Queue Architecture for Scalable Laravel SaaS

A Laravel application that was fine at a thousand users and struggling at ten thousand has usually not outgrown its framework. It has outgrown a queue design that was never designed — jobs were added to the default queue as features shipped, and now a slow report generator sits behind the same workers as password reset emails.

Isolation before anything else

The first change is almost always splitting queues by job class and characteristic, then giving each its own workers. Fast and user-visible work must never wait behind slow batch work.

# horizon.php — isolation by characteristic
'supervisor-interactive' => [
    'queue' => ['notifications', 'webhooks'],
    'maxProcesses' => 10, 'timeout' => 30,
],
'supervisor-batch' => [
    'queue' => ['reports', 'exports'],
    'maxProcesses' => 3,  'timeout' => 900,
],

The timeouts matter as much as the process counts. A thirty-second timeout on interactive work surfaces a hung integration quickly; a fifteen-minute timeout on a report stops a legitimate long job being killed halfway.

Idempotency is not optional

Any queued job will eventually run twice. A worker dies after the side effect but before the acknowledgement, or a retry fires on a job that actually succeeded. If running twice charges a card twice, that is a design defect rather than bad luck.

• Derive a deterministic key from the job payload and record completion against it
• Use database constraints as the final guard, not just an application check
• Prefer upserts over inserts for anything a retry could touch
• Make external calls with an idempotency key where the API supports one

Retries and backpressure

Default retry behaviour is dangerous during an outage. A third-party API returns 500, hundreds of jobs retry immediately, and the queue turns into a load generator aimed at a service that is already unwell.

Exponential backoff plus a bounded attempt count fixes the amplification. Beyond that, jobs should stop retrying and land somewhere a human will look.

public $tries = 5;
public function backoff(): array
{
    return [10, 30, 120, 600];   // seconds, then fail
}

A failed jobs table nobody reads is the same as no error handling. Route failures somewhere with an owner.

Poisoned payloads

One malformed record can block a queue indefinitely, failing and retrying while everything behind it waits. Validate the payload at the start of the job and fail fast on data that can never succeed — that is a permanent failure, not a transient one, and it should not be retried at all.

Watch queue depth

Response time looks healthy right up to the moment a queue backlog becomes a customer-visible delay. Queue depth per queue, oldest-job age, and failure rate are the three numbers that predict the incident.

• Alert on oldest-job age rather than raw depth — 10,000 fast jobs is fine, one stuck job is not
• Track failure rate per queue, not per application
• Confirm workers are alive; a dead worker produces a silent, growing backlog
• Watch memory per worker to catch leaks in long-running processes

In one production system this layer moves two million events a day. None of it is exotic. It is isolation, idempotency, bounded retries, and monitoring that fires before a customer notices.