Laravel / Advanced / Lesson 13

Queues and idempotent jobs

Design deferred work around retries, durable business identity, and transaction boundaries instead of assuming a queue guarantees a single execution.

About 5 min readContent reviewed 2026-10-25

On this page

Lesson goals

Explain retryable delivery and idempotent business outcomes.
Coordinate database commits with queued dispatch.
Choose bounded retries, timeouts, and failure observations.

A job can be attempted more than once

A queue separates accepting a request from performing its work, but workers can fail after a side effect and before acknowledging completion. Treat a durable business operation as potentially repeated, with details depending on the selected driver. The synthetic fixture is invoice event evt-17, invoice identifier 17, and an intended total adjustment of ten units. Repeating that event must not apply the adjustment twice. Idempotency means the same operation identity produces the intended outcome on repetition; it does not mean every handler statement executes once or that retries are harmless without a design.

Keep operation identity separate from an attempt

Choose an identifier that survives retries, such as an immutable event identifier, rather than a new random key generated inside each attempt. The complete PHP model below shows deduplication within one process using synthetic events. Its output verifies the business arithmetic only. An in-memory array is not durable, not shared across workers, and not a distributed idempotency implementation. A real database-backed design needs a uniqueness constraint and a transaction joining the receipt with the local state change. External payment or mail providers require their own idempotency or delivery protocol; a local flag cannot atomically commit a remote side effect.

php Offline arithmetic model, not a durable queue backend

<?php
declare(strict_types=1);
$events = [
    ["id" => "evt-17", "units" => 10],
    ["id" => "evt-17", "units" => 10],
    ["id" => "evt-18", "units" => 5],
];
$seen = [];
$total = 0;
foreach ($events as $event) {
    if (isset($seen[$event["id"]])) { continue; }
    $total += $event["units"];
    $seen[$event["id"]] = true;
}
echo $total, " ", count($seen), PHP_EOL;

Expected output

15 2

The repeated evt-17 contributes once; evt-18 adds five. The result is fifteen units across two accepted identities. No Laravel application or queue is started.

Dispatch only when the required state is committed

A fast worker may otherwise read an invoice before its creating transaction commits. Laravel afterCommit dispatching delays the queued job until the surrounding transactions commit and discards that dispatch if the transaction rolls back. The contextual fragment assumes an existing Invoice model with a status field and an application GenerateInvoice job that accepts the invoice identifier and implements its own idempotent handling. These dependencies are deliberately not executed here. Delayed dispatch is useful ordering, but a process crash between a successful commit and publication can still require an outbox or another durable coordination design when guaranteed handoff matters.

php Contextual Laravel transaction and dispatch fragment

<?php
use App\Jobs\GenerateInvoice;
use Illuminate\Support\Facades\DB;

DB::transaction(function () use ($invoice): void {
    $invoice->update(["status" => "ready"]);
    GenerateInvoice::dispatch($invoice->getKey())->afterCommit();
});

Expected observation in an isolated application fixture: the job is published after a successful outer transaction commit, and this dispatch is discarded on rollback. No database or queue behavior is claimed tested by this fragment.

Bound attempts without mistaking locks for idempotency

Set attempts, backoff, job duration, and external I/O timeouts from measured workload and dependency behavior. For drivers using retry_after, keep worker timeout sufficiently shorter so an old worker is terminated before another attempt becomes eligible; SQS instead uses its configured visibility timeout. Unique-job and WithoutOverlapping mechanisms can coordinate dispatch or concurrent handling, but they do not prove an external effect occurred exactly once. All workers must share the required lock backend. Keep serialized payloads small and avoid placing secrets in jobs. Rehydrate identifiers deliberately, accounting for records deleted or changed before execution.

Observe duplicates and recovery paths

Test a duplicate event, a failure before commit, a failure after the business effect, and a missing target record. A queue fake can verify dispatch intent, but it does not execute the handler or reproduce driver delivery semantics. A synchronous connection likewise does not demonstrate independent worker timing. Record operation identity, attempt, outcome, and release identifier without logging sensitive payloads. Alert on sustained failures and queue delay using application-specific objectives. A replay tool should preserve the original operation identity and authorization context, because creating a new identity can bypass the very duplicate protection the handler relies on.

Common errors and fixes

A retry applies the same invoice adjustment twice.

Why it happens: Attempt delivery was confused with a unique business operation.
How to fix it: Use stable operation identity and durable duplicate protection.
Prevent it next time: Test failure after the side effect, not only before it.

A worker cannot find a newly created invoice.

Why it happens: The job ran before the database transaction committed.
How to fix it: Use documented after-commit dispatch and handle genuine deletion.
Prevent it next time: Test commit and rollback dispatch paths separately.

Two workers process a long attempt together.

Why it happens: Visibility or retry timing permits another attempt before termination.
How to fix it: Align timeouts with the driver and retain idempotent handling.
Prevent it next time: Measure durations and monitor attempts rather than choosing universal values.

Check your understanding

1. The PHP fixture receives evt-18 again after its current events. Give the result and explain what would change after a process restart.

Show worked solution

Within the same model, total remains fifteen and the accepted-identity count remains two. After restart the array is lost, so this model alone would accept duplicates again. Persisting a receipt with a unique operation key and joining it transactionally to the local update addresses that durability requirement; it still does not atomically cover an unrelated remote provider.

2. A transaction rolls back after scheduling GenerateInvoice with afterCommit. State the expected dispatch result and one additional test the handler still needs.