FastAPI / Advanced: lifecycle and reliable service design / Lesson 14

Background work and durable jobs

Distinguish response-adjacent in-process tasks from durable processing with explicit delivery, retry, and recovery contracts.

About 5 min readContent reviewed 2026-10-25

On this page

Lesson goals

Explain the assumptions and boundaries of background work and durable jobs.
Interpret both complete examples and justify their observed outputs.
Diagnose the listed failures and apply the worked solutions while keeping identity, state, and lifecycle ownership explicit.

BackgroundTasks is an execution convenience

FastAPI BackgroundTasks can attach supported work to run after the response is sent according to the underlying response lifecycle. It does not create a durable queue, separate worker fleet, or exactly-once guarantee. A process crash or deployment can lose unfinished in-process work. The first example appends a public synthetic event to a list and returns 202. TestClient waits through that task before its call completes under the tested stack, so the visible list is deterministic. A real HTTP client receiving the response does not thereby prove the task has finished, and a 202 result needs an eventual-outcome contract when the work matters.

python Complete in-process background-task check

from fastapi import BackgroundTasks, FastAPI
from fastapi.testclient import TestClient
recorded = []
def record(operation_id): recorded.append(operation_id)
app = FastAPI()
@app.post("/enqueue-lab", status_code=202)
def enqueue(background: BackgroundTasks):
    background.add_task(record, "lab-a")
    return {"accepted": True}
with TestClient(app) as client:
    response = client.post("/enqueue-lab")
    print(response.status_code, response.json())
    print(recorded)

Expected output

202 {'accepted': True}
['lab-a']

The isolated test client observes the attached synthetic task before returning. The list is not durable storage, and the endpoint creates no real queue or external work.

Choose what can safely be lost

A small best-effort local follow-up can have a different reliability requirement from payment, inventory, or customer notification. State whether work may be lost, repeated, delayed, or reordered, and who observes its outcome. Do not move a critical operation after the response merely to make the endpoint appear faster. If durable acceptance is promised, record the job or intent durably under an explicit transaction boundary before returning that promise. A database write followed by queue publication can still leave a gap, which may need an outbox or other coordination protocol rather than a background callback.

Own failures after the response boundary

Once a response is sent, a failed task cannot generally replace it with a clean new error response. The second example records a known synthetic task failure while the accepted response remains 202. It is an explicit demonstration of separate outcomes, not an instruction to silently swallow errors. A real task needs safe diagnostics, a status or retry decision, and an accountable owner. Task ordering and failure behavior also matter when several callbacks are attached. Avoid logging credentials or full payloads, and do not treat a healthy HTTP status as complete evidence that every attached side effect succeeded.

python Complete separate-task-failure check

from fastapi import BackgroundTasks, FastAPI
from fastapi.testclient import TestClient
outcomes = []
def work():
    try:
        raise ValueError("synthetic task failure")
    except ValueError:
        outcomes.append({"state": "failed", "code": "LAB_FAILURE"})
app = FastAPI()
@app.post("/work", status_code=202)
def accept(background: BackgroundTasks):
    background.add_task(work)
    return {"accepted": True}
with TestClient(app) as client:
    response = client.post("/work")
    print(response.status_code, response.json())
    print(outcomes)

Expected output

202 {'accepted': True}
[{'state': 'failed', 'code': 'LAB_FAILURE'}]

The response is accepted while the known synthetic task result is failed. Recording in a list is only a lab demonstration; production failure state needs appropriate durable ownership.

Use independent inputs and bounded resources

Pass validated stable values or operation identifiers instead of a mutable request object or a yielded session that may already be cleaned up. A synchronous task can consume thread-pool capacity, while an async task can block the event loop if it performs blocking work. Bound execution, payload size, retries, and downstream concurrency. Heavy CPU work or long-running reliable workflows may require a separately designed worker system with its own trust, storage, and shutdown contracts. Selecting a queue library does not decide idempotency, transaction scope, or safe replay of external effects for you.

Test the exact guarantee being claimed

In-process tests can verify callback wiring, validated inputs, known failure recording, and cleanup. They cannot prove survival across process loss or a distributed broker’s delivery contract. Use synthetic work and controlled completion rather than arbitrary sleeps. These examples start no worker service, network server, queue, email sender, or database and perform no external side effect. For a durable design, separately test interruption before and after persistence, duplicate delivery, uncertain external results, and shutdown in an authorized isolated environment. Make the public response say accepted or completed according to the evidence actually established, not according to how quickly the handler returned.

Common errors and fixes

An attached task is called a durable queue

Why it happens: In-process scheduling was confused with persistence and delivery.
How to fix it: Define durable acceptance and recovery where required.
Prevent it next time: Test process-loss and duplicate scenarios separately.

A task captures a closed request session

Why it happens: Request and background lifetimes were mixed.
How to fix it: Pass stable inputs and own resources inside the task.
Prevent it next time: Review cleanup and task ordering under the pinned version.

HTTP 202 is reported as completed side effects

Why it happens: Acceptance and final outcome were conflated.
How to fix it: Expose or record a separate status and failure policy.
Prevent it next time: Align response wording with established evidence.

Check your understanding

1. A process can restart immediately after sending 202. What should a critical job design guarantee before the response?

Show worked solution

If 202 promises durable acceptance, persist the operation or job intent under an appropriate atomic contract first. A callback added to memory cannot establish that guarantee. Define how the client observes completion and how a worker resumes or safely retries.

2. Why can a successful TestClient background test still be insufficient for production delivery?