Apache Airflow / Editorial solution guide
Tasks remain queued without reaching execution
Locate admission, routing and worker-start boundaries before increasing concurrency or retrying work.
Read-only knowledge base ยท No public comments
Recognize the symptoms
Task instances show queued for a sustained period, with no new business output and little or no task-log content. Other DAGs or queues may still make progress.
Understand the likely causes
- The binding constraint can be a resource budget, executor capacity, queue subscription mismatch or worker startup failure. An unavailable log backend can also hide evidence from a task that actually started.
Capture the transition and exact attempt
Record DAG, run, task, map index and attempt, then capture scheduled, queued and any running timestamps from existing evidence. Determine which executor is selected because queued-worker and per-task-container paths have different diagnostics. A task-log panel alone cannot establish whether dispatch occurred; its retrieval path can fail independently. Preserve scheduler and executor error context with sensitive values removed. Do not clear the task merely to obtain a new attempt before understanding whether the old execution might still be active or have produced an uncertain external effect.
Check weighted resource admission
Review the task's pool and slot weight alongside the pool's total and occupied capacity, then inspect DAG and active-run limits at their documented scopes. A five-slot request cannot fit a four-slot budget even when no worker is busy. Backfill admission has its own concurrency boundary in Airflow 3.1, so include that workload in a shared dependency budget. Waiting can be intentional protection for a saturated external system. Raising the pool merely because a task waits can replace controlled backlog with timeouts and retries. The local arithmetic identifies an impossible request but changes no configuration.
pool_total = 4
requested_slots = 5
can_ever_fit = requested_slots <= pool_total
assert not can_ever_fit
print(can_ever_fit)
False
Waiting for free workers cannot make five fit into a declared total of four.
Compare routing with actual worker capability
For an executor using named queues, compare the task's queue with the approved worker subscriptions. A fleet serving default cannot necessarily accept reports. For a container-based path, examine existing creation and startup evidence rather than assuming the same queue model applies. Confirm the required code, provider and runtime are available at the execution target; successful parsing elsewhere does not prove that. Keep routing absence, resource exhaustion and import failure distinct. Each requires a different authorized remedy, and adding generic workers may solve none of them if they cannot receive or start this task.
Repair one boundary and reconcile uncertain work
Choose a correction supported by the identified boundary: a reviewed slot policy, routing alignment, compatible runtime or logging repair. Preserve the original attempt evidence and confirm no duplicate execution remains active before an authorized rerun. If the destination may already contain output, reconcile by the stable business operation identity rather than assuming queued display proves nothing happened. This guide starts no worker, edits no pool, calls no executor API and runs no task-clear command. Its local calculation validates only the supplied budget mismatch; actual dispatch and recovery remain deployment-specific verification work.
Verify the fix
After an authorized correction, observe a bounded synthetic task transition from admission through running to a validated output. Compare queue wait and runtime separately and verify the protected dependency remains healthy. Confirm the original attempt did not create a second effect, and retain routing and version evidence with the result.
Keep it from recurring
- Validate task weights against pool totals, document executor-specific routing and keep worker runtimes reproducible. Alert on sustained queue age with the relevant resource and attempt context rather than automatically increasing every concurrency limit.