RabbitMQ / Intermediate: replicated work, retry safety and operational evidence / Lesson 8
TTL, dead-lettering and bounded retries
Design expiry and terminal handoff without turning failure routing into an infinite loop or an unverified durability promise.
On this page
Lesson goals
- Distinguish message TTL, queue expiry and delivery limits.
- Explain dead-letter routing, safety modes and bounded retry ownership.
Expiry controls different lifetimes
Message TTL bounds residence under a queue’s expiry rules, while queue expiry concerns an unused queue’s lifetime. Expiring an entire queue does not dead-letter its messages. AMQP 0-9-1 per-message expiration is a string containing milliseconds; queue-level message TTL is a numeric policy or argument, and the lower applicable value wins. TTL is not a precise application scheduler. Expired records can remain behind earlier records until cleanup conditions occur, although expired messages are not intentionally delivered under the documented rules. Design delayed retry from the actual queue-type behavior rather than promising exact execution at a millisecond deadline.
A DLX is an exchange with another routing step
A dead-letter exchange is a normal exchange receiving republished messages after supported events such as rejection without requeue, expiry, a length policy or quorum delivery-limit exhaustion. Its target queues still need valid bindings and availability. Merely naming a DLX does not create its topology or guarantee the handoff succeeds. Prefer reviewed policies for operationally changeable settings rather than hardcoding every argument into clients. Preserve the original business identifier and inspect the documented x-death history by queue and reason; one header entry is not a universal count of every application retry or processing attempt.
Default dead-lettering is not automatically lossless
Default dead-letter republishing does not universally use internal publisher confirms and can lose messages when a target is unavailable. Quorum queues support an opt-in at-least-once strategy with documented prerequisites, including reject-publish overflow and a configured DLX; verify required feature support for the exact deployment. It retains source responsibility until targets confirm, consuming resources and allowing duplicate target deliveries during retries. This is stronger transfer evidence, not exactly-once business execution. Changing strategy or overflow can affect retained dead letters, so treat such changes as a reviewed lifecycle operation rather than harmless tuning.
Bound attempts and elapsed wait separately
The offline fixture has three retry delays totaling fourteen seconds. That sum excludes processing and transport time, so it is not the entire business deadline. The JSON policy record names a finite terminal path and original identity. It is documentation rather than an administrative policy payload. Quorum queues in the4.1 baseline inherit a default delivery limit of twenty introduced in 4.0; do not disable it casually to hide repeated failure. Application retry topics, republishing and dead-letter cycles have distinct accounting, so define the attempt convention and terminal ownership explicitly.
delays_s = [2, 4, 8]
assert len(delays_s) == 3
assert sum(delays_s) == 14
print(len(delays_s), sum(delays_s))
3 14
Three waits total 14 seconds before adding execution or transport time; no sleep or network occurs.
{"operation_id":"label-P17-v1","retry_delays_seconds":[2,4,8],"terminal_destination":"lab.review","preserve_original_identity":true,"review_owner":"lab-operations-role"}
The record describes application policy and does not create a queue, DLX or delayed-delivery service.
Quarantine requires a review and replay contract
An invalid schema needs a different path from a temporarily unavailable label provider. Repeating malformed bytes immediately wastes capacity, while silently discarding exhausted work hides a business failure. Define who reviews terminal messages, which corrections are allowed and how replay preserves identity and ordering requirements. Cycles can move later events ahead of an earlier failed event, so verify sequencing consequences. This lesson configures no TTL, DLX or delivery limit and sends no message. It checks local delay arithmetic and a design fixture only; actual expiration, transfer durability and replay behavior remain untested.
Common errors and fixes
Queue expiry is expected to dead-letter every stored message.
- Why it happens
- Queue lifetime was confused with message expiry.
- How to fix it
- Review the documented expiry scope and recovery requirements.
- Prevent it next time
- Test message and queue lifecycle assumptions separately.
A named DLX is treated as guaranteed durable transfer.
- Why it happens
- Target routing and dead-letter strategy were not reviewed.
- How to fix it
- Review topology, target availability and the queue-type safety contract.
- Prevent it next time
- Include unavailable and unroutable targets in acceptance.
Retries circulate indefinitely with no terminal owner.
- Why it happens
- Attempt and elapsed-time budgets were absent.
- How to fix it
- Define bounded retry and an observable reviewed terminal state.
- Prevent it next time
- Preserve original identity through every handoff.
Check your understanding
1. Does fourteen seconds of configured retry wait guarantee completion within fourteen seconds?
Show worked solution
No. Processing, queueing, network delay and terminal handling add time, and TTL-based patterns do not provide a universal exact scheduler. State the full deadline separately from individual delays. Test boundary and exhaustion behavior in an authorized environment before making a service-level promise.
2. Can at-least-once dead-lettering still produce duplicate target messages?
Show worked solution
Yes. Lost confirmations and retried transfers can repeat delivery. The stronger strategy retains source responsibility until confirmed acceptance under its prerequisites, but consumers still need stable identity and duplicate-safe effects. Also budget retained dead letters when a target stays unavailable instead of assuming the source can always discard them immediately.
Key takeaways
- Expiry, retry counts and transfer safety have different scopes.
- A terminal queue needs explicit ownership and replay rules.