Google Cloud / Intermediate: Service Contracts and Operations / Lesson 11
Pub/Sub Delivery and Idempotency
Connect message delivery, acknowledgment, ordering, retries, and application effects without promising exactly-once business behavior.
On this page
Lesson goals
- Distinguish topics, subscriptions, and consumer progress.
- Explain duplicate delivery and acknowledgment timing.
- Model idempotent processing and bounded backlog.
A subscription has its own delivery state
Publishers send messages to a topic; subscriptions define how consumers receive and track them. Multiple subscribers to one subscription generally share its work, while separate subscriptions can deliver the topic’s messages to separate applications. Choose pull, push, or supported export behavior from its documented contract rather than assuming every subscriber sees every message independently.
A successful publish and a successful business effect are different events. Delivery can be repeated under ordinary at-least-once behavior, and publishing retries can create duplicates at a different layer. Define a stable application event identity, payload schema, retention needs, and consumer ownership before relying on a queue as a source of truth.
A duplicate-aware handler needs a durable real design
The first offline program keeps a set of processed synthetic event IDs and counts each only once. It illustrates the idempotency key concept, not a distributed deduplication guarantee. A process-local set is lost on restart and can race across workers. In a real application, the deduplication marker and business effect need an atomic or otherwise justified consistency protocol.
Acknowledge only under the intended success policy. Acknowledging before a required effect is safely recorded can lose work; recording the effect before a failed acknowledgment can cause a duplicate attempt. Handle poison messages, schema errors, and partial effects with a bounded retry and review path rather than an infinite crash loop.
deliveries = [("event-a", 10), ("event-a", 10), ("event-b", 20)]
seen = set()
total = 0
for event_id, amount in deliveries:
if event_id in seen: continue
seen.add(event_id)
total += amount
print(f"deliveries={len(deliveries)} unique={len(seen)} total={total}")
deliveries=3 unique=2 total=30
Three deliveries represent two invented events totaling30; the in-memory set is not production deduplication.
Delivery guarantees have explicit scope
Exactly-once delivery is a documented opt-in capability for supported pull subscriptions, including StreamingPull, with regional requirements. It is not a universal property of push or every export subscription. It also does not turn separate unique publish attempts or a database side effect into exactly-once application processing. Read current acknowledgment and endpoint requirements for the selected client.
Ordering is another optional contract with ordering-key and regional considerations; it is not a default global FIFO across all messages. An expired acknowledgment deadline, flow-control saturation, or failed consumer can affect progress. Keep ordering, delivery uniqueness, application idempotency, and business transaction atomicity as separate requirements.
Measure arrival and processing rates with units
The second program models a static arrival rate of120 messages per minute and processing capacity of100. Backlog grows by twenty per minute in that simplified steady model. It does not measure Pub/Sub throughput, quota, or actual retention. Retry traffic can increase effective arrival and work amplification, so raw unique-event rate alone may understate consumer load.
Use flow control and bounded concurrency to protect memory and downstream services. Increasing acknowledgment deadlines without fixing slow processing can merely hide symptoms. Track oldest-unacked age, backlog, processing errors, and dependency saturation under the product’s documented metrics. Dead-letter forwarding and delivery-attempt behavior need their own configured permissions and semantics.
arrivals_per_minute = 120
processed_per_minute = 100
minutes = 5
growth = max(0, arrivals_per_minute - processed_per_minute)
print(f"growth_per_minute={growth} added_backlog={growth * minutes}")
growth_per_minute=20 added_backlog=100
Under constant invented rates, five minutes add one hundred queued messages.
A consumer contract includes replay and recovery
Define how schema changes, replay, retention expiry, consumer identity, and dead-letter review are handled. A replay can intentionally redeliver old events, so the application’s idempotency horizon must match its business requirements. Protect topic and subscription permissions independently from the consumer’s target-database authority.
Before a real implementation, validate duplicate, out-of-order, delayed, malformed, and partially processed events in an approved isolated environment. Document the acknowledgment boundary and restart behavior. This lesson publishes no message, opens no subscription, and tests no cloud delivery guarantee; only the local examples execute.
Common errors and fixes
A Pub/Sub delivery flag is called exactly-once database processing.
- Why it happens
- Transport acknowledgment and business transaction semantics were conflated.
- How to fix it
- Design event identity and atomic effect/deduplication handling.
- Prevent it next time
- Review publish-side duplicates and restart behavior.
A consumer acknowledges before recording required work.
- Why it happens
- Progress is reported ahead of the durable success boundary.
- How to fix it
- Move acknowledgment under a documented processing policy.
- Prevent it next time
- Test crashes on both sides of the boundary.
Backlog grows despite larger ack deadlines.
- Why it happens
- Processing capacity or downstream saturation remains unresolved.
- How to fix it
- Measure rates, flow control, retries, and dependencies.
- Prevent it next time
- Use bounded admission and consumer health evidence.
Check your understanding
1. A message updates a database, then the process crashes before acknowledgment. What must the next delivery tolerate?
Show worked solution
The prior update may already be committed. The consumer must identify the same application event and avoid repeating a non-idempotent effect, using a durable consistency design rather than a process-local set. Acknowledge according to the defined success boundary. Exactly-once delivery settings alone do not establish an atomic transaction with the application database.
2. Why can an ordered subscription still require explicit application retry and failure policy?
Show worked solution
Ordering constrains supported delivery relationships under its contract, but a consumer can fail, miss an acknowledgment deadline, encounter an invalid payload, or commit an uncertain effect. Define how retries, poison events, downstream errors, and key-level progress are handled. Do not interpret ordering as validation, unlimited throughput, or a guarantee that every business action succeeds once.
Key takeaways
- Delivery, ordering, acknowledgment, and application effects are separate contracts.
- Idempotency must survive the failures the application claims to tolerate.
- Offline message models do not verify Pub/Sub guarantees or quotas.