Software Architecture / Editorial solution guide
Trace lost and duplicate notifications across commit boundaries
Locate dual-write and acknowledgement windows using durable identities and state evidence rather than blind replay.
Read-only knowledge base ยท No public comments
Recognize the symptoms
Some synthetic confirmed reservations never produce mail, while others produce two copies after a worker restart. The booking screen can still be correct. Missing and duplicate messages can arise from different crash windows, so one global instruction to retry does not explain or safely repair both.
Understand the likely causes
- Candidates include publication outside the reservation transaction, early consumer acknowledgement, relay publication followed by a failed sent marker, and external effects without stable deduplication identity. Telemetry gaps can also make delivered work appear missing; distinguish absence of evidence from a verified missing effect.
Correlate the business and delivery identities
Collect a minimal synthetic timeline keyed by reservation identity, outbox event identity, delivery attempt, and consumer outcome. Keep timestamps on a declared basis and distinguish event time from log-ingestion time. Do not copy addresses or tokens into the investigation. A mail-provider acceptance record does not necessarily prove inbox delivery, and an application log statement before commit does not prove durable booking state. Label each evidence source and its exact meaning before inferring where the notification disappeared.
reservation r-21: confirmed in authoritative store
outbox event e-21: pending
relay attempt a-1: publication accepted
outbox event e-21: still pending after relay crash
relay attempt a-2: publication accepted again
The trace supports a relay redelivery explanation. It does not prove that the consumer applied its business effect twice.
Locate the missing intention before replaying anything
If a confirmed reservation has no associated outgoing intention, inspect whether both writes were inside the intended atomic boundary. If an intention exists but is never claimed, inspect relay selection and failure state. If publication is acknowledged but consumption is absent, inspect the broker route and consumer evidence in the actual authorized environment. These are separate boundaries with different owners. Never infer that every missing mail should create a second reservation or a new unrelated event identity. Preserve the original business relationship during correction.
Inspect the repeat-sensitive effect and its atomicity
A consumer that marks a message processed only after an external send can crash between those steps. Marking before the send trades duplicate risk for lost-send risk unless a stronger protocol exists. Determine whether the provider supports a scoped idempotency identity and what its retention and content rules are. If it does not, document the remaining uncertainty and product policy for possible duplicate notifications. An in-memory set cannot protect effects across process loss, and broker delivery guarantees do not automatically cover a separate external provider.
Correct one boundary and replay under explicit limits
A repair plan names affected durable records, the authoritative owner, the intended effect, and the reason replay is safe. Keep the original identity where the protocol requires it. Bound replay concurrency and include a stop condition for unexpected provider errors or state conflicts. Persistent malformed work belongs in an owned unresolved path until a compatible correction exists. The sample plan is editorial; this guide executes no database mutation, broker operation, provider request, or replay command.
scope: synthetic event e-21 only
precondition: consumer duplicate policy is verified
identity: preserve e-21
verification: one permitted business effect and coherent final delivery state
stop: conflicting payload or unexpected irreversible effect
The precondition must be established before execution in a real environment. The example is a proposed plan, not a completed recovery.
Verify the fix
Reproduce the relevant crash windows in an isolated implementation and inspect reservation, outbox, relay, and consumer state after recovery. Verify the number of permitted effects, not just the number of logs. Record any external-delivery uncertainty explicitly and ensure unresolved work remains visible to its owner.
Keep it from recurring
- Keep durable publication intentions aligned with business commits, retain stable identities, test duplicate and delayed delivery, and review acknowledgement ordering when replacing a broker or provider. Separate delivery acceptance from final user receipt in both contracts and telemetry.