MongoDB / Advanced: distributed and recoverable operations / Lesson 14
Replica sets and read/write concerns
Separate routing preference, durability acknowledgment and read visibility. Reason about elections, replication lag and causal sessions without promising universal zero-loss behavior.
On this page
Lesson goals
- Explain primary/secondary roles and election effects.
- Choose read preference and read/write concern from requirements.
Replication copies history rather than providing a backup
A replica set has a primary accepting ordinary writes and secondary members applying replicated operations. Elections can change the primary, so clients need supported discovery and retry behavior rather than a permanently hardcoded writer. Replication lag can make a secondary read older than a primary write. A replicated accidental delete can reach every member, so multiple live copies are not independent historical recovery. Define failure domains, acceptable stale reads and recovery objectives before interpreting replica count as a complete availability guarantee.
Routing and visibility are different settings
Read preference chooses eligible members; read concern controls supported visibility and consistency guarantees. Choosing a secondary does not automatically request majority-committed data, and majority read concern does not route the request to every member or guarantee the newest possible value. The following illustrative read selects the primary and requests majority concern for a training shipment. Actual defaults can be configured and vary with context, so inspect the application and deployment policy instead of assuming one global setting defines all operations.
db.getSiblingDB('quay_lab').shipments.find(
{ _id: 101 }, { _id: 1, state: 1 }
).readPref('primary').readConcern('majority');
Returned state depends on the fixture and timing. The options describe routing and visibility, not a fabricated healthy result.
Acknowledgment defines what success means
Write concern can require a number or majority of acknowledgments and supported journaling behavior. w:1 acknowledgment is not equivalent to majority durability across a replica-set failover. A write-concern timeout can leave the write applied but not acknowledged to the requested level, so timeout does not prove safe failure or justify repeating a financial event blindly. Defaults have deployment and topology nuances; specify the application guarantee and test it. A transaction’s write concern belongs at transaction commit rather than separate operation options within the transaction.
Use a named acknowledgment policy
The single-document status change below requests majority acknowledgment with a finite teaching timeout. Five seconds is illustrative, not a universal production value. If the response reports a write-concern error, preserve the identifier and reconcile the actual operation outcome through the approved application protocol. The document may already have changed. This setting does not make a separate message send part of the write, and it does not replace request idempotency when a client retransmits after losing the response. No write is executed by the course.
db.getSiblingDB('quay_lab').shipments.updateOne(
{ _id: 101, state: 'pending' },
{ $set: { state: 'ready' } },
{ writeConcern: { w: 'majority', wtimeout: 5000 } }
);
With one matching pending document and successful requested acknowledgment, expected state is ready. Timeout leaves an outcome to reconcile.
Sessions and topology need a complete policy
Causally consistent sessions can provide documented ordering guarantees when their read/write concern prerequisites are met; they are not a synonym for a multi-document transaction or linearizable access everywhere. secondaryPreferred can fall back to a primary when suitable secondaries are unavailable, so it is not a promise that reads never reach the writer. During elections, preserve structured labels and bounded retry decisions. Monitor replication health, retained history, storage and backup independence without treating a green process or member count as proof that a particular business write is recoverable.
Common errors and fixes
A secondary read misses a recent primary write
- Why it happens
- Routing allowed a member whose applied history is older.
- How to fix it
- Use the service’s freshness or causal-read policy and inspect replication context.
- Prevent it next time
- Document which reads tolerate lag and how read-your-writes is obtained.
A write-concern timeout is treated as no write
- Why it happens
- Acknowledgment failure was confused with definite nonapplication.
- How to fix it
- Reconcile the event outcome before a potentially duplicate retry.
- Prevent it next time
- Use event identity and explicit timeout semantics.
Majority read concern is assumed to mean latest everywhere
- Why it happens
- Visibility, routing and real-time recency were conflated.
- How to fix it
- Select the documented consistency contract required by the request.
- Prevent it next time
- Test election and lag scenarios with the actual concern combination.
Check your understanding
1. An update times out waiting for majority acknowledgment. May the client safely repeat an increment without checking?
Show worked solution
No. The update may already have been applied, so another increment can duplicate the event. Preserve request identity and reconcile its recorded outcome using the application’s designed protocol. Driver retryable-write support has specific semantics and does not make an unrelated manually retransmitted business event safe automatically. Distinguish the server operation identity from the API event identity.
2. Why is three replica-set members an incomplete backup strategy for accidental deletion?
Show worked solution
Replication can faithfully copy the committed mistake to all members, leaving every current copy without the deleted business data. Keep protected historical artifacts and a tested restore boundary independently of the replica set. Availability against node failure and historical recovery against logical mistakes are different requirements that must both be designed and rehearsed.
Key takeaways
- Read preference, read concern and write concern answer different questions.
- Timeout and failover behavior require outcome reconciliation and independent backup.