Node.js / Intermediate: application boundaries / Lesson 11

Persistence, transactions, and idempotency

Separate repository behavior from transport and reason about concurrent updates, atomic changes, and repeated commands.

About 4 min readContent reviewed 2026-10-23

On this page

Lesson goals

Explain the concepts and trade-offs behind persistence, transactions, and idempotency.
Predict and verify both examples, including their outputs and cleanup behavior.
Diagnose the three common failures and apply the worked solutions to related inputs.

A repository defines behavior

A repository hides storage details behind operations expressed in application terms. A transfer or reserve operation can express a stronger invariant than separate read and write methods. Keep database connections and credentials outside request parsing and pass the repository into domain code. A pool limits open connections but does not by itself prevent lost updates. Define what a successful return guarantees: committed data, a queued write, or only an in-memory change. Those meanings affect response status, retries, and recovery after a process crash.

Await does not create a transaction

Two requests can both read the same balance, await unrelated work, and then write incompatible results. JavaScript callbacks running one at a time does not make the whole asynchronous sequence atomic. The first example demonstrates a version check: the first update advances a revision and the second stale update is rejected. This is an in-memory model of optimistic concurrency, not durable storage. A database can implement a similar condition in an atomic UPDATE and verify affected rows. On conflict, recompute from current state rather than blindly resubmitting an outdated absolute value.

Transactions protect an invariant

A transaction groups changes so they commit together or roll back together, subject to the database’s isolation rules. Transferring units between accounts should not commit a debit without its credit. Check constraints and ownership in the same transactional decision as the write when concurrent changes matter. A transaction that waits on a remote API can hold locks too long; keep it focused and consider an outbox for reliable follow-up work. Always release a checked-out connection in finally and distinguish transaction rollback from evidence that an external side effect was undone.

javascript Complete example: Transactions protect an invariant

let account = { balance: 10, revision: 0 };
function update(expectedRevision, balance) {
  if (account.revision !== expectedRevision) return false;
  account = { balance, revision: expectedRevision + 1 };
  return true;
}
const first = { ...account };
const second = { ...account };
console.log(update(first.revision, first.balance - 3));
console.log(update(second.revision, second.balance - 4));
console.log(account.balance, account.revision);

Expected output

true
false
7 1

A stale revision cannot overwrite the first result. The caller must resolve the conflict using fresh state.

Repeated commands need a contract

A client may retry after losing the response even when the original operation committed. The second example associates a synthetic idempotency key with an input and result; the repeated identical command receives the original result, while reusing the key for different input is rejected. Its Map is a sequential teaching model and loses all state on restart. Production keys need caller scope, a retention policy, and atomic storage with the domain change. A check-then-insert spread across awaits still races unless a transaction or unique constraint coordinates competing requests.

javascript Complete example: Repeated commands need a contract

const receipts = new Map();
let nextId = 1;
function create(key, name) {
  if (receipts.has(key)) {
    const saved = receipts.get(key);
    if (saved.name !== name) throw new Error('Key reused with different input');
    return saved.id;
  }
  const id = nextId++;
  receipts.set(key, { name, id });
  return id;
}
console.log(create('request-a', 'Ada'), create('request-a', 'Ada'));
try { create('request-a', 'Grace'); } catch (error) { console.log(error.message); }

Expected output

1 1
Key reused with different input

Identical sequential retries return one identifier. A changed payload cannot silently borrow the previous command’s result.

Secure and operate the storage boundary

Use parameterized database operations for values, allowlist any dynamic identifiers, and grant the application only required privileges. Put deadlines on connection acquisition and queries, monitor pool wait time, and avoid creating a new pool for every request. Plan migrations and rollback compatibility before deploying a new schema. Tests should cover competing writes, duplicate commands, unavailable storage, and commit failures, using an isolated database when integration guarantees matter. Local memory and JSON files are valuable models, but never claim they provide multi-process coordination or crash durability without an explicit implementation.

Common errors and fixes

A balance update disappears

Why it happens: Two requests overwrote a value read before an await.
How to fix it: Use an atomic conditional update or appropriate transaction.
Prevent it next time: Test concurrent requests against the real repository.

Connection pool exhaustion

Why it happens: Connections were not released on every failure path.
How to fix it: Release in finally and bound acquisition time.
Prevent it next time: Monitor active connections and pool wait time.

A retry creates duplicate records

Why it happens: The response was lost after the original commit.
How to fix it: Persist an idempotency key and outcome atomically.
Prevent it next time: Test retries after commit but before response delivery.

Check your understanding

1. Two debits of 3 and 4 begin from balance 10. What must a correct repository preserve?

Show worked solution

If both are accepted, the final balance must be 3. A version conflict can reject one attempt temporarily, but it must be recomputed against the updated balance before retrying. A final balance of 6 or 7 after claiming both succeeded reveals a lost update.

2. Why must an idempotency key be stored with the write rather than in a later operation?