Node.js / Advanced: reliable Node.js services / Lesson 18

Deployment and observability

Release a repeatable artifact, expose meaningful health signals, and connect logs to request context without leaking sensitive data.

About 4 min readContent reviewed 2026-10-23

On this page

Lesson goals

Explain the concepts and trade-offs behind deployment and observability.
Predict and verify both examples, including their outputs and cleanup behavior.
Diagnose the three common failures and apply the worked solutions to related inputs.

Deploy a known artifact

A deployment should identify the source revision, runtime, dependency lockfile, build steps, and configuration contract. Build and test once in a controlled environment, then promote the reviewed artifact rather than installing arbitrary latest packages during startup. Production should use a supported release with current patches; support labels and dates must be checked against the official release policy. A container is a packaging tool, not an automatic security boundary. Run with appropriate user permissions, define writable locations, and apply resource requests and limits based on measured behavior.

Separate readiness from liveness

Liveness asks whether a process is functioning enough that restarting it is unnecessary. Readiness asks whether this instance should receive new work. A starting or draining instance can be alive but unready. Do not make every liveness probe depend on an external database, or a database outage may trigger a fleet-wide restart storm. Readiness can include essential dependency state with bounded, cached checks and clear failure semantics. The second example models startup, readiness, and draining without launching a real deployment. A probe response is useful only when the routing platform acts on its meaning.

Connect logs to request context

Structured logs use consistent fields such as event, requestId, route, status, and durationMs. Log approved metadata rather than entire request bodies or authorization headers. AsyncLocalStorage can carry a context through asynchronous work without passing the same identifier through every helper. The first example creates two synthetic contexts and shows that each retains its own ID after an await. Validate or replace incoming correlation IDs before logging them; clients should not control unbounded or multiline fields. Context is diagnostic state, not a source of authorization decisions.

javascript Complete example: Connect logs to request context

import { AsyncLocalStorage } from 'node:async_hooks';
import { setTimeout as delay } from 'node:timers/promises';
const context = new AsyncLocalStorage();
async function operation(id) {
  return context.run({ requestId: id }, async () => {
    await delay(1);
    return { event: 'completed', requestId: context.getStore().requestId };
  });
}
const records = await Promise.all([operation('lab-a'), operation('lab-b')]);
for (const record of records) console.log(JSON.stringify(record));

Expected output

{"event":"completed","requestId":"lab-a"}
{"event":"completed","requestId":"lab-b"}

Each async chain retains its own request context. Promise.all preserves display order without claiming the two operations finish in that order.

Choose metrics with manageable cardinality

Track request count, error rate, latency distributions, active work, queue depth, pool waits, memory, and loop delay. Label requests by route template rather than a unique user ID or full URL; unbounded labels can overwhelm the monitoring system. Traces connect stages across services, but propagation and sampling must be intentional, and trace attributes also need redaction. Establish alerts based on sustained user impact and resource exhaustion risk instead of every isolated exception. Include deployment revision in diagnostics so a regression can be correlated with a specific change.

javascript Complete example: Choose metrics with manageable cardinality

let state = 'starting';
function probe() { return { live: state !== 'stopped', ready: state === 'ready' }; }
console.log(JSON.stringify(probe()));
state = 'ready'; console.log(JSON.stringify(probe()));
state = 'draining'; console.log(JSON.stringify(probe()));
state = 'stopped'; console.log(JSON.stringify(probe()));

Expected output

{"live":true,"ready":false}
{"live":true,"ready":true}
{"live":true,"ready":false}
{"live":false,"ready":false}

Readiness changes before the process stops. Real probe endpoints map this state to documented HTTP statuses and platform actions.

Release and recover deliberately

A safe rollout limits exposure, observes the same success indicators used in testing, and has a rollback path. Schema changes must remain compatible with old and new application versions during overlap. Check startup validation, readiness transitions, graceful shutdown, and telemetry flushing in an isolated deployment rehearsal. Do not infer success merely from a process remaining alive. A release is successful when intended behavior, error rates, latency, and data integrity remain acceptable. Document an operational runbook that tells the next person how to inspect, stop, restart, and reconcile the system without guessing or revealing credentials.

Common errors and fixes

A database outage restarts every instance

Why it happens: Liveness depended on a remote dependency.
How to fix it: Separate process health from readiness and dependency state.
Prevent it next time: Rehearse dependency failure in an isolated environment.

Metrics storage explodes

Why it happens: Labels contain unique IDs or full URLs.
How to fix it: Use bounded dimensions such as route templates.
Prevent it next time: Review label cardinality before release.

A release is unreproducible

Why it happens: Startup installed changing dependency versions.
How to fix it: Build from a reviewed lockfile and promote one artifact.
Prevent it next time: Record source revision, runtime, and artifact identity.

Check your understanding

1. An instance is draining accepted requests. What should readiness and liveness report?

Show worked solution

Readiness should be false so no new traffic is assigned, while liveness can remain true during orderly cleanup. After resources close, the process stops. Coordinate the platform’s grace period with the application deadline so forced termination does not precede the promised drain window.

2. Choose safe metric labels and log fields for GET /users/123.