Node.js / Advanced: reliable Node.js services / Lesson 15

Performance experiments and memory budgets

Improve measured bottlenecks with bounded data structures and experiments that preserve correctness.

About 4 min readContent reviewed 2026-10-23

On this page

Lesson goals

Explain the concepts and trade-offs behind performance experiments and memory budgets.
Predict and verify both examples, including their outputs and cleanup behavior.
Diagnose the three common failures and apply the worked solutions to related inputs.

Define what better means

An optimization needs a target and a workload. Higher requests per second can coexist with worse tail latency or unacceptable memory growth. Define the important percentile, error rate, throughput, and resource budget before changing code. Include warm-up and realistic input distributions, and compare equivalent runtime and deployment conditions. A laptop microbenchmark does not model network waits, database contention, or container limits. First confirm that the suspected code consumes meaningful time in a representative profile; optimizing an insignificant helper can produce a faster benchmark without improving the service.

Change the amount of work

Algorithm and data-access choices often dominate syntax tweaks. A repeated linear search inside a loop can become quadratic; constructing a Map once can make later lookups direct at the cost of additional memory and setup. The first example builds an index of synthetic products and computes a total without repeating array searches. Duplicate identifiers require a policy because Map would otherwise keep the last value. An optimization must preserve the original business semantics, including ordering, missing items, numeric precision, and error behavior. Faster incorrect output is a regression.

Bound every cache

A cache trades computation or I/O for retained memory and possible staleness. Define key scope, maximum entries or bytes, eviction, expiry, and invalidation before adding one. The second example implements a tiny bounded least-recently-used teaching cache with Map insertion order. A get refreshes recency and adding a third entry evicts the oldest. Its values are small strings; a real cache also needs a byte budget because two entries can still contain gigabytes. Never mix tenants or users in a cache key when authorization changes the result.

javascript Complete example: Bound every cache

const products = [{ id: 'a', price: 3 }, { id: 'b', price: 5 }];
const index = new Map();
for (const product of products) {
  if (index.has(product.id)) throw new Error('Duplicate product');
  index.set(product.id, product);
}
const basket = [{ id: 'b', count: 2 }, { id: 'a', count: 1 }];
let total = 0;
for (const line of basket) {
  const product = index.get(line.id);
  if (!product) throw new Error('Missing product');
  total += product.price * line.count;
}
console.log(total);

Expected output

One index serves every basket lookup. Explicit duplicate and missing-ID behavior prevents a silent semantic change.

Understand retention and allocation

Garbage collection reclaims objects that are no longer reachable, not objects that the business no longer needs. A global Map, event listener, pending promise closure, or repeating timer can retain data indefinitely. Watch heap usage across comparable load cycles and idle periods rather than calling every spike a leak. process.memoryUsage also includes dimensions beyond JavaScript heap, such as external buffers and resident memory. Streaming can reduce retained input, but concurrency and caches can multiply memory use. Establish a per-request estimate and a process budget, then validate them with measurements.

javascript Complete example: Understand retention and allocation

class SmallCache {
  constructor(limit) { if (!Number.isInteger(limit) || limit < 1) throw new Error('Invalid limit'); this.limit = limit; this.items = new Map(); }
  get(key) {
    if (!this.items.has(key)) return undefined;
    const value = this.items.get(key); this.items.delete(key); this.items.set(key, value); return value;
  }
  set(key, value) {
    this.items.delete(key); this.items.set(key, value);
    if (this.items.size > this.limit) this.items.delete(this.items.keys().next().value);
  }
}
const cache = new SmallCache(2);
cache.set('a', 'A'); cache.set('b', 'B'); cache.get('a'); cache.set('c', 'C');
console.log(cache.get('b'), cache.get('a'), cache.items.size);

Expected output

undefined A 2

Reading a makes b the least recent entry, so inserting c evicts b while retaining the two-entry limit.

Report an experiment honestly

Compare several runs, include variability, and state the input size and environment. Use a monotonic timer such as performance.now for elapsed durations and avoid mixing logging overhead into the measured region. Do not assert a fixed speedup as a correctness test because machines and scheduling differ. Verify output equality separately, then inspect timing distributions and resource changes. Keep the simpler implementation when the measured improvement is negligible or the new invalidation rules create more risk than value. Performance work is complete only when the intended user-visible outcome improves within the stated constraints.

Common errors and fixes

A cache grows forever

Why it happens: No eviction or byte budget was defined.
How to fix it: Add bounded retention and explicit invalidation.
Prevent it next time: Monitor entries, bytes, hit rate, and stale results.

A benchmark is fast but requests are unchanged

Why it happens: The optimized helper was not the service bottleneck.
How to fix it: Profile the representative request path.
Prevent it next time: Define the user-visible target before optimizing.

Heap drops but resident memory stays high

Why it happens: The diagnosis ignored buffers or allocator behavior.
How to fix it: Inspect multiple memory dimensions and retained external data.
Prevent it next time: Use comparable load cycles and several metrics.

Check your understanding

1. A two-entry cache stores two 500 MB buffers. Is its memory safely bounded for a small process?

Show worked solution

Its entry count is bounded, but it can retain about 1 GB before other overhead. Add a byte budget and reject or avoid caching oversized values. Limits must use units connected to the resource being protected, not just a convenient collection length.

2. How do you verify that replacing repeated searches with an index preserves behavior?