Node.js / Advanced: reliable Node.js services / Lesson 13

Bounded concurrency and worker threads

Choose between asynchronous I/O and CPU workers, and bound the number of operations competing for resources.

About 4 min readContent reviewed 2026-10-23

On this page

Lesson goals

Explain the concepts and trade-offs behind bounded concurrency and worker threads.
Predict and verify both examples, including their outputs and cleanup behavior.
Diagnose the three common failures and apply the worked solutions to related inputs.

Match the execution model to the work

Asynchronous I/O lets a process overlap waiting without dedicating a JavaScript worker to each request. CPU-intensive JavaScript still occupies its isolate’s thread and can delay every unrelated request there. Worker threads provide separate JavaScript execution for CPU work within the process; child processes provide a separate process boundary. Neither choice makes an algorithm cheap. First reduce unnecessary work and measure the bottleneck. A worker is usually useful for substantial computation, not as an automatic wrapper around every filesystem call or tiny arithmetic expression.

Bound admission before scaling

A queue needs an active-worker limit and a maximum waiting length. Without both, a burst can become an out-of-memory failure even if worker count is fixed. Decide what happens when capacity is full: reject, defer, or shed lower-priority work. The second example uses a small shared index to assign four items to two asynchronous workers and preserves result order by writing each result into its original slot. This bounds active tasks, but a production queue must also bound the input source rather than first allocating an unlimited array.

Communicate through an explicit boundary

The first example runs the same .mjs file as main thread and worker, branching on isMainThread. workerData supplies a small input and parentPort returns the result. The main thread observes both a message and worker exit so it does not leave a resource running. Messages normally use structured cloning; functions and arbitrary process resources cannot simply be sent across. Transferable ArrayBuffers can avoid copying but transfer ownership, making the sender’s buffer unusable. Shared memory requires synchronization and a much stronger correctness argument than ordinary message passing.

javascript Complete example: Communicate through an explicit boundary

import { Worker, isMainThread, parentPort, workerData } from 'node:worker_threads';
import { once } from 'node:events';
if (isMainThread) {
  const worker = new Worker(new URL(import.meta.url), { workerData: 100 });
  const exited = once(worker, 'exit');
  const [answer] = await once(worker, 'message');
  const [code] = await exited;
  if (code !== 0) throw new Error('Worker failed');
  console.log(answer);
} else {
  let sum = 0;
  for (let i = 1; i <= workerData; i++) sum += i;
  parentPort.postMessage(sum);
}

Expected output

One worker sums 1 through 100 and exits. This tiny workload illustrates messaging, not a claim that worker startup improves its speed.

A worker pool has failure policy

Creating a worker per small request often spends more on startup and data transfer than the computation saves. Reuse a bounded pool for substantial repeated jobs, track task identifiers, and handle message, error, and exit events. If a worker crashes, decide which unfinished task failed and whether retry is safe. A timeout that merely rejects the caller leaves the worker computing; cancellation needs a cooperative protocol or carefully managed termination. Treat untrusted jobs as a separate security problem because worker threads share the process and are not a complete sandbox.

javascript Complete example: A worker pool has failure policy

import { setTimeout as delay } from 'node:timers/promises';
const input = [1, 2, 3, 4];
const output = new Array(input.length);
let next = 0;
async function worker() {
  while (next < input.length) {
    const index = next++;
    await delay(1);
    output[index] = input[index] * 2;
  }
}
await Promise.all([worker(), worker()]);
console.log(output.join(','));

Expected output

2,4,6,8

At most two asynchronous item operations are active. Taking the index happens before await, preventing duplicate assignment within this isolate.

Scale with evidence

Measure queue delay separately from execution time, and include serialization, memory, and worker startup in the cost. Adding workers beyond available CPU capacity can increase context switching and worsen latency. In container environments, configured CPU quotas matter as well as host hardware. Size a pool through representative measurements and preserve headroom for the event loop and runtime work. Test overload, worker crashes, shutdown with queued jobs, and duplicate delivery. A reliable concurrency design explains ownership and limits at every stage instead of treating more promises or threads as a universal performance solution.

Common errors and fixes

Promise.all overloads a dependency

Why it happens: Every item began immediately without an admission limit.
How to fix it: Use bounded workers and a finite waiting queue.
Prevent it next time: Measure active tasks, queue depth, and rejection rate.

A worker timeout leaks CPU work

Why it happens: Only the caller’s promise was rejected.
How to fix it: Implement cancellation or terminate and replace the worker deliberately.
Prevent it next time: Test unfinished tasks after timeout and shutdown.

More workers increase latency

Why it happens: CPU capacity or transfer overhead is already saturated.
How to fix it: Reduce pool size and profile representative jobs.
Prevent it next time: Include startup and serialization in benchmarks.

Check your understanding

1. Choose a model for waiting on ten HTTP calls and for compressing a large CPU-heavy data structure.

Show worked solution

Use bounded asynchronous I/O for waiting calls. For substantial JavaScript CPU work, benchmark a worker pool after examining native APIs and algorithm choices. The selection follows the bottleneck; a thread per waiting call usually adds needless overhead.

2. A pool runs four jobs and queues at most eight. What should happen to a thirteenth simultaneous job?