GraphQL / Intermediate: policy, sources and clients / Lesson 9

Batching, loaders and the N+1 problem

Reduce repeated source work while preserving request isolation, key order and missing-record semantics.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Recognize resolver fanout and distinguish batching from global caching.
Map unordered batch results back to requested key order.

Measure the actual source call pattern

A list of courses can resolve each author independently. A naive implementation makes one call for the course list and one for every author reference, even when several courses share an author. This is the N+1 pattern. It is not unique to GraphQL, but field-by-field execution makes it easy to introduce. Count source calls and key cardinality for a representative operation before choosing a remedy. A join, source-native batch endpoint or request-scoped loader may fit; there is no rule requiring a loader for every field.

Create the loader inside request scope

A DataLoader-style tool groups compatible key loads and commonly memoizes results within its lifetime. The illustrated context factory creates a new author loader for each authenticated request. The batch function still has to apply the caller’s tenant and visibility policy. A process-global singleton would retain request-dependent results and can cause data leaks or stale reads. The snippet uses the real tool’s conceptual interface but is intentionally non-runnable because no package or data source is installed or contacted by this course.

javascript A request-scoped loader boundary

function makeContext(principal, services, DataLoader) {
  return {
    principal,
    authorLoader: new DataLoader(keys =>
      services.authors.findVisibleInKeyOrder(keys, principal))
  };
}

findVisibleInKeyOrder must return one result for every requested key in the same order, including explicit missing/error outcomes.

Preserve cardinality, order and errors

A batch source may return rows in arbitrary order and omit missing records. A loader must not simply forward that array if consumers expect positions to match their keys. Build an index, then map the original key sequence back to values. Decide how missing or denied records are represented without leaking which condition occurred. Per-key failures and a batch-wide source outage may need different handling. Review the library’s cache and error semantics rather than assuming all failures are retried or all null results are harmless.

Verify mapping with a missing middle key

This offline example deliberately receives rows out of order and lacks a2. Mapping through the requested keys produces a1, null, a3 in the expected positions. It also preserves a duplicate a1 position, which matters for generic batch functions even if a particular loader deduplicates cached requests. The local index assumes unique row IDs within the eligible source result. A production adapter should detect conflicting duplicate rows and apply visibility before indexing rather than using an index as a substitute for authorization.

javascript Offline batch-order model

const keys = ["a1", "a2", "a3", "a1"];
const rows = [{id:"a3",name:"Mira"},{id:"a1",name:"Lina"}];
const byId = new Map(rows.map(row => [row.id, row]));
const ordered = keys.map(key => byId.get(key) ?? null);
console.assert(ordered[0].name === "Lina");
console.assert(ordered[1] === null);
console.assert(ordered[2].name === "Mira");
console.assert(ordered[3] === ordered[0]);

All assertions describe local mapping only. They prove neither database batching nor the scheduler behavior of an installed DataLoader.

Invalidate within the right lifecycle

A mutation may change an author after that author has been memoized in the same request context. Clear or prime affected loader entries according to the chosen library and domain result; a successful write does not magically update every cache. Across requests, use an explicit shared-cache policy if one exists rather than stretching request memoization into an unbounded global cache. Batching reduces call count but not necessarily rows, bytes or CPU. Large batches and deeply nested lists still need bounds, deadlines and operation demand controls.

Common errors and fixes

Names are assigned to the wrong authors

Why it happens: Batch rows were returned in source order instead of key order.
How to fix it: Index eligible rows and map the original keys.
Prevent it next time: Test reversed source order and a missing middle record.

A mutation returns stale author data

Why it happens: The request loader still memoized the previous value.
How to fix it: Clear or prime the relevant entry after confirmed success.
Prevent it next time: Test read-write-read within one request lifecycle.

One giant batch still times out

Why it happens: Call reduction was mistaken for bounded work.
How to fix it: Apply source batch limits and bounded nested pagination.
Prevent it next time: Measure rows, bytes and deadlines as well as call count.

Check your understanding

1. A source returns a3 then a1 for keys a1,a2,a3. What must the batch adapter return?

Show worked solution

It must return the a1 value, the documented missing outcome for a2, and the a3 value in that order. Returning only two source rows shifts positions and corrupts association. Index by the declared key and map each original key, preserving length. Include a test where unauthorized and absent records follow the same public disclosure policy.

2. Why is one DataLoader shared by all requests risky even if it is faster?