PyTorch / Storage, restoration, and transformed execution / Lesson 16

Compilation and graph-break contracts

Understand torch.compile as a supported execution transformation with guards, capture limits, and separate performance evidence.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Distinguish ordinary eager execution from graph capture and backend work.
Explain fullgraph and graph-break behavior under the pinned API.
Verify semantics before attributing speed or compilation-count changes.

Compilation is not a new statistical objective

torch.compile transforms a supported Python/tensor execution path through graph capture and a selected backend. It does not change the task’s need for correct labels, shapes, modes, gradients, and evaluation. A compiled function can specialize to properties of inputs or surrounding state and rely on guards for reuse. A later call may require another captured variant when those assumptions do not hold. The exact guard set and internal cache behavior are implementation details to inspect through documented diagnostics, not a list that can be inferred merely from shape or a function name.

Choose a small explicit capture boundary

The illustrative function computes a safe pointwise branch with torch.where and requests backend="eager" and fullgraph=True. This backend choice helps isolate graph capture from lower-level compiler code generation; it is not a promise of acceleration. fullgraph=True requires a single capturable graph under the documented contract and can raise when the program cannot meet that requirement. A successful ordinary eager call alone does not establish capture compatibility. All tensors remain on CPU, and the example has no external I/O, downloaded model, random values, or side-effecting logging inside the captured function.

python Illustrative CPU PyTorch 2.14 API program; torch unavailable, expected output derived

import torch
def reference(x):
    return torch.where(x>0,x*x,-x)
compiled = torch.compile(reference,backend="eager",fullgraph=True)
x = torch.tensor([-2.0,1.0,3.0],dtype=torch.float64,device="cpu")
result = compiled(x)
print("values", result.tolist())
print("same", torch.equal(result,reference(x)))

Expected output

values [2.0, 1.0, 9.0]
same True

The eager backend is a documented graph-capture diagnostic choice, not a code-generation speed claim. No compiler, graph capture, tensor call, or equality check was executed in torch.

Graph breaks and recompilation answer different questions

With a configuration permitting graph breaks, supported regions can be captured around Python work that is not captured. A guard failure instead concerns whether a previously captured variant remains applicable to another call. Either can affect performance, but they are not interchangeable diagnoses. Data-dependent Python control flow, plain-number extraction, unsupported operations, or logging can require investigation under the actual pinned API. Do not remove a meaningful branch merely to avoid an error. Rewrite with supported tensor or control-flow operations only when that preserves the intended function, including boundary and error behavior.

Comparison fixtures must not pretend to be compiler diagnostics

The standard-library program computes the same scalar rule and counts three explicit metadata signatures. It does not implement Dynamo’s guard logic, establish three actual compilations, or predict which shapes a dynamic graph can share. Compare eager and transformed results on representative supported shapes, dtypes, values, modes, and relevant gradients in an available runtime. Use justified tolerances and downstream decisions where exact equality is unnecessary or insufficient. Keep first-call capture and compilation overhead separate from steady execution when evaluating an optimization.

python Executed deterministic standard-library companion

values = [-2.0,1.0,3.0]
result = [x*x if x>0 else -x for x in values]
observed = [((3,),"float64","cpu"),((5,),"float64","cpu"),((3,),"float32","cpu")]
print("values", result)
print("distinct declared signatures", len(set(observed)))

Expected output

values [2.0, 1.0, 9.0]
distinct declared signatures 3

This executed branch calculation and metadata count illustrate a comparison contract. The signatures are not Dynamo guards or a measured compilation count.

Preserve a readable reference and version boundary

Retain the ordinary implementation and a narrow reproducer when a transformation fails or changes behavior. Record the documentation/API version, runtime build, backend, settings, input contract, and actual diagnostics used. Compiled execution, export, serialization, and deployment are separate interfaces with different constraints. This course runs none of them. The torch program is unexecuted, and its output is derived from the specified pointwise formula; only the list calculation and signature counting were executed. No compilation success, graph-break absence, kernel generation, speedup, or training behavior is claimed as tested.

Common errors and fixes

A captured path is assumed valid for every future input

Why it happens: Guard and specialization conditions were ignored.
How to fix it: Test the supported input domain and inspect actual reuse diagnostics.
Prevent it next time: Keep shape, dtype, value-boundary, mode, and error cases in comparison.

backend="eager" is presented as a speed benchmark

Why it happens: A capture diagnostic was confused with optimized code generation.
How to fix it: State what stage the backend exercises and measure performance separately.
Prevent it next time: Separate correctness, capture, lowering, and timing evidence.

A graph-break workaround changes the function

Why it happens: Unsupported logic was removed without preserving semantics.
How to fix it: Reformulate through a supported equivalent or retain the ordinary path.
Prevent it next time: Compare boundary outputs and intended errors before optimizing.

Check your understanding

1. Why is the companion’s count of three signatures not evidence of three torch compilations?

Show worked solution

It is a set count over manually supplied shape, dtype, and device tuples. Dynamo can guard on other properties, share some dynamic cases, or capture multiple regions according to its real execution. Only actual supported diagnostics can establish compilation behavior.

2. What does fullgraph=True add beyond obtaining the right result once in eager execution?