CI/CD / Integration and candidate foundations / Lesson 2

Dependencies, parallelism, and pipeline graphs

Design a dependency graph that expresses evidence requirements instead of accidental stage order.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Calculate a simple critical path.
Distinguish scheduling dependencies from artifact transfer.

Draw the graph before naming stages

A pipeline is a collection of jobs with conditions and dependencies. Stages are a convenient ordering model, but they can impose unnecessary barriers: a documentation job need not wait for an unrelated long packaging task. Start by drawing what each job consumes and produces. Parcel has build, unit, package, and documentation jobs. Packaging consumes the build output and successful unit evidence; documentation is independent. The graph should express those facts explicitly rather than relying on a visually pleasing sequence of columns.

Separate elapsed time from total work

Suppose build takes four minutes, unit three, package two, and documentation six. If unit depends on build and package depends on unit, the release path takes nine minutes before queueing. Documentation can run alongside it, making overall completion nine minutes when sufficient capacity exists. Adding job durations gives fifteen minutes of work, not fifteen minutes of elapsed time. The model excludes startup, artifact transfer, and runner contention, so use it to reason about dependencies rather than promise an exact production duration.

python Compute a declared acyclic graph

duration = {"build": 4, "unit": 3, "package": 2, "docs": 6}
parents = {"build": [], "unit": ["build"], "package": ["unit"], "docs": []}
finish = {}
for job in duration:
    finish[job] = duration[job] + max((finish[p] for p in parents[job]), default=0)
print(finish["package"])
print(max(finish.values()))

Expected output

9
9

Insertion order is deliberately topological. This small program would not handle arbitrary input order or detect cycles; a production planner must validate both. All durations are synthetic minutes.

A scheduling edge is not always a file transfer

Different CI products have different defaults for sharing outputs. A job waiting for build does not universally receive its files, and two jobs running on the same named operating system need not share a filesystem. Specify the artifact name, producer, identity, retention, and download rule. Treat workspaces as disposable unless the chosen executor explicitly says otherwise. A package job that accidentally reads a stale file from a reused directory may appear faster while quietly violating the source-to-artifact contract.

Define failure propagation deliberately

Required jobs must block the release path when their evidence is missing or failed. Optional informational analysis can report separately, but it must be visibly optional. Skipped and canceled are not synonyms for successful. A dependency graph also needs a strategy for final diagnostics: some report collection should run after a test failure without turning that failure green. Avoid a blanket ignore-errors setting. Preserve the failing exit status while collecting bounded logs and reports that help explain it.

text Evidence and diagnostic branches

build -> unit -> package
unit failure -> collect-test-report
docs -> documentation artifact

The report branch observes a failed test and preserves that failure. It does not make package eligible. These arrows illustrate desired behavior; each platform needs its own explicit conditions and artifact rules.

Parallelism consumes a finite resource

Independent jobs can still compete for runner slots, CPU, memory, bandwidth, or external rate limits. Splitting a ten-minute test suite into ten jobs is useful only when queueing, setup, and shared bottlenecks permit improvement. First measure queue delay separately from execution. Then change one graph or capacity assumption and compare equivalent revisions. Avoid accelerating packaging while leaving an unnecessary upstream barrier untouched. The safest optimization removes wasted waiting without weakening the evidence needed to release.

Common errors and fixes

A downstream job cannot find the binary.

Why it happens: Scheduling order was mistaken for guaranteed artifact transfer.
How to fix it: Declare the producer artifact and an explicit consumer download path.
Prevent it next time: Review data edges separately from execution edges.

The pipeline is slower after every test becomes a separate job.

Why it happens: Setup costs and runner queues dominate the new parallel layout.
How to fix it: Measure queue and setup time, then choose a useful shard size.
Prevent it next time: Benchmark representative revisions under realistic capacity limits.

A skipped security check still permits release.

Why it happens: The release condition only checked that no visible job was red.
How to fix it: Require explicit successful evidence for mandatory checks.
Prevent it next time: Test failed, canceled, missing, and skipped states in the release contract.

Check your understanding

1. Documentation now takes twelve minutes. What are release-path and whole-pipeline durations in the model?

Show worked solution

The package path remains nine minutes. Whole-pipeline completion becomes twelve because documentation is independent but longer. Whether release must wait for documentation is a policy decision that should be an explicit edge, not an assumption.

2. A report upload runs after failed tests. Should its success satisfy the test requirement?