CI/CD / Promotion and environment contracts / Lesson 10

Progressive delivery and bounded exposure

Choose a rollout strategy by compatibility, evidence quality, and recovery behavior.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Compare rolling, blue-green, and canary adoption.
Define a stop rule before increasing exposure.

Choose the failure boundary

A rolling release replaces instances gradually and normally creates a period where old and new versions coexist. Blue-green keeps two environments and switches traffic between them, trading additional capacity for a clearer traffic boundary. A canary exposes a limited population before broader adoption. None guarantees safety by itself. Parcel must consider shared database compatibility, session behavior, asynchronous work, and external side effects. Switching traffic back cannot undo a label already purchased from a carrier, so the strategy must match the actual reversible boundary.

A percentage is not a representative sample

Five percent of requests may all come from one geography, one tenant, or one cache-heavy route. A canary that sees no address-label creation cannot reveal a label-creation defect. Define the unit of exposure: request, user, tenant, worker, or region. Keep assignment stable where stateful behavior requires it. Compare relevant populations and account for low traffic. A short quiet window is absence of evidence, not evidence of health. Decide what minimum useful observation is required before expanding.

Specify a bounded evaluation rule

The following complete model requires at least two hundred synthetic observations and an error fraction no greater than one percent. These thresholds are teaching choices, not universal production recommendations. A real rule needs service objectives, traffic context, baseline comparison, and statistical reasoning. It should also distinguish telemetry failure from healthy results. In this model insufficient data produces hold, while an observed bad fraction produces stop; neither automatically expands the rollout.

python Evaluate synthetic canary observations

def decision(requests, errors):
    if requests < 200:
        return "hold"
    return "expand" if errors / requests <= 0.01 else "stop"
for sample in [(50, 0), (1000, 8), (1000, 25)]:
    print(decision(*sample))

Expected output

hold
expand
stop

The program does not generate traffic or access monitoring. It illustrates explicit insufficient-evidence handling and a declared rule, not a statistically validated canary analyzer.

Separate code adoption from feature exposure

A feature flag can keep a new code path disabled after package adoption, allowing a separate exposure decision. That helps only when both paths are tested, defaults are safe, evaluation failure is defined, and the flag has an owner and removal plan. Flags can become persistent configuration complexity. Do not place authorization solely in a client-visible flag, and do not assume disabling a feature reverses stored data or completed external actions. Record package identity and flag state when diagnosing an outcome.

json A conceptual feature exposure contract

{"candidate":"candidate-r8","feature":"new-label-layout","exposure":"lab-cohort","on_evaluation_failure":"old-layout"}

This synthetic record states a failure choice for a non-security presentation feature. It is not an SDK configuration and does not establish authorization or route real traffic. Actual failure behavior must be implemented and tested in the application.

Decide who stops and what remains

Before beginning a rollout, identify stop authority, rollback candidate, incompatible state changes, and the evidence needed to resume. Pausing exposure may leave already updated instances running; rollback may require a different procedure. Keep those states visible. A healthy canary does not prove that a full population will fit capacity or exercise identical paths, so continue observation during expansion. After completion, retain the decision record and remove obsolete temporary controls through normal change review rather than leaving unexplained switches behind.

Common errors and fixes

A canary expands after receiving no useful traffic.

Why it happens: Time elapsed was substituted for representative observations.
How to fix it: Hold until the declared evidence requirement is met or use a reviewed alternative check.
Prevent it next time: Define population, minimum evidence, and telemetry failure behavior.

Switching traffic back is assumed to undo external side effects.

Why it happens: Routing reversibility was confused with business-state reversibility.
How to fix it: Plan compensation and compatibility for completed actions.
Prevent it next time: Inventory stateful and external effects before choosing a strategy.

A feature flag becomes an undocumented permanent dependency.

Why it happens: No owner, expiry, or removal condition was assigned.
How to fix it: Document behavior and remove obsolete paths through review.
Prevent it next time: Track flag lifecycle and test both active paths while needed.

Check your understanding

1. A canary handles read traffic successfully but no label purchases. Can it validate a purchase change?

Show worked solution

No. Its observed population does not exercise the changed behavior. Adjust the controlled cohort or use a separately authorized representative validation path, then observe relevant success and failure signals. Do not infer purchase safety from unrelated read requests.

2. Blue-green shares a database. Is reverting the traffic switch sufficient after a destructive schema change?