Apache Airflow / Advanced: historical processing, deployment and recovery evidence / Lesson 18
Upgrade compatibility and recovery design
Review version transitions and restore requirements through explicit contracts rather than untested operational recipes.
On this page
Lesson goals
- Identify core, Task SDK, provider and runtime compatibility boundaries.
- Define recovery acceptance for orchestration metadata and business artifacts.
One version label does not describe the deployment
An Airflow deployment includes core services, Task SDK behavior, provider packages, Python runtime, DAG or bundle code and external systems. Compatibility must be reviewed across those boundaries rather than inferred from one Airflow number. Airflow 3 changes task metadata access, public authoring imports and scheduling defaults compared with older examples. The selected 3.1.0 baseline is a teaching choice, not a statement about the latest patch or current support. Preserve an inventory of exact versions and enabled experimental features before planning a change. A DAG that imports successfully under one runtime can still fail when a worker resolves different providers or context semantics.
Review behavior changes as well as import paths
Migrating imports to airflow.sdk is necessary for the public authoring interface but does not by itself migrate business behavior. Review unified schedule usage, trigger versus data-interval timetables, manual or asset contexts without interval keys and retry-safe output handling. Legacy SLA configuration is not equivalent to Airflow 3.1 experimental Deadline Alerts, whose supported callback behavior must be checked. Providers also evolve independently and can change connection options or operator behavior under their own compatibility policy. Build fixtures around the workflow's actual accepted inputs and outputs, then compare old and candidate behavior before a separately authorized deployment operation.
Metadata recovery and data recovery are different
The metadata database records orchestration state, but report artifacts, source snapshots, logs, secret references and bundle versions can live elsewhere. Restoring one database does not automatically restore all of them or make external effects disappear. Define recovery point and recovery time in business terms: which intervals may be missing, which unique operations must not repeat and how long until an accepted report is available. Preserve independent evidence of publication when task-state recording can lag the external action. A restored failed task may need reconciliation before rerun rather than immediate blind execution against an already completed destination effect.
Include validation in the recovery clock
The local model adds detection, restore and business-validation budgets. It produces a declared twenty-minute recovery envelope rather than measuring any real system. Excluding verification would report success before proving interval coverage, artifact integrity or duplicate-safe resumption. The second example describes a compatibility inventory with synthetic labels and a list of required checks. It is not a migration configuration and contains no administrative commands. A real plan must also identify rollback constraints for metadata changes and preserve the supported restoration path. Never assume a package downgrade safely reverses a database schema migration or a changed published business result.
detect_min = 3
restore_min = 12
validate_min = 5
recovery_budget_min = detect_min + restore_min + validate_min
assert recovery_budget_min == 20
print(recovery_budget_min)20Twenty minutes is a declared budget including acceptance checks, not measured restoration performance.
{"core_baseline":"3.1.0","task_sdk":"review-exact-version","providers":"review-exact-pins","dag_bundle":"lab-report-v3","required_checks":["interval-context","retry-output","provider-imports","artifact-coverage"]}Review labels identify missing deployment evidence and do not claim installed package compatibility.
Finish with observable business acceptance
Recovery acceptance should include a known historical interval, an interrupted operation and a fresh interval after restoration. Compare unique identities and authoritative output versions, not merely task colors or service process health. Record what was actually tested and which feature or provider combinations remain unverified. Keep the rollback decision and supported cutover boundary with the evidence. This course runs no upgrade, metadata migration, backup restore, scheduler restart or credential action. Only local inventory parsing and timing arithmetic execute; a deployment-readiness claim requires a separate authorized environment and the exact official version-specific upgrade documentation.
Common errors and fixes
Changing imports is treated as a complete Airflow 3 migration.
- Why it happens
- Behavioral changes in schedules, context and providers were ignored.
- How to fix it
- Compare workflow contracts and version-specific breaking changes.
- Prevent it next time
- Keep acceptance fixtures across the migration.
Metadata restore is assumed to restore all report objects.
- Why it happens
- Orchestration state was confused with external data storage.
- How to fix it
- Inventory source, artifact, bundle, log and secret dependencies separately.
- Prevent it next time
- Test business coverage after restoration, not process health alone.
A package downgrade is assumed to reverse every upgrade effect.
- Why it happens
- Schema and publication rollback boundaries were not reviewed.
- How to fix it
- Follow the supported version-specific restoration and rollback plan.
- Prevent it next time
- Record irreversible or separately reversible steps before change.
Check your understanding
1. Can successful API-server startup prove restored reports are complete?
Show worked solution
No. It establishes only part of service availability. Verify the required intervals, source coverage, artifact versions and uncertain external operations. A healthy orchestration service can still point to missing objects or schedule a duplicate effect, so business acceptance remains a separate recovery step within the time budget.
2. Why must experimental features be listed in the compatibility inventory?
Show worked solution
Their behavior and supported interfaces can change differently from assumptions in a stable workflow contract. Listing them helps reviewers identify which exact documentation and tests apply. For this baseline, distinguish3.1 experimental Deadline Alerts and latest-bundle rerun behavior from later versions, and avoid claiming compatibility without evidence.
Key takeaways
- Version inventory must include behavior, providers and runtime dependencies.
- Recovery ends with accepted business coverage and safe resumption.