Apache Airflow / Intermediate: runtime work, control flow and resource budgets / Lesson 8

Branching, trigger rules and joins

Make conditional paths and join behavior match the meaning of successful, skipped and failed work.

About 5 min readContent reviewed 2026-10-28

On this page

Lesson goals

Explain branching as a task-state decision rather than ordinary Python control flow.
Choose a join rule that accepts intended skips without hiding failures.

Select a path at task execution time

A branch task selects downstream task identifiers at runtime and causes unselected paths to be skipped under Airflow's control-flow rules. A Python if statement at DAG definition time instead chooses graph construction based on information available during parsing. Use runtime branching when the decision depends on the current interval's validated data. The branch return value must match actual downstream identifiers, including any task-group prefix. A label such as archive is not enough if the graph node is group.archive. Keep the decision small and deterministic, and record why the chosen path satisfies the business policy for that run.

A skipped sibling changes the default join

The default all_success rule requires all direct upstream tasks to succeed. After an exclusive branch, one sibling is intentionally skipped, so a join with the default can also skip rather than produce the desired final result. none_failed_min_one_success is often suitable when at least one selected path must succeed and no path may fail. It differs from all_done, which waits for terminal upstream states but can run despite failure. Choose from the actual business contract, not whichever rule makes the diagram green. Failed validation must remain visible even when a cleanup or notification task is allowed to run.

Compare two conceptual join decisions

The local model checks a synthetic pair of terminal labels: success and skipped. all-success is false, while no-failure-with-one-success is true. This is a deliberately narrow conceptual comparison, not a complete Airflow trigger-rule evaluator. Actual states include upstream_failed and mapped task behavior, and scheduler timing also matters. A second assertion adds failed to show why the more permissive intended join should still reject a real failure. Keep these negative cases in review, because a join that accepts a branch skip can accidentally be broadened until it masks the exact business fault it was meant to expose.

python Offline conceptual join comparison

states = ['success', 'skipped']
all_success = all(state == 'success' for state in states)
join_ok = not any(state in {'failed', 'upstream_failed'} for state in states) and 'success' in states
assert not all_success
assert join_ok
failed_states = ['success', 'failed']
assert any(state == 'failed' for state in failed_states)
print(all_success, join_ok)

Expected output

False True

The labels show a skip-compatible join; this is not a complete scheduler rule implementation.

Read a simple decision without executing Airflow

The branch function fragment returns publish only for a positive validated row count and otherwise returns record_empty. It assumes both named tasks are immediate valid downstream choices in the surrounding DAG. The fragment is non-runnable because that graph and runtime are not supplied. A real empty dataset and an unavailable dataset must not share the same branch result unless the business policy explicitly says so. Validate source availability first, then choose whether zero rows are a normal report condition. Return actual task identities and use a separately reviewed join rule; the branch function alone does not determine final DAG status.

python Illustrative runtime branch fragment

from airflow.sdk import task

@task.branch
def choose_path(validated_row_count):
    return 'publish' if validated_row_count > 0 else 'record_empty'

The surrounding DAG must define both downstream identities and its intended join rule; no branch is executed.

Review the final status from the business stages

DAG-run status depends on its terminal leaf task instances and Airflow's state rules. A permissive all_done leaf that always succeeds can make the overall status misleading when an earlier business task failed. Cleanup and notifications need to run reliably without redefining what success means for the report. Design an acceptance task or graph structure that preserves the required failure signal, then verify scenarios with one successful branch, all-skipped paths and an upstream failure. This material runs no branch operator, clears no state and changes no trigger rule in a configured system. Only the supplied Boolean model executes locally.

Common errors and fixes

A join skips after one successful selected branch.

Why it happens: Default all_success also requires the intentionally skipped sibling.
How to fix it: Choose a documented join rule matching allowed skips and forbidden failures.
Prevent it next time: Test the whole branch-and-join state matrix.

A branch returns an identifier that is not a valid downstream node.

Why it happens: Task-group prefixes or task identities were ignored.
How to fix it: Return actual supported downstream identifiers.
Prevent it next time: Keep graph identities stable and covered by definition checks.

An all_done final task hides a failed business stage.

Why it happens: Cleanup success was allowed to define overall acceptance.
How to fix it: Preserve the required failure signal in the graph and acceptance logic.
Prevent it next time: Review leaf states under upstream failure.

Check your understanding

1. Which conceptual join result should success plus skipped produce when one selected branch must succeed?

Show worked solution

A join that requires no failed upstream and at least one success can proceed. The default all_success would not satisfy its condition because skipped is not success. Verify the exact documented trigger rule for the deployed version and include an actual failure case so allowing skips does not become allowing failed validation.

2. Should a missing source file automatically take the normal zero-row branch?