Apache Airflow / Basic: definitions, intervals and safe task execution / Lesson 2

TaskFlow, dependencies and execution context

Build a small declarative task graph while keeping actual computation inside task execution.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Explain what calling a decorated task means during DAG construction.
Use task context without importing private runtime internals.

Task calls describe work before workers execute it

Within a DAG definition, calling a TaskFlow-decorated function constructs a task relationship and an output reference rather than immediately running the business function as ordinary Python would. Passing that output reference to another task establishes a dependency and a data contract. This difference is why printing the result while defining a DAG does not show the eventual computed value. Keep graph construction cheap and deterministic. File downloads, database queries and expensive calculations belong in task execution, not at module import. Parsing can happen repeatedly, so an innocent-looking top-level function call can create repeated side effects before any scheduled run exists.

Read a complete illustrative DAG

The example defines a manually triggered report with two tasks. manifest returns two synthetic weights, and total consumes that declared output and sums it. Airflow 3.1 authoring imports come from airflow.sdk; the DAG factory is called so the definition is available for discovery. The expected total is five under these fixtures, but this framework program is not executed in the course. A compatible Airflow installation, metadata state and task runtime would be needed to verify scheduling and XCom behavior. The code includes no external connection, real data or deployment configuration and is labeled non-runnable accordingly.

python Illustrative Airflow 3.1 TaskFlow DAG

from datetime import datetime, timezone
from airflow.sdk import dag, task

@dag(schedule=None, start_date=datetime(2026, 1, 1, tzinfo=timezone.utc), catchup=False)
def lab_weight_report():
    @task
    def manifest():
        return [2, 3]

    @task
    def total(weights):
        return sum(weights)

    total(manifest())

lab_weight_report()

The declared total is 5; no Airflow import or execution is performed here.

Read context only where it exists

Runtime context carries information about the current execution, such as run identity and interval boundaries. The public get_current_context interface is intended for task execution context, not arbitrary top-level import code. Keep context-dependent logic close to the task boundary, then pass explicit plain values into pure helper functions. That separation makes business logic testable without pretending to create an Airflow task instance. It also avoids hiding time or identity dependencies in global variables. Do not import metadata models or private scheduler modules to discover a run from task code; the Airflow 3 public interface establishes a different integration boundary.

Test computation separately from orchestration

The local helper receives a list of integer weights and returns their sum. Its assertions cover an ordinary fixture and an empty manifest, whose defined result is zero. This establishes a small business rule without evaluating DAG parsing, task state or XCom serialization. If an empty manifest should instead fail, change the contract and test that decision explicitly rather than relying on a downstream task to crash accidentally. Validate types and units at the input boundary when real data arrives. A function that silently sums kilograms and grams together remains wrong even if Airflow schedules it perfectly.

python Offline pure task-logic check

def total_weight(weights_g):
    return sum(weights_g)

assert total_weight([2, 3]) == 5
assert total_weight([]) == 0
print(total_weight([2, 3]))

Expected output

This verifies summation only, independently of TaskFlow and task communication.

Use dependencies to express real prerequisites

An explicit dependency arrow means the downstream task waits under its trigger rule; it does not copy arbitrary local variables or reserve the same worker. Passing a TaskFlow output combines dependency and value reference, but it still uses task communication semantics rather than one Python call stack. Use stable task identifiers and small named helper functions so the graph reflects the business stages. Avoid enormous graphs of trivial arithmetic tasks merely to display more boxes: each task has scheduling, logging and retry overhead. Conversely, one giant task can hide useful recovery boundaries, so choose units that have meaningful inputs, outputs and retry behavior.

Common errors and fixes

A network request runs every time the DAG is parsed.

Why it happens: Business work was placed at module import.
How to fix it: Move the operation inside task execution and keep definition-time inputs bounded.
Prevent it next time: Review top-level calls and expensive imports.

A decorated task output is treated as an immediate Python result.

Why it happens: Graph construction was confused with execution.
How to fix it: Pass the output reference to a task or inspect the eventual runtime result.
Prevent it next time: Separate pure helpers from decorated orchestration functions.

Context lookup fails during import.

Why it happens: No task execution context exists at that point.
How to fix it: Read context inside the task and pass explicit values to helpers.
Prevent it next time: Keep runtime context out of global initialization.

Check your understanding

1. Why must the decorated DAG factory be invoked in the example?

Show worked solution

The invocation constructs the workflow definition that Airflow can discover. Merely defining the factory leaves a callable rather than constructing its graph. This is definition-time construction, not execution of manifest and total. Keep that distinction explicit when reviewing module-level code for accidental external side effects.

2. Does the pure-helper test verify XCom can transport the result in the selected deployment?