Apache Spark / Basic: typed relational foundations

Apache Spark β€” Basic: typed relational foundations

Locate execution, preserve measurement meaning and build correct query populations and totals.

6 lessons Β· No account required

What you will learn

  • Explain laziness, schemas, NULL, ANSI behavior and time boundaries.
  • Validate joins and aggregate completeness at the intended grain.

  1. Driver, executors and lazy plans

    Locate a DataFrame operation in Spark execution and distinguish constructing a plan from computing its result.

    Start learning

  2. Schemas, types and source boundaries

    Read meter data through an explicit typed contract and preserve rejected values for diagnosis.

    Start learning

  3. Column expressions, NULL and ANSI behavior

    Make missing values and invalid measurements explicit instead of hiding them inside casts or Python control flow.

    Start learning

  4. Filtering, time zones and deterministic order

    Express a reporting interval and stable output without confusing event instants, wall clocks and partition order.

    Start learning

  5. Joins, grain and reference data

    Enrich readings without multiplying energy or losing sites whose reference data is incomplete.

    Start learning

  6. Aggregations and completeness metrics

    Summarize energy with an explicit population, denominator and missing-reading policy.

    Start learning
Back to course