Free to read ยท Three levels
Data Engineering: correct production data flows
A complete bilingual theory course in reliable source-to-consumer delivery: contracts, formats, SQL, increments, event time, orchestration, quality, storage and recovery.
Start learningIn this course
Build dependable production flows through synthetic shipment events, accepted batch versions and isolated relational fixtures. Eighteen developed lessons connect data meaning to identity, completeness, publication and operational ownership.
Every lesson includes two explained examples, cause/fix/prevention mistakes, complete worked exercises, takeaways and primary references. Safe Python programs and SQLite memory examples were executed in isolation; distributed platform behavior is explicitly modeled rather than claimed as deployed.
What you will learn
- Define data contracts, grain, units, source identity and acceptance boundaries.
- Design repeatable batches, incremental cursors, CDC, event-time correction and replay-safe sinks.
- Evaluate partitioning, orchestration, lineage, table snapshots, security and data-level recovery evidence.
Your learning path
Three levels
Basic: contracts and repeatable batches
Define formats, meaning, relational grain and accepted publication.
- Data flows, ownership and contracts
- Formats, encoding and file boundaries
- Schemas, types, nulls and time
- SQL grain, joins and reconciliation
- ETL, ELT, staging and publication
- Deterministic batches and idempotent runs
Intermediate: change, time and coordination
Coordinate increments, CDC, late data, layout, tasks and quality evidence.
- Incremental cursors and checkpoints
- Change data capture and versioned state
- Event time, windows and late data
- Partitions, file layout and compaction
- Orchestration, dependencies and retries
- Data quality, rejects and lineage
Advanced: distributed correctness and reliability
Review storage/transaction boundaries, backfills, security, service evidence and recovery.
- Warehouses, lakes and snapshot tables
- Delivery semantics and transactional sinks
- Backfills, schema evolution and cutover
- Security, privacy and access boundaries
- Freshness, observability and cost
- Recovery, replay and operational reliability
Who this is for
- Developers and data practitioners building repeatable ingestion, transformation and publication workflows.
- Engineers reviewing production data correctness, storage, privacy, observability and recovery.
Before you start
- Basic Python, tabular data and introductory SQL familiarity.
- Examples use invented records and standard-library tools; no cloud account, scheduler, broker or configured database is required.