Slides (PDF)

The slide deck for the Week 12 in-class session. Structure: teach chapter-by-chapter (Ch1 through Ch8, with two live builds), then one mixed Live Quiz (12 questions, Ch1-Ch8 + gotchas) near the end, with the assignment frame set up front.

Live builds point at the reference-repo checkpoint branches (ch2-hello for §2, ch3-schedule for §3, ch4-taxi for §4, ch5-params for §5), not main's namespaced dags/<student_name>/ example (that is for Ch8 deploy shape only).

Download

week_12__presentation.pdf

What's in it

Welcome, theme (from building pipelines to operating them), where orchestration sits, agenda + assignment frame up front.
§1 Ch1: why orchestration, not cron.
§2 Ch2: Airflow fundamentals + trigger hello_pipeline from the UI (ch2-hello).
§3 Ch3: weekday schedule + catchup on hello_pipeline (ch3-schedule); Practice Ex 2 (taxi cron + retries) is async.
§4 Ch4 (Ex 1 known-good run + live build on ch4-taxi): green taxi_pipeline, then the quality gate on the >> chain.
Break.
§5 Ch5 (live build on ch5-params): parameterized runs, backfill create, and the idempotency proof.
§6 Ch6: testing DAGs (the integrity test; checkpoint ch6-tests).
§7 Ch7: monitoring and reading logs bottom-up.
§8 Ch8: deploying to shared Airflow (namespaced example: main / dags/<student_name>/).
§9 Live Quiz (one mixed round, 12 questions, Ch1-Ch8 + gotchas).
§10 Q&A + Week 13 preview (Big Data at scale / Delta Lake).

Teacher script and pacing notes: GitHub-only week_12__lesson_plan.md.

To regenerate after editing the Marp source: just slides 12 from the repo root.

Exercises (live in class)

Section	Exercise	Start branch	Solution check	What you practice
§2 Ch2	Live: trigger `hello_pipeline`	ch2-hello	`ch2-hello`	Astro start, DAG list, Grid view, first green run
§3 Ch3	Live: weekday schedule + catchup	ch2-hello	ch3-schedule	Edit cron, catchup flood demo, Details / Next Run
§4 Ch4	Ex 1: Run the taxi pipeline locally	ch4-taxi	`ch4-taxi`	Known-good three-task run (plain trigger) + `dbt_test` PASS=8 WARN=3
§4 Ch4 (live build)	Ex 3: Quality gate between ingest and dbt	ch4-taxi	Slide TaskFlow form (no gate `-solution`; optional ex-branch-skip-solution)	TaskFlow call wiring, gate task, `upstream_failed`
§5 Ch5 (live build)	Ex 5: Controlled backfill with idempotency proof	ch5-params	`ch5-params`	`backfill create` over a range, prove reruns leave row counts unchanged
§7 Ch7	Ex 6: Reproduce the 403	ch5-params	`ch5-params` (clean after revert)	Log bottom-up, runbook line, revert + green rerun
§8 Ch8	Deploy to shared Airflow	`ch5-params` (local DAG)	`main` → `dags/<student_name>/taxi_pipeline.py`	Namespace, tag, PR auto-merge

Full instructions: Practice chapter. Extra ex-* / -solution drills are listed there too.

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

CC BY-NC-SA 4.0 Icons

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.