Week 12 - Orchestration

Introduction to Orchestration

Airflow Fundamentals

Scheduling and Triggers

Sequential Pipeline Steps

Parameterized Runs and Backfills

Testing DAGs

Monitoring and Debugging

Deploying to Shared Airflow

Practice

Gotchas & Pitfalls

Assignment: Orchestration

Glossary: Week 12

Career relevance: Week 12

Going Further

History of Data Orchestration

Slides (PDF)

Practice

These exercises help you turn Week 12 concepts into working Airflow behavior. Complete them in order. Each exercise builds toward the assignment.

<aside> 📦 Reference repo: If your taxi_pipeline is broken or you want a known-good starting point, clone lassebenni/nyc-taxi-airflow-reference and git switch ch4-taxi (hardcoded January run) or ch5-params (parameterized monthly). Each checkpoint ships a working DAG plus the dbt project under include/dbt_project/.

</aside>

This is an end-of-week synthesis page, not a chapter-by-chapter drill page. Every exercise reuses the taxi_pipeline you built across Sequential Pipelines, Parameterized Runs and Backfills, Testing DAGs, and Monitoring and Debugging. You are not building a new DAG here; you are exercising the one you already have. If yours is broken, diff against assets/dag_snapshots/taxi_pipeline.py and start from the reference.

<aside> 📝 Practice is optional, but strongly recommended before starting the assignment.

</aside>

Before you run any exercise, set AIRFLOW_STUDENT=yourname in your Astro project's .env and restart the stack (astro dev restart). Without it, your DAG writes to airflow_default on the shared Azure Postgres and collides with every other student who also forgot. Sequential Pipelines covers the why; Practice just needs you to verify the env var is live before Exercise 1.

Suggested pacing

<aside> 💭 If you get blocked for more than 20 minutes, write what you tried, ask for help, and continue with the next exercise. This is professional behavior, not failure.

</aside>

Exercise 1: Run the taxi pipeline locally

Concepts: Astro CLI, DAG discovery, manual trigger.

Primary chapter: Sequential Pipelines

Instructions:

  1. From the Astro project you used during Sequential Pipelines, run astro dev start.
  2. Open the Airflow UI URL Astro printed in your terminal (the http://<project-dirname>.localhost:<random-port> line, not http://localhost:8080: see Airflow Fundamentals) and confirm taxi_pipeline appears in the DAG list.
  3. Keep the DAG paused for the first manual run (a daily schedule can race a scheduled run against your trigger and collide on if_exists="replace"). Then trigger one run. If your DAG is already in the Chapter 5 parameterized form, give it an explicit logical date such as 2024-01-01: in the UI click the dropdown next to TriggerTrigger DAG w/ config and paste {"logical_date": "2024-01-01T00:00:00+00:00"}; or from the terminal run astro dev run dags trigger taxi_pipeline -l 2024-01-01. If you are doing this exercise before Chapter 5, a plain manual trigger is fine: the goal here is simply to confirm that the DAG loads and runs locally.
  4. Watch the Grid view until ingest_taxi_month, dbt_run, and dbt_test are all green. Inspect the dbt_test log and verify the Done. PASS=8 WARN=3 ERROR=0 summary: three warn-level Week 10 data-quality findings (4 duplicates, 3415 null payment_types, 5 orphaned pickup zones) keep the task green.

Success criteria: One green run in your airflow_<yourname> schema, with raw_trips populated and fct_trips rebuilt. If you passed 2024-01-01 as the logical date, that specific partition should have loaded.

Exercise 2: Add scheduling and retries

Concepts: cron schedule, retry configuration.

Primary chapters: Scheduling and Triggers, Monitoring and Debugging

Instructions:

  1. The reference taxi_pipeline uses schedule="@monthly" and default_args={"retries": 2}. Override these per-instance on your own DAG so the scheduler fires at 06:00 UTC on the first of each month, not midnight: change schedule to the cron expression "0 6 1 * *".
  2. Add retry_delay=timedelta(minutes=2) next to retries in default_args.
  3. Reserialize (astro dev run dags reserialize) and open the DAG's Details tab in the UI to confirm both settings are applied.

Success criteria: DAG metadata shows schedule 0 6 1 * *, retries=2, retry_delay=0:02:00.

Exercise 3: Add a quality gate between ingest and dbt

Concepts: task dependencies, TaskFlow return values, >> chaining, DAG integrity test.

Primary chapters: Sequential Pipelines, Testing DAGs

Instructions:

Step 1: Add a check_row_count task between ingest_taxi_month and dbt_run. It takes the row count returned by ingest_taxi_month (an int, not a file path: XCom handles ints cleanly) and raises ValueError if the count is below 1000:

@task()
def check_row_count(row_count: int) -> None:
    if row_count < 1000:
        raise ValueError(f"only {row_count} rows; need at least 1000")

Step 2: Wire it with a TaskFlow call so the int is passed. Do not use bare ingest_taxi_month() >> check_row_count(): that sets order only and leaves row_count unset:

check_row_count(ingest_taxi_month()) >> dbt_run >> dbt_test

Step 3: Before triggering, run the integrity test from Testing DAGs to confirm your edited DAG still parses: astro dev pytest tests/test_dag_integrity.py --args "-v". This is the test-before-run habit that keeps a broken edit off the scheduler.

Step 4: Trigger the DAG normally (plain trigger is fine on the Chapter 4 hardcoded-January DAG). All four tasks should go green.

Step 5: Confirm the Graph view shows the new four-node chain. Optionally raise the threshold to 10_000_000, re-trigger, and confirm dbt_run / dbt_test land in upstream_failed, then revert.

Success criteria: The integrity test passes on the edited DAG; four-node chain visible in Graph view; happy-path run is green; all dependencies fire in order.

Exercise 4: Trigger multiple months and compare ds

Concepts: {{ ds }} templating, parameterized runs.

Primary chapter: Parameterized Runs and Backfills

Instructions:

  1. Manually trigger taxi_pipeline for 2024-01-01, then 2024-02-01, then 2024-03-01 (use the "Trigger DAG w/ config" option if your Airflow version requires an explicit logical date for manual triggers).
  2. Open each run's ingest_taxi_month task and find the ds value in the task log (it will show up in the URL built by parquet_url_for(ds)).
  3. Query airflow_<yourname>.raw_trips grouped by to_char(lpep_pickup_datetime, 'YYYY-MM'). You should see three rows: 2024-01, 2024-02, 2024-03, each with the correct row count for that month's TLC parquet.

Success criteria: Three distinct ds values across three runs; three distinct year-month buckets in raw_trips.

Exercise 5: Controlled backfill with idempotency proof

Concepts: airflow backfill create, idempotent DELETE-then-append.

Primary chapter: Parameterized Runs and Backfills

Instructions:

  1. Run astro dev run backfill create --dag-id taxi_pipeline --from-date 2023-11-01 --to-date 2024-01-31 --max-active-runs 1. Wait for all three runs to go green.
  2. Query raw_trips grouped by year-month: the backfilled 2023-11, 2023-12, and 2024-01 buckets are now present. If you did Exercise 4, you will also still see 2024-02 and 2024-03: delete-then-append only rewrites the months you backfill, so earlier months survive.
  3. Re-run the exact same backfill command. The three backfilled runs should go green again, and their per-month row counts should not change.

Success criteria: The three backfilled buckets (Nov 2023, Dec 2023, Jan 2024) are present with identical counts after the second backfill, and any months loaded in Exercise 4 are untouched. DELETE-then-append keeps each month idempotent.

Exercise 6: Reproduce the Ch7 403 failure

Concepts: log investigation, raise_for_status, 403 vs transient failures.

Primary chapter: Monitoring and Debugging

Instructions:

  1. Edit parquet_url_for in your DAG: append _typo to the filename (as in Monitoring and Debugging): f"{TLC_BASE}/green_tripdata_{year_month}_typo.parquet".
  2. Trigger the DAG. ingest_taxi_month goes red.
  3. Open the task log. Scroll to the bottom, then read upward: the first useful line is HTTPError: 403 Client Error: Forbidden for url: ....
  4. Add one line to your RUNBOOK.md: symptom, check, fix.
  5. Revert the typo. Trigger again. Confirm the run goes green.

Success criteria: You read the 403 from the log without scrolling all over the UI, you wrote one runbook entry, and the revert-and-rerun round-trip completed cleanly.

<aside> 🤓 Curious Geek: Blameless post-mortems

The runbook entry you just wrote is the seed of a post-mortem: a written account of what broke, when, why, and what changes prevent a repeat. Mature data teams write one for every customer-impacting incident, and they write them blamelessly: the document names systems and decisions, never people. The reasoning, popularised by Google's SRE practice, is that engineers who fear being named hide context, and hidden context is what causes the next incident. A weekly review of post-mortems is how a team's collective knowledge grows faster than any single member's experience.

</aside>

Exercise 7: Deploy to the shared Airflow

Concepts: shared-infra workflow, deploy-on-push, multi-student etiquette.

Primary chapters: Testing DAGs, Deploying to Shared Airflow

Instructions:

  1. Take the DAG you iterated on in Exercises 1-6 and copy it into your subdirectory (dags/<yourname>/) of the class's shared Airflow repo.
  2. Namespace the dag_id to <yourname>_<dag_name> and add "student:<yourname>" to the tags list.
  3. Run astro dev pytest tests/test_dag_integrity.py --args "-v" locally before pushing.
  4. Open a pull request, merge once CI (or your local integrity test) is green, and wait up to 60 seconds for the shared dag-processor to re-parse.
  5. Filter the shared UI by your student:<yourname> tag, keep your DAG paused for the first manual run, and trigger with an explicit past logical date (e.g. 2024-01-01) so the TLC parquet actually exists. Verify all three tasks go green: ingest_taxi_month writes to your airflow_<yourname> schema, and dbt_run / dbt_test run dbt through uvx --python 3.11. That works on the shared VM because its Airflow image ships uv and the dbt project lives in the shared include/dbt_project/, so no inline install or image rebuild is needed: see Deploying to Shared Airflow for the rationale.
  6. Find a classmate's DAG on the same UI (different student: tag). Note that you can read it but must not pause or trigger it. Write one sentence on what you observed about their DAG's shape.

Success criteria: Your DAG is parsed by the shared scheduler, all three tasks (ingest_taxi_month → dbt_run → dbt_test) run green, ingest_taxi_month writes to your own airflow_<yourname> schema, the DAG is discoverable via tag filter, and you have not touched any classmate's DAG.

Extra exercises: reference-repo branches

The seven exercises above run against your own taxi_pipeline. For five more focused drills, each isolating one concept from a single chapter, the nyc-taxi-airflow-reference repo ships ready-to-run exercise branches. Each branch starts from a working project with one piece stubbed for you, and has a matching -solution branch to compare against. Clone once, git switch to a branch, and follow its EXERCISE.md:

git clone <https://github.com/lassebenni/nyc-taxi-airflow-reference.git>
cd nyc-taxi-airflow-reference
git switch ex-branch-skip        # then read EXERCISE.md

| --- | --- | --- |

Attempt each branch yourself first, then git switch <branch>-solution to compare against a worked answer.

Career reflection (10 minutes)

After finishing practice, write 5 to 8 lines:

  1. Which failure did you debug?
  2. What evidence proved your fix worked?
  3. Which skill from this week would you mention in a junior data engineering interview?

Add this reflection to your assignment report draft.

<aside> 💡 Using AI to help: Before Exercise 3, paste your taxi_pipeline (redacted: ⚠️ no credentials, no PII, no real database host) into an LLM and ask it to suggest where a quality-gate task could sit without breaking the idempotency contract. Review the suggestion before coding: LLMs occasionally recommend placing the gate after dbt_run, which defeats the purpose of failing fast on bad data.

</aside>

Extra reading


In the next chapter, you will scan the Week 12 gotchas before starting your assignment. Read them fresh: you have just practiced enough that most of them will jump out as "oh, I nearly did that" rather than "I have no idea what this warning is about."