Week 13 - Big Data on Databricks
The slide deck for the Week 13 mid-week in-class session. Students are already on the shared Databricks workspace. Structure: progress check → Exercises A–E frame → lock-in / live Exercises A–E → assignment close. Live exercises use workspace artifacts (SQL editor, notebook, local dbt project, Jobs UI), not a reference-repo git switch walk. The reference branch week-13-ch-4-dbt-solution is catch-up / teacher Job demo only.
Interactive present-mode deck (arrow keys / click). Includes the Ch4 compile-fork and MERGE mini-sim widgets that the PDF cannot show:
https://hackyourfuture.github.io/Learning-Resources/data-track/embeds/week-13-slides/week_13__presentation.html
week_13__presentation.pdf
count(*).show(), live call-out for top borough then solution slide, then avg total_amount (no Jupyter detour).dbt debug, incremental config, compile-fork + MERGE mini-sim (HTML present-mode), commented full model, build twice + Delta history.Teacher script, pacing, and quiz cribs: GitHub-only week_13__lesson_plan.md.
To regenerate after editing the Marp source: just slides-rebuild 13 from the repo root (PDF + HTML + slide-hash comments). Teachers present iframe slides with just slides-present 13. While iterating on the deck locally, just slides-dev 13 watches the Marp file and reloads the browser after each HTML rebuild (~1 min per save).
| Ex / live | Section | Start | Success check | Feeds |
|---|---|---|---|---|
| Exercise A | §2 Ch2 | SQL editor + hyf-dbt-warehouse |
count(*) ≈ 128,202,548 |
Ch2 lock-in |
| Exercise B | §3 Ch3 | Python notebook + shared cluster | Top pickup borough + avg total_amount by payment_type via show() |
Task 1 |
| Exercise C | §4 Ch4 | Own dbt project + warehouse connection details | dbt debug → All checks passed! |
Task 2 |
| Exercise D | §4 Ch4 | Incremental fct_trips |
Two timings; second clearly faster | Task 2 |
| Exercise E | §5 Ch5 | Workflows → Job (demo then own fork) | Green Run now + paused schedule + Run URL | Task 3 |
Live Exercises A–E cover the workspace spine. Ch1 is teacher lock-in only (Hands-on stays in the chapter). Catch-up only (not the class spine): nyc-taxi-dbt-reference branch week-13-ch-4-dbt-solution if the port is broken. Async Practice exercises remain on the Practice page.
The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

Built with ❤️ by the HackYourFuture community · Thank you, contributors
Found a mistake or have a suggestion? Let us know in the feedback form.