Week 15 - Final Project

Project Ideas and Inspiration

Cross-Track Collaboration

The Project Architecture

Ingestion, ACA, and CI/CD

dbt models

Publish to Postgres

Running the Pipeline Locally

Orchestrating with Airflow

Promoting to Production

Secrets & Variables

Optional: Failure Alerting

Optional: Enriching with an LLM

Optional: The Health Page

Optional: Inbound Sync

Team Timelines

Gotchas & Pitfalls

Frequently Asked Questions

Glossary: Week 15

Career relevance: Week 15

Final Project Brief

Capstone orientation for the Data Track. Your team builds a cross-track web app in one monorepo. You own the batch data pipeline: fetch an external source, model it in dbt on Databricks, publish marts to the backend's analytics schema on a schedule (Airflow). Backend and frontend own the app; your interface to them is the mart contract.

<aside> 💡 "Week 1" here means the first calendar week of the project, not HYF curriculum Week 15. Week 6 is the demo.

</aside>

The project is the assessment (no quizzes). Deliverables and demo shape: Requirements and Deliverables. How to build: start at Project Ideas, then Architecture → stage chapters → Path A.

Hard rules (kickoff)

• Pull requests only - no direct pushes to main
• One real source first (dev + prod green) before a second source, enrichment, or tool split - Team Timelines
• Real samples for backend - no fake source JSON
• Transforms in dbt (Pydantic in ingest); Pandas only with written justification
• Laptop writes dev / dev_<name> / analytics_dev only - prod via merge + scheduled run

Week 1 checklist

Do these before week-2 feature work on your own feed:

• [ ] Team brainstorm (must-have vs nice-to-have, all tracks)
• [ ] All DEs pick one source; write a source report - Project Ideas
• [ ] Starter pipeline green: stage maps → Path A → Mode 1 → one prod baseline trigger (Ingestion, dbt, Publish, Running the Pipeline, Airflow, Promoting)
• [ ] Architecture diagram in the team README (edit whenever something changes)
• [ ] Real JSON sample + field list to backend/frontend before dbt or container PRs
• [ ] Mart contract draft with backend (one wide table) - Cross-Track Collaboration

Extra reading

• Requirements and Deliverables
• Team Timelines
• Your team's data/docs/dev_flow.md

Students start at Project Ideas and Inspiration.

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 https://hackyourfuture.net/

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.