Week 13 - Big Data & Streaming

Introduction to Big Data and Streaming

Apache Spark Core Concepts

Databricks

Streaming Theory

Streaming Platforms: Kafka and Azure Event Hubs

Gotchas & Pitfalls

Practice

Assignment

Week 13 Lesson Plan (Teachers)

Week 13 Lesson Plan (Teachers)

Content coming soon...

Suggested Topics to Cover

• Demo: live Databricks notebook walkthrough showing cluster startup, data loading, and a PySpark transformation
• Discussion: when do you actually need Spark vs when is pandas enough
• Demo: create an Azure Event Hub namespace and show messages flowing through the portal
• Workshop: students run a PySpark notebook in Databricks (guided lab)
• Workshop: students explore Azure Event Hubs by sending and receiving test messages (guided lab)
• Whiteboard session: drawing the pub/sub architecture and explaining consumer groups
• Discussion: real-world streaming use cases and when batch is the better choice
• Assessment rubric: working Databricks notebook, understanding of streaming concepts, comparative write-up

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.