Week 14 - Infrastructure & Data Loading

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

CC BY-NC-SA 4.0 Icons

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.

Introduction to IaC and Data Loading

Infrastructure as Code Concepts

Azure Bicep

Data Loading Tools (Fivetran/Airbyte)

Week 14 Lesson Plan (Teachers)

Week 14 - Infrastructure & Data Loading

Welcome to Week 14! Building data pipelines is only half the job. You also need to provision the infrastructure they run on and connect to data sources reliably. This week covers two complementary topics: Infrastructure as Code (IaC) with Azure Bicep for reproducible cloud deployments, and managed data loading tools like Fivetran and Airbyte that replace hand-written ingestion scripts.

By the end of this week, you will have deployed cloud resources using Azure Bicep and configured a data source-to-destination pipeline using Airbyte Cloud.

Learning goals

Understand what Infrastructure as Code means and why it is preferable to manual cloud setup
Write and deploy Azure Bicep templates to provision cloud resources
Understand the Bicep workflow: author a template, deploy with az deployment group create, and verify in the Azure portal
Explain what managed data loading tools like Fivetran and Airbyte do and when to use them instead of custom scripts
Configure a source-to-destination connector in Airbyte Cloud and run a sync
Compare build-vs-buy decisions for data ingestion: custom Python scripts vs managed connectors
Recognize how IaC and managed loading tools fit into a production data platform

Prerequisites

Azure for Students account with active subscription
Azure CLI installed with Bicep support (built-in since Azure CLI 2.20+)
Airbyte Cloud free account

Chapters

Teachers

Lesson plan