Introduction to Cloud and Azure

Your pipeline is containerized. Now you need a place to run it reliably and at scale. The cloud gives you compute, storage, and managed services without maintaining servers yourself.

By the end of this chapter, you should understand what cloud services are, how Azure organizes them, and why this week builds directly on Week 5.

Concepts

What the cloud gives data pipelines

Cloud services give you:

Scalability: run bigger jobs without buying hardware
Reliability: managed services with backups and monitoring
Speed: provision resources in minutes

Cloud service types

Cloud providers offer services at different levels of abstraction:

IaaS (Infrastructure as a Service): you rent virtual machines and manage the operating system, runtime, and application yourself. Example: Azure Virtual Machines.
PaaS (Platform as a Service): the provider manages the infrastructure and runtime. You deploy your code or container. Example: Azure Container Apps, Azure Database for PostgreSQL.
SaaS (Software as a Service): ready-to-use applications. Example: Microsoft 365, Slack.

<aside> 🖼️ Visual: Cloud Service Types

</aside>

Layers of cloud compute

IaaS, PaaS, and SaaS classify services by who manages what. Underneath, cloud compute also stacks in layers:

Physical server: rack hardware in a datacenter (the provider owns it)
Virtual machine (IaaS): you patch the OS; the provider runs the hardware
Docker container on a VM: your Week 5 image; same bytes locally and in ACR
Serverless: a Container App Job: upload the image, trigger a run, pay per execution

Each step hides more infrastructure from you. Week 6 runs at step 4: you provide the container image and environment variables; Azure provisions the VMs and orchestration underneath.

<aside> 🖼️ Visual: Layers of cloud compute

</aside>

"Serverless" still uses servers; you never SSH into them. Managed Postgres (Chapter 4) is a different kind of PaaS: it runs 24/7, not per job. For the full history (bare metal → VMs → containers → serverless), see History of Cloud Computing.

This week you will use mostly PaaS services: Azure Database for PostgreSQL and Azure Container Apps. PaaS is the sweet spot for data engineering because you focus on your pipeline, not on patching servers.

Classifying a service before you reach for it is the small habit that prevents the most common Week 6 mistake: spinning up a VM for a job that a PaaS could run for a fraction of the cost. The fastest way to internalise this is to price the two options against each other.

<aside> ⌨️ Hands on: Open the Azure pricing calculator and compare two ways to run a small Postgres database for this week's pipeline: (a) a Standard_B2s Virtual Machine in West Europe (IaaS, you install Postgres yourself), and (b) an Azure Database for PostgreSQL Flexible Server on Standard_B1ms in the same region (PaaS, Azure runs the server). Note the monthly EUR cost for each and the gap between them. The chapter on Cost Awareness revisits these numbers.

</aside>

Why this course uses Azure

The three major cloud providers are Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). All three offer compute, storage, networking, and managed databases. The skills you learn this week (working with CLIs, deploying containers, connecting to managed databases) transfer between clouds; each provider has its own naming and quirks, but the concepts are the same.

This course uses Azure because it has a dominant market share in business and enterprise in the Netherlands. The ACM (Authority for Consumers and Markets) estimates that Microsoft and AWS each hold 35% to 40% of the market. Many Dutch companies already have Microsoft contracts (Office 365, Active Directory), which makes adopting Azure easier than bringing in a new vendor. If your future employer uses AWS or GCP instead, you will recognize the patterns.

<aside> 📘 Core program connection: In the Core program Systems week you learned the basics of cloud computing. This week focuses on Azure services for data pipelines. Refresh here: https://www.notion.so/hackyourfuture/Cloud-computing-2d750f64ffc9801ca8a7f14dfe7d513d

</aside>

How Azure is organized

Azure has a clear hierarchy for organizing resources:

Tenant (your organization)
└── Subscription (billing boundary)
    └── Resource group (logical container for a project)
        ├── Azure Database for PostgreSQL
        ├── Azure Container Apps environment
        └── Azure Container Registry

<aside> 🖼️ Visual: Azure Resource Hierarchy

</aside>

Tenant: the top-level identity for your organization. In this course, HackYourFuture is the tenant.
Subscription: controls billing and access. Your Azure for Students subscription lives under the shared tenant.
Resource group: a logical container that groups related resources. When you delete a resource group, everything inside it is deleted too.
Region: each resource group and resource is created in a specific Azure region (like westeurope). Keep all resources for a project in the same region to reduce latency and cost.

<aside> ⌨️ Hands on: Open the Azure portal, navigate to your subscription, and find the resource group your teacher created. Identify which region it uses.

</aside>

Shared responsibility

Azure manages the infrastructure, but you still manage:

Your data quality: Azure ensures the database server is running, but if your Python script inserts duplicated or null rows into PostgreSQL, Azure won't fix it.
Access permissions: Azure provides the login system, but you must explicitly grant your Container App permission to read from the database using identities or connection strings.
Configuration values: Azure runs your container, but you must provide the correct environment variables (like API keys) so your code knows what to do.
Cost visibility: Azure bills you for what you use. If you accidentally leave a heavy process running in a loop all weekend, you are responsible for the bill.

When you are unsure whether a concern falls on your side or Azure's, an LLM can help you reason through the shared-responsibility split.

<aside> 💡 Using AI to help: Describe a configuration or security concern (without secrets) and ask an LLM to label which side of the shared-responsibility line it falls on, then verify against the Azure shared responsibility documentation. (⚠️ Ensure no PII or sensitive company data is included!)

</aside>

Use it as a thinking aid, not a source of truth: confirm Azure's docs before you act on the suggestion.

<aside> 🤓 Curious Geek: The first cloud launch

Amazon launched EC2 in 2006, which popularized pay-as-you-go compute. Azure followed soon after and now offers hundreds of managed services. For a deeper look at how cloud computing evolved, from bare metal to VMs to serverless and from CapEx to pay-per-second billing, see History of Cloud Computing.

</aside>

Knowledge Check

1. Your pipeline needs to run once a day and write results to a database. Would you choose IaaS or PaaS for the database, and why?
1. What happens to all the resources inside a resource group when you delete the resource group?
1. A colleague says "the cloud is just someone else's computer." What nuance does this miss about managed services like Azure Database for PostgreSQL?

<aside> 🚀 Try it in the widget: Interactive Quiz: Introduction to Cloud and Azure

</aside>

https://lasse.be/simple-hyf-teach-widget/mcq.html?bank=week_6_ch1_intro_cloud_azure&embed=1

If the IaaS / PaaS / SaaS split still feels fuzzy after reading, this short Azure-fundamentals video walks the three service models with concrete Azure examples.

<aside> 🎬 Struggling with this concept? Watch this beginner-friendly video:

Watch on YouTube

</aside>

https://www.youtube.com/watch?v=9CVBohl6w0Q

Extra reading

Azure Architecture Guide: reference for designing cloud solutions
Describe cloud concepts (Microsoft Learn): official learning path covering IaaS, PaaS, SaaS, and Azure's hierarchy

<aside> 📚 For full courses, books, and community resources, see the optional Going Further page.

</aside>

Next up: Azure CLI and the Portal, where you log in with az, navigate the shared resource group, and learn the one command pattern that works across every Azure service.

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

CC BY-NC-SA 4.0 Icons

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.