Week 14 - Infrastructure as Code
Why use Infrastructure as Code
This page is optional. Nothing here is required for Week 14's learning goals or the assignment. It covers how infrastructure moved from portal clicks and shell scripts to version-controlled templates, first across the industry, then on Azure specifically, and where Azure IaC stands today. Read it in one sitting, or come back when a job interview asks "how did we end up with declarative cloud templates?"
Week 6's History of Cloud Computing traces how cloud infrastructure itself evolved (portals, APIs, CLIs, object storage). This page covers the next step: how teams learned to describe that infrastructure as code.
Background for Why use Infrastructure as Code
<aside> 🤓 Curious Geek: servers used to arrive in a box
Twenty years ago, getting a new server meant ordering hardware, waiting weeks for a truck, and installing it in a data center by hand. The cloud changed that: you create a server with one API call. Infrastructure as Code goes one step further. Your whole setup lives in a text file. You can compare versions, review changes, and rebuild everything in minutes. An infrastructure engineer used to rack physical machines. Today they write the code that describes those machines.
</aside>
Early cloud (mid-2000s) was operated through web portals: create a VM, attach a disk, open a port. That worked for exploration. It failed for production when you needed the same stack in three environments, when a teammate had to rebuild what you clicked last quarter, or when an incident required knowing exactly what was live.
Cloud providers exposed the same actions as HTTP APIs, then wrapped them in CLIs (az, aws, gcloud). Imperative scripts followed: "create this, then that, then set this property." Scripts were an improvement over memory, but they still encoded steps. Re-running them meant inventing "if it already exists, skip" branches for every resource. Drift (someone edited the portal by hand) was invisible until something broke.
For a data engineer today, that history is why "we set it up in the portal once" is a smell, not a strategy. Capstones and production platforms need a file you can review, re-run, and recover from.
Background for IaC concepts
Declarative configuration is older than Azure. CFEngine (Mark Burgess, 1993) described how machines should look and repeatedly converged reality toward that description. Puppet (2005) and Chef (2009) brought the same idea to large server fleets with richer languages and catalogs. Ansible (2012) made "desired state via YAML playbooks" mainstream for many ops teams.
The shared insight, decades before Bicep: you care about the end state, not the click path. Idempotency (apply twice, same result) is what made those tools safe to run on a schedule. When this week's chapters say Bicep is declarative and idempotent, they are borrowing a proven model from configuration management, applied to cloud resource graphs instead of packages and files on a box.
That framing still pays off: every time you re-deploy an unchanged template and Azure reports no meaningful change, you are using the same convergence idea CFEngine shipped in the 1990s.
Background for Azure Bicep and IaC tool landscape
When infrastructure itself became an API, template languages followed:
Azure still deploys ARM under the hood. Bicep did not replace ARM as the control plane; it replaced hand-authored ARM JSON as the human interface.
<aside> 🤓 Curious Geek: ARM JSON was the compiler target first
If you open a compiled Bicep build today, you still see ARM template JSON. Bicep's bet was linguistic: keep Azure's deployment engine, change the language people write. That is why "Bicep compiles to ARM" is not marketing, it is the architecture.
</aside>
Background for Azure Bicep
Microsoft published the first Bicep release, v0.1, as an alpha on 31 August 2020, and demoed it at Ignite that September. It was not a "use this in production" moment: the team said v0.3 would be the release at parity with ARM templates, and that is when they began recommending production use. Bicep is a domain-specific language that authors readable templates and emits ARM JSON.
It stayed Azure-only on purpose: no separate state file to manage, tight VS Code tooling, and resource types that match Azure's API versions (Microsoft.Storage/storageAccounts@2023-01-01). Microsoft's own framing is that Bicep is "a transparent abstraction over a Resource Manager JSON template", and that Azure stores all state so there is nothing for you to operate.
For Azure-centric data teams, that trade-off usually wins. You already live in az and the portal; Bicep is the smallest jump from "I can deploy a storage account" to "I can review infrastructure in a PR." Terraform remains the better default when your company is multi-cloud; that comparison lives on IaC tool landscape.
As a data engineer on this track, Bicep is the skill that lets you stop treating shared Postgres, storage, and jobs as mysterious teacher-provisioned furniture and start owning the file that creates them.
Background for Bicep in practice
Two operational lessons showed up after teams adopted cloud templates at scale.
Preview before apply. Imperative scripts failed loudly; declarative deploys could change live infrastructure quietly. Azure's what-if (and Terraform's plan) exist because shared environments need a diff humans can refuse. Unexpected lines in that diff are often drift: a hand edit in the portal.
Additive by default. Early adopters expected "remove a resource from the file, next deploy deletes it." Standard ARM/Bicep incremental mode does not work that way: resources linger until you delete them explicitly. Azure did ship a complete mode that deletes whatever the template omits, and it turned out to be too blunt for shared scopes. Microsoft now marks it not recommended and slated for gradual deprecation, pointing teams at deployment stacks instead. That surprise is why this week's teardown practice exists; the current options are compared on Deleting infrastructure.
Modules and secrets arrived for the same reasons libraries and vaults arrived in application code: reuse without copy-paste, and keep passwords out of git. @secure() and Key Vault references are the infrastructure echo of patterns you already used for Postgres URLs in earlier weeks.
Background for Going Further
The story does not stop in 2020. Four things changed after Bicep landed, and together they are the state you are learning into.
Bicep became the recommended way to author. Microsoft's own resource reference now tells readers choosing between the two languages to pick Bicep. ARM JSON did not go away, it became the compile target. That is why az bicep build prints JSON, and why error messages sometimes point at line numbers you never wrote.
Deployment stacks replaced the dangerous delete. Stacks went to public preview in September 2023 and reached general availability in May 2024. A stack tracks the resources it created, so removing a resource from your template can actually remove it from Azure, with an explicit --action-on-unmanage flag deciding whether that means detach or delete. This is the supported answer to the "additive by default" surprise above, and it is why complete mode is being retired rather than fixed.
Shared modules got a distribution story. Bicep modules moved beyond relative file paths: you can publish them to a container registry and reference them by version, the way you pin a Python package. On top of that, Microsoft runs Azure Verified Modules, a library of supported, pre-built modules for Bicep and Terraform that encode Microsoft's own recommended defaults for each resource type.
The tool choice stopped being a fight. Bicep for Azure-only estates, Terraform for multi-cloud, and plenty of teams run both. What settled is the practice rather than the tool: templates in git, a preview step before apply, secrets outside the file, and someone accountable for teardown.
For a data engineer, the practical read is that the interesting problems moved up a layer. Writing a storage account in Bicep is now routine. Deciding which layer of infrastructure your team owns, and how a change to it gets reviewed, is the part that still takes judgement.
| Era | Technology | What it solved | What it left behind |
|---|---|---|---|
| Portal / early cloud | Click-ops, then CLI scripts | Fast first create | No reviewable record; re-runs were fragile |
| Config management | CFEngine, Puppet, Chef, Ansible | Declarative desired state on servers | Not a native cloud resource graph |
| First cloud templates | CloudFormation, ARM JSON, Terraform | Multi-resource deploys as files | Verbose JSON; multi-cloud needed Terraform |
| Azure-native authoring | Bicep (2020+) | Readable Azure templates → ARM | Azure-only; additive delete semantics |
| Safe operations | what-if, modules, Key Vault |
Preview, reuse, secret hygiene | Discipline still required (teardown, no portal edits) |
| Current state | Deployment stacks (GA 2024), module registries, Azure Verified Modules | Real lifecycle management and shared modules | Complete mode being retired; layer ownership is the new hard question |
The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*
Built with ❤️ by the HackYourFuture community · Thank you, contributors
Found a mistake or have a suggestion? Let us know in the feedback form.