Week 4

VPC - An analogy about shared resources

Public subnets and Internet Gateway

Private subnets and NAT

Endpoints and Security Groups

Putting it all together

Load balancing - one front door, many clerks

Auto Scaling

Architecture and Tradeoffs

Practice

Assignment

Cloud Track

🗺️ Where part 1 left us

👀 Look at the first row again. Everything the internet sees is one machine, in one Availability Zone. Stop that instance, let its disk fill up, or lose eu-central-1a for an hour, and the site is gone — however carefully the security groups behind it were written.

🏪 One clerk at one counter

Back to the building. Our shop has exactly one clerk, at one counter, in one district. Every visitor walks straight up to them. If they are ill, the shop is closed. If a hundred visitors show up at once, they queue out of the door.

🎉 Today we hire a receptionist (the load balancer), open a second counter in another district (AZ B), and sign a contract with a staffing agency (the Auto Scaling group) that always keeps enough clerks on shift — and replaces any who don't show up.

Source: https://excalidraw.com/#json=lW1NxsB0_ITvXnJdrXueq,thy44V3mOVapT77b34cW6w

The rest of part 2 uses this mapping. Keep it in mind; every new AWS word below has a place in it.

📈 Scaling up: vertical or horizontal

When more visitors arrive than one clerk can handle, the shop needs to scale up: get more capacity. There are two directions to do it, and the difference matters a lot:

• ⬆️ Vertical scaling — give the one clerk a faster desk: move the server to a bigger instance type, with more CPU and memory. Simple, and nothing else in the architecture changes. But there is a ceiling (the biggest instance AWS sells), resizing an EC2 instance means stopping it first, and it is still one clerk. If they are ill, the shop is still closed.
• ↔️ Horizontal scaling — hire more clerks: run more instances of the same size, side by side. There is no real ceiling, adding one doesn't interrupt the others, and losing one clerk only means a slightly longer queue.

Horizontal scaling is what the cloud is built for, and it fixes both problems at once: capacity (more clerks) and availability (no single clerk the shop depends on). But it creates a new question — if there are four clerks, which one does a visitor go to? That is the load balancer's job.

<aside> 💭

Horizontal scaling only works if any server can answer any request. If a server keeps something important on its own disk or in its own memory — a logged-in user's session, an uploaded file — the next request may land on a different server that has never heard of it. Applications built to keep that kind of data elsewhere (a database, S3, a shared cache) are called stateless, and they are the ones that scale horizontally without surprises.

</aside>

🛎️ What a load balancer does

A load balancer sits between your users and your servers, and does three things:

1. 📍 Gives you one stable address. Users talk to the load balancer's DNS name, never to an individual server. Servers can come and go behind it without anyone outside noticing.
2. 🔀 Spreads requests across servers. Each new request goes to one of the servers in its list, so no single machine takes all the load.
3. 🚑 Stops sending traffic to sick servers. It constantly checks each server's health, and quietly takes a failing one out of rotation until it recovers.

<aside> 💭

Always use the load balancer's DNS name (something like hyf-week4-alb-123456789.eu-central-1.elb.amazonaws.com), never the IP addresses it resolves to. AWS changes those IPs as it scales the load balancer itself — the name is the only thing that stays the same.

</aside>

🏆 What we gain: stability and availability

Putting a load balancer in front of several servers buys you two things:

• 🧘 Stability — one server crashing, being restarted or being replaced stops being an event your users notice. You can also update servers one at a time while the others keep serving.
• ⏱️ Availability — the share of time your service actually answers. This is the number the business cares about, and it is usually written as a percentage made of "nines".

Each extra nine divides the downtime you are allowed by ten. A real example of the same idea: AWS promises 99.99 % monthly uptime for a load balancer spread over several Availability Zones, but only 99.9 % for one that lives in a single AZ (Elastic Load Balancing SLA). Multi-AZ is literally worth one extra nine in the contract.

⚠️ One trap to know about: availability multiplies along a chain. If the web tier, the app tier and the database are each 99.9 %, and a request needs all three, the whole site is about 99.9 % × 99.9 % × 99.9 % ≈ 99.7 % — worse than any single part. That is why one single point of failure anywhere in the chain matters.

<aside> 💼

The business tradeoff. Every extra nine costs roughly ten times more effort and money: more servers in more AZs (or even regions), automatic failover, engineers on call at night, and regular tests where you break things on purpose. Whether that is worth it depends on what an hour of downtime costs the business. An internal tool used during office hours can live with 99 %. A web shop loses sales every minute it is down. A payment or hospital system may need five nines. Engineers don't choose this number alone — it is agreed with the business, and written down as an SLA (a promise to customers, often with money back if it is broken) or an SLO (an internal target). Ask "what does an hour of downtime cost us?" before asking "how do we get another nine?".

</aside>

🤔 Which load balancer?

Before looking at how a load balancer routes requests, it helps to know what it can actually see of a request. That depends on the network layer it works at:

• ✉️ A layer 4 load balancer only sees IP addresses and ports — the address on the envelope. It forwards connections without reading what is inside them.
• 💌 A layer 7 load balancer opens the envelope and reads the HTTP request inside: the URL path, the host name, the headers. That is what lets it make decisions like "send /api/* to these servers".

AWS offers several types. You will see their names everywhere, so it is worth recognising them — but in this course we only use the ALB.

<aside> 🛡️

There is a fourth type: the Gateway Load Balancer (GWLB). It works even lower, at layer 3 (the network layer), and is used to pass traffic through a fleet of virtual firewalls or other network appliances. You won't need it in this course — just know the name exists.

</aside>

📖 The vocabulary

An AWS load balancer is built from four pieces, and a request passes through them in this order:

1. 👂 A listener waits on a protocol and port — for us, HTTP:80. Nothing reaches the load balancer except through a listener.
2. 📜 Each listener has rules. A rule says "requests matching this go to that target group". We only use the default rule ("everything"), but a rule could also say "/api/* goes to one group, everything else to another".
3. 👥 A target group is the list of servers that can answer — plus the health check settings for them.
4. 🎯 The targets are the servers themselves: for us, EC2 instances on port 80.

<aside> 💭

Targets don't have to be EC2 instances.

An ALB target group has one of three target types, chosen when you create it:

• instances (by instance ID — what we use)
• IP addresses (any private IP in the VPC; this is how containers running on ECS are usually registered, one IP per container)
• Lambda function (code that runs once per request, with no server for you to manage at all).

A Network Load Balancer can even use a whole ALB as its target. Listeners and rules work exactly the same whatever is behind the target group.

</aside>

🏢🏢 An ALB always lives in at least two Availability Zones

<aside> 📌

The one fact to take away from this page: an Application Load Balancer always lives in at least two Availability Zones. AWS will not let you create one in a single AZ.

</aside>

An Application Load Balancer is not one machine. When you create it you pick one subnet per Availability Zone, and AWS runs a load balancer node in each of them. Its DNS name resolves to all of those nodes at once. If one AZ goes dark, DNS stops returning that node and the others carry on.

AWS makes this non-negotiable: an ALB must be placed in at least two AZs. You cannot build a single-AZ ALB even if you want to. That is why the next page starts by creating a second public subnet, in eu-central-1b. Each subnet you give the ALB must be in a different AZ, and needs at least a /27 CIDR block with at least 8 free IP addresses.

🔄 By default an ALB also does cross-zone load balancing (always on for the load balancer itself; it can only be switched off per target group): a node in AZ A is happy to send a request to a server in AZ B, and the other way around. So traffic is spread evenly across servers, not just across AZs.

🩺 Health checks: "you OK?"