Week 5 - Containers & CI/CD

Introduction to Containers and CI/CD

Dependency Management

Docker Fundamentals

Azure Container Registry

Python CI Pipeline

Week 5 Gotchas & Pitfalls

Practice

Week 5 Assignment: Containerize and Ship a Data Pipeline

Week 5 Lesson Plan

🎒 Week 5 Assignment: Containerize and Ship a Data Pipeline

In Week 3 you built a robust ingestion pipeline. In Week 4 you processed data with Pandas. Now you will package that pipeline as a container and automate its delivery to Azure Container Registry.

Your goal is to make your pipeline reproducible and deployable with one CI workflow.

Task 1: Choose a Pipeline

Pick one pipeline you already built:

• Your Week 3 API ingestion pipeline, or
• Your Week 4 Pandas transformation pipeline

Copy the code into a new assignment folder and make sure it runs locally.

<aside> 💡 Choose the pipeline you understand best. You will debug it inside a container.

</aside>

Task 2: Define Dependencies

Create a requirements.txt or a pyproject.toml with pinned versions. Your pipeline must run in a clean virtual environment.

<aside> ⚠️ If you rely on system-wide packages, your container will fail in CI.

</aside>

Task 3: Write a Dockerfile

Create a Dockerfile that:

1. Uses Python 3.11
2. Installs dependencies
3. Copies your source code
4. Runs the pipeline

Test it locally with docker build and docker run.

Task 4: Add Configuration

Move any secrets or runtime values into environment variables. Your container must read configuration from the environment, not from hardcoded values.

<aside> ⌨️ Hands on: Run your container with a missing env var and confirm it fails fast with a clear error.

</aside>

Task 5: Build a CI Workflow

Create .github/workflows/ci.yml that:

1. Installs dependencies
2. Runs ruff (see Python CI Pipeline)
3. Runs pytest (see Testing with pytest, Week 2)
4. Builds the Docker image

Task 6: Push to Azure Container Registry

Use GitHub Actions to push your image to ACR. The workflow should tag the image with the Git commit SHA.

Task 7: AI Assist Report

Create a short file AI_ASSIST.md and describe:

• The prompt you gave an LLM
• The code or suggestion it returned
• What you changed after reviewing it

Deliverables

You must submit:

• A working Dockerfile
• A requirements.txt or pyproject.toml with pinned versions
• A GitHub Actions workflow that passes
• An image pushed to ACR with a SHA tag
• AI_ASSIST.md

Technical Requirements

• Use a clear project structure, for example:

week5-container-assignment/
├── .github/
│   └── workflows/
│       └── ci.yml
├── src/
│   └── pipeline.py
├── tests/
│   └── test_pipeline.py
├── Dockerfile
├── requirements.txt
└── AI_ASSIST.md

• Use logging instead of print for pipeline status (see Python Setup, Week 1).
• Read all configuration from environment variables.
• Do not commit secrets or .env files.

Evaluation Criteria

Your assignment will be evaluated on:

• Correctness: the pipeline runs in a container without manual fixes
• Reproducibility: clean install works with pinned dependencies
• CI quality: lint and tests run on every push
• Security: no secrets committed and env vars used correctly
• Clarity: README or comments explain how to run the container

Extra reading

• Docker development best practices: guidelines for building efficient images
• GitHub Actions quickstart: get a workflow running in minutes

Submission

1. Create a git branch week5/your-name.
2. Commit your work with a clear message.
3. Push the branch and open a Pull Request.

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 *https://hackyourfuture.net/*

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.