Week 7 - Mid-Track Project

Project Brief

Project Requirements

Week 7 Gotchas & Pitfalls

Technical Interview Preparation

The project ends with a 15-20 minute technical interview. This is not a quiz. It is a conversation about your project, your decisions, and what you learned. This chapter helps you prepare.

What to expect

The interview has four parts:

  1. Technical questions (5-7 min): The interviewer asks one question at each difficulty level about APIs, Pydantic, Docker, and Azure.
  2. Demo (5-7 min): You present your pipeline. Show the architecture, demonstrate the live deployment, and explain the data flow.
  3. Code discussion (5-7 min): The interviewer asks about specific code decisions. "Why did you use a Pydantic model here?" / "What happens if you remove this try/except?"
  4. Code comprehension (5 min): You read a short piece of unfamiliar pipeline code, explain what it does, find a bug, and suggest improvements. Copilot will be off.

You do not need slides. A terminal, the Azure portal, and your code editor are enough.

How to present your architecture

Start with the big picture before diving into details. A good walkthrough follows this order:

  1. What: "My pipeline fetches weather data from Open-Meteo and stores it in Postgres."
  2. Why: "I chose this API because it requires no authentication and returns clean JSON."
  3. How: Walk through the data flow step by step.
Open-Meteo API ──► pipeline.py ──► Pydantic validation ──► Postgres INSERT
                                                         ──► Blob Storage (raw JSON backup)

<aside> 💡 Draw this diagram in your README. You can use a simple text diagram (like above) or a tool like Excalidraw. Having a visual makes your explanation clearer and shows you understand the full flow.

</aside>

What to show live

Common questions and how to answer them

Prepare concise answers for these. You do not need to memorize scripts, but you should be able to answer each one clearly.

"Why did you choose this data source?"

Talk about practical reasons: availability, data format, personal interest. "I chose Open-Meteo because it has no rate limit, returns JSON, and I wanted to focus on the pipeline, not API authentication."

"How does your error handling work?"

Be specific. "If the API returns a non-200 status, I log the error and exit with code 1. If Pydantic validation fails, I log the invalid record and skip it."

"What do your tests cover?"

Describe what you tested and why. "I have tests that verify my Pydantic model accepts valid data and rejects records with out-of-range values. This catches data quality issues before they reach the database."

"What happens if the API is down?"

Honest answers are fine. "Right now, the pipeline fails and the job exits. If I had more time, I would add retry logic with backoff."

"Why Postgres instead of Blob Storage (or vice versa)?"

Explain the trade-off. "I used Postgres because I wanted to query the data with SQL later. Blob Storage would be better for raw archiving."

"What would you change if you had more time?"

This is not a trick question. Show that you can evaluate your own work. "I would add a cron schedule, improve error messages, and add a second data source."

"How do you know the pipeline actually worked?"

Describe your verification steps. "I check the execution history for a successful status, query the row count in Postgres, and list the blobs in storage."

Talking about failures

Interviewers value honesty about failures more than a perfect project. If something broke during the week, talk about it.

Good pattern:

  1. What happened: "My job kept failing with an image pull error."
  2. How you debugged it: "I checked the execution logs and saw it was trying to pull from Docker Hub."
  3. What the fix was: "I had forgotten --registry-server. Once I added it, the job pulled from ACR and ran."
  4. What you learned: "Container Apps does not parse the registry from the image name."

This shows debugging skills, which matter more than a flawless demo.

Practice exercise

<aside> ⌨️ Hands on: Before the interview, set a 3-minute timer and explain your project out loud to yourself (or a rubber duck). Cover: what data you ingest, how validation works, where the data is stored, and how the job runs on Azure. Record yourself on your phone if you want to review it. If you stumble or cannot explain something clearly, those are the areas to prepare.

</aside>

What interviewers are looking for

Signal What it means
Can explain the data flow from source to storage Understands the full pipeline
Knows why specific tools were chosen Made deliberate decisions, not only following a tutorial
Talks about failures and how they were resolved Can debug independently
Admits what they do not know Honest and self-aware
Shows working deployment with verifiable output Actually built and deployed the project

The interview is about understanding, not perfection. A simple project you can explain thoroughly scores higher than a complex one you cannot walk through.

<aside> 💡 Using AI to help: Practice by asking an LLM to interview you about your project. Paste your README and say: "Act as a technical interviewer. Ask me 5 questions about this data pipeline project." Answer out loud, not in the chat. The goal is to practice speaking, not typing.

</aside>

The more you practice explaining your work, the more natural it feels in the actual interview. Even five minutes of talking through your architecture out loud makes a difference.

<aside> 🤓 Curious Geek: Data engineering interviews

Technical interviews for data engineering roles often focus on system design and trade-offs rather than coding puzzles. Companies like Spotify, Netflix, and Airbnb publish blog posts about their data infrastructure. Reading one before your interview can help you connect your project to real-world practices.

</aside>

Knowledge Check

  1. What are the four parts of the Week 7 technical interview, and roughly how long is each?
  2. A classmate says "I built the project, so I can explain it. I don't need to practice." What would you say?
  3. The interviewer asks "what would you change if you had more time?" and your project has obvious gaps. How do you answer?
  4. You are asked to read unfamiliar code during the code comprehension exercise. Copilot is off and you cannot run the code. What is your approach?

Extra reading


Next up: Project Guidelines and Requirements, where you will find the minimum requirements checklist to review before submitting.


The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0 https://hackyourfuture.net/

CC BY-NC-SA 4.0 Icons

Built with ❤️ by the HackYourFuture community · Thank you, contributors

Found a mistake or have a suggestion? Let us know in the feedback form.