Week 2 - Structuring Data Pipelines

Introduction to Data Pipelines

Configuration & Secrets (.env)

Separation of Concerns (I/O vs Logic)

Dataclasses for Data Objects

OOP vs Functional Programming

Functional Composition

Testing with Pytest

Practice

Assignment: A Clean Pipeline

Gotchas & Pitfalls

• Teachers

Back to Track

7. Testing with Pytest

Goal: Confidence in your code.

Concepts to Cover

Why Pytest*: Simplicity over unittest.

*The Structure: test_*.py files and test_ functions.

Assertions*: Using simple assert x == y.

Testing Logic*: Writing tests for the specific transformation functions from the previous chapter.

Fixtures*: Setup/Teardown logic (creating sample data).

Parametrized Tests*: Running the same test with different inputs.

Suggested Exercises

• Write tests for the clean_price function.

• Use pytest.mark.parametrize to test edge cases (zeros, nulls, negative numbers).

*https://hackyourfuture.net/*

Found a mistake or have a suggestion? Let us know in the feedback form.