Schema evolution is an inevitable reality in data engineering, but it doesn't have to mean constant pipeline failures. In this post, we break down three pragmatic patterns for handling upstream changes: ignoring them, failing fast with strict Pydantic validation, or building adaptive "bridges." Learn how to choose the right strategy based on your business's risk tolerance.

The singleton creational design pattern is a great one to reach for if you need only one instance of an object.

stash-pass-cli is a command-line tool for securely managing your passwords locally. All data is encrypted using a master password and never leaves your machine.

Learn how the Context Object Pattern simplifies code by replacing long function signatures with a single structured context object.

The builder pattern is a creational design pattern that seeks to make creating objects flexible by separating the object from the construction of the object. Here, I describe the pattern with an example using Python.

The next time you find yourself in an if/elif/else chain when you are creating instances of something, consider whether the factory method can help your code be cleaner and more extensible for future needs!

Is the data set you provide end users is trustworthy? Focus on these data quality dimensions: completeness, consistency, validity, accuracy, uniqueness, and timeliness.