Exécutions reproductibles, schémas versionnés, contrats de données entre étapes.
Constitution — Python data pipeline
Version 1.1.0 · Ratified 2026-10-12 · Last amended 2026-11-10
-
Reproducible runs. A run is defined by its code version, its configuration and its input snapshot. Running it again with the same three gives the same output.
-
Versioned schemas. Every dataset has a schema with a version. A breaking change bumps the major version and keeps the old one readable until every consumer has moved.
-
Data contracts between stages. Each stage declares what it reads and writes, with types, nullability and allowed ranges. A stage refuses input that breaks the contract instead of guessing.
-
Idempotent stages. Running a stage twice for the same partition leaves the same result; writes replace a partition, they never append duplicates.
-
Lineage. Every output row can be traced to the run and the inputs that produced it.
-
Data quality checks gate publishing. Row counts, null rates and value ranges are checked after each stage; a failed check stops the run before anything downstream reads the output, and the failure names the check.