Module 4 — Versioning data and models

Module 3 asked you to log a data_version parameter with every run. This module makes that parameter mean something: a string that lets anyone, on any machine, retrieve the exact bytes that trained the model. Without that guarantee, tracking is a diary; with it, tracking is an audit trail.

Why git alone is not enough

Git tracks text files well and binary files badly. A 2 GB churn dataset committed once bloats the repository and slows every clone; a monthly refresh multiplies the pain. Beyond half a gigabyte, most teams reach for Git LFS or, better for ML, DVC (Data Version Control).

DVC's idea is simple. The data file stays out of git; only a small .dvc pointer file goes in. The pointer holds the file's SHA-256 hash, which becomes its address in a remote object store. git checkout moves you to a commit, then dvc checkout uses the pointers in that commit to fetch the matching data.

DVC on the churn project

Initialize once at the root of the repository:

dvc init
dvc remote add -d storage s3://inskillops-mlops/dvc

Then version the training CSV:

dvc add data/subscribers.csv
git add data/subscribers.csv.dvc .gitignore
git commit -m "data: subscribers snapshot 2026-08-31"
dvc push

After dvc add, subscribers.csv is in .gitignore and its .dvc pointer is committed. dvc push uploads the actual bytes to S3, addressed by their hash. A colleague who runs git pull && dvc pull on the same commit gets the same 5 GB in data/subscribers.csv.

The link commit → data → model

The three artifacts must be tied by a single ID. The commit hash is the natural anchor, and the tracking parameter is what makes the tie visible:

import subprocess

commit = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode().strip()
data_hash = open("data/subscribers.csv.dvc").read().split("md5: ")[1].split("\n")[0]

mlflow.log_params({
    "git_commit": commit,
    "data_hash": data_hash,
})

Now a run in MLflow points, without ambiguity, at one commit, one dataset content and one produced model. Six months later, "why did this prediction happen?" resolves to git checkout <commit> && dvc pull && python -m src.train.

Model files also deserve versioning

The model.pkl MLflow writes as an artifact is already versioned by the run ID. That is enough for most cases. When several models feed each other — a champion model plus a lightweight distilled variant, a base model plus a fine-tuned one — DVC-versioning the intermediates gives a cleaner lineage than nesting artifacts inside a single run. The same dvc add command works on .pkl, .onnx or .safetensors.

Data lineage: reading a prediction backwards

A production system rarely predicts on raw data. It joins a customer's current tariff, their last three months of usage, and a demographic table. Lineage is the ability to walk that graph backwards from a prediction to its inputs.

DVC pipelines make part of that lineage explicit. A dvc.yaml describes stages:

stages:
  clean:
    cmd: python -m src.clean
    deps:
      - data/subscribers.csv
      - src/clean.py
    outs:
      - data/clean.parquet
  features:
    cmd: python -m src.features
    deps:
      - data/clean.parquet
      - src/features.py
    outs:
      - data/features.parquet
  train:
    cmd: python -m src.train
    deps:
      - data/features.parquet
      - src/train.py
    outs:
      - artifacts/model.pkl
    metrics:
      - reports/metrics.json

dvc repro reruns only the stages whose dependencies changed. The graph is the lineage; you can now answer "if I change clean.py, does the model change?" (yes, because clean.parquet is upstream of train) without reading every file.

When a versioned database is enough

DVC is worth its weight for files — CSVs, Parquets, model binaries. If your training data is a query against a data warehouse (BigQuery, Snowflake, Databricks) that already keeps immutable snapshots (time-travel queries, AS OF SYSTEM TIME), you probably do not need DVC for that data. Version the query and the snapshot timestamp in the tracking parameters:

mlflow.log_params({
    "warehouse_query_hash": sha256(query.encode()).hexdigest()[:12],
    "snapshot_ts": "2026-08-31T00:00:00Z",
})

Anyone can then replay the exact query against the exact snapshot. The rule of thumb: version the files, or version the query — never neither, never both.

Lineage is a governance requirement, not a comfort

For a bank subject to GDPR or a hospital subject to HIPAA, being unable to answer "which rows fed this prediction?" is a compliance failure, not an inconvenience. Lineage is a design constraint from day one, retrofitted with pain later.

Summary

• Git tracks code; DVC tracks the large files by keeping a hash pointer in git and the bytes in a remote store.
• Log the commit hash and data hash as run parameters — that is the link that turns a run ID into a reproducible experiment six months later.
• DVC pipelines make the transformation graph explicit; lineage is walking that graph backwards from a prediction.
• Prefer versioning files with DVC or queries with warehouse snapshots — never both, never neither.

Next module: the model registry — a curated place where the winners of module 3's experiments earn a name, a stage, and the right to be served.