Module 6 — Containerizing a model

The registry now names one anointed model version. This module builds the smallest, safest container that serves it — the exact bytes CI will publish in module 7. The goal is a container that starts in seconds, weighs a few hundred megabytes, and refuses at build time anything that would poison production.

What the container must contain, and what it must not

Must contain: the Python runtime, the runtime dependencies, the inference code, and either the model file or the credentials to fetch it from the registry at startup. Must not contain: the training code, notebooks, training data, developer tools, or any secret hard-coded in the image.

The reason this separation matters is not aesthetic. Every megabyte of the image is a byte to scan, to store, to pull to every node. Every developer tool baked in is a potential vulnerability in an application that need not build software at runtime.

A minimal FastAPI service for the churn model

FastAPI keeps the service short. The essential file is a handler that loads the model once at startup and responds to prediction requests.

# src/serve.py
import os
import mlflow.pyfunc
from fastapi import FastAPI
from pydantic import BaseModel

MODEL_URI = os.environ["MODEL_URI"]  # e.g. "models:/churn-classifier@production"

app = FastAPI()
model = mlflow.pyfunc.load_model(MODEL_URI)  # loaded once, at process start


class Subscriber(BaseModel):
    tenure_months: int
    monthly_charges: float
    contract_type: str
    payment_method: str


@app.post("/predict")
def predict(sub: Subscriber):
    proba = model.predict([sub.model_dump()])[0]
    return {"churn_probability": float(proba)}


@app.get("/healthz")
def health():
    return {"status": "ok"}

Two design choices deserve names. Loading at startup (rather than per-request) is what makes the p50 latency predictable — a single-request cold load would spike the first response to seconds. Reading the model URI from an environment variable is what lets the same image serve any registry alias: @production in prod, @shadow in staging, a pinned version in a regression job.

The Dockerfile, one layer at a time

# syntax=docker/dockerfile:1
FROM python:3.11-slim AS base

# Fixed non-root user for defense in depth
RUN useradd --create-home --uid 10001 app
WORKDIR /home/app

# Dependencies first, so they cache
COPY --chown=app:app requirements.txt .
RUN pip install --no-cache-dir --require-hashes -r requirements.txt

# Then the code
COPY --chown=app:app src/ src/

USER app
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c "import urllib.request, sys; \
    sys.exit(0 if urllib.request.urlopen('http://localhost:8080/healthz').status == 200 else 1)"

CMD ["uvicorn", "src.serve:app", "--host", "0.0.0.0", "--port", "8080"]

Several patterns compound. python:3.11-slim trades a few hundred megabytes of build tools for a smaller attack surface. The lock file installed with --require-hashes inherits module 2's supply-chain protection. Copying dependencies before code keeps the dependency layer cached across code changes, dropping build times from minutes to seconds. A non-root user neutralizes a whole class of container-escape techniques. HEALTHCHECK lets the orchestrator restart a service that has become unresponsive — a broken model load, a leaked file descriptor, a hung MLflow client.

Loading the model: baked vs. pulled

Two patterns exist for getting the model bytes into the container.

Bake the model into the image at build time: COPY artifacts/model.pkl model.pkl and set MODEL_URI=file:./model.pkl. Immutable, self-contained, easy to rollback: deploying v7 means running the v7 image. Downside: every model version rebuilds the image.

Pull the model at startup from the registry: MODEL_URI=models:/churn-classifier@production. The image is the same across model versions; alias moves are picked up by restarting pods. Downside: the container's startup is bound to the registry's availability, and rollback is coupled to alias mechanics.

Neither is universally right. For a single model, low change frequency, and strict rollback SLAs, bake. For many models sharing an image or frequent alias moves, pull. Whichever you pick, log the resolved version in the startup log — otherwise you cannot tell v7 from v8 in an outage.

Image size and security

An untuned ML container easily reaches 3–5 GB — a scikit-learn wheel, a full CUDA runtime, some NLP libraries — and takes a minute to pull on every scale-out. Three practical measures:

• Multi-stage builds: install into a build stage, copy only the site-packages into a runtime stage. Cuts 200–500 MB.
• .dockerignore to exclude .git, data/, tests/, notebooks — a common source of accidental data leaks inside images.
• A vulnerability scanner in CI (Trivy, Grype). Any HIGH or CRITICAL CVE fails the build; scanning after deployment is closing the barn door.

For CPU-only inference on a scikit-learn model, aim for 400–600 MB. For a small PyTorch model, 1.5–2 GB is realistic. Above that, something is bundled you did not intend to bundle.

Do not put secrets in the image

An MLflow URL is fine, credentials are not. The container reads the registry via a service account whose token is injected at runtime by the orchestrator (Kubernetes Secret, ECS task role, systemd credential). A secret baked into an image is one docker save away from a leak.

Summary

• The container carries runtime code + dependencies + model, and nothing else — no training code, no data, no secrets, no dev tools.
• Load the model once at startup, read its URI from an environment variable, and expose a healthcheck the orchestrator can use.
• Choose between baking the model (immutable, easy rollback) and pulling from the registry (one image for many versions); log the resolved version.
• Keep the image small (multi-stage builds, .dockerignore) and scan it for vulnerabilities before it is pushed.

Next module: CI/CD — wiring these containers, the tests, the registry moves and the deployments into a pipeline that no human clicks through.