Module 5 — Error handling and status codes

At the end of module 4, the service returns a probability whenever the caller sends valid input, and a Pydantic 422 whenever it does not. But real ML services have more failure modes: an unknown subscriber that leaks in from a stale CSV, a model call that raises because a category is unseen, a downstream feature store that is briefly down. This module maps every one of those cases to the right HTTP status code, defines a single error format the whole API uses, and closes the door on the single most common leak: an unhandled exception that returns a Python traceback to the caller.

The three status codes an ML API actually uses

Nine tenths of ML API errors fit into three buckets. Getting them right is the difference between an API a mobile team can integrate against and one whose reported errors are useless:

• 400 Bad Request — the request is syntactically valid but violates a business rule the caller can fix by itself. Example: the caller asked for a prediction on a subscriber whose contract ended two years ago. The payload parses, the model would compute a number, but that number is not a probability the business wants to serve.
• 422 Unprocessable Entity — the payload failed schema validation: a required field is missing, an enum value is not in the allowed list, an integer is out of range. This is what Pydantic returns automatically. Never re-raise a 422 from your own code; if you can express the check as a Pydantic constraint, do that instead.
• 500 Internal Server Error — something the caller cannot fix: the model call raised, the feature store timed out, disk is full. A 500 must always be paired with a server-side log that includes the request ID, the failing exception, and enough context to reproduce.

Two status codes exist for narrower ML-specific cases and are worth knowing:

• 404 Not Found for a subscriber ID that does not exist, when the service takes an ID and looks up features on the server side.
• 503 Service Unavailable for a temporary condition — the model is still loading, the feature store is briefly down, the service is being drained for shutdown. A 503 with a Retry-After header is what a good client waits on; a 500 makes the client think the request itself is bad.

400, 422 and 500 cover 90 %; 404 and 503 handle the specific ML cases.

HTTPException: the everyday tool

Raising an HTTPException from a route immediately returns a JSON error response with the given status code and detail. It is the mechanism for every 400 and every 503 you write by hand:

# app/main.py (continued)
from fastapi import FastAPI, HTTPException, status
from datetime import date

from app.schemas import Subscriber, Prediction
from app.scoring import _score_one


app = FastAPI(title="Churn scoring API", version="0.5.0")


@app.post("/predict", response_model=Prediction, tags=["prediction"])
def predict(subscriber: Subscriber) -> Prediction:
    if subscriber.tenure_months == 0 and subscriber.total_charges > 0:
        # A business inconsistency the caller can fix on its side.
        raise HTTPException(
            status_code=status.HTTP_400_BAD_REQUEST,
            detail={
                "code": "inconsistent_subscriber",
                "message": "tenure_months is 0 but total_charges is positive",
                "field": "total_charges",
            },
        )
    return _score_one(subscriber)

Note the shape of detail. FastAPI accepts a string, but a dictionary with a stable code, a human message and, when useful, a field turns errors into something the caller can switch on. Never localize message on the server — leave that to the client, and use code as the translation key.

A global 500 handler that never leaks a traceback

The scariest default in a naive FastAPI app is what happens when an unhandled exception escapes a route: FastAPI logs it and returns {"detail": "Internal Server Error"} — but only in production. In development, or in some middleware configurations, it can return the full traceback, which reveals the source layout, dependency versions, and sometimes secrets. Close the door explicitly:

# app/errors.py
import logging
import uuid
from fastapi import Request
from fastapi.responses import JSONResponse

logger = logging.getLogger("churn")


async def unhandled_exception_handler(request: Request, exc: Exception) -> JSONResponse:
    request_id = getattr(request.state, "request_id", str(uuid.uuid4()))
    # Log with the exception on the server, never on the wire.
    logger.exception(
        "unhandled_exception path=%s method=%s request_id=%s",
        request.url.path, request.method, request_id,
    )
    return JSONResponse(
        status_code=500,
        content={
            "code": "internal_error",
            "message": "The service could not process this request.",
            "request_id": request_id,
        },
    )

Register it once on the app:

# app/main.py (continued)
from app.errors import unhandled_exception_handler

app.add_exception_handler(Exception, unhandled_exception_handler)

Three properties matter. The request ID is echoed in both the log line and the response, so a support ticket that includes it can be joined to the log line in seconds (module 8 automates the ID generation). The message is boring on purpose: it says nothing about the failing exception, its stack, or the file it came from. And the logger.exception call in the handler is the only place the traceback is written — always to the server-side log, never on the wire.

Domain exceptions to HTTP: a small mapper

The scoring layer of module 4 should not know that it lives behind HTTP. It raises domain-typed exceptions, and a mapper turns them into responses:

# app/errors.py (continued)
class ModelNotReady(Exception): ...
class UnknownSubscriber(Exception):
    def __init__(self, subscriber_id: str) -> None:
        self.subscriber_id = subscriber_id
class FeatureStoreTimeout(Exception): ...


async def model_not_ready_handler(request: Request, exc: ModelNotReady) -> JSONResponse:
    return JSONResponse(
        status_code=503,
        headers={"Retry-After": "5"},
        content={"code": "model_not_ready", "message": "Model is still loading."},
    )


async def unknown_subscriber_handler(request: Request, exc: UnknownSubscriber) -> JSONResponse:
    return JSONResponse(
        status_code=404,
        content={
            "code": "subscriber_not_found",
            "message": "No subscriber with this identifier.",
            "subscriber_id": exc.subscriber_id,
        },
    )


async def feature_store_timeout_handler(request: Request, exc: FeatureStoreTimeout) -> JSONResponse:
    return JSONResponse(
        status_code=503,
        headers={"Retry-After": "1"},
        content={"code": "feature_store_timeout", "message": "Feature store did not respond in time."},
    )

Wire them all with one call each on the app:

from app.errors import (
    ModelNotReady, UnknownSubscriber, FeatureStoreTimeout,
    model_not_ready_handler, unknown_subscriber_handler, feature_store_timeout_handler,
)

app.add_exception_handler(ModelNotReady, model_not_ready_handler)
app.add_exception_handler(UnknownSubscriber, unknown_subscriber_handler)
app.add_exception_handler(FeatureStoreTimeout, feature_store_timeout_handler)

The routes and the scoring layer are now cleanly decoupled from HTTP. A unit test raises UnknownSubscriber from a scoring function and asserts against the exception, not against a JSON response. A future gRPC endpoint reuses the same domain exceptions with a different mapper. That is the payoff of not scattering HTTPException deep inside business code.

Rewriting validation errors for a friendlier client experience

The default 422 body from Pydantic is precise but verbose. Some teams prefer a flatter, stable format. FastAPI lets you override the handler for RequestValidationError:

from fastapi.exceptions import RequestValidationError


async def validation_error_handler(request: Request, exc: RequestValidationError) -> JSONResponse:
    return JSONResponse(
        status_code=422,
        content={
            "code": "validation_error",
            "message": "One or more fields are invalid.",
            "errors": [
                {"field": ".".join(str(x) for x in e["loc"][1:]), "type": e["type"], "input": e.get("input")}
                for e in exc.errors()
            ],
        },
    )


app.add_exception_handler(RequestValidationError, validation_error_handler)

Two habits pay off here. The loc[1:] slice drops the leading "body", "query" or "path" so the client sees a plain field path like subscribers.2.monthly_charges. And the type (greater_than_equal, int_parsing, missing, ...) is preserved so a client can map it to a localized message.

The single rule to never break

Never expose a Python traceback to a caller. A traceback reveals your source paths, your library versions, sometimes the values of local variables, and always the internal structure of your code. Every entry point — request handlers, exception handlers, background tasks — must go through the 500 handler above or return an equally sanitized error. In practice, a good smoke test in CI provokes an unhandled exception (raise Exception("test-boom") under a feature flag) and asserts that the response body contains neither Traceback, nor File ", nor any path from the source tree.

A 500 that stays quiet is worse than a 500 that alerts

The temptation on a busy service is to filter 500s out of the alerting rules because they are noisy. Do not. A 500 always means "the service could not answer a request it should have answered". If they are noisy, either the exception handler is misclassifying (a 400 dressed up as a 500), or something in the code path is genuinely broken and needs attention.

In summary

• 400 is a caller-fixable business rule violation; 422 is a schema violation Pydantic already handles; 500 is server-side, always logged with the exception. Add 404 for unknown IDs and 503 with Retry-After for temporary conditions.
• Raise HTTPException with a {"code", "message", ...} detail so clients can switch on a stable tag and localize the message on their side.
• Register a global exception handler that returns a sanitized JSON body with a request_id and logs the traceback server-side. Never leak a traceback to the caller.
• Keep the scoring layer HTTP-free by raising domain exceptions and mapping them to responses at the boundary.

Next module: async requests and background tasks — turning the synchronous batch route of this module into an accepted-then-processed job that scales to files with tens of thousands of subscribers.