Module 10 — Project: a feature store for a scoring use case

Every module of this course has referred to the fraud scoring thread. This one runs the whole thing end to end, measures the training-serving skew before and after, times the online lookup, and then — the module's real question — asks honestly when the store paid for itself and when it did not.

Setting up the whole thing on a laptop

The stack is small enough to fit in a docker-compose.yml and a Feast repo. Nothing here needs cloud infrastructure.

services:
  redis:
    image: redis:7
    ports: ["6379:6379"]

fraud_project/
  docker-compose.yml
  fraud_repo/
    feature_store.yaml
    features.py
    data/transactions.parquet
  train.py
  serve.py
  measure_skew.py

feature_store.yaml and features.py are the files from module 7. data/transactions.parquet holds 500 000 anonymized transactions across 20 000 cards, with a is_fraud label on 3 % of them.

Bring the online store up, apply the definitions, then materialize the history:

docker compose up -d redis
cd fraud_repo
feast apply
feast materialize 2026-01-01T00:00:00 2026-05-14T23:59:59

Training with the store

train.py reads a labelled event dataframe, asks Feast for a point-in-time correct join, and fits a gradient-boosting model:

import pandas as pd, joblib
from feast import FeatureStore
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

fs = FeatureStore(repo_path="fraud_repo")

events = pd.read_parquet("fraud_repo/data/labels.parquet")

training_df = fs.get_historical_features(
    entity_df=events,
    features=[
        "card_activity_1h:n_tx_last_1h",
        "card_activity_1h:sum_amount_last_1h",
        "card_activity_7d:n_tx_last_7d",
        "card_activity_7d:avg_amount_last_7d",
    ],
).to_df()

X = training_df.drop(columns=["card_id", "event_timestamp", "is_fraud"])
y = training_df["is_fraud"]

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)
model = GradientBoostingClassifier().fit(X_tr, y_tr)
joblib.dump(model, "model.joblib")

The AUC on the held-out set lands at 0.82 — the honest number from module 5, not the leaked 0.97.

Serving with the store

serve.py is a small Flask endpoint. Each request receives a card_id, reads features from Redis via Feast, and returns a score.

from flask import Flask, request, jsonify
from feast import FeatureStore
import joblib

app = Flask(__name__)
fs = FeatureStore(repo_path="fraud_repo")
model = joblib.load("model.joblib")

FEATURES = [
    "card_activity_1h:n_tx_last_1h",
    "card_activity_1h:sum_amount_last_1h",
    "card_activity_7d:n_tx_last_7d",
    "card_activity_7d:avg_amount_last_7d",
]

@app.post("/score")
def score():
    card_id = request.json["card_id"]
    online = fs.get_online_features(
        features=FEATURES, entity_rows=[{"card_id": card_id}]
    ).to_dict()
    x = [[online[f.split(":")[1]][0] for f in FEATURES]]
    return jsonify({"score": float(model.predict_proba(x)[0, 1])})

Load-tested with a hundred concurrent requests, the p50 lookup lands under 4 ms and the p99 under 12 ms — comfortably inside a payment terminal's budget. Almost all of that is the Redis round-trip; the model inference is a few tens of microseconds on gradient boosting with four features.

Measuring the skew, before and after

measure_skew.py compares, on the same 500 events, the four features as computed by the training pipeline against the four features returned by the serving path. Before Feast — with the notebook aggregate versus the Redis counter from module 1 — the mean absolute relative difference across features was 6.3 %, and the model's recall dropped from 92 % to 74 %.

After Feast — with get_historical_features for training and get_online_features for serving, both reading the same definition — the mean absolute relative difference is 0.0 % on features unaffected by ongoing materialization lag, and under 0.5 % on features whose serving row is up to 15 minutes older than the training row. Recall at 1 % false-positive rate stabilizes at 88 %.

Two things are worth naming.

Eighty-eight is lower than ninety-two. The 92 % was measured on features computed by the notebook, which slightly leak the future through window inclusivity (module 1). The 88 % is measured on features consistent between training and serving, and it is what the model would sustain over months. Trading four points of nominal recall for a metric you can actually maintain is the right trade.

The remaining 0.5 % is the materialization lag. It is the price of batch instead of streaming for these views. If the fraud team needed to close that gap, module 6 tells them how: push to a streaming source. The cost of doing so should be weighed against the four hundredths of a point of recall it would buy.

What it would have taken to do without

The store is not free. feast apply, materialization jobs, a Redis to run, a registry to keep clean, teams to educate. Was it worth it?

Without a store, three things would have had to happen anyway.

A single implementation of each feature. Somewhere in the codebase would have to be a Python function that computes n_tx_last_1h, called by both training and serving. Writing that is the easy part; making sure both pipelines actually call it, at every version bump, forever, is the hard part.

A point-in-time correct join. Somewhere in the training pipeline would have to be a merge_asof with created_ts filtering and per-feature TTL. Any team member reimplementing training on a Sunday afternoon would have to know this and write it correctly.

An online serving layer. Somewhere would have to be a Redis (or DynamoDB, or Postgres) with the current values, refreshed on a schedule, monitored for freshness, with a schema that matches the training schema.

The feature store gives you the three, plus discovery and governance, in one system that survives team turnover. On a project with one model consumed by one team, that overhead is not worth it — a shared Python module and a merge_asof in the training script are cheaper. On a project with two or more models consuming overlapping features, or one model that will outlive its author by more than a year, the store pays back inside months.

When we would skip the store. A one-off model built by one person for a batch decision — no online serving, no evolution planned, no shared features. Building a Feast repo for it is overkill; a well-written notebook with merge_asof and a saved feature-computation module does the job.

When we would insist on the store. More than one model consuming any shared feature. Any online serving with sub-second latency. Any feature that will need to be shared across teams, whether it is today or not. The moment any of those is true, the store's operational cost becomes visibly smaller than the cost of the coordination it replaces.

Ship a small store first

If you are new to feature stores, start with one feature view and one model. Get feast apply, materialize and get_online_features working end to end, monitor freshness on the one view, and only then propose the store to a second team. A pilot with real numbers wins the argument for wider adoption; a slide deck describing benefits usually does not.

Summary

• The whole fraud scoring project fits on a laptop: Redis in a container, a Feast repo, a train.py and a serve.py of a few dozen lines each.
• Before the store: 6 % feature skew, recall drops from 92 % to 74 %. After the store: near-zero skew off the materialization lag, recall stable at 88 % — a lower number that is actually true.
• Online lookup p99 is inside a payment terminal's budget; the model inference itself is negligible next to the Redis round-trip.
• The store pays for itself on more than one model, shared features, or online serving; on a one-off batch model consumed by one team, a shared Python module and a merge_asof are cheaper.

Next module: the recap and the 40-question exam that closes the course.