Skip to main content

Module 5 — Caching data and resources

Module 1 warned that expensive work runs on every interaction. Now we fix it. Streamlit ships two decorators for that, and choosing between them is the most consequential caching decision you will make in a data app: @st.cache_data for results you want to reuse, and @st.cache_resource for objects you want to share. Getting this wrong is the difference between a snappy dashboard and one where every click reloads a 300 MB model.

The rule of thumb, then the reasons​

Use @st.cache_data for anything that returns data: a DataFrame from a SQL query, a NumPy array from a file, the prediction of a model, an API response as JSON.

Use @st.cache_resource for anything you want to instantiate once and share: a trained scikit-learn model, a database connection pool, a gRPC stub, an authenticated HTTP session.

The two decorators solve two different problems that just happen to look alike, and their internal behavior reflects that.

Why two decorators and not one​

@st.cache_data copies its return value on every cache hit. That copy is deliberate: it protects your app against a subtle bug where one user mutates a DataFrame retrieved from the cache and the next user sees the modification. The cost of the copy is real for large arrays but almost always worth paying, and it is the reason cache_data is safe by default.

@st.cache_resource shares its return value across every session and every user, without copying. That is exactly what you want for a model: loading it once and giving every visitor the same instance. But it also means that if two users call model.fit() on the shared object you are in trouble. Resources should be treated as read-only after they are cached.

Cache_data on the churn CSV​

The historical predictions live in a CSV that is a few dozen megabytes. Parsing it on every rerun is exactly what caching is for.

import streamlit as st
import pandas as pd

@st.cache_data(ttl=15 * 60, show_spinner="Loading historical predictions...")
def load_predictions(path: str) -> pd.DataFrame:
    df = pd.read_csv(path, parse_dates=["scored_at"])
    df["month"] = df["scored_at"].dt.to_period("M").astype(str)
    return df

df = load_predictions("data/predictions_history.csv")
st.metric("Rows loaded", f"{len(df):,}")

The cache key is derived from the function name and the arguments, so calling load_predictions("data/predictions_history.csv") twice in a row costs microseconds on the second call. ttl=15 * 60 says "consider this stale after fifteen minutes"; without a TTL, the cache lives for the lifetime of the process. show_spinner= replaces the default "Running load_predictions..." with a message meant for the end user.

Cache_resource on the model​

Loading a joblib model with a hundred thousand parameters from disk takes seconds. Doing it once per process is a change of magnitude in perceived responsiveness.

import joblib

@st.cache_resource(show_spinner="Loading the churn model...")
def get_model():
    return joblib.load("models/churn_v3.joblib")

model = get_model()
probability = model.predict_proba([features])[0, 1]
st.metric("Churn probability", f"{probability:.0%}")

The first rerun of the app pays the load; every subsequent rerun, regardless of which user triggered it, gets the exact same in-memory object. If your model exposes a thread-unsafe internal state, this is where you would need a lock. For scikit-learn, PyTorch (eval() mode) and TensorFlow (frozen), the shared instance is safe for inference.

Invalidation: the two levers​

Both decorators support explicit invalidation, and both give the user a "Clear cache" button in the app menu. In code, three tools cover the common cases.

load_predictions.clear()                    # clear this entry
st.cache_data.clear()                       # clear every cache_data entry
st.cache_resource.clear()                   # clear every cache_resource entry

For time-based freshness, ttl= accepts seconds or a timedelta. For size-based eviction, max_entries= bounds the number of cached (argument-tuple, value) pairs — useful when the same function is called with many parameter combinations, otherwise the cache grows without a bound.

The trap: mutable objects and unhashable arguments​

Streamlit hashes the arguments of the wrapped function to build the cache key. For most built-in types this is transparent, but two patterns break it.

@st.cache_data
def score(features: dict) -> float:                     # dict is fine
    return model.predict_proba([list(features.values())])[0, 1]

@st.cache_data
def score_batch(df: pd.DataFrame) -> pd.DataFrame:      # DataFrame is fine
    return df.assign(probability=model.predict_proba(df)[:, 1])

Both work: dicts and DataFrames have a well-defined hash in Streamlit's implementation. The failure modes look like:

@st.cache_data
def query(cursor, sql):                                  # cursor is unhashable
    return cursor.execute(sql).fetchall()

A live database cursor cannot be hashed, and neither can most SDK clients. The fix is either to move the client behind @st.cache_resource and take only the arguments that change through cache_data, or to tell Streamlit to skip hashing for that parameter with a leading underscore in its name:

@st.cache_resource
def get_cursor():
    return sqlite3.connect("db.sqlite").cursor()

@st.cache_data(ttl=60)
def query(_cursor, sql):                                 # underscore = skip hash
    return _cursor.execute(sql).fetchall()

rows = query(get_cursor(), "SELECT * FROM predictions LIMIT 100")

The underscore convention is worth remembering; it is short, explicit and the Streamlit team has committed to keeping it.

Choosing wrong: two failure stories​

A @st.cache_data on a model. The decorator will copy the model on every hit. A hundred-megabyte model, three users, three copies in memory. The pod OOMs in production and the dashboard 502s during the weekly business review. @st.cache_resource fixes it in one edit.

A @st.cache_resource on a DataFrame. The DataFrame is shared, not copied. A helpful analyst runs df.rename(columns=..., inplace=True) inside the app to make a chart. Every other user sees the renamed columns, sometimes half-renamed if the mutation runs mid-request. @st.cache_data fixes it in one edit; treating cached resources as read-only avoids the class entirely.

Adding both caches to the running example​

import streamlit as st
import pandas as pd
import joblib

@st.cache_resource(show_spinner="Loading the churn model...")
def get_model():
    return joblib.load("models/churn_v3.joblib")

@st.cache_data(ttl=15 * 60, show_spinner="Loading history...")
def load_history() -> pd.DataFrame:
    return pd.read_csv("data/predictions_history.csv", parse_dates=["scored_at"])

@st.cache_data(ttl=15 * 60)
def customers_at_risk(threshold: float) -> pd.DataFrame:
    df = load_history()
    latest = df.sort_values("scored_at").drop_duplicates("customer_id", keep="last")
    return latest[latest["probability"] >= threshold]

model = get_model()
history = load_history()
risky = customers_at_risk(threshold=st.session_state.get("threshold", 0.60))

st.metric("Customers at risk", f"{len(risky):,}")

Notice how customers_at_risk calls another cached function (load_history). Streamlit handles that naturally: each cache lookup is independent, and changing the threshold only invalidates the outer result. That composition is the daily rhythm of a Streamlit app: small, cached, single-purpose functions.

Cache in the language of your users

show_spinner="Loading historical predictions..." is not decoration — it is the difference between a page that feels stuck and a page that says why it is thinking. On any cache that can miss for more than 200 ms, write the spinner message in the user's language.

In summary​

  • cache_data for values you want to reuse (it copies on read); cache_resource for objects you want to share (it does not copy).
  • Invalidation: ttl= for freshness, max_entries= for memory, func.clear() in code, and the "Clear cache" button in the app menu.
  • Traps: never mutate a cached resource; prefix an unhashable argument with _ to skip hashing; move connection objects behind cache_resource.
  • The running example now loads the model once and the history every fifteen minutes; the next module handles state that must persist across the reruns caching alone cannot.

Next module: session state and forms, the two mechanisms that turn a stateless script into a real interactive application.