Module 6 — Prefilled examples and caching

A demo that opens on an empty page asks the visitor to guess what to try. Half of them will click submit on the placeholder input and see a confusing error; the other half will close the tab. Examples cut that friction to zero: a row of clickable buttons at the bottom of the interface, each preloaded with a real input, ready to send. Caching pushes it further by computing every example's output at startup, so clicking one shows the answer in a hundred milliseconds instead of ten seconds. This module is short, but the effect on a live demo is out of proportion with its size.

Clickable examples on Interface

gr.Interface accepts an examples argument that is a list of lists. Each inner list is one row of inputs, matching the order of inputs. A single-input demo takes a list of one-element lists.

import gradio as gr
from transformers import pipeline

classifier = pipeline("image-classification", model="google/vit-base-patch16-224")

def classify(image):
    predictions = classifier(image)
    return {p["label"]: float(p["score"]) for p in predictions}

demo = gr.Interface(
    fn=classify,
    inputs=gr.Image(type="pil"),
    outputs=gr.Label(num_top_classes=5),
    examples=[
        ["examples/labrador.jpg"],
        ["examples/skyscraper.jpg"],
        ["examples/soup-bowl.jpg"],
        ["examples/blurry-cat.jpg"],
    ],
    examples_per_page=8,
    title="Image classifier",
)
demo.launch()

The paths are relative to the working directory. Gradio uploads each file as a static asset and Docker or a Space picks them up as part of the demo. On a chat interface, examples are plain strings and appear as suggestion chips below the input.

gr.ChatInterface(
    fn=chat_stream,
    examples=[
        "Explain gradient descent in three sentences.",
        "Refactor this Python function for clarity: def f(x): return x*2+1",
        "What is a common misuse of a confusion matrix?",
    ],
    title="Chat with examples",
).launch()

The examples that earn their place

A demo has room for six examples, maybe eight. Filling those slots with easy wins is a wasted opportunity. The best examples show the span of the model: one obvious success, one edge case, one graceful failure. That last category is the one most authors skip, and it is the one that builds trust.

The image classifier from the snippet above pairs a Labrador (obvious success) with a soup bowl (correctly classified but with lower confidence) and a blurry cat (misclassified, or classified with low confidence). A visitor who tries all three learns three things: the model works, it hesitates when the picture is odd, and it fails when the picture is degraded. They leave with a calibrated intuition, not with the impression that the model is magic.

For a language model chat, the same rule applies. Provide one straightforward question, one that pushes the model out of its comfort zone (a niche technical topic, a legal question, an ambiguous request), and one that should not be answered at all (a request that would breach policy, a math problem beyond arithmetic). Reviewers of a chat demo want to see whether the model refuses gracefully and admits uncertainty, not only whether it can rephrase a paragraph.

A demo tuned only for its examples is a trap

Some authors optimize their model on the exact set of examples they show. The demo runs beautifully on the four chips and falls apart on the fifth input a user types. Rotate the examples every few weeks and audit them against the general test set — a demo that has drifted into "example overfit" is worse than one with no examples at all, because it lies more convincingly.

Caching examples: the demo that opens ready

cache_examples runs each example through the function at launch time and stores the output. When a visitor clicks the example later, Gradio serves the cached answer immediately, without touching the model. On a hosted Space with a cold GPU, that shift can turn a five-second wait into a hundred-millisecond animation — and the visitor stays on the page.

gr.Interface(
    fn=classify,
    inputs=gr.Image(type="pil"),
    outputs=gr.Label(num_top_classes=5),
    examples=[["examples/labrador.jpg"], ["examples/skyscraper.jpg"], ["examples/blurry-cat.jpg"]],
    cache_examples=True,      # compute at startup, serve at click
).launch()

Enabling caching moves work: the launch takes longer, the interaction gets faster. On a Space with 5 GB of RAM and a slow cold start, the trade is almost always worth making. On a local development server that you kill and relaunch dozens of times a day, cache_examples=False (the default) is friendlier. Modern Gradio versions accept cache_examples="lazy", which caches an example the first time any user runs it — the middle ground that fits most demos.

Cache invalidation, the little detail that bites

Cached outputs live in the gradio_cached_examples/ directory that Gradio creates next to your app. That directory is keyed by the function's name and by the example's inputs, but it does not track the model version. Change the model, retrain it, swap a checkpoint, and Gradio will keep serving the previous outputs until you clear the directory.

rm -rf gradio_cached_examples

Add that command to the deployment script and to any Makefile rule that swaps a model. A demo that boasts a new version but shows the old predictions on the marketing screenshots damages trust faster than a demo with no examples at all. Two lines of documentation next to your train.py — "if you updated the model, delete the cache" — save a lot of embarrassment.

Streaming and caching do not coexist

A generator cannot be cached, because the shape of its output is a stream, not a value. If your callback yields tokens for the streaming module, cache_examples=True will silently do nothing for that function; Gradio just runs the generator on click as usual. That is fine, and worth remembering: for streaming demos, focus on warmup and time-to-first-token rather than on cached outputs. The queue mechanism from the next module is what buys you the equivalent of caching for streamed responses.

In summary

• examples=[[...], [...], ...] gives visitors a way in; the same list is a string list for gr.ChatInterface.
• Pick examples that cover a success, an edge case, and a graceful failure; that mix builds trust more than three easy wins.
• cache_examples=True computes at launch and serves cached outputs at click — the trade of a slower startup for a much snappier click.
• Cached outputs are keyed by inputs, not by model version: delete gradio_cached_examples/ whenever you swap a model.

Next module: the queue and concurrency knobs, so the demo does not fall over when three users hit it at once.