Skip to main content

Module 4 — Chat interfaces

The audio pipeline is behind us; the third demo of the course starts here and runs all the way to the final project. A chat assistant differs from a one-shot classifier in three ways: turns accumulate into a history, a system prompt frames the assistant's behavior, and answers are usually streamed token by token rather than returned all at once. Gradio ships a specialized wrapper — gr.ChatInterface — that handles the first two out of the box, and the next module will layer streaming on top. This module builds the chat surface itself.

The signature that ChatInterface expects​

gr.ChatInterface is the fastest path to a chat page. It takes a function that receives the user's new message and the history so far, and returns the assistant's answer. Gradio owns the widgets — a chat display, an input box, a submit button, a retry and a clear button — so you write only the model call.

import gradio as gr

def chat(message: str, history: list[dict]) -> str:
    # `history` is a list of dicts like {"role": "user"|"assistant", "content": "..."}
    # It already contains every past turn of this session, in order.
    return f"You said: {message} (I remember {len(history) // 2} exchanges)"

demo = gr.ChatInterface(fn=chat, title="A minimal chat demo")
demo.launch()

The history argument is where sessions become real. It is scoped to the browser tab, so two users see two independent conversations, and it grows every time the user submits a message. The type is list[dict] in the modern messages format — each dict has a role ("user", "assistant" or "system") and a content string. This is the same shape the OpenAI-style APIs use, so most of the time the chat function forwards history + [{"role": "user", "content": message}] to a completion endpoint and returns the assistant's reply.

A system prompt that actually gets used​

A system prompt tells the model what role to play and what constraints to respect: tone, expertise, refusal policy, language. In Gradio, wire it as an additional input so the person running the demo can tune it without editing code.

import gradio as gr

def chat(message: str, history: list[dict], system_prompt: str) -> str:
    messages = [{"role": "system", "content": system_prompt}]
    messages.extend(history)
    messages.append({"role": "user", "content": message})
    reply = call_model(messages)   # placeholder for a real backend
    return reply

demo = gr.ChatInterface(
    fn=chat,
    additional_inputs=[
        gr.Textbox(
            value="You are a concise technical assistant. Answer in at most three sentences.",
            label="System prompt",
            lines=3,
        ),
    ],
    title="Chat with a system prompt",
    description="Edit the prompt above to change the assistant's behavior.",
)
demo.launch()

The extra widget appears in a collapsible panel below the chat, which is exactly right: it belongs to configuration, not to the conversation. additional_inputs accepts a list, so you can add a temperature slider, a checkbox to enable a tool, or a dropdown to switch between models — every extra widget becomes a positional argument to your function, in the order you declared them.

Connecting to a real model​

Two backends cover the vast majority of demos: a local model served by Ollama, and a hosted API like OpenAI, Anthropic or Mistral. The Gradio side is identical in both cases; only the call_model helper changes.

A local model with Ollama​

Course 29 introduced Ollama, which serves quantized open-weight models on a laptop. It speaks the OpenAI chat completions protocol, so the client library is the same as for the hosted APIs.

import gradio as gr
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

def chat(message: str, history: list[dict], system_prompt: str) -> str:
    messages = [{"role": "system", "content": system_prompt}]
    messages.extend(history)
    messages.append({"role": "user", "content": message})
    response = client.chat.completions.create(
        model="llama3.1:8b",
        messages=messages,
        temperature=0.4,
    )
    return response.choices[0].message.content

gr.ChatInterface(
    fn=chat,
    additional_inputs=[gr.Textbox(value="You are a concise assistant.", label="System prompt")],
    title="Chat with a local Llama",
).launch()

A hosted API​

Swap the base URL and the API key, and the same code hits OpenAI's servers instead. The difference is measured only in latency and cost, not in the Gradio wiring.

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def chat(message, history, system_prompt):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "system", "content": system_prompt}, *history, {"role": "user", "content": message}],
        temperature=0.4,
    )
    return response.choices[0].message.content

Never hard-code an API key in a file you plan to publish. Read it from an environment variable, and store it as a Space secret when you host the demo — module 9 walks through that.

Right-to-left rendering for Arabic and Hebrew​

If you plan a multilingual demo, remember that Arabic, Hebrew and Persian text renders correctly only when the container carries dir="rtl". Gradio inherits the browser's default direction, so a mixed-language site should nudge the chat display for the right locales.

with gr.Blocks(css=".chatbot .message[data-lang=ar] { direction: rtl; text-align: right; }") as demo:
    gr.ChatInterface(fn=chat)

The exact selector depends on the Gradio version; keep the css block small and inspect the rendered DOM to confirm. The important point is that a chatbot that mixes English and Arabic without RTL support ships punctuation on the wrong side of the line — a small detail that immediately signals a demo that has never seen a real user.

In summary​

  • gr.ChatInterface(fn=chat) gives you a full chat page from a function that takes (message, history, ...) and returns a string.
  • The history argument is scoped per session and follows the messages format [{"role": ..., "content": ...}], the same shape as OpenAI-style APIs.
  • Put the system prompt in additional_inputs: it belongs to configuration, and users can tune it without editing code.
  • The Gradio wiring is identical for a local Ollama or a hosted API; only the client differs. Read keys from environment variables, never from source.

Next module: making the assistant feel fast by streaming tokens instead of waiting for the full response.