Module 3 — Interactive session and generation parameters
The store has a model in it (module 2). Before wiring it into Python, code or a web UI, spend an hour driving it interactively — the fastest way to learn what each generation parameter actually changes, and to spot right away when a tag is wrong for the job. That is the modest promise of this module: ollama run, five parameters, and the discipline to change one at a time.
Opening a session
ollama run qwen2.5:14b-instruct-q4_K_M
The first call loads the model into RAM (or VRAM) — a few seconds on a workstation, up to a minute on a laptop for a large tag. Subsequent calls in the same session, and any call to the same tag from anywhere on the machine within OLLAMA_KEEP_ALIVE, are instant.
You are dropped at a >>> prompt. Type a prompt, press enter, watch the answer stream. Type /bye to exit. Under the hood you are chatting with the tag's default template applied to your message; the session keeps a rolling history until you clear it or leave.
The session commands
Slash commands run against the current session — they never touch the model on disk.
| Command | Effect |
|---|---|
/set parameter <name> <value> | Change a generation parameter for the session |
/set system "..." | Replace the system prompt for the session |
/show info | Print the running model's metadata |
/show parameters | Print the parameters currently in force |
/save <name> | Save the current session as a new tag |
/load <name> | Reload a saved session |
/clear | Reset the conversation history |
/bye | Quit |
/save contract-review-fr after tuning parameters and system prompt turns the session into a named tag, callable from any other tool the same way as an official tag. Module 5 shows the more powerful Modelfile route; this is the same idea in one keystroke.
The five parameters that matter
Twenty parameters are exposed. Five change what the model says most of the time. Everything else is either a niche knob or a synonym.
Temperature
temperature scales the sharpness of the next-token distribution. At 0, the model always picks the top token, and repeated calls with the same prompt are identical (given the same seed). At 1.0, sampling is spread across the top of the distribution — creative and varied. At 2.0, it is close to random.
For the law firm's assistant, 0 or 0.1 is the default: contract review needs the same answer on re-runs. Reserve 0.7 – 1.0 for brainstorming clause alternatives.
num_ctx
num_ctx is the context window size in tokens. It is the single parameter whose misconfiguration causes the most confusion. Ollama's global default is 2048 for older tags and often 4096 for recent ones — regardless of what the model itself supports. Ask a question that references a fifty-page contract loaded earlier and, if you left the default, the earliest pages are silently dropped from the window. The model then answers about what remains, confidently and wrongly.
Set it deliberately:
/set parameter num_ctx 8192
Bigger costs memory linearly and slows generation. Match num_ctx to the longest prompt the model will actually see, plus the reply budget. For a two-page contract clause and a short answer, 4096 is plenty; for the full contract, 16384 if the model supports it and the machine has the RAM.
num_predict
num_predict caps the length of the reply in tokens. Default -1 means "until the stop token". Set it to a reasonable ceiling in scripts (512 for short answers, 2048 for structured extractions) so a chatty model cannot ramble a minute of wall time.
Seed
seed fixes the random draw. Combined with temperature > 0, it makes generation reproducible. Combined with temperature = 0, it changes nothing — deterministic decoding already ignores it.
stop
stop accepts one or more strings that terminate generation on match. Useful for structured outputs: stop = "\n\n" for one paragraph, or stop = "</answer>" when the prompt asks for tagged output.
Measuring the effect
Change one parameter, ask the same question, compare. The --verbose flag prints timing after each answer:
$ ollama run qwen2.5:14b-instruct-q4_K_M --verbose
>>> Summarize the confidentiality clause of the attached NDA in three bullet points.
[...answer streams...]
total duration: 12.4s
load duration: 2.1s
prompt eval count: 1840 tokens
prompt eval duration: 1.8s
eval count: 182 tokens
eval duration: 8.5s
eval rate: 21.4 tokens/s
Two numbers to watch. Eval rate measures raw generation speed (tokens per second) — it drops with quantization quality and rises with GPU offload (module 7). Prompt eval duration is what a large num_ctx costs when you actually fill it: doubling the prompt tokens roughly doubles this line.
A working baseline for the firm's assistant
/set system "You are the internal assistant of a French-speaking law firm. Answer factually, cite the clause number when relevant, refuse to speculate on missing information."
/set parameter temperature 0
/set parameter num_ctx 8192
/set parameter num_predict 512
/set parameter stop "\n---\n"
That is the tunable baseline you carry into every module that follows.
No error is raised when num_ctx is smaller than the prompt: the runtime drops the oldest tokens and generates from whatever fits. If the answer suddenly ignores an early page of the document, num_ctx is the first suspect.
Summary
ollama runopens a streamed REPL; the model stays warm forOLLAMA_KEEP_ALIVEseconds after the last request.- Slash commands tune parameters and the system prompt in the session;
/saveturns the session into a reusable tag. - Five parameters cover most needs:
temperature,num_ctx,num_predict,seed,stop. --verbosereveals eval rate and prompt eval duration — the two numbers you optimize by tag choice, quantization and hardware.
Next module: driving the same model from Python through the local API, natively and via the OpenAI-compatible endpoint.