Module 5 — Modelfiles and derived models
Module 3 tuned parameters in a session; they lived until /bye. Module 4 passed those parameters on every API call; they lived in the code, easy to drift. This module puts them in one place — a Modelfile — that the runtime treats as a first-class citizen. The firm gets a single tag, firm-fr:latest, that carries the right base model, the right system prompt, and the right defaults, and every tool that already knows how to call an Ollama tag now speaks the firm's dialect for free.
The anatomy of a Modelfile
A Modelfile is a small text file, one directive per line, evaluated top to bottom:
FROM qwen2.5:14b-instruct-q4_K_M
SYSTEM """
You are the internal assistant of the Dupont & Partners law firm.
Answer factually, in the language of the question.
Cite article and clause numbers when the source is provided.
Refuse to speculate on missing information — ask for the document instead.
Never reveal client names or file numbers not present in the current prompt.
"""
PARAMETER temperature 0.1
PARAMETER num_ctx 8192
PARAMETER num_predict 1024
PARAMETER stop "---FIN---"
Four directives cover the vast majority of cases.
FROM names the base tag. It can be an official tag from the library, a locally created tag, or a path to a GGUF file on disk (below).
SYSTEM is the system prompt baked into every request against the derived tag. Multi-line strings are triple-quoted. Anything a caller does not explicitly override in a messages[0] with role: system picks this up.
PARAMETER sets a default for any generation option — temperature, num_ctx, num_predict, stop, repeat_penalty, seed, top_p, top_k. The caller's options on /api/chat override these; unset options fall back to what the Modelfile says.
TEMPLATE overrides the chat template. Ninety-five percent of Modelfiles omit it — the base tag already carries the correct template for its model family, and rewriting it is only necessary when the base and its template disagree, or when you want the derived model to behave like a different family.
Creating the derived model
Save the file above as Modelfile-firm-fr (any name is fine) and run:
ollama create firm-fr -f Modelfile-firm-fr
The runtime reads the file, resolves FROM, and registers a new tag. ollama list now shows firm-fr:latest alongside qwen2.5:14b-instruct-q4_K_M, and no additional blob is downloaded — the new tag is a thin overlay on the base's weights.
From any tool in the firm — Python, the CLI, a LangChain chain, the OpenWebUI in module 8 — firm-fr is now a callable model that already knows how to introduce itself, at what temperature to answer, and how much context to accept.
Iterating
Modelfiles are meant to be re-run. Change a line, re-run ollama create, and the tag is updated in place with a new ID. Callers pick up the change on the next request; no restart, no cache eviction ceremony. In practice, the Modelfile lives in the same git repository as the assistant's code, and every commit that changes a default carries the exact reasoning in its message.
Inspect what the runtime actually resolved:
ollama show firm-fr
The output is the effective Modelfile — including everything inherited from FROM — which is invaluable when a bug turns out to be a template clash inherited from the base.
Importing a fine-tuned GGUF
The finetuning course (course 19) produces a LoRA merged into a GGUF file — say, firm-fr-tuned-q5_K_M.gguf, a version of the base fine-tuned on ten thousand of the firm's anonymized clauses. Ollama can import it directly:
FROM ./models/firm-fr-tuned-q5_K_M.gguf
SYSTEM """
Same firm-fr system prompt as before.
"""
PARAMETER temperature 0.1
PARAMETER num_ctx 8192
ollama create firm-fr-tuned -f Modelfile-firm-fr-tuned
The GGUF is copied into the store and registered under the new tag. From that moment, firm-fr-tuned behaves like any other local model — same API, same performance profile as the quantization implies (module 6), no round-trip to any external service.
Two guardrails on imports. First, the GGUF file must be a chat model if you plan to call /api/chat with a system prompt — a raw base model imported with a chat template will produce garbled output on the first turn. Second, the file's quantization determines the tag's memory footprint from module 6 — the PARAMETER directive does not change it.
Sharing a tag
For a small firm, keeping the tag on each machine is enough — ollama create on the workstation, and copy the Modelfile to the associates' laptops. For a fleet, push to a private registry:
ollama push registry.firm.local/firm-fr:v1
Then anywhere:
ollama pull registry.firm.local/firm-fr:v1
The push transfers the derived overlay, not the base weights — colleagues who already pulled the base download only the small delta. Public sharing on ollama.com follows the same command; internal sharing needs a private registry, which is one of the reasons module 8 discusses network exposure carefully.
If your Python code passes messages=[{"role": "system", "content": "..."}], the Modelfile's SYSTEM is not merged — it is replaced. Either omit the caller's system message so the tag's prompt applies, or write the caller as if the Modelfile did not exist. The most common bug in derived models is a Python caller silently canceling the firm's prompt.
Summary
- A
Modelfilebakes a base tag, a system prompt and a set of default parameters into a new tag withollama create. - The derived tag is a thin overlay — no extra blob is downloaded — and is callable everywhere the base was.
FROM ./file.ggufimports a fine-tuned GGUF (course 19) as a first-class local model.ollama pushto a private registry shares only the overlay; a caller's explicit system prompt overrides the Modelfile's, not merges with it.
Next module: choosing the quantization that fits the machine — the single decision that governs RAM, VRAM and answer quality.