Skip to main content

Module 2 — Pulling, listing and removing models

The service is running (module 1) but its store is empty. The firm's partner asks a reasonable question: "which model should we start with?" — and the equally reasonable answer is "the smallest one that answers a real query correctly in the client's working language". This module turns that answer into commands.

What the library looks like​

Ollama publishes models on ollama.com/library. Each entry is a model family (for example llama3.1, qwen2.5, mistral, phi3, gemma2, nomic-embed-text) and a family carries many tags. A tag encodes three facts stacked together:

  • Size in parameters — 7b, 8b, 13b, 70b
  • Instruction tuning — instruct (chat-tuned) versus base (raw next-token)
  • Quantization — q4_K_M, q5_K_M, q8_0, f16, bf16

The fully qualified name looks like qwen2.5:14b-instruct-q4_K_M. Two shortcuts save typing: a family name alone (qwen2.5) resolves to the family's default tag, and a size alone (qwen2.5:14b) resolves to the default quantization for that size. Both are convenient in a terminal and dangerous in a script — pin the full tag when the answer matters.

Pulling a model​

The command is pull, and it is streamed — you watch each blob arrive, checksummed and deduplicated across tags that share weights:

ollama pull qwen2.5:14b-instruct-q4_K_M

For the law firm, qwen2.5 and llama3.1 are both reasonable starting points. qwen2.5 handles French-language contracts especially well, which matters if part of the practice is bilingual. A 7- or 8-billion-parameter tag at q4_K_M fits in about 6 GB of RAM, runs on the partner's MacBook, and gets full sentences out in seconds. A 70-billion-parameter tag needs a workstation with 48+ GB of RAM or a serious GPU — worth reaching for on the closet server, not on a laptop.

pull is safe to interrupt and safe to re-run. It resumes from the last complete blob and never re-downloads what is already on disk. Two families sharing an identical layer (say, the base weights before quantization) share the blob transparently.

Listing what you have​

ollama list

The output is a plain table: NAME, ID, SIZE, MODIFIED. The SIZE column is what the model actually occupies on disk, which is often less than the sum of tag sizes because of shared blobs — this is the reason du -sh ~/.ollama/models and ollama list | awk will disagree, and neither is wrong.

For a script, ollama list --json returns structured output that survives shell quoting. For a deeper look at any one tag:

ollama show qwen2.5:14b-instruct-q4_K_M

show prints the modelfile the tag was built with — its base, its default system prompt, its default parameters (temperature, context length, stop tokens), and the license. Reading it once for a model you plan to deploy avoids surprises later.

Removing what you no longer use​

The tags stack up faster than intuition suggests. A pull of llama3.1:70b-instruct-q4_K_M, then q5_K_M, then q8_0 to compare quality, fills 100+ GB. ollama rm releases the entry from the catalog:

ollama rm llama3.1:70b-instruct-q8_0

Removal frees any blob no longer referenced by another tag. A blob shared with a still-installed tag stays on disk — which is why the freed space can be less than the tag's reported size. To reclaim everything, remove the tags first, then rely on the runtime's garbage collection at next start, or manually clear ~/.ollama/models/blobs after confirming nothing else needs it.

Picking a model in the client's working language​

The firm handles files in French, English and occasionally Arabic. Two rules of thumb:

  • Match the model to the front-end language, not to the developer's language. qwen2.5 and llama3.1 are strong on European languages; command-r is strong on many languages including Arabic; phi3-mini is small and fine on English, weaker on French.
  • Prefer instruction-tuned tags for an assistant. A base model is a next-token predictor — useful for fill-in-the-blank, hopeless as a chat partner.

Do the smoke test in the target language, on a real prompt. Ten seconds of ollama run on three tags is worth more than ten minutes of benchmark tables from a blog post.

Tags with no explicit quantization

llama3.1:70b without a -qXXX suffix resolves to a default that may change between library revisions. In a script committed to git, always pin the full tag — including the quantization — so a colleague running the code six months later gets the same model.

Summary​

  • Model names decompose into family, size, tuning and quantization; the full tag is the reproducible reference.
  • pull streams, resumes and deduplicates blobs; a shared layer only downloads once.
  • list, show, rm inspect and manage the store; disk size and tag size differ because of blob sharing.
  • Choose the smallest instruction-tuned tag that answers a real query correctly in the front-end language; pin the full tag in code.

Next module: driving a loaded model interactively and setting the generation parameters that actually change what it says.