Skip to main content

Module 2 — Landscape of open small models

Picking the base model matters more than any prompt tweak that follows. A model with the right license, the right languages and the right size lets the next eight modules earn their keep; the wrong one turns them into a rescue operation. This module surveys the five families that matter in 2026 for the 1 to 4 billion parameter range and shows how to read a model card without confusing marketing with fact.

The running example — the support-ticket assistant — needs a model that (a) speaks the language of the tickets fluently, (b) is small enough to run comfortably on a laptop CPU, and (c) has a license that lets a small company ship it to customers without a legal review that outlasts the project.

The five families you should know by name​

Phi (Microsoft Research). The 2026 line goes from Phi-3.5-mini (3.8B) to Phi-4-mini (3.8B) with a strong reasoning slant. Trained on curated and synthetic data, they punch above their weight on benchmarks that reward chain-of-thought. Weaker on non-English languages than Qwen or Gemma. MIT license, so genuinely permissive.

Gemma (Google). Gemma 2 exists at 2B and 9B, Gemma 3 in the same tiers. Solid multilingual coverage, good tool-calling fine-tunes available. Gemma license — mostly permissive but with a specific use-restriction clause; read it before shipping to a large customer.

Qwen (Alibaba). Qwen 2.5 and Qwen 3 offer 0.5B, 1.5B, 3B and 7B instruct variants, with excellent multilingual coverage — especially Chinese, English, Arabic and French — and mature tool-calling. Apache 2.0 for the small tiers, which is as permissive as it gets.

Llama (Meta). Llama 3.2 introduced 1B and 3B tiers optimised for on-device use. Strong English, weaker on smaller languages. Llama Community License — free for organisations under 700M monthly active users, requires attribution and forbids using the outputs to train competing models. Not incompatible with commercial use, but it is not Apache.

Mistral (Mistral AI). Ministral 3B and 8B are the "small" tier alongside Mistral 7B. Strong European-language support. Mistral Research License for the small research releases, Apache 2.0 for some open releases such as Mistral 7B. Check per model, not per company.

Licenses in one paragraph​

The rule of thumb: Apache 2.0 and MIT are safe defaults — you can ship, resell, fork, close-source your derivative. Gemma license, Llama Community License and Mistral Research License are custom licenses that need to be read once by someone who reads licenses for a living, then filed. The two clauses that catch people are (i) restrictions on using model outputs to train other models, and (ii) obligations to reproduce a specific attribution notice in the product. Neither is prohibitive; both must be respected.

Language coverage matters more than the leaderboard​

A model with a MMLU score of 68 in English and 41 in French is not a "68" model for a French support desk — it is a 41 model. Language coverage is the single most under-reported dimension in model cards. Concretely:

  • Qwen 2.5 3B and Gemma 2 2B are the two safest defaults for French, Spanish, Italian, Portuguese and German at the 2–3B tier in 2026.
  • Qwen 2.5 3B is currently the safest for Arabic at that size; Llama 3.2 3B is noticeably weaker.
  • Phi-4-mini is strongest in English and technical reasoning; a French support desk will feel the gap in the summary quality.

The test is one paragraph, not a benchmark: give the candidate model twenty real tickets in your language and read the summaries yourself. A model that produces stilted or comically wrong French will not become fluent through fine-tuning; the base has to be right first.

How to read a model card without being fooled​

Every model on Hugging Face ships with a card. Six lines are the ones that decide.

Parameter count and precision at release. A 3B model in fp16 weighs about 6 GB; the same in int4 weighs about 2 GB. Both are called "the 3B model" and only one fits on a laptop with 8 GB of RAM used by other applications.

Context window. A 4k window versus an 8k or 32k window changes what you can pass in. For our tickets 4k is enough; for longer documents it is not.

License. Written in the card; read it before you write the first prompt, not after the demo.

Language list. Explicit list, or the sentence "trained primarily on English data". The second sentence is the model card's polite way of saying "expect a quality gap".

Training-data date cutoff. A 2024 cutoff model does not know about a policy change from 2025. For a support assistant this rarely matters; for a coding assistant it does.

Reported benchmarks and — importantly — which ones are missing. A card that reports MMLU-Redux and GSM8K but omits multilingual benchmarks is telling you where the model was optimised.

Picking one for the running example​

For the French support-ticket assistant of this course we pick Qwen 2.5 3B Instruct. It is Apache 2.0, so the shipping story is trivial. It speaks French fluently at that size. It has mature tool-calling for module 8. It fits in about 2 GB after 4-bit quantization (module 4), which leaves enough memory on a work laptop for the rest of the applications an agent runs. For an Arabic desk, the same choice; for an English-only technical desk, Phi-4-mini would be a defensible alternative.

In summary​

  • The five families to know at 1–4B are Phi, Gemma, Qwen, Llama and Mistral — each with a different strength profile and a different license.
  • License decides whether you can ship; Apache 2.0 and MIT are safe defaults, the others require a real read.
  • Language coverage dominates leaderboard scores for any non-English desk; test with your own paragraphs before trusting a benchmark.
  • The model card answers most questions in six lines — parameters, precision, context, license, languages, cutoff — read it once, do not confuse it with a datasheet.

Next: distillation — how to teach a small model a specific task by using the large one as a teacher, and how the ticket data set will grow as a result.