Skip to main content

Recap and final exam

Ten modules to go from "we should not send client files to a cloud" to a private assistant that reviews contracts, cites sources and stays inside the office. Here is the course condensed, then the threads that run through it, then the checklist for a real deployment.

The course at a glance​

ModuleThe essential point
1. Installing on Windows, macOS, LinuxOllama is a CLI plus a background HTTP service on 127.0.0.1:11434; OLLAMA_MODELS, OLLAMA_HOST, OLLAMA_KEEP_ALIVE, OLLAMA_NUM_PARALLEL cover every real deployment
2. Pulling, listing, removing modelsA tag is family + size + tuning + quantization; pin the full tag in code, pull deduplicates, rm frees only unshared blobs
3. Interactive session and parametersollama run, /set parameter, and five knobs that matter: temperature, num_ctx, num_predict, seed, stop; --verbose reveals eval rate
4. The local API/api/generate, /api/chat, /api/embeddings; the OpenAI-compatible route at /v1/ swaps existing SDKs by a base-URL change
5. Modelfiles and derived modelsFROM, SYSTEM, PARAMETER, TEMPLATE; ollama create produces a thin overlay tag; a caller's system message overrides, never merges
6. Quantization and memory footprintTotal RAM ≈ parameters × bits / 8 + KV cache; q4_K_M is the default; drop num_ctx first when out of memory
7. Hardware accelerationCUDA, ROCm, Metal picked automatically; num_gpu sets layer offload; measure with --verbose, not with vibes
8. Integrating with applicationsChatOllama, Open WebUI in Docker, and a reverse proxy with authentication — the runtime has no auth of its own
9. Local document Q&ALoader, splitter, nomic-embed-text, Chroma on disk — a RAG chain identical to course 18 with ChatOllama in place of a hosted model
10. Limits of local executionRoute hard questions to a hosted API, pin runtime and model digests, cross the single-machine line when concurrency, size or throughput demands it

The threads running through the course​

The runtime is stateless, the tag carries the state. The service loads and evicts models on demand; nothing about the loaded model is remembered across a restart. This is why the Modelfile matters so much — it is where the system prompt, defaults and base tag live, and it is the artifact that goes in git. Every mistake that surfaces as "the model behaves differently on this machine" traces back to an unpinned tag, a missing digest or a lost Modelfile.

Two numbers decide almost every design choice: RAM and num_ctx. Module 6's formula (weights plus KV cache) explains why the same tag runs on the server and not the laptop, why doubling the context window doubles the memory pressure, why quantization changes what fits, and why partial GPU offload buys speed on models too large to hold. Every out-of-memory triaged in production ends by touching either of these two numbers.

Local means offline for every hop, or it is not local. A pipeline that embeds through a hosted API and generates locally still leaks the chunks to a counterparty. Module 9's insistence on OllamaEmbeddings alongside ChatOllama is what makes the confidentiality argument true. The moment any component reaches out, the argument to the firm collapses.

Hosted versus local is a per-query decision, not a religion. Module 10 spells out what a hosted frontier model still does better and how to route to it with an explicit redaction step. The right shape is 95 percent local, 5 percent hosted, with a policy and an audit trail — never "everything on the cloud we banned" and never "everything on a laptop that cannot do it".

A production checklist​

Before opening the assistant to the associates, tick every box:

  • Model store on a large, fast disk (OLLAMA_MODELS)
  • Service bound only where intended (OLLAMA_HOST); firewall rule to the office subnet
  • Reverse proxy with basic auth or SSO in front of the port
  • Modelfile in git, base tag pinned by digest, Modelfile changes reviewed as code
  • num_ctx chosen per caller in the code, not left to the runtime default
  • RAG index encrypted at rest; backups do not exfiltrate it
  • OLLAMA_KEEP_ALIVE and OLLAMA_NUM_PARALLEL set for the fleet's real usage
  • Runtime updated on a monthly cadence, tags re-pulled by digest
  • Written policy for the "hard question" fallback to a hosted API

The final exam​

The exam has 40 questions covering the ten modules: installing the runtime and reading ollama output, choosing a model tag and interpreting a quantization label, tuning temperature and num_ctx, driving the native and OpenAI-compatible APIs, authoring a Modelfile and diagnosing an override, sizing RAM and diagnosing an out-of-memory failure, adding GPU acceleration and measuring throughput, integrating with LangChain and Open WebUI while keeping the port safe, assembling a local RAG chain, and choosing when to route to a hosted API.

Several questions present situations to diagnose: a caller whose derived model is silently ignored because a system message overrides it, an out-of-memory that only occurs when the context grows, retrieval quality that collapses after an embedder swap. It is judgment that is assessed, not the recitation of flag names.

On success, your certificate of completion is issued immediately; its number is verifiable by any third party on the platform.

Before you start

Take the table above and, for each row, ask yourself "how would I see that I am wrong here?". If you can say why an unpinned tag drifts, why num_ctx=2048 silently truncates a fifty-page contract, why the runtime accepts an unauthenticated caller on 0.0.0.0, and when hosted still wins over local, you are ready. Good luck!

Final exam

Ready to validate this course?

40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.

Start the exam

You need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.