Module 1 — Why ground a model in your own documents
Course 16 built a language model, and course 17 taught it to follow instructions. Both left the same problem unsolved: the model only knows what it was trained on. Ask it about the internal procedure your company updated last month and it will answer confidently — with something it made up. This module explains why, and why the fix is not to retrain the model.
Three limits that a raw model cannot cross
Frozen knowledge. Training happens once, on a snapshot of the world. Everything published, changed or corrected after that date is invisible. On the red thread of this course — around 300 internal procedures — the cutoff is fatal: the moment procedure QUAL-047 is revised, the model still describes the previous version.
Hallucinations. When a question falls into the gap between what the model remembers and what it does not, it does not stay silent. It generates the most plausible continuation. Plausible is not the same as true. A model happily invents a paragraph number, a signatory or a date, in the fluent tone of your legitimate documents. Nothing in the answer signals the fabrication.
Confidentiality. Sending an internal note to a hosted model to make it "learn" the content is, in practice, uploading that note to a third party. Even a fine-tuning provider that promises not to log the data still receives it and processes it on their infrastructure. For many organisations that alone is a reason to look elsewhere.
RAG, fine-tuning and long context: three different tools
The three approaches are often presented as competitors. They are complementary and address different problems.
| Approach | Solves | Cost of a change | Best when |
|---|---|---|---|
| RAG | Frozen knowledge, hallucinations, confidentiality | Reindex a few files | Content moves, sources must be cited |
| Fine-tuning | Style, tone, format of the answer | New training run | The way of speaking matters more than the facts |
| Long context window | Punctual question on a specific document | Zero | The user already knows which document to pass |
Fine-tuning teaches a model to behave differently — talk like a lawyer, always answer in JSON, refuse politically loaded questions. It is expensive to update because every change requires another training run, and it does nothing for factual freshness: the model will still not know about the note published yesterday. Long context is a cousin of RAG, but shifts the retrieval work onto the human: someone has to select the right document and paste it. That does not scale to 300 procedures and one hundred employees.
RAG chooses a different split. Facts live in an index that is cheap to update. The model stays generic and does the linguistic work: understanding the question, reading passages, composing an answer, refusing when the passages do not support one. Style adjustments are handled with a good system prompt, not another training run.
If tomorrow's answer must differ from today's because a source changed, choose RAG. If the answer format must change, adjust the prompt or fine-tune. If the shape of the answer changes, you touch the model; if the content it draws on changes, you touch the index.
The shape of the pipeline
Every RAG system, from a weekend prototype to a production assistant, has the same four stages. The rest of the course develops each in turn.
def answer(question: str) -> dict:
# 1. Retrieve: bring back a handful of passages that look relevant
passages = retriever.search(question, top_k=20)
# 2. Rerank: reorder them with a slower but sharper model
top_passages = reranker.rerank(question, passages, keep=6)
# 3. Compose: build a prompt that carries context and citations
prompt = build_prompt(question, top_passages)
# 4. Generate: let the language model answer, or abstain
reply = llm.generate(prompt)
return {"answer": reply, "citations": [p.source for p in top_passages]}
Everything upstream of step 1 is preparation: extracting text from PDFs and office files (module 2), chopping it into chunks (module 3), turning chunks into vectors and storing them (module 4). Everything downstream of step 4 is engineering: evaluating faithfulness (module 8), caching to keep the bill sensible (module 9), and wiring up the interface with permissions (module 10). Modules 5 and 6 develop steps 1 and 2 respectively, and module 7 develops step 3.
Where RAG still fails, and how to tell
RAG is not a magic wand. It shifts the failure modes rather than removing them.
- The retriever misses the right passage and the model, given only irrelevant context, either hallucinates or abstains for the wrong reason.
- The passage is retrieved but contradicts another one, and the model picks the wrong one, or blends them into a false compromise.
- The question is ambiguous, and the retriever bring back passages for the wrong meaning of a term.
- The model ignores the context and answers from its training, especially on generic questions that look like textbook material.
Module 8 gives the vocabulary to distinguish these cases — recall, faithfulness, coverage — and the tooling to measure each of them on a small annotated set. The point is not to eliminate errors but to make them visible enough to be worked on.
A model that outputs a fragment inside quotation marks may still be paraphrasing or fabricating. Citations are trustworthy only when the pipeline attaches them mechanically to the passages that actually went into the prompt. Module 7 shows how to do this so the check is verifiable, not just decorative.
In summary
- A raw language model has frozen knowledge, hallucinates when asked outside it, and cannot see private content — three limits RAG addresses at once.
- RAG, fine-tuning and long context solve different problems: RAG for moving facts and citations, fine-tuning for the way of speaking, long context for one-off use on a document the user already has.
- Every RAG pipeline has the same four stages — retrieve, rerank, compose, generate — and the course builds them one module at a time on the red thread of 300 internal procedures.
- RAG shifts failure modes rather than removing them: missed retrieval, contradictions, ambiguity, model bypassing context — module 8 makes each of them measurable.
Next module: extracting clean text from the actual files an assistant receives — PDF, HTML and office documents — while keeping the metadata that citations will need.