Lesson 3 — How text is generated
Text generation uses a completely different mechanism from image generation, and the mechanism is startlingly simple relative to what it produces.
One token at a time
The model does exactly one thing: given a sequence of tokens, produce a probability for every possible next token.
"The capital of France is" → Paris 94 %, a 2 %, located 1 %, the 0.8 %, …
Choose one. Append it to the sequence. Ask again. Repeat.
That is generation, in full. An essay, a function, a translation — all produced by running that loop a few thousand times. The process is called autoregressive because each output becomes part of the next input.
Two consequences follow immediately, and both explain behaviour you will have noticed:
Generation is inherently sequential. Token 500 cannot be computed before token 499, which is why responses stream in rather than appearing at once, and why text generation is harder to speed up than image generation.
The model cannot revise. Once a token is emitted it is part of the input and will be built upon. A model that starts down a wrong path tends to continue coherently down it, because the most plausible continuation of a wrong sentence is more wrong sentence. This is also why asking a model to "think step by step" measurably helps: it gives the model tokens in which to work, and each intermediate token conditions the next.
Why this produces coherent writing
It is fair to ask why such a myopic procedure yields structured argument rather than word salad.
Because predicting the next token well requires knowing an enormous amount. To continue "The patient presented with a fever and the doctor prescribed" you need medicine. To continue "In the third chapter, the narrator reveals that" you need to have followed the narrative. To close a bracket correctly you need to have tracked the code's structure.
Training to predict text on a large enough scale forces the model to encode grammar, factual associations, reasoning patterns, code structure and stylistic register — not as goals, but because they are what the prediction task requires. Scale did the rest: the same objective applied to vastly more text and parameters produced qualitatively new behaviour.
Sampling: the settings you actually touch
The model gives a probability distribution. How you draw from it changes everything about the output's character.
Greedy always takes the highest-probability token. Deterministic, and it produces flat, repetitive text and gets stuck in loops.
Temperature rescales the distribution before sampling. Below 1 sharpens it towards the likely tokens; above 1 flattens it towards variety.
| Temperature | Behaviour | Suits |
|---|---|---|
| 0 to 0.3 | Nearly deterministic, repetitive | Extraction, classification, code, structured output |
| 0.5 to 0.8 | Balanced | Explanation, general prose, summarising |
| 0.9 to 1.2 | Varied, surprising | Brainstorming, creative drafts |
| Above 1.5 | Frequently incoherent | Rarely useful |
Top-p (nucleus sampling) keeps only the smallest set of tokens whose probabilities sum to p, then samples among them. This adapts automatically: where the model is confident the set is tiny, where it is uncertain the set is large. It is usually a better default than temperature alone, and most interfaces expose both.
Temperature controls variability, not truthfulness. Setting it to zero does not make a model factual — it makes it produce its single most likely output, which may be confidently and repeatably wrong. Reducing temperature reduces variety in the errors, not their frequency.
The context window
The model sees a fixed maximum number of tokens: your prompt, any documents you supply, the conversation so far, and everything generated so far all share that budget.
Three practical facts about it:
It is not memory. Nothing persists between separate conversations unless a system explicitly re-supplies it. Apparent memory in a product is that system storing text and pasting it back in.
Cost and latency scale with it. You pay per token in and per token out, so a large context is not free even when it is available.
Large windows are used unevenly. Models attend most reliably to the beginning and end of a long context and least reliably to the middle. Putting your actual question at the end of a long document, rather than the start, is a small change that measurably improves results.
Why it fabricates
This is the single most important property to internalise, and it follows directly from the mechanism.
The model produces plausible continuations. A correct citation and an invented one are produced by exactly the same process: both are sequences that look like citations. Nothing in the architecture distinguishes retrieval of something known from construction of something plausible, because the model does not do retrieval at all.
So fabrication is not a defect to be patched. It is what generation is, applied to a case where the training data did not contain a strong enough pattern. And it is worst precisely where it is most dangerous: obscure facts, specific numbers, references, quotations, legal citations, API methods that sound like they should exist.
Mitigations reduce it and none eliminates it: supply the source material in the context so the answer can be grounded in it (RAG), ask for sources and check them, use the model to draft rather than to know, and let it say it does not know.
Structured output, which is where most production value sits
Generation is not only prose. Constraining a model to emit valid JSON, or to select from a fixed set of options, turns it into a general-purpose extractor:
Unstructured customer email →
{"intent": "refund_request", "order": "4471", "sentiment": "angry", "urgency": "high"}
This is quietly the most productive use of text generation in business systems. It is verifiable — the schema either validates or it does not — the output is small and cheap, and it replaces classification pipelines that used to need labelled training data. Low temperature, an explicit schema, and validation on the way out.
In three sentences
Text generation is next-token prediction run in a loop: the model produces a probability for every possible next token, one is chosen, it is appended, and the question is asked again, which is why output streams rather than appearing and why a model that starts wrongly continues wrongly rather than revising. Sampling settings shape the character of the output — temperature and top-p control variability, not correctness, so a temperature of zero yields a confidently repeatable error rather than a true answer — and everything the model sees must fit a fixed context window that is a budget rather than a memory. Fabrication is structural rather than a bug: a correct citation and an invented one come out of the same process, which is why anything factual needs grounding in supplied sources or independent verification, and why constraining output to a validated schema is where generation is most reliably useful.
Next — Lesson 4: using it well →