Lesson 6 — Recap and frequent questions
The five lessons in five sentences
- A discriminative model learns where categories separate while a generative model learns what the data itself looks like, which is far more to know and the reason generation arrived two decades after reliable classification.
- Diffusion adds noise to real images to create exact training targets, learns to reverse it, and generates by denoising pure static under the guidance of your prompt — all in a compressed latent space, which is why it runs on consumer hardware.
- Text generation is next-token prediction in a loop, sampling settings control variability rather than truthfulness, and fabrication is structural because a correct citation and an invented one come from the same process.
- Generation pays off when verifying the output is much cheaper than producing it, which is why drafting, variation and schema extraction work while unverifiable facts and decisions about people do not.
- The limits that produce real consequences are fabrication behind fluent prose, an unsettled copyright position including output that may not be copyrightable at all, memorisation of duplicated training content, synthetic media harms, amplified bias, and inference cost.
The two mechanisms, side by side
Different mechanisms entirely, and the same underlying bargain: enormous training data compressed into weights, sampled to produce something new, with no notion of whether the result is true or good.
A usage guide
| Situation | What to do |
|---|---|
| Need a first draft | Generate, then edit. Do not aim for publishable output in one pass |
| Need facts | Supply the source and ask about it. Never rely on recall |
| Need a consistent style in images | Train or download a LoRA adapter |
| Need a specific composition | Use inpainting or ControlNet, not a longer prompt |
| Need structured data from text | Low temperature, explicit schema, validate the output |
| Output goes public | Human review of every fact and every claim, always |
| Input is confidential | Check retention terms, or run a model locally |
| Output is commercial | Reverse image search, and read your provider's indemnity clause |
| Cost is a concern | Use the smallest model that passes your tests |
Twelve frequent questions
1. Which generative tool should I use?
For text, the frontier hosted models are close enough that workflow integration and data terms matter more than benchmark differences. For images, the hosted services are easiest and the open models are more controllable and run locally. The useful approach is to test two on your actual task for an hour rather than reading comparisons.
2. Is a bigger model always better?
No, and assuming so is expensive. Small models handle classification, extraction, formatting and routine drafting perfectly well at a fraction of the cost and latency. The productive pattern is a small model by default with escalation to a large one when the task genuinely needs it.
3. Why do I get different answers to the same question?
Because sampling is random by design. Set temperature to zero for near-deterministic output where you need reproducibility, remembering that this makes the errors consistent rather than absent.
4. Can a model learn from my corrections?
Not within a conversation in any lasting sense, and not between conversations unless a system re-supplies the context. Persistent adaptation requires fine-tuning or storing your preferences and inserting them into each prompt, which is what products offering "memory" do.
5. What is fine-tuning and do I need it?
Continuing training on your own examples so the model adopts a format, tone or domain convention. It is good at teaching style and poor at teaching facts — for facts, supply them in the context instead. Most teams reach for fine-tuning before exhausting prompting and retrieval, and regret the order.
6. How do I stop it inventing things?
You cannot eliminate it. You can reduce it substantially by supplying source material and instructing the model to answer only from it, requiring citations you then check, preferring extraction over generation, and designing the system so declining to answer is an acceptable outcome.
7. Can I sell what I generate?
Check your provider's terms, which vary, and understand that in several jurisdictions purely machine-generated work may not be copyrightable — so you may be unable to prevent others from using it. Run a reverse image search on anything commercial, since resemblance to memorised training content is a genuine if uncommon risk.
8. Why is it bad at counting and arithmetic?
Because it predicts plausible tokens rather than computing. Numbers are also fragmented by tokenisation, which makes digit-level operations awkward. Modern systems handle this by calling an actual calculator or running code, which is the right architecture: let the model decide what to compute and let a tool compute it.
9. What is the difference between an LLM and generative AI?
Generative AI is the category covering all content production. A large language model is the text member of that family. Image generation uses diffusion, a different mechanism entirely, and multimodal models combine both in one system.
10. Should I disclose that content was generated?
Norms are moving faster than law, and the expectation is spreading across journalism, academic work and some advertising regulation. Assuming disclosure is expected is the safer default, and the cost of over-disclosing is much lower than the cost of being found not to have.
11. Will these tools keep improving at this rate?
Uncertain, and worth being sceptical of confident claims. Gains from simply adding scale have been slowing, high-quality training text is finite, and recent progress has come more from post-training, tool use and reasoning techniques than from raw size. Predictions have a poor track record in both directions.
12. Where should I start if I want to build with this?
Take one task you do repeatedly, where you can check the answer, and automate it end to end including the verification step. That teaches you more about the real constraints — cost, latency, error handling, prompt brittleness — than any amount of experimenting in a chat window. The premium catalogue covers Stable Diffusion, fine-tuning and production pipelines.
Vocabulary you can now use precisely
| Term | Meaning in one line |
|---|---|
| Generative model | Learns what data looks like, so new samples can be drawn from it |
| Diffusion | Learning to reverse added noise, then denoising static into an image |
| Latent space | A compressed representation where generation actually happens |
| Guidance scale | How strongly the prompt pulls generation, trading adherence for naturalness |
| Seed | The number fixing the initial noise, making a result reproducible |
| Inpainting | Regenerating only a masked region of an image |
| ControlNet | Conditioning generation on structure such as pose, edges or depth |
| LoRA | A small adapter file steering a base model towards a style or subject |
| Token | The unit of text a model reads and writes, roughly three-quarters of a word |
| Temperature | How much randomness is applied when choosing the next token |
| Top-p | Sampling only from the smallest set of tokens covering probability p |
| Context window | The fixed token budget holding prompt, history and response |
| Fabrication | Plausible output with no basis, produced by the same process as correct output |
| Model collapse | Degradation from training successive models on synthetic output |
And now?
You know what happens when you type a prompt, which tasks are worth handing over, and which risks are structural rather than temporary. That is enough to make good decisions and to recognise overstatement when you meet it.
Validate it: take the 5-question quiz →
Continue free: Large Language Models covers the text side in depth, including retrieval and agents.
Go deeper: the premium catalogue covers Stable Diffusion, LoRA training, fine-tuning and production pipelines hands-on, with a verifiable certificate after a 40-question examination.