Module 10 — The current limits and the paths forward
Last course module: a dated snapshot — that of the mid-2020s — of the technology's structural limits and the research directions trying to push them back. The goal is not exhaustiveness but lucidity: knowing what current systems cannot do is as useful as knowing their capabilities.
Five structural limits
Data hunger. A child recognizes a giraffe after seeing two; a standard vision model asks for thousands. Statistical learning remains fundamentally sample-inefficient compared to human learning. Practical consequence: data-poor domains — rare diseases, exceptional failures, low-resource languages — remain badly served.
Fragility out of distribution. A model that performs well on its training distribution can fail abruptly — and silently — on slightly different inputs: a replaced sensor, a changed format, a new population (modules 5 and 9). Humans degrade gracefully facing the unusual; current models do not. They do not know that they do not know.
Opacity. A network of billions of parameters does not, by construction, provide a justification for its outputs. Yet entire sectors — credit, healthcare, justice, insurance — require explainable decisions, by ethics and by regulation. Explainability techniques (XAI) produce useful clues (which features weighed), not complete explanations; the gap between what the law demands and what the technique provides remains an open construction site.
Cost. Training the largest models runs into tens of millions of dollars of compute, with the corresponding energy footprint; their inference at scale is not free either. This cost concentrates the ability to train frontier models among a few actors — an industrial and political question as much as a technical one.
Reasoning and the world. Current systems excel at pattern recognition; they remain unreliable at verifiable multi-step reasoning, arithmetic, long-horizon planning, and have no direct grip on the physical world. Language models produce plausible text — including when it is false (hallucinations): plausibility is not truth.
The paths structuring research
Facing each limit, active work directions — worth knowing to read the field's news without getting lost.
Learning with less. Transfer learning (start from a pre-trained model, adapt it with few examples — already the norm), few-shot learning, synthetic data, self-supervision (module 6). This is the most mature path: it has already changed practice.
Quantifying uncertainty and detecting the unusual. Having the model produce not only a prediction but a calibrated confidence level, and detecting out-of-distribution inputs to hand over to a human. Essential for critical uses; the methods exist, their reliability remains uneven.
Opening the box. Post-hoc explainability (attributing the output to the inputs), models constrained to be interpretable where the stakes justify it, and — more recently — mechanistic interpretability: understanding the internal circuits of networks. A young field, with partial but real results.
Cutting costs. Distillation (compressing a large model into a small one), quantization (reducing numerical precision), efficient architectures, specialized hardware. The course on small language models (SLM) details this path, whose progress is fast and very concrete.
Hybridizing. Combining statistical learning with what it lacks: external tools (calculator, database, search engine — the agents approach), retrieval of verifiable knowledge (RAG, the subject of a dedicated course), symbolic verification of outputs. The shared intuition: do not ask the model for everything; surround it with a system.
Reading the future without a crystal ball
Three principles for evaluating the announcements to come — in six months as in six years.
Look for the factor that changed. Data, compute, algorithm: any real breakthrough is explained by at least one of the three (module 2). An announcement that invokes none is a communication exercise.
Distinguish demonstration from deployment. The road between "it works in a demo" and "it holds in production on real cases" is precisely what this course has mapped: data, honest evaluation, robustness, monitoring. Most spectacular announcements are demonstrations.
Distrust linear extrapolations. The field progresses through steps and plateaus, not in a straight line. Confident predictions — enthusiastic or catastrophist — have a poor reliability record in both directions.
The tools will change; the questions of this course — which data, which metric, what error cost, what domain of validity, what monitoring — will remain. They will apply as they are to the next generation of systems. That is the part of your learning that will not go out of date.
Key takeaways
- Five structural limits: data hunger, fragility out of distribution, opacity, cost, reasoning — all active, none solved.
- Five paths facing them: learning with less, calibrating uncertainty, opening the box, cutting costs, hybridizing the model with tools.
- Reading grid for announcements: which factor changed, demonstration or deployment, distrust of extrapolations.
- The method questions of this course will outlive today's tools.
One step remains: the recap page, then the 40-question exam that validates the course and issues your certificate.