Module 2 — From rule-based systems to learning from data
The history of AI is not a cultural detour: it explains why today's tools have the shape they have, and it vaccinates against the cycles of enthusiasm and disappointment that have been recurring, almost identically, for seventy years.
The era of rules: writing intelligence by hand
From the founding Dartmouth conference (1956) until the late 1980s, the dominant approach was symbolic: intelligence would be a matter of logic, hence of rules. You interview specialists, transcribe their knowledge into thousands of "if… then…" rules, and you get an expert system.
Some genuinely worked. MYCIN diagnosed bacterial infections with an accuracy comparable to physicians of the time. XCON configured DEC's computer orders and saved the company millions of dollars a year. In the 1980s, entire companies were built on this promise.
Then the approach hit three walls.
The acquisition wall. Extracting knowledge from experts is slow, expensive and contentious: experts disagree with each other, and much of their competence is intuitive, impossible to phrase as rules.
The maintenance wall. At ten thousand rules, every addition conflicts with what exists. Systems became fragile cathedrals nobody dared to touch.
The real-world wall. Rules assume clean inputs and anticipated situations. Faced with noisy data, an unusual phrasing, an edge case, a rule-based system does not degrade gracefully: it is confidently wrong, or it stops.
The field went through two funding freezes — in the mid-1970s and again in the late 1980s — each time after public promises far beyond actual capabilities. The pattern is always the same: a spectacular demonstration on a narrow case, media extrapolation, contact with the real world, disappointment, budget cuts. Knowing this cycle helps calibrate today's announcements.
The statistical shift: letting the data write the rules
In the 1990s, another tradition took over, driven by three converging forces: more digitized data, more computing power, and theoretical progress in statistical learning.
The idea reverses the workflow. Instead of asking an expert "how do you recognize a spam email?", you collect a hundred thousand emails labeled spam or legitimate, and you let an algorithm find the regularities. Knowledge is no longer written; it is learned.
This reversal changes the nature of the work. The critical resource is no longer the expert who writes rules, but labeled data. The critical skill is no longer logic, but statistics. And the validation question is no longer "are the rules correct?" but "does the model generalize to new cases?" — a subtler question we will treat in depth in module 5.
The 1990s–2000s were the years of classical statistical methods: logistic regression, support vector machines, random forests. They quietly powered fraud detection, credit scoring and the first recommender systems. AI worked, but it no longer made headlines — it was discreetly called "data mining" or "analytics".
2012: the deep learning breakthrough
Neural networks had existed since the 1950s and had disappointed twice. Three missing ingredients came together around 2010:
- Massive data. The internet supplied millions of images, texts and sounds, labeled or labelable. ImageNet, published in 2009, offered 1.2 million images annotated in a thousand categories.
- Parallel computing. Graphics cards (GPUs), designed for video games, turned out to be perfect for the matrix multiplications neural networks are made of.
- Training techniques that finally made deep networks stable: better activation functions, initializations, regularization.
In 2012, the AlexNet network won the ImageNet competition, cutting the error rate from 26% to 16% — a leap of unprecedented magnitude in that contest. Within three years, all of computer vision switched to deep learning; speech and translation followed.
The rest of the story — the transformer architecture (2017), then the large language models — is covered in detail in the dedicated courses. What matters here: each era did not replace the previous one; it was added to it. The random forests of 1995 remain the best choice today on a large share of tabular business problems.
Why this history informs your decisions
Lesson 1 — The shape of the solution follows the shape of the data. Rules fit when knowledge is explicit and stable (regulatory computation, format validation). Learning fits when knowledge is diffused across examples (perception, language, behavior). Many real systems combine both — and that is perfectly fine.
Lesson 2 — Breakthroughs come from the conjunction of data + compute + algorithms. When an announcement promises a revolution, look for which of these three factors actually changed. If none did, it is marketing.
Lesson 3 — The hype-disappointment cycle is structural. Capabilities progress in steps; expectations rise continuously. The gap between the two always closes painfully. A project calibrated on demonstrated capabilities — not on promises — sails through the winters unharmed.
Key takeaways
- Rule-based systems hit the walls of knowledge acquisition, maintenance, and fragility in the real world.
- The 1990s shift: knowledge is no longer written by experts but learned from data; labeled data becomes the critical resource.
- 2012 (AlexNet on ImageNet) marks the deep learning breakthrough, enabled by massive data + GPUs + training techniques.
- Eras add up instead of replacing each other: the right tool depends on the problem, not on fashion.
Next module: what research actually says about the gap between narrow AI — the only kind that exists — and the general AI that fuels the headlines.