Lesson 2 — Convolution, the idea that worked
In 2012 a network called AlexNet cut the error rate on a large image benchmark by roughly ten percentage points in a single year, after a decade of incremental gains. The field reorganised itself around the result within about eighteen months. The core idea it used was not new — it dated to the 1980s — but it finally had enough data and enough compute to demonstrate what it could do.
The idea is convolution.
One filter, applied everywhere
Take a small grid of numbers, say 3 × 3. Slide it across the image. At each position, multiply the filter's numbers by the pixel values underneath and add up the results. Write that single number into a new grid.
That is the entire operation, and it produces a feature map: an image-sized grid where each value says how strongly the filter's pattern is present at that location.
What makes it powerful is what a filter can encode. A filter with negative values on the left and positive on the right responds strongly wherever brightness increases from left to right — a vertical edge. Rotate the arrangement and you get horizontal edges. Other arrangements respond to corners, to blobs, to particular textures.
Crucially, the filters are not designed. They are learned. Training adjusts the nine numbers in each filter to whatever proves useful for predicting the label. The network discovers edge detectors because edges are useful, not because anyone specified them.
Why sliding the same filter matters so much
Two consequences follow from applying one filter across the whole image, and both are large.
Translation equivariance. A vertical edge produces a strong response wherever it appears. The network does not need to learn "edge in the top-left" separately from "edge in the centre" — it learns "edge" once. Position is handled by the operation itself.
Parameter efficiency. A fully connected layer on a 224 × 224 colour image with a thousand units would need roughly 150 million weights for one layer. A convolutional layer with sixty-four 3 × 3 filters needs about 1,700. Fewer parameters means less data required, faster training and far less overfitting.
This is what people mean when they say a convolutional network has a good inductive bias for images: the architecture assumes nearby pixels are related and that patterns can occur anywhere, and both assumptions happen to be true of photographs. Assumptions that hold are worth an enormous amount of training data.
Stacking builds a hierarchy
One convolutional layer finds edges. Feed its output into another and the second layer finds combinations of edges — corners, curves, junctions. A third finds combinations of those: textures, simple parts. Deeper still: eyes, wheels, letters. Deeper again: faces, cars, whole objects.
This hierarchy is not imposed. It emerges from training, and it can be inspected: visualising what early layers of a trained network respond to reliably shows edge and colour detectors, and the resemblance to what is known about early visual processing in mammals is close enough to be interesting.
Two supporting operations make the stack practical:
Pooling shrinks the feature maps, typically by keeping the strongest response in each small neighbourhood. This reduces computation and grants a little tolerance to small shifts, at the cost of positional precision — a trade-off that matters when you need exact boundaries.
Non-linearity is applied after each convolution, usually by setting negative values to zero. Without it, stacking layers would be mathematically equivalent to a single layer, and the hierarchy would collapse.
The receptive field, and why depth helps
A unit in layer 1 sees a 3 × 3 patch of the original image. A unit in layer 2 sees a 3 × 3 patch of layer 1, which corresponds to a 5 × 5 patch of the image. Keep going and each unit sees a progressively larger region. That region is its receptive field.
This is why depth matters more than filter size. To relate two parts of an image forty pixels apart, a convolutional network needs enough stacked layers for their receptive fields to overlap. A single enormous filter could do it in one step but would cost far more parameters. Depth buys long-range context cheaply — and slowly, which is precisely the weakness that transformers later exploited (lesson 4).
Transfer learning: the reason you need less data than you fear
A network trained on a million varied photographs learns filters that are useful for almost any visual task, because edges, textures and shapes are not specific to the categories it was trained on.
So nobody sensible trains from scratch. The standard approach:
- Take a network already trained on a large public dataset.
- Replace its final classification layer with one matching your categories.
- Continue training on your images, usually with a small learning rate.
The early layers already know how to see. You are teaching the last few layers what to look for.
| Approach | Images needed per class | Typical training time |
|---|---|---|
| From scratch | Tens of thousands | Days on multiple GPUs |
| Fine-tuning a pre-trained model | A few hundred to a few thousand | Minutes to hours on one GPU |
| Feature extraction only | Dozens to a few hundred | Minutes |
Transfer learning reduces how many images you need, and it does not reduce how representative they must be. Five hundred images taken under your actual deployment conditions beat fifty thousand scraped from the internet, every time. This is the single most reliable predictor of whether a vision project works, and it is lesson 5's main subject.
In three sentences
Convolution slides a small learned filter across an image and records where its pattern occurs, which handles position automatically and needs a tiny fraction of the parameters a fully connected layer would require. Stacking convolutional layers produces a hierarchy that emerges from training rather than design: edges combine into shapes, shapes into parts, parts into objects, with each layer seeing a larger region than the last. Because early layers learn features useful for nearly any visual task, you start from a pre-trained network and fine-tune it, which cuts the data requirement from tens of thousands of images per class to a few hundred — provided those hundreds actually resemble what the model will meet in production.
Next — Lesson 3: the four tasks →