Module 1 — Convolution, filters and feature maps

Course 07 built dense networks: every input pixel talked to every neuron of the next layer. On the 32 by 32 colour images of CIFAR-10 — the dataset that will follow us through the first six modules — that costs three thousand parameters per neuron. Multiply by a few thousand neurons and the model outweighs the training set. The convolutional layer solves that with two ideas: a small filter that is slid over the image, and the same weights shared across every position.

A convolution is a sliding weighted sum

A 2D convolution takes an input image $I$ and a small kernel $K$ of size $k \times k$. It produces a new image $O$, called a feature map, where each output pixel is a weighted sum of a small neighbourhood of the input:

$$O(i, j) \;=\; \sum_{u=0}^{k-1} \sum_{v=0}^{k-1} I(i + u,\, j + v)\, K(u, v).$$

Let us do it by hand on a 5 by 5 grayscale image with a 3 by 3 kernel. The output is 3 by 3, because a 3 by 3 window fits in three positions horizontally and three positions vertically inside a 5 by 5 image.

import numpy as np

image = np.array([
    [10, 10, 10,  0,  0],
    [10, 10, 10,  0,  0],
    [10, 10, 10,  0,  0],
    [10, 10, 10,  0,  0],
    [10, 10, 10,  0,  0],
], dtype=float)

# Vertical Sobel: detects vertical edges (columns where intensity changes).
kernel = np.array([
    [-1, 0, 1],
    [-2, 0, 2],
    [-1, 0, 1],
], dtype=float)

def conv2d(img, ker):
    kh, kw = ker.shape
    ih, iw = img.shape
    out = np.zeros((ih - kh + 1, iw - kw + 1))
    for i in range(out.shape[0]):
        for j in range(out.shape[1]):
            out[i, j] = np.sum(img[i:i+kh, j:j+kw] * ker)
    return out

print(conv2d(image, kernel))
# [[ 0. -40.  0.]
#  [ 0. -40.  0.]
#  [ 0. -40.  0.]]

The output is zero in flat regions and lights up on the column where intensity drops from 10 to 0. That third column is precisely the vertical edge of our synthetic image: the Sobel kernel just detected an edge without being told where to look.

The Sobel intuition: an edge detector as a fixed CNN filter

The Sobel operators come in two versions. The horizontal one detects rows where intensity changes; the vertical one detects columns. On a real image, combining them approximates the local gradient of pixel intensities.

sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]], dtype=float)
sobel_y = np.array([[-1, -2, -1], [0, 0, 0], [1, 2, 1]], dtype=float)

For decades, computer vision engineers designed such filters by hand: Sobel, Prewitt, Gabor, LoG. A CNN does the same thing, except that the numbers inside every kernel are learned by gradient descent from the labelled data. The first convolutional layer of a trained network almost always contains something that looks like edges and colour blobs, because those turn out to be the features that separate CIFAR-10 classes at the lowest level.

Weight sharing: the reason a CNN scales

Compare two layers on a 32 by 32 colour input, which is 3 072 numbers.

• A dense layer with 32 output neurons has $32 \times 3072 = 98\,304$ weights, plus 32 biases.
• A convolutional layer with 32 kernels of size $3 \times 3 \times 3$ has $32 \times 3 \times 3 \times 3 = 864$ weights, plus 32 biases.

Two orders of magnitude fewer parameters, for a layer that also enforces translation equivariance: shifting a cat by ten pixels to the right shifts the corresponding activations by ten pixels to the right, instead of firing entirely different neurons. That property is baked into the architecture, not learned from data, which is why CNNs need drastically less data than dense networks for the same image task.

Parameters of a Conv2D layer

The formula to memorise: $C_{in} \times k_h \times k_w \times C_{out} + C_{out}$ for the bias. In Keras, Conv2D(32, (3, 3)) on a 3-channel input has exactly 896 parameters. The framework prints it in model.summary(); get in the habit of predicting the number before reading it.

Multiple input channels, multiple output filters

Real images have three channels (red, green, blue). A single kernel therefore spans all input channels: for a 3 by 3 kernel on a colour image, that means 27 weights, not 9. The output of that one kernel is a single-channel feature map: the sum has already collapsed the colour axis.

To produce a multi-channel output, we stack several independent kernels. A Conv2D(64, (3, 3)) layer applied to an RGB image contains 64 kernels of shape $3 \times 3 \times 3$, and produces a 64-channel feature map. Each output channel can specialise: one for red-to-green transitions, one for vertical edges, one for small round textures. The network discovers the specialisation from the data.

import tensorflow as tf

# One convolutional layer on the CIFAR-10 red thread.
inputs = tf.keras.Input(shape=(32, 32, 3))
x = tf.keras.layers.Conv2D(32, (3, 3), activation="relu")(inputs)
print(x.shape)   # (None, 30, 30, 32) — see module 2 for the -2
print(x.dtype)   # float32

Where convolution differs from cross-correlation

Mathematically, a true convolution first flips the kernel horizontally and vertically, then applies the sliding sum. Every deep learning framework instead applies the sliding sum without flipping, which is properly called cross-correlation. The distinction does not matter in practice, because the kernel is learned: if the training data prefers a flipped kernel, gradient descent will learn a flipped one. But it explains why hand-written kernels from a signal processing textbook sometimes look mirrored compared to a CNN filter that does the same thing.

Do not confuse channels and filters

A 3 by 3 kernel on 3 input channels is not three separate 3 by 3 kernels: it is one $3 \times 3 \times 3$ tensor that outputs a single number per position. The number of output channels is set by the number of kernels, not by the number of input channels. Reading a Conv2D summary correctly starts with this distinction.

In summary

• A convolution is a weighted sum on a small window slid over the image; the weights are the same at every position and form a kernel.
• Classic hand-designed filters like Sobel detect edges; a CNN learns similar filters from data, and later layers combine them into more complex patterns.
• Weight sharing cuts parameter counts by orders of magnitude compared to a dense layer, and grants translation equivariance for free.
• A Conv2D(C_out, (k, k)) layer on $C_{in}$ input channels has $C_{in} \cdot k^2 \cdot C_{out} + C_{out}$ parameters, independent of the image size.

Next module: what happens at the borders of the image, and how stride, padding and dilation control the size of the output and the region of the input each output neuron actually sees.