Module 7 — Inception, MobileNet and compute efficiency

VGG and ResNet chased accuracy. Inception and MobileNet chased the same accuracy at a fraction of the cost. That is not a niche concern: as soon as a model has to run on a phone, on a browser tab, or on a video stream, latency and battery become the primary constraints and accuracy becomes a runner-up. This module explains the two structural ideas that shaped efficient CNNs, and warns against the metric that keeps misleading people: floating-point operations per inference.

Inception: run several kernel sizes in parallel

The intuition behind Inception (GoogLeNet, 2014) is that different objects need different receptive fields at the same depth. A face fits in a 5 by 5 region of a low-resolution feature map, an eyebrow in a 3 by 3 region, a small texture in a 1 by 1 patch. Rather than picking one kernel size and hoping, an Inception module runs several in parallel on the same input and concatenates the outputs along the channel axis.

The naive version would be prohibitively expensive: a 5 by 5 convolution on 256 channels is a huge tensor multiplication. Inception fixes that with the same 1 by 1 trick we met in module 5: project down to fewer channels before the expensive spatial convolution, and back up if needed.

import tensorflow as tf
from tensorflow.keras import layers

def inception_module(x, f1, f3_reduce, f3, f5_reduce, f5, pool_proj):
    b1 = layers.Conv2D(f1, 1, activation="relu")(x)
    b3 = layers.Conv2D(f3_reduce, 1, activation="relu")(x)
    b3 = layers.Conv2D(f3, 3, padding="same", activation="relu")(b3)
    b5 = layers.Conv2D(f5_reduce, 1, activation="relu")(x)
    b5 = layers.Conv2D(f5, 5, padding="same", activation="relu")(b5)
    bp = layers.MaxPooling2D(3, strides=1, padding="same")(x)
    bp = layers.Conv2D(pool_proj, 1, activation="relu")(bp)
    return layers.Concatenate(axis=-1)([b1, b3, b5, bp])

The four branches produce four feature maps of the same spatial size but different channel semantics; the concatenation glues them into one wide feature map. GoogLeNet stacks nine such modules and reaches AlexNet-level accuracy with 12 times fewer parameters.

Later versions (Inception-v3, Inception-v4) factorised the 5 by 5 branch as two 3 by 3, and the 3 by 3 as a 1 by 3 followed by a 3 by 1 — repeated applications of the VGG lesson from module 5. Inception-ResNet, as the name suggests, added residual shortcuts on top.

MobileNet: depthwise separable convolutions

A standard 3 by 3 convolution on 256 input channels producing 256 output channels performs $3 \times 3 \times 256 \times 256 = 589\,824$ multiply-adds per output pixel. That is dominated by the channel mixing part. Sifre and colleagues asked: what if we separated the two?

A depthwise separable convolution decomposes the operation into two cheaper ones:

1. Depthwise convolution — one 3 by 3 kernel per channel, applied independently on that channel only. Cost: $3 \times 3 \times 256 = 2\,304$ multiply-adds per output pixel.
2. Pointwise convolution — a 1 by 1 convolution that mixes the 256 channels. Cost: $1 \times 1 \times 256 \times 256 = 65\,536$ multiply-adds per output pixel.

Total: about 68 000, an 8.7× reduction compared to the standard convolution, for the same input-output shape and roughly the same accuracy on ImageNet.

def depthwise_separable(x, filters, strides=1):
    x = layers.DepthwiseConv2D(3, strides=strides, padding="same", use_bias=False)(x)
    x = layers.BatchNormalization()(x)
    x = layers.ReLU(6)(x)  # ReLU6, capped at 6, MobileNet convention
    x = layers.Conv2D(filters, 1, use_bias=False)(x)
    x = layers.BatchNormalization()(x)
    return layers.ReLU(6)(x)

MobileNet-v1 stacks depthwise separable blocks in place of standard convolutions and adds two hyperparameters: the width multiplier (scales the channel count) and the resolution multiplier (scales the input size). MobileNet-v2 added inverted residual blocks — the opposite of the ResNet bottleneck: expand channels, run depthwise, project down, with a shortcut on the narrow ends.

FLOPs versus real-world latency

Papers report multiply-add operations (MACs) or floating-point operations (FLOPs) as an efficiency proxy. That number is easy to compute and easy to compare, and it is systematically misleading in production.

Reason FLOPs mislead	Consequence
Memory bandwidth	Depthwise convolutions have great FLOP counts but read the input twice; on GPU they underuse the arithmetic units.
Kernel launch overhead	A layer split into many small ops (Inception, depthwise + pointwise) pays a fixed cost per kernel. Latency does not divide by the FLOPs ratio.
Hardware-specific optimisations	On mobile NPUs, 3 by 3 conv has hand-tuned kernels; a 5 by 5 conv can be slower even with fewer FLOPs.
Parallelism	ResNet runs its blocks sequentially; Inception's parallel branches only pay off when the hardware can execute them concurrently.

The practical answer is boring: measure end-to-end latency on your target device, not FLOPs. On a laptop CPU, MobileNet-v2 can be slower than ResNet50 despite four times fewer FLOPs, because the CPU cannot exploit the depthwise sparsity as well as a mobile GPU can.

FLOPs is a floor, not a ceiling

FLOPs give a lower bound on how fast a network could ever run; they say nothing about how fast it will run on your box. When choosing a mobile model, benchmark on the actual phone; when choosing a server model, benchmark inside your target inference runtime (TensorRT, ONNX Runtime, TFLite). The paper's headline number is the best case, rarely reproducible.

A glance at EfficientNet

EfficientNet (2019) turned the scaling question into a joint optimisation. Instead of scaling depth alone (ResNet), or width alone (WideResNet), or resolution alone (VGG variants), it scales the three dimensions together with a compound coefficient $\phi$:

$$\text{depth} = \alpha^{\phi}, \quad \text{width} = \beta^{\phi}, \quad \text{resolution} = \gamma^{\phi}, \quad \alpha \cdot \beta^2 \cdot \gamma^2 \approx 2.$$

EfficientNet-B0 through B7 correspond to increasing values of $\phi$. On ImageNet, EfficientNet-B0 reaches ResNet50 accuracy at a quarter of the FLOPs, and B7 beats ResNet152 at half the FLOPs. Same warning as above: measured latency on GPU is closer, because EfficientNet's Swish activation is more expensive than ReLU.

Bringing it back to the red thread

The waste-sorting case study of modules 8 and 9 will use ResNet50 as the pretrained backbone, precisely because it is the default balanced choice. But keep in mind: if the same model had to run on a smartphone camera to sort waste in real time, MobileNet-v3 or EfficientNet-B0 would replace ResNet50, and everything else in modules 8 and 9 would stay identical. The choice of backbone is orthogonal to the transfer procedure.

PyTorch equivalent

torch.nn.Conv2d(in, out, kernel_size, groups=in) gives you a depthwise convolution when groups == in_channels. That single argument is how PyTorch generalises the depthwise idea. Following it with a 1 by 1 Conv2d reproduces MobileNet's separable block.

In summary

• Inception runs several kernel sizes in parallel and concatenates them, with 1 by 1 projections to keep the cost down; parameter efficiency, not raw depth, was its innovation.
• Depthwise separable convolutions split the standard convolution into a per-channel spatial pass and a 1 by 1 channel-mixing pass, cutting FLOPs by 8 to 9 times at similar accuracy.
• FLOPs are not latency; measure on the target device, because memory bandwidth, kernel launch overhead and hardware specialisation dominate reality.
• EfficientNet scales depth, width and resolution jointly rather than independently, and provides a family of models that trace out the current Pareto frontier of accuracy against FLOPs.

Next module: data augmentation for images, the training-time technique that made ImageNet-scale networks possible and that the modern pipelines run on the GPU as part of the model itself.