Module 7 — Instance segmentation with Mask R-CNN

Module 6 taught the model to say "these pixels are road, those are pavement". It cannot say "this pixel belongs to car number 3, that one to car number 5". At the crossroads, when two vehicles overlap in the same lane, semantic segmentation returns a single blob labelled "car" and the counting logic is lost. Instance segmentation restores the count.

The task, precisely

Semantic segmentation gives one class per pixel. Instance segmentation gives one object identifier per pixel of a countable class. Two adjacent cars produce two disjoint masks with the same class "car" but different ids. Pixels that belong to no object (road, sky) receive no id — this is the difference with panoptic segmentation, which unifies both.

For our crossroads, instance masks let module 8 track a specific vehicle across frames even when it moves through partial occlusion; a box alone often loses identity when two boxes fuse and split.

Mask R-CNN: one extra head on Faster R-CNN

Mask R-CNN (2017) starts from Faster R-CNN (module 2) and adds a single small head. Everything else — backbone, feature pyramid, region proposal network, box classification and regression heads — stays identical. The new mask head takes each region's features and predicts a binary mask of the object inside the region, at a fixed resolution (typically $28 \times 28$).

Three details make the design work:

1. RoI Align, not RoI pooling. As module 2 mentioned, the double-quantisation of RoI pooling shifts mask boundaries by half a pixel or more. For boxes it is invisible, for masks it collapses accuracy. RoI Align was introduced specifically for Mask R-CNN.
2. Per-class masks, decoupled from classification. The mask head outputs $C$ masks per region (one per class), and only the mask of the predicted class is kept. This lets the mask loss reward geometric accuracy without also having to reason about the class.
3. Small, fixed mask resolution. $28 \times 28$ per region is enough because the mask is later upsampled and refitted into the box coordinates at inference. Training at higher resolution helps only marginally and costs a lot.

Running Mask R-CNN on the crossroads

Torchvision ships a pretrained Mask R-CNN with the same interface as Faster R-CNN:

import torch
import torchvision
from torchvision.models.detection import maskrcnn_resnet50_fpn_v2
from torchvision.io import read_image
from torchvision.transforms.functional import convert_image_dtype

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = maskrcnn_resnet50_fpn_v2(weights="DEFAULT").eval().to(device)

image = convert_image_dtype(read_image("crossroads_0001.jpg"), torch.float32).to(device)

with torch.no_grad():
    outputs = model([image])

boxes = outputs[0]["boxes"].cpu()      # [N, 4]
scores = outputs[0]["scores"].cpu()    # [N]
labels = outputs[0]["labels"].cpu()    # [N] COCO ids
masks = outputs[0]["masks"].cpu()      # [N, 1, H, W] float in [0, 1]

masks[i, 0] is a soft mask; binarise it with a threshold (0.5 by default). Overlaying masks colour-coded by instance id is the fastest way to spot annotation or ordering bugs:

import numpy as np

keep = scores > 0.6
binary_masks = (masks[keep, 0] > 0.5).numpy()
overlay = np.zeros((*image.shape[-2:], 3), dtype=np.uint8)
for i, m in enumerate(binary_masks):
    colour = np.random.randint(0, 255, size=3)
    overlay[m] = colour

Where instance and semantic differ in practice

Two situations force the distinction:

• Two objects of the same class touching or overlapping. On the crossroads, two cars in the same queue, or a pedestrian standing right next to another. Semantic segmentation returns a single connected region and cannot count. Instance segmentation returns two masks with different ids.
• Object counting with per-instance features. If downstream logic needs to compute per-vehicle area, speed or trajectory, it needs an instance id to attach that feature to. Semantic segmentation forces a fragile connected-component pass that fails in exactly the overlap case above.

If neither situation applies to your problem, do not pay the instance-segmentation cost. On a satellite image of rooftops where every roof is spatially isolated, semantic segmentation plus connected components is faster and simpler.

Panoptic segmentation, in one paragraph

Panoptic segmentation unifies semantic and instance. It splits classes into two groups: things (countable objects: car, person, dog) get instance masks with ids; stuff (uncountable surfaces: road, sky, grass) get a single semantic mask. Every pixel receives exactly one label from exactly one group.

Panoptic is the right target when the scene has both — which our crossroads does. In practice the panoptic quality metric (PQ) is still a research topic, and most deployed pipelines run two models (a detector plus a semantic segmenter) and combine their outputs in post-processing. Full panoptic models like Mask2Former exist and are excellent, but they carry their own training complexity; for a first project, module 10 will stay with detection plus semantic segmentation.

Segment Anything as an annotation booster

The Segment Anything Model (SAM, 2023) changed the economics of mask annotation. Given an image and a prompt — a point, a box, or a rough polygon — SAM outputs a high-quality mask. It is class-agnostic: it does not know what the object is, it just isolates it.

The workflow this enables:

# pseudocode; the actual API is in the segment_anything package
from segment_anything import SamPredictor, sam_model_registry

sam = sam_model_registry["vit_h"](checkpoint="sam_vit_h.pth").cuda()
predictor = SamPredictor(sam)
predictor.set_image(image)

# annotator draws a rough box around a car; SAM refines it into a pixel mask
mask, score, _ = predictor.predict(box=np.array([100, 100, 300, 250]))

An annotator who used to draw a polygon for 10 minutes per vehicle now clicks a rough box and accepts SAM's mask in 10 seconds. On a 10 000-image annotation job this is not a productivity gain, it is the difference between the project being feasible or not.

SAM masks are excellent but not correct

SAM refuses to invent classes but happily returns crisp masks around the wrong object — a shadow, a reflection, a road marking. Annotators must still review every mask. The workflow is faster because drawing is replaced by review; skipping review to save even more time reintroduces label noise at scale.

Metrics for masks

Mask R-CNN uses the same COCO evaluator from module 5, with iouType="segm" instead of "bbox". The IoU is now computed on masks rather than boxes:

$$\text{IoU}_\text{mask} = \frac{|M_p \cap M_g|}{|M_p \cup M_g|}$$

Every threshold and interpretation from module 5 carries over. Mask AP is typically 3 to 6 points lower than box AP for the same model, because a badly-shaped mask around a well-placed box still fails the mask IoU threshold.

Add a mask visualisation to the training loop early

Every mistake in mask coordinates, one-hot encoding, or channel order produces a visually obvious overlay long before it affects the metric. Save five overlays every epoch to a directory that TensorBoard reads, and scroll through them; you will catch two thirds of the mask bugs on the first epoch.

In summary

• Instance segmentation attaches an object id to every pixel of countable objects; semantic segmentation only assigns a class.
• Mask R-CNN = Faster R-CNN + a small per-class mask head reading RoI-Aligned features; RoI Align was invented for this task and matters most here.
• Panoptic unifies "things" (instance masks) and "stuff" (semantic masks); the crossroads pipeline typically ships two models plus post-processing rather than a full panoptic model.
• Segment Anything turns polygon drawing into mask review and can make a segmentation annotation job an order of magnitude cheaper, provided the review step is not skipped.

Next module: multi-object tracking, where instance masks acquire a persistent identity across video frames.