Module 5 — Detection metrics: IoU and mean average precision

Modules 2 and 3 built detectors, module 4 turned their raw outputs into a clean detection list. This module answers the question every stakeholder eventually asks: how good is it?. On the crossroads we cannot ship "seems fine on the last five frames". We need a number that survives cross-team scrutiny, and that number is mAP on a held-out set, read from the same COCO report every serious vision paper uses.

IoU: the atomic notion

Every detection metric ultimately rests on intersection over union. Given a predicted box and a ground-truth box, IoU measures how much they agree:

$$\text{IoU} = \frac{|A \cap B|}{|A \cup B|}$$

where $A$ and $B$ are the two boxes seen as pixel sets. IoU is 0 for disjoint boxes, 1 for identical ones. A useful piece of intuition: IoU is not linear. Shifting a well-aligned box by a quarter of its width does not drop IoU by a quarter; it drops it much faster.

By hand:

def iou(a, b):
    # a, b as (x1, y1, x2, y2)
    x1 = max(a[0], b[0])
    y1 = max(a[1], b[1])
    x2 = min(a[2], b[2])
    y2 = min(a[3], b[3])
    inter = max(0, x2 - x1) * max(0, y2 - y1)
    area_a = (a[2] - a[0]) * (a[3] - a[1])
    area_b = (b[2] - b[0]) * (b[3] - b[1])
    return inter / (area_a + area_b - inter)

print(iou((10, 10, 50, 50), (30, 30, 70, 70)))   # 0.1428...

That single value is what every downstream metric aggregates.

True positive, false positive, false negative

A detection is a true positive when its IoU with a ground-truth box of the same class exceeds a threshold, and no better-scoring detection has already matched that ground-truth. A detection that fails either condition is a false positive. A ground-truth box that no detection matched is a false negative.

The matching order matters. The standard rule is:

1. Sort detections by score, descending.
2. For each detection, find the highest-IoU ground-truth of the same class that is not yet matched.
3. If that IoU passes the threshold, mark the detection TP and the ground-truth as taken. Otherwise mark the detection FP.
4. Any ground-truth still untaken at the end is a FN.

This is a bipartite matching, greedy by score. It is deterministic and reproducible, which is exactly why every framework converges on it.

The precision-recall curve

At any confidence threshold $t$, we can compute:

$$\text{precision} = \frac{TP}{TP + FP}, \qquad \text{recall} = \frac{TP}{TP + FN}$$

Sweeping $t$ from high to low traces a precision-recall curve for one class. Each new detection added (as we lower $t$) either becomes a TP (recall rises, precision may hold or dip) or a FP (recall stays, precision drops). The curve typically starts high-precision, low-recall on the left and slides towards low-precision, higher-recall on the right.

Average precision for that class is the area under this curve, computed with a specific interpolation. COCO uses a 101-point interpolation:

$$\text{AP} = \frac{1}{101} \sum_{r \in \{0, 0.01, \dots, 1.0\}} p_{\text{interp}}(r)$$

where $p_{\text{interp}}(r)$ is the maximum precision at any recall greater than or equal to $r$. The interpolation matters because it smooths the "zigzag" of the raw curve into a monotone envelope.

From AP to mAP

Mean average precision averages AP across classes. Two flavours dominate:

• mAP@0.5 (Pascal VOC style): AP computed with a single IoU threshold of 0.5, then averaged across classes. Permissive; a box that half-overlaps the true one still counts.
• mAP@[0.5:0.95] (COCO style, "the primary metric"): AP is computed at ten IoU thresholds from 0.5 to 0.95 in steps of 0.05, then averaged across thresholds and classes. Strict; the model has to localise as well as classify.

The COCO report also reports AP by object size: AP_S for small objects (area < 32² pixels), AP_M for medium (32² to 96²), AP_L for large. This split is decisive on the crossroads, where distant pedestrians are small objects and buses are large; a global mAP hides which side of that spectrum the model fails.

Reading a COCO report

pycocotools evaluates predictions against ground truth in one call.

from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
import json

gt = COCO("annotations/instances_val.json")

# predictions in COCO format: image_id, category_id, bbox=[x, y, w, h], score
with open("predictions.json") as f:
    preds = json.load(f)

dt = gt.loadRes(preds)
evaluator = COCOeval(gt, dt, iouType="bbox")
evaluator.evaluate()
evaluator.accumulate()
evaluator.summarize()

The output is a twelve-line table:

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.421
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.634
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.448
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.213
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.467
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.598
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.312
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.503
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.550
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.330
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.596
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.703

Read it in three passes. First line is the headline number quoted in papers. The AP_S / AP_M / AP_L split tells you where the model struggles. The AR at maxDets=1 versus maxDets=100 tells you whether it fails by ranking (top detection is wrong) or by recall (correct detections exist but low-scored).

A single mAP number without the breakdown is nearly useless

Two models with the same 0.42 mAP can differ by 30 points on small objects. On the crossroads that difference is whether distant pedestrians get counted. Always report the six main lines together, not just the top one, whenever anyone asks how the model performs.

Common numeric mistakes

Three errors show up in almost every first evaluation:

1. Ground-truth in wrong format: COCO expects [x, y, w, h], torchvision expects [x1, y1, x2, y2]. A silent format mismatch gives an mAP near zero without any exception being raised.
2. Confidence threshold too high at eval time: as module 4 explained, mAP wants the low-confidence tail. Setting conf=0.5 at prediction time can halve the reported mAP.
3. Class id mismatch: your dataset uses ids 1 and 2 for car and pedestrian, but the exported predictions still carry COCO's ids 3 and 1. The report then shows 0.0 AP everywhere and it looks catastrophic; it is a mapping bug.

Track mAP over training with a fixed evaluation call

Wire the same COCOeval.summarize call into your training loop, once per epoch, on a fixed validation subset. The learning curve of mAP@[0.5:0.95] is the single most informative training-time signal: it flattens out well before loss does, and that flattening is your real early-stopping signal.

In summary

• IoU is the atomic notion; it is not linear in translation, and a modest shift can drop it sharply.
• A detection is a true positive when a greedy score-sorted matching pairs it with a same-class ground-truth above the IoU threshold; anything else is a false positive or a false negative.
• AP is the area under a class's precision-recall curve with a specific interpolation; mAP@[0.5:0.95] averages AP across ten IoU thresholds and all classes, and is the primary COCO metric.
• Never quote mAP without the AP_S / AP_M / AP_L breakdown: the crossroads passes or fails on distant pedestrians, and the global number hides where the failures actually are.

Next module: semantic segmentation, where the atomic unit shifts from a box to a pixel.