Module 10 — Project: detection on a custom crossroads dataset
Modules 1 to 9 built every piece of the crossroads pipeline separately. This module puts them together. The deliverable is a fine-tuned YOLOv8 detector on a custom crossroads dataset, evaluated with COCO metrics, whose errors we understand by class and by object size, and which we export for deployment.
The starting point
The dataset is 3 000 crossroads frames captured over three weeks from a fixed camera at a suburban intersection. Two classes: car, pedestrian. Annotations exist in COCO format, produced by two annotators using CVAT following the guide from module 9. Their median agreement IoU is 0.89, so the label ceiling is not a bottleneck.
The split is video-aware: 20 continuous days for train (2 200 frames), 5 for validation (500), 3 for test (300). No frame from a training day appears in validation or test. This alone is the difference between a paper report and a defensible number.
Converting COCO to YOLO format
Ultralytics YOLO consumes YOLO-format labels. The COCO-to-YOLO conversion is short but must be done once and versioned:
import json
from pathlib import Path
def coco_to_yolo(coco_json, images_root, labels_root):
with open(coco_json) as f:
coco = json.load(f)
id_to_size = {img["id"]: (img["width"], img["height"], img["file_name"]) for img in coco["images"]}
cat_to_idx = {c["id"]: i for i, c in enumerate(coco["categories"])}
lines = {}
for a in coco["annotations"]:
w_img, h_img, name = id_to_size[a["image_id"]]
x, y, w, h = a["bbox"]
cx = (x + w / 2) / w_img
cy = (y + h / 2) / h_img
wn = w / w_img
hn = h / h_img
cls = cat_to_idx[a["category_id"]]
lines.setdefault(name, []).append(f"{cls} {cx:.6f} {cy:.6f} {wn:.6f} {hn:.6f}")
Path(labels_root).mkdir(parents=True, exist_ok=True)
for name, ls in lines.items():
(Path(labels_root) / (Path(name).stem + ".txt")).write_text("\n".join(ls))
The YOLO layout expects images/train, images/val, labels/train, labels/val in parallel folders. A one-file data.yaml describes the classes and paths:
path: /data/crossroads
train: images/train
val: images/val
test: images/test
names:
0: car
1: pedestrian
Fine-tuning YOLOv8
Ultralytics handles training in a single call. The choices worth thinking about: model size, image size, epochs, and the augmentation config.
from ultralytics import YOLO
model = YOLO("yolov8s.pt") # small: 11 M params, good starting point
results = model.train(
data="data.yaml",
epochs=100,
imgsz=960, # larger than default 640, distant pedestrians are small
batch=16,
optimizer="AdamW",
lr0=1e-3,
patience=15,
close_mosaic=10, # disable mosaic in the last 10 epochs (module 9)
hsv_h=0.015, hsv_s=0.5, hsv_v=0.3, # colour jitter kept modest, camera is fixed
flipud=0.0, fliplr=0.5, # never flip vertically (module 9)
mixup=0.0, # detector project, skip mixup
project="runs/crossroads",
name="yolov8s-960",
)
Two choices deserve a comment. Image size 960 trades throughput for the ability to see distant pedestrians whose bounding boxes are under 20 pixels wide; from module 5, small-object AP dominates our downstream count quality. AdamW at 1e-3 with warmup is more stable than SGD for fine-tuning from a pretrained checkpoint on a small dataset.
Evaluating with COCO metrics
Ultralytics prints its own mAP during training, but the number of record is the one from pycocotools, which module 5 taught us to read:
model = YOLO("runs/crossroads/yolov8s-960/weights/best.pt")
# predict on the test set, save COCO-format json
model.val(data="data.yaml", split="test", save_json=True, project="runs/crossroads/eval")
# then feed predictions.json into pycocotools as in module 5
from pycocotools.coco import COCO
from pycocotools.cocoeval import COCOeval
gt = COCO("annotations/test.json")
dt = gt.loadRes("runs/crossroads/eval/predictions.json")
ev = COCOeval(gt, dt, iouType="bbox")
ev.evaluate(); ev.accumulate(); ev.summarize()
On this dataset a reasonable outcome is:
AP @[0.5:0.95] all = 0.463
AP @[0.5:0.95] car = 0.612
AP @[0.5:0.95] pedestrian= 0.314
AP @[0.5:0.95] small = 0.238
AP @[0.5:0.95] medium = 0.502
AP @[0.5:0.95] large = 0.647
The interesting signal is not the headline 0.463 but the two spreads: car scores twice as high as pedestrian, and small twice as high as large has to be inverted — we scored twice as low on small as on large. Both point to the same underlying case: distant pedestrians, which are both small and rare.
Error analysis by class and by size
Running the COCO evaluator is cheap; running it sliced is where the diagnosis comes from.
def summarize_slice(ev, class_id=None, area_range=None):
p = ev.params
p.catIds = [class_id] if class_id is not None else p.catIds
if area_range is not None:
p.areaRng = [area_range]
p.areaRngLbl = ["slice"]
ev.evaluate(); ev.accumulate(); ev.summarize()
Slice by class, then by size, then by class and size together. On this project, the four-cell table {car, pedestrian} x {small, large} reveals that pedestrian-small AP is 0.11 while car-small AP is 0.44. The model's problem is not "small objects", it is "distant pedestrians specifically", which suggests either scraping more of them (module 9) or fine-tuning image size higher yet.
Confusion matrix on detections answers a different question: which classes get mistaken for which. matplotlib's imshow on a matrix of TP-normalised counts is enough. On our crossroads, pedestrians pushing bicycles get labelled car on 4 % of frames, which pinpoints a class-definition ambiguity worth escalating back to the annotation guide.
Do not report only 0.463. Report per-class, and the two lowest. If pedestrian AP is 0.31 and the whole point of the system is pedestrian safety, the model is not ready even if the headline looks respectable.
Exporting for deployment
The trained checkpoint is 22 MB of PyTorch state. Deployment usually wants something smaller and faster.
model.export(format="onnx", imgsz=960, half=True, simplify=True)
# → runs/crossroads/yolov8s-960/weights/best.onnx
ONNX is a portable graph any modern inference runtime consumes: ONNXRuntime, TensorRT, OpenVINO, TFLite through onnx-tf. Two options matter: half=True uses float16 weights, halving the file and speeding inference on GPUs with tensor cores; simplify=True folds constant operations and removes dead ones.
On an NVIDIA RTX 3060 the ONNX+TensorRT version of this model runs at 4 ms per frame at 960 pixels, comfortably above the 40 ms budget for 25 fps. On a Jetson Orin Nano the same model runs at 22 ms, still fits, and is the deployment target for edge cameras.
Ultralytics exports the raw predictions; NMS and class filtering happen in Python around the ONNX call. For deployment robustness, bake NMS into the graph with export(nms=True) or wrap it in a preprocessing script that ships with the model. A model that answers differently in production than in eval, because postprocessing lives in a different codebase, is the single most common integration bug at deployment time.
Wiring it back into the pipeline
The last hop is trivial in code and decisive in behaviour: feed the exported model into the tracker from module 8, and count. Ultralytics track() accepts an ONNX or PyTorch model transparently.
from ultralytics import YOLO
model = YOLO("best.onnx", task="detect")
results = model.track(source="live_stream.rtsp", tracker="bytetrack.yaml", persist=True, conf=0.4)
The pipeline the course promised — detect, track, count — now runs end to end on a real camera, evaluated on a labelled test set, with an accuracy number every downstream stakeholder can defend.
In summary
- The video-aware split (whole days into train, val or test) is what makes the reported mAP defensible; without it, near-duplicate frames inflate scores by tens of points.
- YOLOv8-s at 960 pixels, AdamW at , mosaic closed in the last 10 epochs is a reasonable default; the levers to think about are model size, image size and class-aware augmentation.
- Error analysis by class and by object size, not the headline mAP, tells you where the model actually fails; on this crossroads, distant pedestrians dominate the error budget.
- ONNX export with
half=Trueandsimplify=True, then wired back into the tracker from module 8, gives a real-time pipeline whose numbers a downstream team can trust.
Next module: the recap and the 40-question final exam that closes the course.