Object Detection: YOLO, SSD & Faster R-CNN

Go from "is there a cat?" to "draw a box around every cat, dog, and car." You'll measure box overlap with IoU, clean up duplicates with NMS, and know which detector to reach for.

Learn Object Detection: YOLO, SSD & Faster R-CNN in our free AI & Machine Learning course — a beginner-friendly interactive lesson with worked examples, a…

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Imagine handing a friend a busy holiday photo and a marker pen. Classification is asking "is there a dog in this photo?" — one yes/no answer. Object detection is asking your friend to go further: "draw a box around every dog, every person, and every car, and write what each one is." That is the whole job — find the objects and box them.

Now imagine your friend gets over-excited and scribbles five boxes around the same dog. You'd keep the neatest box and cross out the rest. That clean-up step is exactly what Non-Max Suppression does, and the way you decide two boxes are "the same dog" is by measuring their overlap — that's IoU .

A classifier outputs one label for the whole image: "cat". A detector must output a list of objects, and for each one it predicts three things:

A bounding box is the rectangle drawn around an object. The most common format is the two corners: [x1, y1, x2, y2] — the top-left corner (x1, y1) and the bottom-right corner (x2, y2) , all in pixels. (YOLO often uses a centre format [cx, cy, w, h] instead — same rectangle, different numbers.)

How do you score a predicted box against the true box? You use IoU (Intersection over Union) : the area where the two boxes overlap, divided by the total area they cover together.

IoU = overlap_area / (area_a + area_b - overlap_area)

Here is the full, commented version. Read it, then run it — the output is at the bottom.

The two max(0, ...) calls are important: if the boxes don't overlap, x2 - x1 goes negative, and clamping it to 0 keeps the intersection at 0 instead of producing a fake positive area.

Predicting box coordinates from scratch is hard. Instead, many detectors start from anchor boxes : a fixed set of reference rectangles in a range of shapes and sizes (a tall one for people, a wide one for cars, a square one, and so on). The model doesn't invent a box — it picks the closest anchor and nudges its size and position to fit the object.

Think of anchors as templates printed on tracing paper. The model slides the best-matching template over the object and tweaks it slightly, which is far easier than drawing freehand.

There are two broad families of detector, and they trade speed against accuracy:

Predict every box and class in a single pass over the image. "You Only Look Once." Very fast (real-time video, mobile, edge devices), slightly weaker on tiny or crowded objects.

First propose regions that might hold an object, then classify and refine each one. Slower, but typically more accurate, especially on small and overlapping objects.

A good default: reach for YOLO when you need speed or real-time, and Faster R-CNN when accuracy on hard images matters more than frame rate.

In practice you rarely write IoU and NMS by hand — a library does it. This is read-only (it needs pip install ultralytics and an image), but it shows how little code real detection takes. Ultralytics runs NMS for you, so the boxes come back already de-duplicated.

A raw detector fires dozens of overlapping boxes around each object. Non-Maximum Suppression (NMS) tidies them up with a simple rule: sort boxes by confidence, keep the most confident one, then throw away every other box that overlaps it too much (IoU above a threshold, usually 0.5). Repeat until none are left.

Here is NMS written out in plain Python, reusing the IoU function. Run it and watch four boxes become two.

One number summarises a detector's quality: mAP (mean Average Precision) . For each class you measure precision across recall levels to get its Average Precision , then average across all classes. A box counts as correct only if its IoU with the truth clears a threshold.

This is read-only (it needs the library and a dataset), but it shows how you'd measure mAP in practice:

Setting the match threshold too low counts sloppy boxes as correct; too high rejects boxes that are actually fine.

Raw model output has many overlapping boxes. Without NMS you draw five boxes on one object.

If 95% of your training boxes are "car" and 5% are "bicycle", the model learns to ignore bicycles and your per-class mAP for the rare class collapses — even though overall accuracy looks fine.

Tiny objects (distant faces, far-off signs) shrink to a few pixels after downsampling and vanish, or no anchor shape fits them.

Time to fly solo. Using IoU, count how many predicted boxes actually hit the ground-truth object. The starter block gives you the brief and the data — write the logic yourself.

Lesson complete — you can detect and localise objects!

You can describe a detection as class + box + confidence, compute IoU by hand, run NMS to remove duplicates, choose between one-stage and two-stage detectors, and read an mAP score. These are the building blocks behind every modern detector.

🚀 Up next: Semantic Segmentation — go beyond boxes and label every single pixel in the image.

Practice quiz

• Browse all free courses
• Cheat sheets & quizzes
• Coding dictionary