Classify images by learning from examples
See why hand-written rules fail, then classify images by comparing them to labeled examples.
- Explain why recognizing objects with hand-written rules is so hard
- Classify an image with a nearest-neighbor classifier
- Use separate training and test images
Image classification assigns one label to a whole image: cat, dog, car. You do it effortlessly, but try writing the rules: “a cat has pointy ears and whiskers”... seen from which side? In what light? Half hidden behind a sofa? The same cat produces wildly different pixel grids from one photo to the next.
Try it
Why pixels are hard
Each photo below is still “a cat”, yet its pixels look completely different from a typical cat photo. Name the challenge each one shows.
“A cat photographed from directly behind”
“A black cat in a dim room at night”
“Only a cat’s tail sticking out from under a blanket”
“A tabby cat lying on a tabby-patterned rug”
“A cat in harsh sunlight with deep shadows”
“A cat peeking around a doorframe”
Learn from examples instead
Instead of rules, give the computer many labeled examples and let it compare. The simplest learned classifier is nearest neighbor: to label a new image, find the most similar training image and copy its label. Similarity can be the L1 distance - add up the absolute differences between corresponding pixels.
Stanford’s CS231n course notes use exactly this classifier to introduce image classification. It works for tiny, tidy images, but raw pixel distance is easily fooled: a shifted or darker image looks “far away” even though it shows the same thing. That is why modern classifiers learn features instead.
first = [0, 255, 0, 255]
second = [0, 200, 50, 255]
print(sum(abs(a - b) for a, b in zip(first, second)))105
Key takeaways
Viewpoint, lighting, occlusion and clutter make hand-written recognition rules hopeless.
Nearest neighbor labels an image like its most similar training example, using a pixel distance such as L1.
Raw pixel distance is fragile - which is why modern models learn features.
Lesson quiz
6 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: apply computer vision with Python
Use small pixel arrays to explore vision concepts, run your code against sample images, and connect each result to the larger computer vision idea.
Build a nearest-neighbor classifier
The first line is N. Each of the next N lines is a label followed by a JSON list of pixels, like cat [0, 255, 0]. The last line is a JSON list of pixels to classify. Print the label of the training image with the smallest L1 distance (ties: the earlier training image).
- Bright beats dark
- Pattern match
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…