Capstone: teach Neuro to read doodles
Build, train, debug and evaluate a doodle classifier from scratch in numpy - every piece of the track in one project.
- Set up data, a model and a loss, and sanity-check them before training
- Implement backpropagation and a mini-batch training loop for a real classifier
- Evaluate with accuracy and a confusion matrix, and classify brand-new inputs
This is the moment. Neuro’s camera captures 5×5 doodles, and there are four kinds of line to recognize: vertical, horizontal, diagonal () and anti-diagonal (/). Vertical and horizontal lines can appear in any column or row, and the camera is noisy - about 8% of the pixels come out flipped.
You’ll build the whole thing from scratch: a 25 → 16 → 4 network (ReLU, softmax), cross-entropy loss, backprop, mini-batch gradient descent - then evaluate it honestly and let it read doodles it has never seen.
Step 1: look at the data
Before any modeling, look at what you’re modeling. The dataset generator draws each kind of line and flips random pixels. There are 400 training doodles and a separate validation set of 100 that training never sees.
1import numpy as np
2
3CLASSES = ["vertical", "horizontal", "diagonal", "anti-diagonal"]
4
5def draw(label, position):
6 canvas = np.zeros((5, 5))
7 if label == 0:
8 canvas[:, position] = 1
9 elif label == 1:
10 canvas[position, :] = 1
11 elif label == 2:
12 canvas = np.eye(5)
13 else:
14 canvas = np.fliplr(np.eye(5))
15 return canvas.ravel()
16
17def make_doodles(per_class, noise, rng):
18 X, y = [], []
19 for label in range(4):
20 for _ in range(per_class):
21 pixels = draw(label, rng.integers(0, 5))
22 flips = rng.random(25) < noise
23 X.append(np.abs(pixels - flips))
24 y.append(label)
25 order = rng.permutation(len(y))
26 return np.array(X)[order], np.array(y)[order]
27
28rng = np.random.default_rng(7)
29X_train, y_train = make_doodles(100, 0.08, rng)
30X_val, y_val = make_doodles(25, 0.08, rng)
31
32print(X_train.shape, y_train.shape, X_val.shape)
33print("examples per class:", np.bincount(y_train))
34for index in range(3):
35 print(CLASSES[y_train[index]])
36 for row in X_train[index].reshape(5, 5):
37 print("".join("#" if pixel else "." for pixel in row))(400, 25) (400,) (100, 25) examples per class: [100 100 100 100] horizontal ##### ..... ..... ..... ..... horizontal ..... ..... .#... ##### .#... anti-diagonal ...## ...#. ..#.. .#..# #....
Already a lesson: the second horizontal line picked up two stray pixels, and the anti-diagonal has two extras of its own - noise the network must learn to ignore. The classes are balanced (100 each), so chance-level accuracy is 25%: the number to beat.
Step 2: sanity-check the untrained model
With He-initialized weights, the untrained network should be roughly uniform over 4 classes: loss near and accuracy near 25%. If it isn’t, something is broken before training even starts - the check from the debugging lesson. That’s exercise 1.
Step 3: train
The backward pass is the training loop’s heart. For softmax with cross-entropy the logit gradient is “probabilities minus one-hot labels” - in code, copy the probabilities and subtract 1 at each correct class. From there it’s the rules you know: dW = inputᵀ @ upstream, biases sum over the batch, and ReLU passes gradient only where its input was positive. Exercise 2 puts it together with shuffled mini-batches.
1import numpy as np
2
3probabilities = np.array([[0.7, 0.1, 0.1, 0.1],
4 [0.2, 0.2, 0.5, 0.1]])
5labels = np.array([0, 3])
6d_logits = probabilities.copy()
7d_logits[np.arange(2), labels] -= 1 # probabilities - one-hot labels
8print(d_logits)[[-0.3 0.1 0.1 0.1] [ 0.2 0.2 0.5 -0.9]]
Step 4: evaluate honestly
Accuracy is one number; a confusion matrix shows which classes get mixed up. Row = true class, column = predicted class, so the diagonal holds the correct answers. Then the real test: brand-new doodles, typed in as # and . - exercise 3.
Expect the validation accuracy to land in the mid-90s rather than at 100%. With 8% of pixels flipped, a few doodles are genuinely ambiguous - even to a human.
Where to go next
- Train a real model: MNIST or Fashion-MNIST digits with PyTorch, first with an MLP and then a CNN.
- Read the classics: AlexNet, ResNet (“Deep Residual Learning”) and “Attention Is All You Need” are all readable now.
- Build a tiny language model: a character-level transformer trained on a book - every piece is from this track.
- Go deeper on related tracks here: Computer Vision, NLP and Large Language Models build on everything you’ve learned.
Key takeaways
Look at the data first, and know the chance-level baseline.
Check the initial loss (≈ ln K) before training.
Train with shuffled mini-batches while monitoring a held-out validation set.
Evaluate with a confusion matrix and genuinely new inputs, not just one accuracy number.
Lesson quiz
6 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: write Python
Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.
Sanity-check the untrained network
The starter generates the doodles and sets up init_params. Write forward(params, X) returning (z1, h, probabilities) for the 25 → 16 → 4 network (ReLU hidden layer, stable softmax output), and cross_entropy(probabilities, y).
The input is a seed for the weights. Print the untrained model’s validation loss to 3 decimals, ln 4 for comparison, and its validation accuracy to 2 decimals.
- Seed 0
- Seed 1
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Train Neuro’s network
Now make it learn. Write backward(params, X, y, z1, h, probabilities) returning the gradients for W1, b1, W2 and b2, and train(...): each epoch, shuffle with rng.permutation, step through mini-batches, and update every parameter with params[name] -= lr * grads[name].
The input is epochs learning_rate. The starter reports validation loss and accuracy every 5 epochs.
- 20 epochs, lr 0.5
- A tiny learning rate
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Read brand-new doodles
The starter trains the finished network (20 epochs, learning rate 0.5). Finish the job:
- Print the confusion matrix on the validation set: 4 rows (true class) of 4 counts (predicted class), separated by spaces.
- The input contains new doodles: 5 lines of
#and.each, separated by blank lines. For each, print the predicted class and its probability to 2 decimals:doodle 1: vertical (0.99).
- Four clean doodles
- Messy doodles
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…