What is deep learning?
Meet layered neural networks, see why depth matters, and find where deep learning shines.
- Explain deep learning as learning layered representations from raw data
- Describe why deep learning took off in the 2010s
- Judge when deep learning is - and isn’t - the right tool
Meet Neuro, the little robot at Doodle Lab. Neuro can’t recognize a single doodle yet. Over this track you’ll build Neuro a brain from scratch - out of nothing but arrays of numbers and a few lines of calculus - and by the capstone it will tell lines from crosses on its own.
Deep learning is machine learning with neural networks that have many layers. Each layer turns its input into a new representation, and the layers are learned together, end to end, from examples:
- In an image network, early layers end up detecting edges, middle layers detect textures and parts (an eye, a wheel), and late layers detect whole objects.
- Nobody programs those detectors. They emerge because they help the network make better predictions.
That’s the big difference from classic machine learning, where people hand-craft the features (“count the red pixels”, “average word length”) and a simpler model learns from them. Deep learning learns the features too.
Why did it take until 2012? Three ingredients finally came together:
- Data - the internet produced labeled datasets with millions of examples (ImageNet has 14 million images).
- Compute - graphics cards (GPUs) turned out to be brilliant at the matrix multiplications networks are made of.
- Tricks - better activations (ReLU), initializations, normalization and regularization made deep networks trainable at all. You’ll learn every one of these in this track.
Why layers? XOR in three neurons
A single neuron draws one straight line through its inputs, so it can’t compute XOR (“exactly one input is on”) - no single line separates the two “on” corners from the two “off” corners. Add a hidden layer and it’s easy. Here are hand-picked weights; press Run:
1import numpy as np
2
3def step(z):
4 return (z > 0).astype(int)
5
6W1 = np.array([[1, 1], [1, 1]]) # two hidden neurons, each sums both inputs
7b1 = np.array([-0.5, -1.5]) # hidden 1 fires on "at least one", hidden 2 on "both"
8W2 = np.array([1, -1]) # output: "at least one" but not "both"
9b2 = -0.5
10
11for x in [(0, 0), (0, 1), (1, 0), (1, 1)]:
12 hidden = step(W1 @ np.array(x) + b1)
13 output = step(W2 @ hidden + b2)
14 print(x, "->", output)(0, 0) -> 0 (0, 1) -> 1 (1, 0) -> 1 (1, 1) -> 0
The hidden layer re-represents the input (“at least one on”, “both on”), and in that new space the answer is a simple straight-line decision. That’s deep learning in miniature: each layer makes the next layer’s job easier. The difference in real networks is that the weights are learned - which is what Unit 2 is all about.
Try it
Deep learning or something simpler?
Deep learning shines with lots of raw, unstructured data like images, audio and text. For small tables of numbers, simpler models usually win. Sort each task.
“Spot cats in millions of photos”
“Predict a house price from 6 columns and 300 sales”
“Turn speech into text”
“Translate between languages”
“Forecast next month’s sales from 3 years of monthly totals”
“Flag fraud in a bank’s table of transaction features”
“Generate an image from a text description”
Key takeaways
Deep learning = neural networks with many layers, trained end to end from examples.
Each layer learns a new representation; together they learn features nobody hand-coded.
Data, GPUs and training tricks made it practical around 2012.
It shines on large unstructured data; for small tables, simpler models often win.
Lesson quiz
7 questions · pass with 5 correct · up to 50 XP
Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.
Practice: write Python
Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.
Wire up XNOR by hand
XNOR is the opposite of XOR: it outputs 1 when both inputs are the same. Using the same two hidden neurons as the XOR example (“at least one” and “both”), choose the output weights W2 and bias b2 so the network computes XNOR.
Each input line is two bits. Print a b -> output for each.
- All four inputs
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Neuro’s first neuron
Neuro’s first job: decide whether a doodle is a tall shape. Each input line has three features of a doodle - height width ink (ink is the fraction of the canvas drawn on). A single neuron computes z = 1.5·height − 1.0·width − 0.5·ink − 0.2 and says tall when z > 0, otherwise not tall.
Print z = 0.85 -> tall with z rounded to two decimals. Use numpy for the weighted sum (weights @ features).
- Three doodles
- A square
Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.
Questions about this lesson
Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.
Loading posts…