What Is Deep Learning? How Computers Learn to See, Hear and Talk

How does your phone know that a photo has your dog in it? How does a voice assistant turn your mumbled "set a timer for ten minutes" into an actual timer? How does a chatbot write a poem about your cat?

The answer to all three is the same: deep learning. It's the technology behind almost every big leap in AI over the last fifteen years, from recognizing faces to translating languages to chatting with you in plain English.

The name sounds intimidating, but the core idea is surprisingly simple. This article explains it in everyday language, step by step, with no math beyond adding up a few numbers. You'll even get to watch a real (tiny) deep learning system learn.

Where deep learning fits

Artificial intelligence is the big umbrella: computers doing tasks that usually need human intelligence. Inside it is machine learning (explained here), where computers learn from examples instead of following rules written by people. And inside machine learning is deep learning: learning with large neural networks.

So what's a neural network? To understand that, we start with its smallest piece.

The building block: an artificial neuron

Your brain has around 86 billion nerve cells, called neurons, connected to each other. Deep learning borrows that idea very loosely. An artificial neuron is not a real cell, just a tiny bit of math that makes one small decision.

Here's how it works, using a decision you might make on a summer morning:

1. The neuron receives some inputs: is it sunny, is it warm, are friends coming, is it a long drive?
2. Each input has a weight, a number that says how much it matters. Sunshine matters a lot (+3). A long drive counts against going (−4).
3. The neuron multiplies each input by its weight and adds everything up.
4. If the total is high enough, the neuron "fires": go to the beach!

That's it. One neuron is just a weighted vote. The magic is in the weights: change them, and the same neuron makes different decisions. Learning, for a neural network, means finding good weights.

Many neurons, many layers

One neuron can only make a simple decision. But connect thousands or millions of them together, arranged in layers, and you get a neural network:

• The input layer takes in the raw information, like the colors of each tiny dot (pixel) in a photo.
• The hidden layers in the middle pass signals along, each neuron combining the outputs of the layer before it.
• The output layer gives the answer, such as how likely the photo shows a cat or a dog.

Every line in that picture is a weight. Real networks have a lot of them: a well-known chatbot model from 2020, GPT-3, had 175 billion.

And here's where the name comes from: a network with many hidden layers is called deep. That's all "deep" means in deep learning. Not deep thoughts, just deep stacks.

Why layers are so powerful

Something remarkable happens when you stack layers. Each layer learns to notice something a little more complex than the layer before it. In a network that recognizes photos, researchers have looked inside and found a pattern like this:

The early layers notice simple edges. The next ones combine edges into shapes and textures. Deeper layers combine shapes into parts, like eyes and ears. The final layers put the parts together into a whole object: a cat!

The amazing part: nobody programs these steps. No one tells the network "look for ears". It discovers that ears are useful clues all by itself, simply by being trained on lots of labeled photos.

How a network learns

A brand-new network has random weights, so its answers are pure guesswork. Training it is a loop:

1. Guess. Show the network an example, like a photo, and let it make a prediction.
2. Compare. Check the prediction against the right answer.
3. Measure. Calculate how wrong it was. This is called the error (or "loss").
4. Adjust. Nudge every weight a tiny bit in the direction that would have made the error smaller.

Then do it again. And again. Millions of times, over millions of examples.

The clever part is step 4. With millions of weights, how do you know which way to turn each one? A method called backpropagation works backward from the error, through every layer, and figures out how much each weight was to blame. Think of a giant sound mixing board with millions of knobs, and a helper who tells you exactly which way to turn each knob to make the music sound a little better.

Watch a real network learn

Talk is cheap, so here's a real one. We trained a tiny neural network (one small hidden layer of 8 neurons) on a simple task: look at a dot's position and decide whether it's inside the circle (blue) or outside it (orange). The shading shows what the network guesses for every spot.

Before training, the network's guesses are random, and it gets about half right, no better than flipping a coin. After 50 rounds of guess-compare-adjust, a rough shape appears. After 400 rounds, it has discovered the circle on its own, and gets 95% right. Nobody told it the word "circle". It found the pattern in the examples.

Here's its progress round by round:

This shape, slow start, rapid improvement, then leveling off, shows up in giant AI systems too. They just use billions of weights and examples instead of 8 neurons and 240 dots.

Why deep learning took off when it did

Neural networks aren't new. The first artificial neuron that could learn, the perceptron, was built by Frank Rosenblatt in 1958. For decades, though, neural networks were mostly a curiosity. Three things changed around 2010:

1. Data. The internet created enormous collections of photos, text and audio to learn from.
2. Speed. Graphics chips (GPUs), built for video games, turned out to be perfect for the huge number of simple calculations neural networks need.
3. Better techniques. Researchers found ways to train much deeper networks without them getting stuck.

The turning point came in 2012 at the ImageNet challenge, a yearly competition to recognize objects in over a million photos. A deep network called AlexNet made dramatically fewer mistakes than everything before it. Within three years, deep networks beat a human's score:

The deep learning pioneers Geoffrey Hinton, Yann LeCun and Yoshua Bengio received the 2018 Turing Award, often called the Nobel Prize of computing, and Hinton shared the actual 2024 Nobel Prize in Physics with John Hopfield for their foundational work on neural networks.

Where you meet deep learning every day

Today, deep learning powers much of the technology around you:

• Photos: face unlock, searching your photo library for "beach", blurring the background in video calls.
• Voice: voice assistants, automatic captions, dictation.
• Language: translation apps and AI chatbots. The chatbots are built on a type of deep network called a transformer, introduced by Google researchers in 2017. (Read more in What Is an LLM?)
• Recommendations: what shows, songs and posts you're offered next.
• Health: helping doctors spot signs of disease in X-rays, scans and eye images.
• Science: predicting the 3D shapes of proteins, which helps research into medicines.
• Creative tools: generating images, music and video from a text description.

The limits of deep learning

Deep learning is powerful, but it has real weaknesses worth knowing about:

That point about being fooled is real and a little spooky. In a famous 2014 experiment, researchers added a carefully designed pattern of noise to a photo of a panda. To a human, the picture looked exactly the same. The network, which had correctly said "panda", now said "gibbon" with over 99% confidence. Deep networks recognize patterns very differently from how we do.

A small glossary

• Deep learning: machine learning using neural networks with many layers.
• Artificial neuron: a tiny unit that multiplies its inputs by weights, adds them up and passes on a signal.
• Weight: a number that says how much one input matters. Learning means adjusting weights.
• Neural network: many artificial neurons connected in layers.
• Layer: a group of neurons at the same stage. Input, hidden and output layers.
• Hidden layer: any layer between the input and the output.
• Training: teaching a network by showing it examples and adjusting its weights.
• Error (or loss): a measure of how wrong the network's guess was.
• Backpropagation: the method that works backward from the error to figure out how to adjust each weight.
• GPU: a graphics chip, originally for video games, that's very good at the calculations deep learning needs.
• Transformer: a type of deep network, introduced in 2017, behind today's AI chatbots.

Wrapping up

Deep learning sounds complicated, but it's built from a very simple piece: a neuron that adds up weighted inputs. Stack millions of them in many layers, show them millions of examples, and keep nudging the weights to make the errors smaller. Out of that simple recipe come systems that can recognize faces, understand speech, translate languages and hold a conversation.

It isn't magic, and it isn't a brain. It's pattern-finding at an enormous scale. Now, the next time your phone finds every photo of your dog in a second, you'll know there's a deep stack of layers behind it, each one noticing something a little more complex than the last.