What Is Machine Learning? How Computers Learn From Examples

How does your bank know a payment might be fraud before you do? How does a music app guess which song you'll want next? How does a real estate website estimate what a house is worth?

Nobody sat down and wrote rules for every possible payment, song or house. Instead, these computers learned from examples. That's machine learning, and it's the engine behind most of the "smart" technology you use every day.

This article explains machine learning in plain language, with pictures and simple examples. No math or programming needed, just curiosity.

The big idea: learning from examples

Traditional software follows instructions a person wrote: "If this, then do that." That works well when the rules are clear, like calculating tax or sorting a list.

But some problems are too messy for rules. What exactly makes a house worth $420,000 instead of $380,000? The size, the number of rooms, the neighborhood, the condition, the view... and the way they combine is complicated. Writing all of that out by hand would be nearly impossible.

Machine learning takes a different approach. You give the computer lots of examples where the answer is already known, and it figures out the patterns itself:

There are three key words here:

• Data (or examples): the past cases the computer learns from.
• Training: the process of finding patterns in that data.
• Model: what the computer ends up with, a kind of recipe that turns new information into a prediction.

The term "machine learning" goes back to 1959, when an IBM researcher named Arthur Samuel used it to describe a checkers program that got better by playing games against itself.

The simplest model: drawing a line

Let's see training in action with the simplest model there is. Imagine you run a pizza shop and want to price a new, bigger pizza. Here are your current prices (these are made up for the example):

Training here means finding the line that sits as close as possible to all the dots. The computer tries different angles and heights and keeps the one with the smallest total error. (This is a real calculation, the same one spreadsheets use when you add a "trendline".)

Once it has the line, the model can make predictions for sizes it has never seen, like an 18-inch pizza. That's the whole point of machine learning: learn from the past, predict the new.

Real models work with many more inputs than one (size, toppings, location, day of the week...) and much more complicated shapes than a straight line. But the idea is exactly the same.

Three ways machines learn

Machine learning comes in three main flavors, depending on what kind of examples you have:

1. Supervised learning: learning with the answers

The examples come with the right answers, like a student practicing with an answer key. "This email is spam. This one isn't." "This house sold for $350,000." It's by far the most common type, and it does two main kinds of jobs:

• Classification: picking a category. Spam or not spam? Cat or dog? Fraud or normal?
• Regression: predicting a number. What price? How many minutes? How much rain?

2. Unsupervised learning: finding patterns on its own

Here there are no answers at all, just data. The computer's job is to find structure by itself, such as groups of similar things. A shop might have no idea how its customers naturally split up, so it lets the computer look:

Nobody told the computer there were three kinds of customers. It found the groups, and then people looked at them and gave them names. (This is a real result from a popular method called k-means clustering, run on made-up data.) The same idea helps spot unusual activity, like a credit card suddenly being used in a strange way.

3. Reinforcement learning: learning by trial and reward

Here the computer learns by doing. It tries actions, gets a reward when things go well and a penalty when they don't, and gradually figures out what works. It's how you might train a dog with treats.

Reinforcement learning taught computers to play games like Go and chess at superhuman levels, helps robots learn to walk and grab things, and is one of the steps used to make AI chatbots more helpful, by rewarding the answers people prefer.

The biggest trap: memorizing instead of learning

Imagine a student who memorizes the answers to every practice test but never understands the subject. They ace the practice tests and then fail the real exam, because the questions are different.

Machine learning models can do exactly the same thing. It's called overfitting, and it's one of the most important ideas in the whole field. Here are three models learning from the same 12 example dots:

• Too simple: a straight line can't bend, so it misses the real wave-shaped pattern. This is called underfitting.
• Just right: a gently curved model follows the real trend and ignores the random noise.
• Too complicated: a model with lots of freedom bends itself to pass close to every single dot, noise and all. It "memorized" the examples, and its guesses in between are wild.

Here's what that looks like in numbers. We measured each model's average error on the 12 examples it learned from, and on 100 new dots it never saw:

The blue line (practice tests) keeps getting better as the model gets more complicated. The orange line (the real exam) gets better at first, then much worse. Doing well on the examples you trained on proves very little.

How do you know if a model is good?

The obvious measure is accuracy: what percentage of the time is it right? But accuracy can be badly misleading, especially when you're looking for something rare.

A "model" that simply says "not fraud" every single time would be right 999 times out of 1,000. That's 99.9% accuracy, and it's completely useless, because it never catches any fraud.

So people who build models also ask questions like:

• Of the real fraud cases, how many did it catch? (Missing fraud is costly.)
• Of the payments it flagged, how many were really fraud? (Too many false alarms, and customers get annoyed when their cards are blocked.)

Which mistakes matter more depends on the job. A medical screening test should rarely miss a disease, even if that means some false alarms. A spam filter should rarely throw away a real email, even if a bit of spam gets through.

Garbage in, garbage out

A model can only be as good as the examples it learned from (see What Is Data Labeling? for how those examples get their answers). Some common problems:

• Too little data: a few dozen examples rarely capture the real pattern.
• Wrong or messy data: typos, mislabeled examples and missing values teach the wrong lessons.
• Unrepresentative data: a model trained only on summer sales will be confused in winter. A face recognition system trained mostly on one group of people works worse for others.
• Bias from the past: if past decisions were unfair, like who got approved for a loan, a model trained on them can learn and repeat that unfairness.
• The world changes: people's habits, prices and fraud tricks shift over time, so models need checking and retraining.

Machine learning all around you

A small glossary

• Machine learning: computers learning patterns from examples instead of following hand-written rules.
• Data: the examples a model learns from.
• Feature: one piece of information about an example, like a house's size or number of rooms.
• Label: the right answer attached to an example, like "spam" or a sale price.
• Training: the process of finding patterns in the data.
• Model: the result of training, which turns new inputs into predictions.
• Prediction: the model's answer for a new case.
• Supervised learning: learning from examples that include the right answers.
• Unsupervised learning: finding structure, like groups, in data without answers.
• Reinforcement learning: learning by trial and error, guided by rewards.
• Classification: predicting a category. Regression: predicting a number.
• Overfitting: memorizing the training examples instead of learning the general pattern.
• Test data: examples kept aside to check how well a model works on things it hasn't seen.
• Bias: unfair or lopsided patterns a model picks up from its data.

Wrapping up

Machine learning is a simple idea with huge consequences: instead of telling a computer exactly what to do, you show it lots of examples and let it find the patterns. Supervised learning learns from examples with answers, unsupervised learning finds groups on its own, and reinforcement learning learns by trial and reward.

The art is in the details: having good, fair data, avoiding models that memorize instead of learn, testing on examples the model has never seen, and measuring the mistakes that actually matter. Get those right, and machine learning can spot fraud, recommend a great song, or estimate a price in a split second. And now you know what's going on behind the scenes when it does.