What Is a Transformer in AI? The Idea Behind ChatGPT, Explained Simply

ChatGPT, Claude, Gemini, the AI that translates your messages, the one that writes captions for videos: almost all of today's headline-grabbing AI has one thing in common. Under the hood, they're all built on the same design, called a transformer.

It's even in the name: the "T" in ChatGPT stands for transformer (Generative Pre-trained Transformer). And no, it has nothing to do with robots that turn into cars.

This article explains what a transformer is in plain language: the problem it solved, its big idea (called attention), what happens inside one, and why it took over AI in just a few years.

The problem: reading one word at a time

Language is tricky because meaning depends on context. "Bank" means something different in "river bank" and "bank account". To understand a word, you often need to look at other words, sometimes far away in the sentence.

Before 2017, most AI read text the way you might read through a keyhole: one word at a time, left to right, carrying a summary of what came before. These models (called recurrent neural networks) had two big problems:

• They forgot. By the end of a long sentence, the early words had faded. In "The cat that chased the mouse was hungry", the model might lose track of who was hungry.
• They were slow. Each word had to wait for the previous one, so the work couldn't be split up and done in parallel. That made it hard to train them on really large amounts of text.

The big idea: attention

In 2017, eight researchers at Google published a paper with a now-famous title: "Attention Is All You Need". They introduced the transformer, originally to translate between languages, and its key ingredient was attention.

Attention is a simple idea: when the AI processes a word, it looks at every other word in the text and decides which ones matter most for understanding it. Then it blends in information from those words.

Here's the classic example. What does "it" mean in these two sentences?

You instantly know that a tired "it" is the animal, and a wide "it" is the street. Changing one word at the end flips the meaning. Attention lets a transformer make the same connection: when it processes "it", it gives the most weight to "animal" in one sentence and to "street" in the other.

Crucially, attention works on all words at once. Every word looks at every other word at the same time, no matter how far apart they are:

This solved both old problems at once:

• Nothing gets forgotten: a word at the end can look directly at a word at the beginning.
• It's fast: all those comparisons can happen in parallel on graphics chips (GPUs), so transformers can be trained on enormous amounts of text.

What happens inside a transformer

Here's the journey of a piece of text through a transformer, simplified:

1. Text in. For example: "The sky is".
2. Tokens become numbers. The text is split into tokens (words or pieces of words), and each one becomes a long list of numbers that captures its meaning. The model also adds information about each token's position, since attention on its own doesn't know word order.
3. Attention. Every token gathers information from the others, as described above.
4. Thinking step. Each token's numbers are updated with what it learned, by a small neural network.
5. Repeat. Steps 3 and 4 form one layer, and large models stack dozens of layers, often around a hundred. Early layers pick up simple things like grammar; later layers capture more abstract meaning.
6. Next word out. For a chatbot, the final step produces a list of likely next words. The model picks one, adds it to the text, and runs again to produce the next word, and the next.

Why transformers took over

Because transformers learn so efficiently from huge amounts of data, researchers could make them much, much bigger, and bigger turned out to mean better:

Parameters are the adjustable numbers inside a model that get tuned during training. In five years, the largest models grew by about 10,000 times. (Many of today's top models don't publish their size, so the newest giants aren't on this chart.)

They've also learned to read much longer texts at once. The amount of text a model can consider at one time is called its context window:

From a few pages to thousands of pages in about five years. That's why you can now give a chatbot a whole report, contract or book and ask questions about it.

The "Attention Is All You Need" paper has become one of the most cited papers in modern computer science, and its ideas spread far beyond chatbots.

Not just for words

Anything that can be split into pieces can be fed to a transformer:

• Images: Vision Transformers (2020) cut a picture into small square patches and treat them like words, letting each patch "attend" to the others.
• Speech: speech recognition systems like OpenAI's Whisper use transformers to turn audio into text.
• Science: DeepMind's AlphaFold 2, which predicts the 3D shapes of proteins, relies heavily on attention.
• Code, music, video: coding assistants, music generators and video generators are commonly built on transformers too.

The limits of transformers

Transformers are powerful, but they aren't perfect:

• Long texts are expensive. Remember that every word looks at every other word. Double the length of the text, and the number of comparisons goes up about four times. That's why huge context windows require clever engineering and a lot of computing power.
• They're hungry. Training a large transformer takes enormous amounts of data, electricity and money.
• They don't truly "understand" like people. Attention finds patterns in how words relate. That's remarkably powerful, but transformers can still make confident mistakes (called hallucinations).
• They're hard to explain. With billions of numbers and dozens of layers, it's difficult to say exactly why a transformer produced a particular answer. Researchers are working on ways to look inside.

Researchers are also exploring alternatives and add-ons that handle long texts more cheaply, but as of today, transformers remain the foundation of most cutting-edge AI.

A small glossary

• Transformer: a neural network design, introduced in 2017, that processes all words at once using attention.
• Attention: a way for each word to look at all the other words and decide which ones matter most for understanding it.
• Attention head: one of several attention "spotlights" working in parallel, each focusing on different relationships.
• Token: a word or piece of a word, the unit a transformer reads and writes.
• Position information: extra numbers that tell the model where each token sits in the text.
• Layer: one round of attention plus a thinking step. Big models stack dozens.
• Parameters: the adjustable numbers inside a model, tuned during training.
• Context window: how much text a model can consider at once.
• GPT: Generative Pre-trained Transformer, the family of models behind ChatGPT.
• Recurrent neural network (RNN): an older design that read text one word at a time.

Wrapping up

A transformer is the design behind nearly all of today's leading AI. Its big idea, attention, lets every word look at every other word at once and decide what matters, just like you instantly know whether "it" means the animal or the street. That made AI both smarter about context and much faster to train, which let researchers scale models up dramatically.

So the next time you chat with an AI, translate a message or get an automatic caption, you'll know what's working behind the scenes: layer after layer of words paying attention to each other.