The math toolkit: vectors and matrices

Computers can’t see a house, a song or a sentence. They see numbers. So the first step in almost every AI system is to turn each thing into a vector: an ordered list of numbers.

A house might become $(1800, 3, 2, 12)$: 1800 square feet, 3 bedrooms, 2 bathrooms, 12 years old. Each position always means the same thing, so two houses can be compared number by number. Each number is a feature, and a vector with 4 numbers lives in 4-dimensional space. Real AI vectors often have hundreds or thousands of dimensions; we draw 2-D ones because we can see them.

Vectors add position by position. Drawn as arrows, that means putting them tip to tail:

The most important operation in all of AI is the dot product: multiply matching positions, then add everything up.

$a \cdot b = a_1 b_1 + a_2 b_2 + \dots + a_n b_n$

For $a = (3, 4)$ and $b = (4, 1)$: $3 \times 4 + 4 \times 1 = 16$. The result is a single number that measures how much two vectors point the same way: large and positive when they agree, zero when they’re at right angles, negative when they point in opposite directions.

A matrix is a grid of numbers - a stack of vectors. A spreadsheet of 100 houses with 4 features each is a 100 × 4 matrix; a grayscale photo is a matrix of pixel brightnesses.

Multiplying a matrix by a vector is just one dot product per row:

$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 5 \\ 6 \end{bmatrix} = \begin{bmatrix} 1 \cdot 5 + 2 \cdot 6 \\ 3 \cdot 5 + 4 \cdot 6 \end{bmatrix} = \begin{bmatrix} 17 \\ 39 \end{bmatrix}$

That is how a model scores a whole batch of examples at once: each row is one example, the vector holds the weights, and out come all the scores. Graphics chips (GPUs) do millions of these multiply-and-adds in parallel, which is why they made deep learning practical.