The Math Toolkit: Vectors & Matrices

You do not need a math degree to understand AI, but a handful of linear algebra ideas show up constantly — in how data is represented, how a neuron combines its inputs, and how an image is stored. This module covers exactly the ideas you'll need, and no more.

What is a vector?

A vector is just an ordered list of numbers. In AI, a vector is how you represent "one thing" as numbers a computer can work with. A house might be represented as the vector

(1800, 3, 2, 12)

meaning (square feet, bedrooms, bathrooms, age in years). A word can be represented as a vector too (more on that in Module 10). Geometrically, a 2-number vector like

(4, 2)

is an arrow from the origin to the point (4, 2).

The dot product

The single most important operation in this whole guide is the dot product. Given two vectors of the same length, multiply their matching entries together and add up the results:

(a_1, a_2, a_3) \cdot (b_1, b_2, b_3) = a_1b_1 + a_2b_2 + a_3b_3

Geometrically, the dot product tells you how much two vectors point in the same direction. A large positive dot product means they're aligned; a dot product near zero means they're roughly perpendicular (unrelated); a negative dot product means they point in opposite directions. Try it below.

a = (3, 4)

b = (4, 1)

|a| = 5.00

a · b = 3×4 + 4×1 = 16

angle between a and b ≈ 39°

Positive dot product: the vectors point in broadly the same direction.

Key idea: When you see a neuron computing "weighted sum of inputs" in Module 7, that's a dot product between the input vector and the weight vector. When a search engine measures how similar two pieces of text are, it's often a dot product between their vector representations. It's everywhere.

What is a matrix?

A matrix is a grid of numbers — think of it as a stack of vectors, or a table with rows and columns. Matrices show up in two places you'll care about here: an entire dataset is usually stored as a matrix (one row per example, one column per feature), and a digital image is literally a matrix of brightness values (you'll see this directly in Module 9).

Multiplying a vector by a matrix is, under the hood, just a series of dot products — one between the input vector and each row of the matrix. That single fact is why "matrix multiplication" is the operation GPUs are built to do extremely fast, and why it powers nearly all of modern deep learning.