Neural Networks: The Basics

Neural networks power almost everything people mean when they say "AI" today — image recognition, language models, recommendation engines. The building block behind all of that complexity is surprisingly small: a single artificial neuron.

A single neuron

A neuron takes a vector of inputs, multiplies each one by its own weight, adds them up (that's the dot product from Module 2), adds one more number called the bias, and passes the result through an activation function.

As an equation: $z = w_1x_1 + w_2x_2 + w_3x_3 + b$, then $y = \text{activation}(z)$. This should look familiar — it's multiple linear regression from Module 5, with one extra step (the activation function) tacked on the end.

Why activation functions matter

Without an activation function, stacking neurons together would still just be linear regression no matter how many layers you added — a line combined with a line is still a line. Activation functions introduce non-linearity, which is what lets networks learn curved, complex decision boundaries instead of only straight ones. A common choice is the sigmoid function, which squashes any input into a value between 0 and 1 — convenient for representing "how confident am I this is class A?"

From one neuron to a network

A neural network arranges neurons into layers: an input layer (your raw features), one or more hidden layers (where the actual pattern-finding happens), and an output layer (the prediction). Every neuron in one layer typically connects to every neuron in the next layer, each connection carrying its own weight.

Each hidden layer learns to combine the previous layer's outputs into new, more abstract features. In an image classifier, early layers might detect simple edges, middle layers might detect shapes made of those edges, and later layers might detect entire objects made of those shapes — a progression you'll see again in Module 9.

"Deep learning" simply refers to networks with many hidden layers stacked on top of each other. More layers means the network can represent more complex functions — but it also means more parameters to learn, which is exactly where Module 8's training algorithm comes in.