0x70Lesson 8 of 11

Find structure without labels

Run k-means by hand, choose k with the elbow method, and compress features with PCA.

24 min 6-question quiz 2 code exercises

By the end of this lesson you can

Step through the k-means algorithm
Use inertia and the elbow method to choose k
Explain what PCA does and why to interpret clusters cautiously

Clustering groups examples so that similar ones end up together - without any labels. The classic algorithm, k-means, repeats two steps until nothing changes:

Assign each point to its nearest centroid.
Update each centroid to the mean of its assigned points.

Each step can only lower the inertia - the total squared distance from points to their centroids:

\text{inertia} = \sum_{i=1}^{n} \lVert x_i - \mu_{c(i)} \rVert^2

so the algorithm always converges - though to a local best that depends on where the centroids started. In practice you run it several times from different starts and keep the lowest inertia.

Try it

Up to 15 XP

Run k-means by hand

Three groups of customers, plotted by visits per month and average basket size. The three centroids start bunched in the middle. Alternate “Assign points” and “Move centroids” and watch the inertia fall until nothing changes.

Diamonds are centroids. Next step: assign each point to its nearest centroid.

Choosing k, and compressing features

More clusters always lower inertia (with $k = n$ it is zero), so you can’t just minimize it. The elbow method plots inertia against $k$ and picks the point where adding clusters stops helping much. Better still: check whether the clusters are useful for your question.

Principal component analysis (PCA) finds new axes - combinations of the original features - along which the data varies most. Keeping the top two or three lets you plot high-dimensional data or feed a smaller input to another model. Like clustering, PCA depends on feature scaling: standardize first.

elbow.py

inertia_by_k = {1: 245.2, 2: 98.7, 3: 9.8, 4: 7.9, 5: 6.4}
drops = {k: round(inertia_by_k[k - 1] - inertia_by_k[k], 1) for k in range(2, 6)}
print(drops)

Output

{2: 146.5, 3: 88.9, 4: 1.9, 5: 1.5}

Key takeaways

k-means alternates assign and update until assignments stop changing.
Inertia always falls with more clusters; use the elbow and domain sense to choose k.
Scale features first; validate that clusters mean something before naming them.

Lesson quiz

6 questions · pass with 5 correct · up to 50 XP

Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.

Practice: write Python

Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.

Exercise 1

k-means from scratch

+25 XP

Line 1 is a JSON list of initial centroids; each following line is a point x y. Run k-means until assignments stop changing. Print iterations: N, then each final centroid as centroid I: (X, Y) with M points (2 decimals), then inertia: V (3 decimals). Ties in distance go to the lower centroid index. Compare the two tests: same data, different starts.

A good start
A bad start gets stuck

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Exercise 2

Pick k with the elbow

+25 XP

Each line is k inertia, for increasing k. The relative improvement of k is $\frac{I_{k-1} - I_k}{I_{k-1}}$ . Print each k=K improvement P% (whole percent) and finally elbow: k=K - the last k whose improvement is at least 20%.

Customer data

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Questions about this lesson

Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.

Loading posts…