- Foundation
- About 15 min
- 5 practice questions
Unsupervised learning: clustering and k-means step by step
Explain how unsupervised learning finds groups in data that has no labels, run k-means by hand on a small example, and tell unsupervised learning apart from supervised and reinforcement learning.
If something looks wrong, please tell us.
By the end of this lesson you can
- Explain what unsupervised learning is and when it is useful.
- Run the assign-and-update steps of k-means on a small set of numbers.
- Tell unsupervised learning apart from supervised learning and reinforcement learning.
Labels are expensive. Someone must mark every email as spam or not, or every X-ray as normal or not. Very often an organisation has plenty of data but no labels at all: a list of customers and their purchases, or thousands of complaint letters. Unsupervised learning finds structure in such data on its own, without an answer key. Exams ask how it differs from supervised learning and how its best-known method, k-means, works.
Finding groups without labels
The most common unsupervised task is clustering: putting similar examples into groups (clusters) so that members of a group are alike and different from other groups. A clothing shop may group its customers by how often they visit and how much they spend, without anyone deciding the groups in advance. The model might find 'frequent small buyers', 'occasional big buyers' and 'festival-season shoppers'; a person then decides what each group means and what to do with it. Other unsupervised tasks include spotting unusual examples, such as an odd card transaction, and reducing many features to a few that keep most of the information.
k-means step by step
k-means is the best-known clustering method. Each group is described by its centre (the average of its members). The steps are:
The k-means loop
- Assign Put every point in the group of its nearest centre.
- Update Move each centre to the average of the points in its group.
Then back to step 1: Assign
- Choose k, the number of groups you want.
- Pick k starting centres, often k of the data points chosen at random.
- Assign: put every point in the group of its nearest centre.
- Update: move each centre to the average of the points in its group.
- Repeat assign and update until no point changes group. The algorithm then stops.
k-means: assign each point to its nearest centre, move each centre to the average of its points, and repeat until nothing changes.
Worked example 1: k-means on four numbers
The monthly spending of four customers, in thousands of rupees, is 2, 4, 10 and 12. Take k = 2 with starting centres 2 and 4.
| Round | Assign (groups) | Update (new centres) |
|---|---|---|
| 1 | 2 goes to centre 2; 4, 10 and 12 are nearer centre 4 | 2 and (4 + 10 + 12) ÷ 3 = 26 ÷ 3 ≈ 8.67 |
| 2 | 4 is now nearer 2 (distance 2) than 8.67 (distance 4.67): groups {2, 4} and {10, 12} | (2 + 4) ÷ 2 = 3 and (10 + 12) ÷ 2 = 11 |
| 3 | No point changes group | Stop: centres 3 and 11 |
The result is a low-spending group and a high-spending group. Notice that the poor starting centres were corrected automatically after two rounds.
Worked example 2: k-means with two features
Four customers are described by (visits a month, spending in thousands of rupees): A (1, 2), B (2, 1), C (8, 9), D (9, 8). Take k = 2 with starting centres at A and C. B is close to A and far from C, so it joins A; D is close to C, so it joins C. Update: centre 1 = average of A and B = ((1 + 2) ÷ 2, (2 + 1) ÷ 2) = (1.5, 1.5); centre 2 = average of C and D = (8.5, 8.5). In the next round no point changes group, so the algorithm stops. With two features, each part of the centre is averaged separately.
Choosing k and other cautions
k is chosen by people, not learned. Too few groups lump different customers together; too many split real groups apart. The final groups can also depend on the starting centres, so k-means is often run several times from different starts and the tightest result is kept. Do not confuse the two k's: in k-NN, k is the number of neighbours that vote; in k-means, k is the number of groups.
How it differs from supervised learning
| Item | Supervised learning | Unsupervised learning |
|---|---|---|
| Data | Labelled examples | Unlabelled examples |
| Goal | Predict a known kind of answer | Discover groups or structure |
| Checking results | Compare predictions with true labels | A person judges whether the groups are useful |
| Example | Spam filter, rent prediction | Customer groups, unusual transactions |
A third type: reinforcement learning
In reinforcement learning there is no fixed set of examples. An agent (the learner) takes actions in an environment and receives a reward or penalty after them. By trial and error it learns which actions earn the most reward over time. Suppose a game agent gets +10 for reaching a goal and −1 for each step. A 4-step route earns 10 − 4 = 6 and a 7-step route earns 10 − 7 = 3, so the agent learns to prefer the shorter route. Game-playing programs and robots learning to walk are classic examples.
Three types of machine learning
-
Supervised
- Learns from labelled examples
- Predicts a known kind of answer
- Spam filter, rent prediction
-
Unsupervised
- Works on unlabelled examples
- Discovers groups or structure
- Customer groups, unusual transactions
-
Reinforcement
- An agent takes actions in an environment
- Learns from rewards or penalties by trial and error
- Game-playing programs, robots learning to walk
Key takeaways: unsupervised learning works on unlabelled data; clustering groups similar examples. k-means repeats assign-and-update until no point moves. k is chosen by people, and the result can depend on the start. Supervised learning learns from labels, unsupervised learning finds structure, and reinforcement learning learns from rewards.
Check your understanding
-
1. Which situation is an example of unsupervised learning?
- Predicting exam marks from hours of study using past students' marks
- Marking emails as spam using emails already marked by users
- Grouping news articles by similar topics when no topics are given in advance
- Teaching a program chess by rewarding wins
Show answer
Answer: C. Grouping articles with no predefined topics is clustering on unlabelled data, which is unsupervised. Predicting marks and marking spam use labelled examples (supervised), and rewarding wins is reinforcement learning.
-
2. In k-means, after points are assigned to groups, what happens to each group's centre?
- It is deleted
- It moves to the average of the points in its group
- It moves to the point farthest from it
- It is given a label by the model
Show answer
Answer: B. The update step moves each centre to the mean (average) of the points assigned to it. Assign and update are repeated until no point changes group.
-
3. The values 3, 5, 15 and 17 are clustered by k-means with k = 2, starting centres 3 and 17. After one assign-and-update round, where are the centres?
- 3 and 17
- 5 and 15
- 8 and 12
- 4 and 16
Show answer
Answer: D. Assign: 3 and 5 are nearer centre 3; 15 and 17 are nearer centre 17. Update: (3 + 5) ÷ 2 = 4 and (15 + 17) ÷ 2 = 16. So the new centres are 4 and 16, and a further round changes nothing.
-
4. In k-means clustering, what does k stand for?
- The number of groups to be formed
- The number of neighbours that vote
- The number of labels in the data
- The number of times the algorithm runs
Show answer
Answer: A. In k-means, k is the number of clusters, chosen by people before the algorithm runs. The number of neighbours that vote is the k of k-nearest neighbours, a different (supervised) method.
-
5. In reinforcement learning, what is the 'agent'?
- The person who labels the data
- The learner that takes actions and receives rewards
- The group centre in clustering
- The reward itself
Show answer
Answer: B. The agent is the learner. It acts in an environment, receives rewards or penalties and gradually learns which actions earn the most reward. Reinforcement learning uses no labelled examples and no cluster centres.
Sign in to keep track of the lessons you finish.