- Foundation
- About 16 min
- 5 practice questions
Common machine learning algorithms explained in plain words
Understand how linear regression, decision trees, k-nearest neighbours and k-means clustering work, with small worked examples and a comparison table.
If something looks wrong, please tell us.
An algorithm is the method a learning program follows to build a model from data. There are many, but four simple ones explain most of the ideas you will meet: linear regression, decision trees, k-nearest neighbours and k-means clustering. You do not need heavy maths to understand them. Each one is a common-sense idea made precise.
Linear regression: fitting a straight line
Linear regression predicts a number by drawing the straight line that best fits the data. For one feature the line is prediction = slope × feature + intercept. Training chooses the slope and intercept that make the predictions as close as possible to the real answers, usually by making the sum of squared errors as small as possible (the least squares method). It is a supervised regression method.
Worked example 1. Suppose a model for monthly flat rent in a town has learned: rent = 10 × area in square feet + 2,000. For a flat of 800 square feet: rent = 10 × 800 + 2,000 = 8,000 + 2,000 = ₹10,000. The slope says each extra square foot adds ₹10; the intercept is the base amount. (Figures are illustrative.)
Decision trees: a chain of questions
A decision tree predicts by asking a series of yes/no questions about the features, like a flowchart. Each question splits the data, and each final branch (a leaf) gives the answer. Training chooses the questions that separate the labels most cleanly. Trees are easy to explain to a person, and they work for both classification and regression, but a very deep tree can memorise noise in the training data.
Is the applicant's monthly income above ₹30,000?
-
No
- Predict 'reject'
-
Yes: more than two existing loans?
- Yes: predict 'reject'
- No: predict 'approve'
- Is the applicant's monthly income above ₹30,000? If no: predict 'reject'.
- If yes: does the applicant already have more than two loans? If yes: predict 'reject'.
- If no: predict 'approve'.
For an applicant earning ₹45,000 a month with one existing loan, the first answer is yes and the second is no, so the tree predicts 'approve'. (An illustrative tree, not a real lending rule.)
k-nearest neighbours (k-NN): ask the neighbours
k-NN classifies a new example by finding the k training examples closest to it and taking a majority vote of their labels. There is no real training step: the algorithm simply stores the data and does its work when a question arrives. The value of k is chosen by the user; an odd k avoids ties between two classes. Because it measures distance, features should be on similar scales.
Worked example 2. A new fruit has features (3, 4). Training fruits: P (3, 5) apple; Q (4, 4) apple; T (3, 1) orange; R (0, 0) orange; S (6, 8) orange. Distances from (3, 4): P = 1; Q = 1; T = 3; R = √(9 + 16) = 5; S = √(9 + 16) = 5. With k = 3, the nearest are P, Q and T: two apples and one orange. The majority vote predicts apple. Notice that with k = 5 all five would vote and orange would win 3 to 2, so the choice of k matters.
k-means clustering: finding group centres
k-means is unsupervised. You choose the number of groups k, and the algorithm repeats two steps until nothing changes: (1) assign each point to its nearest centre; (2) move each centre to the average of the points assigned to it.
The two repeating steps of k-means
- Assign Put each point with its nearest centre.
- Update Move each centre to the average of the points assigned to it.
Then back to step 1: Assign
Worked example 3. Monthly spending of four customers, in thousands of rupees: 2, 4, 10, 12. Take k = 2 with starting centres 2 and 4. Round 1: 2 goes to centre 2; 4, 10 and 12 are nearer 4. New centres: 2 and (4 + 10 + 12) ÷ 3 = 26 ÷ 3 ≈ 8.67. Round 2: 4 is now nearer 2 (distance 2) than 8.67 (distance 4.67), so the groups become {2, 4} and {10, 12}. New centres: 3 and 11. Round 3: no point changes group, so the algorithm stops with a low-spending group and a high-spending group.
| Algorithm | Type | Predicts | Main idea | Watch out for |
|---|---|---|---|---|
| Linear regression | Supervised | A number | Best-fitting straight line | Fails when the relationship is curved |
| Decision tree | Supervised | Category or number | Chain of yes/no questions | Deep trees memorise noise |
| k-nearest neighbours | Supervised | Usually a category | Majority vote of the k closest examples | Slow on large data; needs scaled features |
| k-means | Unsupervised | Group membership | Move centres to the average of their points | k must be chosen; result depends on the start |
Line for numbers (linear regression); questions for decisions (tree); neighbours vote (k-NN); centres move to averages (k-means).
Key takeaways
- Linear regression and decision trees learn from labelled data; k-means needs no labels.
- k in k-NN is the number of neighbours that vote; k in k-means is the number of groups.
- Simple methods are often tried first because they are fast and easy to explain.
Check your understanding
-
1. A model has learned: price of a used scooter = −5,000 × age in years + 60,000. What price does it predict for a 4-year-old scooter?
- ₹20,000
- ₹55,000
- ₹40,000
- ₹80,000
Show answer
Answer: C. Price = −5,000 × 4 + 60,000 = −20,000 + 60,000 = ₹40,000.
-
2. Which algorithm is unsupervised?
- Linear regression
- Decision tree
- k-nearest neighbours
- k-means clustering
Show answer
Answer: D. k-means groups unlabelled data into k clusters. Linear regression, decision trees and k-NN all learn from labelled examples, so they are supervised.
-
3. Using k-NN with k = 3, a new point's three nearest neighbours are labelled 'pass', 'fail' and 'pass'. What is the prediction?
- Pass
- Fail
- No prediction is possible
- Both pass and fail
Show answer
Answer: A. k-NN takes a majority vote among the k nearest neighbours. Two of the three say 'pass', so the prediction is 'pass'.
-
4. In k-means, after points are assigned to groups, what happens to each group's centre?
- It is deleted
- It moves to the average of the points in its group
- It moves to the point farthest from it
- It is replaced by a random label
Show answer
Answer: B. The update step of k-means moves each centre to the mean (average) of the points assigned to it. Assign and update are repeated until the groups stop changing.
-
5. Why are decision trees often liked by banks and other organisations that must explain their decisions?
- They never make mistakes
- They need no data
- Their predictions follow a readable chain of yes/no questions
- They only work on images
Show answer
Answer: C. A decision tree's prediction can be traced through its questions, so a person can see why it decided. Trees still make mistakes, need data to learn, and work on ordinary tables of features.
Sign in to keep track of the lessons you finish.