• Foundation
  • About 16 min
  • 5 practice questions

Supervised learning: classification, regression and three simple methods

Decide whether a supervised task is classification or regression, and explain in plain words how linear regression, decision trees and k-nearest neighbours make predictions, working small examples by hand.

If something looks wrong, please tell us.

By the end of this lesson you can

  • Decide whether a supervised task is classification or regression.
  • Make a prediction by hand with a regression line, a decision tree and a k-nearest neighbours vote.
  • Compare the strengths and weaknesses of these three methods.

In supervised learning, every training example comes with its correct label, like a student practising with an answer key. The model studies features and labels together and learns to map one to the other, so that it can predict the label of a new example. Most ML in everyday use, from spam filters to loan checks, is supervised. This lesson shows the two kinds of supervised task and three simple methods that solve them.

Classification and regression

Two kinds of supervised task

  • Classification The label is a category.
    • Spam or not spam
    • A handwritten digit 0 to 9
    • Whether a loan will be repaid
    • Which disease a leaf photo shows
  • Regression The label is a number on a continuous scale.
    • The rent of a flat
    • Tomorrow's temperature
    • The expected wheat yield of a field in quintals
  • Classification: the label is a category. Examples: spam or not spam; a handwritten digit 0 to 9; whether a loan will be repaid; which disease a leaf photo shows.
  • Regression: the label is a number on a continuous scale. Examples: the rent of a flat; tomorrow's temperature; the expected wheat yield of a field in quintals.

Ask: is the answer a category or a quantity? Category = classification. Quantity = regression.

Worked example 1: naming the task

  1. A dairy cooperative predicts how many litres of milk a farm will supply next month from past records. The answer is a quantity: regression.
  2. A hospital sorts chest X-rays into 'normal' and 'needs review', learning from X-rays already marked by doctors. The answer is a category: classification.
  3. An app names a leaf's condition as 'healthy', 'rust' or 'blight'. Three categories instead of two is still classification.

Linear regression: fitting a straight line

Linear regression predicts a number by drawing the straight line that best fits the data. With one feature the line is prediction = slope × feature + intercept. Training chooses the slope and intercept that make the predictions as close as possible to the real answers, usually by making the sum of the squared errors as small as possible (the least squares method).

Worked example 2. Suppose a model for monthly flat rent in a town has learned: rent = 10 × area in square feet + 2,000. For a flat of 800 square feet: rent = 10 × 800 + 2,000 = 8,000 + 2,000 = ₹10,000. The slope says each extra square foot adds ₹10; the intercept is the base amount. (The figures are illustrative.) A straight line cannot follow a curved relationship well, which is the method's main weakness.

Decision trees: a chain of questions

A decision tree predicts by asking a series of yes/no questions about the features, like a flowchart. Each question splits the data, and each final branch (a leaf) gives the answer. Training picks the questions that separate the labels most cleanly. Here is a small illustrative tree, not a real lending rule:

Is the applicant's monthly income above ₹30,000?

  • No
    • Predict 'reject'
  • Yes: more than two existing loans?
    • Yes: predict 'reject'
    • No: predict 'approve'
A small illustrative tree, not a real lending rule.
  1. Is the applicant's monthly income above ₹30,000? If no: predict 'reject'.
  2. If yes: does the applicant already have more than two loans? If yes: predict 'reject'.
  3. If no: predict 'approve'.

For an applicant earning ₹45,000 a month with one existing loan, the first answer is yes and the second is no, so the tree predicts 'approve'. Anyone can trace why, which is why trees are popular where decisions must be explained. They work for both classification and regression, but a very deep tree can memorise accidents in its training data.

k-nearest neighbours: ask the neighbours

k-nearest neighbours (k-NN) classifies a new example by finding the k training examples closest to it and taking a majority vote of their labels. It has almost no training step: it stores the data and does its work when a question arrives. People choose k; an odd k avoids ties between two classes. Because it measures distance, the features should be on similar scales.

Worked example 3. A new fruit has features (3, 4). Training fruits: P (3, 5) apple; Q (4, 4) apple; T (3, 1) orange; R (0, 0) orange; S (6, 8) orange. Distances from (3, 4): P = 1; Q = 1; T = 3; R = √(9 + 16) = 5; S = √(9 + 16) = 5. With k = 3 the nearest are P, Q and T: two apples and one orange, so the prediction is apple. With k = 5 all five vote and orange wins 3 to 2. The choice of k matters.

Three supervised methods compared
MethodPredictsMain ideaWatch out for
Linear regressionA numberBest-fitting straight lineCurved relationships
Decision treeCategory or numberChain of yes/no questionsVery deep trees memorise noise
k-nearest neighboursUsually a categoryMajority vote of the k closest examplesSlow on large data; needs similar scales

Key takeaways: supervised learning needs labelled examples. Classification predicts categories and regression predicts numbers. Linear regression fits a line, a decision tree asks questions, and k-NN lets the nearest examples vote. Simple methods like these are often tried first because they are fast and easy to explain.

Check your understanding

  1. 1. A model predicts the number of passengers who will board a train next Sunday, using past records. Which kind of task is this?

    1. Classification
    2. Clustering
    3. Reinforcement learning
    4. Regression
    Show answer

    Answer: D. The output is a quantity (a number of passengers) and past records give known answers, so it is supervised regression. Classification would predict a category; clustering and reinforcement learning do not learn from labelled answers in this way.

  2. 2. A model decides whether a leaf photo shows 'healthy', 'rust' or 'blight', trained on photos labelled by experts. This is:

    1. Regression
    2. Classification
    3. Reinforcement learning
    4. Clustering
    Show answer

    Answer: B. The label is one of three categories and labelled examples are available, so this is supervised classification. Having more than two categories does not change that.

  3. 3. A model has learned: price of a used scooter = −5,000 × age in years + 60,000. What price does it predict for a 4-year-old scooter?

    1. ₹20,000
    2. ₹55,000
    3. ₹40,000
    4. ₹80,000
    Show answer

    Answer: C. Price = −5,000 × 4 + 60,000 = −20,000 + 60,000 = ₹40,000. The negative slope means each extra year lowers the price by ₹5,000.

  4. 4. Using k-NN with k = 3, a new point's three nearest neighbours are labelled 'pass', 'fail' and 'pass'. What is the prediction?

    1. Pass
    2. Fail
    3. No prediction is possible
    4. Both pass and fail
    Show answer

    Answer: A. k-NN takes a majority vote among the k nearest neighbours. Two of the three say 'pass', so the prediction is 'pass'.

  5. 5. Why are decision trees often liked by organisations that must explain their decisions?

    1. They never make mistakes
    2. They need no data
    3. Their predictions follow a readable chain of yes/no questions
    4. They only work on images
    Show answer

    Answer: C. A decision tree's prediction can be traced through its questions, so a person can see why it decided. Trees still make mistakes, need labelled data to learn, and work on ordinary tables of features.

Privacy · Terms · Refunds · Delivery