Revision notes

Machine Learning Fundamentals: Foundation

The key points, rules to remember, tables and diagrams from all 5 lessons, on one page for quick revision. To keep a copy, use your browser's Print and choose "Save as PDF".

Lesson 1

Data and features: examples, labels, data types and clean data

A dataset

  • Rows
    • Each row is one example (record or instance)
    • One mango, one loan application, one email
  • Features
    • The columns the model uses as inputs
  • Label
    • The column holding the answer to predict
    • Also called the target
A tiny labelled dataset
MangoWeight (g)Colour score (1-10)Softness (1-5)Ripe? (label)
125084Yes
218031No
323074Yes
421042No
Common kinds of data
TypeWhat it looks likeExamples
NumericalNumbers that can be measured or countedWeight in grams, monthly rent, number of rooms
CategoricalNames of groups with no orderState, blood group, payment mode (UPI, cash, card)
OrdinalCategories with a natural orderSmall, medium, large; class 8, class 10, class 12
UnstructuredNot in neat columnsPhotos, voice recordings, free text such as complaints

Cleaning data before learning

  1. Remove duplicates The same entry twice would count double.
  2. Fix wrong values Impossible or inconsistent values, such as an age of 205.
  3. Handle missing values Drop the row, or fill the gap with the mean, median or most common value.
  4. Check the labels A wrong label teaches the wrong lesson.
  5. Record every change So the same cleaning can be repeated on new data.

Garbage in, garbage out: a model can be no better than the data it learns from.

Key takeaways: a row is an example, the inputs are features and the answer is the label. Categories are converted to numbers, usually by one-hot encoding, and ordinal data keeps its order. Clean data before learning: remove duplicates, fix errors, handle missing values and check labels. The median is a safer fill than the mean when extreme values exist. Data that is wrong or unrepresentative produces a model that is wrong or unfair.

Read the full lesson

Lesson 2

Supervised learning: classification, regression and three simple methods

Two kinds of supervised task

  • Classification The label is a category.
    • Spam or not spam
    • A handwritten digit 0 to 9
    • Whether a loan will be repaid
    • Which disease a leaf photo shows
  • Regression The label is a number on a continuous scale.
    • The rent of a flat
    • Tomorrow's temperature
    • The expected wheat yield of a field in quintals

Ask: is the answer a category or a quantity? Category = classification. Quantity = regression.

Is the applicant's monthly income above ₹30,000?

  • No
    • Predict 'reject'
  • Yes: more than two existing loans?
    • Yes: predict 'reject'
    • No: predict 'approve'
A small illustrative tree, not a real lending rule.
Three supervised methods compared
MethodPredictsMain ideaWatch out for
Linear regressionA numberBest-fitting straight lineCurved relationships
Decision treeCategory or numberChain of yes/no questionsVery deep trees memorise noise
k-nearest neighboursUsually a categoryMajority vote of the k closest examplesSlow on large data; needs similar scales

Key takeaways: supervised learning needs labelled examples. Classification predicts categories and regression predicts numbers. Linear regression fits a line, a decision tree asks questions, and k-NN lets the nearest examples vote. Simple methods like these are often tried first because they are fast and easy to explain.

Read the full lesson

Lesson 3

Unsupervised learning: clustering and k-means step by step

The k-means loop

  1. Assign Put every point in the group of its nearest centre.
  2. Update Move each centre to the average of the points in its group.

Then back to step 1: Assign

First choose k and pick k starting centres; stop when no point changes group.

k-means: assign each point to its nearest centre, move each centre to the average of its points, and repeat until nothing changes.

Each round assigns, then updates
RoundAssign (groups)Update (new centres)
12 goes to centre 2; 4, 10 and 12 are nearer centre 42 and (4 + 10 + 12) ÷ 3 = 26 ÷ 3 ≈ 8.67
24 is now nearer 2 (distance 2) than 8.67 (distance 4.67): groups {2, 4} and {10, 12}(2 + 4) ÷ 2 = 3 and (10 + 12) ÷ 2 = 11
3No point changes groupStop: centres 3 and 11
Supervised versus unsupervised
ItemSupervised learningUnsupervised learning
DataLabelled examplesUnlabelled examples
GoalPredict a known kind of answerDiscover groups or structure
Checking resultsCompare predictions with true labelsA person judges whether the groups are useful
ExampleSpam filter, rent predictionCustomer groups, unusual transactions

Three types of machine learning

  • Supervised
    • Learns from labelled examples
    • Predicts a known kind of answer
    • Spam filter, rent prediction
  • Unsupervised
    • Works on unlabelled examples
    • Discovers groups or structure
    • Customer groups, unusual transactions
  • Reinforcement
    • An agent takes actions in an environment
    • Learns from rewards or penalties by trial and error
    • Game-playing programs, robots learning to walk

Key takeaways: unsupervised learning works on unlabelled data; clustering groups similar examples. k-means repeats assign-and-update until no point moves. k is chosen by people, and the result can depend on the start. Supervised learning learns from labels, unsupervised learning finds structure, and reinforcement learning learns from rewards.

Read the full lesson

Lesson 4

Model training and testing: training, validation and test sets and the workflow

Shares vary from project to project; 80% training and 20% test is also common when there is no separate validation set
SetUsed forA typical share
Training setLearning: the model adjusts its learned numbers (its parameters, such as the slope of a line) to fit these examplesAbout 70%
Validation setComparing models and choosing settings fixed before training (hyperparameters), such as k in k-NN or the depth of a treeAbout 15%
Test setOne final, honest check on examples the model has never seenAbout 15%

2,000 labelled deliveries (70/15/15 split)

  • Training set: 1,400
    • Learning: the model adjusts its parameters
  • Validation set: 300
    • Comparing models and choosing hyperparameters
  • Test set: 300
    • One final, honest check
    • 255 of 300 right: 85% test accuracy

Train on the training set, tune on the validation set, judge once on the test set, and keep monitoring after deployment.

The machine learning workflow

  1. Collect Data that matches the real task and people.
  2. Clean Fix errors, missing values, duplicates and labels.
  3. Train Train models; compare them on the validation set.
  4. Test Once, on the untouched test set.
  5. Deploy Put it into use in an app, website or office process.
  6. Monitor Keep checking results; retrain when they slip.

Key takeaways: the training set teaches, the validation set chooses and the test set judges, once. A score on training data shows memory, not learning, and leaked or reused test data inflates scores. The workflow is collect, clean, train, test, deploy, monitor, and monitoring continues for as long as the model is in use, because data drifts.

Read the full lesson

Lesson 5

Overfitting and evaluation: accuracy, the confusion matrix, precision and recall

Reading training and test scores together
SituationTraining scoreTest scoreTypical fixes
UnderfittingLowLowA more flexible model, more useful features, longer training
Good fitHighClose to trainingKeep it; monitor it on new data
OverfittingVery highMuch lowerMore data, a simpler model, earlier stopping, limiting tree depth

The four outcomes in a confusion matrix

  • Predicted fraud
    • True positive (TP): really fraud
    • False positive (FP): really genuine, a false alarm
  • Predicted genuine
    • False negative (FN): really fraud, a missed fraud
    • True negative (TN): really genuine

Accuracy = (TP + TN) ÷ total. Precision = TP ÷ (TP + FP): of those flagged, how many were right? Recall = TP ÷ (TP + FN): of the real positives, how many were caught?

Confusion matrix for the better model
ItemPredicted fraudPredicted genuineTotal
Actually fraudTP = 8FN = 210
Actually genuineFP = 12TN = 978990
Total209801,000

Precision or recall?

  • Precision TP ÷ (TP + FP): of those flagged, how many were right?
    • Matters most when a false positive is costly
    • Spam filter: hiding a genuine email such as an interview call letter
  • Recall TP ÷ (TP + FN): of the real positives, how many were caught?
    • Matters most when a false negative is costly
    • Disease screening: a missed patient
Raising one usually lowers the other, so the choice is a decision for people.

Key takeaways: a big gap between training and test scores signals overfitting; low scores on both signal underfitting. Accuracy can look excellent while a model misses every rare case. Build the confusion matrix first, then calculate precision = TP ÷ (TP + FP) and recall = TP ÷ (TP + FN), and choose the measure that matches the costlier mistake.

Read the full lesson

Privacy · Terms · Refunds · Delivery