Revision notes

Chapter 12 Introduction to probability theory: Foundation

The key points, rules to remember, tables and diagrams from all 5 lessons, on one page for quick revision. To keep a copy, use your browser's Print and choose "Save as PDF".

Lesson 1

Random experiments, sample spaces and types of events

Sample spaces of some random experiments
ExperimentSample spacen(S)
Tossing two coins{HH, HT, TH, TT}2² = 4
Tossing three coins{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}2³ = 8
Tossing a coin and rolling a die{H1, ..., H6, T1, ..., T6}2 × 6 = 12
Rolling two dice{(1, 1), (1, 2), ..., (6, 6)}6² = 36
Tossing n coinsall strings of n letters H or T2ⁿ

Types of events

  • Mutually exclusive Cannot occur together: Aᵢ ∩ Aⱼ = ∅ for i ≠ j.
  • Mutually inclusive Can occur together: Aᵢ ∩ Aⱼ ≠ ∅.
  • Exhaustive Together they cover S: A₁ ∪ A₂ ∪ ... ∪ Aₖ = S.
  • Mutually exclusive and exhaustive No overlaps, and nothing left out: a partition of S.
  • Equally likely Each has the same chance of occurring.
NotationMeaning in words
A ∪ BA or B or both occurs (at least one of them)
A ∩ B (also written AB)A and B both occur
ĀA does not occur
A ∩ B̄only A occurs (A but not B)
Ā ∩ B̄neither A nor B occurs

Key takeaways: a random experiment has known possible outcomes but an unpredictable result; the sample space lists them all; events are subsets of S, so a finite S with n points has 2ⁿ events. Mutually exclusive means no overlap; exhaustive means the union is S; a pair of complementary events is both.

Read the full lesson

Lesson 2

Classical and axiomatic probability, and odds

P(A) = n(A)/n(S) = (number of cases favourable to A)/(exhaustive number of cases in S), when all outcomes are equally likely.

— Classical (a priori) probability

Odds and probability

  • Odds in favour a : b
    • P(A) = a/(a + b)
    • P(Ā) = b/(a + b)
    • Example 3 : 5 gives P(A) = 3/8
  • Odds against b : a
    • P(A) = a/(a + b) still
    • Read carefully which number is first
    • Example: against 4 : 11 gives P(A) = 11/15

Key takeaways: classical probability counts equally likely cases; the axioms require non-negative probabilities that add to 1 and are additive for mutually exclusive events; a proposed assignment fails if any value is negative or the total is not 1; odds a : b in favour mean P = a/(a + b).

Read the full lesson

Lesson 3

Basic theorems: complements and the addition theorem

P(A ∪ B) = P(A) + P(B) − P(A ∩ B). For three events: P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B) − P(B ∩ C) − P(C ∩ A) + P(A ∩ B ∩ C).

— Addition theorem

The four regions made by two events

  • Only A: A ∩ B̄ P(A) − P(A ∩ B)
  • Both: A ∩ B Given, or found from the addition theorem
  • Only B: Ā ∩ B P(B) − P(A ∩ B)
  • Neither: Ā ∩ B̄ 1 − P(A ∪ B)

Key takeaways: every rule here follows from the three axioms; use 1 − P(Ā) for at least questions; subtract the overlap once for two events, and add back the triple overlap for three; neither is 1 − P(A ∪ B) and not both is 1 − P(A ∩ B).

Read the full lesson

Lesson 4

Conditional probability and independent events

P(B/A) = P(A ∩ B)/P(A), provided P(A) > 0. It is read: the probability of B, given that A has happened.

— Definition of conditional probability

Mutually exclusive versus independent

  • Mutually exclusive
    • A property of the sets: A ∩ B = ∅
    • P(A ∩ B) = 0
    • If one happens, the other cannot
  • Independent
    • A property of the probabilities
    • P(A ∩ B) = P(A) P(B)
    • If one happens, the other's chance is unchanged

Key takeaways: P(B/A) = P(A ∩ B)/P(A), or count within A; the multiplication theorem chains probabilities in sequence; without replacement the second probability changes. Independence means P(A ∩ B) = P(A)P(B), it carries over to complements, and it is a different idea from mutual exclusiveness.

Read the full lesson

Lesson 5

Total probability and Bayes' theorem

P(B) = P(A₁) P(B/A₁) + P(A₂) P(B/A₂) + ... + P(Aₙ) P(B/Aₙ)

— Total probability of B

A two-stage experiment as a tree

  • Cause A₁, probability P(A₁)
    • B happens: P(B/A₁)
    • Route probability P(A₁) P(B/A₁)
  • Cause A₂, probability P(A₂)
    • B happens: P(B/A₂)
    • Route probability P(A₂) P(B/A₂)
  • Cause Aₙ, probability P(Aₙ)
    • B happens: P(B/Aₙ)
    • Route probability P(Aₙ) P(B/Aₙ)

P(Aᵢ/B) = P(Aᵢ) P(B/Aᵢ) / [P(A₁) P(B/A₁) + P(A₂) P(B/A₂) + ... + P(Aₙ) P(B/Aₙ)]

— Bayes' theorem
Bayes' theorem as a table (three machines)
MachinePrior P(Aᵢ)P(D/Aᵢ)Branch P(Aᵢ)P(D/Aᵢ)Posterior P(Aᵢ/D)
A0.50.020.01010/29
B0.30.030.0099/29
C0.20.050.01010/29
Total10.0291

Key takeaways: the causes must be mutually exclusive and exhaustive; total probability adds the branch products P(Aᵢ)P(B/Aᵢ); Bayes' theorem divides one branch by that total to reverse the condition; a table of priors, branch products and posteriors keeps the working tidy and lets you check that the posteriors add to 1.

Read the full lesson

Privacy · Terms · Refunds · Delivery