Revision notes

Chapter 13 Probability: Foundation

The key points, rules to remember, tables and diagrams from all 5 lessons, on one page for quick revision. To keep a copy, use your browser's Print and choose "Save as PDF".

Lesson 1

Conditional probability and its properties

If E and F are events of the same sample space and P(F) ≠ 0, the conditional probability of E given F is P(E|F) = P(E ∩ F)/P(F).

Finding P(E|F) by counting

  1. Restrict the sample space to the outcomes of F
  2. Count the outcomes of F that are also in E (that is, E ∩ F)
  3. P(E|F) = n(E ∩ F) ÷ n(F)
Knowing that F has occurred makes F the new sample space.
  • P(E|F) is the probability of E when F is known to have occurred.
  • P(E|F) = P(E ∩ F)/P(F), defined only when P(F) ≠ 0.
  • With equally likely outcomes, P(E|F) = n(E ∩ F)/n(F): F becomes the new sample space.
  • Conditional probability obeys the usual rules: P(E′|F) = 1 − P(E|F) and the addition rule.
  • P(E|F) and P(F|E) are generally different.
Read the full lesson

Lesson 2

The multiplication theorem on probability

P(E ∩ F) = P(E) P(F|E) = P(F) P(E|F). For three events: P(E ∩ F ∩ G) = P(E) P(F|E) P(G|E ∩ F).

Two draws without replacement from 6 red and 4 blue

  • First red (6/10)
    • Second red (5/9): RR = 30/90
    • Second blue (4/9): RB = 24/90
  • First blue (4/10)
    • Second red (6/9): BR = 24/90
    • Second blue (3/9): BB = 12/90
Multiply along a branch; the four results add to 90/90 = 1.
  • P(E ∩ F) = P(E) P(F|E): multiply, using the conditional probability for the later event.
  • Without replacement, update the counts after each draw.
  • With replacement, the draws do not affect each other.
  • For 'in either order', add the probabilities of the separate orders.
  • A tree diagram organises the multiplications.
Read the full lesson

Lesson 3

Independent events

E and F are independent if and only if P(E ∩ F) = P(E) × P(F). Otherwise they are dependent.

Two different ideas

  • Mutually exclusive
    • Cannot happen together
    • E ∩ F = φ, so P(E ∩ F) = 0
    • Defined from the events themselves
    • Example: 'odd' and 'even' on one die
  • Independent
    • One does not affect the chance of the other
    • P(E ∩ F) = P(E) P(F)
    • Defined through the probabilities
    • Example: head on a coin and 6 on a die
If P(E) > 0 and P(F) > 0, mutually exclusive events are never independent, since then P(E) P(F) > 0 = P(E ∩ F).
  • Independent: P(E ∩ F) = P(E) P(F); equivalently P(E|F) = P(E).
  • Test independence by computing both sides; do not judge by appearance.
  • Mutually exclusive events with positive probabilities are always dependent.
  • Complements of independent events are independent.
  • P(at least one) = 1 − product of the probabilities that each fails.
Read the full lesson

Lesson 4

Partitions and the theorem of total probability

A partition of S

  • Pairwise disjoint
    • No outcome in two of the events
  • Exhaustive
    • Their union is the whole of S
  • Non-zero probabilities
    • P(Eᵢ) > 0 for each i

Theorem of total probability: P(A) = P(E₁) P(A|E₁) + P(E₂) P(A|E₂) + … + P(Eₙ) P(A|Eₙ).

Working for worked example 2
MachineP(machine)P(defective | machine)Product
A0.50.020.010
B0.30.030.009
C0.20.040.008
Total10.027
  • A partition is a set of pairwise disjoint, exhaustive events with non-zero probabilities.
  • P(A) = Σ P(Eⱼ) P(A|Eⱼ): weight each route's conditional probability by the chance of that route.
  • Check that the P(Eⱼ) add to 1 before using them.
  • A table or tree keeps the products organised.
  • The theorem of total probability is the denominator of Bayes' theorem in the next lesson.
Read the full lesson

Lesson 5

Bayes' theorem

Bayes' theorem: P(Eᵢ|A) = P(Eᵢ) P(A|Eᵢ) / [P(E₁) P(A|E₁) + P(E₂) P(A|E₂) + … + P(Eₙ) P(A|Eₙ)], for i = 1, 2, …, n.

Applying Bayes' theorem

  1. List the hypotheses E₁, …, Eₙ (a partition) and their priors P(Eᵢ)
  2. Write the likelihoods P(A|Eᵢ) of the observed result
  3. Multiply: P(Eᵢ) P(A|Eᵢ) for each i
  4. Add the products to get P(A)
  5. Posterior P(Eᵢ|A) = its product ÷ P(A)
Prior and posterior in worked example 3
HypothesisPriorLikelihood of a positive testProductPosterior
Has the condition0.010.950.009519/118 ≈ 0.16
Does not have it0.990.050.049599/118 ≈ 0.84
Total10.0591
  • Bayes' theorem finds the probability of a cause Eᵢ given an observed result A.
  • Posterior = P(Eᵢ) P(A|Eᵢ) ÷ Σ P(Eⱼ) P(A|Eⱼ); the denominator is the total probability of A.
  • Priors matter: a rare cause stays fairly unlikely even after favourable evidence.
  • The posteriors over a partition always add to 1, which is a good check.
  • A random variable assigns a real number to each outcome of an experiment.
Read the full lesson

Privacy · Terms · Refunds · Delivery