Step 7: Probability
- Probability measures how likely an event is, on a scale from 0 to 1. With equally likely outcomes, it is just counting: favourable outcomes ÷ total outcomes.
- The key rules: complement \( P(A’) = 1 – P(A) \); addition \( P(A \text{ or } B) = P(A) + P(B) – P(A \text{ and } B) \); multiplication \( P(A \text{ and } B) = P(A) \times P(B \mid A) \).
- Conditional probability \( P(B \mid A) = P(A \text{ and } B) \div P(A) \) shrinks the sample space to the cases where A happened.
- Main exam point: \( P(A \mid B) \neq P(B \mid A) \). With a rare condition, even an accurate test produces more false positives than true ones, so always ask for the base rate.
Randomness, sample spaces and events
When you roll a die, you can’t say which number will come up. But roll it 6,000 times and each number will turn up roughly 1,000 times. That is the heart of probability: single outcomes are unpredictable, but the long-run pattern is predictable.
The sample space \( S \) is the set of all possible outcomes. For one die roll, \( S = \{1, 2, 3, 4, 5, 6\} \). An event is any collection of outcomes; “rolling an even number” is the event \( \{2, 4, 6\} \). The probability of an event, \( P(A) \), always lies between 0 (impossible) and 1 (certain), and the probabilities of all outcomes in \( S \) add up to 1.
When every outcome is equally likely, probability is just counting:
For example, the chance of rolling more than 4 on a fair die is \( 2 \div 6 = 1/3 \approx 0.333 \), because only 5 and 6 qualify.
The law of large numbers says that as you repeat an experiment more and more times, the observed proportion of an event settles towards its true probability.
The complement and addition rules
The complement of event A, written \( A’ \) (“not A”), is everything in the sample space outside A. Since A and not-A cover everything:
This is often the fastest route to an answer. Whenever a question says “at least one”, work out the probability of “none” and subtract from 1. For four coin flips, the only way to get no heads is four tails: \( 0.5^4 = 0.0625 \). So \( P(\text{at least one head}) = 1 – 0.0625 = 0.9375 \).
The addition rule handles “A or B”. Two events are mutually exclusive if they cannot happen together; a single die roll can’t be both a 2 and a 5. If events can overlap, simply adding their probabilities counts the overlap twice, so you subtract it once.
For mutually exclusive events \( P(A \text{ and } B) = 0 \), so the rule becomes \( P(A) + P(B) \).
One card is drawn from a standard 52-card deck. Find the probability that it is a heart or a face card (J, Q, K).
Forgetting to subtract the 3 overlapping cards would give the wrong answer of 25/52.
The multiplication rule and independence
The multiplication rule handles “A and B”: both events happening.
\( P(B \mid A) \) means “the probability of B given that A happened”. Successive coin flips are independent. Drawing two cards without putting the first back is not, because removing the first card changes what is left.
A bag holds 6 red and 4 blue marbles. Find the probability of drawing two reds (a) with replacement and (b) without replacement.
The second answer is smaller because removing a red leaves fewer reds.
Tree diagrams
When an experiment happens in stages, a tree diagram keeps track. Each branch carries the probability of that step. Multiply along a path to get the probability of that whole sequence, and add across paths that give the event you want.
Machine A makes 60% of a factory’s items, and 3% of its items are defective. Machine B makes the other 40%, with 5% defective. Find (a) the overall proportion defective and (b) the probability that a defective item came from machine A.
Machine A makes most of the items (60%) but slightly fewer than half the defects, because its defect rate is lower. Reversing the condition changed the answer.
Conditional probability and two-way tables
A conditional probability asks about one event given that another has already happened. It shrinks your attention to just the cases where the given event occurred.
Read it as: of all the times A happens, what fraction also have B? For example, if 40% of students play a sport and 25% play a sport and an instrument, then \( P(\text{instrument} \mid \text{sport}) = 0.25 \div 0.40 = 0.625 \). A and B are independent exactly when \( P(B \mid A) = P(B) \).
Two-way tables make this easy. Here 200 students are classified by laptop ownership and commuting:
| Commutes | Does not commute | Row total | |
|---|---|---|---|
| Owns laptop | 70 | 60 | 130 |
| No laptop | 50 | 20 | 70 |
| Column total | 120 | 80 | 200 |
- Joint (one inner cell over the grand total): \( P(\text{laptop and commutes}) = 70/200 = 0.35 \)
- Marginal (a row or column total over the grand total): \( P(\text{laptop}) = 130/200 = 0.65 \)
- Conditional (restrict to one row or column): \( P(\text{laptop} \mid \text{commutes}) = 70/120 \approx 0.583 \)
The denominator tells you which kind you are calculating.
Reversing the condition: the medical-test problem
The probability that someone carries an umbrella given that it is raining is high. The probability that it is raining given that someone carries an umbrella is much lower, because people carry umbrellas on cloudy days too. Same two events, very different conditional probabilities.
Mixing these up is called the prosecutor’s fallacy. To reverse a condition properly, you need the base rate.
Here is the classic case. A disease affects 1% of people. A test correctly detects 99% of people who have it, and correctly clears 95% of people who don’t, so it has a 5% false-positive rate. You test positive. How likely is it that you have the disease? Most people guess 95–99%. Imagine a large group and count.
Imagine 10,000 people and find \( P(\text{disease} \mid \text{positive}) \).
Even after a positive result from an accurate test, there is roughly an 83% chance you do not have the disease. The 5% error rate is small, but it is applied to 9,900 healthy people, so false alarms swamp the 99 true detections. That is why doctors usually order a second, different test.
Practise: 6 questions
Try each one on paper first, then check the worked answer.
EasyQ1. A bag has 5 red, 3 green and 2 yellow marbles. Find \( P(\text{green}) \) and \( P(\text{not yellow}) \). Show worked answer
\( P(\text{green}) = 3/10 = 0.3 \).
\( P(\text{yellow}) = 2/10 = 0.2 \), so \( P(\text{not yellow}) = 1 – 0.2 = 0.8 \).
EasyQ2. \( P(A) = 0.5 \), \( P(B) = 0.4 \) and \( P(A \text{ and } B) = 0.2 \). Find \( P(A \text{ or } B) \). Show worked answer
The overlap is subtracted once so it is not counted twice.
MediumQ3. A fair die is rolled twice. Find the probability of at least one 6. Show worked answer
The rolls are independent: \( P(\text{no 6 on either}) = \dfrac{5}{6} \times \dfrac{5}{6} = \dfrac{25}{36} \).
\( P(\text{at least one 6}) = 1 – \dfrac{25}{36} = \dfrac{11}{36} \approx 0.306 \).
MediumQ4. Using the laptop and commuting table, find \( P(\text{owns laptop} \mid \text{does not commute}) \) and \( P(\text{does not commute} \mid \text{owns laptop}) \). Why are they different? Show worked answer
Restrict to the 130 laptop owners: \( 60/130 \approx 0.462 \).
They use different denominators: reversing the condition changes the group you are looking at, so \( P(A \mid B) \neq P(B \mid A) \).
MediumQ5. \( P(A) = 0.5 \), \( P(B) = 0.4 \) and \( P(A \text{ and } B) = 0.30 \). Are A and B independent? Show worked answer
The actual value is 0.30, and \( 0.30 \neq 0.20 \), so A and B are not independent.
(Equivalently, \( P(B \mid A) = 0.30 \div 0.5 = 0.6 \), which differs from \( P(B) = 0.4 \).)
ExamQ6. A disease affects 2% of a population. A test detects 90% of cases and has a 10% false-positive rate. Using 10,000 people, find the probability that a person who tests positive has the disease, and explain why it is so much lower than 90%. [6 marks] Show worked answer
True positives: \( 0.90 \times 200 = 180 \) [1].
False positives: \( 0.10 \times 9800 = 980 \) [1].
Total positives: \( 180 + 980 = 1160 \) [1].
\( P(\text{disease} \mid \text{positive}) = 180 \div 1160 \approx 0.155 \), about 16% [1].
Explanation: the healthy group is 49 times larger than the sick group, so its 10% error rate produces far more false alarms (980) than true detections (180). The low base rate, not the test’s quality, drives the result [1].
Check yourself
1. Two events each have probability above 0 and cannot happen together. They are…
2. A fair coin has landed heads five times in a row. The probability of heads on the next flip is…
3. In a two-way table, which denominator gives a conditional probability?
Your score: 0 out of 3
The short version
- Probabilities lie between 0 and 1; with equally likely outcomes, count favourable over total. Past flips do not change the next one.
- Complement for “at least one”; addition for “or” (subtract the overlap); multiplication for “and” (use \( P(B \mid A) \) unless independent).
- Mutually exclusive is not the same as independent. Trees: multiply along, add across.
- \( P(B \mid A) = P(A \text{ and } B) \div P(A) \), and \( P(A \mid B) \neq P(B \mid A) \). With a rare condition, false positives can swamp true ones.
- The full chapter in Statistics Made Simple has 54 practice questions with worked answers, including more base-rate problems.
Further reading on EconTutorials
Older in-depth articles on this topic from our blog: