Probability · Lesson 4 of 9
Bayes' theorem
We usually know how likely the evidence is given the cause: how often a test is positive when someone is ill. We want the reverse: how likely the cause is given the evidence. Bayes’ theorem turns one into the other, and it is the most reliably tested idea in this module.
A question to hold on to
0.1% of a town has TB. A test is positive for 99.9% of people with TB, and for just 0.2% of people without it. Your test is positive. What is the chance you have TB?
Delhi School of Economics entrance, 2015Solved at the end ↓
The problem
The question points the wrong way
Most data come as P(evidence | cause): a test’s accuracy, a witness’s honesty, the colour mix in an urn. Most questions ask for P(cause | evidence). Lesson 3 showed these are different numbers. Confusing them has a name in courtrooms, the prosecutor’s fallacy: “an innocent person would match this evidence with probability 1 in 1,000, so the chance he is innocent is 1 in 1,000.” It isn’t, and this lesson shows why.
Pause and predict
At a clinic, 10% of patients have liver disease and 5% are alcoholics. Among patients with liver disease, 7% are alcoholics. What is P(liver disease | alcoholic)? (JNU 2020)
Interactive
Think in people, not percentages
Percentages of percentages are hard to hold in your head. Counts of people are easy. Imagine 1,000 people, split them by the cause, then by the evidence, and look only at the group the evidence puts you in. Each dot below is one person.
0.1% have TB; the test catches 99.9% of cases and wrongly flags 0.2% of healthy people.
1 ill, tested positive
0 ill, tested negative
2 well, tested positive
997 well, tested negative
P(ill | positive)
33%
Of every 1,000 people, about 3 test positive, and 1 of them are ill: 1 ÷ 3. Exactly: 0.001 × 0.999 ÷ (0.001 × 0.999 + 0.999 × 0.002) = 0.3333
Each dot is one person. 1,000 people.
With the TB numbers, one person in 1,000 has TB and tests positive, while about two of the 999 healthy people also test positive. So a positive test puts you among about three people, only one of whom is ill. The test is excellent; the disease is rare. When the condition is rare, the false positives from the huge healthy group outnumber the true ones. Switch to the specialist clinic, where one in five is ill, and the same test becomes convincing.
Pause and predict
1% of people have a disease. A test detects 99% of cases and falsely flags 1% of healthy people. A random person tests positive. P(ill)? (DSE 2005)
The theorem
The theorem, and where it comes from
The multiplication rule gives the overlap two ways: P(A ∩ B) = P(B) P(A | B) = P(A) P(B | A). Divide by P(B) and you have Bayes. The bottom line is P(B) itself, usually found by total probability over the possible causes.
Bayes' theorem
A₁, …, Aₙ are the possible causes (a partition); B is the evidence you observed.
The words matter less than the picture, but papers use them. Prior: P(Aᵢ), what you believed before the evidence. Likelihood: P(B | Aᵢ), how well each cause explains the evidence. Posterior: P(Aᵢ | B), your belief after. In the tree, Bayes is simply: of all the paths that end in B, what share start with Aᵢ? Switch the tree below to “Backward”.
0.1% of a town has TB. The test is positive with probability 0.999 if a person has TB, and 0.002 if not. (DSE 2015)
Start
Has TB1/1000
Positive 999/10001/1000 × 999/1000 = 999/1000000
Negative 1/10001/1000 × 1/1000 = 1/1000000
No TB999/1000
Positive 1/500999/1000 × 1/500 = 999/500000
Negative 499/500999/1000 × 499/500 = 498501/500000
Given positive: each branch’s share of the highlighted paths
P(has tb | positive)999/1000000 ÷ 2997/1000000 = 1/3 ≈ 0.333
P(no tb | positive)999/500000 ÷ 2997/1000000 = 2/3 ≈ 0.667
Bayes in one sentence
The posterior is the prior × likelihood of your cause, divided by the same product added over every cause that could have produced the evidence. Numerator: your path. Denominator: all paths to the evidence.
Recognise the set-up
Five patterns the papers use
Almost every Bayes question in the bank is one of these five. The method is always the same: name the causes, the evidence, then share out.
1 · Which urn? · JNU 2011
Urn 1 has 3 red and 7 blue balls; urn 2 has 6 red and 4 blue. An urn is picked at random and a red ball is drawn. P(it came from urn 2)?
- Paths to red.Urn 1: ½ × 3/10 = 3/20. Urn 2: ½ × 6/10 = 6/20.
- Urn 2's share.
Answer
2/3. With equal priors, the answer is just the ratio of red counts: 6 : 3.
2 · Knew it or guessed? · IIT JAM 2025
A four-option question. Anubhab knows the answer with probability 0.5; if not, he guesses and is right with probability 0.25. He got it right. P(he knew it)?
- Paths to a right answer.Knew: 0.5 × 1 = 0.5. Guessed: 0.5 × 0.25 = 0.125.
- Share.0.5 / 0.625 = 0.8.
- The general form (DSE 2018, 2020).With m options and P(knows) = p: . Check: m = 4, p = ½ gives 2/2.5 = 0.8.
Answer
0.8.
3 · Two witnesses agree · JNU 2014
A tells the truth with probability 0.6 and B with 0.8, independently. Both make the same statement. P(it is true)?
- They agree in two ways.Both truthful: 0.6 × 0.8 = 0.48. Both lying (a yes/no statement, so two lies agree): 0.4 × 0.2 = 0.08.
- Share of the agreement that is true.
Answer
6/7. In general (JNU 2012): pq / [pq + (1 − p)(1 − q)].
4 · Boxes and coins · DSE 2004, 2005; CUET 2024
Three boxes: A has two gold coins, B two silver, C one of each. A box and a drawer are picked at random, and the drawer holds gold. P(the box is A)?
- The tempting answer is 1/2.“It’s A or C, so 50–50.” But A and C are not equally likely to show gold.
- Count the gold coins instead.There are three gold coins, each equally likely to be the one you found. Two of them are in box A.
- Or run Bayes.
Answer
2/3 (Bertrand’s box paradox). CUET 2024 dressed it as three coins, two-headed, two-tailed and ordinary: given a head, P(the other side is a tail) = P(ordinary coin | head) = 1/3.
5 · Who is it? · DSE 2008
80% of statisticians are shy but only 15% of economists. At a gathering, 90% are economists and 10% statisticians. A shy person is met at random. P(statistician)?
- Paths to shy.Statistician: 0.10 × 0.80 = 0.080. Economist: 0.90 × 0.15 = 0.135.
- Share.0.080 / 0.215 = 80/215 ≈ 0.37.
Answer
80/215. Shyness points towards statisticians, but there are nine economists for every statistician, so most shy people are still economists.
A shortcut
The odds form: Bayes in your head
With two causes, there is a faster way. Write beliefs as odds, “ill : well”, and the evidence as a likelihood ratio: how much more likely the evidence is under one cause than the other. Then multiply.
Bayes in odds form
The TB test: prior odds 1 : 999; likelihood ratio 0.999 / 0.002 ≈ 500. Posterior odds ≈ 500 : 999, about 1 : 2, so P ≈ 1/3. The evidence multiplied the odds by 500, but they started so low that even that only reaches one in three.
Updating twice. Yesterday’s posterior is today’s prior. A second, independent positive test multiplies the odds by 500 again: 1 : 2 becomes 500 : 2, and P(TB) rises from 1/3 to about 0.996. This is why doctors confirm a positive screen with a second test.
Solving the opening question
The opening question, solved
DSE 2015 · advanced
0.1% have TB. P(positive | TB) = 0.999 and P(positive | no TB) = 0.002. Given a positive test, what is P(TB)?
- Two causes of a positive test.TB: 0.001 × 0.999 = 0.000999. No TB: 0.999 × 0.002 = 0.001998.
- TB's share of the positives.
- In people.Per 1,000: 1 true positive and about 2 false positives. One in three.
Answer
1/3. A 99.9%-accurate test, and still a two-in-three chance that the positive is a false alarm.
Watch out for
The mistakes examiners are fishing for
- Reporting the likelihood as the answer. P(positive | ill) = 99.9% is not P(ill | positive). The option equal to the test’s accuracy is almost always the trap.
- Ignoring the base rate. The prior matters as much as the evidence. A rare cause stays fairly unlikely even after strong evidence.
- Forgetting a path in the denominator. Every cause that could produce the evidence belongs at the bottom, including “guessed and got lucky” and “both witnesses lied”.
- Assuming equal cases. In the three-box problem, “A or C” is not 50–50 because A is twice as likely to show gold.
Revise
The lesson on one screen
The lesson on one screen
- Bayes' theorem
- P(Aᵢ | B) = P(Aᵢ) P(B | Aᵢ) / Σ P(Aⱼ) P(B | Aⱼ).
- In the tree
- Of all the paths ending in the evidence, the share that start with your cause.
- In people
- Imagine 1,000 people; count those with the evidence; count how many of them have the cause.
- Words
- Prior P(A); likelihood P(B | A); posterior P(A | B).
- Odds form
- Posterior odds = prior odds × likelihood ratio. Update again with each new independent piece of evidence.
- Patterns
- Which urn; knew or guessed, mp/(1 + (m − 1)p); witnesses, pq/(pq + (1 − p)(1 − q)); boxes and coins; who is it.
- The trap
- The option equal to the test's accuracy, or that ignores how rare the cause is.
Practice
Check yourself
Reverse the conditional
A test
20% of patients have a condition. A test is positive for 90% of those who do and 10% of those who don’t. Given a positive test, P(condition)?
Urns
Urn A has 3 red and 2 black balls; urn B has 1 red and 4 black. A coin picks the urn and a red ball is drawn. P(urn A)?
Guessing
A five-option question. A student knows the answer with probability 0.6 and otherwise guesses. She answered correctly. P(she knew)?
Factories
Machine A makes 60% of output with 2% defective; machine B makes 40% with 5% defective. A defective item is found. P(it came from B)?
Quick questions
1Suppose P(A ∩ B) > 0. Then P(A | B) = P(B | A) exactly when
2Prior odds are 1 : 4 and the evidence has likelihood ratio 8. The posterior probability is
3Boxes hold gold–gold, silver–silver and gold–silver coins. A random drawer shows silver. P(the other drawer of that box is silver)?
4A test is 99% accurate for ill and well people alike. As the disease becomes rarer, P(ill | positive)
Real questions
Past papers, by level
Foundation
Definitions and one-step problems. Most CUET and UGC NET questions sit here.
1 questionPractise →
Standard
Two or three steps, up to the hardest JAM, JNU and Hyderabad questions.
7 questionsPractise →
Delhi School level
This topic’s hardest questions, from the DSE entrance, are worked through in Lesson 9: Conditioning and Bayes.
32 questionsLesson 9 →
All 8 past-paper questions for this lesson, easiest first: practise the full set →
These are questions from the TestStop bank. A free account answers 10 a day, each with its worked solution; All-Access opens every set with no daily limit.
Read next
All resources →StatisticsProbability 5: random variables, expectation and varianceA random variable as a number attached to each outcome, the pmf and the CDF, expectation as a balance point, variance, linear transformations, the indicator trick and Chebyshev's bound.MicroeconomicsAsymmetric information, every typeAdverse selection with a live lemons market, moral hazard, signalling, screening and the principal–agent problem.StatisticsProbability, from zero to exam levelA module of lessons that starts at "what is a sample space" and ends at the DSE's hardest questions: counting, the rules, conditional probability, Bayes' theorem, random variables, the discrete and continuous distributions, and a final lesson of hard problems, each with its own interactives and graded past-paper sets.