Bayes’ theorem answers a question you rarely have a direct answer to, using the reverse question that you
do have an answer to.
P(A|B) = P(B|A) · P(A) / P(B)
Holds when P(B) > 0.
The four parts have names worth knowing:
- P(A) — the prior, what you believed about A before you knew anything.
- P(B|A) — the likelihood, how probable the evidence B is when A really holds.
- P(B) — how probable the evidence B is at all.
- P(A|B) — the posterior, what you believe about A after the evidence.
The theorem follows directly from
conditional probability written both ways
round. P(A∩B) can be computed as P(A|B)·P(B) and as P(B|A)·P(A). Set those equal and divide by P(B), and
Bayes’ theorem is what is left.
A test example
A test detects the disease in 90% of those who have it and gives a false alarm in 10% of those who do
not. The disease is present in 1% of the population. You test positive. How likely is it that you have it?
The answer is not 90%. Count a thousand people:
- 10 are ill. The test finds 9 of them.
- 990 are healthy. At a 10% false-alarm rate the test flags 99 of them.
That is 9 + 99 = 108 positive results, and only 9 of them are ill: 9/108 ≈ 8.3%.
In the formula’s terms: P(B|A) = 0.9, P(A) = 0.01, P(B) = 0.108, and
P(A|B) = 0.9 · 0.01 / 0.108 ≈ 0.083.
The gap between 90% and 8% is not an arithmetic slip but the prior. The disease is so rare that the false
alarms from the large healthy group drown out the true alarms from the small ill one. This is a
common misreading of medical tests, and it has a name: the base rate fallacy.
The denominator, when it is not given
P(B) is often not written out in the problem. It is then computed from the other numbers — every route to
B, added up:
P(B) = P(B|A) · P(A) + P(B|Ā) · P(Ā)
Those are the two rows of the thousand-person count: 0.9 · 0.01 + 0.1 · 0.99 = 0.009 + 0.099 = 0.108. The
rule is called the law of total probability, and it is almost always the first step before Bayes.
Common mistakes
- Confusing P(A|B) with P(B|A). Bayes’ theorem converts one into the other.
- Ignoring the prior. An accurate test for a rare disease still produces mostly false alarms. For a rare
disease, P(A) decides the answer more than the quality of the test does.
- Forgetting the second route in the denominator. Divide by P(B|A)·P(A) alone and you get 1, which
gives away that half the denominator is missing.
- Thinking a second test does not help. It does: after the first test the prior is 8.3% rather than
1%, and if the second test errs independently of the first, a second positive lifts it to around 45%.
Common questions
Why is the answer so much lower than the test’s accuracy?
Because the healthy are so many more. Being wrong 10% of the time about 990 people is more cases than
being right 90% of the time about 10 people.
What are prior and posterior?
Before and after the evidence. The prior is what you believed beforehand, the posterior what you believe
afterwards. Bayes’ theorem is the rule for how evidence should move your belief.
Do I need the law of total probability every time?
Only when P(B) is not stated. If the problem says “8% of all tests come back positive”, you can put that
straight in.
Can I do it without the formula?
Yes, and for many people it is quicker: set a total of 1000, fill in the four boxes and divide the true
positives by all the positives. It is the same arithmetic in whole numbers instead of decimals.