Diagnostic testing and Positive Predictive Value (PPV)
Self-paced bonus activity · apply the probability rules to screeners, base rates, and false positives
Welcome to a self-paced bonus activity. In the M06 pre-study you used the three rules of probability — complement, addition, and multiplication — plus conditional probability, on the fake-news dataset where you could count every article directly. Here we apply those same rules to a setting where counting isn’t an option, but the stakes are very real: diagnostic screening. When someone tests positive on a depression, autism, or suicide-risk screener, how much should you actually trust that result? The answer comes straight from the multiplication and addition rules — plus one counterintuitive twist about how the rarity of a condition drives the rate of false alarms. Everything happens on this page — nothing here goes into your lab notebook, and none of it is graded.
Why this matters for psychology students
Across almost every applied area of psychology, we rely on screeners — short, structured questionnaires designed to quickly flag people who might have a particular condition. A few you’ll run into often:
- PHQ-9 (Patient Health Questionnaire) — nine items; screens for major depression in primary care and college counseling centers.
- AUDIT (Alcohol Use Disorders Identification Test) — ten items; screens for harmful drinking, used in everything from ER triage to campus wellness programs.
- M-CHAT-R (Modified Checklist for Autism in Toddlers) — twenty items completed by parents; flags toddlers who should be referred for a full autism evaluation.
- C-SSRS (Columbia Suicide Severity Rating Scale) — a brief structured interview used in crisis hotlines, emergency departments, research studies, and clinical trials.
Each of these returns a binary result — positive (screen endorsed) or negative (screen not endorsed) — against a cutoff. And each one has known sensitivity and specificity that come from validation studies: researchers administered the screener alongside a much longer “gold standard” diagnostic assessment, and measured how often the screener agreed. Those two numbers are what let you tell a PHQ-9 score of 12 from a PHQ-9 score of 6 — they’re doing real work.
But screeners are not diagnoses. A positive screen is an invitation to the next step in a decision chain — usually a structured clinical interview or multi-method assessment — not the end of the story. The reason is exactly what this activity is about: the probability that someone truly has the condition given a positive screen has its own name — the Positive Predictive Value, or PPV — and it depends not just on how accurate the screener is, but on how common the condition is in the population being screened (the prevalence, or base rate).
That one fact has enormous practical consequences for PPV:
- A depression screener validated in a psychiatry outpatient clinic (where prevalence might be 40%) loses much of its PPV when used on a general-population college survey (where prevalence is closer to 10%) — and because real screeners’ specificity sits well below the idealized 0.98 in this activity, the positives can end up mostly false.
- An autism screener with excellent sensitivity and specificity still yields a low PPV when used for universal screening of all 18-month-olds, because autism is relatively uncommon.
- A suicide-risk screener that works well in an emergency department (high prevalence) behaves very differently in a low-risk workplace wellness program.
As a psychology student, you’ll make decisions about screeners often — when reading research and judging what results mean, when choosing measures for your own studies, when designing prevention or intervention programs, and when interpreting positive results in an applied setting. Being able to reason about PPV and base rates is the difference between blindly trusting a test and knowing what a positive result actually implies. (You’ll derive PPV from first principles in Part C — no memorization needed.)
The math you’re about to do is exactly that reasoning tool — applied in its simplest form.
The scenario
Imagine a rare disorder affects 1 in 1,000 people in the general population. There is a highly accurate diagnostic test for it. You just received a positive test result. The treatment is invasive, expensive, and stressful. Before anything else, you want to know:
“What is the probability I actually have the disorder, given that I tested positive?”
That’s a conditional probability: \(P(\text{Disorder} \mid \text{Positive})\).
We’ll get there one step at a time. But first we need to understand what the scenario is actually telling us.
A quick tour of sensitivity and specificity
Every diagnostic test has two accuracy numbers attached to it. They describe what happens to two different groups of people — people who do have the condition, and people who don’t — when they take the test.
Sensitivity and specificity in plain language
Picture two groups:
Group 1 — people who have the disorder. When each person in this group takes the test, some will correctly test positive (a true positive) and some will incorrectly test negative (a false negative).
- Sensitivity = the proportion of Group 1 who correctly test positive.
- In probability notation: \(P(\text{Positive} \mid \text{Disorder})\).
Group 2 — people who do NOT have the disorder. When each person in this group takes the test, some will correctly test negative (a true negative) and some will incorrectly test positive (a false positive).
- Specificity = the proportion of Group 2 who correctly test negative.
- In probability notation: \(P(\text{Negative} \mid \text{No Disorder})\).
The false positive rate is just \(1 - \text{specificity}\) — i.e., \(P(\text{Positive} \mid \text{No Disorder})\).
For our test:
| Number | Name | Plain-language meaning | Probability notation |
|---|---|---|---|
| 0.001 | Prevalence | About 1 in 1,000 people have the disorder | \(P(\text{Disorder}) = 0.001\) |
| 0.99 | Sensitivity | 99% of people with the disorder test positive | \(P(\text{Positive} \mid \text{Disorder}) = 0.99\) |
| 0.98 | Specificity | 98% of people without the disorder test negative | \(P(\text{Negative} \mid \text{No Disorder}) = 0.98\) |
| 0.02 | False-positive rate | 2% of people without the disorder incorrectly test positive | \(P(\text{Positive} \mid \text{No Disorder}) = 0.02\) |
Sensitivity and specificity are conditional probabilities — there’s always a “given that” component. “Given you have the disorder…” or “given you don’t.” These are exactly the same structure as \(P(\text{Fake} \mid \text{With !})\) from the fake-news cross-tab in the M06 pre-study — the bar means “given.”
Part A — Match each number to its meaning
Before you touch any R code, make sure you can identify what each number in the scenario is. Pick the correct translation:
And one more — which probability expression describes the sensitivity of the test?
Once those click, the rest of this activity is just arithmetic on the rules you already know.
Part B — Build the contingency table step by step
We’re going to build a 2×2 table of probabilities — just like the fake-news cross-tab in the M06 pre-study, but we’ll have to compute the cells instead of counting them from data. Every person in the population ends up in exactly one of these four cells:
| Disorder | No Disorder | Row total | |
|---|---|---|---|
| Positive test | True Positive | False Positive | \(P(\text{Positive})\) |
| Negative test | False Negative | True Negative | \(P(\text{Negative})\) |
| Column total | \(0.001\) | \(0.999\) | \(1.000\) |
The column totals are filled in already — 0.001 of the population has the disorder; by the complement rule, 0.999 does not.
We still need the four interior cells. Each one is a joint probability like \(P(\text{Positive and Disorder})\), and there’s a trick for computing every one of them: the multiplication rule you practiced in the M06 pre-study.
\[P(A \text{ and } B) \;=\; P(A \mid B) \times P(B)\]
For each cell, we’ll pick:
- a conditional probability from the scenario (sensitivity, or 1 - specificity, or specificity, or 1 - sensitivity), and
- a marginal from the column totals (0.001 or 0.999),
and multiply them. Let’s do the first one together, entirely by hand, to see how it works.
Step 1 — Work one cell by hand
Goal: the top-left cell — the probability of both testing positive AND having the disorder.
We want \(P(\text{Positive and Disorder})\). Using the multiplication rule:
\[ P(\text{Positive and Disorder}) \;=\; P(\text{Positive} \mid \text{Disorder}) \times P(\text{Disorder}) \]
Now we just plug in. The first term, \(P(\text{Positive} \mid \text{Disorder})\), is the sensitivity — you wrote that down in Part A. It’s 0.99. The second term, \(P(\text{Disorder})\), is the prevalence — the column total. It’s 0.001.
\[ P(\text{Positive and Disorder}) \;=\; 0.99 \times 0.001 \;=\; 0.00099 \]
So 0.00099 of the population — about 99 in 100,000 — are true positives: they have the disorder and the test correctly catches it. That’s the top-left cell.
Now let’s do that same calculation in R so you can see the code pattern. Just run the chunk below — no blanks to fill in yet.
Got it? Good — that’s the pattern for every remaining cell.
Step 2: Positive and no disorder
Same rule, different pieces:
\[P(\text{Positive and No Disorder}) \;=\; P(\text{Positive} \mid \text{No Disorder}) \times P(\text{No Disorder})\]
\(P(\text{Positive} \mid \text{No Disorder})\) is the false positive rate — that’s \(1 - \text{specificity} = 1 - 0.98 = 0.02\).
Your task: fill the two blanks — the false positive rate, and the probability of not having the disorder. The multiplication on the next line is written for you.
\(1 - 0.98 = 0.02\) and \(1 - 0.001 = 0.999\).
About 0.01998 of the population doesn’t have the disorder AND tests positive — false positives.
Look at this closely: the false-positive cell (0.01998) is about 20 times larger than the true-positive cell (0.00099), even though the test catches 99% of true cases. Note that 99% is the test’s sensitivity, not its overall accuracy — accuracy here is about 98%, and neither number is what you actually wanted to know. That gap is the base-rate fallacy: judging how likely a positive result is to be real from the test’s accuracy alone, while ignoring how rare the condition is to begin with.
Step 3: The probability of testing positive
Now use the addition rule — every positive test is either a true positive OR a false positive, and those two are mutually exclusive:
\[P(\text{Positive}) \;=\; P(\text{Positive and Disorder}) \;+\; P(\text{Positive and No Disorder})\]
Your task: fill the two blanks with the values you computed in Steps 1 and 2, so the sum gives the total probability of testing positive.
Step 1 gave you 0.00099. Step 2 gave you 0.01998.
About 2.1% of the whole population tests positive. Of those, most are false positives.
Part C — Compute the PPV
Here’s the question that kicked off this whole activity:
“What is the probability I actually have the disorder, given that I tested positive?”
In probability notation, that is:
\[P(\text{Disorder} \mid \text{Positive})\]
And in plain English: out of everyone who tests positive, what proportion actually has the disorder? This is the conditional probability we flagged at the start of the activity — the Positive Predictive Value (PPV) — and now we have everything we need to compute it.
What PPV means
Positive Predictive Value (PPV) is the probability that a person truly has the condition given that they tested positive. It answers a very practical question: “How much should I trust a positive result?”
A PPV close to 1 means “a positive result almost always means the person has the condition.” A PPV close to 0 means “a positive result is usually a false alarm.”
PPV depends on the test and on how common the condition is (prevalence). Same test, different population → different PPV.
Now — how do we actually compute it?
Remember from the M06 pre-study — the fake-news cross-tab: a conditional probability is a joint probability divided by a marginal probability. That’s the structure:
\[P(\text{Disorder} \mid \text{Positive}) \;=\; \frac{P(\text{Disorder and Positive})}{P(\text{Positive})}\]
- The numerator is the top-left cell of your table: 0.00099 (Step 1).
- The denominator is the row total for “Positive”: 0.02097 (Step 3).
Divide them:
Your task: fill the two blanks — the numerator (the true-positive cell from Step 1) and the denominator (the total probability of testing positive from Step 3).
Numerator: 0.00099 (from Step 1 — the true-positive cell). Denominator: 0.02097 (from Step 3 — the total probability of testing positive).
PPV ≈ 0.047.
Let that land: even with a test that is 99% sensitive and 98% specific, fewer than 5% of positive results correspond to someone who actually has the disorder — because the disorder is so rare that false positives vastly outnumber true positives.
Putting it all together, here is the one-line Bayes’-theorem formula for PPV:
\[\text{PPV} \;=\; \frac{\text{sensitivity} \times \text{prevalence}}{\text{sensitivity} \times \text{prevalence} + (1 - \text{specificity}) \times (1 - \text{prevalence})}\]
The numerator is your true-positive cell. The denominator is the full “Positive” row total. You just did it — one step at a time.
Part D — Explore how prevalence changes PPV
The function calculate_ppv() below wraps the whole computation you just did into a single line of R. Run the chunk once to define the function — then you can call it with different values.
Now change the prevalence. Keep sensitivity at 0.99 and specificity at 0.98. Try prevalence = 0.10 and prevalence = 0.50 — watch what happens to PPV:
Your task: fill the prevalence blank and run the cell, then change the number and run it again. The same test on the same person — only the population changes.
Run the cell once with prevalence = 0.10, then change it to 0.50 and run again. The same test, the same person — only the population changes.
As prevalence rises from 0.1% → 10% → 50%, the PPV jumps from 5% → 85% → 98%. Same test, same accuracy — completely different real-world meaning.
That’s why a depression screener validated in a psychiatric clinic (where prevalence is high) may produce mostly false positives when used on a general college population (where prevalence is lower). A screener’s sensitivity and specificity are properties of the test itself and do not change across populations — but the PPV does, because it depends on how common the condition is in the group you’re actually screening. Same test, different base rate → different meaning of a positive result.
Back to the M06 pre-study → return to the pre-study