Revenge, Two Weeks Later

Lecture · Module 10 · Mon Nov 2

How today works

The M10 Module fit three regressions on benevolence in the REACH trial. Today we fit the same three on a different outcome from the same questionnaire: revenge. Same data, same code, new answers. That gives you a second pass through everything the Module covered, and then we use what you’ve learned to catch a mistake applied researchers make often.

Today’s plan

Part	The question	The Module section it revisits
Meet revenge	Where do people start?	The REACH study and its measures
Part 1	Does revenge now predict revenge two weeks later?	Part 1: a continuous predictor (the line, centering, R² and \(\sigma\), inference, CI vs PI)
Part 2	Did the workbook reduce revenge?	Part 2: a binary predictor (the slope as a mean difference, the t-test, Cohen’s d)
Part 3	Does revenge differ across the six sites?	Part 3: a categorical predictor (dummies, the reference group, the omnibus F)
Part 4	What goes wrong when you study only the most (or least) vengeful people?	Part 1: regression toward the mean

Each part follows the Module’s steps in the Module’s order, so you can keep the Module open beside this page.

---

Meet revenge

REACH’s questionnaire, the TRIM-18, has three subscales. The Module used benevolence (goodwill toward the person who hurt you). Today’s outcome is revenge: five items such as “I’ll make him or her pay” and “I want to see him or her hurt and miserable,” each rated from 1 (strongly disagree) to 5 (strongly agree). Each person’s score is the average of the five, so it runs from 1 to 5, and higher means more revenge. The file has it twice: t1_revenge at baseline and t2_revenge two weeks later.

Pair-and-share · predict first

Benevolence scores sat near the middle of the 1–5 scale, spread fairly evenly around it. Where do you think baseline revenge scores sit? In the middle too, or piled up near one end? Sketch the histogram you expect, then open the answer.

Piled up at the bottom. Nearly 3 in 10 participants (28.7%) gave the lowest possible answer, 1, on every item, and the rest trail off to the right (mean 2.00, SD 1.02). Most people who are dealing with a hurt don’t endorse wanting revenge, and the scale can’t go below 1 to show how much they don’t. Statisticians call a pile-up at the lowest possible value a floor effect. Keep it in mind: it matters at the end of today, and again in M12, when we check the model’s assumptions.

---

Part 1 · Does revenge now predict revenge two weeks later?

As in the Module’s Part 1, we start with Hong Kong’s waitlist arm: 173 people who completed both questionnaires and had no workbook during the two weeks. A small sample keeps the picture readable, and with no workbook, the question is simply how revenge carries over from one questionnaire to the next, two weeks later.

Plot before you fit

Pair-and-share · predict first

In the Module, people one point higher on baseline benevolence were predicted to be about 0.62 points higher at follow-up. Will revenge’s slope be steeper, flatter, or about the same? Take a guess, then run the plot.

The cloud rises from lower left to upper right, and a straight line is a reasonable summary of it. Notice the few people at a baseline of 1: they can’t score any lower, at either time.

Fit the line and read it

Read the output · five questions

Answer each from your output before you open the answers.

1. What does the slope say, in one sentence about people?
2. What does the intercept say, and is it a useful number here?
3. What do R² and \(\sigma\) say?
4. What do the slope’s 95% CI and its test say?
5. Your guess from the pair-and-share: steeper, flatter, or the same as benevolence’s?

1. Slope, 0.597. Comparing two people in this arm whose baseline revenge differs by 1 point, the one with the higher baseline is predicted to score about 0.60 points higher two weeks later. It’s a comparison between people, not a statement about one person’s change.
2. Intercept, 0.994. It’s the predicted follow-up score for a baseline of 0, and the scale starts at 1, so no one can have that baseline: it locates the line but isn’t worth interpreting. Centering fixes that, as in the Module: subtract the mean baseline (2.82) from everyone’s score and refit, and the intercept becomes 2.68, the predicted follow-up score for someone at the average baseline, which is exactly the arm’s average follow-up score. The slope doesn’t change.
3. R² = .450: baseline revenge accounts for about 45% of the variation in follow-up revenge. (In a simple regression, R² is the correlation squared: r = .671, and .671² = .450.) \(\sigma\) = 0.645: the typical scale of the scatter around the line; if the errors are roughly Normal, about two-thirds of people land within ±0.645 points of it.
4. 95% CI [0.497, 0.696], with \(t_\text{obs}\) = 11.8 on 171 degrees of freedom and p < .001. The slope’s SE, 0.050, is the sample-to-sample wobble the Module’s bootstrap made visible. The test of a zero slope isn’t very informative here: no one expected revenge two weeks apart to be unrelated. The CI is the more useful part: it rules out slopes near 0 and slopes near 1. (The F in glance(), 140.1, is the slope’s \(t_\text{obs}\) squared: in a simple regression they are the same test.)
5. About the same, a little flatter: 0.60 against benevolence’s 0.62. Both are well below 1. Hold on to that: Part 4 is about what a slope below 1 means.

One person, or the average person?

The county’s question from earlier lectures comes back in a new form: what follow-up score should we expect for someone who starts at a baseline revenge of 4? There are two versions of that question, and they have different answers.

Pair-and-share · which interval answers which question?

A clinic wants to know what to expect on average for clients who start at 4. A client who starts at 4 asks what she should expect. Which interval goes with which question, and why is one so much wider?

Both give the same point prediction, 3.38. The confidence interval, [3.23, 3.53], answers the clinic: it’s the range of plausible values for the average follow-up score among people who start at 4, and it reflects only how precisely the line is estimated. The prediction interval, [2.10, 4.66], answers the client: it covers one new person, so it adds the person-to-person scatter around the line (\(\sigma\) = 0.64). With 173 people, the line is pinned down fairly well, but individuals still scatter widely around it.

---

Part 2 · Did the workbook reduce revenge?

Now the question REACH was built to answer, with revenge as the outcome. As in the Module’s Part 2, we use everyone with a follow-up score, in both arms: 2,057 on the waitlist and 1,907 who got the workbook. The predictor is the factor treat; lm() codes it as 0 for the waitlist (Control) and 1 for the workbook (Treatment).

Pair-and-share · what do the two numbers mean?

1. What is the intercept, 1.90?
2. What is the slope, −0.359? What does its sign say?
3. Can we say the workbook caused the difference?

1. The waitlist’s average follow-up revenge, 1.90. With a 0/1 predictor, the intercept is the mean of the group coded 0.
2. The workbook arm’s mean minus the waitlist’s: 1.54 − 1.90 = −0.36 points, 95% CI [−0.41, −0.31]. Negative means less revenge with the workbook. For a binary predictor, the slope is the difference between the two group means.
3. Mostly, yes, with one caution. Chance decided who got the workbook, so the two arms were comparable at the start, and that’s what supports a causal reading. The caution is who we’re analyzing: only people who completed the follow-up. The workbook arm lost more people (16.7% against 10.9% on the waitlist), and if dropping out was related to revenge, the comparison is no longer quite the one randomization set up. The Module’s listwise-deletion box (in Meet the data) explains why.

The same comparison, two ways. A regression with a binary predictor and the standard (equal-variance) two-sample t-test are the same model. Check it:

The statistic matches the regression’s \(t_\text{obs}\) (−13.07), and the interval matches its CI. How big is the effect? Cohen’s d is the difference in SD units: the slope divided by \(\sigma\) from glance(), −0.359 / 0.865 = −0.42. By Cohen’s rough benchmarks, that falls between a small (0.2) and a medium (0.5) effect.

---

Part 3 · Does revenge differ across the six sites?

The Module’s Part 3 compared baseline benevolence across REACH’s six sites. Do the same with baseline revenge, using everyone in the trial:

Pair-and-share · read the dummies

1. Which site is the reference group, and where is its mean in the output?
2. What does the Hong Kong coefficient mean?
3. What do R² and the F-test in glance() say?

1. Colombia, the first level of site. Its mean baseline revenge is the intercept, 1.60, the lowest of the six sites.
2. Hong Kong’s mean minus Colombia’s: 2.72 − 1.60 = 1.11 points, 95% CI [1.00, 1.22]. Each dummy’s coefficient compares that site with the reference group. Hong Kong has the highest mean, which is one reason Part 1’s arm had so few people at the floor.
3. F(5, 4,592) = 96.7, p < .001: the sites’ means are not all equal. But R² = .095: site accounts for under a tenth of the variation in revenge. Most of the differences are between people within a site.

---

Part 4 · A common mistake: studying only the most vengeful people

Now imagine REACH had been a different study, with the same people but one arm: everyone gets the workbook, and there is no waitlist. A researcher wants to show that the workbook helps the people who need it most, the ones consumed by revenge, so he analyzes only the most vengeful: those with baseline revenge of 3 or higher (about 1 in 5 people, 21.0%). He compares their revenge before the workbook with their revenge two weeks later. It’s a single-arm study: one group, measured before and after.

REACH’s workbook arm lets us run his study with real people: its 403 most vengeful participants who completed both questionnaires.

Pair-and-share · predict first

Two weeks after getting the workbook, how much will his most vengeful group’s average revenge have changed? Write down a number.

Run his study. The chunk keeps REACH’s most vengeful workbook participants and tests their average change against zero with t_test(), the one-sample test from M08.

What would he conclude?

His most vengeful group’s revenge fell by 1.50 points on a 1-to-5 scale, \(t_\text{obs}\)(402) = −29.6, p < .001. What would he write in his abstract? What would a reader conclude about the workbook?

But REACH had a waitlist. 431 of REACH’s most vengeful people didn’t get the workbook during those two weeks. Run the same analysis on them:

These people received nothing

The waitlist’s most vengeful group fell by 0.74 points, \(t_\text{obs}\)(430) = −16.5, p < .001: a large, statistically significant “improvement” among people who got no workbook at all. Take a minute with your neighbor: what does that mean for his result?

Why did people who got nothing improve?

Here is every waitlist participant who completed both questionnaires (2,057 people). Two lines help read the picture. The dashed line is where people would sit if each person’s revenge stayed exactly the same. The regression line is the one you’d fit to these points.

The regression line is flatter than the no-change line: its slope is 0.62, not 1. That is the pattern you met in Part 1, and it has a name: regression toward the mean. People who score far above average on one occasion tend, as a group, to score closer to the average on the next.

Why it happens. Anyone’s revenge score on a given day is part lasting (how they generally feel about the person who hurt them) and part that-day (how raw the hurt felt that week, how they read a question, their mood when they filled it in). The most vengeful scores at baseline belong mostly to people whose that-day part happened to run high. Two weeks later, the lasting part is still there but the that-day part doesn’t repeat, so the group’s average falls back toward the mean. No one has to change for this to happen. Selecting people because they scored high builds part of the drop in, on average.

The regression line predicts it. For the waitlist’s most vengeful group, average baseline 3.62, the line predicts 0.648 + 0.621 × 3.62 = 2.89 two weeks later. They actually averaged 2.88. The whole waitlist, by comparison, changed by just −0.12: the big drop came from whom he selected, not from the passage of time.

Pair-and-share · so what did the workbook do?

Across all of REACH’s most vengeful participants, the workbook group dropped by −1.50 points and the waitlist group by −0.74. What is the workbook’s effect for people who start this vengeful? Why is the waitlist group the right comparison?

About 0.76 points: −1.50 − (−0.74) = −0.76. The waitlist group was selected the same way (baseline 3 or higher), so it regresses toward the mean by the same amount, and chance, not the researcher, decided who in that group got the workbook. Subtracting its drop removes the part of the change that selection alone produces. A single-arm study has nothing to subtract: his −1.50 mixes the workbook’s effect with regression toward the mean, and no statistic computed on one group can separate them.

The other direction: did the workbook harm the least vengeful?

Researchers who select their worst-off participants rarely run the same analysis on their best-off ones. Try it: the people who started with no revenge at all, the lowest possible score of 1 on every item.

Pair-and-share · predict first

Suppose he had analyzed only the workbook participants who started at 1. What would he find two weeks later?

Their revenge rose by 0.19 points, \(t_\text{obs}\)(546) = 8.54, p < .001. By his logic, the workbook made the least vengeful people more vengeful. No one would write that conclusion, yet it is exactly the same analysis as “the most vengeful were really helped.” Here are both selections, in both arms:

In each selection, both arms move the same way: down for the most vengeful, up for the least vengeful, each back toward the middle. That shared movement is regression toward the mean. (At the floor it can only go one way, because a score of 1 can stay or rise but not fall.) The workbook’s effect is the gap between the arms within each selection: −0.76 among the most vengeful, and only −0.08 among the least vengeful, who had almost no revenge to lose.

Pair-and-share · can he keep his claim?

Suppose he reports his single-arm result, “the most vengeful were really helped,” and lists regression toward the mean as a limitation. Is that good enough?

No. A limitation is a caveat on a result. Here, regression toward the mean is a large part of the result itself: in REACH, about half of his −1.50 happened in people who got nothing. With one arm, he has no way to tell how much of the change the workbook caused, so the claim that it helped can’t be supported, however carefully he words the limitation. The standard solution is a comparison group selected the same way, ideally with chance deciding who gets the intervention, as in REACH. (Some designs can shrink the problem, for example selecting people with one screening questionnaire and then measuring a fresh baseline before the intervention, but they rest on extra assumptions, and they don’t rule out the other things a comparison group rules out, such as the passage of time.)

The sentence of the day

Select people because they score extremely, high or low, and, as a group, they will tend to look less extreme next time, with or without your intervention: the worst off will look helped, and the best off will look harmed. A comparison group selected the same way is what tells you how much of the change your intervention caused.

The same applies anywhere people enter a study because a score is high (or low): a clinic that enrolls its most anxious clients, a tutoring program for the lowest scorers on a first exam, a school that flags its most-absent students.

TipGoing further — regression toward the mean with no real change at all

You can watch regression toward the mean appear with no real change at all. In this simulation, each person has a lasting level of revenge that stays exactly the same, and each questionnaire adds a little that-day noise. Selecting the top fifth at baseline still produces a drop. Run it, then try a larger or smaller noise and rerun: the more that-day noise there is in each score, the bigger the drop.

---

Wrapping up

What today covered, with revenge

The Module’s idea	Today’s revenge result
A continuous predictor: plot first, fit, read the slope as a comparison between people	Slope 0.60: two people 1 point apart at baseline are predicted 0.60 points apart two weeks later
Centering gives the intercept a meaning	Centered intercept 2.68: the predicted follow-up score at the average baseline
R² and \(\sigma\)	R² = .45 (r = .67, squared); \(\sigma\) = 0.645
Inference for the slope	95% CI [0.50, 0.70]; F = \(t_\text{obs}^2\)
CI vs PI	At baseline 4: the average person [3.23, 3.53], one person [2.10, 4.66]
A binary predictor: the slope is a mean difference; same model as the t-test	The workbook lowered revenge by 0.36 points; d = −0.42
A categorical predictor: dummies and a reference group	Hong Kong 1.11 points above Colombia; site R² = .095
Regression toward the mean	With no workbook, the most vengeful dropped 0.74 points and the least vengeful rose 0.27

What’s next.

Wednesday’s lab starts the three-lab Bread + Peace analysis: two simple regressions on U.S. presidential elections, and a forecast of 2020 and 2024 with a CI and a PI.

Next Monday, revenge comes back. You’ll put baseline revenge into the model alongside the workbook, the multiple-regression version of today’s comparison, and see how adjusting for where people started sharpens the estimate of what the workbook did.