An annotated example report
Project 2 · a competent reproduction, marked up with what would raise it
How to read this page
Below is a short example reproduction report on an invented paper, written the way a competent team often writes a first draft. It is not a model answer — it is a competent one, which is where most first drafts land.
After each section you will find a boxed note — What would raise it — naming the move that separates competent from strong. The notes are the point of this page. This project’s rubric rewards characterizing a discrepancy rather than smoothing it over — the notes are what that difference looks like on the page.
The paper, the numbers, and the discrepancy are fabricated for illustration. Do not cite them.
The Project 1 annotated example does the same thing for the data brief.
The example report
1 · The published question and claim
Marchetti and Okonkwo (2021) asked whether undergraduates who completed a brief sleep-hygiene workshop reported lower daytime fatigue than a waitlist group. They report that the workshop group scored lower on a fatigue scale at follow-up, t(178) = 2.61, p = .010, d = 0.39.
What would raise it
The claim is reported accurately and the statistics are copied correctly, which is the requirement. What is missing is the scope of the claim: the paper’s own conclusion was about a workshop delivered to volunteers at one university, and saying so here sets up the discussion nine sections later. State the claim at the size the authors actually made it.
2 · Recognize the design · 3 · Parameter and hypotheses
Two independent groups, one continuous outcome, measured once. That is an independent-samples design, so the parameter is the difference in population mean fatigue between workshop and waitlist, and H0 is that the difference is zero.
What would raise it
Correct, and stated in the team’s own words rather than copied — which is what the rubric asks. A stronger version would say why the design is independent rather than paired here, since the paper measured everyone at follow-up only. Naming what the design is not shows you recognized it rather than inherited it.
4 · Data and analytic-sample reconstruction
The posted dataset has 214 rows. The paper reports 180 in the analysis. Applying the exclusions described in the Method — attrition before follow-up, and one group assignment recorded as unknown — leaves us with 186.
What would raise it
This is the most valuable paragraph in the report and it stops one sentence early. A six-person gap is a finding, not an inconvenience: which exclusion is ambiguous, and what would settle it? A strong version says “the paper does not state whether partial-completion cases were dropped; if we also exclude the four participants missing two or more scale items, we reach 182, and at 180 we would additionally need a criterion the paper does not describe.” That is a reader who could act on your work.
5 · Reconstruct the descriptive results · 6 · Assess the design and model
Group means and SDs are close to the published table. A Q-Q plot of the outcome shows mild right skew in both groups, and group variances are similar, so we used Welch’s t-test.
What would raise it
The assumption check is present and the choice of Welch is defensible. But “close to the published table” is doing a lot of unexamined work — how close? Reporting the two sets of means side by side, before the test, is what lets a reader see whether a later discrepancy came from the sample or the analysis. The rubric asks for the descriptive reconstruction before the test for exactly this reason.
7 · Reproduce the test · 8 · Original versus reproduced result
| Quantity | Original | Ours | Match |
|---|---|---|---|
| Analytic N | 180 | 186 | close |
| Test statistic | t(178) = 2.61 | t(181.4) = 2.38 | close |
| p-value | .010 | .018 | same direction |
| Effect size | d = 0.39 | d = 0.35 | close |
Our reproduction found the same direction and a slightly smaller effect.
What would raise it
The table is exactly right — this is the deliverable, and it is built the way the rubric asks. The single sentence under it is where the report goes quiet. Note that the df differ in kind, not just size: 178 is a pooled t, 181.4 is Welch’s. That means the team did not run the same test the authors ran, which is a legitimate choice but has to be said, because it is one candidate explanation for the p-value moving.
9 · Discussion and scope
The finding reproduced in direction and approximate magnitude. The p-value moved from .010 to .018, likely because our analytic sample is slightly larger. This does not undermine the original result. Limitations include our uncertainty about the exclusions and the fact that we used Welch’s test.
What would raise it
The honesty is right and the team resists overclaiming, which many drafts fail to do. But “likely because our analytic sample is slightly larger” is a guess presented as a conclusion — and it is checkable. Running the test again on the 180 closest to the paper’s description, and reporting whether p returns to .010, converts a guess into evidence. That single extra run is the difference between a team that noticed a discrepancy and a team that characterized one.
The last sentence also lists limitations rather than weighing them. Which of the two matters more for whether the original finding stands?
What this example is not
It does not show a strong report, deliberately. The distance from competent to strong here is not more analysis — it is a handful of specific moves:
- reporting the claim at the size the authors made it
- treating a sample-size gap as a finding to diagnose, not a nuisance to note
- showing the descriptive comparison before the test, so a later gap can be located
- saying when you ran a different test from the authors’, and why that matters
- checking the explanation you offer, when checking it costs one more line of code
The last one is the whole project in miniature. A reproduction that says “probably because…” has stopped one step short of its own contribution.