Project Rubric

Shared structure for Project 1 and Project 2

How this rubric works

Both projects are scored on the same six dimensions, but the content of each dimension differs. “Technical correctness” means wrangling and descriptive analysis in Project 1 and reconstructing a published test in Project 2.

Weights total 100 points per project. The first four dimensions are graded on the repository and the report together — the rendered document and the repository that produces it are one deliverable, not two. Presentation is graded on the 30-minute session. Peer accountability is graded at the team-member level: any member can be asked to defend any part of the analysis during Q&A.

This rubric is meant to make the standards visible early, so your team can build toward them rather than guess at them late. Read it now and it becomes a checklist you can work against; read it the night before and it becomes a source of dread. Nothing in it should be a surprise.

What this rubric does not reward. Not the number of commits. Not the difficulty of the technique. Not, in Project 2, whether the result reproduced. A careful project that finds a discrepancy and characterizes it honestly scores higher than a careless one that happens to match. That is good news for teams doing real analytic work rather than trying to guess what outcome would look best.

The six dimensions

Dimension	Points
1 · Technical correctness	25
2 · Substantive interpretation	20
3 · Documentation & reproducibility	15
4 · Project-specific quality	20
5 · Presentation	15
6 · Peer accountability	5
Total	100

1. Technical correctness · 25 pts

Does the code run? Does the analysis do what it claims to do? This dimension is about whether the pipeline is trustworthy from start to finish.

Project 1 (Pew Data Brief)	Project 2 (NHST Reproduction)
The pipeline runs end-to-end from `data/raw/` to the analytic dataset with no manual steps	The analysis runs end-to-end and the reported numbers come from the code, not from the paper
Filters, recodes, and derived variables do what the codebook says they do	The analytic sample is reconstructed to match the paper’s stated N, or the gap is explained
Denominators are tracked; every percentage names what it is a percentage of	The test matches the design; assumptions are checked or explicitly acknowledged
Figures are built in ggplot2 from the analytic data, not assembled by hand	The APA result line carries the test statistic, df where applicable, p, the course-standard effect size, and the appropriate CI where the M09 workflow supplies one
Missingness is handled deliberately and the choice is stated	Every number in the original-versus-reproduced table matches the code output exactly

Using survey weights is optional, but state whether they are used or not plainly.

2. Substantive interpretation · 20 pts

Does the prose make sense? Does the argument hang together? This is where clear thinking has to show up on the page.

The question comes first. The report answers a question the team posed, rather than assembling evidence for a conclusion chosen in advance.
Measurement is traced from the survey item or the paper’s instrument through to the analytic variable, and at least one real limitation is named.
Findings are interpreted, not just displayed — the reader is told what the pattern means, not left to infer it from a figure.
Limitations name something specific to this analysis. In your report’s closing section — “What the findings mean — and do not mean” in Project 1, the discussion in Project 2 — a generic caution earns nothing, because it is equally true of every study ever run and so tells your reader nothing about yours. “Correlation is not causation” is the classic example of a sentence that costs you nothing to write and buys the reader nothing to read.

What earns full marks is naming the particular group your sample cannot speak for, or the particular thing your measure cannot capture. For instance: “the ATP panel is recruited from U.S. adults with internet access, so these percentages describe that group rather than all U.S. adults” — a limitation a reader could actually act on.
(Project 2) The discussion owns the outcome. Whether the result reproduced, partially reproduced, or did not, the team characterizes what happened and what would explain it.

3. Documentation & reproducibility · 15 pts

Could a stranger clone your repository and rebuild the work? If not, the project is harder to trust and harder to learn from.

README.md is a working front door — question, summary, repository map, how to reproduce, required packages, and where the data come from.
documentation/codebook.csv uses all seven columns and documents every variable that enters the analysis. source and derivation trace each analytic variable back to its raw origin. See the Analytic Codebook Guide.
documentation/data-provenance.md records the source, the wave or version, the access date, and the steps needed to obtain the file again.
Raw data stays raw. The original file lives in data/raw/ and is never edited by hand; everything downstream is produced by code into data/derived/. A tracked data/README.md explains what belongs in each folder.
Paths are project-relative via here(). The analysis does not depend on files outside the project, objects left in the Environment, or numbers pasted from the Console.
The clean-clone test passed, and the report says who ran it and when.

Documentation discipline is one of the most portable skills in the course, and one of the easiest to skip when people are rushed — which is why it carries its own dimension rather than being folded into technical correctness. A brilliant analysis nobody can rerun is worth less than a modest one that reproduces on the first try.

4. Project-specific quality · 20 pts

This is the part of the rubric where the two projects really diverge. The shared standards above still apply; the items below are the features that make each project successful on its own terms.

Project 1 · The data brief

All required sections are present and doing their jobs: question, measurement, data and analytic sample, Table 1, headline figure, comparison figure, and what the findings mean.
Table 1 describes the analytic sample — the people actually analyzed — with the denominator stated.
Figure 1 carries the headline result and is readable without the surrounding prose. Its title states the finding rather than naming the variables.
Figure 2 earns its place by qualifying, comparing, or complicating Figure 1 — not by restating it in another geometry.
At least one figure is your team’s own analytic contribution — not a direct reproduction of a figure or breakdown from the Pew report that led you to the wave. This is the requirement that makes the brief an analysis rather than a restatement of a report. You are not expected to prove Pew has never published something anywhere; the standard is independence from your source report.
The brief reads as a document written for its stated reader, not as a lab notebook with the code hidden.

Project 2 · Reproduction fidelity

The reproduction plan was written before the target test was run, and records the sample, exclusions, variables, test, and the published numbers the team would compare against. (The bar for “reproduced” is the course’s three-way standard, not one the team sets.)
The paper’s design is correctly recognized, and the parameter and hypotheses are stated in the team’s own words.
The original-versus-reproduced comparison is explicit, with the paper’s numbers and yours side by side.
Discrepancies are characterized rather than smoothed over — the team names the most likely source and says what would settle it.
The required results figure makes the focal effect visible in the raw or minimally summarized data, in a form suited to the design, and meets the M03 standard for titles, labels, and provenance.
The descriptive reconstruction is reported before the test, so the reader can see whether the sample itself matches.

5. Presentation · 15 pts

A 30-minute session in five parts — roughly 20 minutes of content and 10 for questions (see the schedule in each project brief). This is graded as a talk and as an act of teaching your classmates something.

Pacing fits the five-part budget. The setup does not eat the findings, and the talk ends in time to leave the full ten minutes for questions.
The finding is the spine — the question and its answer, not a chronological tour of what the team did.
The behind-the-analysis segment shows real work — a decision the team argued about, a dead end, a judgment call, a correction — specific enough that a classmate could apply it.
Whatever medium you use, curate what the audience sees. Slides are optional and a carefully navigated report is a legitimate presentation tool — but do not scroll chronologically through the whole document or read it aloud. No walls of text.
Q&A is handled honestly. Clear answers where the team knows; a straight “we don’t know” where it doesn’t.

6. Peer accountability · 5 pts

Every team member should be able to defend any part of the analysis. Questions may go to any one of you, and “I didn’t work on that part” is not enough.

This category is small in points but important in spirit, and it is scored individually: if a team member cannot meaningfully engage with the analysis during Q&A, that student’s peer-accountability score may fall to zero. Teammates who can defend the work keep theirs. Divide labor by section, not by competence — everyone needs some hands-on time with the data and the code.

What separates strong from competent

Strong submissions show the team’s thinking. A comment explains why a tricky filter exists; the codebook’s derivation column reads like field notes; the closing section names a specific limitation the team actually worried about; the discussion says which result surprised them.

Competent submissions show the team’s output. The code runs, the figures are clean, the report is tidy — but a reader cannot always tell which choices were deliberate and which were defaults.

The rubric rewards the difference.

Grade scale

Total points	Grade
93–100	A
90–92	A−
87–89	B+
83–86	B
80–82	B−
< 80	flagged for revision; meeting with instructor

Late policy

Both projects are graded from your GitHub repository, not from an upload. Add the instructor (KimberlyHenry) and lab instructor (alliekom) as collaborators before the deadline, and make sure everything you want read has been pushed. We grade the repository as it stands when the deadline passes.

Both projects have a hard 5 PM deadline on the Friday before presentation week — the written deliverable is submitted, and then you present the following week. Project 1 is due Friday, Oct 23 (talks Oct 26–30); Project 2 is due Friday, Dec 4 (talks Dec 7–11). Late work is not accepted. Because the deliverable is the repository, this means we grade the repo as it stands at 5 PM, and anything pushed afterwards is not counted.

The practical consequence is simple: the last push matters more than the last edit. Work finished on someone’s laptop but not pushed does not exist for grading. Push early in the day, then open the repository on github.com in a browser and confirm that what you expect is actually there.

Presentations cannot be made up — if a team member misses the presentation for a documented reason, the remaining members carry the talk.

One last thing

A rubric is a strange document to read. It is written in the language of points and criteria, and read end to end it can make a project sound like an obstacle course.

It isn’t one. Every dimension above describes something a working researcher actually does: state a clear question, trace a measure back to what it measures, keep the data honest, document decisions well enough that someone else can follow them, and say plainly what the evidence supports. You have done every one of those things in smaller pieces already, across M01 to M05. This is the first time they arrive together.

Some of it will be harder than it looks on the page, and that is expected. Teams routinely lose a day to a join that will not behave or a variable that turns out to mean something other than they assumed. That is not a sign the project is going badly — it is the project. What separates teams that finish well is not avoiding those afternoons; it is asking early. Bring it to lab, bring it to office hours, or email us. We would far rather help you in Week 2 than read about it in Week 4.