Data Wrangling

Pre-Study · Module 4 · Fri Sep 4

Welcome to the Module 4 pre-study. You’ve already studied Module 4 — tidy data, the three families of dplyr verbs, the pipe as “then,” and the worked WDI case study that ended in a chart. That was the textbook treatment. This pre-study is where you start writing pipelines yourself.

In this pre-study, you will build a data pipeline step by step. You’ll start with the core wrangling verbs — filter(), mutate(), select(), group_by(), and summarize() — then add arrange(), a ggplot() from M03, and pivot_wider(). That is the core toolkit for the rest of the semester.

How this page is organized

Two short videos introduce the two ideas at the heart of M04 — what tidy data is and why its shape matters, and how group_by() + summarize() work. Most of your time on this page will be spent in ten build-it-yourself activities — each one adds a small piece, so the syntax accumulates naturally as you go.

Every code activity uses a three-tab panel:

  • ✍️ Your Code — the starter with blanks for you to fill in
  • 💡 Hint — a nudge in the right direction
  • 👀 Spoiler — the full working code if you get stuck

Work through the ✍️ tab first. Run it, see what happens, debug if it errors. Only open 💡 or 👀 after you’ve tried it yourself — struggling a little is how this stuff sticks.

Plan to spend about an hour. The goal is not speed — it is to run code, make small mistakes, fix them, and start recognizing the shape of a pipeline.

Video 1 — Tidy data and the three families of verbs

What to listen for:

  • The three families of verbs — row, column, and group operations, the whole toolbox organized into three small groups.
  • Tidy data’s three rules — one variable per column, one observation per row, one value per cell — shown on a real table next to an untidy version of the same data.
  • Why the shape matters — a question that takes one line on tidy data becomes awkward gymnastics on the untidy version.
  • The pipe, pronounced “then” — how a data frame flows through a chain of verbs so a pipeline reads as one plain-English sentence.

Quick check

Answer each question — you’ll see green (correct) or pink (incorrect) feedback as you click.

1. You want to add a new column called approx_gdp_total_bil to wdi_2022 that holds each country’s approximate total GDP in billions of US dollars (population × GDP per capita, divided by a billion). Which family of verbs does this task call for, and which specific verb is the workhorse for it?

2. A classmate wants to analyze change over time, treating year as a variable and each country-year as one observation. Their table has one row per country and three columns — le_1960, le_1990, and le_2022 — each holding life expectancy in that year. Is this table tidy for that analysis?

Video 1 mapped out the row and column families. Now put them to work. The six activities below climb steadily: one condition, then a new column, then two conditions, then sorting, then reading a recode, and finally writing one of your own. Each adds a single piece, so take them in order.

Activity 1.1 — Filter to one region

Start with the simplest step: keep just the rows that match a question. Here’s the research question: which Sub-Saharan African countries do we have data on?

The verb is filter(). Inside filter(), write a condition using == (double-equal, the equality test). Remember: = is for assigning a value, == is for testing one.

Your task: fill the single blank inside filter() with the operator that tests whether region equals "Sub-Saharan Africa".

You want to test whether the region equals that string. Use the double-equal operator.

That’s it — you just wrote your first wrangling pipeline. The full wdi_2022 dataset covers 207 countries; this pipeline keeps only the subset whose region is Sub-Saharan Africa, so the output has many fewer rows than the input.

What you should see: 45 rows — one per Sub-Saharan African country — and all five of the original columns. filter() removes rows; it never touches columns.

Activity 1.2 — Add a column, then select what to keep

Now you’ll add a column to that same filtered data — each country’s approximate total GDP in billions of dollars, computed as population times GDP per capita, divided by a billion — and then use select() to keep just the columns you want to see. Both are column operations: mutate() creates a column, and select() keeps (or drops) columns. You’re extending Activity 1.1’s code, not starting over.

Inside mutate(), give the new column a name on the left of = and the expression that defines it on the right. Then list the columns you want inside select(). The / 1e9 is filled in for you — recall from the Module that 1e9 is R’s shorthand for 1,000,000,000, so dividing by it turns dollars into billions of dollars. That’s exactly what the _bil on the end of the column name is recording: the suffix tells your reader the unit.

Your task: fill the two blanks — inside mutate(), the variable that multiplies by GDP per capita to give each country’s total; inside select(), the new column you just created.

Before you run it, predict the shape: these are both column operations — will the number of rows change?

Two blanks to fill:

  • Inside mutate(): each country’s total GDP is its population multiplied by its GDP per capita.
  • Inside select(): name the column you just created — approx_gdp_total_bil — so the output shows just country and its total GDP.

Two column operations in a row: mutate() added a new approx_gdp_total_bil column (keeping all the others), then select() narrowed the output to just country and approx_gdp_total_bil. mutate() creates columns; select() keeps the ones you name. This is how every derived variable in the rest of the course will be built.

Notice how much easier the rescaled numbers are to read. Angola’s total GDP comes back as about 131 billion; without the / 1e9 that same value prints as 131212208930, and you’d be counting digits to work out its size. Rescaling into sensible units is a small courtesy to whoever reads the table next — which is usually you, three weeks later.

What you should see: the same 45 rows as Activity 1.1, but only two columns — country and approx_gdp_total_bil. The row count is unchanged because neither verb touches rows.

Activity 1.3 — Filter on two conditions at once

Activity 1.1 kept the rows matching one condition. Real questions usually carry more than one. Here’s ours: which Sub-Saharan African countries have a life expectancy above 65 years? That’s a region and a threshold.

Inside filter(), separate conditions with a comma. The comma means and — a row has to satisfy every condition to survive. You’ll also need a comparison operator this time: > for greater than (its relatives are <, >=, and <=).

Your task: fill the single blank with the operator that keeps only the countries whose life_expectancy is above 65.

Before you run it: Activity 1.1 returned 45 countries. Will this return more or fewer?

You want the countries whose life_expectancy is greater than 65. That’s a single character.

Adding another and condition can never increase the result — it either removes more rows or, if every surviving row already satisfies it, leaves the count unchanged. Here it shrinks — 18 countries here, down from the 45 in Activity 1.1. Every extra condition joined by a comma is another hurdle each row must clear. (If you wanted either condition to qualify a row rather than both, you’d use | — the vertical bar, meaning or — but and is what the vast majority of real filters need.)

Activity 1.4 — Sort the survivors

You have the right rows. Now put them in a useful order: which of those countries has the highest life expectancy?

Sorting rows is arrange(), another row operation. On its own arrange() sorts smallest-first; to flip it, wrap the column in desc() (short for descending). You’re extending Activity 1.3’s pipeline by one line.

Your task: fill the single blank inside arrange() with the wrapper that puts the highest life expectancy at the top.

You want the largest life expectancy at the top, so you need the wrapper that reverses the default smallest-first sort.

The highest-life-expectancy country in the region is now the first row. Drop the desc() and you’d get the same 18 countries in the opposite order — arrange() never adds or removes rows, it only reorders them. Read the whole thing aloud and it’s one sentence: “take wdi_2022, then keep Sub-Saharan African countries above 65, then sort by life expectancy, highest first.”

Activity 1.5 — Read and predict a case_when() recode

The biggest step under this video, and it comes in two parts — read one, then write one. Often you don’t want a raw number — you want a label. Here you’ll turn a continuous gdp_per_capita into a three-level band — an illustrative grouping variable of the kind you may build for tables and models later in this course.

This one is deliberately more about reading than typing: only one blank, because the thing worth practicing here is predicting what the arms do, not retyping them.

These bands are invented

The $25,000 and $5,000 cutoffs below were made up for this activity. They are not World Bank income classifications — those exist, use different thresholds, and are based on gross national income per capita rather than GDP. That is exactly why the labels below state their own boundaries: a band called "$25,000 or more" cannot be mistaken for an official category the way "High income" could.

The verb is case_when(), and it lives inside mutate() — because you’re creating a column, this is still a column operation. Each line inside it is an arm with the shape condition ~ value_if_true. That ~ is a tilde; read it as “gives you.”

Your task: fill the single blank — the label the middle arm hands back for the band between the other two. Follow the naming pattern the other two arms set.

Before you run it, answer this — it’s the whole point of the activity: a country with a GDP per capita of $30,000 satisfies the first arm (>= 25000). But it also satisfies the second (>= 5000). Which label does it end up with?

The blank is the value the middle arm hands back — the label for the band between the other two. Follow the pattern the other two arms set and name the boundaries this arm actually catches: "$5,000 to under $25,000" (in quotes — it is a text label, not a column name).

The last arm, TRUE ~ NA_character_, is pre-filled and is the fallback — the Module’s habit of saying out loud that anything matching no arm should become NA.

count() at the end is the Module’s shortcut for group_by() |> summarize(n = n()) — it tallies how many countries landed in each tier.

Now the answer to the prediction: that $30,000 country is labeled “$25,000 or more” — 55 countries land there — even though it also satisfies the >= 5000 arm. R reads the arms top to bottom and stops at the first one that’s true, so the arms below never see it.

That rule is what lets you write plain thresholds instead of spelling out both ends of every band (gdp_per_capita >= 5000 & gdp_per_capita < 25000). And it’s why arm order is part of the logic: move the >= 5000 arm to the top and every country above $5,000 becomes “Middle income” — no error, no warning, just a wrong variable. Write the arms most specific first.

Now look at that last arm, TRUE ~ NA_character_ — the fallback. TRUE is a condition that is always true, so it catches anything still unclaimed by the time R reaches it. Here it never fires: the three tiers add up to all 207 countries, so every case was already covered. Write it anyway. Had a country been missing its gdp_per_capita value, no arm would match it, and the fallback states out loud what should happen — it becomes NA, an honest “we don’t know,” rather than getting quietly assigned a tier it didn’t earn. (The _character_ on NA_character_ is there because every arm has to hand back the same type of value, and the other three all hand back text.)

Activity 1.6 — Now write the arms yourself

Activity 1.5 was about reading. This one is about writing: same verb, different variable, and this time the arms are yours.

Your task: add a column called life_band that sorts each country by life_expectancy into three bands — 80 years or more, 70 to under 80, and under 70 — then count how many countries land in each.

Three things to decide, and one of them is the lesson from 1.5:

  • The conditions. Each arm is condition ~ label. Write plain one-sided thresholds (life_expectancy >= 80), not both ends at once. Because R stops at the first true arm, the arm above has already taken everything past a band’s upper edge — so the middle arm needs only >= 70, never >= 70 & < 80.
  • The order. R stops at the first arm that’s true, so most specific first. Put the >= 70 arm above the >= 80 arm and every country over 70 lands in the same band — no error, no warning, just a wrong variable.
  • The labels. Use text that states its own boundaries, the way 1.5’s did — "80 years or more", not "High". Same reason: a self-describing label can’t be mistaken for an official category.

The final line counts the result. You’ve met that verb already in 1.5.

  • The three conditions all compare life_expectancy against a number: one uses >= 80, one uses >= 70, one uses < 70. Ordering them highest-threshold-first is what makes the middle band mean “70 to under 80” without you writing & life_expectancy < 80.
  • The three labels are quoted text — they are values, not column names.
  • The last line is the tally shortcut for group_by() |> summarize(n = n()).

What you should see: three rows and no NA row — 45 countries at 80 years or more, 101 from 70 to under 80, and 61 under 70. They sum to 207, which is the check that every country matched exactly one arm.

Try deliberately breaking it: move the >= 70 arm above the >= 80 arm and re-run. The 80 years or more band empties out and the middle band swells to 146 — R never complains, because nothing is wrong from its point of view. That silence is the reason arm order counts as part of the logic rather than a matter of style.

You’ve now used five verbs. Before moving on to groups, put each one back in its family — the taxonomy from the Module:

  • filter() and arrange() changed rows — which ones you kept (1.1, 1.3) and what order they came in (1.4). Neither ever altered a column.
  • mutate() and select() changed columns — creating one (1.2) and choosing which to show. Neither ever added or removed a row.
  • case_when() lived inside mutate() (1.5, 1.6), because its job was producing the values for a new column — which makes it a column operation too, not a family of its own.

That’s the whole point of the taxonomy: once you can name which family a question belongs to, the verb nearly picks itself. Next up is the third family — groups.

Video 2 — group_by + summarize: split, apply, combine

What to listen for:

  • group_by() doesn’t change the rows or columns. It tags the data with grouping information so the next verb works separately within each group. The printed tibble may show a small Groups: line, but the data values themselves don’t change.
  • summarize() collapses. Each group folds down to a single summary row — many rows in, one row per group out.
  • Grouped mutate() vs. grouped summarize(). mutate() keeps every row and adds group context; summarize() collapses to one row per group. This is the distinction that trips up the most beginners.
  • Reporting coverage alongside a summary. n() counts the rows in each group. When the summarized variable can be missing, also count the non-missing values the statistic actually used — a mean built from 3 countries and one built from 40 are worth reading differently, and so are two means with the same n() but different amounts of missing data.

Now for the group questions — and the full pipeline. Activities 2.1 through 2.4 put group_by() + summarize() to work, chain everything into a research-question-to-chart pipeline, and finish with a reshape.

Activity 2.1 — Group and summarize, with n()

Time for the most important pattern in M04. The research question is: what is the unweighted mean of the country-level life-expectancy estimates in each region, and how many countries go into each one?

This is a group question, and it is the pattern you will reuse more than any other this semester — so this time you write the whole shape yourself, not just the last piece.

Your task: end up with one row per region, carrying two columns — mean_life, the average life expectancy of the countries in that region, and n_countries, how many countries that average was built from. There are four blanks: the verb that splits the data into regions, the verb that collapses each group down to a single row, the function that averages a column, and the function that counts the rows in a group. (na.rm = TRUE is written in for you — we unpack that argument right after you run this.)

Reporting a count alongside a mean is a useful habit — it tells you how the summary is composed, and how much any single row could move it.

Before you run it, predict the shape: will the result have one row per country, or one row per region?

Four blanks, and you have met all four in the Module:

  • The first two are the grouped-summary pair — one verb marks the groups, the next one collapses them. They almost always appear together, in that order.
  • The third is the ordinary averaging function, the same one you would use outside a pipeline.
  • The fourth takes no arguments at all and only means anything inside a grouped verb: it is the letter n followed by empty parentheses.

207 country rows collapsed to 7 region rows — one row per group. Having n_countries alongside each mean is what makes the table readable: North America’s mean is computed from just 3 countries, so each one contributes a third of it — any single country moves that mean far more than any single country moves a mean built from 45. These are also unweighted means: each country counts once, regardless of population.

Because wdi_2022 is complete on life expectancy, n() here gives both the number of country rows in the region and the number of values contributing to the mean. On data with missing values those two counts can differ — see the box below.

One subtlety about n() — and why it doesn’t bite here

n() counts rows in the group. mean(life_expectancy, na.rm = TRUE) averages the non-missing values. Those are not always the same number — in a group of 20 rows where 3 are missing life expectancy, n() says 20 while the mean rests on 17. Print them side by side and you would be quietly reporting the wrong denominator.

In this pre-study it is not an issue: wdi_2022 has no missing values at all, so every row contributes a value and n_countries is exactly the count behind each mean. That is why the code above is safe as written.

So why write na.rm = TRUE at all, if nothing is missing? Because of what mean() does without it. If even one country in a region is missing life expectancy, that region’s mean comes back as NA — not an average of the countries that do have data, and not a warning. Just NA in one row of an otherwise ordinary-looking table. The other regions compute normally, which is exactly what makes it easy to skim past.

You usually can’t tell by looking whether a column is complete, and a file that is complete today may not be after the next update. So na.rm = TRUE costs nothing here and buys a result that still works on data with gaps — build the habit now, while it changes none of your answers. Its catch is the mirror image of the first one: it drops the missing values silently, and the group still reports its full row count even though fewer values went into the mean. That is precisely what the count beside it is for.

On messier data — which is most data — count the values, not the rows:

summarize(
  n_rows     = n(),                                 # rows in the group
  n_observed = sum(!is.na(life_expectancy)),        # values the mean actually used
  mean_life  = mean(life_expectancy, na.rm = TRUE)
)

sum(!is.na(x)) reads as “count the values that are not missing.” When the two counts agree, n() alone was fine. When they don’t, the second one is the number that belongs next to your mean in a write-up. The M04 Module works through why this matters, and why na.rm = TRUE is not the same thing as listwise deletion.

What you should see: 7 rows — one per region, not one per country. If you got 207 rows, the summarize() did not collapse and something is off upstream.

Activity 2.2 — Chain into a full pipeline

Time to put it all together. Here’s a fresh research question: among countries with life expectancy above 70, what are the top-ranked regions by mean GDP per capita?

You’ll need four verbs in one pipeline — filter() to keep only the high-life-expectancy countries, group_by() to partition by region, summarize() to compute mean GDP per region (with n() alongside), and arrange() to sort by the mean descending. That’s the longest pipeline you’ve built yet — four verbs, and the first to pull a group operation into the row-and-column work you did under Video 1. Go one line at a time.

Your task: fill the four blanks — the column to group by, the mean-GDP expression, the count expression, and the wrapper inside arrange() that sorts largest-first.

You’ve used every one of these verbs already — assemble them:

  • group_by(___) — group by the region column, so you get one row per region.
  • mean_gdp = ___ — the mean of gdp_per_capita; add na.rm = TRUE, just like Activity 2.1.
  • n_countries = ___ — the count of rows in each group: n(), with empty parentheses.
  • arrange(___(mean_gdp)) — to sort largest-first, wrap mean_gdp in desc().

Read this pipeline aloud: “Start with wdi_2022, then keep only countries with life expectancy above 70, then group by region, then compute the mean GDP per capita and the country count, then arrange by mean GDP descending.”

One thing to read carefully: because filter() ran first, n_countries is the number of countries in each region with life expectancy above 70 — not the total number of countries in the region.

Activity 2.3 — Close the M03 → M04 loop with a chart

Last step — turn the Activity 2.2 table into a chart. The pipeline below is identical to 2.2 right up through the summarize(); from there you pipe that one-row-per-region result straight into a ggplot() from M03 to draw a bar chart of mean GDP per capita by region — one bar per region, each bar’s height showing that region’s mean. This is where wrangling pays off visually: the data frame your pipeline produces becomes a chart a reader can scan in seconds.

Your task: fill the single blank on the ggplot() line — the y-axis. Use the column holding the number you want the bar heights to represent, the regional mean GDP you computed in the summarize() step. Everything else in the chart is written for you.

One piece of that code is new: fct_reorder(region, mean_gdp) on the x-axis. You don’t need to memorize it — just read it as “reorder the region labels by mean GDP so the bars come out sorted by height rather than alphabetically.”

The y-axis blank is the variable you just computed in the summarize step. Which column has the regional mean GDP?

There it is — the full M03 → M04 loop. Prepared country-level data went into the pipeline; a communication-oriented bar chart of regional means came out. Every step was one verb. Pipe to filter, pipe to group, pipe to summarize, pipe to arrange, pipe to ggplot, plus to geom_col, plus to labs. You just wrote a real analysis.

One subtlety worth noting: the arrange() step sorts the printed table, but the order of the bars in the chart is set by fct_reorder(region, mean_gdp) inside aes() — not by arrange().

Notice the shape the pipeline produced just before ggplot(): one row per region, with the summary columns ggplot expects (one for the x-axis category, one for the y-axis value). That’s the answer to “why do we wrangle?” — wrangling reshapes raw data into the precise shape your next tool (ggplot, a statistical model, a publication table) demands. The verbs are how you get there. (And notice the pipeline still computes n_countries even though the chart never plots it — a nice habit: keeping the count alongside a summary shows you how each number is composed, and how easily any one row could move it.)

Activity 2.4 — reshape long → wide with pivot_wider()

This final activity switches from wdi_2022 to wdi_trends, the longitudinal dataset — the first time on this page you use the country-year data. wdi_trends is in long format: each row is one country-year observation, with year in a column of its own — the shape that makes grouped summaries and time-axis plots straightforward. But sometimes you need the opposite shape — wide format: one row per country with a separate column for each year, the layout a printed reference table often uses. The verb that produces it is pivot_wider(), the mirror of pivot_longer(). The M04 Module covers both in Part 4; here you’ll practice the long → wide direction.

How pivot_wider() works

pivot_wider() takes one column whose values should become column names (names_from) and another column whose values should become the cells of those new columns (values_from). Everything else stays put as identifying columns.

In this activity, the simplest way to think about it is: year becomes the new column names; life expectancy becomes the values inside those columns.

The more general rule is: the unique values of names_from become new columns, and the values of values_from fill those columns in.

One small wrinkle: when the new column names would otherwise be bare numbers (like 1960, 1990, 2022), it helps to add a names_prefix argument so the columns become y_1960, y_1990, y_2022. That makes the new names cleaner and easier to work with later.

Step 1 — Look at the long-format slice

To keep the result on one screen, we’ll first carve out a small slice of wdi_trends: four countries (Japan, India, Nigeria, United States) at three time points (1960, 1990, 2022). Run this chunk to see what you’re starting from:

There’s a new move in that code, worth pausing on. Every activity so far just ran a pipeline and let its result print. Here the first line begins mini_long <- — you save the pipeline’s output into a named object with the assignment arrow <-, the same arrow you used back in M02 to load a dataset (something <- read_rds(...)). Everything to the right of the arrow runs, and the finished data frame is stored under the name mini_long. The last line — just mini_long on its own — prints it so you can look at what you saved.

Why save it instead of just printing? Because Step 2 builds directly on it: you’ll pipe mini_long into the reshape rather than retyping the whole filter-and-select. Storing an intermediate result under a name and then continuing from it is how longer analyses get assembled — one named step at a time.

You should see 12 rows × 3 columns — long format. Each row is one (country, year) observation. Notice that year is repeated three times for each country, and country is repeated four times for each year.

Step 2 — Pivot wider so years become columns

Now reshape so the unique years become three new columns, with life_expectancy filling the cells. The result should have 4 rows × 4 columns — one row per country, one country identifier column, and three year columns named y_1960, y_1990, y_2022.

Run Step 1 first, so that mini_long exists in your sandbox — this chunk starts from it. (If you reloaded the page, re-run Step 1 before this one.)

Before you run it, predict the shape: reshaping long → wide here gives you fewer rows and more columns — how many of each?

One thing to check before any widen

pivot_wider() has to put one value in each cell. That works here because mini_long holds at most one life-expectancy value for each country-year. If a country-year appeared twice, R would have no way to choose — and instead of erroring it would hand back a column of lists, which breaks the next verb you run in a confusing way.

One line checks it in advance:

mini_long |>
  count(country, year) |>
  filter(n > 1)

Zero rows is the answer you want. If rows come back, find out why before widening — duplicated records, a key you did not expect to repeat, or a genuine one-to-many relationship that needs summarizing first. The M04 Module works through the same check on the full dataset.

Your task: fill the names_from and values_from blanks so mini_long widens into one row per country with one column per year.

How to approach the two blanks. pivot_wider() is asking you two questions, and you answer each one by naming a column of mini_long. Print mini_long again if it helps — it has three columns to choose from.

  1. Which column holds the values that should become the new column headings? That column goes in names_from.
  2. Which column holds the values that should fill the cells underneath those headings? That one goes in values_from.

You name two of the three columns. The third is not forgotten — pivot_wider() treats whatever you left out as the identifier for each row, which is why the result comes back with one row per country rather than one row per country-year. Decide on your two, check them against the target shape above, then run it.

Work out what you want the finished table to look like, then read it back onto the two questions. The new column headings should be the years — y_1960, y_1990, y_2022. The cells underneath them should hold life-expectancy numbers. Each of those two phrases points at one column of mini_long.

(names_prefix and names_sort are filled in for you — they are what turn bare 1960 into y_1960 and put the columns in ascending order.)

You should now see a 4×4 table — one row per country, columns named country, y_1960, y_1990, y_2022, each numeric cell holding the life expectancy for that country-year.

Step 3 — Same data, two shapes

You have the same information in mini_long (12 rows × 3 columns) and mini_wide (4 rows × 4 columns). Neither is more “correct” than the other — they answer different questions:

  • Long format answers “how do I compute the mean life expectancy across all 12 country-year combinations?” in one line: mini_long |> summarize(mean_life = mean(life_expectancy)). Try it.
  • Wide format answers “what does a printed yearbook of these countries look like?” in zero lines — it’s already the yearbook.

The art is choosing the shape that fits your next step. Long is often the convenient shape for grouped analysis and plotting, because year and the measured value each sit in their own column, which is what group_by() and aes() want. Wide is often the convenient shape for paired calculations and presentation tables — subtracting one year from another is a single mutate() when the two years are columns, and human readers scan a rows-and-columns table more easily than a long stack. Neither shape is “correct” on its own; pick the one that makes your next operation obvious.

pivot_wider() converts long → wide. Its mirror, pivot_longer(), converts wide → long. They’re the only two reshape verbs you’ll need this semester.

More practice — pivot_longer with familiar Pew data

Want to practice the opposite direction? Open the Pew social-media activity → — a self-paced tutorial that uses pivot_longer() to reshape the same Pew Research Center series you saw in M03’s lab, then builds the line chart from raw vectors. It’s the same data, the same kind of operation, but going the other way around the reshape pair. Independent of the M04 lab; do it if you have curiosity to burn.

Quick check

Answer each question — you’ll see green (correct) or pink (incorrect) feedback as you click.

1. A colleague’s pipeline computes the mean depression score in each treatment group but doesn’t include n(). Which of these is the MOST important reason to add n()?

2. True or False: After wdi_2022 |> group_by(region) |> summarize(mean_life = mean(life_expectancy, na.rm = TRUE)), the result has one row per country.

The result has one row per region — summarize() collapses each group to a single row. If you wanted one row per country with the region mean added as context, you’d use grouped mutate() instead.

3. Here’s another grouped mutate() — this one ranks each country within its region by life expectancy and saves the result as ranked. (min_rank() numbers the rows 1, 2, 3, …; wrapping the column in desc() makes rank 1 the highest value.)

ranked <- wdi_2022 |>
  group_by(region) |>
  mutate(rank_in_region = min_rank(desc(life_expectancy)))

What is true of ranked?

Grouped mutate() keeps every row and leaves the grouping attached. summarize() peels off the last grouping variable for you; mutate() does not. The Module walks through this surprise in its Ungrouping section.

Here’s the finished pattern — the same grouped mutate(), with the grouping cleared the moment the per-group work is done. Run it:

One row, as you’d expect. (with_ties = FALSE guarantees it: by default slice_max() keeps every row tied at the cutoff, so it can hand back more than the n you asked for.) Now delete the ungroup() line and run it again. The identical slice_max() call hands back 7 rows — the top country in every region — because the leftover grouping made it run within each group. Nothing errors; you just quietly get the wrong answer. That’s the whole bug, and ungroup() is the whole fix.

The habit: ungroup() once the per-group work is done.

Three things to carry into lecture

  • Every wrangling pipeline is verbs chained with the pipe. You read the pipe as “then,” and a well-written pipeline reads aloud as one English sentence.
  • Match the verb to the question family. Rows → filter(). Columns → mutate() or select(). Groups → group_by() plus summarize().
  • Report the relevant denominator alongside any aggregation. n() counts rows; when a statistic drops missing values, also report the number of observed values it actually used (sum(!is.na(x))). A summary built from a few rows reads differently from one built on broader coverage — and is far more sensitive to any single row.