# Attrition Analysis Evidence Across Research Papers *Generated 2026-04-09. Based on exhaustive keyword search using James_analysis_code.txt search terms plus additional terms (sample size, response rate, follow-up, panel, wave, filter, screen, selection, opt-out, recontact, subset).* --- ## Paper 1: Lee et al. (2022) — "Negative partisanship is not more prevalent than positive partisanship" **Journal:** Nature Human Behaviour, Vol 6, pp. 951–963 **Study type:** Multi-method (ANES repeated cross-sections 1978–2020, CCES 2016 survey, SSI 2016 survey, two pre-registered one-shot behavioral experiments). Not longitudinal. | # | Term | Line | Quote | Attrition-relevant? | | -------------- | ----------- | ------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | 1 | drop | 636 | "No participants **dropped** out or declined participation." | **YES** — blanket assertion of zero dropout in Reporting Summary checklist | | 2 | exclud | 634 | "We did not **exclude** any data points." | **YES** — assertion of zero exclusions in Reporting Summary | | 3 | restrict | 119 | "we **restrict** our analysis to only those who identify with one of the two major parties (n = 887)" | Borderline — design restriction, not attrition | | 4 | restrict | 315 | "we **restricted** the data to face-to-face interviews" | Borderline — mode restriction, not attrition | | 5 | sample size | 355 | "Our **sample size** was determined on the basis of a power analysis" | No — power analysis | | 6 | depart | 13, 21 | "Our findings consistently **depart** from..." / "constitutes a **departure** from..." | No — conceptual divergence | | 7 | complete | 333, 339 | "**complete** question wording" / "**completed** a practice session" | No — procedural | | 8 | lost | 357 | "those in the opposing group **lost** rewards" | No — game mechanics | | 9 | panel | 119, 333, 335 | Describes SSI panel, Bovitz panel | No — sampling frame | | **Supplement** | missing | 321, 357, 393 | "**Missing** values are not included in the percentage calculation." (×3) | No — table footnotes about demographic percentages | **Assessment: MINIMAL attrition handling.** Two single-sentence declarations in the Reporting Summary (zero dropout, zero exclusions) with no supporting evidence, no CONSORT diagram, no differential attrition analysis, and no flow chart. The claim of zero dropout is unsubstantiated. --- ## Paper 2: Stagnaro & Amsalem (2025) — "Factual knowledge can reduce attitude polarization" **Journal:** Nature Communications 16:3809 **Study type:** Two-wave randomized experiment (Wave 1: N=1,011; Wave 2: N=881 one month later). | # | Term | Line | Quote | Attrition-relevant? | | ---- | -------------- | ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | | 1 | attrit | 201 | "30 (2.96%) completely **attrited** the study, giving us a final sample of N = 1011" | **YES** — within-wave attrition count and rate | | 2 | attrition | 205 | "to prevent **attrition** issues, participants were given several opportunities to exit the main sections of the study (at a cost)...For details on our efforts to mitigate **attrition**, see Supplementary Note 9." | **YES** — describes attrition mitigation design, references supplementary details | | 3 | lost | 205 | "preserve those observations we would have **lost** due to attrition" | **YES** — rationale for attrition mitigation | | 4 | retention | 43 | "we recontacted all participants a month later with a **retention** rate of 87% (N = 881)" | **YES** — between-wave retention rate | | 5 | exit | 205 | "participants were given several opportunities to **exit** the main sections of the study" | **YES** — early exit mechanism for attrition prevention | | 6 | opt-out | 205 | "the '**opt-out**' option if they wished to end the study early" | **YES** — same attrition mitigation mechanism | | 7 | wave | 241 | "All 1,011 participants who completed **Wave** 1...were recontacted for **Wave** 2...881 (87%) participated in **Wave** 2. The **Wave** 2 participants were comparable to those from **Wave** 1 on political ideology, party identification, gender, and condition (see Supplementary Note 13). Further, looking at the **Wave** 1 results while subsetting just on those who returned in **Wave** 2 yields substantively identical results (see Supplementary Note 14). Thus, there appears to be no obvious 'type' of participant who was available at **Wave** 1 but not at **Wave** 2." | **YES** — the primary attrition analysis passage: balance checks + robustness subsetting | | 8 | filter | 201 | "632 were **filtered** out for cellphone use, not wanting to do a long study...or because they indicated themselves to be True Independent" | Borderline — pre-randomization screening (1673 entered → 632 filtered → 1041 continued → 30 attrited → N=1011) | | 9 | complete | 201 | "30 (2.96%) **completely** attrited the study" | **YES** — part of attrition report | | 10 | complete | 241 | "All 1,011 participants who **completed** Wave 1 of the study were recontacted for Wave 2" | **YES** — defines recontact universe | | 11 | non-compliance | 51 | "results hold when using instrumental variables to estimate the **complier** average treatment effects, which take into account **non-compliance**" | Borderline — addresses treatment non-compliance (Supplementary Note 2), distinct from attrition | | 12 | exclud | 115 | "**excluding** the 8.4% of participants whose attitudes were already moderate" | No — analytic exclusion for coding purposes | | 13 | subset | 241 | "**subsetting** just on those who returned in Wave 2 yields substantively identical results" | **YES** — direct attrition robustness check | **Assessment: SUBSTANTIAL attrition handling.** The paper provides: (1) within-wave attrition count and rate (2.96%), (2) a designed attrition-mitigation mechanism (opt-out → skip to outcome measures), (3) between-wave retention rate (87%), (4) balance checks comparing Wave 1 and Wave 2 participants on observables (Supplementary Note 13), (5) a subsetting robustness check (Supplementary Note 14), and (6) a conclusion that attrition was not systematic. Full details deferred to supplementary notes 9, 13, and 14. No CONSORT diagram. No explicit differential attrition test by treatment arm in the main text. --- ## Paper 3: Vlasceanu et al. (2024) — "Addressing climate change with behavioral science: A global intervention tournament in 63 countries" **Journal:** Science Advances 10, eadj5778 **Study type:** Cross-sectional global megastudy. Single-wave between-subjects randomized experiment (11 interventions + control) across 59,440 participants from 63 countries. | # | Term | Line | Quote | Attrition-relevant? | | -------------- | --------- | ---- | ------------------------------------------------------------ | ------------------------------------------------------------ | | 1 | complete | 215 | "A total of 83,927 **completed** the study. Of them, 59,440 participants...who passed the two attention checks...and correctly **completed** the WEPT demo were included in the analyses." | **YES** — reveals 83,927 completed but only 59,440 analyzed (29.2% excluded) | | 2 | fail | 215 | "removing participants who **failed** these preregistered attention checks risks contributing to a selection bias in the sample (54)" | **YES** — sole acknowledgment that exclusion criteria could introduce bias | | 3 | remove | 215 | "**removing** participants who failed these preregistered attention checks risks contributing to a selection bias" | **YES** — same passage | | 4 | remove | 225 | "which **removed** from the experiment any participants choosing an incorrect answer" | **YES** — attention check #1 actively removed participants in real-time | | 5 | screen | 215 | "we a priori determined we would **screen** participants according to these criteria to ensure data quality" | **YES** — describes screening rationale | | 6 | exclud | 247 | "Participants who indicated that they do not use social media were **excluded** from this analysis (i.e., a third of the sample)." | **YES** — outcome-specific exclusion of ~1/3 of sample | | 7 | exit | 255 | "Participants were allowed to **exit** the task at any point with no penalty." | Borderline — within-task self-selection on WEPT, by design | | 8 | complete | 255 | "Participants were not allowed to advance the page until they correctly **completed** the WEPT demonstration." | Borderline — screening gate for WEPT demo | | 9 | complete | 91 | "half of all participants (50.7% of total sample and 53.1% of control condition sample) **completed** all eight pages of the WEPT" | Borderline — task engagement, not study attrition | | 10 | selection | 215 | "risks contributing to a **selection** bias in the sample (54)" | **YES** — acknowledges selection bias risk | | **Supplement** | — | — | No attrition-related terms found in supplement (only "completed" referring to WEPT scoring on line 661 and "Missing" in Bayesian context) | No attrition evidence in supplement | **Assessment: LIMITED attrition handling.** The paper reveals a substantial 29.2% exclusion rate (83,927 → 59,440) but provides: (1) no breakdown by exclusion criterion, (2) no differential attrition analysis across the 12 conditions, (3) no comparison of excluded vs. included participants, (4) no CONSORT diagram, (5) no analysis of whether exclusion varied by country or recruitment method. The sole defense is one sentence acknowledging selection bias risk and stating exclusion criteria were preregistered. For the social media outcome, an additional ~1/3 of participants were excluded with no analysis of this subsample. --- ## Paper 4: Voelkel et al. (2024) — "Megastudy testing 25 treatments to reduce antidemocratic attitudes and partisan animosity" **Journal:** Science 386, eadh4764 **Study type:** Large-scale between-subjects randomized experiment (N=32,059) with preregistered 2-week follow-up (N=8,644). | # | Term | Line | Quote | Attrition-relevant? | | ---- | --------- | ----------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | 1 | attrit | 417 | "possible demand effects, robustness of results for outcomes with left-censored distributions, **differential attrition**, multiple testing, and measurement -- in detail in the supplementary materials, section S13" | **YES** — sole explicit mention; defers all detail to supplement S13 | | 2 | follow-up | 249 | "We tested the durability of experimental effects ~2 weeks later in a preregistered **follow-up** survey (n = 8644 participants; supplementary materials, section S9)." | Borderline — reports follow-up N but does not discuss the ~73% non-response (32,059 → 8,644) as attrition | | 3 | follow-up | 273 | "effects of treatments on support for undemocratic practices and partisan violence in the 2-week **follow-up** survey" | No — reports durability results | | 4 | exclud | 512 | "we did not include independents who reported not being closer to one of the parties" | Borderline — eligibility restriction by design | | 5 | exclud | 512 | "We **exclude** 2020 because of mode differences" | No — excluding a data year from trend analysis | | 6 | exclud | 505 | "**Excluding** this treatment from the analysis..." | No — sensitivity analysis removing a condition | | 7 | fail | 361, 373, 87, 411 | "prior research...failed to find effects" / "public fails to agree" / "democracy...failing" | No — incidental usage | | 8 | panel | 121, 417 | "nonprobability opt-in internet **panels**" | No — sampling frame | | 9 | complete | 435, 295 | "participants **completed** measures" / "**complete** wording of these items" | No — procedural | | 10 | left | 53, 417 | "left panel" / "**left-censored** distributions" | No — figure layout / statistical term | | 11 | censor | 417 | "**left-censored** distributions" | No — floor effects, not attrition | | 12 | abandon | 188 | "Americans never **abandon** the principles" | No — treatment content | | 13 | lost | 367 | "their party **lost** was illegitimate" | No — election context | **Assessment: MINIMAL attrition handling in main text.** Only one mention of "differential attrition" as a listed concern deferred entirely to supplementary section S13. The follow-up survey retained only ~27% of the original sample (8,644 of 32,059) but this massive loss is not discussed as attrition anywhere in the main text. No attrition rates, no CONSORT diagram, no balance checks, no differential analysis appear in the main paper. --- ## Overall Summary | Paper | Attrition mentioned? | Attrition analyzed? | Key evidence | Rating | | ----------------------------- | ------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | --------------- | | **Lee et al. (2022)** | Barely — two sentences in Reporting Summary | No analysis performed | "No participants dropped out" and "We did not exclude any data points" — unsubstantiated claims | **Minimal** | | **Stagnaro & Amsalem (2025)** | Yes — multiple passages | Yes — within-wave rate, between-wave retention, balance checks, robustness subsetting | 2.96% within-wave attrition; 87% between-wave retention; balance checks (Supp. Note 13); subsetting robustness (Supp. Note 14); designed opt-out mechanism (Supp. Note 9) | **Substantial** | | **Vlasceanu et al. (2024)** | Partially — one passage | Minimal — acknowledges bias risk but no analysis | 29.2% excluded (83,927 → 59,440) via attention checks/WEPT demo; one sentence acknowledging selection bias; no differential analysis | **Limited** | | **Voelkel et al. (2024)** | Barely — one phrase | Deferred entirely to supplement S13 | "differential attrition" listed among concerns in one sentence; ~73% follow-up non-response not discussed; all detail in supplement | **Minimal** | ### Key findings: 1. **Stagnaro & Amsalem** is the only paper with meaningful attrition analysis in the main text: explicit attrition rates, a mitigation mechanism, balance checks, and robustness tests. Even so, full details are in supplementary notes. 2. **Vlasceanu et al.** has a 29.2% participant exclusion rate — the largest sample loss of any paper — but devotes only one sentence to acknowledging potential selection bias. No differential analysis by condition, country, or recruitment method is provided. 3. **Voelkel et al.** has a ~73% loss from main study to follow-up (32,059 → 8,644) but does not discuss this as attrition in the main text, mentioning "differential attrition" only as one item in a list of concerns deferred to supplementary materials. 4. **Lee et al.** claims zero dropout and zero exclusions in checklist responses but provides no evidence to support these claims. 5. None of the four papers include a CONSORT-style flow diagram in their main text.