Compgenerics study 6 (preregistered) analysis
Marianna Zhang
2026-04-21
Summary
This study was a preregistered replication of a prior CogSci study, with book covers removed and statements in randomized order.
We replicated our prior findings where hearing generic statements about a novel social group increased the inferred inductive potential of the group (relative to baseline), while hearing specific statements decreased inductive potential.
Methods
Participants
| n_collect | attn_check | AI | check_task | n | n_excl | excl_rate |
|---|---|---|---|---|---|---|
| 452 | 6 | 4 | 1 | 444 | 8 | 0.01769912 |
Data was collected from 452 adults via Prolific on Monday 4/21/26 - Tuesday 4/22/2026. Participants required to be in the United States, fluent in English, and having not participated in prior studies under this protocol. Participants were paid $2.00 for an estimated 8 minute task.
| condition | n |
|---|---|
| generic | 147 |
| baseline | 152 |
| specific | 145 |
Exclusion criteria
Of the recruited participants, 8 (1.8%) were excluded for meeting at least 1 of the following exclusion criteria:
failing the attention check (i.e., did not select 100% on slider when asked to during induction task) (n = 6 participants)
admitting to use of AI after being explicitly informed use was prohibited (n = 4 participants)
failing the task check (n = 1 participants)
Participants who failed the sound check were included, since a few participants mentioned technical difficulties with the Qualtrics automatically progressing past that video.
Demographics
We used the Prolific representative sample feature to recruit a sample representative on sex, age, and ethnicity (simplified US Census categories).
Age
| age | ||
| mean | sd | n |
|---|---|---|
| 45.42 | 16.06 | 444 |
- The sample skewed young in age.
Gender
| gender | n | prop |
|---|---|---|
| Female | 225 | 50.7% |
| Male | 209 | 47.1% |
| Non-binary | 6 | 1.4% |
| Prefer not to specify | 2 | 0.5% |
| Genderqueer | 1 | 0.2% |
| really sick of scoail construct of gender. I am who i am and like what i like. i refuse to participate for any team... | 1 | 0.2% |
- The sample reflected the diversity of the gender identities in the US.
Race
| race | n | prop |
|---|---|---|
| White, Caucasian, or European American | 271 | 61.0% |
| Black or African American | 51 | 11.5% |
| Hispanic or Latino/a | 35 | 7.9% |
| East Asian | 13 | 2.9% |
| South or Southeast Asian | 12 | 2.7% |
| White, Caucasian, or European American,Hispanic or Latino/a | 7 | 1.6% |
| White, Caucasian, or European American,Native American, American Indian, or Alaska Native | 7 | 1.6% |
| Middle Eastern or North African | 5 | 1.1% |
| Native American, American Indian, or Alaska Native | 5 | 1.1% |
| White, Caucasian, or European American,South or Southeast Asian | 5 | 1.1% |
| Prefer not to specify | 4 | 0.9% |
| White, Caucasian, or European American,Black or African American | 3 | 0.7% |
| South or Southeast Asian,East Asian | 2 | 0.5% |
| White, Caucasian, or European American,Black or African American,Native American, American Indian, or Alaska Native | 2 | 0.5% |
| White, Caucasian, or European American,East Asian | 2 | 0.5% |
| White, Caucasian, or European American,Hispanic or Latino/a,Black or African American | 2 | 0.5% |
| White, Caucasian, or European American,Hispanic or Latino/a,Native American, American Indian, or Alaska Native | 2 | 0.5% |
| White, Caucasian, or European American,Middle Eastern or North African | 2 | 0.5% |
| Black or African American,Native American, American Indian, or Alaska Native | 1 | 0.2% |
| Cape Verdean | 1 | 0.2% |
| European American | 1 | 0.2% |
| HUMAN RACE | 1 | 0.2% |
| Hispanic or Latino/a,Black or African American | 1 | 0.2% |
| Hispanic or Latino/a,Black or African American,Native American, American Indian, or Alaska Native | 1 | 0.2% |
| Hispanic or Latino/a,East Asian | 1 | 0.2% |
| Mixed | 1 | 0.2% |
| Multi-race | 1 | 0.2% |
| Native Hawaiian or other Pacific Islander | 1 | 0.2% |
| White, Caucasian, or European American,Black or African American,East Asian | 1 | 0.2% |
| White, Caucasian, or European American,Native American, American Indian, or Alaska Native,East Asian | 1 | 0.2% |
| White, Caucasian, or European American,Native Hawaiian or other Pacific Islander | 1 | 0.2% |
| mixed race | 1 | 0.2% |
- The sample was also racially diverse, with White Americans slightly overrepresented and Hispanic Americans undererepresented.
Education
| education | n | prop |
|---|---|---|
| Less than high school | 4 | 0.9% |
| High school/GED | 62 | 14.0% |
| Some college | 114 | 25.7% |
| Bachelor's (B.A., B.S.) | 180 | 40.5% |
| Master's (M.A., M.S.) | 62 | 14.0% |
| Doctoral (Ph.D., J.D., M.D.) | 18 | 4.1% |
| Prefer not to specify | 4 | 0.9% |
- A slight majority of the sample had completed at least a college education.
Behavioral analyses
Large circles with bars indicate means with 95% confidence intervals.
Small dots indicate individual trial-level responses.
## Analysis of Deviance Table (Type II Wald chisquare tests)
##
## Response: prevalence
## Chisq Df Pr(>Chisq)
## condition 81.918 2 < 0.00000000000000022 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1## contrast estimate SE df z.ratio p.value
## generic - baseline 0.621 0.113 Inf 5.481 <0.0001
## generic - specific 1.031 0.115 Inf 8.981 <0.0001
## baseline - specific 0.410 0.114 Inf 3.602 0.0003
##
## Results are given on the log odds ratio (not the response) scale.As predicted, participants gave different prevalence estimates in the different conditions (Chisq(2) = 81.92, p < .001).
Specifically, participants in the generic condition inferred higher prevalence of test features than participants in the baseline condition (z = 5.48, p < .001), while participants in the specific condition inferred lower prevalence of test features than participants in the baseline condition (z = 3.6, p < .001).
Model comparison
Footnotes
## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Tahoe 26.5.2
##
## Matrix products: default
## BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## time zone: America/New_York
## tzcode source: internal
##
## attached base packages:
## [1] grid stats graphics grDevices utils datasets methods
## [8] base
##
## other attached packages:
## [1] emmeans_2.0.1 car_3.1-3 carData_3.0-5 glmmTMB_1.1.14
## [5] lubridate_1.9.4 forcats_1.0.1 stringr_1.6.0 dplyr_1.1.4
## [9] purrr_1.2.1 readr_2.1.6 tidyr_1.3.2 tibble_3.3.1
## [13] ggplot2_4.0.3 tidyverse_2.0.0 gt_1.3.0 scales_1.4.0
## [17] janitor_2.2.1 here_1.0.2
##
## loaded via a namespace (and not attached):
## [1] Rdpack_2.6.5 gridExtra_2.3 sandwich_3.1-1
## [4] rlang_1.1.7 magrittr_2.0.4 multcomp_1.4-29
## [7] snakecase_0.11.1 otel_0.2.0 compiler_4.5.2
## [10] mgcv_1.9-4 systemfonts_1.3.1 vctrs_0.7.1
## [13] pkgconfig_2.0.3 crayon_1.5.3 fastmap_1.2.0
## [16] backports_1.5.0 labeling_0.4.3 rmarkdown_2.30
## [19] tzdb_0.5.0 nloptr_2.2.1 ragg_1.5.0
## [22] bit_4.6.0 xfun_0.56 cachem_1.1.0
## [25] jsonlite_2.0.0 parallel_4.5.2 cluster_2.1.8.1
## [28] R6_2.6.1 bslib_0.10.0 stringi_1.8.7
## [31] RColorBrewer_1.1-3 boot_1.3-32 rpart_4.1.24
## [34] jquerylib_0.1.4 numDeriv_2016.8-1.1 estimability_1.5.1
## [37] Rcpp_1.1.2 knitr_1.51 zoo_1.8-15
## [40] base64enc_0.1-3 Matrix_1.7-6 splines_4.5.2
## [43] nnet_7.3-20 timechange_0.3.0 tidyselect_1.2.1
## [46] rstudioapi_0.18.0 abind_1.4-8 yaml_2.3.12
## [49] TMB_1.9.19 codetools_0.2-20 lattice_0.22-7
## [52] withr_3.0.2 S7_0.2.1 coda_0.19-4.1
## [55] evaluate_1.0.5 foreign_0.8-90 survival_3.8-6
## [58] xml2_1.5.2 pillar_1.11.1 checkmate_2.3.3
## [61] reformulas_0.4.3.1 generics_0.1.4 vroom_1.6.7
## [64] rprojroot_2.1.1 hms_1.1.4 minqa_1.2.8
## [67] xtable_1.8-4 glue_1.8.0 Hmisc_5.2-5
## [70] tools_4.5.2 data.table_1.18.0 lme4_2.0-6
## [73] fs_1.6.6 mvtnorm_1.3-3 rbibutils_2.4.1
## [76] colorspace_2.1-2 nlme_3.1-168 htmlTable_2.4.3
## [79] Formula_1.2-5 cli_3.6.5 textshaping_1.0.4
## [82] ggthemes_5.2.0 gtable_0.3.6 sass_0.4.10
## [85] digest_0.6.39 TH.data_1.1-5 htmlwidgets_1.6.4
## [88] farver_2.1.2 htmltools_0.5.9 lifecycle_1.0.5
## [91] bit64_4.6.0-1 MASS_7.3-65