Structural skew
Study 2a pilot
Marianna Zhang
2026-07-15
Overview
Project goals
The goal of this project is to establish if children and adults can adjust their generalizations about a social group to account for sampling skew.
Previously on..
In Studies 1a-c (boat studies), we found that 4yo struggle to adjust their generalizations against structural skew.
This result makes an interesting contrast compared to infant literature, where even infants suspend inference/generalization from a sample of marbles skewed by an agent’s preferences.
In Study 2, we use a paradigm involving agent preference to test if 4yo’s struggle has to do with structural vs agentic sources of skew, or if 4yo’s struggle has to do with adjusting vs suspending inference.
In an initial pilot with adults, our goal is to confirm that the paradigm does not confuse adult participants.
Results
Adults (n=14/condition) passed the ball training and familiarization checks, indicating that they knew what soccer and basketball balls were and that they knew Alex either had no preference (not skewed) or had a preference for one sport (skewed).
Nevertheless, participants in both conditions generalized from the sample in their prediction trials, predicting that most Gorps would like the sample-majority sport (i.e., the sport that the majority of Alex’s Gorp friends preferred). In explanations for their predictions, only 1 participant in the skewed condition reported reasoning about what Gorps might have liked independently of Alex’s selections; they ended up defaulting to the idea that Gorps would like both equally (either because of familiarization, or because of a true prior).
In addition, most participants in both conditions reported that Gorps on Gorp Planet would like the sample-majority sport “the same” or “more” than Alex’s Gorp friends. Only 1 participant in the skewed condition reported that Gorps on Gorp Planet would like the sample-majority sport “less” than the observed sample, although confusingly this person also predicted 3 out of 4 Gorps like the sample-majority sport and reported that they believed “more people liked [sample-majority sport] than [alternative sport]”.
Methods
Participants
Data was collected from 30 adults recruited via Prolific on Weds 7/15/2026 as a standard sample. Participants were required to be in the United States, fluent in English, and have not participated in any previous studies in this project.
Participants were paid $2.00 for an estimated 8 minute task. In fact, the study generally took about 8.5 minutes for participants.
| condition | participants |
|---|---|
| not_skewed | 14 |
| skewed | 14 |
The final sample included 28 adults (n = 14-14 in each of the 2 conditions).
Exclusion criteria
Participants were excluded if they failed the sound check or task check.
| Exclusion reasons | |||||
| n_collect | sound_check | check_task | n | n_excl | excl_rate |
|---|---|---|---|---|---|
| 30 | 1 | 1 | 28 | 2 | 6.67% |
Demographics
Age
| age | ||
| mean | sd | n |
|---|---|---|
| 40.39 | 10.30 | 28 |
Gender
| gender | n | prop |
|---|---|---|
| Female | 17 | 60.7% |
| Male | 11 | 39.3% |
Race
| race | n | prop |
|---|---|---|
| White, Caucasian, or European American | 22 | 78.6% |
| Black or African American | 2 | 7.1% |
| East Asian | 2 | 7.1% |
| White, Caucasian, or European American,Hispanic or Latino/a | 1 | 3.6% |
| White, Caucasian, or European American,Middle Eastern or North African | 1 | 3.6% |
Education
| education | n | prop |
|---|---|---|
| High school/GED | 5 | 17.9% |
| Some college | 7 | 25.0% |
| Bachelor's (B.A., B.S.) | 12 | 42.9% |
| Master's (M.A., M.S.) | 1 | 3.6% |
| Doctoral (Ph.D., J.D., M.D.) | 3 | 10.7% |
Procedure
This study was administered as a Qualtrics survey, and approved by the NYU IRB (IRB-FY2024-9169).
After providing their consent, participants completed a captcha and sound check, and were asked to watch videos sound on. Participants then watched the following videos in order:
In the warmup phase, to confirm participants’ understanding of balls and sports, participants heard the narrator label a soccer ball and a basketball, and were asked to click on each.
In the alternate counterbalance version, the left/right position of soccer and basketball buttons was switched on this and all questions in the study.
In the familiarization phase, participants were introduced to an agent called Alex (depicted using a photograph of a white female child), and learned how Alex chooses friends by watching her make friends at a playground.
Each trial showed pictures of two children (pictures matched on race and gender; races and genders varied across trials), one holding a soccer ball and another holding a basketball. Children were unique to each trial.
In the skewed condition, Alex approached the child holding the soccer ball on 5 out of 6 trials, and approached the child holding the basketball on the remaining oddball trial. Trial order was randomized, such that the oddball trial always appeared in 2nd, 3rd, 4th, or 5th position.
In the not skewed condition, Alex approached children of each sport on 3 out of 6 trials. Sport selections alternated, with the first selection being randomized.
In the alternate counterbalance version, the position of children was fixed, while Alex’s selections were switched, such that the skewed condition saw Alex approach mostly basketball.
As familiarization phase checks, participants were asked to (in the following fixed order):
Familiarization: friends check: Predict which child Alex might befriend between a novel soccer kid and a novel basketball kid, to confirm their understanding of the agent’s preference. The images used were fixed images of a Black girl holding a basketball (fixed), and another Black girl holding a soccer ball (fixed), their positions counterbalanced on screen.
After responding, participants in the skewed condition were told that Alex will probably choose the kid holding the soccer ball, because Alex likes soccer (or basketball, in the alternate counterbalance version). Participants in the not skewed condition were told, “it might be hard for Alex to choose, because Alex likes soccer and basketball”.
Familiarization: sport base rate check: Confirm their understanding of whether kids at the playground liked basketball, soccer, or both the same. After responding, all participants were told that kids at the playground liked both the same.
- In the sample observation phase, participants observed the same sample of Gorps that Alex befriends on Gorp Planet. In both conditions, Alex befriends 8 Gorps, 6 of which (fixed positions and colors) like soccer. The remaining 2 Gorps (fixed positions and colors: light pink, light yellow) like basketball. (In the alternate counterbalance version, the 6 like basketball, and the remaining 2 like soccer.)
- In the inference phase, participants had to make inferences about Gorps in general, after Alex left. They completed the below measures in fixed order:
Inference: prediction trials: As one of our dependent measures, participants were asked to predict the sport preferences of 4 novel Gorps (a brown, pink, purple, and teal Gorp in fixed order). These four trials were averaged into a prediction proportion for each participant.
For piloting purposes, participants were also asked how they decided their responses in the prediction task.
Inference: comparison forced-choice: As another dependent measure, adults were shown Alex’s Gorp friends again, and were asked to make a forced-choice comparison: whether Gorps on Gorp Planet like soccer (or, in counterbalanced version, basketball) less, the same, or more than Alex’s Gorp friends.
- Finally, participants were asked for any problems or confusion they had, what they thought the task was about, and demographic information.
Results
Ball training
After labeling, all participants correctly identified a soccer ball and a basketball.
Familiarization: friends check
Both conditions largely passed the friends check.
Participants in the skewed condition understood Alex would befriend kids who prefered one sport (aligned with their counterbalance condition), while participants in the not skewed condition appeared more mixed.
All participants received information about the expected response after their response.
Familiarization: sport base rate check
Participants were asked to recall that Alex met many kids at the park today, and to recall which sport more of the kids on the playground liked: basketball, soccer, or did they like them the same.
The correct answer to this question is “the same”, as every trial showed a basketball kid and a soccer kid.
Participants mostly answered this question correctly; participants who gave incorrect answers were still included. All participants received the correct answer after their response.
Inference: prediction trials
Participants were asked to predict the sport preferences (soccer or basketball) of 4 novel group members (a brown, pink, purple, and teal Gorp in fixed order).
First prediction trial:
In the skewed condition, only 1 participant (participant 16) made at-chance predictions. They reported, “since I didn’t get the chance to see alex friending gorps like i did in the first set, it was fairly difficult to tell if gorps leaned more towards soccer or basket ball. so I went with the idea that they like both equally. so i decided to choose that two liked soccer and the other two liked basket ball, since everyone has their own preferences and I doubt it would change on a different planet.”
This participant’s explanation is the only one that mentions anything like accounting for Alex’s preferences, and it seems they fell back on an at-chance prior about people liking soccer and basketball equally (unclear if their prior is from familiarization, or pre-existing).
Confusingly, this participant later on went to select that Gorps on Gorp Planet like the sample-majority sport “the same” as Alex’s Gorp friends.
Inference: comparison forced-choice
Participants were shown Alex’s Gorp friends again, and were asked to infer whether Gorps on Gorp Planet liked the sample-majority sport “less”, “the same”, or “more” than Alex’s Gorp friends.
Only 1 participant (participant 30) in the skewed condition reported that Gorps on Gorp Planet would like the sample-majority sport “less” than the observed sample. Confusingly, this person previously predicted that 3 out of 4 Gorps like the sample-majority sport and reported that they believed “more people liked [sample-majority sport] than [alternative sport]”.
Session info
## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Tahoe 26.5
##
## Matrix products: default
## BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## time zone: America/New_York
## tzcode source: internal
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] effectsize_1.0.1 emmeans_2.0.1 nnet_7.3-20 lmerTest_3.2-0
## [5] lme4_1.1-38 Matrix_1.7-4 car_3.1-3 carData_3.0-5
## [9] viridis_0.6.5 viridisLite_0.4.2 ggtext_0.1.2 lubridate_1.9.4
## [13] forcats_1.0.1 stringr_1.6.0 dplyr_1.1.4 purrr_1.2.1
## [17] readr_2.1.6 tidyr_1.3.2 tibble_3.3.1 ggplot2_4.0.1
## [21] tidyverse_2.0.0 gt_1.3.0 scales_1.4.0 janitor_2.2.1
## [25] here_1.0.2 knitr_1.51
##
## loaded via a namespace (and not attached):
## [1] Rdpack_2.6.5 gridExtra_2.3 sandwich_3.1-1
## [4] rlang_1.1.7 magrittr_2.0.4 multcomp_1.4-29
## [7] snakecase_0.11.1 otel_0.2.0 compiler_4.5.2
## [10] systemfonts_1.3.1 vctrs_0.7.1 pkgconfig_2.0.3
## [13] crayon_1.5.3 fastmap_1.2.0 backports_1.5.0
## [16] labeling_0.4.3 rmarkdown_2.30 tzdb_0.5.0
## [19] nloptr_2.2.1 ragg_1.5.0 bit_4.6.0
## [22] xfun_0.56 cachem_1.1.0 jsonlite_2.0.0
## [25] cluster_2.1.8.1 parallel_4.5.2 R6_2.6.1
## [28] bslib_0.10.0 stringi_1.8.7 RColorBrewer_1.1-3
## [31] rpart_4.1.24 boot_1.3-32 jquerylib_0.1.4
## [34] numDeriv_2016.8-1.1 estimability_1.5.1 Rcpp_1.1.1
## [37] zoo_1.8-15 base64enc_0.1-3 parameters_0.28.3
## [40] splines_4.5.2 timechange_0.3.0 tidyselect_1.2.1
## [43] rstudioapi_0.18.0 abind_1.4-8 yaml_2.3.12
## [46] codetools_0.2-20 lattice_0.22-7 withr_3.0.2
## [49] bayestestR_0.17.0 S7_0.2.1 coda_0.19-4.1
## [52] evaluate_1.0.5 foreign_0.8-90 survival_3.8-6
## [55] xml2_1.5.2 pillar_1.11.1 checkmate_2.3.3
## [58] reformulas_0.4.3.1 insight_1.4.5 generics_0.1.4
## [61] vroom_1.6.7 rprojroot_2.1.1 hms_1.1.4
## [64] minqa_1.2.8 xtable_1.8-4 glue_1.8.0
## [67] Hmisc_5.2-5 tools_4.5.2 data.table_1.18.0
## [70] fs_1.6.6 mvtnorm_1.3-3 grid_4.5.2
## [73] rbibutils_2.4.1 datawizard_1.3.0 colorspace_2.1-2
## [76] nlme_3.1-168 htmlTable_2.4.3 Formula_1.2-5
## [79] cli_3.6.5 textshaping_1.0.4 ggthemes_5.2.0
## [82] gtable_0.3.6 sass_0.4.10 digest_0.6.39
## [85] TH.data_1.1-5 htmlwidgets_1.6.4 farver_2.1.2
## [88] htmltools_0.5.9 lifecycle_1.0.5 gridtext_0.1.5
## [91] bit64_4.6.0-1 MASS_7.3-65