Structural skew
Study 2b
Marianna Zhang
2026-08-26
Overview
Project goals
The goal of this project is to establish if children and adults can adjust their generalizations about a social group to account for sampling skew.
Previously on..
In the previous study (2a), the sport preferences observed among a sample of novel group members was skewed by an agent’s probabilistic preference for befriending those who like one sport over another. Although adult participants understood the agent’s preferences in the skewed condition, they nonetheless generalized from the sample to the population, similarly to as when the sample was not skewed (a Bayesian analysis indicated moderate evidence in favor of no condition difference). This result suggests adults fail to account for probabilistic skew in social generalization.
As an interesting contrast, infants succeed in discounting samples skewed by an agent’s deterministic preference in physical generalization (Xu & Denison, 2009). Does the difference lie in adjusting inference from probabilistically skewed samples, vs discounting deterministically skewed samples entirely? To examine this question, in this study, the agent’s preference was made deterministic (similar to the agent’s deterministic preference in the infant marble sampling task), and the sample accordingly was made uniform to align with the agent’s selection process.
Results
Adult participants passed all the checks, indicating that they knew what soccer and basketball balls were, that they knew Alex either had no preference (not skewed) or had a preference for one sport (skewed) among kids, and among Gorps.
In contrast to Study 2a, where adults failed to account for probabilistic skew, adult participants here showed sensitivity to deterministic skew. Specifically:
Adults showed less generalization of the sample’s sport in the skewed condition than in the not skewed condition. However, they still fell short of the model prediction.
When comparing Alex’s Gorp friends and Gorps on Gorp Planet, adults in the skewed condition were more likely to select Alex’s Gorp friends as liking soccer more, and less likely to select that they like soccer the same. Note that this is a pretty easy response here, as all of Alex’s Gorp friends liked soccer, so it is impossible to like soccer proportionally more than Alex’s Gorp friends.
As a result, it appears that adults do account for skewed sampling when it is entirely deterministic, consistent with the infant sampling studies.
Methods
Participants
Data was collected from 201 adults recruited via Prolific on Weds 8/26/2026 as a standard sample. Participants were required to be in the United States and have not participated in any previous studies in this project.
Participants were paid $2.25 for an estimated 9 minute task. In fact, the study generally took about 11 minutes for participants.
| condition | participants |
|---|---|
| not_skewed | 98 |
| skewed | 99 |
The final sample included 197 adults (n = 98-99 in each of the 2 conditions).
Exclusion criteria
Participants were excluded if they failed the sound check, task check, or admitted to AI usage despite pledging to not use AI.
| Exclusion reasons | |||||
| n_collect | sound_check | check_task | n | n_excl | excl_rate |
|---|---|---|---|---|---|
| 201 | 1 | 4 | 197 | 4 | 1.99% |
Demographics
Age
| age | ||
| mean | sd | n |
|---|---|---|
| 41.37 | 13.08 | 197 |
Gender
| gender | n | prop |
|---|---|---|
| Female | 108 | 54.8% |
| Male | 84 | 42.6% |
| Prefer not to specify | 3 | 1.5% |
| Non-binary | 2 | 1.0% |
Race
| race | n | prop |
|---|---|---|
| White, Caucasian, or European American | 128 | 65.0% |
| Black or African American | 22 | 11.2% |
| Hispanic or Latino/a | 8 | 4.1% |
| East Asian | 7 | 3.6% |
| Prefer not to specify | 5 | 2.5% |
| White, Caucasian, or European American,Hispanic or Latino/a | 5 | 2.5% |
| South or Southeast Asian | 4 | 2.0% |
| White, Caucasian, or European American,Black or African American | 4 | 2.0% |
| Middle Eastern or North African | 2 | 1.0% |
| Native American, American Indian, or Alaska Native | 2 | 1.0% |
| White, Caucasian, or European American,East Asian | 2 | 1.0% |
| White, Caucasian, or European American,Hispanic or Latino/a,Native American, American Indian, or Alaska Native | 2 | 1.0% |
| Foundational Black American | 1 | 0.5% |
| Israeli | 1 | 0.5% |
| South or Southeast Asian,East Asian | 1 | 0.5% |
| White, Caucasian, or European American,Native American, American Indian, or Alaska Native | 1 | 0.5% |
| White, Caucasian, or European American,South or Southeast Asian | 1 | 0.5% |
| persian/indian | 1 | 0.5% |
Education
| education | n | prop |
|---|---|---|
| High school/GED | 21 | 10.7% |
| Some college | 45 | 22.8% |
| Bachelor's (B.A., B.S.) | 97 | 49.2% |
| Master's (M.A., M.S.) | 27 | 13.7% |
| Doctoral (Ph.D., J.D., M.D.) | 3 | 1.5% |
| Prefer not to specify | 3 | 1.5% |
| NA | 1 | 0.5% |
Procedure
This study was administered as a Qualtrics survey, and approved by the NYU IRB (IRB-FY2024-9169).
After providing their consent, participants completed a captcha and sound check, and were asked to watch videos sound on. Participants then watched the following videos in order:
In the warmup phase, to confirm participants’ understanding of balls and sports, participants heard the narrator label a soccer ball and a basketball, and were asked to click on each.
In the alternate counterbalance version, the left/right position of soccer and basketball buttons was switched on this and all questions in the study.
In the familiarization phase, participants were introduced to an agent called Alex (depicted using a photograph of a white female child), and learned how Alex chooses friends by watching her make friends at a playground.
Each trial showed pictures of two children (pictures matched on race and gender; races and genders varied across trials), one holding a soccer ball and another holding a basketball. Children were unique to each trial.
In the skewed condition, Alex approached the child holding the soccer ball on 5 out of 6 trials, and approached the child holding the basketball on the remaining oddball trial. Trial order was randomized, such that the oddball trial always appeared in 2nd, 3rd, 4th, or 5th position.
In the not skewed condition, Alex approached children of each sport on 3 out of 6 trials. Sport selections alternated, with the first selection being randomized.
In the alternate counterbalance version, the position of children was fixed, while Alex’s selections were switched, such that the skewed condition saw Alex approach mostly basketball.
- As familiarization phase checks, participants were asked to (in the following fixed order):
- Familiarization: friends check: Predict which child Alex might befriend between a novel soccer kid and a novel basketball kid, to confirm their understanding of the agent’s preference. The images used were fixed images of a Black girl holding a basketball (fixed), and another Black girl holding a soccer ball (fixed), their positions counterbalanced on screen.
After responding, participants in the skewed condition were told that Alex will probably choose the kid holding the soccer ball, because Alex likes soccer (or basketball, in the alternate counterbalance version). Participants in the not skewed condition were told, “it might be hard for Alex to choose, because Alex likes soccer and basketball”.
- Familiarization: sport base rate check: Confirm their understanding of whether kids at the playground liked basketball, soccer, or both the same. After responding, all participants were told that kids at the playground liked both the same.
- In the sample observation phase, participants observed the same sample of Gorps that Alex befriends on Gorp Planet. In both conditions, Alex befriends 8 Gorps, 6 of which (fixed positions and colors) like soccer. The remaining 2 Gorps (fixed positions and colors: light pink, light yellow) like basketball. (In the alternate counterbalance version, the 6 like basketball, and the remaining 2 like soccer.)
- In the inference phase, participants had to make inferences about Gorps in general, after Alex left. They completed the below measures in fixed order:
Inference: prediction trials: As one of our dependent measures, participants were asked to predict the sport preferences of 4 novel Gorps (a brown, pink, purple, and teal Gorp in fixed order). These four trials were averaged into a prediction proportion for each participant.
For piloting purposes, participants were also asked how they decided their responses in the prediction task.
Inference: comparison forced-choice: As another dependent measure, adults were shown Alex’s Gorp friends again, and were asked to make a forced-choice comparison: Who liked soccer more? Alex’s Gorp friends, Gorps on Gorp Planet, or do they like soccer the same. In the counterbalanced version, this question asked about basketball.
For piloting purposes, participants were also asked how they decided their response to this question.
- As a final friends check, participants were asked which Gorp Alex might befriend between a soccer Gorp and a basketball Gorp, to check whether they understood the agent’s preference for kids extended to Gorps. The images used were fixed images of a pink Gorp and a purple Gorp from the prediction phase (which had soccer ball and which had basketball was counterbalanced).
- Finally, participants were asked for any problems or confusion they had, what they thought the task was about, and demographic information.
Checks
Ball training
After labeling, all participants correctly identified a soccer ball and a basketball.
Familiarization: friends check (kids)
Both conditions largely passed the friends check.
Participants in the skewed condition understood Alex would befriend kids who prefered one sport (aligned with their counterbalance condition), while participants in the not skewed condition appeared more mixed.
All participants received information about the expected response after their response.
Familiarization: sport base rate check
Participants were asked to recall that Alex met many kids at the park today, and to recall which sport more of the kids on the playground liked: basketball, soccer, or did they like them the same.
The correct answer to this question is “the same”, as every trial showed a basketball kid and a soccer kid.
Participants mostly answered this question correctly; participants who gave incorrect answers were still included. All participants received the correct answer after their response.
Agent friends check (Gorps)
Both conditions largely passed the agent friends check.
Participants in the skewed condition understood Alex would befriend the Gorp who preferred one sport (aligned with their counterbalance condition), while participants in the not skewed condition appeared more mixed.
Results
Inference: prediction trials
Participants were asked to predict the sport preferences (soccer or basketball) of 4 novel group members (a brown, pink, purple, and teal Gorp in fixed order).
First prediction trial:
## Generalized linear mixed model fit by maximum likelihood (Laplace
## Approximation) [glmerMod]
## Family: binomial ( logit )
## Formula: infer_sport ~ condition + (1 | participant)
## Data: data_tidy
##
## AIC BIC logLik -2*log(L) df.resid
## 775.6 789.6 -384.8 769.6 785
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.6701 0.2476 0.2476 0.4601 1.0861
##
## Random effects:
## Groups Name Variance Std.Dev.
## participant (Intercept) 1.12 1.058
## Number of obs: 788, groups: participant, 197
##
## Fixed effects:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 2.5331 0.2617 9.679 < 0.0000000000000002 ***
## conditionskewed -1.7632 0.2860 -6.164 0.000000000709 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## conditnskwd -0.824Inference: comparison forced-choice
Participants were shown Alex’s Gorp friends again, and were asked to infer whether Gorps on Gorp Planet liked the sample-majority sport “less”, “the same”, or “more” than Alex’s Gorp friends.
## # weights: 9 (4 variable)
## initial value 216.426621
## iter 10 value 165.428594
## final value 165.428588
## converged
## # weights: 6 (2 variable)
## initial value 216.426621
## iter 10 value 171.253990
## iter 10 value 171.253990
## iter 10 value 171.253990
## final value 171.253990
## converged## Analysis of Deviance Table (Type II tests)
##
## Response: infer_comp
## LR Chisq Df Pr(>Chisq)
## condition 11.651 2 0.002952 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Participants made different responses to the comparison forced-choice question depending on condition (LR Chisq(2) = 11.65, p = 0.003).
##
## Call:
## glm(formula = infer_comp_pop ~ condition, family = binomial,
## data = .)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -2.7300 0.4213 -6.479 0.0000000000922 ***
## conditionskewed -0.2038 0.6230 -0.327 0.744
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 84.851 on 196 degrees of freedom
## Residual deviance: 84.743 on 195 degrees of freedom
## AIC: 88.743
##
## Number of Fisher Scoring iterations: 5##
## Call:
## glm(formula = infer_comp_same ~ condition, family = binomial,
## data = .)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) 0.4140 0.2064 2.006 0.04486 *
## conditionskewed -0.9302 0.2928 -3.177 0.00149 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 272.97 on 196 degrees of freedom
## Residual deviance: 262.61 on 195 degrees of freedom
## AIC: 266.61
##
## Number of Fisher Scoring iterations: 4##
## Call:
## glm(formula = dv_comp_sample ~ condition, family = binomial,
## data = .)
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.6779 0.2137 -3.171 0.00152 **
## conditionskewed 0.9833 0.2950 3.333 0.00086 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 271.63 on 196 degrees of freedom
## Residual deviance: 260.18 on 195 degrees of freedom
## AIC: 264.18
##
## Number of Fisher Scoring iterations: 4Specifically, participants in the skewed condition were more likely to say that Alex’s Gorp friends liked the sample-exclusive sport more (b = 0.98, z = 3.33, p < .001), and less likely to say that Alex’s Gorp friends and Gorps on Gorp Planet liked the sample-exclusive sport “the same” (b = -0.93, z = -3.18, p = 0.001). There was no significant difference in saying that Gorps on Gorp Planet liked the sample-exclusive sport more (b = -0.2, z = -0.33, p = 0.744), which was an infrequent response in both conditions.
Supplementary results
Inference: comparison forced-choice vs prediction
Participants’ responses to the comparison forced-choice question did not appear to be clearly related to their earlier responses to the prediction trials.
The comparison forced-choice asked who likes the sample-majority sport more: Alex’s Gorp friends, Gorps on Gorp Planet, or if they like it the same.
Since 8 out of 8 of Alex’s Gorp friends liked the sample-majority sport, matching the sample proportion would correspond with predictions of 1 (100%). As a result:
predictions of 0, 0.25, and 0.5, 0.75 should correspond with responding “Alex’s Gorp friends”
predictions of 1 should correspond with responding “the same”
Session info
## R version 4.5.2 (2025-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Tahoe 26.5.2
##
## Matrix products: default
## BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.1
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## time zone: America/New_York
## tzcode source: internal
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] car_3.1-3 carData_3.0-5 nnet_7.3-20
## [4] tidybayes_3.0.7 broom.mixed_0.2.9.6 brms_2.23.0
## [7] Rcpp_1.1.2 lmerTest_3.2-0 lme4_2.0-6
## [10] Matrix_1.7-6 viridis_0.6.5 viridisLite_0.4.2
## [13] ggtext_0.1.2 lubridate_1.9.4 forcats_1.0.1
## [16] stringr_1.6.0 dplyr_1.1.4 purrr_1.2.1
## [19] readr_2.1.6 tidyr_1.3.2 tibble_3.3.1
## [22] ggplot2_4.0.3 tidyverse_2.0.0 gt_1.3.0
## [25] scales_1.4.0 janitor_2.2.1 here_1.0.2
## [28] knitr_1.51
##
## loaded via a namespace (and not attached):
## [1] RColorBrewer_1.1-3 tensorA_0.36.2.1 rstudioapi_0.18.0
## [4] jsonlite_2.0.0 magrittr_2.0.4 TH.data_1.1-5
## [7] estimability_1.5.1 farver_2.1.2 nloptr_2.2.1
## [10] rmarkdown_2.30 fs_1.6.6 ragg_1.5.0
## [13] vctrs_0.7.1 minqa_1.2.8 base64enc_0.1-3
## [16] htmltools_0.5.9 distributional_0.6.0 broom_1.0.12
## [19] Formula_1.2-5 sass_0.4.10 parallelly_1.46.1
## [22] bslib_0.10.0 htmlwidgets_1.6.4 sandwich_3.1-1
## [25] emmeans_2.0.1 zoo_1.8-15 cachem_1.1.0
## [28] lifecycle_1.0.5 pkgconfig_2.0.3 R6_2.6.1
## [31] fastmap_1.2.0 rbibutils_2.4.1 future_1.69.0
## [34] snakecase_0.11.1 digest_0.6.39 numDeriv_2016.8-1.1
## [37] colorspace_2.1-2 furrr_0.3.1 rprojroot_2.1.1
## [40] textshaping_1.0.4 Hmisc_5.2-5 labeling_0.4.3
## [43] timechange_0.3.0 abind_1.4-8 compiler_4.5.2
## [46] bit64_4.6.0-1 withr_3.0.2 htmlTable_2.4.3
## [49] S7_0.2.1 backports_1.5.0 MASS_7.3-65
## [52] loo_2.9.0 tools_4.5.2 foreign_0.8-90
## [55] otel_0.2.0 glue_1.8.0 nlme_3.1-168
## [58] gridtext_0.1.5 grid_4.5.2 checkmate_2.3.3
## [61] cluster_2.1.8.1 generics_0.1.4 gtable_0.3.6
## [64] tzdb_0.5.0 data.table_1.18.0 hms_1.1.4
## [67] xml2_1.5.2 pillar_1.11.1 ggdist_3.3.3
## [70] vroom_1.6.7 posterior_1.6.1 splines_4.5.2
## [73] lattice_0.22-7 survival_3.8-6 bit_4.6.0
## [76] tidyselect_1.2.1 reformulas_0.4.3.1 arrayhelpers_1.1-2
## [79] gridExtra_2.3 xfun_0.56 bridgesampling_1.2-1
## [82] matrixStats_1.5.0 stringi_1.8.7 yaml_2.3.12
## [85] boot_1.3-32 evaluate_1.0.5 codetools_0.2-20
## [88] cli_3.6.5 rpart_4.1.24 RcppParallel_5.1.11-1
## [91] xtable_1.8-4 systemfonts_1.3.1 Rdpack_2.6.5
## [94] jquerylib_0.1.4 globals_0.18.0 coda_0.19-4.1
## [97] svUnit_1.0.8 parallel_4.5.2 rstantools_2.6.0
## [100] bayesplot_1.15.0 Brobdingnag_1.2-9 listenv_0.10.0
## [103] ggthemes_5.2.0 mvtnorm_1.3-3 crayon_1.5.3
## [106] rlang_1.1.7 multcomp_1.4-29