Overview

Project goals

The goals of this project are to establish:

if children and adults generalize from a sample to a social group from a sample that is, unbeknownst to them, structurally skewed, resulting in inaccurate beliefs about the group
if children and adults can adjust their generalization from a sample to a social group to account for the fact that the sample was skewed by a structural process

This study focuses on question (2) in adults.

Previously on..

In Study 1a, we found that adults estimated the height of a novel social group to be taller after seeing the exact same sample of group members be generated from a skewed sampling process (a short boat cut many Zarpies off, with a few making it on by scrunching) versus a not skewed sampling process (a tall boat let all Zarpies board). This was true whether adults had information about the true height of the social group, or when that information was occluded.

This result could be explained by the main hypothesis that adults adjust their inferences about a population to account for structural skew, or by a simpler perceptual hypothesis that adults generalize from the sample to the population, weighting the scrunching Zarpies more because of their perceptual salience.

In addition, in study 1a, we found an unexpected pattern where adults also estimated the sample to be taller in the occluded skewed condition than the other conditions, even though the sample was the same across all conditions.

Study goals

The primary goal of this study was to rule out an alternative perceptual explanation for the results in Study 1a, and to replicate the unexpected pattern in Study 1a.

We re-ran the critical two conditions from Study 1a (occluded skewed and occluded not skewed), and added a third control condition to address the perceptual hypothesis (occluded not skewed control, where a short boat let all Zarpies board, with a few making it on by scrunching).

Results

We replicated the main effect from Study 1a, where participants in the skewed condition made taller population inferences than those in the not skewed condition.

However, this was qualified by a marginal interaction with DV order, where the effect only occurred when participants were asked about the population question after the sample question, rather than before (see population inference order effects).

We were able to rule out the simpler perceptual explanation, since participants in the skewed condition (short boat, some board, some scrunch) still made taller population inferences and were still more likely to explicitly say “taller” than the no skewed control condition (short boat, all board, some scrunch) (see [population representation] and explicit comparison).
However, the unusual phenomenon from Study 1a persisted, where participants in the skewed condition made taller sample inferences than those in the not skewed control condition, who in turn made taller sample inferences than those in the not skewed condition, despite all participants seeing the exact same sample (see sample representation).

This resulted in no interaction between condition and sample/population DV when comparing the skewed vs either of the not skewed conditions (ps = .16), and only a marginal difference between population and sample questions in the skewed condition (t(94) = -1.74, p = .085). (see sample vs population).
There’s no overall interaction between condition and DV order on the sample question, but there are some hints of DV order effects. When the sample question was asked first (immediately after the sample was observed), there was no effect of condition on the sample question (p = .47). However, when the population question was asked first, there was a significant effect of condition on the sample question (p = .003), such that those in the skewed condition made taller sample inferences than those in the not skewed control condition, who in turn made taller sample inferences than those in the not skewed condition (see sample representation order effects).
Asking about the population first, then the sample, might be difficult because adults have to retain their representation of the sample in memory over the intervening population question. This might lead responses on the population question to bleed over into responses on the sample question?

Methods

The study was preregistered on OSF.

Participants

Data was collected from 300 adults recruited via Prolific on Tues 9/2/2025 as a US representative sample on sex and ethnicity (simplified). Participants were required to be in the United States, fluent in English, and have not participated in any pilots of this study.

Participants were paid $2.50 for an estimated 9-11 minute task. In fact, the study generally took about 10 minutes for participants.

The final sample included 273 adults (n = 83-95 in each of the 3 conditions).

boarding	participants
not skewed	95
not skewed control	83
skewed	95

Exclusion criteria

27 participants (9.0% of all participants) were excluded for meeting at least 1 of the following exclusion criteria:

failing the sound check (n = 1 participants)
failing to select the correct task description (i.e., did not select “Watching videos about fictional people from an island”) (n = 3 participants)
failing the memory check (n = 5 participants)
failing the comprehension check (n = 19 participants)

Memory check

Participants overwhelmingly passed the memory check for the Quaffa boarding sequence, i.e., “no”, not all the Quaffas made it onto the boat.

Participants who made incorrect responses were excluded.

Comprehension check

Participants overwhelmingly passed the comprehension check for the Zarpie boarding sequence. Note the correct answer to this question depends on condition:

In the skewed condition, the correct answer is “no”, not all of the Zarpies made it onto the boat.
In the not skewed and not skewed control condition, the correct answer is “yes”, all of the Zarpies made it onto the boat.

Demographics

This study recruited a sample representative of the US on age, sex, and ethnicity (simplified US Census categories), using the representative sample feature on Prolific.

mean	sd	n
age
46.10	15.89	273

The sample was largely young and middle-aged.

gender	n	prop
Female	136	49.8%
Male	131	48.0%
Non-binary	5	1.8%
Prefer not to specify	1	0.4%

The sample was diverse in terms of gender identities in the US.

race	n	prop
White, Caucasian, or European American	173	63.4%
Black or African American	31	11.4%
Hispanic or Latino/a	24	8.8%
East Asian	10	3.7%
White, Caucasian, or European American,Hispanic or Latino/a	9	3.3%
South or Southeast Asian	8	2.9%
White, Caucasian, or European American,Black or African American	4	1.5%
White, Caucasian, or European American,South or Southeast Asian	3	1.1%
White, Caucasian, or European American,East Asian	2	0.7%
Black or African American,Native American, American Indian, or Alaska Native	1	0.4%
Hispanic or Latino/a,Native Hawaiian or other Pacific Islander	1	0.4%
Native American, American Indian, or Alaska Native	1	0.4%
Native Hawaiian or other Pacific Islander	1	0.4%
Prefer not to specify	1	0.4%
White, Caucasian, or European American,Middle Eastern or North African	1	0.4%
White, Caucasian, or European American,Native American, American Indian, or Alaska Native	1	0.4%
multi	1	0.4%
multiracial	1	0.4%

The sample was also racially diverse.

education	n	prop
Less than high school	2	0.7%
High school/GED	32	11.7%
Some college	83	30.4%
Bachelor's (B.A., B.S.)	120	44.0%
Master's (M.A., M.S.)	28	10.3%
Doctoral (Ph.D., J.D., M.D.)	5	1.8%
Prefer not to specify	3	1.1%

The sample was about evenly split in whether they had attained a college degree or not.

Procedure

This study was administered as a Qualtrics survey, and approved by the NYU IRB (IRB-FY2024-9169).

After providing their consent, participants completed a captcha and sound check, and were asked to watch videos sound on. Participants then watched the following videos in order:

In the prior setting and familiarization phase, participants saw a photorealistic picture of 5 human adults and then another picture of a different 5 adults appear on screen against a grid. These adults were all 10 gridline units tall.

Prior setting and familiarization.

In the boat training phase, participants were shown a parade of fictional animals attempting to board the boat, to illustrate how the boat works. In the skewed condition and not skewed control condition, the boat was 6 units tall. In the not skewed condiiton, the boat was 10 units tall.
- The boat height was specified to be accidental (“When the boat builders were building the boat, they started building the boat from the bottom, but ran out of the special wood they needed for the boat! So the boat ended up being this tall. It might be hard for anyone who is taller than the boat to get on the boat.”), to avoid any justificatory reasoning about the height of the boat being informative about the height of Zarpies or vice versa.
- To communicate how the boat functions to exclude those shorter than the boat, participants then watched a parade of 20 fictional animals (Quaffas, taken from Foster-Hanson et al., 2019) attempt to board the boat, one at a time, from shortest to tallest.
- The height of animals were scaled to the height of the boat, such that 10 animals were always shorter than the boat (these animals boarded successfully) and 10 animals were always taller than the boat (all but one were unable to board; the third quaffa successfully boards by bending its head).
Quaffas in the skewed condition. Note the Quaffas are short, since the skewed condition involves a short boat.
- Participants were asked a memory check: “Did all of the animals board the boat?” (yes/no), and received an affirmation (if they said “no”) or correction (if they said “yes”).
In the boat boarding phase, participants learned that Zarpies live on Zarpie island, and saw an island with many Zarpies overhead. Participants learned that all the grownup Zarpies’ names were put into a hat, and some of their names “were drawn out of a hat to try and visit us”. Participants saw then saw a parade of Zarpies attempt to board the boat to visit us, one at a time. Participants were told that they were all grown-up Zarpies.
- In all conditions, the heights of Zarpies were occluded behind a curtain that showed only their feet.
- In the not skewed condition, the boat is 10 units tall. 6 Zarpies attempt to board, all of whom successfully make it on (6 out of 6 successful = 100% successful). Of the 6 who board, none had to stoop to board.
- In the not skewed control condition, the boat is 6 units tall. 6 Zarpies attempt to board, all of whom successfully make it on (6 out of 6 successful = 100% successful). Of the 6 who board, 2 had to stoop to board.
- In the skewed condition, the boat is 6 units tall. 20 Zarpies attempt to board, 6 of whom successfully make it on (6 out of 16 successful = 30% successful). Of the 6 who make it on, 2 had to stoop to board.
After the boat boarding phase, participants were asked a comprehension check: “Did all of the Zarpies board the boat?” (yes/no), and received either an affirmation (if they said “no” in the skewed condition, or “yes” in the not skewed or not skewed control conditions) or correction (if they said “yes” in the skewed condition, or “no” in the not skewed or not skewed control condition).
In the sample observation phase, all participants saw the Zarpies who successfully boarded the boat get off the boat to visit us. The Zarpies got off one at a time, and each waved/descrunched if relevant. The height of this observed sample (4, 5, 6, 6, 7, 8) was held constant across conditions.
- To emphasize the height of the Zarpies relative to the boat, participants watched Zarpies deboard the boat, wave, reboard the boat (with any Zarpies taller than the boat stooping down again to board again), and deboard again (with any Zarpies taller than the boat straightening up again).

Observed sample in skewed condition. Note the observed sample is the same, but the height of the boat is short in the skewed condition and not skewed control condition, vs tall in the not skewed condition.

Participants were asked the following DVs in the following order:

Participants were asked the average height of the Zarpies who visited (Sample representation) and the average height of Zarpies on Zarpie island (Population inference), in counterbalanced order.
Participants were then asked an explicit comparison question asking them to compare the heights of Zarpies on Zarpie island to that of Zarpies who visited: shorter, about the same, or taller (see explicit comparison).

Finally, participants were asked for any problems or confusion they had, what they thought the task was about (see [Participant feedback]), and demographic information.

Primary results

Sample representation

As a check that they could retrieve the mean of the sample they observed, participants were asked, “Which picture shows the average height of the Zarpies who visited?”, and had to choose between a Zarpie of height 4, 5, 6, 7, or 8.

We did not preregister any hypotheses on this measure.

Sample representation question in the skewed/not skewed control conditions, with the response options in red boxes.

Since all participants saw the same sample, participants should provide the same response, regardless of condition. The sample was (4, 5, 6, 6, 7, 8), so the the response is expected to be the mean of the sample, which is 6 (indicated with red dots below).

Unexpectedly, there was a main effect of condition, such that those in the skewed sample condition reported significantly taller estimates of the sample than those in the not skewed control condition, which in turn reported significantly taller estimates of the sample than those in the not skewed condition (FIXME add stats). This was unexpected, since all conditions observed the same sample.

As in Study 1a, participants in the skewed condition reported taller heights than the true mean of the sample, which is 6 (not skewed: t(94) = 3.73, p < .001), as did those in the not skewed control condition (t(82) = 2.58, p = 0.012).

Population inference

To assess how tall participants thought Zarpies in general are, participants were asked: “Which picture shows the average height of Zarpies on Zarpie island?” Response options were a Zarpie of height 4, 5, 6, 7, or 8.

Population representation question in the skewed condition, with the response options in red boxes.

We pre-registered that if participants adjust their inferences about the population based on biases in the sampling process:

Participants in the not skewed condition should infer that the population is like the sample, since everyone was able to board, so they should respond with something similar to sample mean, i.e., 6.
Participants in the skewed condition should infer the population is taller than the sample, since Zarpies taller than the boat were unable to board, so they should report significantly taller height than those in the unrestricted condition.

Participants reported the population was taller in the skewed condition than the not skewed condition and than in the not skewed control condition, consistent with adjustment against skew.

Explicit comparison

Participants were explicitly asked to compare the population to the sample: “Do you think the Zarpies on Zarpie island are shorter, the same, or taller in height than the Zarpies who visited?”

We pre-registered that if adults do adjust, they should be more likely to say “taller” in the skewed condition than the not skewed condition for both the visible and occluded sets of conditions.

Notably, “the same” is an extremely common response across all conditions.

boarding	shorter	the same	taller
not skewed	8%	87%	4%
not skewed control	7%	89%	4%
skewed	11%	49%	40%

Participants’ explicit comparisons were significantly different between conditions (p < .001, Fisher’s exact).

## Analysis of Deviance Table (Type II tests)
## 
## Response: dv_comp_taller
##          LR Chisq Df         Pr(>Chisq)    
## boarding   57.537  2 0.0000000000003207 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

##  contrast                        estimate    SE  df z.ratio p.value
##  not skewed - not skewed control    0.159 0.779 Inf   0.204  0.8384
##  not skewed - skewed               -2.719 0.552 Inf  -4.925  <.0001
##  not skewed control - skewed       -2.878 0.624 Inf  -4.610  <.0001
## 
## Results are given on the log odds ratio (not the response) scale. 
## P value adjustment: fdr method for 3 tests

## Analysis of Deviance Table (Type II tests)
## 
## Response: dv_comp_same
##          LR Chisq Df       Pr(>Chisq)    
## boarding   47.937  2 0.00000000003895 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

##  contrast                        estimate    SE  df z.ratio p.value
##  not skewed - not skewed control   -0.173 0.469 Inf  -0.369  0.7124
##  not skewed - skewed                1.955 0.371 Inf   5.272  <.0001
##  not skewed control - skewed        2.128 0.408 Inf   5.211  <.0001
## 
## Results are given on the log odds ratio (not the response) scale. 
## P value adjustment: fdr method for 3 tests

## 
## Call:
## glm(formula = dv_comp_shorter ~ boarding, family = binomial, 
##     data = .)
## 
## Coefficients:
##                            Estimate Std. Error z value       Pr(>|z|)    
## (Intercept)                 -2.3865     0.3695  -6.459 0.000000000105 ***
## boardingnot skewed control  -0.1656     0.5623  -0.294          0.768    
## boardingskewed               0.2464     0.4983   0.495          0.621    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 162.53  on 272  degrees of freedom
## Residual deviance: 161.91  on 270  degrees of freedom
## AIC: 167.91
## 
## Number of Fisher Scoring iterations: 5

##  contrast                        estimate    SE  df z.ratio p.value
##  not skewed - not skewed control    0.166 0.562 Inf   0.294  0.7684
##  not skewed - skewed               -0.246 0.498 Inf  -0.495  0.7684
##  not skewed control - skewed       -0.412 0.540 Inf  -0.763  0.7684
## 
## Results are given on the log odds ratio (not the response) scale. 
## P value adjustment: fdr method for 3 tests

Participants were more likely to say that Zarpies on Zarpie island are “taller” and less likely to say “the same” compared to Zarpies who visited, in the skewed condition compared to the two not skewed conditions. Responses that they are “shorter” were rare and did not differ across conditions.

Secondary results

Sample vs population

As an implicit comparison, we can compare each participant’s sample representation and population inference to each other.

## Anova Table (Type II tests)
## 
## Response: response
##             Sum Sq  Df F value      Pr(>F)    
## boarding     4.211   1 21.9114 0.000003992 ***
## dv           0.516   1  2.6841      0.1022    
## boarding:dv  0.379   1  1.9720      0.1611    
## Residuals   72.253 376                        
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## Anova Table (Type II tests)
## 
## Response: response
##             Sum Sq  Df F value  Pr(>F)  
## boarding     1.021   1  4.7725 0.02958 *
## dv           0.475   1  2.2188 0.13723  
## boarding:dv  0.415   1  1.9385 0.16471  
## Residuals   75.311 352                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

There is no interaction between the boarding condition (skewed vs not skewed, or skewed vs not skewed control) and the DV (sample vs pop), suggesting participants in the skewed condition did not generalize from sample to population differently than participants in the not skewed conditions.

## 
##  Paired t-test
## 
## data:  data %>% filter(boarding == "not skewed") %>% pull(dv_sample) and data %>% filter(boarding == "not skewed") %>% pull(dv_pop)
## t = -0.25693, df = 94, p-value = 0.7978
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  -0.09187349  0.07082086
## sample estimates:
## mean difference 
##     -0.01052632

## 
##  Paired t-test
## 
## data:  data %>% filter(boarding == "not skewed control") %>% pull(dv_sample) and data %>% filter(boarding == "not skewed control") %>% pull(dv_pop)
## t = 0, df = 82, p-value = 1
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  -0.09022406  0.09022406
## sample estimates:
## mean difference 
##               0

In both of the not skewed conditions, participants did not give different responses to sample and population questions (not skewed: t(94) = -0.26, p = 0.798, not skewed control: t(82) = 0, p = 1). This is expected since in the not skewed conditions, the sample and the population are identical.

## 
##  Paired t-test
## 
## data:  data %>% filter(boarding == "skewed") %>% pull(dv_sample) and data %>% filter(boarding == "skewed") %>% pull(dv_pop)
## t = -1.7402, df = 94, p-value = 0.0851
## alternative hypothesis: true mean difference is not equal to 0
## 95 percent confidence interval:
##  -0.29297760  0.01929338
## sample estimates:
## mean difference 
##      -0.1368421

In the skewed condition, participants gave marginally different responses to the population versus sample questions (t(94) = -1.74, p = 0.085). This suggests that participants may have only marginally adjusted their generalization from the sample based on the sampling process they observed.

Sample vs population by explicit comparison

Order effects

Participants saw the two DVs in counterbalanced order:

pop_sample = population DV first, then sample DV
sample_pop = sample DV first, then population DV

There was a marginal order effect on the sample question, such that participants made marginally taller inferences about the sample if they were asked about the sample after the population (pop_sample) versus before the population (sample_pop).

There was also a marginal condition by order interaction on the population question, such that participants made marginally taller inferences about the sample if they were asked about the sample after the population (pop_sample) versus before the population (sample_pop).

In summary: * sample question then population question –> no differences between any conditions on sample question. skewed > not skewed control, not skewed on population question. * population question then sample question –> no differences between any conditions on population question. skewed, not skewed control > not skewed on sample question.

There were no order effects on the explicit comparison question.

Sample representation order effects

The condition effect on the sample representation was consistent, regardless of DV order, as evidenced by a main effect of boarding condition and no effect of DV order or interaction between the two.

## Anova Table (Type II tests)
## 
## Response: dv_sample
##                      Sum Sq  Df F value   Pr(>F)   
## boarding             1.1406   2  5.1811 0.006201 **
## cb_dvorder           0.2145   1  1.9486 0.163896   
## boarding:cb_dvorder  0.3558   2  1.6163 0.200561   
## Residuals           29.3906 267                    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

If you run pairwise comparisons anyway: when the sample question was asked second (after the population question), participants made taller inferences about the sample in the skewed condition and in the not skewed control conditions compared to the not skewed condition. No condition differences between sample representations were found when the sample question was asked first (before the population question).

## cb_dvorder = pop_sample:
##  contrast                        estimate     SE  df t.ratio p.value
##  not skewed - not skewed control  -0.1957 0.0732 267  -2.674  0.0119
##  not skewed - skewed              -0.2153 0.0654 267  -3.293  0.0034
##  not skewed control - skewed      -0.0196 0.0735 267  -0.267  0.7897
## 
## cb_dvorder = sample_pop:
##  contrast                        estimate     SE  df t.ratio p.value
##  not skewed - not skewed control  -0.0380 0.0693 267  -0.548  0.6670
##  not skewed - skewed              -0.0677 0.0711 267  -0.951  0.6670
##  not skewed control - skewed      -0.0297 0.0689 267  -0.431  0.6670
## 
## P value adjustment: fdr method for 3 tests

When people were asked about the height of the sample before being asked about the population, participants all three conditions responded no differently from the true mean (6).

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "sample_pop" & boarding == "not skewed") %>% select(dv_sample)
## t = 1, df = 42, p-value = 0.323
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  5.976324 6.070188
## sample estimates:
## mean of x 
##  6.023256

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "sample_pop" & boarding == "not skewed control") %>% select(dv_sample)
## t = 1.7693, df = 48, p-value = 0.0832
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  5.991649 6.130800
## sample estimates:
## mean of x 
##  6.061224

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "sample_pop" & boarding == "skewed") %>% select(dv_sample)
## t = 1.6656, df = 43, p-value = 0.1031
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  5.980837 6.200982
## sample estimates:
## mean of x 
##  6.090909

When people were asked about the height of the sample after being asked about the population, participants in the not skewed condition responded no differently from the true mean (6), but participants in the not skewed control condition responded marginally higher than the true mean, and participants in the skewed condition responded significantly higher than the true mean.

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "pop_sample" & boarding == "not skewed") %>% select(dv_sample)
## t = -0.57361, df = 51, p-value = 0.5688
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  5.913464 6.048075
## sample estimates:
## mean of x 
##  5.980769

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "pop_sample" & boarding == "not skewed control") %>% select(dv_sample)
## t = 1.9768, df = 33, p-value = 0.05647
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  5.994851 6.358090
## sample estimates:
## mean of x 
##  6.176471

## 
##  One Sample t-test
## 
## data:  data %>% filter(cb_dvorder == "pop_sample" & boarding == "skewed") %>% select(dv_sample)
## t = 3.4922, df = 50, p-value = 0.001012
## alternative hypothesis: true mean is not equal to 6
## 95 percent confidence interval:
##  6.083301 6.308856
## sample estimates:
## mean of x 
##  6.196078

Population inference order effects

There was a marginal interaction between condition and DV order on population representations, such that when population was asked first (before the sample question), participants in the three conditions did not give different responses from each other, but when population was asked second (after the sample question), participants in the skewed condition gave taller responses than those in the not skewed control condition, which in turn gave taller responses than those in the not skewed condition.

## 
## Call:
## lm(formula = dv_pop ~ boarding * cb_dvorder, data = data)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -2.17647 -0.17647 -0.08163  0.00000  1.98077 
## 
## Coefficients:
##                                                 Estimate Std. Error t value
## (Intercept)                                      6.01923    0.06627  90.834
## boardingnot skewed control                       0.12783    0.10539   1.213
## boardingskewed                                   0.15724    0.09417   1.670
## cb_dvordersample_pop                            -0.01923    0.09850  -0.195
## boardingnot skewed control:cb_dvordersample_pop -0.04620    0.14518  -0.318
## boardingskewed:cb_dvordersample_pop              0.25185    0.13917   1.810
##                                                            Pr(>|t|)    
## (Intercept)                                     <0.0000000000000002 ***
## boardingnot skewed control                                   0.2262    
## boardingskewed                                               0.0962 .  
## cb_dvordersample_pop                                         0.8454    
## boardingnot skewed control:cb_dvordersample_pop              0.7506    
## boardingskewed:cb_dvordersample_pop                          0.0715 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.4779 on 267 degrees of freedom
## Multiple R-squared:  0.07605,    Adjusted R-squared:  0.05875 
## F-statistic: 4.395 on 5 and 267 DF,  p-value: 0.0007307

## cb_dvorder = pop_sample:
##  contrast                        estimate     SE  df t.ratio p.value
##  not skewed - not skewed control  -0.1278 0.1054 267  -1.213  0.3394
##  not skewed - skewed              -0.1572 0.0942 267  -1.670  0.2885
##  not skewed control - skewed      -0.0294 0.1058 267  -0.278  0.7812
## 
## cb_dvorder = sample_pop:
##  contrast                        estimate     SE  df t.ratio p.value
##  not skewed - not skewed control  -0.0816 0.0999 267  -0.818  0.4143
##  not skewed - skewed              -0.4091 0.1025 267  -3.992  0.0003
##  not skewed control - skewed      -0.3275 0.0992 267  -3.299  0.0017
## 
## P value adjustment: fdr method for 3 tests

Explicit comparison order effects

There are no order effects on the explicit comparison question.

## # weights:  21 (12 variable)
## initial  value 299.921155 
## iter  10 value 166.439452
## iter  20 value 165.300655
## final  value 165.300653 
## converged

## Call:
## multinom(formula = dv_comp ~ boarding * cb_dvorder, data = data)
## 
## Coefficients:
##             (Intercept) boardingnot skewed control boardingskewed
## the same  2.72994855875             -0.02187805960      -1.631314
## taller   -0.00006176681              0.00008991661       0.865074
##          cb_dvordersample_pop boardingnot skewed control:cb_dvordersample_pop
## the same           -0.7284555                                       0.4182594
## taller             -1.6095077                                       0.2231417
##          boardingskewed:cb_dvordersample_pop
## the same                            2.072139
## taller                              2.995761
## 
## Std. Errors:
##          (Intercept) boardingnot skewed control boardingskewed
## the same   0.5958587                  0.9425448      0.7222991
## taller     0.8164786                  1.2909852      0.9188422
##          cb_dvordersample_pop boardingnot skewed control:cb_dvordersample_pop
## the same            0.7629392                                        1.178194
## taller              1.3663011                                        2.028991
##          boardingskewed:cb_dvordersample_pop
## the same                            1.136754
## taller                              1.611531
## 
## Residual Deviance: 330.6013 
## AIC: 354.6013

## Analysis of Deviance Table (Type II tests)
## 
## Response: dv_comp
##                     LR Chisq Df       Pr(>Chisq)    
## boarding              60.930  4 0.00000000000185 ***
## cb_dvorder             0.135  2           0.9349    
## boarding:cb_dvorder    5.859  4           0.2099    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Session info

## R version 4.4.2 (2024-10-31)
## Platform: aarch64-apple-darwin20
## Running under: macOS Sequoia 15.6.1
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRblas.0.dylib 
## LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## time zone: America/New_York
## tzcode source: internal
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] effectsize_1.0.0 emmeans_1.10.4   nnet_7.3-19      lmerTest_3.1-3  
##  [5] lme4_1.1-35.5    Matrix_1.7-1     car_3.1-3        carData_3.0-5   
##  [9] ggtext_0.1.2     lubridate_1.9.3  forcats_1.0.0    stringr_1.5.1   
## [13] dplyr_1.1.4      purrr_1.0.2      readr_2.1.5      tidyr_1.3.1     
## [17] tibble_3.2.1     ggplot2_3.5.1    tidyverse_2.0.0  gt_0.11.1       
## [21] scales_1.3.0     janitor_2.2.0    here_1.0.1      
## 
## loaded via a namespace (and not attached):
##  [1] gridExtra_2.3       sandwich_3.1-1      rlang_1.1.4        
##  [4] magrittr_2.0.3      multcomp_1.4-26     snakecase_0.11.1   
##  [7] compiler_4.4.2      systemfonts_1.1.0   vctrs_0.6.5        
## [10] pkgconfig_2.0.3     crayon_1.5.3        fastmap_1.2.0      
## [13] backports_1.5.0     labeling_0.4.3      rmarkdown_2.29     
## [16] markdown_1.13       tzdb_0.4.0          nloptr_2.1.1       
## [19] ragg_1.3.2          bit_4.5.0.1         xfun_0.49          
## [22] cachem_1.1.0        jsonlite_1.8.9      parallel_4.4.2     
## [25] cluster_2.1.6       R6_2.5.1            bslib_0.8.0        
## [28] stringi_1.8.4       boot_1.3-31         rpart_4.1.23       
## [31] jquerylib_0.1.4     numDeriv_2016.8-1.1 estimability_1.5.1 
## [34] Rcpp_1.0.13         knitr_1.49          zoo_1.8-12         
## [37] base64enc_0.1-3     parameters_0.24.0   splines_4.4.2      
## [40] timechange_0.3.0    tidyselect_1.2.1    rstudioapi_0.17.1  
## [43] abind_1.4-8         yaml_2.3.10         codetools_0.2-20   
## [46] lattice_0.22-6      withr_3.0.2         bayestestR_0.15.0  
## [49] coda_0.19-4.1       evaluate_1.0.1      foreign_0.8-87     
## [52] survival_3.7-0      xml2_1.3.6          pillar_1.10.0      
## [55] checkmate_2.3.2     insight_1.0.0       generics_0.1.3     
## [58] vroom_1.6.5         rprojroot_2.0.4     hms_1.1.3          
## [61] commonmark_1.9.2    munsell_0.5.1       minqa_1.2.8        
## [64] glue_1.8.0          Hmisc_5.1-3         tools_4.4.2        
## [67] data.table_1.15.4   mvtnorm_1.3-1       grid_4.4.2         
## [70] datawizard_0.13.0   colorspace_2.1-1    nlme_3.1-166       
## [73] htmlTable_2.4.3     Formula_1.2-5       cli_3.6.3          
## [76] textshaping_0.4.0   ggthemes_5.1.0      gtable_0.3.5       
## [79] sass_0.4.9          digest_0.6.37       TH.data_1.1-2      
## [82] htmlwidgets_1.6.4   farver_2.1.2        htmltools_0.5.8.1  
## [85] lifecycle_1.0.4     gridtext_0.1.5      bit64_4.5.2        
## [88] MASS_7.3-61

Structural skew

Study 1b

Marianna Zhang

2025-09-02