Primary results
Induction task
Analyses of the induction task were logistic regressions unless otherwise specified, predicting prevalence (.01-.99) with participant and test feature as random intercepts. Test feature (“can snap with their toes”, etc.) is technically nested within test feature type (physical, diet, personality), but since each test feature is unique to each test feature type, a model with the nesting term is analytically equivalent to the previous model, so the nesting term was omitted for simplicity of specification.
By condition x test feature type
# condition x test feature type
glmm_condition_testfeaturetype <-
glmmTMB(prevalence ~ condition * test_feature_type + (1|participant),
# prevalence ~ condition * test_feature_type + (1|participant) + (1|test_feature_type:test_feature),
data = data_tidy,
family = beta_family(link = "logit"))
glmm_condition_testfeaturetype %>%
Anova()There is no significant interaction between condition and test feature type on prevalence (\(\chi^2\)(4) = 4.38, p = 0.358), based on an ANOVA conducted on a logistic regression with condition, test feature type, and their interaction as fixed effects, with random intercepts per participant.
The only significant effect was a main effect of test feature type (\(\chi^2\)(2) = 230.59, p < .001).
# condition x test feature type
brm_condition_testfeaturetype <-
brm(prevalence ~ condition * test_feature_type + (1|participant),
data = data_tidy,
family = beta_family(link = "logit"),
save_pars = save_pars(all = TRUE))
# vs null
brm_condition_testfeaturetype_null <-
brm(prevalence ~ condition + test_feature_type + (1|participant),
data = data_tidy,
family = beta_family(link = "logit"),
save_pars = save_pars(all = TRUE))
bf_condition_testfeaturetype <-
bayes_factor(brm_condition_testfeaturetype, brm_condition_testfeaturetype_null)A Bayesian analysis revealed moderate evidence against an interaction between condition and test feature type on prevalence (BF = 0.19), comparing Bayesian logistic models with and without the interaction, with default priors.
By condition x test feature
We can look at how prevalence judgments vary by condition and individual test features.
By test feature type match
Another way to look at the data is to code responses by whether the test feature type matched the feature type that received generic statements in that condition. If they match (e.g., a diet generic condition responding to a diet test question), we can code that as a match, or if they mismatch (e.g., diet generic condition responding to a personality test question), we can code that as a mismatch.
In this study, match = heard generics about that feature type, mismatch = heard specifics about that feature type.
If the chosen clusters capture some systematicity in how people generalize, matches should result in higher prevalence estimates than mismatches. Indeed, that’s what we find.
# condition
glmm_condition_test_match <-
glmmTMB(prevalence ~ condition_test_match + (1|participant) + (1|test_feature),
data = data_tidy,
family = beta_family(link = "logit"))
glmm_condition_test_match %>%
Anova()There is a marginal effect of whether one heard generics or specifics about features in the test feature type on prevalence, in an ANOVA conducted on a logistic regression with match as a main effect, and with participant and test feature as random intercepts (\(\chi^2\)(1) = 2.8, p = 0.094).
By cosine similarity
# average cosine similarity of the test feature, to the training features in that condition
glmm_cosine_similarity <-
glmmTMB(prevalence ~ cosine_similarity + (1|participant) + (1|test_feature),
data = data_tidy,
family = beta_family(link = "logit"))
glmm_cosine_similarity %>%
summary()There is no effect of average cosine similarity of the test feature to the genericized training features in the condition (\(\chi^2\)(1) = 0.82, p = 0.365), in a logistic model with random intercepts per participant and test feature.
Group characterization
Participants were asked to describe what characterizes Zarpies as a group. TBD