2. Models predicting word knowledge by measure
Christina
2021-02-12
- 1 Analyses including picture-naming data (fewer words) - raw AoAs
- 1.1 Estimate only effects of measure: in general, how much do they differ?
- 1.2 Estimate how word characteristics (frequency, preschoolness, helpfulness, concreteness) interact with measure
- 1.3 Pairwise comparisons of slopes for different measures
- 1.4 Reliability of frequency interaction
- 1.5 Reliability of concreteness interaction
- 1.6 Reliability of preschoolness interaction
- 1.7 Reliability of helpfulness interaction
- 1.8 Check effect of concreteness for larger dataset (no picture-naming)
- 2 Analyses including picture-naming data (fewer words) - Inverse AoAs
- 3 Analyses excluding picture-naming data (more words) - Inverse AoAs
- 3.1 Estimate only effects of measure: in general, how much do they differ?
- 3.2 Add interactions with word-level characteristics
- 3.3 Pairwise comparisons of slopes for different measures
- 3.4 Reliability of frequency interaction
- 3.5 Reliability of concreteness interaction
- 3.6 Reliability of preschoolness interaction
- 3.7 Reliability of helpfulness interaction
complete_predictors_all <- read_csv("data/complete_predictors_all_vars.csv") %>%
rename(picture_naming_aoa = morrison_aoa_threshold_years, wordbank_inv = wordbank_threshold_inv) %>%
pivot_longer(cols = c("KupermanAoA", "wordbank_aoa_years", "picture_naming_aoa",
"prop_say_naive_combined","kuperman_inv",
"picture_naming_inv", "wordbank_inv"),
names_to = "measure", values_to = "value") %>%
select(num_item_id, word, category, wordnet_pos, preschoolness, helpfulness, frequency=childes_adult_log_freq,
concreteness, measure, value)
#set measure as factor for later analyses
complete_predictors_all$measure <- as.factor(complete_predictors_all$measure)
complete_predictors_wordbank_all <- read_csv("data/complete_predictors_wordbank.csv") %>%
rename(picture_naming_aoa = morrison_aoa_threshold_years, wordbank_inv = wordbank_threshold_inv) %>%
pivot_longer(cols = c("KupermanAoA", "wordbank_aoa_years", "picture_naming_aoa",
"prop_say_naive_combined","kuperman_inv",
"picture_naming_inv", "wordbank_inv"),
names_to = "measure", values_to = "value") %>%
select(num_item_id, word, category, wordnet_pos, preschoolness, helpfulness, frequency=childes_adult_log_freq,
concreteness, measure, value)
#set measure as factor for later analyses
complete_predictors_all$measure <- as.factor(complete_predictors_all$measure)
aoas <- c("KupermanAoA","wordbank_aoa_years","picture_naming_aoa")
#dfs with inverted AoAs (for easier comparison to proportion data)
##smaller df that includes only words for which we have all data, including picture-naming norms
complete_predictors <- complete_predictors_all %>% filter(!(measure %in% aoas))
##larger df that excludes picture-naming norms for a larger complete dataset
complete_predictors_wordbank <- complete_predictors_wordbank_all %>% filter(!(measure %in% aoas) & measure != "picture_naming_inv")
#dfs with raw AoAs - excludes proportion data
##smaller df that includes words for which we have all data including picture-naming
complete_predictors_aoas <- complete_predictors_all %>% filter(measure %in% aoas)
##larger df that excludes picture-naming norms for a larger complete dataset
complete_predictors_wordbank_aoas <- complete_predictors_wordbank_all %>% filter(measure %in% aoas)
#df of only kuperman and wordbank AoAs
kuperman_wb_aoas <- complete_predictors_wordbank_all %>% filter(measure %in% c("KupermanAoA","wordbank_aoa_years"))Codebook:
wordbank_threshold_inv: 1/wordbank_threshold_aoa so that interpretation is consistent with naive proportion data
kuperman_inv: 1/KupermanAoA
picture_naming_inv: 1/picture_naming_aoa
prop_say_naive_combined: Proportion of children estimated to produce the word at 18/24 months (combined across age group because r = .95), from a survey of naive adults (N=87).
wordbank_production_24: Proportion of children reported in wordbank to produce word at 24 months (for comparison to naive data)
preschoolness: on a scale of 1-5, how much is the word associated with preschoolers (MTurk)
helpfulness: on a scale of 1-5, how helpful would it be for a preschooler to know this word (MTurk)
frequency: log frequency based on adult speech in CHILDES
concreteness: concreteness norms from Brysbaert et al. - adults asked to rate on a scale of 1-5
1 Analyses including picture-naming data (fewer words) - raw AoAs
1.1 Estimate only effects of measure: in general, how much do they differ?
#relevel measure so that picture-naming (closest thing we have to ground truth) is reference group
complete_predictors_aoas$measure <- relevel(complete_predictors_aoas$measure, ref = "picture_naming_aoa")
measure_aoa <- lmer(value ~ measure + (1|word), data=complete_predictors_aoas)
summary(measure_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ measure + (1 | word)
## Data: complete_predictors_aoas
##
## REML criterion at convergence: 1021
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -1.3649 -0.5894 -0.0307 0.2997 5.4675
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.2809 0.5300
## Residual 0.8904 0.9436
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 3.1887 0.1009 306.7233 31.60 < 2e-16 ***
## measureKupermanAoA 0.9818 0.1244 228.0000 7.89 1.25e-13 ***
## measurewordbank_aoa_years -1.3184 0.1244 228.0000 -10.60 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) msrKAA
## mesrKprmnAA -0.617
## msrwrdbnk__ -0.617 0.5001.1.1 Plot what this main effect looks like
ggplot(complete_predictors_aoas, aes(x = measure, y = value, color = measure, fill = measure))+
geom_jitter(height=0.1, width=0.1)+
geom_violin(alpha=.2)+
theme_classic()+
labs(y="age of acquisition")
1.2 Estimate how word characteristics (frequency, preschoolness, helpfulness, concreteness) interact with measure
omnibus_aoa <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_aoas)
summary(omnibus_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: complete_predictors_aoas
##
## REML criterion at convergence: 899.4
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.9267 -0.3891 -0.0499 0.3066 4.1847
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.09515 0.3085
## Residual 0.67802 0.8234
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 20.66385 3.96179 320.29807 5.216
## frequency -0.86982 0.07958 320.29840 -10.930
## measureKupermanAoA -13.37824 5.24675 220.00032 -2.550
## measurewordbank_aoa_years -15.63595 5.24675 220.00034 -2.980
## preschoolness 0.37796 0.12219 320.29840 3.093
## helpfulness 0.62591 0.13874 320.29840 4.511
## concreteness -2.89647 0.80939 320.29807 -3.579
## frequency:measureKupermanAoA 0.53990 0.10540 220.00000 5.123
## frequency:measurewordbank_aoa_years 0.70619 0.10540 220.00000 6.700
## measureKupermanAoA:preschoolness -0.76850 0.16182 220.00000 -4.749
## measurewordbank_aoa_years:preschoolness -0.43716 0.16182 220.00000 -2.702
## measureKupermanAoA:helpfulness -0.73424 0.18374 220.00000 -3.996
## measurewordbank_aoa_years:helpfulness -0.62298 0.18374 220.00000 -3.391
## measureKupermanAoA:concreteness 2.99750 1.07191 220.00032 2.796
## measurewordbank_aoa_years:concreteness 2.51609 1.07191 220.00034 2.347
## Pr(>|t|)
## (Intercept) 3.29e-07 ***
## frequency < 2e-16 ***
## measureKupermanAoA 0.011458 *
## measurewordbank_aoa_years 0.003205 **
## preschoolness 0.002154 **
## helpfulness 9.05e-06 ***
## concreteness 0.000399 ***
## frequency:measureKupermanAoA 6.58e-07 ***
## frequency:measurewordbank_aoa_years 1.72e-10 ***
## measureKupermanAoA:preschoolness 3.68e-06 ***
## measurewordbank_aoa_years:preschoolness 0.007440 **
## measureKupermanAoA:helpfulness 8.79e-05 ***
## measurewordbank_aoa_years:helpfulness 0.000827 ***
## measureKupermanAoA:concreteness 0.005625 **
## measurewordbank_aoa_years:concreteness 0.019798 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.3 Pairwise comparisons of slopes for different measures
measure_frequency_aoa <- emtrends(omnibus_aoa, "measure", var = "frequency")
print(measure_frequency_aoa)## measure frequency.trend SE df lower.CL upper.CL
## picture_naming_aoa -0.870 0.0796 320 -1.026 -0.71324
## KupermanAoA -0.330 0.0796 320 -0.486 -0.17334
## wordbank_aoa_years -0.164 0.0796 320 -0.320 -0.00705
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_frequency_aoa)## contrast estimate SE df t.ratio p.value
## picture_naming_aoa - KupermanAoA -0.540 0.105 220 -5.123 <.0001
## picture_naming_aoa - wordbank_aoa_years -0.706 0.105 220 -6.700 <.0001
## KupermanAoA - wordbank_aoa_years -0.166 0.105 220 -1.578 0.2574
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_preschoolness_aoa <- emtrends(omnibus_aoa, "measure", var = "preschoolness")
print(measure_preschoolness_aoa)## measure preschoolness.trend SE df lower.CL upper.CL
## picture_naming_aoa 0.3780 0.122 320 0.138 0.618
## KupermanAoA -0.3905 0.122 320 -0.631 -0.150
## wordbank_aoa_years -0.0592 0.122 320 -0.300 0.181
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_preschoolness_aoa)## contrast estimate SE df t.ratio p.value
## picture_naming_aoa - KupermanAoA 0.769 0.162 220 4.749 <.0001
## picture_naming_aoa - wordbank_aoa_years 0.437 0.162 220 2.702 0.0203
## KupermanAoA - wordbank_aoa_years -0.331 0.162 220 -2.048 0.1034
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_helpfulness_aoa <- emtrends(omnibus_aoa, "measure", var = "helpfulness")
print(measure_helpfulness_aoa)## measure helpfulness.trend SE df lower.CL upper.CL
## picture_naming_aoa 0.62591 0.139 320 0.353 0.899
## KupermanAoA -0.10833 0.139 320 -0.381 0.165
## wordbank_aoa_years 0.00293 0.139 320 -0.270 0.276
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_helpfulness_aoa)## contrast estimate SE df t.ratio p.value
## picture_naming_aoa - KupermanAoA 0.734 0.184 220 3.996 0.0003
## picture_naming_aoa - wordbank_aoa_years 0.623 0.184 220 3.391 0.0024
## KupermanAoA - wordbank_aoa_years -0.111 0.184 220 -0.606 0.8173
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_concreteness_aoa <- emtrends(omnibus_aoa, "measure", var = "concreteness")
print(measure_concreteness_aoa)## measure concreteness.trend SE df lower.CL upper.CL
## picture_naming_aoa -2.896 0.809 320 -4.49 -1.30
## KupermanAoA 0.101 0.809 320 -1.49 1.69
## wordbank_aoa_years -0.380 0.809 320 -1.97 1.21
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_concreteness_aoa)## contrast estimate SE df t.ratio p.value
## picture_naming_aoa - KupermanAoA -2.998 1.07 220 -2.796 0.0155
## picture_naming_aoa - wordbank_aoa_years -2.516 1.07 220 -2.347 0.0516
## KupermanAoA - wordbank_aoa_years 0.481 1.07 220 0.449 0.8948
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimates1.4 Reliability of frequency interaction
Reference group: Kuperman AoA (shallowest slope)
1.4.1 Model
frequency_aoa <- lmer(value ~ frequency + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_aoas)
summary(frequency_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency + preschoolness * measure + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors_aoas
##
## REML criterion at convergence: 938.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.0212 -0.5415 -0.0549 0.3496 5.4110
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.04721 0.2173
## Residual 0.82182 0.9065
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 18.33211 4.18336 329.47397 4.382
## frequency -0.45445 0.05129 110.00000 -8.860
## preschoolness 0.16553 0.12493 325.59399 1.325
## measureKupermanAoA -10.34739 5.73957 221.99984 -1.803
## measurewordbank_aoa_years -11.67156 5.73957 221.99975 -2.034
## helpfulness 0.52017 0.14610 329.21868 3.560
## concreteness -2.85587 0.85808 329.82206 -3.328
## preschoolness:measureKupermanAoA -0.49238 0.16798 222.00000 -2.931
## preschoolness:measurewordbank_aoa_years -0.07600 0.16798 222.00000 -0.452
## measureKupermanAoA:helpfulness -0.59680 0.20012 222.00000 -2.982
## measurewordbank_aoa_years:helpfulness -0.44321 0.20012 222.00000 -2.215
## measureKupermanAoA:concreteness 2.94472 1.18006 221.99984 2.495
## measurewordbank_aoa_years:concreteness 2.44705 1.18006 221.99975 2.074
## Pr(>|t|)
## (Intercept) 1.58e-05 ***
## frequency 1.58e-14 ***
## preschoolness 0.186107
## measureKupermanAoA 0.072773 .
## measurewordbank_aoa_years 0.043189 *
## helpfulness 0.000425 ***
## concreteness 0.000973 ***
## preschoolness:measureKupermanAoA 0.003730 **
## preschoolness:measurewordbank_aoa_years 0.651405
## measureKupermanAoA:helpfulness 0.003181 **
## measurewordbank_aoa_years:helpfulness 0.027794 *
## measureKupermanAoA:concreteness 0.013310 *
## measurewordbank_aoa_years:concreteness 0.039264 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_aoa, frequency_aoa)## Data: complete_predictors_aoas
## Models:
## frequency_aoa: value ~ frequency + preschoolness * measure + helpfulness * measure +
## frequency_aoa: concreteness * measure + (1 | word)
## omnibus_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_aoa: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## frequency_aoa 15 946.40 1004.05 -458.20 916.40
## omnibus_aoa 17 904.08 969.42 -435.04 870.08 46.322 2 8.738e-11 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.4.2 Plot
freq_aoa_plot <- ggplot(complete_predictors_aoas, aes(x = frequency, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
freq_aoa_plot
1.5 Reliability of concreteness interaction
Reference group: Kuperman AoA
1.5.1 Model
concreteness_aoa <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness +
(1|word), data=complete_predictors_aoas)
summary(concreteness_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness + (1 | word)
## Data: complete_predictors_aoas
##
## REML criterion at convergence: 912
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.0507 -0.3775 -0.0309 0.2952 5.0032
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.0880 0.2966
## Residual 0.6995 0.8363
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 11.79380 2.60437 118.96079 4.528
## frequency -0.86808 0.08031 323.44888 -10.809
## measureKupermanAoA 1.08856 0.88841 222.00001 1.225
## measurewordbank_aoa_years -3.49259 0.88841 222.00001 -3.931
## preschoolness 0.36351 0.12321 323.31050 2.950
## helpfulness 0.59351 0.13958 322.88617 4.252
## concreteness -1.05861 0.52165 109.99990 -2.029
## frequency:measureKupermanAoA 0.53707 0.10704 222.00000 5.017
## frequency:measurewordbank_aoa_years 0.70382 0.10704 222.00000 6.575
## measureKupermanAoA:preschoolness -0.74494 0.16414 222.00000 -4.539
## measurewordbank_aoa_years:preschoolness -0.41739 0.16414 222.00000 -2.543
## measureKupermanAoA:helpfulness -0.68141 0.18563 222.00001 -3.671
## measurewordbank_aoa_years:helpfulness -0.57864 0.18563 222.00001 -3.117
## Pr(>|t|)
## (Intercept) 1.42e-05 ***
## frequency < 2e-16 ***
## measureKupermanAoA 0.221764
## measurewordbank_aoa_years 0.000113 ***
## preschoolness 0.003407 **
## helpfulness 2.77e-05 ***
## concreteness 0.044837 *
## frequency:measureKupermanAoA 1.07e-06 ***
## frequency:measurewordbank_aoa_years 3.43e-10 ***
## measureKupermanAoA:preschoolness 9.28e-06 ***
## measurewordbank_aoa_years:preschoolness 0.011673 *
## measureKupermanAoA:helpfulness 0.000303 ***
## measurewordbank_aoa_years:helpfulness 0.002068 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_aoa, concreteness_aoa)## Data: complete_predictors_aoas
## Models:
## concreteness_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness *
## concreteness_aoa: measure + concreteness + (1 | word)
## omnibus_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_aoa: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## concreteness_aoa 15 909.32 966.97 -439.66 879.32
## omnibus_aoa 17 904.08 969.42 -435.04 870.08 9.2428 2 0.009839 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.5.2 Plot
concreteness_aoa_plot <- ggplot(complete_predictors_aoas, aes(x = concreteness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
concreteness_aoa_plot
1.6 Reliability of preschoolness interaction
Reference group: picture_naming_inv (shallowest slope)
1.6.1 Model
complete_predictors_aoas_presch <- complete_predictors_aoas
complete_predictors_aoas_presch$measure <- relevel(as.factor(complete_predictors_aoas_presch$measure), ref = "picture_naming_aoa")
omnibus_presch_aoa <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_aoas_presch)
preschoolness_aoa <- lmer(value ~ frequency*measure + preschoolness + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_aoas_presch)
summary(omnibus_presch_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: complete_predictors_aoas_presch
##
## REML criterion at convergence: 899.4
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.9267 -0.3891 -0.0499 0.3066 4.1847
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.09515 0.3085
## Residual 0.67802 0.8234
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 20.66385 3.96179 320.29807 5.216
## frequency -0.86982 0.07958 320.29840 -10.930
## measureKupermanAoA -13.37824 5.24675 220.00032 -2.550
## measurewordbank_aoa_years -15.63595 5.24675 220.00034 -2.980
## preschoolness 0.37796 0.12219 320.29840 3.093
## helpfulness 0.62591 0.13874 320.29840 4.511
## concreteness -2.89647 0.80939 320.29807 -3.579
## frequency:measureKupermanAoA 0.53990 0.10540 220.00000 5.123
## frequency:measurewordbank_aoa_years 0.70619 0.10540 220.00000 6.700
## measureKupermanAoA:preschoolness -0.76850 0.16182 220.00000 -4.749
## measurewordbank_aoa_years:preschoolness -0.43716 0.16182 220.00000 -2.702
## measureKupermanAoA:helpfulness -0.73424 0.18374 220.00000 -3.996
## measurewordbank_aoa_years:helpfulness -0.62298 0.18374 220.00000 -3.391
## measureKupermanAoA:concreteness 2.99750 1.07191 220.00032 2.796
## measurewordbank_aoa_years:concreteness 2.51609 1.07191 220.00034 2.347
## Pr(>|t|)
## (Intercept) 3.29e-07 ***
## frequency < 2e-16 ***
## measureKupermanAoA 0.011458 *
## measurewordbank_aoa_years 0.003205 **
## preschoolness 0.002154 **
## helpfulness 9.05e-06 ***
## concreteness 0.000399 ***
## frequency:measureKupermanAoA 6.58e-07 ***
## frequency:measurewordbank_aoa_years 1.72e-10 ***
## measureKupermanAoA:preschoolness 3.68e-06 ***
## measurewordbank_aoa_years:preschoolness 0.007440 **
## measureKupermanAoA:helpfulness 8.79e-05 ***
## measurewordbank_aoa_years:helpfulness 0.000827 ***
## measureKupermanAoA:concreteness 0.005625 **
## measurewordbank_aoa_years:concreteness 0.019798 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1summary(preschoolness_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors_aoas_presch
##
## REML criterion at convergence: 917.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.7612 -0.4104 -0.0220 0.2672 4.6634
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.07408 0.2722
## Residual 0.74123 0.8609
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 20.32103 4.06747 326.20305 4.996
## frequency -0.78263 0.07893 320.04272 -9.916
## measureKupermanAoA -12.72267 5.48397 222.00014 -2.320
## measurewordbank_aoa_years -15.26303 5.48397 222.00016 -2.783
## preschoolness -0.02393 0.07875 110.00000 -0.304
## helpfulness 0.61535 0.14245 326.20991 4.320
## concreteness -2.75789 0.83047 326.11509 -3.321
## frequency:measureKupermanAoA 0.37317 0.10391 222.00000 3.591
## frequency:measurewordbank_aoa_years 0.61135 0.10391 222.00000 5.884
## measureKupermanAoA:helpfulness -0.71406 0.19206 222.00000 -3.718
## measurewordbank_aoa_years:helpfulness -0.61150 0.19206 222.00000 -3.184
## measureKupermanAoA:concreteness 2.73249 1.11924 222.00013 2.441
## measurewordbank_aoa_years:concreteness 2.36534 1.11924 222.00016 2.113
## Pr(>|t|)
## (Intercept) 9.56e-07 ***
## frequency < 2e-16 ***
## measureKupermanAoA 0.021251 *
## measurewordbank_aoa_years 0.005846 **
## preschoolness 0.761790
## helpfulness 2.07e-05 ***
## concreteness 0.000999 ***
## frequency:measureKupermanAoA 0.000405 ***
## frequency:measurewordbank_aoa_years 1.47e-08 ***
## measureKupermanAoA:helpfulness 0.000254 ***
## measurewordbank_aoa_years:helpfulness 0.001662 **
## measureKupermanAoA:concreteness 0.015414 *
## measurewordbank_aoa_years:concreteness 0.035688 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_presch_aoa, preschoolness_aoa)## Data: complete_predictors_aoas_presch
## Models:
## preschoolness_aoa: value ~ frequency * measure + preschoolness + helpfulness * measure +
## preschoolness_aoa: concreteness * measure + (1 | word)
## omnibus_presch_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_presch_aoa: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## preschoolness_aoa 15 922.66 980.31 -446.33 892.66
## omnibus_presch_aoa 17 904.08 969.42 -435.04 870.08 22.582 2 1.248e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.6.2 Plot
presch_aoa_plot <- ggplot(complete_predictors_aoas, aes(x = preschoolness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
presch_aoa_plot
1.7 Reliability of helpfulness interaction
Reference group: Kuperman AoA (shallowest slope)
1.7.1 Model
helpfulness_aoa <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness + concreteness*measure +
(1|word), data=complete_predictors_aoas)
summary(helpfulness_aoa)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness +
## concreteness * measure + (1 | word)
## Data: complete_predictors_aoas
##
## REML criterion at convergence: 914
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.8646 -0.4526 -0.0480 0.2658 4.6840
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.0783 0.2798
## Residual 0.7286 0.8536
## Number of obs: 345, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 20.56718 4.04716 325.46175 5.082
## frequency -0.83192 0.08078 324.43522 -10.299
## measureKupermanAoA -13.22135 5.43870 222.00056 -2.431
## measurewordbank_aoa_years -15.50284 5.43870 222.00052 -2.850
## preschoolness 0.36874 0.12480 325.44006 2.955
## helpfulness 0.17350 0.08942 110.00000 1.940
## concreteness -2.62511 0.82421 324.96602 -3.185
## frequency:measureKupermanAoA 0.47840 0.10808 221.99997 4.426
## frequency:measurewordbank_aoa_years 0.65401 0.10808 221.99997 6.051
## measureKupermanAoA:preschoolness -0.75355 0.16770 221.99997 -4.493
## measurewordbank_aoa_years:preschoolness -0.42448 0.16770 221.99997 -2.531
## measureKupermanAoA:concreteness 2.55709 1.10527 222.00055 2.314
## measurewordbank_aoa_years:concreteness 2.14241 1.10527 222.00052 1.938
## Pr(>|t|)
## (Intercept) 6.31e-07 ***
## frequency < 2e-16 ***
## measureKupermanAoA 0.01585 *
## measurewordbank_aoa_years 0.00478 **
## preschoolness 0.00336 **
## helpfulness 0.05490 .
## concreteness 0.00159 **
## frequency:measureKupermanAoA 1.50e-05 ***
## frequency:measurewordbank_aoa_years 6.06e-09 ***
## measureKupermanAoA:preschoolness 1.13e-05 ***
## measurewordbank_aoa_years:preschoolness 0.01206 *
## measureKupermanAoA:concreteness 0.02161 *
## measurewordbank_aoa_years:concreteness 0.05385 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_aoa, helpfulness_aoa)## Data: complete_predictors_aoas
## Models:
## helpfulness_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness +
## helpfulness_aoa: concreteness * measure + (1 | word)
## omnibus_aoa: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_aoa: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## helpfulness_aoa 15 918.70 976.36 -444.35 888.70
## omnibus_aoa 17 904.08 969.42 -435.04 870.08 18.623 2 9.038e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.7.2 Plot
help_aoa_plot <- ggplot(complete_predictors_aoas, aes(x = helpfulness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
help_aoa_plot
1.8 Check effect of concreteness for larger dataset (no picture-naming)
omnibus_for_conc <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data = kuperman_wb_aoas)
summary(omnibus_for_conc)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: kuperman_wb_aoas
##
## REML criterion at convergence: 1296.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.7644 -0.3683 0.0138 0.3592 5.7629
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.05037 0.2244
## Residual 0.24102 0.4909
## Number of obs: 790, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 8.818961 0.388443 757.368189
## frequency -0.254119 0.021569 757.368193
## measurewordbank_aoa_years -5.117637 0.499610 389.999976
## preschoolness -0.363172 0.032514 757.368191
## helpfulness -0.009086 0.043462 757.368189
## concreteness -0.393819 0.050734 757.368192
## frequency:measurewordbank_aoa_years 0.151079 0.027742 389.999976
## measurewordbank_aoa_years:preschoolness 0.296224 0.041819 389.999982
## measurewordbank_aoa_years:helpfulness 0.046555 0.055901 389.999982
## measurewordbank_aoa_years:concreteness 0.187745 0.065254 389.999977
## t value Pr(>|t|)
## (Intercept) 22.703 < 2e-16 ***
## frequency -11.782 < 2e-16 ***
## measurewordbank_aoa_years -10.243 < 2e-16 ***
## preschoolness -11.170 < 2e-16 ***
## helpfulness -0.209 0.83446
## concreteness -7.762 2.70e-14 ***
## frequency:measurewordbank_aoa_years 5.446 9.14e-08 ***
## measurewordbank_aoa_years:preschoolness 7.083 6.61e-12 ***
## measurewordbank_aoa_years:helpfulness 0.833 0.40546
## measurewordbank_aoa_years:concreteness 2.877 0.00423 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) frqncy msrw__ prschl hlpfln cncrtn frq:__
## frequency -0.613
## msrwrdbnk__ -0.643 0.394
## prescholnss -0.221 -0.064 0.142
## helpfulness -0.497 -0.039 0.320 -0.079
## concretenss -0.874 0.429 0.562 0.102 0.241
## frqncy:ms__ 0.394 -0.643 -0.613 0.041 0.025 -0.276
## msrwrdbnk__yrs:p 0.142 0.041 -0.221 -0.643 0.051 -0.065 -0.064
## msrwrdbnk__yrs:h 0.320 0.025 -0.497 0.051 -0.643 -0.155 -0.039
## msrwrdbnk__yrs:c 0.562 -0.276 -0.874 -0.065 -0.155 -0.643 0.429
## msrwrdbnk__yrs:p msrwrdbnk__yrs:h
## frequency
## msrwrdbnk__
## prescholnss
## helpfulness
## concretenss
## frqncy:ms__
## msrwrdbnk__yrs:p
## msrwrdbnk__yrs:h -0.079
## msrwrdbnk__yrs:c 0.102 0.2411.8.1 Concreteness interaction reliability
kwb_conc <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness +
(1|word), data = kuperman_wb_aoas)
summary(kwb_conc)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness + (1 | word)
## Data: kuperman_wb_aoas
##
## REML criterion at convergence: 1300.8
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -2.7639 -0.3870 0.0199 0.3594 5.8122
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.04813 0.2194
## Residual 0.24551 0.4955
## Number of obs: 790, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 8.191056 0.321768 518.966610
## frequency -0.237016 0.020804 730.840427
## measurewordbank_aoa_years -3.861826 0.245351 391.000001
## preschoolness -0.357058 0.032569 759.004747
## helpfulness 0.010292 0.043097 752.632556
## concreteness -0.299947 0.038852 390.000029
## frequency:measurewordbank_aoa_years 0.116873 0.025298 390.999999
## measurewordbank_aoa_years:preschoolness 0.283996 0.041988 391.000000
## measurewordbank_aoa_years:helpfulness 0.007799 0.054756 391.000002
## t value Pr(>|t|)
## (Intercept) 25.456 < 2e-16 ***
## frequency -11.393 < 2e-16 ***
## measurewordbank_aoa_years -15.740 < 2e-16 ***
## preschoolness -10.963 < 2e-16 ***
## helpfulness 0.239 0.811
## concreteness -7.720 9.86e-14 ***
## frequency:measurewordbank_aoa_years 4.620 5.22e-06 ***
## measurewordbank_aoa_years:preschoolness 6.764 4.92e-11 ***
## measurewordbank_aoa_years:helpfulness 0.142 0.887
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) frqncy msrw__ prschl hlpfln cncrtn frq:__
## frequency -0.575
## msrwrdbnk__ -0.381 0.329
## prescholnss -0.224 -0.086 0.176
## helpfulness -0.502 -0.087 0.385 -0.091
## concretenss -0.808 0.340 0.000 0.078 0.186
## frqncy:ms__ 0.207 -0.608 -0.542 0.077 0.103 0.000
## msrwrdbnk__yrs:p 0.104 0.073 -0.274 -0.645 0.068 0.000 -0.120
## msrwrdbnk__yrs:h 0.231 0.099 -0.607 0.069 -0.635 0.000 -0.163
## msrwrdbnk__yrs:p
## frequency
## msrwrdbnk__
## prescholnss
## helpfulness
## concretenss
## frqncy:ms__
## msrwrdbnk__yrs:p
## msrwrdbnk__yrs:h -0.108anova(omnibus_for_conc, kwb_conc)## Data: kuperman_wb_aoas
## Models:
## kwb_conc: value ~ frequency * measure + preschoolness * measure + helpfulness *
## kwb_conc: measure + concreteness + (1 | word)
## omnibus_for_conc: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_for_conc: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## kwb_conc 11 1276.0 1327.4 -627.02 1254.0
## omnibus_for_conc 12 1269.8 1325.8 -622.87 1245.8 8.2964 1 0.003972 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 11.8.2 Compare concreteness slopes for Kuperman and Wordbank
measure_concreteness_kwb <- emtrends(omnibus_for_conc, "measure", var = "concreteness")
print(measure_concreteness_kwb)## measure concreteness.trend SE df lower.CL upper.CL
## KupermanAoA -0.394 0.0507 757 -0.493 -0.294
## wordbank_aoa_years -0.206 0.0507 757 -0.306 -0.106
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_concreteness_kwb)## contrast estimate SE df t.ratio p.value
## KupermanAoA - wordbank_aoa_years -0.188 0.0653 390 -2.877 0.0042
##
## Degrees-of-freedom method: kenward-roger1.8.3 Plot
conc_kwb_aoa_plot <- ggplot(kuperman_wb_aoas, aes(x = concreteness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
conc_kwb_aoa_plot
2 Analyses including picture-naming data (fewer words) - Inverse AoAs
2.1 Estimate only effects of measure: in general, how much do they differ?
#relevel measure so that picture-naming (closest thing we have to ground truth) is reference group
complete_predictors$measure <- fct_relevel(complete_predictors$measure, c("picture_naming_inv", "wordbank_inv", "prop_say_naive_combined,","kuperman_inv"))
measure_aoa_inv <- lmer(value ~ measure + (1|word), data=complete_predictors)
summary(measure_aoa_inv)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ measure + (1 | word)
## Data: complete_predictors
##
## REML criterion at convergence: -497.8
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2841 -0.4628 0.0162 0.4941 2.3808
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.007813 0.08839
## Residual 0.014070 0.11862
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 0.37846 0.01379 329.85072 27.435 < 2e-16
## measurewordbank_inv 0.16799 0.01564 342.00000 10.739 < 2e-16
## measurekuperman_inv -0.12774 0.01564 342.00000 -8.166 6.2e-15
## measureprop_say_naive_combined 0.19416 0.01564 342.00000 12.412 < 2e-16
##
## (Intercept) ***
## measurewordbank_inv ***
## measurekuperman_inv ***
## measureprop_say_naive_combined ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) msrwr_ msrkp_
## msrwrdbnk_n -0.567
## msrkprmn_nv -0.567 0.500
## msrprp_sy__ -0.567 0.500 0.5002.1.1 Plot what this main effect looks like
ggplot(complete_predictors, aes(x = measure, y = value, color = measure, fill = measure))+
geom_jitter(height=0.1, width=0.1)+
geom_violin(alpha=.2)+
theme_classic()
2.2 Add interaction terms
omnibus <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors)
summary(omnibus)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: complete_predictors
##
## REML criterion at convergence: -642.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2129 -0.4553 -0.0091 0.4356 3.2627
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002692 0.05188
## Residual 0.009596 0.09796
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -0.82582 0.49946 384.62912
## frequency 0.06945 0.01003 384.62895
## measurewordbank_inv 0.42422 0.62420 330.00002
## measurekuperman_inv 0.85819 0.62420 330.00003
## measureprop_say_naive_combined 0.03084 0.62420 329.99998
## preschoolness -0.02781 0.01540 384.62895
## helpfulness -0.05365 0.01749 384.62895
## concreteness 0.19358 0.10204 384.62911
## frequency:measurewordbank_inv -0.02286 0.01254 330.00000
## frequency:measurekuperman_inv -0.05099 0.01254 330.00000
## frequency:measureprop_say_naive_combined 0.06156 0.01254 330.00000
## measurewordbank_inv:preschoolness 0.04998 0.01925 330.00000
## measurekuperman_inv:preschoolness 0.05225 0.01925 330.00000
## measureprop_say_naive_combined:preschoolness 0.08583 0.01925 330.00000
## measurewordbank_inv:helpfulness 0.05094 0.02186 330.00000
## measurekuperman_inv:helpfulness 0.05753 0.02186 330.00000
## measureprop_say_naive_combined:helpfulness 0.10780 0.02186 330.00000
## measurewordbank_inv:concreteness -0.07652 0.12752 330.00002
## measurekuperman_inv:concreteness -0.19000 0.12752 330.00004
## measureprop_say_naive_combined:concreteness -0.16889 0.12752 329.99998
## t value Pr(>|t|)
## (Intercept) -1.653 0.09906 .
## frequency 6.923 1.87e-11 ***
## measurewordbank_inv 0.680 0.49722
## measurekuperman_inv 1.375 0.17011
## measureprop_say_naive_combined 0.049 0.96063
## preschoolness -1.805 0.07185 .
## helpfulness -3.067 0.00231 **
## concreteness 1.897 0.05855 .
## frequency:measurewordbank_inv -1.823 0.06916 .
## frequency:measurekuperman_inv -4.066 5.98e-05 ***
## frequency:measureprop_say_naive_combined 4.909 1.44e-06 ***
## measurewordbank_inv:preschoolness 2.596 0.00985 **
## measurekuperman_inv:preschoolness 2.714 0.00700 **
## measureprop_say_naive_combined:preschoolness 4.458 1.13e-05 ***
## measurewordbank_inv:helpfulness 2.330 0.02040 *
## measurekuperman_inv:helpfulness 2.632 0.00889 **
## measureprop_say_naive_combined:helpfulness 4.932 1.30e-06 ***
## measurewordbank_inv:concreteness -0.600 0.54889
## measurekuperman_inv:concreteness -1.490 0.13719
## measureprop_say_naive_combined:concreteness -1.324 0.18630
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 12.3 Reliability of frequency interaction
Reference group: Kuperman AoA (shallowest slope)
2.3.1 Model
frequency <- lmer(value ~ frequency + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors)
summary(frequency)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency + preschoolness * measure + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors
##
## REML criterion at convergence: -586.1
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.13997 -0.53364 0.02236 0.51589 2.97192
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002083 0.04565
## Residual 0.012030 0.10968
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -0.808571 0.533086 413.513290
## frequency 0.066382 0.006458 109.999998
## preschoolness -0.026234 0.015912 400.929066
## measurewordbank_inv 0.295881 0.694419 333.000114
## measurekuperman_inv 0.571955 0.694419 333.000097
## measureprop_say_naive_combined 0.376403 0.694419 333.000067
## helpfulness -0.052868 0.018617 412.539460
## concreteness 0.193284 0.109351 414.900631
## preschoolness:measurewordbank_inv 0.038289 0.020324 332.999999
## preschoolness:measurekuperman_inv 0.026169 0.020324 332.999999
## preschoolness:measureprop_say_naive_combined 0.117308 0.020324 332.999999
## measurewordbank_inv:helpfulness 0.045117 0.024212 332.999999
## measurekuperman_inv:helpfulness 0.044548 0.024212 332.999999
## measureprop_say_naive_combined:helpfulness 0.123470 0.024212 332.999998
## measurewordbank_inv:concreteness -0.074284 0.142773 333.000113
## measurekuperman_inv:concreteness -0.185018 0.142773 333.000098
## measureprop_say_naive_combined:concreteness -0.174905 0.142773 333.000067
## t value Pr(>|t|)
## (Intercept) -1.517 0.13009
## frequency 10.279 < 2e-16 ***
## preschoolness -1.649 0.10000
## measurewordbank_inv 0.426 0.67032
## measurekuperman_inv 0.824 0.41073
## measureprop_say_naive_combined 0.542 0.58815
## helpfulness -2.840 0.00474 **
## concreteness 1.768 0.07787 .
## preschoolness:measurewordbank_inv 1.884 0.06044 .
## preschoolness:measurekuperman_inv 1.288 0.19878
## preschoolness:measureprop_say_naive_combined 5.772 1.79e-08 ***
## measurewordbank_inv:helpfulness 1.863 0.06328 .
## measurekuperman_inv:helpfulness 1.840 0.06667 .
## measureprop_say_naive_combined:helpfulness 5.100 5.73e-07 ***
## measurewordbank_inv:concreteness -0.520 0.60321
## measurekuperman_inv:concreteness -1.296 0.19591
## measureprop_say_naive_combined:concreteness -1.225 0.22142
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus, frequency)## Data: complete_predictors
## Models:
## frequency: value ~ frequency + preschoolness * measure + helpfulness * measure +
## frequency: concreteness * measure + (1 | word)
## omnibus: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## frequency 19 -646.73 -568.24 342.37 -684.73
## omnibus 22 -721.83 -630.94 382.91 -765.83 81.095 3 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 12.3.2 Plot
freq_115_plot <- ggplot(complete_predictors, aes(x = frequency, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
freq_115_plot
2.3.3 Pairwise slope comparisons
measure_frequency <- emtrends(omnibus, "measure", var = "frequency")
print(measure_frequency)## measure frequency.trend SE df lower.CL upper.CL
## picture_naming_inv 0.0695 0.01 385 0.04973 0.0892
## wordbank_inv 0.0466 0.01 385 0.02687 0.0663
## kuperman_inv 0.0185 0.01 385 -0.00126 0.0382
## prop_say_naive_combined 0.1310 0.01 385 0.11129 0.1507
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_frequency)## contrast estimate SE df t.ratio
## picture_naming_inv - wordbank_inv 0.0229 0.0125 330 1.823
## picture_naming_inv - kuperman_inv 0.0510 0.0125 330 4.066
## picture_naming_inv - prop_say_naive_combined -0.0616 0.0125 330 -4.909
## wordbank_inv - kuperman_inv 0.0281 0.0125 330 2.243
## wordbank_inv - prop_say_naive_combined -0.0844 0.0125 330 -6.733
## kuperman_inv - prop_say_naive_combined -0.1125 0.0125 330 -8.976
## p.value
## 0.2642
## 0.0003
## <.0001
## 0.1139
## <.0001
## <.0001
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 4 estimates2.4 Reliability of concreteness interaction
Reference group: Kuperman AoA
2.4.1 Model
concreteness <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness +
(1|word), data=complete_predictors)
summary(concreteness)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness + (1 | word)
## Data: complete_predictors
##
## REML criterion at convergence: -647.6
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2720 -0.4722 -0.0048 0.4366 3.4687
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002693 0.05189
## Residual 0.009592 0.09794
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -0.30047 0.32773 118.74891
## frequency 0.06935 0.01003 385.93577
## measurewordbank_inv 0.05492 0.10404 333.00001
## measurekuperman_inv -0.05882 0.10404 333.00001
## measureprop_say_naive_combined -0.78426 0.10404 333.00001
## preschoolness -0.02695 0.01539 385.55459
## helpfulness -0.05173 0.01743 384.39859
## concreteness 0.08473 0.06568 109.99989
## frequency:measurewordbank_inv -0.02279 0.01254 333.00001
## frequency:measurekuperman_inv -0.05081 0.01254 333.00001
## frequency:measureprop_say_naive_combined 0.06172 0.01254 333.00001
## measurewordbank_inv:preschoolness 0.04938 0.01922 333.00001
## measurekuperman_inv:preschoolness 0.05075 0.01922 333.00001
## measureprop_say_naive_combined:preschoolness 0.08450 0.01922 333.00001
## measurewordbank_inv:helpfulness 0.04959 0.02174 333.00001
## measurekuperman_inv:helpfulness 0.05418 0.02174 333.00001
## measureprop_say_naive_combined:helpfulness 0.10482 0.02174 333.00001
## t value Pr(>|t|)
## (Intercept) -0.917 0.36109
## frequency 6.914 1.97e-11 ***
## measurewordbank_inv 0.528 0.59792
## measurekuperman_inv -0.565 0.57219
## measureprop_say_naive_combined -7.538 4.55e-13 ***
## preschoolness -1.751 0.08071 .
## helpfulness -2.967 0.00319 **
## concreteness 1.290 0.19971
## frequency:measurewordbank_inv -1.818 0.06995 .
## frequency:measurekuperman_inv -4.053 6.29e-05 ***
## frequency:measureprop_say_naive_combined 4.924 1.34e-06 ***
## measurewordbank_inv:preschoolness 2.569 0.01063 *
## measurekuperman_inv:preschoolness 2.640 0.00867 **
## measureprop_say_naive_combined:preschoolness 4.396 1.48e-05 ***
## measurewordbank_inv:helpfulness 2.281 0.02317 *
## measurekuperman_inv:helpfulness 2.492 0.01318 *
## measureprop_say_naive_combined:helpfulness 4.822 2.16e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus, concreteness)## Data: complete_predictors
## Models:
## concreteness: value ~ frequency * measure + preschoolness * measure + helpfulness *
## concreteness: measure + concreteness + (1 | word)
## omnibus: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## concreteness 19 -724.87 -646.38 381.44 -762.87
## omnibus 22 -721.83 -630.94 382.91 -765.83 2.9553 3 0.39862.4.2 Plot
conc_115_plot <- ggplot(complete_predictors, aes(x = concreteness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
conc_115_plot
2.4.3 Pairwise slope comparisons
measure_concreteness <- emtrends(omnibus, "measure", var = "concreteness")
print(measure_concreteness)## measure concreteness.trend SE df lower.CL upper.CL
## picture_naming_inv 0.19358 0.102 385 -0.00704 0.394
## wordbank_inv 0.11707 0.102 385 -0.08356 0.318
## kuperman_inv 0.00358 0.102 385 -0.19704 0.204
## prop_say_naive_combined 0.02470 0.102 385 -0.17593 0.225
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_concreteness)## contrast estimate SE df t.ratio
## picture_naming_inv - wordbank_inv 0.0765 0.128 330 0.600
## picture_naming_inv - kuperman_inv 0.1900 0.128 330 1.490
## picture_naming_inv - prop_say_naive_combined 0.1689 0.128 330 1.324
## wordbank_inv - kuperman_inv 0.1135 0.128 330 0.890
## wordbank_inv - prop_say_naive_combined 0.0924 0.128 330 0.724
## kuperman_inv - prop_say_naive_combined -0.0211 0.128 330 -0.166
## p.value
## 0.9320
## 0.4447
## 0.5480
## 0.8101
## 0.8873
## 0.9984
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 4 estimates2.5 Reliability of preschoolness interaction
Reference group: picture_naming_inv (shallowest slope)
2.5.1 Model
complete_predictors_presch <- complete_predictors
complete_predictors_presch$measure <- relevel(as.factor(complete_predictors_presch$measure), ref = "picture_naming_inv")
omnibus_presch <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_presch)
preschoolness <- lmer(value ~ frequency*measure + preschoolness + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_presch)
summary(omnibus_presch)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: complete_predictors_presch
##
## REML criterion at convergence: -642.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2129 -0.4553 -0.0091 0.4356 3.2627
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002692 0.05188
## Residual 0.009596 0.09796
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -0.82582 0.49946 384.62912
## frequency 0.06945 0.01003 384.62895
## measurewordbank_inv 0.42422 0.62420 330.00002
## measurekuperman_inv 0.85819 0.62420 330.00003
## measureprop_say_naive_combined 0.03084 0.62420 329.99998
## preschoolness -0.02781 0.01540 384.62895
## helpfulness -0.05365 0.01749 384.62895
## concreteness 0.19358 0.10204 384.62911
## frequency:measurewordbank_inv -0.02286 0.01254 330.00000
## frequency:measurekuperman_inv -0.05099 0.01254 330.00000
## frequency:measureprop_say_naive_combined 0.06156 0.01254 330.00000
## measurewordbank_inv:preschoolness 0.04998 0.01925 330.00000
## measurekuperman_inv:preschoolness 0.05225 0.01925 330.00000
## measureprop_say_naive_combined:preschoolness 0.08583 0.01925 330.00000
## measurewordbank_inv:helpfulness 0.05094 0.02186 330.00000
## measurekuperman_inv:helpfulness 0.05753 0.02186 330.00000
## measureprop_say_naive_combined:helpfulness 0.10780 0.02186 330.00000
## measurewordbank_inv:concreteness -0.07652 0.12752 330.00002
## measurekuperman_inv:concreteness -0.19000 0.12752 330.00004
## measureprop_say_naive_combined:concreteness -0.16889 0.12752 329.99998
## t value Pr(>|t|)
## (Intercept) -1.653 0.09906 .
## frequency 6.923 1.87e-11 ***
## measurewordbank_inv 0.680 0.49722
## measurekuperman_inv 1.375 0.17011
## measureprop_say_naive_combined 0.049 0.96063
## preschoolness -1.805 0.07185 .
## helpfulness -3.067 0.00231 **
## concreteness 1.897 0.05855 .
## frequency:measurewordbank_inv -1.823 0.06916 .
## frequency:measurekuperman_inv -4.066 5.98e-05 ***
## frequency:measureprop_say_naive_combined 4.909 1.44e-06 ***
## measurewordbank_inv:preschoolness 2.596 0.00985 **
## measurekuperman_inv:preschoolness 2.714 0.00700 **
## measureprop_say_naive_combined:preschoolness 4.458 1.13e-05 ***
## measurewordbank_inv:helpfulness 2.330 0.02040 *
## measurekuperman_inv:helpfulness 2.632 0.00889 **
## measureprop_say_naive_combined:helpfulness 4.932 1.30e-06 ***
## measurewordbank_inv:concreteness -0.600 0.54889
## measurekuperman_inv:concreteness -1.490 0.13719
## measureprop_say_naive_combined:concreteness -1.324 0.18630
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1summary(preschoolness)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors_presch
##
## REML criterion at convergence: -642
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.4421 -0.4126 -0.0238 0.4537 3.1603
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002568 0.05067
## Residual 0.010094 0.10047
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -0.785718 0.506875 393.085074
## frequency 0.059255 0.009840 376.010180
## measurewordbank_inv 0.381587 0.639943 333.000032
## measurekuperman_inv 0.813619 0.639943 333.000034
## measureprop_say_naive_combined -0.042378 0.639943 333.000026
## preschoolness 0.019207 0.009915 109.999999
## helpfulness -0.052416 0.017751 393.106995
## concreteness 0.177373 0.103492 392.805119
## frequency:measurewordbank_inv -0.012019 0.012125 333.000000
## frequency:measurekuperman_inv -0.039653 0.012125 333.000000
## frequency:measureprop_say_naive_combined 0.080177 0.012125 333.000000
## measurewordbank_inv:helpfulness 0.049624 0.022412 333.000002
## measurekuperman_inv:helpfulness 0.056155 0.022412 333.000002
## measureprop_say_naive_combined:helpfulness 0.105546 0.022412 333.000001
## measurewordbank_inv:concreteness -0.059283 0.130608 333.000033
## measurekuperman_inv:concreteness -0.171986 0.130608 333.000035
## measureprop_say_naive_combined:concreteness -0.139290 0.130608 333.000026
## t value Pr(>|t|)
## (Intercept) -1.550 0.12192
## frequency 6.022 4.11e-09 ***
## measurewordbank_inv 0.596 0.55139
## measurekuperman_inv 1.271 0.20448
## measureprop_say_naive_combined -0.066 0.94724
## preschoolness 1.937 0.05529 .
## helpfulness -2.953 0.00334 **
## concreteness 1.714 0.08734 .
## frequency:measurewordbank_inv -0.991 0.32229
## frequency:measurekuperman_inv -3.270 0.00119 **
## frequency:measureprop_say_naive_combined 6.613 1.50e-10 ***
## measurewordbank_inv:helpfulness 2.214 0.02749 *
## measurekuperman_inv:helpfulness 2.506 0.01270 *
## measureprop_say_naive_combined:helpfulness 4.709 3.65e-06 ***
## measurewordbank_inv:concreteness -0.454 0.65019
## measurekuperman_inv:concreteness -1.317 0.18881
## measureprop_say_naive_combined:concreteness -1.066 0.28698
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_presch, preschoolness)## Data: complete_predictors_presch
## Models:
## preschoolness: value ~ frequency * measure + preschoolness + helpfulness * measure +
## preschoolness: concreteness * measure + (1 | word)
## omnibus_presch: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_presch: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## preschoolness 19 -707.28 -628.79 372.64 -745.28
## omnibus_presch 22 -721.83 -630.94 382.91 -765.83 20.548 3 0.0001307 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 12.5.2 Plot
presch_115_plot <- ggplot(complete_predictors, aes(x = preschoolness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
presch_115_plot
2.5.3 Pairwise slope comparisons
measure_preschoolness <- emtrends(omnibus, "measure", var = "preschoolness")
print(measure_preschoolness)## measure preschoolness.trend SE df lower.CL upper.CL
## picture_naming_inv -0.0278 0.0154 385 -0.05809 0.00248
## wordbank_inv 0.0222 0.0154 385 -0.00811 0.05246
## kuperman_inv 0.0244 0.0154 385 -0.00585 0.05473
## prop_say_naive_combined 0.0580 0.0154 385 0.02773 0.08831
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_preschoolness)## contrast estimate SE df t.ratio
## picture_naming_inv - wordbank_inv -0.04998 0.0193 330 -2.596
## picture_naming_inv - kuperman_inv -0.05225 0.0193 330 -2.714
## picture_naming_inv - prop_say_naive_combined -0.08583 0.0193 330 -4.458
## wordbank_inv - kuperman_inv -0.00226 0.0193 330 -0.118
## wordbank_inv - prop_say_naive_combined -0.03585 0.0193 330 -1.862
## kuperman_inv - prop_say_naive_combined -0.03358 0.0193 330 -1.744
## p.value
## 0.0482
## 0.0351
## 0.0001
## 0.9994
## 0.2466
## 0.3025
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 4 estimates2.6 Reliability of helpfulness interaction
Reference group: Kuperman AoA (shallowest slope)
2.6.1 Model
helpfulness <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness + concreteness*measure +
(1|word), data=complete_predictors)
summary(helpfulness)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness +
## concreteness * measure + (1 | word)
## Data: complete_predictors
##
## REML criterion at convergence: -637.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.5224 -0.4155 0.0028 0.4568 2.8569
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002538 0.05037
## Residual 0.010213 0.10106
## Number of obs: 460, groups: word, 115
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -8.143e-01 5.088e-01 3.948e+02
## frequency 6.493e-02 1.015e-02 3.918e+02
## measurewordbank_inv 4.133e-01 6.439e-01 3.330e+02
## measurekuperman_inv 8.459e-01 6.439e-01 3.330e+02
## measureprop_say_naive_combined 7.801e-03 6.439e-01 3.330e+02
## preschoolness -2.670e-02 1.569e-02 3.947e+02
## helpfulness 4.156e-04 1.126e-02 1.100e+02
## concreteness 1.612e-01 1.036e-01 3.933e+02
## frequency:measurewordbank_inv -1.860e-02 1.280e-02 3.330e+02
## frequency:measurekuperman_inv -4.617e-02 1.280e-02 3.330e+02
## frequency:measureprop_say_naive_combined 7.059e-02 1.280e-02 3.330e+02
## measurewordbank_inv:preschoolness 4.894e-02 1.986e-02 3.330e+02
## measurekuperman_inv:preschoolness 5.107e-02 1.986e-02 3.330e+02
## measureprop_say_naive_combined:preschoolness 8.363e-02 1.986e-02 3.330e+02
## measurewordbank_inv:concreteness -4.597e-02 1.309e-01 3.330e+02
## measurekuperman_inv:concreteness -1.555e-01 1.309e-01 3.330e+02
## measureprop_say_naive_combined:concreteness -1.042e-01 1.309e-01 3.330e+02
## t value Pr(>|t|)
## (Intercept) -1.600 0.110293
## frequency 6.394 4.61e-10 ***
## measurewordbank_inv 0.642 0.521384
## measurekuperman_inv 1.314 0.189875
## measureprop_say_naive_combined 0.012 0.990341
## preschoolness -1.702 0.089523 .
## helpfulness 0.037 0.970617
## concreteness 1.555 0.120663
## frequency:measurewordbank_inv -1.453 0.147126
## frequency:measurekuperman_inv -3.608 0.000356 ***
## frequency:measureprop_say_naive_combined 5.516 6.98e-08 ***
## measurewordbank_inv:preschoolness 2.465 0.014206 *
## measurekuperman_inv:preschoolness 2.572 0.010536 *
## measureprop_say_naive_combined:preschoolness 4.212 3.26e-05 ***
## measurewordbank_inv:concreteness -0.351 0.725619
## measurekuperman_inv:concreteness -1.188 0.235582
## measureprop_say_naive_combined:concreteness -0.796 0.426333
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus, helpfulness)## Data: complete_predictors
## Models:
## helpfulness: value ~ frequency * measure + preschoolness * measure + helpfulness +
## helpfulness: concreteness * measure + (1 | word)
## omnibus: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## helpfulness 19 -703.21 -624.71 370.60 -741.21
## omnibus 22 -721.83 -630.94 382.91 -765.83 24.621 3 1.853e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 12.6.2 Plot
help_115_plot <- ggplot(complete_predictors, aes(x = helpfulness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
help_115_plot
2.6.3 Pairwise slope comparisons
measure_helpfulness <- emtrends(omnibus, "measure", var = "helpfulness")
print(measure_helpfulness)## measure helpfulness.trend SE df lower.CL upper.CL
## picture_naming_inv -0.05365 0.0175 385 -0.0880 -0.0193
## wordbank_inv -0.00271 0.0175 385 -0.0371 0.0317
## kuperman_inv 0.00388 0.0175 385 -0.0305 0.0383
## prop_say_naive_combined 0.05415 0.0175 385 0.0198 0.0885
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_helpfulness)## contrast estimate SE df t.ratio
## picture_naming_inv - wordbank_inv -0.05094 0.0219 330 -2.330
## picture_naming_inv - kuperman_inv -0.05753 0.0219 330 -2.632
## picture_naming_inv - prop_say_naive_combined -0.10780 0.0219 330 -4.932
## wordbank_inv - kuperman_inv -0.00659 0.0219 330 -0.301
## wordbank_inv - prop_say_naive_combined -0.05686 0.0219 330 -2.601
## kuperman_inv - prop_say_naive_combined -0.05027 0.0219 330 -2.300
## p.value
## 0.0933
## 0.0439
## <.0001
## 0.9905
## 0.0476
## 0.1001
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 4 estimates3 Analyses excluding picture-naming data (more words) - Inverse AoAs
3.1 Estimate only effects of measure: in general, how much do they differ?
#relevel measure so that picture-naming (closest thing we have to ground truth) is reference group
complete_predictors_wordbank$measure <- fct_relevel(as.factor(complete_predictors_wordbank$measure),
c("wordbank_inv","prop_say_naive_combined,","kuperman_inv"))
measure_aoa_wb <- lmer(value ~ measure + (1|word), data=complete_predictors_wordbank)
summary(measure_aoa_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ measure + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -1519
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.04565 -0.40539 0.00589 0.36801 2.84046
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.006403 0.08002
## Residual 0.011477 0.10713
## Number of obs: 1184, groups: word, 396
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) 0.507937 0.006720 941.282821 75.591
## measurekuperman_inv -0.265340 0.007614 786.940092 -34.851
## measureprop_say_naive_combined 0.016907 0.007638 788.528089 2.214
## Pr(>|t|)
## (Intercept) <2e-16 ***
## measurekuperman_inv <2e-16 ***
## measureprop_say_naive_combined 0.0271 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) msrkp_
## msrkprmn_nv -0.567
## msrprp_sy__ -0.565 0.4983.1.1 Plot what this main effect looks like
ggplot(complete_predictors_wordbank, aes(x = measure, y = value, color = measure, fill = measure))+
geom_violin(alpha=.4)+
geom_jitter(height=0.1, width=0.1, alpha=.2)+
theme_classic()
3.2 Add interactions with word-level characteristics
omnibus_wb <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_wordbank)
summary(omnibus_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness * measure + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -1993.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.4493 -0.4089 -0.0174 0.3830 5.0502
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.003285 0.05731
## Residual 0.007435 0.08622
## Number of obs: 1181, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 0.029733 0.074503 982.906510
## frequency 0.028415 0.004137 982.906100
## measurekuperman_inv -0.032033 0.087747 777.241602
## measureprop_say_naive_combined -0.805139 0.088269 780.482946
## preschoolness 0.018640 0.006236 982.905903
## helpfulness -0.008855 0.008336 982.906033
## concreteness 0.057833 0.009731 982.906349
## frequency:measurekuperman_inv -0.014893 0.004872 777.241842
## frequency:measureprop_say_naive_combined 0.062472 0.004926 783.190794
## measurekuperman_inv:preschoolness 0.003056 0.007345 777.241968
## measureprop_say_naive_combined:preschoolness 0.062909 0.007374 779.431631
## measurekuperman_inv:helpfulness 0.008049 0.009818 777.241888
## measureprop_say_naive_combined:helpfulness 0.019944 0.009855 779.303170
## measurekuperman_inv:concreteness -0.036386 0.011461 777.241700
## measureprop_say_naive_combined:concreteness 0.034017 0.011537 780.881788
## t value Pr(>|t|)
## (Intercept) 0.399 0.68992
## frequency 6.869 1.15e-11 ***
## measurekuperman_inv -0.365 0.71517
## measureprop_say_naive_combined -9.121 < 2e-16 ***
## preschoolness 2.989 0.00287 **
## helpfulness -1.062 0.28838
## concreteness 5.943 3.87e-09 ***
## frequency:measurekuperman_inv -3.057 0.00232 **
## frequency:measureprop_say_naive_combined 12.682 < 2e-16 ***
## measurekuperman_inv:preschoolness 0.416 0.67742
## measureprop_say_naive_combined:preschoolness 8.531 < 2e-16 ***
## measurekuperman_inv:helpfulness 0.820 0.41254
## measureprop_say_naive_combined:helpfulness 2.024 0.04333 *
## measurekuperman_inv:concreteness -3.175 0.00156 **
## measureprop_say_naive_combined:concreteness 2.948 0.00329 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 13.3 Pairwise comparisons of slopes for different measures
measure_frequency_wb <- emtrends(omnibus_wb, "measure", var = "frequency")
print(measure_frequency_wb)## measure frequency.trend SE df lower.CL upper.CL
## wordbank_inv 0.0284 0.00414 982 0.0203 0.0365
## kuperman_inv 0.0135 0.00414 982 0.0054 0.0216
## prop_say_naive_combined 0.0909 0.00420 998 0.0826 0.0991
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_frequency_wb)## contrast estimate SE df t.ratio p.value
## wordbank_inv - kuperman_inv 0.0149 0.00487 776 3.057 0.0065
## wordbank_inv - prop_say_naive_combined -0.0625 0.00493 782 -12.682 <.0001
## kuperman_inv - prop_say_naive_combined -0.0774 0.00493 782 -15.705 <.0001
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_preschoolness_wb <- emtrends(omnibus_wb, "measure", var = "preschoolness")
print(measure_preschoolness_wb)## measure preschoolness.trend SE df lower.CL upper.CL
## wordbank_inv 0.0186 0.00624 982 0.00640 0.0309
## kuperman_inv 0.0217 0.00624 982 0.00946 0.0339
## prop_say_naive_combined 0.0815 0.00627 988 0.06924 0.0939
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_preschoolness_wb)## contrast estimate SE df t.ratio p.value
## wordbank_inv - kuperman_inv -0.00306 0.00734 776 -0.416 0.9090
## wordbank_inv - prop_say_naive_combined -0.06291 0.00737 778 -8.531 <.0001
## kuperman_inv - prop_say_naive_combined -0.05985 0.00737 778 -8.116 <.0001
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_helpfulness_wb <- emtrends(omnibus_wb, "measure", var = "helpfulness")
print(measure_helpfulness_wb)## measure helpfulness.trend SE df lower.CL upper.CL
## wordbank_inv -0.008855 0.00834 982 -0.02521 0.0075
## kuperman_inv -0.000806 0.00834 982 -0.01716 0.0156
## prop_say_naive_combined 0.011089 0.00838 988 -0.00535 0.0275
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_helpfulness_wb)## contrast estimate SE df t.ratio p.value
## wordbank_inv - kuperman_inv -0.00805 0.00982 776 -0.820 0.6908
## wordbank_inv - prop_say_naive_combined -0.01994 0.00985 778 -2.024 0.1072
## kuperman_inv - prop_say_naive_combined -0.01189 0.00985 778 -1.207 0.4494
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimatesmeasure_concreteness_wb <- emtrends(omnibus_wb, "measure", var = "concreteness")
print(measure_concreteness_wb)## measure concreteness.trend SE df lower.CL upper.CL
## wordbank_inv 0.0578 0.00973 982 0.03874 0.0769
## kuperman_inv 0.0214 0.00973 982 0.00235 0.0405
## prop_say_naive_combined 0.0919 0.00982 992 0.07258 0.1111
##
## Degrees-of-freedom method: kenward-roger
## Confidence level used: 0.95pairs(measure_concreteness_wb)## contrast estimate SE df t.ratio p.value
## wordbank_inv - kuperman_inv 0.0364 0.0115 776 3.175 0.0044
## wordbank_inv - prop_say_naive_combined -0.0340 0.0115 780 -2.948 0.0092
## kuperman_inv - prop_say_naive_combined -0.0704 0.0115 780 -6.102 <.0001
##
## Degrees-of-freedom method: kenward-roger
## P value adjustment: tukey method for comparing a family of 3 estimates3.4 Reliability of frequency interaction
Reference group: Kuperman AoA (flattest slope) ### Model
frequency_wb <- lmer(value ~ frequency + preschoolness*measure + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_wordbank)
summary(frequency_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency + preschoolness * measure + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -1774.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -4.0429 -0.4330 -0.0134 0.4068 3.4837
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.002432 0.04931
## Residual 0.010041 0.10020
## Number of obs: 1181, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -1.383e-01 7.188e-02 9.174e+02
## frequency 4.364e-02 3.049e-03 3.941e+02
## preschoolness 1.717e-02 6.719e-03 1.084e+03
## measurekuperman_inv -1.963e-01 8.059e-02 7.791e+02
## measureprop_say_naive_combined -1.321e-01 8.159e-02 7.876e+02
## helpfulness -1.006e-02 8.988e-03 1.085e+03
## concreteness 7.318e-02 9.969e-03 1.019e+03
## preschoolness:measurekuperman_inv 1.620e-03 8.518e-03 7.791e+02
## preschoolness:measureprop_say_naive_combined 6.953e-02 8.547e-03 7.815e+02
## measurekuperman_inv:helpfulness 6.869e-03 1.140e-02 7.791e+02
## measureprop_say_naive_combined:helpfulness 2.536e-02 1.144e-02 7.815e+02
## measurekuperman_inv:concreteness -2.137e-02 1.203e-02 7.791e+02
## measureprop_say_naive_combined:concreteness -2.598e-02 1.218e-02 7.877e+02
## t value Pr(>|t|)
## (Intercept) -1.924 0.0547 .
## frequency 14.312 < 2e-16 ***
## preschoolness 2.555 0.0107 *
## measurekuperman_inv -2.436 0.0151 *
## measureprop_say_naive_combined -1.619 0.1059
## helpfulness -1.119 0.2632
## concreteness 7.341 4.34e-13 ***
## preschoolness:measurekuperman_inv 0.190 0.8492
## preschoolness:measureprop_say_naive_combined 8.134 1.63e-15 ***
## measurekuperman_inv:helpfulness 0.603 0.5470
## measureprop_say_naive_combined:helpfulness 2.217 0.0269 *
## measurekuperman_inv:concreteness -1.776 0.0761 .
## measureprop_say_naive_combined:concreteness -2.132 0.0333 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_wb, frequency_wb)## Data: complete_predictors_wordbank
## Models:
## frequency_wb: value ~ frequency + preschoolness * measure + helpfulness * measure +
## frequency_wb: concreteness * measure + (1 | word)
## omnibus_wb: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_wb: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## frequency_wb 15 -1850.3 -1774.2 940.17 -1880.3
## omnibus_wb 17 -2085.9 -1999.6 1059.93 -2119.9 239.52 2 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 13.4.1 Plot
freq_plot <- ggplot(filter(complete_predictors_wordbank, measure != "picture_naming_inv"),
aes(x = frequency, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
freq_plot
3.5 Reliability of concreteness interaction
Reference group: Kuperman AoA ### Model
concreteness_wb <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness*measure + concreteness +
(1|word), data=complete_predictors_wordbank)
summary(concreteness_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness *
## measure + concreteness + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -1971.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.7393 -0.4207 -0.0134 0.3819 4.5825
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.003172 0.05632
## Residual 0.007772 0.08816
## Number of obs: 1181, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) 0.037055 0.060233 562.460939
## frequency 0.028216 0.003995 922.759101
## measurekuperman_inv -0.275414 0.043652 779.293478
## measureprop_say_naive_combined -0.578107 0.043965 783.346800
## preschoolness 0.018569 0.006286 996.337843
## helpfulness -0.009081 0.008307 977.784921
## concreteness 0.056739 0.007149 392.334768
## frequency:measurekuperman_inv -0.008263 0.004501 779.293563
## frequency:measureprop_say_naive_combined 0.056597 0.004580 789.016595
## measurekuperman_inv:preschoolness 0.005426 0.007470 779.293597
## measureprop_say_naive_combined:preschoolness 0.060557 0.007498 781.349561
## measurekuperman_inv:helpfulness 0.015560 0.009742 779.293562
## measureprop_say_naive_combined:helpfulness 0.012592 0.009763 780.481294
## t value Pr(>|t|)
## (Intercept) 0.615 0.53868
## frequency 7.063 3.21e-12 ***
## measurekuperman_inv -6.309 4.70e-10 ***
## measureprop_say_naive_combined -13.149 < 2e-16 ***
## preschoolness 2.954 0.00321 **
## helpfulness -1.093 0.27457
## concreteness 7.937 2.20e-14 ***
## frequency:measurekuperman_inv -1.836 0.06675 .
## frequency:measureprop_say_naive_combined 12.358 < 2e-16 ***
## measurekuperman_inv:preschoolness 0.726 0.46784
## measureprop_say_naive_combined:preschoolness 8.077 2.51e-15 ***
## measurekuperman_inv:helpfulness 1.597 0.11062
## measureprop_say_naive_combined:helpfulness 1.290 0.19749
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_wb, concreteness_wb)## Data: complete_predictors_wordbank
## Models:
## concreteness_wb: value ~ frequency * measure + preschoolness * measure + helpfulness *
## concreteness_wb: measure + concreteness + (1 | word)
## omnibus_wb: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_wb: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## concreteness_wb 15 -2053.0 -1976.9 1041.5 -2083.0
## omnibus_wb 17 -2085.9 -1999.6 1059.9 -2119.9 36.866 2 9.877e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 13.5.1 Plot
conc_plot <- ggplot(filter(complete_predictors_wordbank, measure != "picture_naming_inv"),
aes(x = concreteness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
conc_plot
3.6 Reliability of preschoolness interaction
Reference group: Kuperman AoA ### Model
preschoolness_wb <- lmer(value ~ frequency*measure + preschoolness + helpfulness*measure + concreteness*measure +
(1|word), data=complete_predictors_wordbank)
summary(preschoolness_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness + helpfulness * measure +
## concreteness * measure + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -1922.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.6550 -0.3928 -0.0049 0.3837 5.6839
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.003002 0.05479
## Residual 0.008295 0.09107
## Number of obs: 1181, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -2.781e-02 7.556e-02 1.005e+03
## frequency 2.749e-02 4.243e-03 1.022e+03
## measurekuperman_inv -2.396e-02 9.039e-02 7.793e+02
## measureprop_say_naive_combined -6.368e-01 9.088e-02 7.825e+02
## preschoolness 4.042e-02 4.580e-03 3.913e+02
## helpfulness -1.117e-02 8.544e-03 1.021e+03
## concreteness 6.129e-02 9.964e-03 1.020e+03
## frequency:measurekuperman_inv -1.476e-02 5.136e-03 7.793e+02
## frequency:measureprop_say_naive_combined 6.543e-02 5.189e-03 7.855e+02
## measurekuperman_inv:helpfulness 8.374e-03 1.034e-02 7.793e+02
## measureprop_say_naive_combined:helpfulness 2.604e-02 1.038e-02 7.817e+02
## measurekuperman_inv:concreteness -3.687e-02 1.204e-02 7.793e+02
## measureprop_say_naive_combined:concreteness 2.357e-02 1.212e-02 7.830e+02
## t value Pr(>|t|)
## (Intercept) -0.368 0.71293
## frequency 6.480 1.43e-10 ***
## measurekuperman_inv -0.265 0.79105
## measureprop_say_naive_combined -7.007 5.25e-12 ***
## preschoolness 8.825 < 2e-16 ***
## helpfulness -1.307 0.19143
## concreteness 6.151 1.11e-09 ***
## frequency:measurekuperman_inv -2.874 0.00416 **
## frequency:measureprop_say_naive_combined 12.608 < 2e-16 ***
## measurekuperman_inv:helpfulness 0.810 0.41814
## measureprop_say_naive_combined:helpfulness 2.509 0.01232 *
## measurekuperman_inv:concreteness -3.062 0.00228 **
## measureprop_say_naive_combined:concreteness 1.945 0.05209 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_wb, preschoolness_wb)## Data: complete_predictors_wordbank
## Models:
## preschoolness_wb: value ~ frequency * measure + preschoolness + helpfulness * measure +
## preschoolness_wb: concreteness * measure + (1 | word)
## omnibus_wb: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_wb: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## preschoolness_wb 15 -2001.5 -1925.4 1015.8 -2031.5
## omnibus_wb 17 -2085.9 -1999.6 1059.9 -2119.9 88.365 2 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 13.6.1 Plot
presch_plot <- ggplot(filter(complete_predictors_wordbank, measure != "picture_naming_inv"),
aes(x = preschoolness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
presch_plot
3.7 Reliability of helpfulness interaction
Reference group: Kuperman AoA ### Model
helpfulness_wb <- lmer(value ~ frequency*measure + preschoolness*measure + helpfulness + concreteness*measure +
(1|word), data=complete_predictors_wordbank)
summary(helpfulness_wb)## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: value ~ frequency * measure + preschoolness * measure + helpfulness +
## concreteness * measure + (1 | word)
## Data: complete_predictors_wordbank
##
## REML criterion at convergence: -2004.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.5484 -0.4061 -0.0182 0.3865 4.8688
##
## Random effects:
## Groups Name Variance Std.Dev.
## word (Intercept) 0.003278 0.05725
## Residual 0.007455 0.08634
## Number of obs: 1181, groups: word, 395
##
## Fixed effects:
## Estimate Std. Error df
## (Intercept) -1.148e-02 7.017e-02 8.754e+02
## frequency 2.823e-02 4.138e-03 9.841e+02
## measurekuperman_inv 3.718e-03 7.625e-02 7.792e+02
## measureprop_say_naive_combined -7.157e-01 7.654e-02 7.813e+02
## preschoolness 1.809e-02 6.231e-03 9.823e+02
## helpfulness 4.232e-04 6.119e-03 3.912e+02
## concreteness 6.044e-02 9.605e-03 9.619e+02
## frequency:measurekuperman_inv -1.474e-02 4.875e-03 7.792e+02
## frequency:measureprop_say_naive_combined 6.290e-02 4.928e-03 7.851e+02
## measurekuperman_inv:preschoolness 3.535e-03 7.332e-03 7.792e+02
## measureprop_say_naive_combined:preschoolness 6.401e-02 7.364e-03 7.817e+02
## measurekuperman_inv:concreteness -3.865e-02 1.114e-02 7.792e+02
## measureprop_say_naive_combined:concreteness 2.823e-02 1.119e-02 7.820e+02
## t value Pr(>|t|)
## (Intercept) -0.164 0.870132
## frequency 6.823 1.56e-11 ***
## measurekuperman_inv 0.049 0.961125
## measureprop_say_naive_combined -9.351 < 2e-16 ***
## preschoolness 2.903 0.003779 **
## helpfulness 0.069 0.944896
## concreteness 6.293 4.74e-10 ***
## frequency:measurekuperman_inv -3.022 0.002589 **
## frequency:measureprop_say_naive_combined 12.763 < 2e-16 ***
## measurekuperman_inv:preschoolness 0.482 0.629817
## measureprop_say_naive_combined:preschoolness 8.692 < 2e-16 ***
## measurekuperman_inv:concreteness -3.470 0.000549 ***
## measureprop_say_naive_combined:concreteness 2.522 0.011862 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1anova(omnibus_wb, helpfulness_wb)## Data: complete_predictors_wordbank
## Models:
## helpfulness_wb: value ~ frequency * measure + preschoolness * measure + helpfulness +
## helpfulness_wb: concreteness * measure + (1 | word)
## omnibus_wb: value ~ frequency * measure + preschoolness * measure + helpfulness *
## omnibus_wb: measure + concreteness * measure + (1 | word)
## npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
## helpfulness_wb 15 -2085.7 -2009.6 1057.8 -2115.7
## omnibus_wb 17 -2085.9 -1999.6 1059.9 -2119.9 4.1849 2 0.12343.7.1 Plot
help_plot <- ggplot(filter(complete_predictors_wordbank, measure != "picture_naming_inv"),
aes(x = helpfulness, y = value, color = as.factor(measure)))+
geom_point()+
geom_smooth(method="lm")+
scale_color_brewer(palette = "Set1")+
theme_classic()
help_plot