We use weakly regularizing priors.
Primary model
including both correct and incorrect responses
## Family: gaussian
## Links: mu = identity; sigma = identity
## Formula: rt ~ type.numeric * context.numeric + (type.numeric * context.numeric | item) + (1 + type.numeric * context.numeric | participant)
## Data: filter(for_mod, is_critical) (Number of observations: 2067)
## Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
## total post-warmup draws = 4000
##
## Multilevel Hyperparameters:
## ~item (Number of levels: 32)
## Estimate Est.Error l-95% CI
## sd(Intercept) 54.58 12.79 30.86
## sd(type.numeric) 148.34 26.45 100.65
## sd(context.numeric) 44.83 25.01 3.20
## sd(type.numeric:context.numeric) 44.11 33.19 1.69
## cor(Intercept,type.numeric) -0.44 0.22 -0.82
## cor(Intercept,context.numeric) -0.13 0.36 -0.76
## cor(type.numeric,context.numeric) -0.22 0.35 -0.82
## cor(Intercept,type.numeric:context.numeric) -0.03 0.43 -0.80
## cor(type.numeric,type.numeric:context.numeric) 0.04 0.42 -0.77
## cor(context.numeric,type.numeric:context.numeric) 0.11 0.43 -0.75
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 82.29 1.00 2158
## sd(type.numeric) 206.20 1.00 2039
## sd(context.numeric) 96.45 1.01 1159
## sd(type.numeric:context.numeric) 123.52 1.00 1840
## cor(Intercept,type.numeric) 0.03 1.00 1060
## cor(Intercept,context.numeric) 0.61 1.00 3027
## cor(type.numeric,context.numeric) 0.58 1.00 3346
## cor(Intercept,type.numeric:context.numeric) 0.77 1.00 5028
## cor(type.numeric,type.numeric:context.numeric) 0.80 1.00 4640
## cor(context.numeric,type.numeric:context.numeric) 0.84 1.00 3220
## Tail_ESS
## sd(Intercept) 2561
## sd(type.numeric) 3098
## sd(context.numeric) 1620
## sd(type.numeric:context.numeric) 1767
## cor(Intercept,type.numeric) 1993
## cor(Intercept,context.numeric) 3084
## cor(type.numeric,context.numeric) 2490
## cor(Intercept,type.numeric:context.numeric) 2856
## cor(type.numeric,type.numeric:context.numeric) 2836
## cor(context.numeric,type.numeric:context.numeric) 3217
##
## ~participant (Number of levels: 88)
## Estimate Est.Error l-95% CI
## sd(Intercept) 219.27 19.16 184.89
## sd(type.numeric) 69.05 33.79 4.72
## sd(context.numeric) 30.97 22.24 1.25
## sd(type.numeric:context.numeric) 86.20 48.94 5.32
## cor(Intercept,type.numeric) -0.05 0.27 -0.59
## cor(Intercept,context.numeric) 0.17 0.38 -0.65
## cor(type.numeric,context.numeric) 0.02 0.44 -0.81
## cor(Intercept,type.numeric:context.numeric) -0.33 0.33 -0.86
## cor(type.numeric,type.numeric:context.numeric) -0.10 0.41 -0.81
## cor(context.numeric,type.numeric:context.numeric) -0.10 0.44 -0.84
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 258.46 1.00 1093
## sd(type.numeric) 130.41 1.00 627
## sd(context.numeric) 82.66 1.00 1509
## sd(type.numeric:context.numeric) 187.16 1.01 1104
## cor(Intercept,type.numeric) 0.49 1.00 3634
## cor(Intercept,context.numeric) 0.81 1.00 4526
## cor(type.numeric,context.numeric) 0.83 1.00 2905
## cor(Intercept,type.numeric:context.numeric) 0.42 1.00 4508
## cor(type.numeric,type.numeric:context.numeric) 0.70 1.00 2307
## cor(context.numeric,type.numeric:context.numeric) 0.76 1.00 2301
## Tail_ESS
## sd(Intercept) 1947
## sd(type.numeric) 1134
## sd(context.numeric) 1925
## sd(type.numeric:context.numeric) 1452
## cor(Intercept,type.numeric) 2325
## cor(Intercept,context.numeric) 2613
## cor(type.numeric,context.numeric) 2518
## cor(Intercept,type.numeric:context.numeric) 2288
## cor(type.numeric,type.numeric:context.numeric) 2917
## cor(context.numeric,type.numeric:context.numeric) 3152
##
## Regression Coefficients:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## Intercept 999.86 27.25 945.94 1053.00 1.01 546
## type.numeric -65.94 30.67 -126.25 -7.63 1.00 2392
## context.numeric 21.86 19.28 -16.02 60.33 1.00 4329
## type.numeric:context.numeric 57.18 34.00 -8.77 123.27 1.00 4987
## Tail_ESS
## Intercept 1096
## type.numeric 2863
## context.numeric 3007
## type.numeric:context.numeric 3071
##
## Further Distributional Parameters:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma 359.22 5.99 347.72 371.25 1.00 4192 3188
##
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
Additional models
Only correct
It’s slightly clearer if we filter for only the correct answers
## Family: gaussian
## Links: mu = identity; sigma = identity
## Formula: rt ~ type.numeric * context.numeric + (type.numeric * context.numeric | item) + (1 + type.numeric * context.numeric | participant)
## Data: filter(filter(for_mod, is_critical), correct == 1) (Number of observations: 2018)
## Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
## total post-warmup draws = 4000
##
## Multilevel Hyperparameters:
## ~item (Number of levels: 32)
## Estimate Est.Error l-95% CI
## sd(Intercept) 55.31 12.72 32.23
## sd(type.numeric) 151.14 25.58 106.62
## sd(context.numeric) 38.43 24.01 2.27
## sd(type.numeric:context.numeric) 45.52 34.15 2.06
## cor(Intercept,type.numeric) -0.49 0.22 -0.87
## cor(Intercept,context.numeric) -0.07 0.38 -0.75
## cor(type.numeric,context.numeric) -0.15 0.39 -0.82
## cor(Intercept,type.numeric:context.numeric) 0.03 0.43 -0.79
## cor(type.numeric,type.numeric:context.numeric) 0.01 0.41 -0.77
## cor(context.numeric,type.numeric:context.numeric) 0.09 0.45 -0.80
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 82.08 1.00 1602
## sd(type.numeric) 205.69 1.00 1997
## sd(context.numeric) 89.73 1.00 1224
## sd(type.numeric:context.numeric) 127.03 1.00 1770
## cor(Intercept,type.numeric) 0.01 1.00 777
## cor(Intercept,context.numeric) 0.69 1.00 3121
## cor(type.numeric,context.numeric) 0.66 1.00 3135
## cor(Intercept,type.numeric:context.numeric) 0.80 1.00 4239
## cor(type.numeric,type.numeric:context.numeric) 0.77 1.00 4553
## cor(context.numeric,type.numeric:context.numeric) 0.85 1.00 2178
## Tail_ESS
## sd(Intercept) 2091
## sd(type.numeric) 2558
## sd(context.numeric) 1565
## sd(type.numeric:context.numeric) 2095
## cor(Intercept,type.numeric) 1058
## cor(Intercept,context.numeric) 2670
## cor(type.numeric,context.numeric) 2166
## cor(Intercept,type.numeric:context.numeric) 2587
## cor(type.numeric,type.numeric:context.numeric) 3260
## cor(context.numeric,type.numeric:context.numeric) 2782
##
## ~participant (Number of levels: 88)
## Estimate Est.Error l-95% CI
## sd(Intercept) 218.70 17.98 186.62
## sd(type.numeric) 63.05 32.33 5.32
## sd(context.numeric) 33.51 23.02 1.75
## sd(type.numeric:context.numeric) 78.39 50.09 4.12
## cor(Intercept,type.numeric) -0.16 0.29 -0.72
## cor(Intercept,context.numeric) 0.17 0.37 -0.61
## cor(type.numeric,context.numeric) 0.07 0.44 -0.78
## cor(Intercept,type.numeric:context.numeric) -0.25 0.35 -0.85
## cor(type.numeric,type.numeric:context.numeric) -0.01 0.42 -0.78
## cor(context.numeric,type.numeric:context.numeric) -0.12 0.44 -0.84
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 256.42 1.00 991
## sd(type.numeric) 123.17 1.01 524
## sd(context.numeric) 86.04 1.00 1231
## sd(type.numeric:context.numeric) 183.16 1.00 1023
## cor(Intercept,type.numeric) 0.46 1.00 3104
## cor(Intercept,context.numeric) 0.82 1.00 3926
## cor(type.numeric,context.numeric) 0.82 1.00 2496
## cor(Intercept,type.numeric:context.numeric) 0.54 1.00 3505
## cor(type.numeric,type.numeric:context.numeric) 0.76 1.00 2359
## cor(context.numeric,type.numeric:context.numeric) 0.74 1.00 1973
## Tail_ESS
## sd(Intercept) 2081
## sd(type.numeric) 633
## sd(context.numeric) 2066
## sd(type.numeric:context.numeric) 1893
## cor(Intercept,type.numeric) 1799
## cor(Intercept,context.numeric) 2477
## cor(type.numeric,context.numeric) 2482
## cor(Intercept,type.numeric:context.numeric) 2317
## cor(type.numeric,type.numeric:context.numeric) 2540
## cor(context.numeric,type.numeric:context.numeric) 2890
##
## Regression Coefficients:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## Intercept 995.03 26.70 942.96 1046.48 1.01 477
## type.numeric -72.67 30.99 -133.63 -11.04 1.00 1593
## context.numeric 22.14 18.08 -13.69 58.60 1.00 3822
## type.numeric:context.numeric 84.41 33.47 17.07 148.81 1.00 4330
## Tail_ESS
## Intercept 1050
## type.numeric 2565
## context.numeric 2541
## type.numeric:context.numeric 3168
##
## Further Distributional Parameters:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma 349.42 6.07 337.67 361.28 1.00 3422 3103
##
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
Sum multi-word
As mentioned, there’s some question of what to do with the multi-word
versus one-word stuff. Here we
- filter for only correct answers
- add up the RTs
- add a predictor for whether it was multiword (because multi-word
will be longer RTs overall)
## Family: gaussian
## Links: mu = identity; sigma = identity
## Formula: rt ~ type.numeric * context.numeric + multi + (type.numeric * context.numeric | item) + (1 + type.numeric * context.numeric + multi | participant)
## Data: grouped_mod (Number of observations: 1355)
## Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
## total post-warmup draws = 4000
##
## Multilevel Hyperparameters:
## ~item (Number of levels: 32)
## Estimate Est.Error l-95% CI
## sd(Intercept) 93.01 25.24 47.96
## sd(type.numeric) 166.99 33.38 105.58
## sd(context.numeric) 44.05 31.02 1.71
## sd(type.numeric:context.numeric) 54.47 41.27 2.26
## cor(Intercept,type.numeric) -0.18 0.25 -0.66
## cor(Intercept,context.numeric) 0.03 0.40 -0.73
## cor(type.numeric,context.numeric) -0.12 0.40 -0.83
## cor(Intercept,type.numeric:context.numeric) 0.11 0.43 -0.74
## cor(type.numeric,type.numeric:context.numeric) -0.02 0.43 -0.81
## cor(context.numeric,type.numeric:context.numeric) 0.08 0.45 -0.78
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 148.93 1.00 984
## sd(type.numeric) 239.39 1.00 2395
## sd(context.numeric) 114.26 1.00 1282
## sd(type.numeric:context.numeric) 151.07 1.00 2253
## cor(Intercept,type.numeric) 0.32 1.00 1246
## cor(Intercept,context.numeric) 0.78 1.00 4674
## cor(type.numeric,context.numeric) 0.68 1.00 4585
## cor(Intercept,type.numeric:context.numeric) 0.83 1.00 5086
## cor(type.numeric,type.numeric:context.numeric) 0.80 1.00 5388
## cor(context.numeric,type.numeric:context.numeric) 0.84 1.00 3374
## Tail_ESS
## sd(Intercept) 1448
## sd(type.numeric) 2349
## sd(context.numeric) 1857
## sd(type.numeric:context.numeric) 1913
## cor(Intercept,type.numeric) 1210
## cor(Intercept,context.numeric) 3020
## cor(type.numeric,context.numeric) 3192
## cor(Intercept,type.numeric:context.numeric) 3092
## cor(type.numeric,type.numeric:context.numeric) 2696
## cor(context.numeric,type.numeric:context.numeric) 2938
##
## ~participant (Number of levels: 88)
## Estimate Est.Error l-95% CI
## sd(Intercept) 205.65 22.90 161.23
## sd(type.numeric) 99.46 46.66 7.07
## sd(context.numeric) 55.12 34.43 2.63
## sd(multiTRUE) 284.15 32.61 223.18
## sd(type.numeric:context.numeric) 91.08 58.64 4.42
## cor(Intercept,type.numeric) -0.23 0.26 -0.70
## cor(Intercept,context.numeric) 0.12 0.34 -0.57
## cor(type.numeric,context.numeric) 0.09 0.39 -0.68
## cor(Intercept,multiTRUE) 0.71 0.13 0.42
## cor(type.numeric,multiTRUE) -0.03 0.27 -0.63
## cor(context.numeric,multiTRUE) 0.23 0.33 -0.51
## cor(Intercept,type.numeric:context.numeric) -0.21 0.34 -0.80
## cor(type.numeric,type.numeric:context.numeric) -0.03 0.39 -0.74
## cor(context.numeric,type.numeric:context.numeric) -0.15 0.39 -0.81
## cor(multiTRUE,type.numeric:context.numeric) -0.31 0.33 -0.85
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 252.39 1.00 2100
## sd(type.numeric) 184.53 1.01 729
## sd(context.numeric) 127.10 1.00 1015
## sd(multiTRUE) 352.92 1.00 1659
## sd(type.numeric:context.numeric) 214.36 1.00 1193
## cor(Intercept,type.numeric) 0.35 1.00 2589
## cor(Intercept,context.numeric) 0.74 1.00 4890
## cor(type.numeric,context.numeric) 0.77 1.00 2474
## cor(Intercept,multiTRUE) 0.92 1.00 848
## cor(type.numeric,multiTRUE) 0.47 1.00 749
## cor(context.numeric,multiTRUE) 0.78 1.00 736
## cor(Intercept,type.numeric:context.numeric) 0.54 1.01 3383
## cor(type.numeric,type.numeric:context.numeric) 0.71 1.00 3216
## cor(context.numeric,type.numeric:context.numeric) 0.64 1.00 2090
## cor(multiTRUE,type.numeric:context.numeric) 0.45 1.00 3779
## Tail_ESS
## sd(Intercept) 2753
## sd(type.numeric) 779
## sd(context.numeric) 1357
## sd(multiTRUE) 2563
## sd(type.numeric:context.numeric) 1879
## cor(Intercept,type.numeric) 1934
## cor(Intercept,context.numeric) 2688
## cor(type.numeric,context.numeric) 2880
## cor(Intercept,multiTRUE) 1275
## cor(type.numeric,multiTRUE) 824
## cor(context.numeric,multiTRUE) 1586
## cor(Intercept,type.numeric:context.numeric) 2360
## cor(type.numeric,type.numeric:context.numeric) 3076
## cor(context.numeric,type.numeric:context.numeric) 2861
## cor(multiTRUE,type.numeric:context.numeric) 3233
##
## Regression Coefficients:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## Intercept 1036.34 39.33 963.90 1118.35 1.00 1172
## type.numeric -90.82 38.10 -166.00 -15.71 1.00 2872
## context.numeric 33.26 26.24 -18.04 84.61 1.00 3826
## multiTRUE 722.07 55.93 607.27 824.82 1.00 1275
## type.numeric:context.numeric 109.77 44.00 23.92 196.73 1.00 5315
## Tail_ESS
## Intercept 1771
## type.numeric 3081
## context.numeric 2752
## multiTRUE 2112
## type.numeric:context.numeric 2834
##
## Further Distributional Parameters:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma 411.95 9.22 394.32 429.95 1.00 2279 2559
##
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
Residualize
Using length + freq as predictors per-participant in an lm, then
using those (fixef estimates) to calculate a “predicted” value for
critical words and subtracting that off.

Residualize, then sum
## Family: gaussian
## Links: mu = identity; sigma = identity
## Formula: resid ~ type.numeric * context.numeric + multi + (type.numeric * context.numeric | item) + (1 + type.numeric * context.numeric | participant)
## Data: grouped_residuals (Number of observations: 1355)
## Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
## total post-warmup draws = 4000
##
## Multilevel Hyperparameters:
## ~item (Number of levels: 32)
## Estimate Est.Error l-95% CI
## sd(Intercept) 60.35 19.32 19.95
## sd(type.numeric) 185.79 32.24 127.34
## sd(context.numeric) 46.88 32.12 1.99
## sd(type.numeric:context.numeric) 68.50 48.78 2.37
## cor(Intercept,type.numeric) -0.28 0.27 -0.76
## cor(Intercept,context.numeric) 0.04 0.41 -0.76
## cor(type.numeric,context.numeric) -0.08 0.40 -0.80
## cor(Intercept,type.numeric:context.numeric) 0.18 0.42 -0.70
## cor(type.numeric,type.numeric:context.numeric) -0.14 0.41 -0.83
## cor(context.numeric,type.numeric:context.numeric) 0.09 0.44 -0.77
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 99.51 1.00 1090
## sd(type.numeric) 254.28 1.00 2112
## sd(context.numeric) 117.83 1.00 1210
## sd(type.numeric:context.numeric) 178.21 1.00 1915
## cor(Intercept,type.numeric) 0.29 1.00 920
## cor(Intercept,context.numeric) 0.80 1.00 3856
## cor(type.numeric,context.numeric) 0.72 1.00 4078
## cor(Intercept,type.numeric:context.numeric) 0.84 1.00 4316
## cor(type.numeric,type.numeric:context.numeric) 0.71 1.00 4531
## cor(context.numeric,type.numeric:context.numeric) 0.85 1.00 3147
## Tail_ESS
## sd(Intercept) 764
## sd(type.numeric) 2715
## sd(context.numeric) 1641
## sd(type.numeric:context.numeric) 1787
## cor(Intercept,type.numeric) 1514
## cor(Intercept,context.numeric) 2657
## cor(type.numeric,context.numeric) 2412
## cor(Intercept,type.numeric:context.numeric) 2865
## cor(type.numeric,type.numeric:context.numeric) 2726
## cor(context.numeric,type.numeric:context.numeric) 3177
##
## ~participant (Number of levels: 88)
## Estimate Est.Error l-95% CI
## sd(Intercept) 92.52 18.06 56.15
## sd(type.numeric) 96.30 43.08 9.80
## sd(context.numeric) 50.77 34.64 2.47
## sd(type.numeric:context.numeric) 90.53 61.34 4.28
## cor(Intercept,type.numeric) -0.32 0.31 -0.83
## cor(Intercept,context.numeric) -0.02 0.42 -0.79
## cor(type.numeric,context.numeric) 0.08 0.42 -0.74
## cor(Intercept,type.numeric:context.numeric) -0.15 0.40 -0.82
## cor(type.numeric,type.numeric:context.numeric) 0.00 0.43 -0.79
## cor(context.numeric,type.numeric:context.numeric) -0.10 0.43 -0.84
## u-95% CI Rhat Bulk_ESS
## sd(Intercept) 127.61 1.00 1396
## sd(type.numeric) 174.74 1.00 837
## sd(context.numeric) 126.55 1.00 1265
## sd(type.numeric:context.numeric) 224.81 1.01 1066
## cor(Intercept,type.numeric) 0.41 1.00 2134
## cor(Intercept,context.numeric) 0.77 1.00 3591
## cor(type.numeric,context.numeric) 0.81 1.00 3159
## cor(Intercept,type.numeric:context.numeric) 0.71 1.00 3948
## cor(type.numeric,type.numeric:context.numeric) 0.78 1.00 3076
## cor(context.numeric,type.numeric:context.numeric) 0.75 1.00 2336
## Tail_ESS
## sd(Intercept) 1532
## sd(type.numeric) 797
## sd(context.numeric) 2016
## sd(type.numeric:context.numeric) 2008
## cor(Intercept,type.numeric) 1623
## cor(Intercept,context.numeric) 2214
## cor(type.numeric,context.numeric) 2877
## cor(Intercept,type.numeric:context.numeric) 2794
## cor(type.numeric,type.numeric:context.numeric) 2979
## cor(context.numeric,type.numeric:context.numeric) 3214
##
## Regression Coefficients:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## Intercept -32.24 23.27 -78.20 13.86 1.00 2726
## type.numeric -104.11 38.44 -177.83 -24.67 1.00 2708
## context.numeric 28.10 25.52 -22.68 77.96 1.00 5782
## multiTRUE -72.01 30.34 -131.54 -11.51 1.00 2967
## type.numeric:context.numeric 100.91 44.48 14.12 188.15 1.00 7052
## Tail_ESS
## Intercept 3006
## type.numeric 2834
## context.numeric 3176
## multiTRUE 2814
## type.numeric:context.numeric 3161
##
## Further Distributional Parameters:
## Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma 419.94 9.01 402.51 437.74 1.00 2818 3109
##
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).