CEL_combined_alignment
Ellise Suffill & Gary Lupyan
9/7/2020
CATEGORY ALIGNMENT (WITHIN CONDITION COMPARISONS)
In sorting, how similar are people’s sorts within conditions, and how does this vary across conditions?
in-lab
With labels people have significantly more similar sorts than do Baseline or No labels people. Baseline and No labels people do not significantly differ in alignment.
in-lab plot
in-lab models
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_inlab
##
## REML criterion at convergence: -231.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.5503 -0.5982 0.0240 0.5932 3.4683
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.11184 0.3344
## subj_code_2 (Intercept) 0.10207 0.3195
## Residual 0.03746 0.1935
## Number of obs: 2713, groups: subj_code_1, 126; subj_code_2, 126
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.30669 0.07088 195.57535 -32.543 < 2e-16 ***
## condNL 0.11997 0.10132 194.89618 1.184 0.238
## condWL 0.41741 0.10346 200.33342 4.035 7.78e-05 ***
## scale(base) 0.17929 0.01410 2477.13270 12.714 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condNL condWL
## condNL -0.698
## condWL -0.690 0.480
## scale(base) 0.053 -0.014 -0.126## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_NLWL
##
## REML criterion at convergence: -309.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.1900 -0.5995 0.0267 0.5774 3.6821
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.12262 0.3502
## subj_code_2 (Intercept) 0.11302 0.3362
## Residual 0.03317 0.1821
## Number of obs: 1723, groups: subj_code_1, 82; subj_code_2, 82
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.15807 0.07606 131.61459 -28.373 <2e-16 ***
## condWL 0.30606 0.10938 133.62399 2.798 0.0059 **
## scale(base) 0.18561 0.01653 1664.05209 11.227 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condWL
## condWL -0.700
## scale(base) 0.072 -0.116online
No labels and With labels people do not significantly differ in alignment
online plot
online models
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online
##
## REML criterion at convergence: -1229.3
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -6.5156 -0.6459 -0.0255 0.6030 3.4967
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.06369 0.2524
## subj_code_2 (Intercept) 0.06247 0.2499
## Residual 0.01637 0.1279
## Number of obs: 1446, groups: subj_code_1, 75; subj_code_2, 75
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -1.475711 0.059991 106.059474 -24.60 <2e-16 ***
## condWL -0.002472 0.083159 105.900246 -0.03 0.976
## scale(base) 0.194524 0.016436 809.796893 11.84 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condWL
## condWL -0.721
## scale(base) 0.012 -0.007online 2
With labels = greater alignment than NL (p=.016)
add in baseline
online 2 plot
online 2 models
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online2
## Control: lmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05))
##
## REML criterion at convergence: -400.9
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -4.0511 -0.6421 -0.0024 0.6329 3.9326
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.05731 0.2394
## subj_code_2 (Intercept) 0.05351 0.2313
## Residual 0.03546 0.1883
## Number of obs: 2305, groups: subj_code_1, 116; subj_code_2, 116
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.00524 0.05636 175.74949 -35.580 <2e-16 ***
## condNL -0.05396 0.07765 175.06444 -0.695 0.488
## condWL 0.06870 0.07969 178.76205 0.862 0.390
## scale(base) 0.33204 0.01440 1172.83440 23.065 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condNL condWL
## condNL -0.723
## condWL -0.711 0.508
## scale(base) 0.040 0.034 -0.123## refitting model(s) with ML (instead of REML)## Data: data_full_online2
## Models:
## m0: I(log(is_match + 0.001)) ~ 1 + scale(base) + (1 | subj_code_1) +
## m0: (1 | subj_code_2)
## m1: I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## m1: (1 | subj_code_2)
## Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
## m0 5 -407.02 -378.31 208.51 -417.02
## m1 7 -405.49 -365.29 209.75 -419.49 2.4735 2 0.2903Look at specific comparisons
BL v NL - not sig diff
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online2_BLNL
##
## REML criterion at convergence: -97.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.8572 -0.6585 0.0009 0.6253 3.7327
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.06687 0.2586
## subj_code_2 (Intercept) 0.05254 0.2292
## Residual 0.03986 0.1997
## Number of obs: 1564, groups: subj_code_1, 78; subj_code_2, 78
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.09099 0.05854 124.79823 -35.719 <2e-16 ***
## condNL -0.05154 0.08070 124.58436 -0.639 0.524
## scale(base) 0.30794 0.01616 920.70288 19.060 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condNL
## condNL -0.726
## scale(base) -0.030 0.042BL v WL - not sig diff
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online2_BLWL
##
## REML criterion at convergence: -314.8
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -4.2165 -0.6460 -0.0361 0.6184 4.0620
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.05044 0.2246
## subj_code_2 (Intercept) 0.04683 0.2164
## Residual 0.03391 0.1841
## Number of obs: 1444, groups: subj_code_1, 75; subj_code_2, 75
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -1.92957 0.05325 125.76657 -36.24 <2e-16 ***
## condWL 0.04294 0.07528 127.49875 0.57 0.569
## scale(base) 0.37224 0.01841 586.37716 20.23 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condWL
## condWL -0.717
## scale(base) 0.119 -0.165NL v WL - WL sig higher alignment than NL
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online2_NLWL
##
## REML criterion at convergence: -413.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.4643 -0.6360 0.0241 0.6169 3.8334
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.06042 0.2458
## subj_code_2 (Intercept) 0.06431 0.2536
## Residual 0.03206 0.1791
## Number of obs: 1602, groups: subj_code_1, 79; subj_code_2, 79
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.05552 0.05680 105.77675 -36.186 <2e-16 ***
## condWL 0.16981 0.08257 108.20192 2.057 0.0421 *
## scale(base) 0.29075 0.01722 894.16901 16.887 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condWL
## condWL -0.699
## scale(base) 0.121 -0.170ALIGNMENT CONTROLLING FOR SCREEN SIZE IN ONLINE 2
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + scale(subj_1_area):scale(subj_2_area) +
## (1 | subj_code_1) + (1 | subj_code_2)
## Data: data_full_online2_2
##
## REML criterion at convergence: -63.2
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.2423 -0.6392 -0.0123 0.6103 3.6776
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.17089 0.4134
## subj_code_2 (Intercept) 0.15861 0.3983
## Residual 0.03711 0.1926
## Number of obs: 2268, groups: subj_code_1, 115; subj_code_2, 115
##
## Fixed effects:
## Estimate Std. Error df t value
## (Intercept) -1.986e+00 5.414e-02 2.259e+02 -36.686
## scale(subj_1_area):scale(subj_2_area) -3.042e-03 5.612e-03 2.077e+03 -0.542
## Pr(>|t|)
## (Intercept) <2e-16 ***
## scale(subj_1_area):scale(subj_2_area) 0.588
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr)
## s(_1_):(_2_ -0.027## boundary (singular) fit: see ?isSingular
## boundary (singular) fit: see ?isSingular## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: align_by_screen ~ 1 + cond + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_full_online2_2
##
## REML criterion at convergence: -1253.7
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -3.3162 -0.6867 -0.0209 0.6372 4.0253
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.00000 0.0000
## subj_code_2 (Intercept) 0.00000 0.0000
## Residual 0.03325 0.1823
## Number of obs: 2268, groups: subj_code_1, 115; subj_code_2, 115
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) 2.487e-03 7.092e-03 2.264e+03 0.351 0.726
## condNL 2.543e-03 9.440e-03 2.264e+03 0.269 0.788
## condWL -1.057e-02 1.012e-02 2.264e+03 -1.044 0.296
## scale(base) 1.800e-02 4.122e-03 2.264e+03 4.366 1.32e-05 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) condNL condWL
## condNL -0.739
## condWL -0.719 0.501
## scale(base) 0.087 0.080 -0.272
## convergence code: 0
## boundary (singular) fit: see ?isSingularINLAB VS. ONLINE ALIGNMENT
Suggests people in online task are overall more aligned than inlab? Might be being driven by high BL alignment for online 2 ppts. Again, overall, WL people more aligned than BL; NL and BL don’t sig differ.
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: I(log(is_match + 0.001)) ~ 1 + exp * cond + scale(base) + (1 |
## subj_code_1) + (1 | subj_code_2)
## Data: data_both
## Control: lmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05))
##
## REML criterion at convergence: -587.4
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.7410 -0.6271 0.0173 0.6108 3.7771
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.08462 0.2909
## subj_code_2 (Intercept) 0.07700 0.2775
## Residual 0.03682 0.1919
## Number of obs: 5018, groups: subj_code_1, 242; subj_code_2, 242
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.25889 0.06189 359.38000 -36.498 < 2e-16 ***
## exponline 0.20524 0.09159 359.69601 2.241 0.025643 *
## condNL 0.11467 0.08842 357.63588 1.297 0.195514
## condWL 0.35382 0.09007 364.10030 3.928 0.000102 ***
## scale(base) 0.24930 0.01031 3745.52715 24.181 < 2e-16 ***
## exponline:condNL -0.18207 0.12840 358.78260 -1.418 0.157058
## exponline:condWL -0.22620 0.13047 359.21757 -1.734 0.083838 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) expnln condNL condWL scl(b) exp:NL
## exponline -0.676
## condNL -0.698 0.472
## condWL -0.691 0.467 0.480
## scale(base) 0.064 -0.041 -0.011 -0.105
## expnln:cnNL 0.482 -0.713 -0.689 -0.332 0.023
## expnln:cnWL 0.474 -0.702 -0.331 -0.685 0.019 0.500combined NL v WL only
Again, WL has higher alignment than NL Effect of xperiment disappears so again seems to back up idea that higher online alignment in full model is from online:BL people.
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: I(log(is_match + 0.001)) ~ 1 + exp * cond + scale(base) + (1 |
## subj_code_1) + (1 | subj_code_2)
## Data: data_bothNLWL
## Control: lmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05))
##
## REML criterion at convergence: -708.5
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.3718 -0.6139 0.0300 0.5939 3.7924
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.09453 0.3075
## subj_code_2 (Intercept) 0.09017 0.3003
## Residual 0.03262 0.1806
## Number of obs: 3325, groups: subj_code_1, 161; subj_code_2, 161
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.12695 0.06753 240.45502 -31.495 < 2e-16 ***
## exponline 0.02918 0.09586 238.68985 0.304 0.76110
## condWL 0.27064 0.09692 242.32905 2.792 0.00565 **
## scale(base) 0.22695 0.01209 2852.15987 18.772 < 2e-16 ***
## exponline:condWL -0.04549 0.13779 238.98915 -0.330 0.74159
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) expnln condWL scl(b)
## exponline -0.702
## condWL -0.700 0.489
## scale(base) 0.071 -0.010 -0.096
## expnln:cnWL 0.487 -0.696 -0.697 -0.003look at just baseline model for alignment across in-lab and online 2
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula:
## I(log(is_match + 0.001)) ~ 1 + exp + scale(base) + (1 | subj_code_1) +
## (1 | subj_code_2)
## Data: data_both
## Control: lmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05))
##
## REML criterion at convergence: -579.6
##
## Scaled residuals:
## Min 1Q Median 3Q Max
## -5.7455 -0.6276 0.0183 0.6113 3.7808
##
## Random effects:
## Groups Name Variance Std.Dev.
## subj_code_1 (Intercept) 0.08598 0.2932
## subj_code_2 (Intercept) 0.07855 0.2803
## Residual 0.03688 0.1920
## Number of obs: 5018, groups: subj_code_1, 242; subj_code_2, 242
##
## Fixed effects:
## Estimate Std. Error df t value Pr(>|t|)
## (Intercept) -2.10823 0.03684 352.27206 -57.219 <2e-16 ***
## exponline 0.07176 0.05328 353.58807 1.347 0.179
## scale(base) 0.25475 0.01022 3556.28842 24.921 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Correlation of Fixed Effects:
## (Intr) expnln
## exponline -0.692
## scale(base) 0.016 -0.038