TME analyses

#Read in responses and filter based on accuracy

## Warning: Column `category` joining character vector and factor, coercing
## into character vector

## Warning: Column `subjCode` joining character vector and factor, coercing
## into character vector

#Read in existing (already processed) TME studies

Subject-level graph of overall accuracy and naming frequency

## Warning: Removed 1 rows containing non-finite values (stat_smooth).

## Warning: Removed 1 rows containing missing values (geom_point).

What factors predict accuracy? (typicality, nameability, naming frequency during study)

accuracy_randomSlope <- glmer(isRight ~ kinds_c + avgFam_c + typ_c*simpson_c + simpson_c*NamingFreq_Study_c + version+
                                        (1|category) + (typ_c*simpson_c|subjCode),
                                      data=tme_all, family=binomial)

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
## control$checkConv, : Model failed to converge with max|grad| = 0.00458298
## (tol = 0.001, component 1)

accuracy_randomInt <- glmer(isRight ~ kinds_c+ avgFam_c + typ_c*simpson_c*version_c + simpson_c*NamingFreq_Study_c +
                                        (1|category) + (1|subjCode), data=tme_all, family=binomial)

accuracy_randomInt_simpler <- glmer(isRight ~ kinds_c+ avgFam_c + typ_c*simpson_c*version_c + NamingFreq_Study_c +
                                        (1|category) + (1|subjCode), data=tme_all, family=binomial)

anova(accuracy_randomSlope, accuracy_randomInt)

## Data: tme_all
## Models:
## accuracy_randomInt: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c * version_c + 
## accuracy_randomInt:     simpson_c * NamingFreq_Study_c + (1 | category) + (1 | subjCode)
## accuracy_randomSlope: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c + simpson_c * 
## accuracy_randomSlope:     NamingFreq_Study_c + version + (1 | category) + (typ_c * 
## accuracy_randomSlope:     simpson_c | subjCode)
##                      Df    AIC    BIC  logLik deviance Chisq Chi Df
## accuracy_randomInt   14 7522.6 7616.9 -3747.3   7494.6             
## accuracy_randomSlope 20 7529.6 7664.3 -3744.8   7489.6 4.962      6
##                      Pr(>Chisq)
## accuracy_randomInt             
## accuracy_randomSlope     0.5487

anova(accuracy_randomInt, accuracy_randomInt_simpler) #fit isn't significantly different

## Data: tme_all
## Models:
## accuracy_randomInt_simpler: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c * version_c + 
## accuracy_randomInt_simpler:     NamingFreq_Study_c + (1 | category) + (1 | subjCode)
## accuracy_randomInt: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c * version_c + 
## accuracy_randomInt:     simpson_c * NamingFreq_Study_c + (1 | category) + (1 | subjCode)
##                            Df    AIC    BIC  logLik deviance  Chisq Chi Df
## accuracy_randomInt_simpler 13 7522.6 7610.2 -3748.3   7496.6              
## accuracy_randomInt         14 7522.6 7616.9 -3747.3   7494.6 2.0455      1
##                            Pr(>Chisq)
## accuracy_randomInt_simpler           
## accuracy_randomInt             0.1527

summary(accuracy_randomInt_simpler)

## Generalized linear mixed model fit by maximum likelihood (Laplace
##   Approximation) [glmerMod]
##  Family: binomial  ( logit )
## Formula: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c * version_c +  
##     NamingFreq_Study_c + (1 | category) + (1 | subjCode)
##    Data: tme_all
## 
##      AIC      BIC   logLik deviance df.resid 
##   7522.6   7610.2  -3748.3   7496.6     6197 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.9827 -0.9099  0.4431  0.7220  2.0744 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  subjCode (Intercept) 0.5803   0.7618  
##  category (Intercept) 0.2353   0.4851  
## Number of obs: 6210, groups:  subjCode, 138; category, 45
## 
## Fixed effects:
##                            Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                0.814666   0.118751   6.860 6.87e-12 ***
## kinds_c                    0.300176   0.122459   2.451  0.01424 *  
## avgFam_c                  -0.300293   0.372345  -0.806  0.41996    
## typ_c                     -0.026025   0.015645  -1.663  0.09622 .  
## simpson_c                 -0.387080   0.273602  -1.415  0.15714    
## version_c                 -0.459866   0.143976  -3.194  0.00140 ** 
## NamingFreq_Study_c         0.179738   0.058078   3.095  0.00197 ** 
## typ_c:simpson_c            0.018005   0.048507   0.371  0.71050    
## typ_c:version_c            0.007368   0.023435   0.314  0.75320    
## simpson_c:version_c        0.195316   0.177577   1.100  0.27138    
## typ_c:simpson_c:version_c  0.056696   0.072496   0.782  0.43418    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) knds_c avgFm_ typ_c  smpsn_ vrsn_c NmF_S_ typ_c:s_
## kinds_c      0.006                                                   
## avgFam_c    -0.001 -0.363                                            
## typ_c       -0.007 -0.001 -0.005                                     
## simpson_c    0.004  0.182 -0.380  0.005                              
## version_c   -0.518 -0.003  0.001  0.006  0.003                       
## NmngFrq_St_ -0.011  0.003 -0.001 -0.001 -0.001  0.035                
## typ_c:smps_  0.002  0.004  0.000 -0.019 -0.009 -0.002 -0.001         
## typ_c:vrsn_  0.005  0.000  0.004 -0.668 -0.003 -0.005 -0.001  0.013  
## smpsn_c:vr_  0.006  0.001 -0.001 -0.004 -0.291 -0.003 -0.001  0.013  
## typ_c:sm_:_ -0.001 -0.003 -0.001  0.013  0.006  0.003  0.003 -0.669  
##             typ_c:v_ smp_:_
## kinds_c                    
## avgFam_c                   
## typ_c                      
## simpson_c                  
## version_c                  
## NmngFrq_St_                
## typ_c:smps_                
## typ_c:vrsn_                
## smpsn_c:vr_  0.008         
## typ_c:sm_:_ -0.003   -0.014

summary(accuracy_randomInt)

## Generalized linear mixed model fit by maximum likelihood (Laplace
##   Approximation) [glmerMod]
##  Family: binomial  ( logit )
## Formula: isRight ~ kinds_c + avgFam_c + typ_c * simpson_c * version_c +  
##     simpson_c * NamingFreq_Study_c + (1 | category) + (1 | subjCode)
##    Data: tme_all
## 
##      AIC      BIC   logLik deviance df.resid 
##   7522.6   7616.9  -3747.3   7494.6     6196 
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.9792 -0.9041  0.4424  0.7207  2.1227 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  subjCode (Intercept) 0.5809   0.7621  
##  category (Intercept) 0.2351   0.4848  
## Number of obs: 6210, groups:  subjCode, 138; category, 45
## 
## Fixed effects:
##                               Estimate Std. Error z value Pr(>|z|)    
## (Intercept)                   0.815316   0.118775   6.864 6.68e-12 ***
## kinds_c                       0.300299   0.122425   2.453  0.01417 *  
## avgFam_c                     -0.301052   0.372279  -0.809  0.41870    
## typ_c                        -0.025955   0.015648  -1.659  0.09718 .  
## simpson_c                    -0.389480   0.273552  -1.424  0.15451    
## version_c                    -0.459791   0.144052  -3.192  0.00141 ** 
## NamingFreq_Study_c            0.180159   0.058107   3.100  0.00193 ** 
## typ_c:simpson_c               0.018106   0.048518   0.373  0.70902    
## typ_c:version_c               0.007138   0.023439   0.305  0.76073    
## simpson_c:version_c           0.186028   0.177767   1.046  0.29534    
## simpson_c:NamingFreq_Study_c -0.103235   0.071911  -1.436  0.15112    
## typ_c:simpson_c:version_c     0.056989   0.072517   0.786  0.43194    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) knds_c avgFm_ typ_c  smpsn_ vrsn_c NmF_S_ typ_c:s_
## kinds_c      0.006                                                   
## avgFam_c    -0.001 -0.363                                            
## typ_c       -0.007 -0.001 -0.005                                     
## simpson_c    0.003  0.182 -0.380  0.004                              
## version_c   -0.518 -0.003  0.001  0.005  0.003                       
## NmngFrq_St_ -0.011  0.003 -0.001 -0.001 -0.002  0.035                
## typ_c:smps_  0.002  0.004  0.000 -0.021 -0.009 -0.002 -0.001         
## typ_c:vrsn_  0.005  0.000  0.004 -0.668 -0.003 -0.005 -0.001  0.014  
## smpsn_c:vr_  0.007  0.001 -0.001 -0.004 -0.290 -0.004  0.001  0.013  
## smps_:NF_S_ -0.006 -0.001  0.002 -0.003  0.006  0.001 -0.006 -0.002  
## typ_c:sm_:_ -0.001 -0.003 -0.001  0.014  0.006  0.003  0.003 -0.669  
##             typ_c:v_ smp_:_ s_:NF_
## kinds_c                           
## avgFam_c                          
## typ_c                             
## simpson_c                         
## version_c                         
## NmngFrq_St_                       
## typ_c:smps_                       
## typ_c:vrsn_                       
## smpsn_c:vr_  0.008                
## smps_:NF_S_  0.007    0.036       
## typ_c:sm_:_ -0.006   -0.014 -0.003

Findings from simplified random-intercepts model:
Significant effect of kinds: accuracy increases as category members are more conceptually distinct
Marginal effect of typicality: accuracy decreases as studied exemplar was less typical (=higher typ value)
Accuracy was lower overall in version 2 (shorter stimulus duration in study phase)
Effect of naming frequency during study: those who named more were more accurate
No interaction between typicality, nameability, or version

Simpson vs. naming frequency (raw)

Simpson vs. naming frequency (median split)

Graph subject-level interaction between typ and overall naming frequency

Each person contributes 3 data points: their mean accuracy for typ = 2, 5, and 8

## Warning: Removed 3 rows containing non-finite values (stat_smooth).

## Warning: Removed 3 rows containing missing values (geom_point).

How do naming frequency during study, nameability, and typicality affect typicality shift?

Absolute value (random slope doesn’t converge)

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
## control$checkConv, : Model failed to converge with max|grad| = 0.0460782
## (tol = 0.002, component 1)

## refitting model(s) with ML (instead of REML)

## Data: filter(tme_all, isRight == 0)
## Models:
## abs_typ_shift_randomInt_simpler: abs_typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c * 
## abs_typ_shift_randomInt_simpler:     version_c + NamingFreq_Study_c + (1 | subjCode) + (1 | category)
## abs_typ_shift_randomInt: abs_typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c * 
## abs_typ_shift_randomInt:     version_c + simpson_c * NamingFreq_Study_c + (1 | subjCode) + 
## abs_typ_shift_randomInt:     (1 | category)
## abs_typ_shift_randomSlope: abs_typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c * 
## abs_typ_shift_randomSlope:     version_c + simpson_c * NamingFreq_Study_c + (1 | category) + 
## abs_typ_shift_randomSlope:     (simpson_c * typ_c | subjCode)
##                                 Df    AIC    BIC  logLik deviance   Chisq
## abs_typ_shift_randomInt_simpler 14 9489.1 9569.6 -4730.5   9461.1        
## abs_typ_shift_randomInt         15 9491.0 9577.3 -4730.5   9461.0  0.0459
## abs_typ_shift_randomSlope       24 9486.4 9624.5 -4719.2   9438.4 22.6579
##                                 Chi Df Pr(>Chisq)   
## abs_typ_shift_randomInt_simpler                     
## abs_typ_shift_randomInt              1   0.830400   
## abs_typ_shift_randomSlope            9   0.007012 **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: abs_typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c *  
##     version_c + NamingFreq_Study_c + (1 | subjCode) + (1 | category)
##    Data: filter(tme_all, isRight == 0)
## 
## REML criterion at convergence: 9499.5
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -1.7548 -0.7875 -0.1029  0.5814  2.8062 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  subjCode (Intercept) 0.02556  0.1599  
##  category (Intercept) 0.04396  0.2097  
##  Residual             3.35550  1.8318  
## Number of obs: 2330, groups:  subjCode, 138; category, 45
## 
## Fixed effects:
##                             Estimate Std. Error         df t value
## (Intercept)                3.128e+00  6.520e-02  7.820e+01  47.981
## kinds_c                   -3.019e-02  7.892e-02  4.453e+01  -0.383
## avgFam_c                  -2.398e-01  2.361e-01  4.088e+01  -1.016
## simpson_c                 -4.956e-02  2.040e-01  9.290e+01  -0.243
## typ_c                      2.643e-01  2.167e-02  2.281e+03  12.197
## version_c                 -3.454e-02  8.219e-02  1.114e+02  -0.420
## NamingFreq_Study_c        -2.928e-03  3.286e-02  1.070e+02  -0.089
## simpson_c:typ_c            1.235e-01  6.813e-02  2.283e+03   1.813
## simpson_c:version_c        3.370e-01  2.392e-01  2.285e+03   1.409
## typ_c:version_c           -9.204e-02  3.118e-02  2.268e+03  -2.952
## simpson_c:typ_c:version_c  8.166e-02  9.738e-02  2.279e+03   0.839
##                           Pr(>|t|)    
## (Intercept)                < 2e-16 ***
## kinds_c                    0.70387    
## avgFam_c                   0.31559    
## simpson_c                  0.80864    
## typ_c                      < 2e-16 ***
## version_c                  0.67506    
## NamingFreq_Study_c         0.92917    
## simpson_c:typ_c            0.07004 .  
## simpson_c:version_c        0.15911    
## typ_c:version_c            0.00319 ** 
## simpson_c:typ_c:version_c  0.40181    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) knds_c avgFm_ smpsn_ typ_c  vrsn_c NmF_S_ smpsn_c:t_
## kinds_c      0.055                                                     
## avgFam_c    -0.013 -0.354                                              
## simpson_c   -0.049  0.140 -0.312                                       
## typ_c       -0.029 -0.006 -0.027  0.014                                
## version_c   -0.598 -0.011  0.000  0.045  0.022                         
## NmngFrq_St_  0.028  0.000  0.006 -0.024  0.004  0.070                  
## smpsn_c:ty_  0.007  0.016  0.025 -0.045 -0.044 -0.006 -0.014           
## smpsn_c:vr_  0.052  0.017  0.007 -0.571 -0.006 -0.058  0.010  0.032    
## typ_c:vrsn_  0.023  0.011  0.024 -0.011 -0.693 -0.028  0.007  0.032    
## smpsn_c:_:_ -0.004 -0.014 -0.013  0.031  0.032  0.020  0.019 -0.698    
##             smpsn_c:v_ typ_:_
## kinds_c                      
## avgFam_c                     
## simpson_c                    
## typ_c                        
## version_c                    
## NmngFrq_St_                  
## smpsn_c:ty_                  
## smpsn_c:vr_                  
## typ_c:vrsn_  0.020           
## smpsn_c:_:_ -0.039     -0.035

Raw value (random slope model has singular fit)

## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl =
## control$checkConv, : Model failed to converge with max|grad| = 0.0255303
## (tol = 0.002, component 1)

## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: 
## typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c * version_c +  
##     NamingFreq_Study_c + (1 | subjCode) + (1 | category)
##    Data: filter(tme_all, isRight == 0)
## 
## REML criterion at convergence: 10903.3
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -2.5346 -0.6965  0.1567  0.8115  1.8714 
## 
## Random effects:
##  Groups   Name        Variance Std.Dev.
##  subjCode (Intercept) 0.08741  0.2956  
##  category (Intercept) 0.41661  0.6455  
##  Residual             6.00807  2.4511  
## Number of obs: 2330, groups:  subjCode, 138; category, 45
## 
## Fixed effects:
##                             Estimate Std. Error         df t value
## (Intercept)                1.042e+00  1.261e-01  6.635e+01   8.263
## kinds_c                   -1.373e-01  1.731e-01  4.184e+01  -0.793
## avgFam_c                  -1.291e+00  5.241e-01  4.086e+01  -2.463
## simpson_c                  1.070e+00  4.015e-01  5.743e+01   2.665
## typ_c                      1.021e+00  2.914e-02  2.265e+03  35.035
## version_c                 -2.639e-01  1.165e-01  1.286e+02  -2.266
## NamingFreq_Study_c        -9.660e-04  4.661e-02  1.242e+02  -0.021
## simpson_c:typ_c           -1.251e-01  9.164e-02  2.266e+03  -1.365
## simpson_c:version_c       -5.026e-02  3.213e-01  2.269e+03  -0.156
## typ_c:version_c           -3.310e-02  4.186e-02  2.254e+03  -0.791
## simpson_c:typ_c:version_c  1.995e-01  1.308e-01  2.264e+03   1.525
##                           Pr(>|t|)    
## (Intercept)               8.62e-12 ***
## kinds_c                    0.43214    
## avgFam_c                   0.01806 *  
## simpson_c                  0.00999 ** 
## typ_c                      < 2e-16 ***
## version_c                  0.02513 *  
## NamingFreq_Study_c         0.98350    
## simpson_c:typ_c            0.17248    
## simpson_c:version_c        0.87572    
## typ_c:version_c            0.42923    
## simpson_c:typ_c:version_c  0.12733    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) knds_c avgFm_ smpsn_ typ_c  vrsn_c NmF_S_ smpsn_c:t_
## kinds_c      0.025                                                     
## avgFam_c    -0.004 -0.360                                              
## simpson_c   -0.017  0.170 -0.359                                       
## typ_c       -0.022 -0.005 -0.017  0.010                                
## version_c   -0.437 -0.006  0.000  0.032  0.021                         
## NmngFrq_St_  0.019  0.000  0.003 -0.016  0.003  0.066                  
## smpsn_c:ty_  0.004  0.009  0.015 -0.034 -0.043 -0.006 -0.013           
## smpsn_c:vr_  0.039  0.011  0.004 -0.392 -0.006 -0.058  0.010  0.032    
## typ_c:vrsn_  0.018  0.007  0.015 -0.007 -0.692 -0.027  0.007  0.031    
## smpsn_c:_:_ -0.002 -0.007 -0.008  0.024  0.031  0.019  0.017 -0.696    
##             smpsn_c:v_ typ_:_
## kinds_c                      
## avgFam_c                     
## simpson_c                    
## typ_c                        
## version_c                    
## NmngFrq_St_                  
## smpsn_c:ty_                  
## smpsn_c:vr_                  
## typ_c:vrsn_  0.020           
## smpsn_c:_:_ -0.040     -0.035

Compare absolute value random-intercept models

## refitting model(s) with ML (instead of REML)

## Data: filter(tme_all, isRight == 0 & !is.na(NamingFreq_Study_c))
## Models:
## abs_model1: abs_typ_adv_choice ~ kinds_c + avgFam_c + (1 | subjCode) + (1 | 
## abs_model1:     category)
## abs_model2: abs_typ_adv_choice ~ kinds_c + avgFam_c + simpson_c * typ_c * 
## abs_model2:     version_c + NamingFreq_Study_c + (1 | subjCode) + (1 | category)
##            Df    AIC    BIC  logLik deviance Chisq Chi Df Pr(>Chisq)    
## abs_model1  6 9688.0 9722.5 -4838.0   9676.0                            
## abs_model2 14 9489.1 9569.6 -4730.5   9461.1 214.9      8  < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Typicality shift model summaries

Absolute value:
Significant effect of typ: typicality shift is greater when studied object was less typical (driven by typ=8)
Marginal interaction between typicality and nameability: slope for typ==2 is slightly different from typ==5 and typ==8? (see plots below) Interaction between typicality and version: when studied typ==8, shift is smaller for version 2 than version 1
Model including typ, simpson, naming better fit than base model

Raw value:
Significant effect of familiarity: for more familiar object categories, when people are wrong, they choose a less typical item than the one they studied
Significant effect of typ: typicality shift is greater when studied object was less typical
Significant effect of nameability: incorrect responses for more-nameable objects tend to be more typical than studied object
Significant effect of version: typicality shift is smaller for version 2
No interactions between typ, simpson, or self-reported naming

Scatterplots for typicality shift depending on studied typ and version

Absolute value - multiple responses per participant

Absolute value - 1 response per participant for each value of typ

Raw value - multiple responses per participant

Raw value - 1 response per participant for each value of typ

Scatterplots for typicality shift depending on studied typ and nameability

Absolute value - multiple responses per participant

Absolute value - 1 response per participant for each value of typ

Raw value - multiple responses per participant

Raw value - 1 response per participant for each value of typ

Density plots for typicality shift depending on studied typ and version

Absolute value

Raw value

Histograms for typicality shift depending on studied typ

Absolute value

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

## Warning: Removed 12 rows containing missing values (geom_bar).

Raw value

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

## Warning: Removed 12 rows containing missing values (geom_bar).