library (tidyverse)library (lavaan)
Warning: package 'lavaan' was built under R version 4.3.3
This is lavaan 0.6-18
lavaan is FREE software! Please report any bugs.
Warning: package 'tidySEM' was built under R version 4.3.3
Loading required package: OpenMx
OpenMx may run faster if it is compiled to take advantage of multiple cores.
Registered S3 method overwritten by 'tidySEM':
method from
predict.MxModel OpenMx
here() starts at /Users/mcfrank/Projects/peekbank/peekbank-development
Warning: package 'psych' was built under R version 4.3.3
Attaching package: 'psych'
The following object is masked from 'package:OpenMx':
tr
The following object is masked from 'package:lavaan':
cor2cov
The following objects are masked from 'package:ggplot2':
%+%, alpha
<- readRDS (here ("cached_intermediates" ,"1_d_sub.Rds" ))<- readRDS (here ("cached_intermediates" ,"1_d_trial.Rds" ))
Our goal here is to understand the dimensionality of the data.
Let’s use the approach advocated by Nilam, namely test for a single dimension of latent variation.
Full dataset
<- d_sub |> ungroup () |> select (dataset_name, rt, rt_var, long_window_accuracy, long_window_acc_var, prod, comp) <- d_sub_mat |> ungroup () |> mutate (across (all_of (c ("rt" , "rt_var" , "long_window_accuracy" , "long_window_acc_var" , "prod" , "comp" )), ~ scale (.x)[,1 ]))
EFA
First we will fit EFA. You can do this over unscaled data, result is the same.
fa.parallel (select (d_sub_mat, - dataset_name), fa = "fa" , use = "pairwise.complete.obs" )
Parallel analysis suggests that the number of factors = 3 and the number of components = NA
There is some variation on the second dimension, though.
fa (select (d_sub_mat, - dataset_name), nfactor = 2 ,use = "pairwise" , rotate = "varimax" )
Factor Analysis using method = minres
Call: fa(r = select(d_sub_mat, -dataset_name), nfactors = 2, rotate = "varimax",
use = "pairwise")
Standardized loadings (pattern matrix) based upon correlation matrix
MR1 MR2 h2 u2 com
rt 0.83 0.01 0.68 0.32 1.0
rt_var 0.82 -0.02 0.67 0.33 1.0
long_window_accuracy -0.53 0.38 0.43 0.57 1.8
long_window_acc_var 0.42 -0.02 0.18 0.82 1.0
prod -0.15 0.84 0.73 0.27 1.1
comp 0.08 0.78 0.61 0.39 1.0
MR1 MR2
SS loadings 1.84 1.45
Proportion Var 0.31 0.24
Cumulative Var 0.31 0.55
Proportion Explained 0.56 0.44
Cumulative Proportion 0.56 1.00
Mean item complexity = 1.1
Test of the hypothesis that 2 factors are sufficient.
df null model = 15 with the objective function = 1.97 with Chi Square = 4070.01
df of the model are 4 and the objective function was 0.15
The root mean square of the residuals (RMSR) is 0.1
The df corrected root mean square of the residuals is 0.2
The harmonic n.obs is 506 with the empirical chi square 155.88 with prob < 1.1e-32
The total n.obs was 2065 with Likelihood Chi Square = 299.49 with prob < 1.4e-63
Tucker Lewis Index of factoring reliability = 0.727
RMSEA index = 0.189 and the 90 % confidence intervals are 0.171 0.208
BIC = 268.96
Fit based upon off diagonal values = 0.96
Measures of factor score adequacy
MR1 MR2
Correlation of (regression) scores with factors 0.91 0.90
Multiple R square of scores with factors 0.83 0.81
Minimum correlation of possible factor scores 0.66 0.62
CFA
We start with a one factor CFA.
<- "F1 =~ rt + rt_var + long_window_accuracy + long_window_acc_var + prod + comp" <- cfa (fa_model, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
summary (fit, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 28 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 18
Used Total
Number of observations 1978 2065
Number of missing patterns 16
Model Test User Model:
Test statistic 396.575
Degrees of freedom 9
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 2158.416
Degrees of freedom 15
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.819
Tucker-Lewis Index (TLI) 0.699
Robust Comparative Fit Index (CFI) 0.516
Robust Tucker-Lewis Index (TLI) 0.194
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -10321.435
Loglikelihood unrestricted model (H1) -10123.148
Akaike (AIC) 20678.871
Bayesian (BIC) 20779.488
Sample-size adjusted Bayesian (SABIC) 20722.301
Root Mean Square Error of Approximation:
RMSEA 0.148
90 Percent confidence interval - lower 0.135
90 Percent confidence interval - upper 0.160
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.364
90 Percent confidence interval - lower 0.336
90 Percent confidence interval - upper 0.393
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.185
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.887 0.022 40.058 0.000 0.887 0.883
rt_var 0.944 0.026 36.812 0.000 0.944 0.874
lng_wndw_ccrcy -0.488 0.025 -19.603 0.000 -0.488 -0.486
lng_wndw_cc_vr 0.380 0.026 14.779 0.000 0.380 0.379
prod -0.177 0.050 -3.511 0.000 -0.177 -0.177
comp 0.023 0.063 0.363 0.716 0.023 0.023
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.004 0.023 0.160 0.873 0.004 0.004
.rt_var 0.019 0.026 0.723 0.470 0.019 0.017
.lng_wndw_ccrcy 0.001 0.023 0.043 0.966 0.001 0.001
.lng_wndw_cc_vr -0.001 0.023 -0.033 0.974 -0.001 -0.001
.prod 0.005 0.044 0.118 0.906 0.005 0.005
.comp -0.005 0.062 -0.089 0.929 -0.005 -0.005
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.223 0.023 9.628 0.000 0.223 0.220
.rt_var 0.275 0.025 11.062 0.000 0.275 0.236
.lng_wndw_ccrcy 0.772 0.027 28.128 0.000 0.772 0.764
.lng_wndw_cc_vr 0.861 0.030 29.054 0.000 0.861 0.856
.prod 0.969 0.061 15.995 0.000 0.969 0.969
.comp 0.996 0.084 11.789 0.000 0.996 0.999
F1 1.000 1.000 1.000
Missing data coverage:
rt rt_var lng_w_ lng___ prod comp
rt 0.943
rt_var 0.789 0.789
long_window_accuracy 0.874 0.765 0.929
long_window_acc_var 0.874 0.765 0.929 0.929
prod 0.245 0.215 0.254 0.254 0.262
comp 0.132 0.109 0.134 0.134 0.141 0.141
Look at observed vs. fitted correlations.
inspect (fit, what= "cor.all" )
rt rt_var lng_w_ lng___ prod comp F1
rt 1.000
rt_var 0.772 1.000
long_window_accuracy -0.429 -0.425 1.000
long_window_acc_var 0.335 0.331 -0.184 1.000
prod -0.156 -0.155 0.086 -0.067 1.000
comp 0.020 0.020 -0.011 0.009 -0.004 1.000
F1 0.883 0.874 -0.486 0.379 -0.177 0.023 1.000
rt rt_var lng_w_ lng___ prod comp
rt 1.000
rt_var 0.781 1.000
long_window_accuracy -0.409 -0.394 1.000
long_window_acc_var 0.304 0.292 -0.405 1.000
prod -0.095 -0.093 0.388 -0.051 1.000
comp 0.012 0.024 0.308 0.020 0.738 1.000
graph_sem (model = fit, text_size = 2 ) + theme (panel.background = element_rect (fill = "white" ))
Extract F1.
$ f1 <- lavPredict (fit, newdata = d_sub_mat_s)ggplot (d_sub, aes (x = log_age, y = f1)) + geom_point (alpha = .1 ) + geom_smooth () + geom_smooth (method = "lm" , col = "green" )
Don't know how to automatically pick scale for object of type
<lavaan.matrix/matrix>. Defaulting to continuous.
`geom_smooth()` using method = 'gam' and formula = 'y ~ s(x, bs = "cs")'
Warning: Removed 87 rows containing non-finite outside the scale range
(`stat_smooth()`).
`geom_smooth()` using formula = 'y ~ x'
Warning: Removed 87 rows containing non-finite outside the scale range
(`stat_smooth()`).
Warning: Removed 87 rows containing missing values or values outside the scale range
(`geom_point()`).
This seems somewhat satisfactory (all loadings are in the same direction). But to be honest, the fit indices are not great. This is probably driven by some of the action in the second dimension in the scree plot. Not sure what is going on there, but part of it has to do with the relative lack of correlation between RT and comprehension. I wonder if that has to do with the relatively small amount of comprehension data we have.
Alternative models
Speed/accuracy separate
Compare to model with separate speed and accuracy. This one is better fitting.
<- "accuracy =~ long_window_accuracy + long_window_acc_var + prod + comp speed =~ rt + rt_var" <- cfa (fa2_model, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
summary (fit2, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 35 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 19
Used Total
Number of observations 1978 2065
Number of missing patterns 16
Model Test User Model:
Test statistic 179.668
Degrees of freedom 8
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 2158.416
Degrees of freedom 15
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.920
Tucker-Lewis Index (TLI) 0.850
Robust Comparative Fit Index (CFI) 0.666
Robust Tucker-Lewis Index (TLI) 0.373
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -10212.982
Loglikelihood unrestricted model (H1) -10123.148
Akaike (AIC) 20463.963
Bayesian (BIC) 20570.170
Sample-size adjusted Bayesian (SABIC) 20509.807
Root Mean Square Error of Approximation:
RMSEA 0.104
90 Percent confidence interval - lower 0.091
90 Percent confidence interval - upper 0.118
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 0.999
Robust RMSEA 0.321
90 Percent confidence interval - lower 0.292
90 Percent confidence interval - upper 0.351
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.148
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
accuracy =~
lng_wndw_ccrcy 0.831 0.038 21.971 0.000 0.831 0.828
lng_wndw_cc_vr -0.492 0.030 -16.206 0.000 -0.492 -0.492
prod 0.439 0.050 8.847 0.000 0.439 0.439
comp 0.302 0.079 3.833 0.000 0.302 0.300
speed =~
rt 0.902 0.025 35.636 0.000 0.902 0.901
rt_var 0.937 0.028 32.892 0.000 0.937 0.866
Covariances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
accuracy ~~
speed -0.552 0.031 -18.062 0.000 -0.552 -0.552
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.lng_wndw_ccrcy -0.002 0.023 -0.069 0.945 -0.002 -0.002
.lng_wndw_cc_vr 0.001 0.023 0.041 0.968 0.001 0.001
.prod 0.026 0.042 0.619 0.536 0.026 0.026
.comp 0.134 0.069 1.958 0.050 0.134 0.134
.rt 0.004 0.023 0.154 0.878 0.004 0.004
.rt_var 0.019 0.026 0.743 0.457 0.019 0.018
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.lng_wndw_ccrcy 0.318 0.055 5.768 0.000 0.318 0.315
.lng_wndw_cc_vr 0.760 0.032 23.646 0.000 0.760 0.758
.prod 0.806 0.055 14.545 0.000 0.806 0.807
.comp 0.919 0.081 11.289 0.000 0.919 0.910
.rt 0.189 0.033 5.758 0.000 0.189 0.188
.rt_var 0.293 0.035 8.301 0.000 0.293 0.250
accuracy 1.000 1.000 1.000
speed 1.000 1.000 1.000
graph_sem (model = fit2, text_size = 2 )
Big improvement.
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit2 8 20464 20570 179.67
fit 9 20679 20780 396.58 216.91 0.33039 1 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Variation separated
Compare to model with vars separate but acc and RT grouped. This is definitely worse.
<- "language =~ rt + long_window_accuracy + prod + comp variability =~ long_window_acc_var + rt_var" <- cfa (fa2v_model, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
Warning: lavaan->lav_object_post_check():
covariance matrix of latent variables is not positive definite ; use
lavInspect(fit, "cov.lv") to investigate.
summary (fit2v, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 40 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 19
Used Total
Number of observations 1978 2065
Number of missing patterns 16
Model Test User Model:
Test statistic 381.246
Degrees of freedom 8
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 2158.416
Degrees of freedom 15
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.826
Tucker-Lewis Index (TLI) 0.673
Robust Comparative Fit Index (CFI) 0.516
Robust Tucker-Lewis Index (TLI) 0.093
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -10313.771
Loglikelihood unrestricted model (H1) -10123.148
Akaike (AIC) 20665.542
Bayesian (BIC) 20771.749
Sample-size adjusted Bayesian (SABIC) 20711.385
Root Mean Square Error of Approximation:
RMSEA 0.154
90 Percent confidence interval - lower 0.141
90 Percent confidence interval - upper 0.167
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.386
90 Percent confidence interval - lower 0.355
90 Percent confidence interval - upper 0.418
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.185
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
language =~
rt 0.845 0.025 34.245 0.000 0.845 0.841
lng_wndw_ccrcy -0.498 0.025 -20.026 0.000 -0.498 -0.496
prod -0.175 0.050 -3.502 0.000 -0.175 -0.175
comp 0.036 0.062 0.573 0.567 0.036 0.036
variability =~
lng_wndw_cc_vr 0.382 0.026 14.708 0.000 0.382 0.380
rt_var 0.860 0.034 25.599 0.000 0.860 0.799
Covariances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
language ~~
variability 1.139 0.038 29.986 0.000 1.139 1.139
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.003 0.023 0.144 0.885 0.003 0.003
.lng_wndw_ccrcy 0.001 0.023 0.044 0.965 0.001 0.001
.prod 0.008 0.044 0.181 0.856 0.008 0.008
.comp -0.010 0.062 -0.157 0.876 -0.010 -0.010
.lng_wndw_cc_vr -0.001 0.023 -0.031 0.975 -0.001 -0.001
.rt_var 0.019 0.026 0.746 0.456 0.019 0.018
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.296 0.028 10.414 0.000 0.296 0.293
.lng_wndw_ccrcy 0.761 0.027 27.965 0.000 0.761 0.754
.prod 0.970 0.061 16.004 0.000 0.970 0.969
.comp 0.995 0.084 11.787 0.000 0.995 0.999
.lng_wndw_cc_vr 0.862 0.030 28.917 0.000 0.862 0.855
.rt_var 0.418 0.043 9.803 0.000 0.418 0.361
language 1.000 1.000 1.000
variability 1.000 1.000 1.000
graph_sem (model = fit2v, text_size = 2 )
Model comparison. There is an improvement.
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit2v 8 20666 20772 381.25
fit 9 20679 20780 396.58 15.329 0.085113 1 9.032e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Which of these is better?
Warning: lavaan->lavTestLRT():
some models have the same degrees of freedom
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit2 8 20464 20570 179.67
fit2v 8 20666 20772 381.25 201.58 0 0
The speed/accuracy model has vastly better fit.
CDI vs. LWL
Compare to model with CDI variables separated. Note this is confounded by dataset.
<- "task =~ rt + long_window_accuracy + long_window_acc_var + rt_var CDI =~ prod + comp" <- cfa (fa2t_model, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
Warning: lavaan->lav_object_post_check():
some estimated ov variances are negative
summary (fit2t, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 60 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 19
Used Total
Number of observations 1978 2065
Number of missing patterns 16
Model Test User Model:
Test statistic 244.150
Degrees of freedom 8
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 2158.416
Degrees of freedom 15
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.890
Tucker-Lewis Index (TLI) 0.793
Robust Comparative Fit Index (CFI) 0.893
Robust Tucker-Lewis Index (TLI) 0.800
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -10245.223
Loglikelihood unrestricted model (H1) -10123.148
Akaike (AIC) 20528.445
Bayesian (BIC) 20634.652
Sample-size adjusted Bayesian (SABIC) 20574.289
Root Mean Square Error of Approximation:
RMSEA 0.122
90 Percent confidence interval - lower 0.109
90 Percent confidence interval - upper 0.136
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.181
90 Percent confidence interval - lower 0.153
90 Percent confidence interval - upper 0.211
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.095
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
task =~
rt 0.887 0.022 40.070 0.000 0.887 0.882
lng_wndw_ccrcy -0.488 0.025 -19.600 0.000 -0.488 -0.486
lng_wndw_cc_vr 0.380 0.026 14.780 0.000 0.380 0.379
rt_var 0.944 0.026 36.851 0.000 0.944 0.875
CDI =~
prod 1.713 0.921 1.859 0.063 1.713 1.712
comp 0.494 0.276 1.792 0.073 0.494 0.434
Covariances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
task ~~
CDI -0.104 0.075 -1.377 0.168 -0.104 -0.104
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.003 0.023 0.148 0.883 0.003 0.003
.lng_wndw_ccrcy 0.001 0.023 0.042 0.966 0.001 0.001
.lng_wndw_cc_vr -0.001 0.023 -0.033 0.974 -0.001 -0.001
.rt_var 0.019 0.026 0.718 0.473 0.019 0.017
.prod 0.005 0.044 0.117 0.907 0.005 0.005
.comp 0.346 0.065 5.333 0.000 0.346 0.303
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.223 0.023 9.703 0.000 0.223 0.221
.lng_wndw_ccrcy 0.772 0.027 28.137 0.000 0.772 0.764
.lng_wndw_cc_vr 0.861 0.030 29.053 0.000 0.861 0.856
.rt_var 0.274 0.025 11.041 0.000 0.274 0.235
.prod -1.933 3.160 -0.612 0.541 -1.933 -1.931
.comp 1.056 0.274 3.849 0.000 1.056 0.812
task 1.000 1.000 1.000
CDI 1.000 1.000 1.000
graph_sem (model = fit2t, text_size = 2 )
Model comparison. There is an improvement.
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit2t 8 20528 20635 244.15
fit 9 20679 20780 396.58 152.43 0.27669 1 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Which of these is better?
Warning: lavaan->lavTestLRT():
some models have the same degrees of freedom
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
fit2 8 20464 20570 179.67
fit2t 8 20528 20635 244.15 64.482 0 0
No comprehension
Removing comprehension doesn’t seem to clean up the misfit issues.
<- "F1 =~ rt + rt_var + long_window_accuracy + long_window_acc_var + prod" <- cfa (fa_prod_model, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
summary (fit_prod, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 20 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 15
Used Total
Number of observations 1978 2065
Number of missing patterns 11
Model Test User Model:
Test statistic 238.019
Degrees of freedom 5
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 1999.727
Degrees of freedom 10
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.883
Tucker-Lewis Index (TLI) 0.766
Robust Comparative Fit Index (CFI) 0.836
Robust Tucker-Lewis Index (TLI) 0.671
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -9927.537
Loglikelihood unrestricted model (H1) -9808.528
Akaike (AIC) 19885.075
Bayesian (BIC) 19968.922
Sample-size adjusted Bayesian (SABIC) 19921.267
Root Mean Square Error of Approximation:
RMSEA 0.153
90 Percent confidence interval - lower 0.137
90 Percent confidence interval - upper 0.170
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.225
90 Percent confidence interval - lower 0.197
90 Percent confidence interval - upper 0.253
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.087
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.887 0.022 40.067 0.000 0.887 0.883
rt_var 0.943 0.026 36.818 0.000 0.943 0.874
lng_wndw_ccrcy -0.488 0.025 -19.617 0.000 -0.488 -0.486
lng_wndw_cc_vr 0.380 0.026 14.780 0.000 0.380 0.379
prod -0.179 0.050 -3.555 0.000 -0.179 -0.179
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.004 0.023 0.164 0.870 0.004 0.004
.rt_var 0.019 0.026 0.729 0.466 0.019 0.017
.lng_wndw_ccrcy 0.001 0.023 0.042 0.966 0.001 0.001
.lng_wndw_cc_vr -0.001 0.023 -0.033 0.974 -0.001 -0.001
.prod 0.005 0.044 0.121 0.904 0.005 0.005
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.223 0.023 9.635 0.000 0.223 0.220
.rt_var 0.275 0.025 11.085 0.000 0.275 0.236
.lng_wndw_ccrcy 0.771 0.027 28.129 0.000 0.771 0.764
.lng_wndw_cc_vr 0.861 0.030 29.055 0.000 0.861 0.856
.prod 0.969 0.061 15.997 0.000 0.969 0.968
F1 1.000 1.000 1.000
graph_sem (model = fit_prod, text_size = 2 )
Without variances
Without variance.
<- "F1 =~ rt + long_window_accuracy + prod + comp" <- cfa (fa_model_novar, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714 741 1001 1100 1121 1123 1125 1149 1157
1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242
1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333
1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423 1544 1553 1687 1704
1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977 1979 1983 2001 2009
2027 2047 2048.
summary (fit_novar, fit.measures = TRUE , standardized = TRUE )
lavaan 0.6-18 ended normally after 33 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 12
Used Total
Number of observations 1978 2065
Number of missing patterns 11
Model Test User Model:
Test statistic 138.242
Degrees of freedom 2
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 531.946
Degrees of freedom 6
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.741
Tucker-Lewis Index (TLI) 0.223
Robust Comparative Fit Index (CFI) 0.418
Robust Tucker-Lewis Index (TLI) -0.745
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -6184.951
Loglikelihood unrestricted model (H1) -6115.830
Akaike (AIC) 12393.901
Bayesian (BIC) 12460.979
Sample-size adjusted Bayesian (SABIC) 12422.855
Root Mean Square Error of Approximation:
RMSEA 0.186
90 Percent confidence interval - lower 0.160
90 Percent confidence interval - upper 0.212
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.621
90 Percent confidence interval - lower 0.551
90 Percent confidence interval - upper 0.693
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.186
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.454 0.041 11.062 0.000 0.454 0.453
lng_wndw_ccrcy -0.886 0.083 -10.709 0.000 -0.886 -0.882
prod -0.457 0.069 -6.610 0.000 -0.457 -0.458
comp -0.344 0.098 -3.523 0.000 -0.344 -0.342
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.003 0.023 0.145 0.885 0.003 0.003
.lng_wndw_ccrcy -0.002 0.023 -0.091 0.927 -0.002 -0.002
.prod 0.005 0.041 0.109 0.913 0.005 0.005
.comp 0.139 0.069 2.020 0.043 0.139 0.138
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.796 0.041 19.396 0.000 0.796 0.794
.lng_wndw_ccrcy 0.224 0.143 1.568 0.117 0.224 0.222
.prod 0.789 0.069 11.433 0.000 0.789 0.790
.comp 0.893 0.088 10.150 0.000 0.893 0.883
F1 1.000 1.000 1.000
graph_sem (model = fit_novar) + theme (panel.background = element_rect (fill = "white" ))
Fit models with dataset grouping
One big worry here is that it’s between-group confounding that leads to model misfit, not actual factor structure. To deal with this, we fit the same model but with grouping by dataset.
We can’t really handle the missing data here sadly, because there’s too much missing prod and comp data.
<- "F1 =~ rt + rt_var + long_window_accuracy + long_window_acc_var" <- cfa (fa_model_reduced, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 366 379 409 430 466 616 618 621
623 633 636 637 645 646 649 666 670 676 680 689 714 741 1001 1100 1121
1123 1125 1149 1157 1166 1167 1168 1173 1178 1181 1188 1190 1197 1200 1206
1217 1218 1237 1242 1243 1244 1246 1249 1251 1257 1260 1266 1282 1303 1315
1319 1321 1322 1333 1335 1354 1360 1363 1365 1371 1383 1387 1388 1419 1423
1544 1553 1687 1704 1795 1801 1815 1824 1839 1841 1868 1881 1931 1938 1977
1979 1983 2001 2009 2027 2047 2048.
summary (fit_reduced, fit.measures= TRUE , standardize= TRUE )
lavaan 0.6-18 ended normally after 15 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 12
Used Total
Number of observations 1973 2065
Number of missing patterns 5
Model Test User Model:
Test statistic 159.652
Degrees of freedom 2
P-value (Chi-square) 0.000
Model Test Baseline Model:
Test statistic 1908.840
Degrees of freedom 6
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.917
Tucker-Lewis Index (TLI) 0.751
Robust Comparative Fit Index (CFI) 0.934
Robust Tucker-Lewis Index (TLI) 0.801
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -9197.869
Loglikelihood unrestricted model (H1) -9118.043
Akaike (AIC) 18419.738
Bayesian (BIC) 18486.786
Sample-size adjusted Bayesian (SABIC) 18448.662
Root Mean Square Error of Approximation:
RMSEA 0.200
90 Percent confidence interval - lower 0.174
90 Percent confidence interval - upper 0.227
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA 0.212
90 Percent confidence interval - lower 0.183
90 Percent confidence interval - upper 0.243
P-value H_0: Robust RMSEA <= 0.050 0.000
P-value H_0: Robust RMSEA >= 0.080 1.000
Standardized Root Mean Square Residual:
SRMR 0.062
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.890 0.022 39.864 0.000 0.890 0.885
rt_var 0.944 0.026 36.566 0.000 0.944 0.874
lng_wndw_ccrcy -0.485 0.025 -19.486 0.000 -0.485 -0.482
lng_wndw_cc_vr 0.379 0.026 14.741 0.000 0.379 0.378
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.002 0.023 0.086 0.932 0.002 0.002
.rt_var 0.016 0.026 0.616 0.538 0.016 0.015
.lng_wndw_ccrcy 0.001 0.023 0.048 0.962 0.001 0.001
.lng_wndw_cc_vr -0.001 0.023 -0.037 0.970 -0.001 -0.001
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.218 0.024 9.223 0.000 0.218 0.216
.rt_var 0.277 0.026 10.843 0.000 0.277 0.237
.lng_wndw_ccrcy 0.775 0.027 28.204 0.000 0.775 0.767
.lng_wndw_cc_vr 0.862 0.030 29.063 0.000 0.862 0.857
F1 1.000 1.000 1.000
Now the grouped dataset.
<- cfa (fa_model_reduced, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' , group = "dataset_name" )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 366 379.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 409 430.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 466.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 741.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1001.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1100 1121 1123 1125.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1149 1157 1166 1167 1168 1173
1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242 1243 1244 1246 1249
1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333 1335 1354 1360 1363
1365 1371 1383 1387 1388 1419 1423.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1544 1553 1687 1704.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1795 1801 1815 1824 1839 1841
1868 1881 1931 1938 1977 1979 1983 2001 2009 2027 2047 2048.
Warning: lavaan->lav_object_post_check():
some estimated ov variances are negative
summary (fit_reduced_grouped, fit.measures= TRUE , standardize= TRUE )
Warning: lavaan->lav_fit_cfi_lavobject():
computation of robust CFI failed.
Warning: lavaan->lav_fit_rmsea_lavobject():
computation of robust RMSEA failed.
lavaan 0.6-18 ended normally after 8655 iterations
Estimator ML
Optimization method NLMINB
Number of model parameters 240
Number of observations per group: Used Total
adams_marchman_2018 270 270
bacon_gendercues 38 38
baumgartner_2014 98 100
ferguson_eyetrackingr 26 28
fmw_2013 178 179
frank_tablet_2016 88 104
gazetriggered_2020 47 48
mahr_coartic 29 29
newman_genderdistractor 19 19
perry_cowpig 42 42
pomper_prime 63 63
pomper_saffran_2016 60 60
pomper_salientme 43 44
potter_canine 67 67
potter_remix 40 44
reflook_v4 267 310
ronfard_2021 40 40
swingley_aslin_2002 50 50
weaver_zettersten_2024 244 248
yurovsky_2017 264 282
Number of missing patterns per group:
adams_marchman_2018 3
bacon_gendercues 1
baumgartner_2014 5
ferguson_eyetrackingr 3
fmw_2013 3
frank_tablet_2016 5
gazetriggered_2020 4
mahr_coartic 1
newman_genderdistractor 2
perry_cowpig 1
pomper_prime 3
pomper_saffran_2016 3
pomper_salientme 4
potter_canine 4
potter_remix 5
reflook_v4 5
ronfard_2021 4
swingley_aslin_2002 2
weaver_zettersten_2024 5
yurovsky_2017 5
Model Test User Model:
Test statistic 287.425
Degrees of freedom 40
P-value (Chi-square) 0.000
Test statistic for each group:
adams_marchman_2018 0.483
bacon_gendercues 14.124
baumgartner_2014 3.825
ferguson_eyetrackingr 3.128
fmw_2013 34.554
frank_tablet_2016 7.251
gazetriggered_2020 0.740
mahr_coartic 10.958
newman_genderdistractor 6.783
perry_cowpig 1.218
pomper_prime 31.629
pomper_saffran_2016 34.664
pomper_salientme 30.753
potter_canine 20.430
potter_remix 9.798
reflook_v4 26.384
ronfard_2021 9.965
swingley_aslin_2002 17.395
weaver_zettersten_2024 2.257
yurovsky_2017 21.085
Model Test Baseline Model:
Test statistic 1999.897
Degrees of freedom 120
P-value 0.000
User Model versus Baseline Model:
Comparative Fit Index (CFI) 0.868
Tucker-Lewis Index (TLI) 0.605
Robust Comparative Fit Index (CFI) NA
Robust Tucker-Lewis Index (TLI) NA
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -8187.501
Loglikelihood unrestricted model (H1) -8043.788
Akaike (AIC) 16855.002
Bayesian (BIC) 18195.956
Sample-size adjusted Bayesian (SABIC) 17433.467
Root Mean Square Error of Approximation:
RMSEA 0.250
90 Percent confidence interval - lower 0.224
90 Percent confidence interval - upper 0.278
P-value H_0: RMSEA <= 0.050 0.000
P-value H_0: RMSEA >= 0.080 1.000
Robust RMSEA NA
90 Percent confidence interval - lower NA
90 Percent confidence interval - upper NA
P-value H_0: Robust RMSEA <= 0.050 NA
P-value H_0: Robust RMSEA >= 0.080 NA
Standardized Root Mean Square Residual:
SRMR 0.070
Parameter Estimates:
Standard errors Standard
Information Observed
Observed information based on Hessian
Group 1 [adams_marchman_2018]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.716 0.044 16.351 0.000 0.716 0.894
rt_var 0.720 0.047 15.245 0.000 0.720 0.846
lng_wndw_ccrcy -0.390 0.044 -8.786 0.000 -0.390 -0.530
lng_wndw_cc_vr 0.395 0.043 9.145 0.000 0.395 0.549
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.140 0.049 2.874 0.004 0.140 0.175
.rt_var 0.281 0.052 5.418 0.000 0.281 0.331
.lng_wndw_ccrcy -0.132 0.045 -2.943 0.003 -0.132 -0.179
.lng_wndw_cc_vr 0.304 0.044 6.941 0.000 0.304 0.422
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.129 0.032 4.074 0.000 0.129 0.200
.rt_var 0.206 0.035 5.922 0.000 0.206 0.284
.lng_wndw_ccrcy 0.391 0.036 10.921 0.000 0.391 0.719
.lng_wndw_cc_vr 0.361 0.033 10.828 0.000 0.361 0.699
F1 1.000 1.000 1.000
Group 2 [bacon_gendercues]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.762 0.107 7.153 0.000 0.762 0.946
rt_var 0.622 0.098 6.353 0.000 0.622 0.866
lng_wndw_ccrcy -0.444 0.122 -3.628 0.000 -0.444 -0.561
lng_wndw_cc_vr 0.327 0.117 2.805 0.005 0.327 0.466
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.118 0.131 0.900 0.368 0.118 0.146
.rt_var -0.041 0.117 -0.353 0.724 -0.041 -0.057
.lng_wndw_ccrcy 0.608 0.128 4.738 0.000 0.608 0.769
.lng_wndw_cc_vr -0.433 0.114 -3.796 0.000 -0.433 -0.616
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.069 0.068 1.004 0.315 0.069 0.106
.rt_var 0.130 0.051 2.560 0.010 0.130 0.251
.lng_wndw_ccrcy 0.429 0.103 4.174 0.000 0.429 0.685
.lng_wndw_cc_vr 0.386 0.093 4.134 0.000 0.386 0.783
F1 1.000 1.000 1.000
Group 3 [baumgartner_2014]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 1.368 0.154 8.867 0.000 1.368 1.052
rt_var 1.673 0.279 5.992 0.000 1.673 0.819
lng_wndw_ccrcy -0.490 0.108 -4.518 0.000 -0.490 -0.531
lng_wndw_cc_vr -0.033 0.136 -0.246 0.805 -0.033 -0.028
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.080 0.138 -0.575 0.565 -0.080 -0.061
.rt_var 0.276 0.242 1.140 0.254 0.276 0.135
.lng_wndw_ccrcy -0.812 0.096 -8.446 0.000 -0.812 -0.880
.lng_wndw_cc_vr 0.226 0.126 1.798 0.072 0.226 0.190
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.180 0.342 -0.528 0.597 -0.180 -0.107
.rt_var 1.377 0.591 2.330 0.020 1.377 0.330
.lng_wndw_ccrcy 0.611 0.110 5.533 0.000 0.611 0.718
.lng_wndw_cc_vr 1.415 0.211 6.707 0.000 1.415 0.999
F1 1.000 1.000 1.000
Group 4 [ferguson_eyetrackingr]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.755 0.125 6.066 0.000 0.755 1.063
rt_var 0.837 0.205 4.081 0.000 0.837 0.897
lng_wndw_ccrcy -0.781 0.205 -3.807 0.000 -0.781 -0.717
lng_wndw_cc_vr 0.480 0.135 3.558 0.000 0.480 0.669
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.765 0.163 -4.687 0.000 -0.765 -1.076
.rt_var -0.631 0.244 -2.583 0.010 -0.631 -0.677
.lng_wndw_ccrcy 0.287 0.214 1.345 0.179 0.287 0.264
.lng_wndw_cc_vr -0.637 0.141 -4.527 0.000 -0.637 -0.888
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.065 0.040 -1.648 0.099 -0.065 -0.129
.rt_var 0.170 0.091 1.864 0.062 0.170 0.196
.lng_wndw_ccrcy 0.577 0.213 2.704 0.007 0.577 0.486
.lng_wndw_cc_vr 0.284 0.094 3.023 0.003 0.284 0.552
F1 1.000 1.000 1.000
Group 5 [fmw_2013]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.727 0.067 10.866 0.000 0.727 0.826
rt_var 0.637 0.068 9.378 0.000 0.637 0.701
lng_wndw_ccrcy -0.638 0.080 -7.973 0.000 -0.638 -0.657
lng_wndw_cc_vr 0.463 0.073 6.343 0.000 0.463 0.539
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.173 0.066 2.623 0.009 0.173 0.197
.rt_var 0.171 0.068 2.497 0.013 0.171 0.188
.lng_wndw_ccrcy 0.014 0.073 0.189 0.850 0.014 0.014
.lng_wndw_cc_vr 0.547 0.065 8.454 0.000 0.547 0.636
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.246 0.064 3.838 0.000 0.246 0.317
.rt_var 0.421 0.061 6.940 0.000 0.421 0.509
.lng_wndw_ccrcy 0.535 0.083 6.473 0.000 0.535 0.568
.lng_wndw_cc_vr 0.523 0.068 7.705 0.000 0.523 0.709
F1 1.000 1.000 1.000
Group 6 [frank_tablet_2016]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.883 0.219 4.037 0.000 0.883 0.648
rt_var 1.000 0.267 3.742 0.000 1.000 0.800
lng_wndw_ccrcy -0.599 0.182 -3.291 0.001 -0.599 -0.533
lng_wndw_cc_vr 0.805 0.182 4.426 0.000 0.805 0.718
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.101 0.150 0.675 0.500 0.101 0.074
.rt_var 0.047 0.173 0.271 0.786 0.047 0.038
.lng_wndw_ccrcy 0.114 0.135 0.844 0.399 0.114 0.101
.lng_wndw_cc_vr -0.173 0.133 -1.307 0.191 -0.173 -0.155
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 1.077 0.356 3.028 0.002 1.077 0.580
.rt_var 0.561 0.310 1.809 0.070 0.561 0.359
.lng_wndw_ccrcy 0.905 0.201 4.493 0.000 0.905 0.716
.lng_wndw_cc_vr 0.607 0.223 2.715 0.007 0.607 0.484
F1 1.000 1.000 1.000
Group 7 [gazetriggered_2020]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.578 0.135 4.291 0.000 0.578 0.857
rt_var 0.164 0.062 2.637 0.008 0.164 0.470
lng_wndw_ccrcy -0.358 0.138 -2.590 0.010 -0.358 -0.462
lng_wndw_cc_vr 0.373 0.147 2.538 0.011 0.373 0.482
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.044 0.100 0.443 0.658 0.044 0.066
.rt_var -0.342 0.054 -6.313 0.000 -0.342 -0.979
.lng_wndw_ccrcy 0.155 0.115 1.344 0.179 0.155 0.200
.lng_wndw_cc_vr 0.322 0.115 2.804 0.005 0.322 0.417
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.121 0.129 0.940 0.347 0.121 0.266
.rt_var 0.095 0.024 4.023 0.000 0.095 0.779
.lng_wndw_ccrcy 0.471 0.114 4.123 0.000 0.471 0.787
.lng_wndw_cc_vr 0.458 0.119 3.838 0.000 0.458 0.767
F1 1.000 1.000 1.000
Group 8 [mahr_coartic]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.156 0.054 2.901 0.004 0.156 0.522
rt_var 0.122 0.054 2.260 0.024 0.122 0.434
lng_wndw_ccrcy -0.704 0.111 -6.316 0.000 -0.704 -0.978
lng_wndw_cc_vr 0.608 0.122 4.978 0.000 0.608 0.805
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.378 0.056 -6.808 0.000 -0.378 -1.264
.rt_var -0.485 0.052 -9.305 0.000 -0.485 -1.728
.lng_wndw_ccrcy 0.513 0.134 3.836 0.000 0.513 0.712
.lng_wndw_cc_vr 0.591 0.140 4.214 0.000 0.591 0.783
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.065 0.018 3.666 0.000 0.065 0.727
.rt_var 0.064 0.018 3.641 0.000 0.064 0.812
.lng_wndw_ccrcy 0.022 0.079 0.283 0.777 0.022 0.043
.lng_wndw_cc_vr 0.200 0.072 2.778 0.005 0.200 0.351
F1 1.000 1.000 1.000
Group 9 [newman_genderdistractor]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 1.171 0.252 4.640 0.000 1.171 1.237
rt_var 0.913 0.305 2.997 0.003 0.913 0.736
lng_wndw_ccrcy -0.393 0.223 -1.767 0.077 -0.393 -0.414
lng_wndw_cc_vr 0.036 0.143 0.250 0.803 0.036 0.049
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.049 0.217 -0.224 0.822 -0.049 -0.052
.rt_var 0.208 0.286 0.729 0.466 0.208 0.168
.lng_wndw_ccrcy 0.047 0.218 0.215 0.830 0.047 0.049
.lng_wndw_cc_vr -0.046 0.168 -0.276 0.783 -0.046 -0.063
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.476 0.559 -0.851 0.395 -0.476 -0.531
.rt_var 0.705 0.406 1.737 0.082 0.705 0.458
.lng_wndw_ccrcy 0.747 0.250 2.986 0.003 0.747 0.828
.lng_wndw_cc_vr 0.536 0.174 3.084 0.002 0.536 0.998
F1 1.000 1.000 1.000
Group 10 [perry_cowpig]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.632 0.128 4.940 0.000 0.632 0.946
rt_var 0.487 0.118 4.137 0.000 0.487 0.725
lng_wndw_ccrcy -0.159 0.108 -1.465 0.143 -0.159 -0.240
lng_wndw_cc_vr 0.240 0.112 2.142 0.032 0.240 0.352
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.335 0.103 -3.244 0.001 -0.335 -0.501
.rt_var -0.051 0.104 -0.490 0.624 -0.051 -0.076
.lng_wndw_ccrcy -0.284 0.102 -2.779 0.005 -0.284 -0.429
.lng_wndw_cc_vr 0.186 0.105 1.768 0.077 0.186 0.273
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.047 0.130 0.361 0.718 0.047 0.105
.rt_var 0.214 0.088 2.428 0.015 0.214 0.475
.lng_wndw_ccrcy 0.413 0.091 4.538 0.000 0.413 0.943
.lng_wndw_cc_vr 0.406 0.091 4.442 0.000 0.406 0.876
F1 1.000 1.000 1.000
Group 11 [pomper_prime]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.491 0.058 8.410 0.000 0.491 0.983
rt_var 0.567 0.082 6.884 0.000 0.567 0.821
lng_wndw_ccrcy -0.259 0.095 -2.722 0.006 -0.259 -0.352
lng_wndw_cc_vr 0.292 0.121 2.412 0.016 0.292 0.350
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.599 0.063 -9.519 0.000 -0.599 -1.199
.rt_var -0.525 0.088 -5.994 0.000 -0.525 -0.760
.lng_wndw_ccrcy 0.801 0.094 8.562 0.000 0.801 1.087
.lng_wndw_cc_vr -0.224 0.106 -2.112 0.035 -0.224 -0.268
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.008 0.036 0.225 0.822 0.008 0.033
.rt_var 0.155 0.053 2.940 0.003 0.155 0.326
.lng_wndw_ccrcy 0.476 0.086 5.536 0.000 0.476 0.876
.lng_wndw_cc_vr 0.612 0.115 5.335 0.000 0.612 0.878
F1 1.000 1.000 1.000
Group 12 [pomper_saffran_2016]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 1.176 0.177 6.660 0.000 1.176 1.096
rt_var 0.584 0.150 3.903 0.000 0.584 0.736
lng_wndw_ccrcy -0.225 0.095 -2.358 0.018 -0.225 -0.305
lng_wndw_cc_vr 0.187 0.125 1.494 0.135 0.187 0.222
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.405 0.140 -2.902 0.004 -0.405 -0.377
.rt_var -0.445 0.106 -4.202 0.000 -0.445 -0.562
.lng_wndw_ccrcy 1.150 0.095 12.055 0.000 1.150 1.556
.lng_wndw_cc_vr -0.747 0.109 -6.884 0.000 -0.747 -0.889
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.232 0.363 -0.640 0.522 -0.232 -0.202
.rt_var 0.288 0.103 2.790 0.005 0.288 0.459
.lng_wndw_ccrcy 0.495 0.091 5.448 0.000 0.495 0.907
.lng_wndw_cc_vr 0.672 0.125 5.374 0.000 0.672 0.951
F1 1.000 1.000 1.000
Group 13 [pomper_salientme]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt -0.023 0.013 -1.859 0.063 -0.023 -0.034
rt_var -23.989 11.227 -2.137 0.033 -23.989 -26.580
lng_wndw_ccrcy -0.005 0.005 -0.864 0.388 -0.005 -0.005
lng_wndw_cc_vr -0.002 0.005 -0.457 0.648 -0.002 -0.002
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.618 0.115 -5.382 0.000 -0.618 -0.885
.rt_var -0.504 0.151 -3.326 0.001 -0.504 -0.558
.lng_wndw_ccrcy 0.955 0.150 6.357 0.000 0.955 1.005
.lng_wndw_cc_vr -0.396 0.183 -2.169 0.030 -0.396 -0.343
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.487 0.113 4.306 0.000 0.487 0.999
.rt_var -574.650 538.637 -1.067 0.286 -574.650 -705.511
.lng_wndw_ccrcy 0.903 0.202 4.472 0.000 0.903 1.000
.lng_wndw_cc_vr 1.336 0.299 4.472 0.000 1.336 1.000
F1 1.000 1.000 1.000
Group 14 [potter_canine]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.662 0.123 5.366 0.000 0.662 0.714
rt_var 0.736 0.139 5.288 0.000 0.736 0.779
lng_wndw_ccrcy -0.325 0.114 -2.861 0.004 -0.325 -0.432
lng_wndw_cc_vr 0.567 0.156 3.633 0.000 0.567 0.554
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.226 0.114 1.978 0.048 0.226 0.243
.rt_var -0.104 0.119 -0.869 0.385 -0.104 -0.110
.lng_wndw_ccrcy 0.367 0.093 3.965 0.000 0.367 0.488
.lng_wndw_cc_vr 0.212 0.126 1.687 0.092 0.212 0.207
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.421 0.117 3.588 0.000 0.421 0.491
.rt_var 0.351 0.141 2.489 0.013 0.351 0.393
.lng_wndw_ccrcy 0.461 0.093 4.963 0.000 0.461 0.813
.lng_wndw_cc_vr 0.725 0.173 4.185 0.000 0.725 0.693
F1 1.000 1.000 1.000
Group 15 [potter_remix]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.917 0.175 5.231 0.000 0.917 0.887
rt_var 0.878 0.213 4.118 0.000 0.878 0.864
lng_wndw_ccrcy -0.431 0.202 -2.137 0.033 -0.431 -0.426
lng_wndw_cc_vr 0.438 0.286 1.532 0.126 0.438 0.321
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.061 0.165 -0.368 0.713 -0.061 -0.059
.rt_var -0.083 0.179 -0.466 0.641 -0.083 -0.082
.lng_wndw_ccrcy 0.310 0.191 1.628 0.103 0.310 0.307
.lng_wndw_cc_vr 0.416 0.261 1.595 0.111 0.416 0.305
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.227 0.224 1.016 0.310 0.227 0.213
.rt_var 0.262 0.195 1.341 0.180 0.262 0.253
.lng_wndw_ccrcy 0.836 0.240 3.480 0.001 0.836 0.818
.lng_wndw_cc_vr 1.675 0.472 3.552 0.000 1.675 0.897
F1 1.000 1.000 1.000
Group 16 [reflook_v4]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.890 0.063 14.038 0.000 0.890 0.940
rt_var 0.907 0.079 11.478 0.000 0.907 0.873
lng_wndw_ccrcy -0.314 0.074 -4.255 0.000 -0.314 -0.311
lng_wndw_cc_vr 0.364 0.075 4.852 0.000 0.364 0.363
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.286 0.060 -4.800 0.000 -0.286 -0.302
.rt_var -0.384 0.070 -5.467 0.000 -0.384 -0.370
.lng_wndw_ccrcy 0.248 0.067 3.686 0.000 0.248 0.246
.lng_wndw_cc_vr -0.592 0.067 -8.839 0.000 -0.592 -0.589
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.104 0.080 1.299 0.194 0.104 0.116
.rt_var 0.257 0.082 3.117 0.002 0.257 0.238
.lng_wndw_ccrcy 0.918 0.089 10.315 0.000 0.918 0.903
.lng_wndw_cc_vr 0.877 0.087 10.099 0.000 0.877 0.869
F1 1.000 1.000 1.000
Group 17 [ronfard_2021]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.086 0.069 1.245 0.213 0.086 0.261
rt_var 0.217 0.067 3.220 0.001 0.217 0.531
lng_wndw_ccrcy -0.754 0.160 -4.702 0.000 -0.754 -0.760
lng_wndw_cc_vr 0.836 0.142 5.892 0.000 0.836 0.948
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt -0.497 0.053 -9.426 0.000 -0.497 -1.508
.rt_var -0.458 0.067 -6.837 0.000 -0.458 -1.119
.lng_wndw_ccrcy 0.452 0.159 2.843 0.004 0.452 0.456
.lng_wndw_cc_vr 0.441 0.142 3.114 0.002 0.441 0.500
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.101 0.024 4.163 0.000 0.101 0.932
.rt_var 0.120 0.030 4.074 0.000 0.120 0.718
.lng_wndw_ccrcy 0.415 0.155 2.673 0.008 0.415 0.422
.lng_wndw_cc_vr 0.079 0.164 0.483 0.629 0.079 0.102
F1 1.000 1.000 1.000
Group 18 [swingley_aslin_2002]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.573 0.171 3.349 0.001 0.573 0.666
rt_var 0.649 0.178 3.640 0.000 0.649 0.835
lng_wndw_ccrcy -0.240 0.134 -1.790 0.073 -0.240 -0.321
lng_wndw_cc_vr 0.122 0.151 0.807 0.420 0.122 0.143
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.143 0.122 1.179 0.239 0.143 0.167
.rt_var -0.098 0.117 -0.830 0.406 -0.098 -0.126
.lng_wndw_ccrcy -0.321 0.106 -3.042 0.002 -0.321 -0.430
.lng_wndw_cc_vr 0.541 0.121 4.474 0.000 0.541 0.633
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.413 0.173 2.380 0.017 0.413 0.557
.rt_var 0.183 0.195 0.937 0.349 0.183 0.303
.lng_wndw_ccrcy 0.500 0.108 4.624 0.000 0.500 0.897
.lng_wndw_cc_vr 0.715 0.145 4.939 0.000 0.715 0.980
F1 1.000 1.000 1.000
Group 19 [weaver_zettersten_2024]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 0.884 0.073 12.096 0.000 0.884 0.915
rt_var 0.913 0.083 10.970 0.000 0.913 0.831
lng_wndw_ccrcy -0.139 0.039 -3.584 0.000 -0.139 -0.248
lng_wndw_cc_vr 0.285 0.059 4.849 0.000 0.285 0.351
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.468 0.063 7.463 0.000 0.468 0.484
.rt_var 0.624 0.072 8.664 0.000 0.624 0.567
.lng_wndw_ccrcy -0.838 0.036 -23.189 0.000 -0.838 -1.496
.lng_wndw_cc_vr -0.018 0.052 -0.335 0.737 -0.018 -0.022
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.151 0.096 1.573 0.116 0.151 0.162
.rt_var 0.374 0.107 3.502 0.000 0.374 0.310
.lng_wndw_ccrcy 0.294 0.027 10.823 0.000 0.294 0.938
.lng_wndw_cc_vr 0.577 0.055 10.446 0.000 0.577 0.877
F1 1.000 1.000 1.000
Group 20 [yurovsky_2017]:
Latent Variables:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
F1 =~
rt 1.121 0.092 12.206 0.000 1.121 0.916
rt_var 0.962 0.104 9.297 0.000 0.962 0.872
lng_wndw_ccrcy -0.376 0.077 -4.870 0.000 -0.376 -0.367
lng_wndw_cc_vr 0.310 0.084 3.675 0.000 0.310 0.285
Intercepts:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.048 0.080 0.599 0.549 0.048 0.039
.rt_var -0.154 0.082 -1.886 0.059 -0.154 -0.139
.lng_wndw_ccrcy 0.048 0.067 0.709 0.479 0.048 0.046
.lng_wndw_cc_vr -0.205 0.071 -2.873 0.004 -0.205 -0.189
Variances:
Estimate Std.Err z-value P(>|z|) Std.lv Std.all
.rt 0.240 0.152 1.576 0.115 0.240 0.161
.rt_var 0.293 0.111 2.637 0.008 0.293 0.240
.lng_wndw_ccrcy 0.907 0.090 10.101 0.000 0.907 0.865
.lng_wndw_cc_vr 1.084 0.105 10.325 0.000 1.084 0.919
F1 1.000 1.000 1.000
We can look at invariance.
<- cfa (fa_model_reduced, d_sub_mat_s, std.lv= TRUE , missing= 'fiml' , group = "dataset_name" , group.equal = "loadings" )
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 366 379.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 409 430.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 466.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 616 618 621 623 633 636 637 645
646 649 666 670 676 680 689 714.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 741.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1001.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1100 1121 1123 1125.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1149 1157 1166 1167 1168 1173
1178 1181 1188 1190 1197 1200 1206 1217 1218 1237 1242 1243 1244 1246 1249
1251 1257 1260 1266 1282 1303 1315 1319 1321 1322 1333 1335 1354 1360 1363
1365 1371 1383 1387 1388 1419 1423.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1544 1553 1687 1704.
Warning: lavaan->lav_data_full():
some cases are empty and will be ignored: 1795 1801 1815 1824 1839 1841
1868 1881 1931 1938 1977 1979 1983 2001 2009 2027 2047 2048.
Warning: lavaan->lav_object_post_check():
some estimated ov variances are negative
# summary(fit_reduced_grouped_loadings, fit.measures=TRUE, standardize=TRUE) lavTestLRT (fit_reduced_grouped, fit_reduced_grouped_loadings)
Chi-Squared Difference Test
Df AIC BIC Chisq Chisq diff RMSEA Df diff
fit_reduced_grouped 40 16855 18196 287.43
fit_reduced_grouped_loadings 97 16936 17959 482.68 195.25 0.1568 57
Pr(>Chisq)
fit_reduced_grouped
fit_reduced_grouped_loadings < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
This test suggests that we reject the null and the non-invariant model is better.
