Quiroga et al. (2019) Reanalysis
library(pacman); p_load(lavaan, semPlot, qpcR, egg)
QEAMat = '
1
0.19 1
0.05 0.23 1
0.29 0.43 0.45 1
0.33 0.57 0.39 0.68 1
0.3 0.42 0.38 0.49 0.69 1
-0.1 0.16 0.32 0.23 0.19 0.15 1
0.24 0.26 0.51 0.6 0.53 0.51 0.27 1
0.06 0.27 0.33 0.47 0.31 0.31 0.1 0.38 1
0.02 0.15 0.21 0.27 0.2 0.19 0.21 0.37 0.21 1
0.3 0.34 0.31 0.38 0.55 0.32 0.03 0.32 0.17 0.13 1
0.34 0.23 0.17 0.37 0.5 0.38 0.14 0.4 0.17 0.16 0.36 1
-0.05 0.05 0.15 0.13 0.08 0.01 0.05 0.05 0.1 0.02 0.13 0.02 1
0.33 0.51 0.37 0.43 0.6 0.55 0.19 0.41 0.3 0.26 0.54 0.51 0.22 1
0.37 0.36 0.22 0.3 0.47 0.4 0 0.27 0.15 0.2 0.51 0.44 0.21 0.56 1
0.03 0.22 0.32 0.23 0.24 0.1 0.08 0.18 0.21 0.07 0.31 0.1 0.57 0.29 0.2 1'
#Variable names
Names = list("ArtofB", "Blek", "CrazyPool", "EDGE", "Hook", "RailMaze", "SpaceInvd", "Splatoon", "SkyJump", "Unposs", "D48", "PMAS", "DATPSA", "DATAR", "DATSR", "TPRVD")
#SDs
SDs <- c(3.36, 4.22, 2.77, 1.06, 3.23, 2.18, 355.91, 4.52, 34.77, 11.25, 3.26, 11.64, 13.22, 3.54, 4.68, 51.37)
#Convert to VCV
QEAMat.cor = getCov(QEAMat, names = Names)
QEAMat.cov = lavaan::cor2cov(R = QEAMat.cor, sds = SDs)
N <- 134
#For plotting
LATS <- list(
GFP = c("D48", "DATAR"),
GVP = c("PMAS", "DATSR"),
GSP = c("DATPSA", "TPRVD"),
GFV = c("ArtofB", "Blek", "Hook", "RailMaze"),
GVV = c("CrazyPool", "EDGE", "Splatoon"),
GSV = c("SpaceInvd", "SkyJump", "Unposs"))I suspected ArtofB would not load on GVV and so attempted a model with it and found that it, indeed, did not load on GVV when cross-loaded with GFV, so I removed it here.
Analysis
The motivating question for this reanalysis is the question of how much of the deviation from a correlation of 1 between general factors in Quiroga et al.’s (2019) study is due to model-based psychometric sampling error. Even if the answer for their data is “not much”, a lack of breadth might mean that substantial psychometric sampling issues still impact the result.
#Lower-order Factor Model, Used
QEALO <- '
gVG =~ ArtofB + Blek + CrazyPool + EDGE + Hook + RailMaze + SpaceInvd + Splatoon + SkyJump + Unposs
gPsy =~ D48 + PMAS + DATPSA + DATAR + DATSR + TPRVD
Hook ~~ Blek + RailMaze
DATPSA ~~ TPRVD'
#Higher-order Factor Model, Theory-Supported, Unused
QEAHOF <- '
GFP =~ D48 + DATAR
GVP =~ PMAS + DATSR
GSP =~ DATPSA + TPRVD
GFV =~ ArtofB + Blek + Hook + RailMaze
GVV =~ CrazyPool + EDGE + Splatoon
GSV =~ SpaceInvd + SkyJump + Unposs
gVG =~ GFV + GVV + GSV
gPsy =~ GFP + GVP + GSP'
QEALOF <- cfa(QEALO, sample.cov = QEAMat.cov, sample.nobs = N, std.lv = T, orthogonal = F)
QEAHOFF <- cfa(QEAHOF, sample.cov = QEAMat.cov, sample.nobs = N, std.lv = T, control=list(rel.tol=1e-6))
summary(QEALOF, stand = T, fit = T); summary(QEAHOFF, stand = T, fit = T)## lavaan 0.6-7 ended normally after 81 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of free parameters 36
##
## Number of observations 134
##
## Model Test User Model:
##
## Test statistic 169.122
## Degrees of freedom 100
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 862.238
## Degrees of freedom 120
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.907
## Tucker-Lewis Index (TLI) 0.888
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -6975.267
## Loglikelihood unrestricted model (H1) -6890.706
##
## Akaike (AIC) 14022.535
## Bayesian (BIC) 14126.857
## Sample-size adjusted Bayesian (BIC) 14012.980
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.072
## 90 Percent confidence interval - lower 0.053
## 90 Percent confidence interval - upper 0.090
## P-value RMSEA <= 0.05 0.032
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.070
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## gVG =~
## ArtofB 1.213 0.297 4.081 0.000 1.213 0.362
## Blek 2.255 0.360 6.260 0.000 2.255 0.536
## CrazyPool 1.546 0.232 6.665 0.000 1.546 0.560
## EDGE 0.838 0.080 10.518 0.000 0.838 0.794
## Hook 2.635 0.239 11.010 0.000 2.635 0.820
## RailMaze 1.492 0.175 8.506 0.000 1.492 0.687
## SpaceInvd 98.890 31.953 3.095 0.002 98.890 0.279
## Splatoon 3.224 0.354 9.102 0.000 3.224 0.716
## SkyJump 16.561 2.988 5.542 0.000 16.561 0.478
## Unposs 3.852 0.999 3.857 0.000 3.852 0.344
## gPsy =~
## D48 2.187 0.266 8.218 0.000 2.187 0.673
## PMAS 7.074 0.975 7.257 0.000 7.074 0.610
## DATPSA 2.799 1.219 2.295 0.022 2.799 0.212
## DATAR 2.924 0.269 10.861 0.000 2.924 0.829
## DATSR 3.146 0.382 8.239 0.000 3.146 0.675
## TPRVD 17.600 4.639 3.794 0.000 17.600 0.344
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .Blek ~~
## .Hook 1.612 0.665 2.425 0.015 1.612 0.247
## .Hook ~~
## .RailMaze 0.831 0.333 2.498 0.012 0.831 0.287
## .DATPSA ~~
## .TPRVD 334.946 61.884 5.412 0.000 334.946 0.542
## gVG ~~
## gPsy 0.787 0.052 15.252 0.000 0.787 0.787
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .ArtofB 9.735 1.214 8.018 0.000 9.735 0.869
## .Blek 12.588 1.646 7.649 0.000 12.588 0.712
## .CrazyPool 5.226 0.681 7.675 0.000 5.226 0.686
## .EDGE 0.412 0.066 6.230 0.000 0.412 0.370
## .Hook 3.372 0.578 5.837 0.000 3.372 0.327
## .RailMaze 2.491 0.356 6.988 0.000 2.491 0.528
## .SpaceInvd 115947.468 14328.913 8.092 0.000 115947.468 0.922
## .Splatoon 9.882 1.413 6.994 0.000 9.882 0.487
## .SkyJump 925.652 117.836 7.855 0.000 925.652 0.771
## .Unposs 110.781 13.784 8.037 0.000 110.781 0.882
## .D48 5.767 0.828 6.961 0.000 5.767 0.547
## .PMAS 84.444 11.536 7.320 0.000 84.444 0.628
## .DATPSA 165.631 20.406 8.117 0.000 165.631 0.955
## .DATAR 3.886 0.789 4.928 0.000 3.886 0.312
## .DATSR 11.845 1.704 6.952 0.000 11.845 0.545
## .TPRVD 2309.410 288.922 7.993 0.000 2309.410 0.882
## gVG 1.000 1.000 1.000
## gPsy 1.000 1.000 1.000## lavaan 0.6-7 ended normally after 95 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of free parameters 39
##
## Number of observations 134
##
## Model Test User Model:
##
## Test statistic 161.794
## Degrees of freedom 97
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 862.238
## Degrees of freedom 120
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.913
## Tucker-Lewis Index (TLI) 0.892
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -6971.603
## Loglikelihood unrestricted model (H1) -6890.706
##
## Akaike (AIC) 14021.207
## Bayesian (BIC) 14134.222
## Sample-size adjusted Bayesian (BIC) 14010.856
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.071
## 90 Percent confidence interval - lower 0.051
## 90 Percent confidence interval - upper 0.089
## P-value RMSEA <= 0.05 0.043
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.067
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## GFP =~
## D48 0.191 1.349 0.142 0.887 2.195 0.676
## DATAR 0.255 1.806 0.141 0.888 2.931 0.831
## GVP =~
## PMAS 1.389 2.686 0.517 0.605 7.194 0.620
## DATSR 0.617 1.196 0.516 0.606 3.195 0.685
## GSP =~
## DATPSA 7.252 1.485 4.885 0.000 7.745 0.588
## TPRVD 46.436 9.806 4.736 0.000 49.589 0.969
## GFV =~
## ArtofB 0.410 0.167 2.453 0.014 1.252 0.374
## Blek 0.837 0.301 2.779 0.005 2.557 0.608
## Hook 0.966 0.342 2.826 0.005 2.953 0.918
## RailMaze 0.532 0.186 2.866 0.004 1.627 0.749
## GVV =~
## CrazyPool 0.744 0.172 4.316 0.000 1.606 0.582
## EDGE 0.403 0.085 4.750 0.000 0.871 0.824
## Splatoon 1.546 0.326 4.737 0.000 3.340 0.742
## GSV =~
## SpaceInvd 63.065 32.066 1.967 0.049 108.510 0.306
## SkyJump 10.547 4.794 2.200 0.028 18.147 0.524
## Unposs 2.594 1.205 2.152 0.031 4.462 0.398
## gVG =~
## GFV 2.889 1.116 2.588 0.010 0.945 0.945
## GVV 1.914 0.467 4.100 0.000 0.886 0.886
## GSV 1.400 0.659 2.124 0.034 0.814 0.814
## gPsy =~
## GFP 11.446 81.313 0.141 0.888 0.996 0.996
## GVP 5.084 9.971 0.510 0.610 0.981 0.981
## GSP 0.375 0.126 2.967 0.003 0.351 0.351
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## gVG ~~
## gPsy 0.832 0.050 16.505 0.000 0.832 0.832
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .D48 5.732 0.826 6.940 0.000 5.732 0.543
## .DATAR 3.846 0.911 4.220 0.000 3.846 0.309
## .PMAS 82.760 12.367 6.692 0.000 82.760 0.615
## .DATSR 11.536 1.990 5.798 0.000 11.536 0.531
## .DATPSA 113.461 25.363 4.474 0.000 113.461 0.654
## .TPRVD 159.021 870.988 0.183 0.855 159.021 0.061
## .ArtofB 9.637 1.198 8.044 0.000 9.637 0.860
## .Blek 11.139 1.455 7.655 0.000 11.139 0.630
## .Hook 1.637 0.500 3.277 0.001 1.637 0.158
## .RailMaze 2.070 0.298 6.949 0.000 2.070 0.439
## .CrazyPool 5.039 0.677 7.448 0.000 5.039 0.661
## .EDGE 0.358 0.072 4.934 0.000 0.358 0.321
## .Splatoon 9.126 1.436 6.357 0.000 9.126 0.450
## .SpaceInvd 113944.164 14773.702 7.713 0.000 113944.164 0.906
## .SkyJump 870.809 148.396 5.868 0.000 870.809 0.726
## .Unposs 105.727 14.613 7.235 0.000 105.727 0.842
## .GFP 1.000 0.008 0.008
## .GVP 1.000 0.037 0.037
## .GSP 1.000 0.877 0.877
## .GFV 1.000 0.107 0.107
## .GVV 1.000 0.214 0.214
## .GSV 1.000 0.338 0.338
## gVG 1.000 1.000 1.000
## gPsy 1.000 1.000 1.000resid(QEALOF, "cor"); resid(QEAHOFF, "cor")## $type
## [1] "cor.bollen"
##
## $cov
## ArtofB Blek CrzyPl EDGE Hook RailMz SpcInv Splatn SkyJmp Unposs
## ArtofB 0.000
## Blek -0.004 0.000
## CrazyPool -0.153 -0.070 0.000
## EDGE 0.002 0.004 0.005 0.000
## Hook 0.033 0.010 -0.070 0.029 0.000
## RailMaze 0.051 0.051 -0.005 -0.055 0.007 0.000
## SpaceInvd -0.201 0.010 0.164 0.009 -0.039 -0.042 0.000
## Splatoon -0.019 -0.124 0.109 0.032 -0.057 0.018 0.070 0.000
## SkyJump -0.113 0.014 0.062 0.090 -0.082 -0.018 -0.033 0.038 0.000
## Unposs -0.105 -0.034 0.018 -0.003 -0.082 -0.046 0.114 0.124 0.046 0.000
## D48 0.108 0.056 0.013 -0.041 0.115 -0.044 -0.118 -0.059 -0.083 -0.052
## PMAS 0.166 -0.028 -0.099 -0.011 0.106 0.050 0.006 0.056 -0.060 -0.005
## DATPSA -0.111 -0.040 0.056 -0.003 -0.057 -0.105 0.003 -0.070 0.020 -0.037
## DATAR 0.094 0.160 0.004 -0.088 0.065 0.102 0.008 -0.057 -0.012 0.036
## DATSR 0.178 0.075 -0.077 -0.122 0.034 0.035 -0.148 -0.110 -0.104 0.018
## TPRVD -0.068 0.075 0.168 0.015 0.018 -0.086 0.005 -0.014 0.081 -0.023
## D48 PMAS DATPSA DATAR DATSR TPRVD
## ArtofB
## Blek
## CrazyPool
## EDGE
## Hook
## RailMaze
## SpaceInvd
## Splatoon
## SkyJump
## Unposs
## D48 0.000
## PMAS -0.051 0.000
## DATPSA -0.013 -0.110 0.000
## DATAR -0.018 0.004 0.044 0.000
## DATSR 0.056 0.028 0.067 0.001 0.000
## TPRVD 0.078 -0.110 0.000 0.005 -0.032 0.000## $type
## [1] "cor.bollen"
##
## $cov
## D48 DATAR PMAS DATSR DATPSA TPRVD ArtofB Blek Hook RailMz
## D48 0.000
## DATAR -0.022 0.000
## PMAS -0.050 0.006 0.000
## DATSR 0.057 0.003 0.015 0.000
## DATPSA -0.009 0.049 -0.106 0.071 0.000
## TPRVD 0.081 0.008 -0.107 -0.029 0.000 0.000
## ArtofB 0.102 0.087 0.161 0.172 -0.111 -0.070 0.000
## Blek 0.018 0.114 -0.061 0.039 -0.049 0.057 -0.037 0.000
## Hook 0.064 0.003 0.061 -0.015 -0.069 -0.005 -0.013 0.012 0.000
## RailMaze -0.076 0.063 0.022 0.004 -0.111 -0.100 0.020 -0.036 0.003 0.000
## CrazyPool 0.021 0.015 -0.091 -0.068 0.062 0.174 -0.132 -0.066 -0.057 0.015
## EDGE -0.029 -0.073 0.000 -0.108 0.005 0.023 0.032 0.010 0.047 -0.027
## Splatoon -0.048 -0.043 0.067 -0.098 -0.063 -0.006 0.008 -0.118 -0.040 0.045
## SpaceInvd -0.109 0.019 0.014 -0.139 0.007 0.010 -0.188 0.017 -0.026 -0.026
## SkyJump -0.069 0.006 -0.046 -0.088 0.027 0.089 -0.091 0.025 -0.060 0.008
## Unposs -0.051 0.037 -0.004 0.019 -0.036 -0.022 -0.095 -0.036 -0.081 -0.039
## CrzyPl EDGE Splatn SpcInv SkyJmp Unposs
## D48
## DATAR
## PMAS
## DATSR
## DATPSA
## TPRVD
## ArtofB
## Blek
## Hook
## RailMaze
## CrazyPool 0.000
## EDGE -0.030 0.000
## Splatoon 0.079 -0.011 0.000
## SpaceInvd 0.192 0.048 0.106 0.000
## SkyJump 0.110 0.159 0.100 -0.060 0.000
## Unposs 0.043 0.033 0.157 0.088 0.001 0.000LOFP <- semPaths(QEALOF, "model", "std", title = F, residuals = F, mar = c(2, 1, 3, 1), groups = "LATS", layout = "circle2", posCol = c("skyblue4", "red"), pastel = T, sizeMan = 7, edge.label.cex = 1.2)
HOFP <- semPaths(QEAHOFF, "model", "std", title = F, residuals = F, mar = c(2, 1, 3, 1), groups = "LATS", layout = "circle2", posCol = c("skyblue4", "red"), pastel = T, bifactor = c("gVG", "gPsy"), sizeMan = 7, edge.label.cex = 1.2)
AIC <- c(fitMeasures(QEALOF, "aic"), fitMeasures(QEAHOFF, "aic"))
AICweights <- akaike.weights(AIC)$weights
AMods <- data.frame(Model = factor(rep(c("Lower-Order", "Higher-Order"))),
CRIT = factor(rep('AIC', each = 2)),
values = AICweights)
BIC <- c(fitMeasures(QEALOF, "bic"), fitMeasures(QEAHOFF, "bic"))
BICweights <- akaike.weights(BIC)$weights
BMods <- data.frame(Model = factor(rep(c("Lower-Order", "Higher-Order"))),
CRIT = factor(rep('BIC', each = 2)),
values = BICweights)
AW <- ggplot(AMods, aes(Model, values, fill = Model)) + geom_bar(stat = "identity", position = "dodge", show.legend = F) + coord_cartesian(ylim = c(0,1)) + xlab('Akaike Weights') + ylab('Normalized Probability') + theme_bw()
BW <- ggplot(BMods, aes(Model, values, fill = Model)) + geom_bar(stat = "identity", position = "dodge", show.legend = F) + coord_cartesian(ylim = c(0,1)) + xlab('Schwarz Weights') + ylab('Normalized Probability') + theme_bw()
ggarrange(AW, BW, ncol = 2)
It doesn’t look like the model had much of any effect on the result arrived at by Quiroga et al. (2019), but there are violations of local independence that look to be related to their group factors across assessment types. The higher-order model did not fit significantly better or have much more weight going for it than the lower-order model in any area. The relatively modest relationship between the general variance in these “batteries” is probably just a sampling issue and, perhaps, a bit of a reliability one, what with the small number of indicators per factor and uncertain reliabilities of videogames.
This is an interesting area of research!
References
Quiroga, M. A., Diaz, A., Román, F. J., Privado, J., & Colom, R. (2019). Intelligence and video games: Beyond “brain-games.” Intelligence, 75(C), 85–94.