German Student Outcomes and COVID-19
Setup
library(pacman); p_load(lavaan, psych, DT)
FITM <- c("chisq", "df", "nPar", "cfi", "rmsea", "rmsea.ci.lower", "rmsea.ci.upper", "aic", "bic")
datatable(An1Data, extensions = c("Buttons", "FixedColumns"), options = list(dom = 'Bfrtip', buttons = c('copy', 'csv', 'print'), scrollX = T, fixedColumns = list(leftColumns = 3)))datatable(An2Data, extensions = c("Buttons", "FixedColumns"), options = list(dom = 'Bfrtip', buttons = c('copy', 'csv', 'print'), scrollX = T, fixedColumns = list(leftColumns = 3)))Rationale
Breit et al. (2023) have a paper on German student’s intelligence test results during the COVID-19 pandemic. Unfortunately, their data is unusable. I tried tons of rotations and nothing worked. These models just fit terribly and testing for measurement invariance is impossible with their data. Any papers using this data are similarly compromised, and perhaps papers using the BIS-HB in general are as well. That means there may be issues with Breit, Scherrer & Preckel (2021a, b).
Analysis
Analysis 1
set.seed(100)
fa.parallel(An1Data[,7:13])## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.
## Parallel analysis suggests that the number of factors = 3 and the number of components = 1fa(An1Data[,7:13], nfactors = 1)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Factor Analysis using method = minres
## Call: fa(r = An1Data[, 7:13], nfactors = 1)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 h2 u2 com
## S 0.76 0.57 0.43 1
## M 0.76 0.58 0.42 1
## C 0.69 0.48 0.52 1
## R 0.81 0.65 0.35 1
## F 0.87 0.76 0.24 1
## N 0.84 0.70 0.30 1
## V 0.85 0.72 0.28 1
##
## MR1
## SS loadings 4.47
## Proportion Var 0.64
##
## Mean item complexity = 1
## Test of the hypothesis that 1 factor is sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.34 with Chi Square of 3602.34
## The degrees of freedom for the model are 14 and the objective function was 2.7
##
## The root mean square of the residuals (RMSR) is 0.06
## The df corrected root mean square of the residuals is 0.08
##
## The harmonic number of observations is 495 with the empirical chi square 79.91 with prob < 2.9e-11
## The total number of observations was 495 with Likelihood Chi Square = 1324.45 with prob < 3e-274
##
## Tucker Lewis Index of factoring reliability = 0.45
## RMSEA index = 0.435 and the 90 % confidence intervals are 0.416 0.455
## BIC = 1237.59
## Fit based upon off diagonal values = 0.99fa(An1Data[,7:13], nfactors = 2)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed
## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An1Data[, 7:13], nfactors = 2)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 h2 u2 com
## S 0.75 -0.05 0.56 0.4363 1.0
## M 0.76 -0.21 0.62 0.3774 1.1
## C 0.77 0.64 1.00 -0.0037 1.9
## R 0.81 -0.25 0.72 0.2821 1.2
## F 0.87 0.02 0.75 0.2513 1.0
## N 0.83 -0.11 0.70 0.2987 1.0
## V 0.84 -0.03 0.71 0.2897 1.0
##
## MR1 MR2
## SS loadings 4.54 0.52
## Proportion Var 0.65 0.07
## Cumulative Var 0.65 0.72
## Proportion Explained 0.90 0.10
## Cumulative Proportion 0.90 1.00
##
## Mean item complexity = 1.2
## Test of the hypothesis that 2 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.34 with Chi Square of 3602.34
## The degrees of freedom for the model are 8 and the objective function was 2.44
##
## The root mean square of the residuals (RMSR) is 0.05
## The df corrected root mean square of the residuals is 0.08
##
## The harmonic number of observations is 495 with the empirical chi square 49.94 with prob < 4.2e-08
## The total number of observations was 495 with Likelihood Chi Square = 1194.43 with prob < 1.5e-252
##
## Tucker Lewis Index of factoring reliability = 0.128
## RMSEA index = 0.547 and the 90 % confidence intervals are 0.522 0.574
## BIC = 1144.8
## Fit based upon off diagonal values = 0.99fa(An1Data[,7:13], nfactors = 3)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An1Data[, 7:13], nfactors = 3)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 h2 u2 com
## S 0.75 0.01 0.22 0.62 0.3832 1.2
## M 0.76 -0.15 0.24 0.66 0.3384 1.3
## C 0.77 0.63 -0.16 1.00 -0.0047 2.0
## R 0.86 -0.39 -0.35 1.02 -0.0154 1.8
## F 0.86 0.04 0.05 0.74 0.2564 1.0
## N 0.83 -0.07 0.07 0.69 0.3089 1.0
## V 0.84 -0.01 -0.02 0.70 0.2992 1.0
##
## MR1 MR2 MR3
## SS loadings 4.59 0.58 0.26
## Proportion Var 0.66 0.08 0.04
## Cumulative Var 0.66 0.74 0.78
## Proportion Explained 0.85 0.11 0.05
## Cumulative Proportion 0.85 0.95 1.00
##
## Mean item complexity = 1.3
## Test of the hypothesis that 3 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.34 with Chi Square of 3602.34
## The degrees of freedom for the model are 3 and the objective function was 2.19
##
## The root mean square of the residuals (RMSR) is 0.04
## The df corrected root mean square of the residuals is 0.11
##
## The harmonic number of observations is 495 with the empirical chi square 34.42 with prob < 1.6e-07
## The total number of observations was 495 with Likelihood Chi Square = 1071.46 with prob < 5.7e-232
##
## Tucker Lewis Index of factoring reliability = -1.097
## RMSEA index = 0.848 and the 90 % confidence intervals are 0.807 0.892
## BIC = 1052.85
## Fit based upon off diagonal values = 1fa(An1Data[,7:13], nfactors = 4)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An1Data[, 7:13], nfactors = 4)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 h2 u2 com
## S 0.81 -0.10 0.57 -0.02 1.00 0.0021 1.8
## M 0.75 -0.16 -0.08 0.03 0.59 0.4097 1.1
## C 0.76 0.65 -0.05 -0.09 1.01 -0.0066 2.0
## R 0.81 -0.24 -0.25 0.13 0.80 0.1990 1.4
## F 0.85 0.06 -0.01 0.12 0.74 0.2574 1.0
## N 0.88 -0.18 -0.13 -0.44 1.01 -0.0088 1.6
## V 0.85 0.02 -0.04 0.28 0.80 0.2040 1.2
##
## MR1 MR2 MR3 MR4
## SS loadings 4.67 0.55 0.41 0.31
## Proportion Var 0.67 0.08 0.06 0.04
## Cumulative Var 0.67 0.74 0.80 0.85
## Proportion Explained 0.79 0.09 0.07 0.05
## Cumulative Proportion 0.79 0.88 0.95 1.00
##
## Mean item complexity = 1.5
## Test of the hypothesis that 4 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.34 with Chi Square of 3602.34
## The degrees of freedom for the model are -1 and the objective function was 1.92
##
## The root mean square of the residuals (RMSR) is 0.03
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 495 with the empirical chi square 21.52 with prob < NA
## The total number of observations was 495 with Likelihood Chi Square = 937.6 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 6.534
## Fit based upon off diagonal values = 1An1Data2002 <- subset(An1Data, Sample == 1)
An1Data2012 <- subset(An1Data, Sample == 2)
An1Data2022 <- subset(An1Data, Sample == 3)
An1Mod <- '
GeneralFactor =~ S + M + C + R + F + N + V'
An1InitFit <- cfa(An1Mod, An1Data, std.lv = T)
An1InitFit02 <- cfa(An1Mod, An1Data2002, std.lv = T)
An1InitFit12 <- cfa(An1Mod, An1Data2012, std.lv = T)
An1InitFit22 <- cfa(An1Mod, An1Data2022, std.lv = T)
round(cbind(Overall = fitMeasures(An1InitFit, FITM),
Year2002 = fitMeasures(An1InitFit02, FITM),
Year2012 = fitMeasures(An1InitFit12, FITM),
Year2022 = fitMeasures(An1InitFit22, FITM)), 3)## Overall Year2002 Year2012 Year2022
## chisq 1330.081 531.465 506.757 323.969
## df 14.000 14.000 14.000 14.000
## npar 14.000 14.000 14.000 14.000
## cfi 0.636 0.640 0.623 0.545
## rmsea 0.436 0.448 0.418 0.449
## rmsea.ci.lower 0.416 0.416 0.388 0.407
## rmsea.ci.upper 0.456 0.481 0.450 0.492
## aic 25702.048 9473.261 10496.609 5602.391
## bic 25760.912 9518.270 10542.856 5640.198The data from the first analysis is unworkable across all years or in any particular year.
Analysis 2
set.seed(100)
fa.parallel(An2Data[,1:7])## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.
## Parallel analysis suggests that the number of factors = 5 and the number of components = 1fa.parallel(An2Data[,9:15])## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.
## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully
## Parallel analysis suggests that the number of factors = 5 and the number of components = 1fa(An2Data[,1:7], nfactors = 1)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 1)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 h2 u2 com
## St1 0.73 0.54 0.46 1
## Mt1 0.69 0.48 0.52 1
## Ct1 0.63 0.40 0.60 1
## Rt1 0.80 0.64 0.36 1
## Ft1 0.87 0.76 0.24 1
## Nt1 0.81 0.66 0.34 1
## Vt1 0.83 0.69 0.31 1
##
## MR1
## SS loadings 4.18
## Proportion Var 0.60
##
## Mean item complexity = 1
## Test of the hypothesis that 1 factor is sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are 14 and the objective function was 3.07
##
## The root mean square of the residuals (RMSR) is 0.07
## The df corrected root mean square of the residuals is 0.09
##
## The harmonic number of observations is 314 with the empirical chi square 68.69 with prob < 3.3e-09
## The total number of observations was 468 with Likelihood Chi Square = 1422.31 with prob < 2.6e-295
##
## Tucker Lewis Index of factoring reliability = 0.357
## RMSEA index = 0.464 and the 90 % confidence intervals are 0.444 0.485
## BIC = 1336.23
## Fit based upon off diagonal values = 0.99fa(An2Data[,1:7], nfactors = 2)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed
## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 2)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 h2 u2 com
## St1 0.72 -0.05 0.52 0.4769 1.0
## Mt1 0.70 -0.26 0.55 0.4474 1.3
## Ct1 0.73 0.69 1.01 -0.0081 2.0
## Rt1 0.80 -0.25 0.71 0.2911 1.2
## Ft1 0.87 -0.02 0.75 0.2513 1.0
## Nt1 0.81 -0.03 0.65 0.3504 1.0
## Vt1 0.82 -0.05 0.68 0.3183 1.0
##
## MR1 MR2
## SS loadings 4.25 0.62
## Proportion Var 0.61 0.09
## Cumulative Var 0.61 0.70
## Proportion Explained 0.87 0.13
## Cumulative Proportion 0.87 1.00
##
## Mean item complexity = 1.2
## Test of the hypothesis that 2 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are 8 and the objective function was 2.77
##
## The root mean square of the residuals (RMSR) is 0.06
## The df corrected root mean square of the residuals is 0.09
##
## The harmonic number of observations is 314 with the empirical chi square 41.09 with prob < 2e-06
## The total number of observations was 468 with Likelihood Chi Square = 1279.79 with prob < 5.5e-271
##
## Tucker Lewis Index of factoring reliability = -0.018
## RMSEA index = 0.583 and the 90 % confidence intervals are 0.557 0.611
## BIC = 1230.6
## Fit based upon off diagonal values = 0.99fa(An2Data[,1:7], nfactors = 3)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 3)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 h2 u2 com
## St1 0.80 -0.12 -0.59 1.01 -0.0075 1.9
## Mt1 0.69 -0.23 0.15 0.55 0.4515 1.3
## Ct1 0.72 0.71 0.05 1.02 -0.0152 2.0
## Rt1 0.81 -0.25 0.27 0.79 0.2071 1.4
## Ft1 0.86 0.00 0.09 0.74 0.2574 1.0
## Nt1 0.80 -0.02 0.00 0.64 0.3628 1.0
## Vt1 0.82 -0.04 0.05 0.67 0.3301 1.0
##
## MR1 MR2 MR3
## SS loadings 4.32 0.63 0.46
## Proportion Var 0.62 0.09 0.07
## Cumulative Var 0.62 0.71 0.77
## Proportion Explained 0.80 0.12 0.08
## Cumulative Proportion 0.80 0.92 1.00
##
## Mean item complexity = 1.4
## Test of the hypothesis that 3 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are 3 and the objective function was 2.55
##
## The root mean square of the residuals (RMSR) is 0.05
## The df corrected root mean square of the residuals is 0.12
##
## The harmonic number of observations is 314 with the empirical chi square 28.65 with prob < 2.6e-06
## The total number of observations was 468 with Likelihood Chi Square = 1177.45 with prob < 5.7e-255
##
## Tucker Lewis Index of factoring reliability = -1.511
## RMSEA index = 0.915 and the 90 % confidence intervals are 0.872 0.96
## BIC = 1159
## Fit based upon off diagonal values = 0.99fa(An2Data[,1:7], nfactors = 4)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 4)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 h2 u2 com
## St1 0.79 0.10 -0.61 0.04 1.01 -0.0112 1.9
## Mt1 0.76 -0.47 0.15 0.43 1.01 -0.0052 2.4
## Ct1 0.70 0.66 0.27 0.12 1.01 -0.0128 2.3
## Rt1 0.84 -0.25 0.16 -0.47 1.03 -0.0261 1.9
## Ft1 0.85 0.01 0.07 0.06 0.73 0.2748 1.0
## Nt1 0.79 0.03 -0.03 -0.06 0.63 0.3696 1.0
## Vt1 0.81 0.00 0.01 -0.05 0.65 0.3469 1.0
##
## MR1 MR2 MR3 MR4
## SS loadings 4.40 0.73 0.50 0.43
## Proportion Var 0.63 0.10 0.07 0.06
## Cumulative Var 0.63 0.73 0.80 0.87
## Proportion Explained 0.73 0.12 0.08 0.07
## Cumulative Proportion 0.73 0.85 0.93 1.00
##
## Mean item complexity = 1.7
## Test of the hypothesis that 4 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are -1 and the objective function was 2.1
##
## The root mean square of the residuals (RMSR) is 0.03
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 314 with the empirical chi square 11.08 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 968.32 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 7.227
## Fit based upon off diagonal values = 1fa(An2Data[,1:7], nfactors = 5)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 5)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 MR5 h2 u2 com
## St1 0.79 0.10 -0.59 0.13 0.10 1.00 -0.00333 2.0
## Mt1 0.75 -0.48 0.18 0.39 -0.13 1.00 -0.00269 2.5
## Ct1 0.70 0.65 0.28 0.11 -0.02 1.01 -0.00720 2.4
## Rt1 0.84 -0.25 0.14 -0.45 0.16 1.01 -0.01252 1.9
## Ft1 0.84 0.00 0.09 0.09 0.09 0.73 0.26816 1.1
## Nt1 0.85 0.05 -0.11 -0.22 -0.46 1.00 -0.00066 1.7
## Vt1 0.81 0.00 0.04 0.01 0.26 0.73 0.27123 1.2
##
## MR1 MR2 MR3 MR4 MR5
## SS loadings 4.47 0.73 0.50 0.44 0.34
## Proportion Var 0.64 0.10 0.07 0.06 0.05
## Cumulative Var 0.64 0.74 0.81 0.88 0.93
## Proportion Explained 0.69 0.11 0.08 0.07 0.05
## Cumulative Proportion 0.69 0.80 0.88 0.95 1.00
##
## Mean item complexity = 1.8
## Test of the hypothesis that 5 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are -4 and the objective function was 1.62
##
## The root mean square of the residuals (RMSR) is 0.02
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 314 with the empirical chi square 4.1 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 747.48 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 2.209
## Fit based upon off diagonal values = 1fa(An2Data[,1:7], nfactors = 6)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 1:7], nfactors = 6)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 MR5 MR6 h2 u2 com
## St1 0.78 0.09 -0.57 0.21 -0.08 -0.07 0.99 0.011 2.1
## Mt1 0.74 -0.44 0.18 0.28 0.26 0.19 0.96 0.040 2.6
## Ct1 0.69 0.66 0.25 0.02 0.08 0.08 1.00 0.005 2.3
## Rt1 0.83 -0.27 0.11 -0.40 -0.19 -0.14 1.00 0.005 1.9
## Ft1 0.88 0.01 0.17 0.17 0.10 -0.38 1.00 0.005 1.6
## Nt1 0.84 0.04 -0.21 -0.34 0.31 0.15 1.00 0.005 1.9
## Vt1 0.86 -0.01 0.09 0.10 -0.45 0.21 1.00 0.005 1.7
##
## MR1 MR2 MR3 MR4 MR5 MR6
## SS loadings 4.55 0.72 0.51 0.44 0.42 0.28
## Proportion Var 0.65 0.10 0.07 0.06 0.06 0.04
## Cumulative Var 0.65 0.75 0.83 0.89 0.95 0.99
## Proportion Explained 0.66 0.10 0.07 0.06 0.06 0.04
## Cumulative Proportion 0.66 0.76 0.83 0.90 0.96 1.00
##
## Mean item complexity = 2
## Test of the hypothesis that 6 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.13 with Chi Square of 3308.84
## The degrees of freedom for the model are -6 and the objective function was 0
##
## The root mean square of the residuals (RMSR) is 0
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 314 with the empirical chi square 0 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 0 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 1.006
## Fit based upon off diagonal values = 1
## Measures of factor score adequacy
## MR1 MR2 MR3 MR4 MR5 MR6
## Correlation of (regression) scores with factors 1 0.99 0.99 0.99 0.99 0.99
## Multiple R square of scores with factors 1 0.98 0.98 0.98 0.98 0.97
## Minimum correlation of possible factor scores 1 0.97 0.96 0.96 0.96 0.94fa(An2Data[,9:15], nfactors = 1)## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 1)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 h2 u2 com
## St2 0.70 0.49 0.51 1
## Mt2 0.73 0.53 0.47 1
## Ct2 0.70 0.49 0.51 1
## Rt2 0.75 0.56 0.44 1
## Ft2 0.89 0.80 0.20 1
## Nt2 0.85 0.71 0.29 1
## Vt2 0.85 0.72 0.28 1
##
## MR1
## SS loadings 4.30
## Proportion Var 0.61
##
## Mean item complexity = 1
## Test of the hypothesis that 1 factor is sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are 14 and the objective function was 2.97
##
## The root mean square of the residuals (RMSR) is 0.07
## The df corrected root mean square of the residuals is 0.08
##
## The harmonic number of observations is 306 with the empirical chi square 59.06 with prob < 1.7e-07
## The total number of observations was 468 with Likelihood Chi Square = 1377.45 with prob < 1.2e-285
##
## Tucker Lewis Index of factoring reliability = 0.389
## RMSEA index = 0.456 and the 90 % confidence intervals are 0.436 0.477
## BIC = 1291.37
## Fit based upon off diagonal values = 0.99fa(An2Data[,9:15], nfactors = 2)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed
## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 2)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 h2 u2 com
## St2 0.69 0.05 0.48 0.5199 1.0
## Mt2 0.72 0.20 0.56 0.4388 1.1
## Ct2 0.78 -0.62 1.00 -0.0036 1.9
## Rt2 0.75 0.26 0.63 0.3683 1.2
## Ft2 0.88 0.00 0.78 0.2197 1.0
## Nt2 0.84 0.15 0.73 0.2681 1.1
## Vt2 0.84 -0.01 0.71 0.2944 1.0
##
## MR1 MR2
## SS loadings 4.38 0.52
## Proportion Var 0.63 0.07
## Cumulative Var 0.63 0.70
## Proportion Explained 0.89 0.11
## Cumulative Proportion 0.89 1.00
##
## Mean item complexity = 1.2
## Test of the hypothesis that 2 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are 8 and the objective function was 2.69
##
## The root mean square of the residuals (RMSR) is 0.05
## The df corrected root mean square of the residuals is 0.09
##
## The harmonic number of observations is 306 with the empirical chi square 38.56 with prob < 5.9e-06
## The total number of observations was 468 with Likelihood Chi Square = 1244.09 with prob < 2.9e-263
##
## Tucker Lewis Index of factoring reliability = 0.029
## RMSEA index = 0.575 and the 90 % confidence intervals are 0.548 0.602
## BIC = 1194.9
## Fit based upon off diagonal values = 0.99fa(An2Data[,9:15], nfactors = 3)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 3)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 h2 u2 com
## St2 0.77 0.30 -0.56 1.00 -0.0026 2.2
## Mt2 0.71 0.11 0.15 0.54 0.4560 1.1
## Ct2 0.78 -0.62 -0.13 1.01 -0.0069 2.0
## Rt2 0.77 0.18 0.37 0.76 0.2445 1.6
## Ft2 0.87 -0.03 0.04 0.77 0.2339 1.0
## Nt2 0.83 0.13 0.06 0.71 0.2869 1.1
## Vt2 0.83 -0.05 0.07 0.70 0.2984 1.0
##
## MR1 MR2 MR3
## SS loadings 4.45 0.54 0.50
## Proportion Var 0.64 0.08 0.07
## Cumulative Var 0.64 0.71 0.78
## Proportion Explained 0.81 0.10 0.09
## Cumulative Proportion 0.81 0.91 1.00
##
## Mean item complexity = 1.4
## Test of the hypothesis that 3 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are 3 and the objective function was 2.44
##
## The root mean square of the residuals (RMSR) is 0.04
## The df corrected root mean square of the residuals is 0.12
##
## The harmonic number of observations is 306 with the empirical chi square 25.94 with prob < 9.8e-06
## The total number of observations was 468 with Likelihood Chi Square = 1125.57 with prob < 1e-243
##
## Tucker Lewis Index of factoring reliability = -1.354
## RMSEA index = 0.894 and the 90 % confidence intervals are 0.852 0.939
## BIC = 1107.13
## Fit based upon off diagonal values = 0.99fa(An2Data[,9:15], nfactors = 4)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 4)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 h2 u2 com
## St2 0.76 0.27 0.57 -0.16 1.01 -0.0069 2.2
## Mt2 0.79 -0.34 0.10 0.52 1.01 -0.0077 2.2
## Ct2 0.76 0.52 -0.39 0.08 1.01 -0.0114 2.3
## Rt2 0.80 -0.42 -0.20 -0.40 1.02 -0.0189 2.2
## Ft2 0.86 0.03 -0.03 0.04 0.75 0.2498 1.0
## Nt2 0.82 -0.06 0.05 -0.10 0.69 0.3070 1.0
## Vt2 0.82 0.02 -0.07 0.03 0.68 0.3165 1.0
##
## MR1 MR2 MR3 MR4
## SS loadings 4.53 0.64 0.53 0.47
## Proportion Var 0.65 0.09 0.08 0.07
## Cumulative Var 0.65 0.74 0.81 0.88
## Proportion Explained 0.73 0.10 0.09 0.08
## Cumulative Proportion 0.73 0.84 0.92 1.00
##
## Mean item complexity = 1.7
## Test of the hypothesis that 4 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are -1 and the objective function was 1.89
##
## The root mean square of the residuals (RMSR) is 0.02
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 306 with the empirical chi square 7.37 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 873.12 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 6.507
## Fit based upon off diagonal values = 1fa(An2Data[,9:15], nfactors = 5)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Warning in fa.stats(r = r, f = f, phi = phi, n.obs = n.obs, np.obs = np.obs, :
## The estimated weights for the factor scores are probably incorrect. Try a
## different factor score estimation method.## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : An
## ultra-Heywood case was detected. Examine the results carefully## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 5)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 MR5 h2 u2 com
## St2 0.76 0.26 0.58 -0.08 0.12 1.00 -2.1e-03 2.2
## Mt2 0.78 -0.31 0.02 0.54 -0.05 1.00 -4.5e-03 2.2
## Ct2 0.76 0.54 -0.38 0.02 -0.07 1.01 -9.1e-03 2.3
## Rt2 0.80 -0.42 -0.18 -0.37 0.16 1.01 -9.0e-03 2.2
## Ft2 0.86 0.05 -0.05 0.06 0.11 0.76 2.4e-01 1.1
## Nt2 0.88 -0.11 0.11 -0.20 -0.41 1.00 2.3e-05 1.6
## Vt2 0.83 0.04 -0.10 0.05 0.19 0.73 2.7e-01 1.1
##
## MR1 MR2 MR3 MR4 MR5
## SS loadings 4.58 0.64 0.54 0.49 0.26
## Proportion Var 0.65 0.09 0.08 0.07 0.04
## Cumulative Var 0.65 0.75 0.82 0.89 0.93
## Proportion Explained 0.70 0.10 0.08 0.07 0.04
## Cumulative Proportion 0.70 0.80 0.88 0.96 1.00
##
## Mean item complexity = 1.8
## Test of the hypothesis that 5 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are -4 and the objective function was 1.51
##
## The root mean square of the residuals (RMSR) is 0.02
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 306 with the empirical chi square 3.52 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 694.11 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 2.101
## Fit based upon off diagonal values = 1fa(An2Data[,9:15], nfactors = 6)## Loading required namespace: GPArotation## Warning in fac(r = r, nfactors = nfactors, n.obs = n.obs, rotate = rotate, : I
## am sorry, to do these rotations requires the GPArotation package to be
## installed## Factor Analysis using method = minres
## Call: fa(r = An2Data[, 9:15], nfactors = 6)
## Standardized loadings (pattern matrix) based upon correlation matrix
## MR1 MR2 MR3 MR4 MR5 MR6 h2 u2 com
## St2 0.75 0.19 0.59 -0.03 -0.15 -0.04 0.97 0.032 2.2
## Mt2 0.77 -0.29 -0.03 0.53 0.11 0.08 0.98 0.019 2.2
## Ct2 0.75 0.56 -0.29 -0.07 0.16 0.07 1.00 0.005 2.3
## Rt2 0.79 -0.42 -0.19 -0.37 -0.09 -0.10 1.00 0.005 2.2
## Ft2 0.90 0.07 -0.05 0.09 0.09 -0.41 1.00 0.005 1.5
## Nt2 0.87 -0.15 0.17 -0.22 0.30 0.24 1.00 0.005 1.7
## Vt2 0.87 0.07 -0.18 0.08 -0.41 0.17 1.00 0.005 1.7
##
## MR1 MR2 MR3 MR4 MR5 MR6
## SS loadings 4.66 0.64 0.53 0.48 0.34 0.28
## Proportion Var 0.67 0.09 0.08 0.07 0.05 0.04
## Cumulative Var 0.67 0.76 0.83 0.90 0.95 0.99
## Proportion Explained 0.67 0.09 0.08 0.07 0.05 0.04
## Cumulative Proportion 0.67 0.77 0.84 0.91 0.96 1.00
##
## Mean item complexity = 2
## Test of the hypothesis that 6 factors are sufficient.
##
## The degrees of freedom for the null model are 21 and the objective function was 7.27 with Chi Square of 3373.42
## The degrees of freedom for the model are -6 and the objective function was 0
##
## The root mean square of the residuals (RMSR) is 0
## The df corrected root mean square of the residuals is NA
##
## The harmonic number of observations is 306 with the empirical chi square 0 with prob < NA
## The total number of observations was 468 with Likelihood Chi Square = 0 with prob < NA
##
## Tucker Lewis Index of factoring reliability = 1.006
## Fit based upon off diagonal values = 1
## Measures of factor score adequacy
## MR1 MR2 MR3 MR4 MR5 MR6
## Correlation of (regression) scores with factors 1 0.99 0.98 0.99 0.99 0.99
## Multiple R square of scores with factors 1 0.99 0.97 0.98 0.98 0.98
## Minimum correlation of possible factor scores 1 0.97 0.94 0.95 0.96 0.96Long1 <- '
g1 =~ St1 + Mt1 + Ct1 + Rt1 + Ft1 + Nt1 + Vt1'
Long2 <- '
g2 =~ St2 + Mt2 + Ct2 + Rt2 + Ft2 + Nt2 + Vt2'
LongConfigural <- '
g1 =~ NA*St1 + lambda1*St1 + Mt1 + Ct1 + Rt1 + Ft1 + Nt1 + Vt1
g2 =~ NA*St2 + lambda1*St2 + Mt2 + Ct2 + Rt2 + Ft2 + Nt2 + Vt2
St1 ~ i1*1
Mt1 ~ 1
Ct1 ~ 1
Rt1 ~ 1
Ft1 ~ 1
Nt1 ~ 1
Vt1 ~ 1
St2 ~ i1*1
Mt2 ~ 1
Ct2 ~ 1
Rt2 ~ 1
Ft2 ~ 1
Nt2 ~ 1
Vt2 ~ 1
St1 ~~ St1
Mt1 ~~ Mt1
Ct1 ~~ Ct1
Rt1 ~~ Rt1
Ft1 ~~ Ft1
Nt1 ~~ Nt1
Vt1 ~~ Vt1
St2 ~~ St2
Mt2 ~~ Mt2
Ct2 ~~ Ct2
Rt2 ~~ Rt2
Ft2 ~~ Ft2
Nt2 ~~ Nt2
Vt2 ~~ Vt2
g1 ~ 0*1
g2 ~ 1
g1 ~~ 1*g1
g2 ~~ g2
g1 ~~ g2'
Long1 <- cfa(Long1, An2Data)
Long2 <- cfa(Long2, An2Data)
ConfiguralFit <- cfa(LongConfigural,
An2Data)
round(cbind(Overall = fitMeasures(Long1, FITM),
Time1 = fitMeasures(Long2, FITM),
Time2 = fitMeasures(ConfiguralFit, FITM)), 3)## Overall Time1 Time2
## chisq 910.827 913.358 2221.694
## df 14.000 14.000 76.000
## npar 14.000 14.000 43.000
## cfi 0.576 0.591 0.514
## rmsea 0.460 0.464 0.349
## rmsea.ci.lower 0.435 0.438 0.336
## rmsea.ci.upper 0.485 0.489 0.361
## aic 15849.026 15939.690 24076.798
## bic 15901.019 15991.496 24225.008References
Breit, M., Scherrer, V., Blickle, J., & Preckel, F. (2023). Students’ intelligence test results after six and sixteen months of irregular schooling due to the COVID-19 pandemic. PLOS ONE, 18(3), e0281779. https://doi.org/10.1371/journal.pone.0281779
Breit, M., Scherrer, V., & Preckel, F. (2021). Temporal stability of specific ability scores and intelligence profiles in high ability students. Intelligence, 86, 101538. https://doi.org/10.1016/j.intell.2021.101538
Breit, M., Scherrer, V., & Preckel, F. (2021). Temporal stability and change in manifest intelligence scores: Four complementary analytic approaches. MethodsX, 9, 101613. https://doi.org/10.1016/j.mex.2021.101613
sessionInfo()## R version 4.2.2 (2022-10-31 ucrt)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 19045)
##
## Matrix products: default
##
## locale:
## [1] LC_COLLATE=English_United States.utf8
## [2] LC_CTYPE=English_United States.utf8
## [3] LC_MONETARY=English_United States.utf8
## [4] LC_NUMERIC=C
## [5] LC_TIME=English_United States.utf8
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] DT_0.27 psych_2.2.9 lavaan_0.6-14 pacman_0.5.1
##
## loaded via a namespace (and not attached):
## [1] rstudioapi_0.14 magrittr_2.0.3 knitr_1.42 MASS_7.3-58.1
## [5] mnormt_2.1.1 pbivnorm_0.6.0 lattice_0.20-45 R6_2.5.1
## [9] rlang_1.0.6 quadprog_1.5-8 fastmap_1.1.0 highr_0.10
## [13] tools_4.2.2 parallel_4.2.2 grid_4.2.2 nlme_3.1-160
## [17] xfun_0.37 cli_3.6.0 jquerylib_0.1.4 ellipsis_0.3.2
## [21] crosstalk_1.2.0 htmltools_0.5.4 yaml_2.3.7 digest_0.6.31
## [25] htmlwidgets_1.6.1 sass_0.4.5 cachem_1.0.6 evaluate_0.20
## [29] rmarkdown_2.20 compiler_4.2.2 bslib_0.4.2 stats4_4.2.2
## [33] jsonlite_1.8.4