Statistical analysis was carried out in R, version 3.4.3, using packages lavaan, lavaan.survey, semPlot, dplyr, psych, ICC, Amelia, BaylorEdPsych, haven, survey, semTools, knitr, kableExtra, and gplots.

Missing data

## [1] 6196

Visualization of missing data

For full dataset

Missing data are marked in red. Ordered by the ammount of missing data.

After discarding subjects with more than a third missing item values

As can be seen here, after discarding blank rows, there is very little missing data

Sample characteristics

N

## [1] 6196

Gender

## 
##    1    2 
## 3031 3165

Gender == 1 (Girls) represents 48.92% of the sample.

Age in years

Mean

## [1] 11.82297

SD

## [1] 0.4052831

Percentage of missing data

There is very little missing data. Regardless of what imputation procedure is applied, it won’t have much effect.

## [1] "0.581%"

Descriptive statistics

Practically all the items are right skewed. Children choose answer option 4 (“Strongly disagree”) infrequently - usually some ~5-6%.

vars n mean sd skew kurtosis se
C6_B1A 1 6196 2.46 0.73 -0.24 -0.35 0.01
C6_B1B 2 6196 2.03 0.83 0.24 -0.85 0.01
C6_B1C 3 6196 1.66 0.79 0.94 0.02 0.01
C6_B1D 4 6196 2.05 0.83 0.29 -0.69 0.01
C6_B1E 5 6196 2.39 0.78 -0.10 -0.49 0.01
C6_B1F 6 6196 2.09 0.85 0.34 -0.61 0.01
C6_B1G 7 6196 1.94 0.81 0.40 -0.66 0.01
C6_B1H 8 6196 1.83 0.92 0.81 -0.39 0.01
C6_B1I 9 6196 2.30 0.80 0.04 -0.56 0.01
C6_B1J 10 6196 2.10 0.81 0.22 -0.67 0.01
C6_B1K 11 6196 1.79 0.78 0.60 -0.47 0.01
C6_B1L 12 6196 1.51 0.74 1.34 1.02 0.01
C6_B1M 13 6196 1.80 0.76 0.55 -0.45 0.01
C6_B1N 14 6196 1.86 0.75 0.40 -0.66 0.01
C6_B1O 15 6196 1.71 0.76 0.72 -0.32 0.01
C6_B2A 16 6196 2.20 0.92 0.31 -0.77 0.01
C6_B2B 17 6196 1.95 0.71 0.41 0.07 0.01
C6_B2C 18 6196 2.19 0.92 0.32 -0.75 0.01
C6_B2D 19 6196 2.20 0.90 0.32 -0.69 0.01
C6_B2E 20 6196 2.16 1.03 0.41 -1.02 0.01
C6_B2F 21 6196 1.83 0.67 0.50 0.33 0.01
C6_B2G 22 6196 2.22 0.80 0.30 -0.32 0.01
C6_B2H 23 6196 2.09 1.01 0.51 -0.86 0.01
C6_B2I 24 6196 2.01 0.84 0.54 -0.31 0.01
C6_B2J 25 6196 1.91 0.85 0.66 -0.25 0.01
C6_B2K 26 6196 2.33 1.01 0.17 -1.09 0.01
C6_B2L 27 6196 2.24 1.08 0.30 -1.21 0.01
C6_B2M 28 6196 1.51 0.75 1.44 1.47 0.01
C6_B2N 29 6196 2.09 1.07 0.51 -1.04 0.01
C6_B2O 30 6196 2.04 0.91 0.50 -0.63 0.01
C6_B2P 31 6196 2.07 0.74 0.36 -0.05 0.01
C6_B2Q 32 6196 1.82 0.85 0.83 -0.01 0.01
C6_B2R 33 6196 2.18 0.82 0.33 -0.38 0.01
C6_B2S 34 6196 2.26 0.87 0.23 -0.62 0.01

Polychoric correlation heatmap

As obvious from the heatmap, the items loaded by domain-general factors of Motivation (INSMOT), Learning Strategies (EFFPER), and Self-Belief (SELFEF, CEXP) tend to correlate stronger among each other than with subject-specific factors. Correlations between domain-specific (Math, Language; lower right quadrant) are as expected. strong correlations between Math and Language constructs, respectively, but weak cross-relationships.

Item C6_B2D (“Při hodinách českého jazyka si nevím rady”) behaves awkwardly (see the white-blue “cross”). No or negative correlations with math items, which is okay, but very low correlations with other “language” items. Please note that this item was the only one inversely scaled (which probably wasn’t the best idea as it may have induced response bias making the instroment seem to work better as it really would).

Polychoric correlation matrix

Matrix can be scrolled in every direction

C6_B1A C6_B1B C6_B1C C6_B1D C6_B1E C6_B1F C6_B1G C6_B1H C6_B1I C6_B1J C6_B1K C6_B1L C6_B1M C6_B1N C6_B1O C6_B2A C6_B2B C6_B2C C6_B2D C6_B2E C6_B2F C6_B2G C6_B2H C6_B2I C6_B2J C6_B2K C6_B2L C6_B2M C6_B2N C6_B2O C6_B2P C6_B2Q C6_B2R C6_B2S
C6_B1A 1
C6_B1B 0.54 1
C6_B1C 0.36 0.51 1
C6_B1D 0.3 0.45 0.52 1
C6_B1E 0.68 0.58 0.39 0.38 1
C6_B1F 0.4 0.49 0.4 0.34 0.45 1
C6_B1G 0.43 0.54 0.52 0.52 0.5 0.44 1
C6_B1H 0.22 0.27 0.48 0.32 0.28 0.27 0.28 1
C6_B1I 0.42 0.42 0.35 0.37 0.46 0.47 0.42 0.29 1
C6_B1J 0.51 0.49 0.38 0.32 0.54 0.48 0.42 0.29 0.46 1
C6_B1K 0.34 0.5 0.55 0.58 0.41 0.39 0.58 0.33 0.41 0.44 1
C6_B1L 0.36 0.44 0.72 0.47 0.39 0.4 0.51 0.63 0.37 0.43 0.58 1
C6_B1M 0.43 0.59 0.47 0.48 0.5 0.53 0.55 0.33 0.48 0.52 0.56 0.53 1
C6_B1N 0.51 0.53 0.44 0.39 0.57 0.48 0.51 0.32 0.45 0.57 0.52 0.49 0.64 1
C6_B1O 0.32 0.4 0.48 0.58 0.36 0.34 0.51 0.31 0.35 0.35 0.59 0.51 0.5 0.48 1
C6_B2A 0.28 0.23 0.22 0.27 0.3 0.21 0.29 0.18 0.28 0.25 0.27 0.26 0.27 0.26 0.3 1
C6_B2B 0.51 0.49 0.36 0.3 0.51 0.39 0.42 0.26 0.39 0.49 0.39 0.39 0.47 0.53 0.37 0.33 1
C6_B2C 0.27 0.21 0.24 0.18 0.29 0.22 0.22 0.31 0.25 0.29 0.24 0.28 0.24 0.26 0.24 0.3 0.32 1
C6_B2D 0.24 0.24 0.18 0.13 0.17 0.17 0.2 0 0.1 0.21 0.18 0.17 0.21 0.24 0.16 -0.1 0.21 0 1
C6_B2E 0.24 0.24 0.23 0.23 0.22 0.21 0.25 0.05 0.16 0.2 0.26 0.24 0.26 0.22 0.22 0.18 0.26 0.12 0.14 1
C6_B2F 0.48 0.47 0.39 0.32 0.5 0.42 0.42 0.26 0.39 0.51 0.43 0.41 0.49 0.53 0.39 0.31 0.7 0.32 0.21 0.28 1
C6_B2G 0.39 0.38 0.32 0.27 0.36 0.33 0.35 0.17 0.28 0.38 0.35 0.33 0.38 0.42 0.29 0.13 0.48 0.19 0.55 0.28 0.47 1
C6_B2H 0.32 0.27 0.25 0.28 0.36 0.23 0.29 0.15 0.29 0.29 0.28 0.24 0.27 0.31 0.29 0.55 0.35 0.25 -0.1 0.14 0.35 0.09 1
C6_B2I 0.28 0.31 0.37 0.32 0.31 0.3 0.33 0.36 0.34 0.33 0.37 0.4 0.38 0.35 0.39 0.34 0.36 0.68 0.05 0.19 0.39 0.26 0.36 1
C6_B2J 0.42 0.32 0.26 0.2 0.43 0.31 0.32 0.19 0.29 0.43 0.28 0.27 0.32 0.38 0.26 0.38 0.45 0.27 0.03 0.13 0.46 0.19 0.64 0.31 1
C6_B2K 0.23 0.23 0.23 0.24 0.21 0.2 0.25 0.03 0.14 0.16 0.26 0.22 0.25 0.21 0.22 0.15 0.25 0.1 0.14 0.81 0.26 0.26 0.11 0.17 0.12 1
C6_B2L 0.31 0.23 0.19 0.2 0.33 0.19 0.24 0.14 0.25 0.27 0.22 0.19 0.22 0.26 0.23 0.51 0.32 0.23 -0.12 0.09 0.3 0.04 0.86 0.3 0.69 0.1 1
C6_B2M 0.17 0.16 0.23 0.18 0.19 0.17 0.19 0.32 0.17 0.22 0.25 0.32 0.24 0.22 0.25 0.24 0.21 0.54 0 0.14 0.24 0.15 0.2 0.53 0.18 0.12 0.18 1
C6_B2N 0.2 0.2 0.21 0.21 0.18 0.19 0.23 0.07 0.15 0.16 0.25 0.23 0.25 0.2 0.21 0.28 0.23 0.13 0.12 0.73 0.25 0.24 0.1 0.2 0.09 0.74 0.06 0.21 1
C6_B2O 0.4 0.3 0.24 0.19 0.42 0.27 0.29 0.2 0.31 0.4 0.26 0.25 0.29 0.36 0.23 0.45 0.42 0.27 -0.02 0.08 0.42 0.16 0.7 0.33 0.76 0.06 0.73 0.21 0.06 1
C6_B2P 0.5 0.48 0.36 0.3 0.5 0.43 0.42 0.23 0.37 0.56 0.38 0.38 0.45 0.5 0.35 0.26 0.61 0.29 0.26 0.26 0.66 0.48 0.31 0.36 0.53 0.27 0.31 0.2 0.23 0.46 1
C6_B2Q 0.28 0.25 0.29 0.29 0.33 0.21 0.31 0.21 0.27 0.28 0.3 0.31 0.28 0.29 0.33 0.47 0.32 0.25 -0.05 0.16 0.31 0.13 0.7 0.36 0.52 0.13 0.69 0.29 0.15 0.58 0.33 1
C6_B2R 0.34 0.35 0.3 0.23 0.3 0.33 0.33 0.13 0.26 0.39 0.32 0.31 0.34 0.39 0.26 0.08 0.42 0.17 0.53 0.28 0.45 0.71 0.03 0.22 0.19 0.27 -0.01 0.13 0.24 0.15 0.54 0.1 1
C6_B2S 0.32 0.34 0.31 0.3 0.34 0.32 0.3 0.32 0.36 0.34 0.34 0.34 0.39 0.36 0.32 0.31 0.4 0.57 0.09 0.18 0.4 0.32 0.27 0.66 0.25 0.18 0.22 0.47 0.21 0.28 0.38 0.29 0.3 1

Mean item correlation

Mean inter-item polychoric correlation

## [1] 0.3220143

CFA model

10-factor structure, all factors intercorrelated.

There was a need to include an error covariance between items B2L and B2H; “Matematika je pro mě jedním z nejlepších předmětů” and “Nechtěl/a bych nechat matematiky, protože mě matematika baví.” Superficially, the items’ content validity is pretty much alike, but they load on different factors (INTMAT, SCMATH). This error covariance had such a great impact on model fit, that without it, the model refused to converge under any reasonable estimator.

Some very rough guide to model fit interpretation:

The model was estimated using the Weighted Least Squares Means- and Variance-adjusted fit function while explicitly modeling the ordered nature of the indicators. The given type of estimator (1) is robust with respect to the assumption of normal distribution of errors (especially kurtosis; not likely in Likert scales), (2) induces less bias in parameter estimation and model fit test of misspecified models, and (3) the proportion of Type I errors in assessing correctly specified models with the given data is way more similar to the apriori defined nominal α value, as compared to, e.g., the method of maximum likelihood (Beauducel, Herzberg, 2009).

The model failed the model test. That means, that we can reject the hypothesis of exact fit (which is not very surprising). CFI and TLI are below the cutpoint of .95 for good approximate fit, RMSEA seems quite good - especially its upped bound CI does not cross .05. Given SRMR, there does not seem to be much global absolute misfit. However, significant chi^2 indicates beyond-chance deviations of the data from the theoretized structure. Further detailed model diagnostics are needed.

The model applies the sampling weights (vaha6) and accounts for the two-level hierarchical structure of the data (children nested within classes nested within schools).

Model test and approximate fit indices

##          chisq.scaled             df.scaled         pvalue.scaled 
##               618.812                80.000                 0.000 
##            cfi.scaled            tli.scaled          rmsea.scaled 
##                 0.935                 0.933                 0.033 
## rmsea.ci.lower.scaled rmsea.ci.upper.scaled                  srmr 
##                 0.032                 0.034                 0.038 
##                  pnfi                   bic 
##                 0.801            435531.886

Power analysis

For approximate fit (based on the RMSEA distribution). Statistical power for the detection of a likely misspecified model (RMSEA > .08).

## [1] 1

Given the model and the sample size, there is almost certainty that a badly fitting model would be flagged by RMSEA.

Parameter estimates

The following parts of the output is of interest: “Latent variables” shows factor loadings (all of them significant); Standardized estimates can be found in collumn “Std.all”. “Covariances” show the correlations between the factors. Standardized estimates are to be found in collumn “Std.all”. For the given target interpretation, Intercepts, Thresholds, Intercepts (…) are likely of secondary interest.

## lavaan 0.6-3 ended normally after 114 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                        148
## 
##   Number of observations                          6196
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                    6349.546     618.812
##   Degrees of freedom                               481          80
##   P-value (Chi-square)                           0.000       0.000
##   Scaling correction factor                                 10.261
##     for the mean and variance adjusted correction (MLMV)
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   INSMOT =~                                                             
##     C6_B1C            1.000                               0.597    0.755
##     C6_B1H            0.890    0.029   30.365    0.000    0.531    0.585
##     C6_B1L            1.013    0.022   46.920    0.000    0.605    0.805
##   EFFPER =~                                                             
##     C6_B1D            1.000                               0.553    0.667
##     C6_B1G            1.035    0.024   42.745    0.000    0.573    0.700
##     C6_B1K            1.038    0.023   46.045    0.000    0.574    0.737
##     C6_B1O            0.907    0.023   39.924    0.000    0.502    0.660
##   SELFEF =~                                                             
##     C6_B1A            1.000                               0.468    0.645
##     C6_B1E            1.162    0.026   44.355    0.000    0.544    0.702
##     C6_B1J            1.184    0.030   40.065    0.000    0.555    0.683
##     C6_B1N            1.143    0.030   37.901    0.000    0.535    0.707
##   CEXP =~                                                               
##     C6_B1B            1.000                               0.585    0.701
##     C6_B1F            0.873    0.023   37.319    0.000    0.511    0.590
##     C6_B1I            0.810    0.020   40.195    0.000    0.474    0.587
##     C6_B1M            0.934    0.021   44.944    0.000    0.546    0.714
##   INTREA =~                                                             
##     C6_B2E            1.000                               0.879    0.850
##     C6_B2K            0.990    0.017   59.383    0.000    0.870    0.860
##     C6_B2N            0.925    0.017   53.962    0.000    0.813    0.756
##   INTMAT =~                                                             
##     C6_B2A            1.000                               0.529    0.571
##     C6_B2H            1.504    0.037   40.899    0.000    0.795    0.798
##     C6_B2Q            1.133    0.032   35.079    0.000    0.599    0.700
##   COMLRN =~                                                             
##     C6_B2C            1.000                               0.647    0.704
##     C6_B2I            1.061    0.023   45.663    0.000    0.687    0.815
##     C6_B2M            0.640    0.025   25.181    0.000    0.414    0.545
##     C6_B2S            0.954    0.024   39.050    0.000    0.617    0.717
##   SCVERB =~                                                             
##     C6_B2G            1.000                               0.647    0.806
##     C6_B2D            0.751    0.024   31.917    0.000    0.485    0.537
##     C6_B2R            0.991    0.022   45.357    0.000    0.641    0.776
##   SCMATH =~                                                             
##     C6_B2J            1.000                               0.668    0.788
##     C6_B2L            1.246    0.024   51.700    0.000    0.833    0.777
##     C6_B2O            1.137    0.019   60.285    0.000    0.760    0.834
##   SCACAD =~                                                             
##     C6_B2B            1.000                               0.534    0.745
##     C6_B2F            0.952    0.021   46.370    0.000    0.508    0.751
##     C6_B2P            1.018    0.024   42.972    0.000    0.543    0.733
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##  .C6_B2H ~~                                                             
##    .C6_B2L            0.199    0.010   19.190    0.000    0.199    0.491
##   INSMOT ~~                                                             
##     EFFPER            0.256    0.010   24.411    0.000    0.773    0.773
##     SELFEF            0.172    0.007   23.647    0.000    0.615    0.615
##     CEXP              0.239    0.009   26.144    0.000    0.685    0.685
##     INTREA            0.145    0.011   12.612    0.000    0.275    0.275
##     INTMAT            0.123    0.008   16.053    0.000    0.389    0.389
##     COMLRN            0.193    0.009   21.798    0.000    0.499    0.499
##     SCVERB            0.147    0.008   17.369    0.000    0.381    0.381
##     SCMATH            0.126    0.008   15.950    0.000    0.315    0.315
##     SCACAD            0.165    0.008   21.959    0.000    0.517    0.517
##   EFFPER ~~                                                             
##     SELFEF            0.196    0.007   28.347    0.000    0.756    0.756
##     CEXP              0.277    0.010   28.476    0.000    0.855    0.855
##     INTREA            0.175    0.011   16.400    0.000    0.360    0.360
##     INTMAT            0.147    0.008   18.149    0.000    0.504    0.504
##     COMLRN            0.183    0.009   20.504    0.000    0.512    0.512
##     SCVERB            0.169    0.008   21.085    0.000    0.473    0.473
##     SCMATH            0.145    0.008   17.054    0.000    0.391    0.391
##     SCACAD            0.185    0.007   25.124    0.000    0.626    0.626
##   SELFEF ~~                                                             
##     CEXP              0.256    0.008   30.702    0.000    0.934    0.934
##     INTREA            0.127    0.008   15.425    0.000    0.309    0.309
##     INTMAT            0.139    0.006   22.391    0.000    0.561    0.561
##     COMLRN            0.152    0.007   21.301    0.000    0.502    0.502
##     SCVERB            0.173    0.007   25.312    0.000    0.570    0.570
##     SCMATH            0.179    0.007   24.254    0.000    0.573    0.573
##     SCACAD            0.207    0.007   31.332    0.000    0.829    0.829
##   CEXP ~~                                                               
##     INTREA            0.175    0.010   17.550    0.000    0.340    0.340
##     INTMAT            0.155    0.008   19.867    0.000    0.502    0.502
##     COMLRN            0.199    0.008   23.684    0.000    0.525    0.525
##     SCVERB            0.201    0.009   22.120    0.000    0.531    0.531
##     SCMATH            0.178    0.009   19.894    0.000    0.456    0.456
##     SCACAD            0.233    0.008   30.830    0.000    0.748    0.748
##   INTREA ~~                                                             
##     INTMAT            0.101    0.010    9.878    0.000    0.217    0.217
##     COMLRN            0.144    0.011   12.666    0.000    0.254    0.254
##     SCVERB            0.191    0.012   15.864    0.000    0.336    0.336
##     SCMATH            0.073    0.011    6.667    0.000    0.124    0.124
##     SCACAD            0.170    0.010   17.640    0.000    0.361    0.361
##   INTMAT ~~                                                             
##     COMLRN            0.171    0.008   20.608    0.000    0.500    0.500
##     SCVERB            0.057    0.007    7.800    0.000    0.167    0.167
##     SCMATH            0.323    0.012   27.897    0.000    0.914    0.914
##     SCACAD            0.157    0.008   20.926    0.000    0.558    0.558
##   COMLRN ~~                                                             
##     SCVERB            0.135    0.008   16.115    0.000    0.322    0.322
##     SCMATH            0.177    0.009   19.247    0.000    0.409    0.409
##     SCACAD            0.181    0.007   24.362    0.000    0.524    0.524
##   SCVERB ~~                                                             
##     SCMATH            0.071    0.010    7.199    0.000    0.164    0.164
##     SCACAD            0.227    0.009   25.623    0.000    0.656    0.656
##   SCMATH ~~                                                             
##     SCACAD            0.206    0.009   24.107    0.000    0.578    0.578
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .C6_B1C            1.677    0.016  107.827    0.000    1.677    2.119
##    .C6_B1H            1.820    0.017  105.536    0.000    1.820    2.003
##    .C6_B1L            1.532    0.014  106.452    0.000    1.532    2.038
##    .C6_B1D            2.042    0.016  124.305    0.000    2.042    2.462
##    .C6_B1G            1.970    0.017  117.143    0.000    1.970    2.410
##    .C6_B1K            1.807    0.016  112.653    0.000    1.807    2.317
##    .C6_B1O            1.707    0.015  115.139    0.000    1.707    2.243
##    .C6_B1A            2.495    0.015  167.344    0.000    2.495    3.433
##    .C6_B1E            2.423    0.015  161.919    0.000    2.423    3.127
##    .C6_B1J            2.131    0.015  140.064    0.000    2.131    2.625
##    .C6_B1N            1.881    0.015  126.601    0.000    1.881    2.482
##    .C6_B1B            2.065    0.016  129.200    0.000    2.065    2.478
##    .C6_B1F            2.108    0.017  124.365    0.000    2.108    2.435
##    .C6_B1I            2.309    0.017  132.240    0.000    2.309    2.863
##    .C6_B1M            1.819    0.015  120.010    0.000    1.819    2.380
##    .C6_B2E            2.215    0.021  103.849    0.000    2.215    2.143
##    .C6_B2K            2.376    0.021  114.360    0.000    2.376    2.349
##    .C6_B2N            2.139    0.022   99.351    0.000    2.139    1.989
##    .C6_B2A            2.212    0.017  127.634    0.000    2.212    2.390
##    .C6_B2H            2.108    0.025   84.518    0.000    2.108    2.114
##    .C6_B2Q            1.839    0.020   91.561    0.000    1.839    2.149
##    .C6_B2C            2.210    0.018  126.203    0.000    2.210    2.404
##    .C6_B2I            2.014    0.016  123.617    0.000    2.014    2.390
##    .C6_B2M            1.526    0.014  106.245    0.000    1.526    2.007
##    .C6_B2S            2.251    0.016  138.176    0.000    2.251    2.615
##    .C6_B2G            2.254    0.017  136.456    0.000    2.254    2.808
##    .C6_B2D            2.232    0.019  116.466    0.000    2.232    2.471
##    .C6_B2R            2.213    0.021  105.537    0.000    2.213    2.682
##    .C6_B2J            1.929    0.020   95.075    0.000    1.929    2.276
##    .C6_B2L            2.254    0.028   81.826    0.000    2.254    2.103
##    .C6_B2O            2.075    0.019  108.949    0.000    2.075    2.277
##    .C6_B2B            1.979    0.014  143.925    0.000    1.979    2.762
##    .C6_B2F            1.848    0.014  134.672    0.000    1.848    2.730
##    .C6_B2P            2.101    0.014  150.398    0.000    2.101    2.834
##     INSMOT            0.000                               0.000    0.000
##     EFFPER            0.000                               0.000    0.000
##     SELFEF            0.000                               0.000    0.000
##     CEXP              0.000                               0.000    0.000
##     INTREA            0.000                               0.000    0.000
##     INTMAT            0.000                               0.000    0.000
##     COMLRN            0.000                               0.000    0.000
##     SCVERB            0.000                               0.000    0.000
##     SCMATH            0.000                               0.000    0.000
##     SCACAD            0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .C6_B1C            0.270    0.010   26.509    0.000    0.270    0.431
##    .C6_B1H            0.543    0.019   29.103    0.000    0.543    0.658
##    .C6_B1L            0.199    0.008   25.071    0.000    0.199    0.352
##    .C6_B1D            0.381    0.010   39.469    0.000    0.381    0.555
##    .C6_B1G            0.340    0.011   30.521    0.000    0.340    0.509
##    .C6_B1K            0.278    0.008   33.792    0.000    0.278    0.457
##    .C6_B1O            0.327    0.009   37.067    0.000    0.327    0.565
##    .C6_B1A            0.309    0.008   37.957    0.000    0.309    0.585
##    .C6_B1E            0.304    0.008   40.283    0.000    0.304    0.507
##    .C6_B1J            0.352    0.008   45.083    0.000    0.352    0.533
##    .C6_B1N            0.287    0.008   36.339    0.000    0.287    0.501
##    .C6_B1B            0.353    0.009   39.687    0.000    0.353    0.508
##    .C6_B1F            0.488    0.017   29.446    0.000    0.488    0.652
##    .C6_B1I            0.426    0.011   40.014    0.000    0.426    0.655
##    .C6_B1M            0.286    0.009   31.360    0.000    0.286    0.490
##    .C6_B2E            0.296    0.016   18.773    0.000    0.296    0.277
##    .C6_B2K            0.266    0.012   22.515    0.000    0.266    0.260
##    .C6_B2N            0.496    0.014   34.831    0.000    0.496    0.428
##    .C6_B2A            0.577    0.014   42.107    0.000    0.577    0.673
##    .C6_B2H            0.361    0.014   26.713    0.000    0.361    0.363
##    .C6_B2Q            0.374    0.012   32.304    0.000    0.374    0.510
##    .C6_B2C            0.427    0.014   30.457    0.000    0.427    0.505
##    .C6_B2I            0.239    0.011   21.349    0.000    0.239    0.336
##    .C6_B2M            0.406    0.012   33.148    0.000    0.406    0.703
##    .C6_B2S            0.360    0.012   31.174    0.000    0.360    0.486
##    .C6_B2G            0.226    0.011   20.321    0.000    0.226    0.350
##    .C6_B2D            0.580    0.016   35.994    0.000    0.580    0.711
##    .C6_B2R            0.271    0.013   21.081    0.000    0.271    0.397
##    .C6_B2J            0.272    0.008   34.499    0.000    0.272    0.378
##    .C6_B2L            0.456    0.013   35.740    0.000    0.456    0.396
##    .C6_B2O            0.253    0.009   26.783    0.000    0.253    0.305
##    .C6_B2B            0.229    0.007   34.295    0.000    0.229    0.445
##    .C6_B2F            0.200    0.007   29.562    0.000    0.200    0.436
##    .C6_B2P            0.254    0.008   30.416    0.000    0.254    0.463
##     INSMOT            0.357    0.015   23.666    0.000    1.000    1.000
##     EFFPER            0.306    0.013   23.101    0.000    1.000    1.000
##     SELFEF            0.219    0.010   22.761    0.000    1.000    1.000
##     CEXP              0.342    0.012   28.723    0.000    1.000    1.000
##     INTREA            0.772    0.022   35.303    0.000    1.000    1.000
##     INTMAT            0.280    0.014   20.088    0.000    1.000    1.000
##     COMLRN            0.419    0.016   25.441    0.000    1.000    1.000
##     SCVERB            0.418    0.015   28.232    0.000    1.000    1.000
##     SCMATH            0.447    0.017   26.798    0.000    1.000    1.000
##     SCACAD            0.285    0.011   25.713    0.000    1.000    1.000

Regarding factor loadings, some items show rather lower loadings, namely C6_B1H, C6_B1F, C6_B1I, C6_B2A, C6_B2M, and C6_B2D with the lowest loading of .537 (it also has the highest variance and the second highest SE). All loadings were significant.

Mean factor loading

## [1] 0.715

For how many of the items does the factor explain more than half of their variance (lambda = sqrt(2)/2, ~.707)?

## [1] "70.6%"

How many of the items show a loading greater than .6?

## [1] "82.4%"

Latent correlations

For convenience, latent correlations between the SAL factors in form of a matrix.

The data show high subject-specificity of self-concept and interest constructs. Domain-general self-concept constructs (SELFEF, CEXP, ACACAD) intercorrelate highly.

INSMOT EFFPER SELFEF CEXP INTREA INTMAT COMLRN SCVERB SCMATH SCACAD
INSMOT 1
EFFPER 0.77 1
SELFEF 0.61 0.76 1
CEXP 0.68 0.86 0.93 1
INTREA 0.28 0.36 0.31 0.34 1
INTMAT 0.39 0.5 0.56 0.5 0.22 1
COMLRN 0.5 0.51 0.5 0.53 0.25 0.5 1
SCVERB 0.38 0.47 0.57 0.53 0.34 0.17 0.32 1
SCMATH 0.31 0.39 0.57 0.46 0.12 0.91 0.41 0.16 1
SCACAD 0.52 0.63 0.83 0.75 0.36 0.56 0.52 0.66 0.58 1

Diagram

Model diagnostics

Modification indices

The next step is then to look at the modification indices table. Collumn “mi” stands for the modification index. It represents the change in chi square statistics if you free the given parameter. “~” stands for regression path (read as “predicted by”), “~~” denotes a correlation. “sepc.all” is the value of correlation or standardized regression path that the model missed. Showing only the 10 most severe misspecifications, sorted by magnitude.

Especially the item B2L (“Matematika je pro mě jedním z nejlepších předmětů”) does not seem to work well at all. It shows high cross-loadings on other factors (its cross-loading on INTMAT would be .88(!) should it be modelled). Please note that this item already has an error covariance with B2H. However, no modifications to the model were carried out.

lhs op rhs mi epc sepc.lv sepc.all sepc.nox
347 INTMAT =~ C6_B2L 423.43 1.81 0.95 0.89 0.89
685 C6_B1A ~~ C6_B1E 398.71 0.09 0.09 0.30 0.30
952 C6_B2Q ~~ C6_B2L 322.34 0.11 0.11 0.28 0.28
473 SCACAD =~ C6_B2L 262.50 -0.42 -0.23 -0.21 -0.21
408 SCVERB =~ C6_B2L 247.56 -0.27 -0.17 -0.16 -0.16
470 SCACAD =~ C6_B2D 233.12 -0.48 -0.26 -0.29 -0.29
344 INTMAT =~ C6_B2D 205.50 -0.31 -0.16 -0.18 -0.18
472 SCACAD =~ C6_B2J 200.34 0.30 0.16 0.19 0.19
900 C6_B2N ~~ C6_B2A 194.02 0.11 0.11 0.20 0.20
264 CEXP =~ C6_B1G 192.31 0.61 0.36 0.44 0.44

Heatmap of standardized residuals

Model test indicates the presence of model misspecification. Apart from (global) approximate fit indices, it is necessary to also analyze local sources of causal misspecification based on a matrix of standardized residuals.

The same as shown by residuals can be seen on residuals heatmap. Rather strong residuals are among the subject-specific factor items. Especially B2L and B2D.

Matrix of raw residuals

C6_B1C C6_B1H C6_B1L C6_B1D C6_B1G C6_B1K C6_B1O C6_B1A C6_B1E C6_B1J C6_B1N C6_B1B C6_B1F C6_B1I C6_B1M C6_B2E C6_B2K C6_B2N C6_B2A C6_B2H C6_B2Q C6_B2C C6_B2I C6_B2M C6_B2S C6_B2G C6_B2D C6_B2R C6_B2J C6_B2L C6_B2O C6_B2B C6_B2F C6_B2P
C6_B1C 0
C6_B1H -0.03 0
C6_B1L -0.01 0.04 0
C6_B1D 0.05 -0.02 -0.03 0
C6_B1G 0.01 -0.06 -0.03 -0.01 0
C6_B1K 0.03 -0.05 0.03 0 -0.02 0
C6_B1O 0.01 -0.03 0.01 0.05 -0.03 0.02 0
C6_B1A -0.01 -0.04 -0.05 -0.06 0.02 -0.07 -0.05 0
C6_B1E 0 -0.01 -0.04 -0.02 0.06 -0.04 -0.05 0.12 0
C6_B1J 0 0 0 -0.05 0.01 0.02 -0.03 -0.02 -0.01 0
C6_B1N 0.04 0.03 0.05 -0.01 0.07 0.07 0.06 -0.05 -0.03 0 0
C6_B1B 0.06 -0.04 -0.05 -0.01 0.06 -0.01 -0.06 0.04 0.04 -0.01 -0.01 0
C6_B1F 0.03 0 -0.03 -0.04 0.02 -0.05 -0.05 -0.04 -0.01 0.02 0 0.01 0
C6_B1I -0.01 0.03 -0.03 -0.01 0.03 -0.01 -0.03 0 0.01 0.03 0 -0.04 0.06 0
C6_B1M 0.01 0 0.03 0 0.04 0.04 0.02 -0.08 -0.06 -0.01 0.08 0 0.01 -0.01 0
C6_B2E 0.04 -0.06 0.02 0 0.01 0.01 -0.01 0.04 0.02 0 0.01 0.01 0.02 0.01 0.01 0
C6_B2K 0.02 -0.08 -0.02 0.01 0 -0.01 -0.02 0.01 -0.01 -0.05 -0.01 -0.02 0.01 -0.03 0 0 0
C6_B2N 0.02 -0.04 0.02 0 0 0 0 0.01 -0.02 -0.03 0.01 -0.02 0.01 0 0.01 -0.01 0 0
C6_B2A 0.02 0.03 0.03 0.06 0.06 0.04 0.08 0.05 0.05 0.02 0.01 0.01 0.02 0.09 0.04 0.07 0.05 0.15 0
C6_B2H -0.01 -0.04 -0.05 -0.03 -0.03 -0.06 -0.03 -0.02 0 -0.04 -0.05 -0.04 -0.03 0.02 -0.04 -0.02 -0.04 -0.03 0.02 0
C6_B2Q 0.04 0.02 0.02 0.01 0.02 0 0.04 -0.02 0 -0.03 -0.03 -0.03 -0.02 0.03 -0.01 0.02 0 0.02 0 0.04 0
C6_B2C -0.06 0.07 -0.05 -0.07 -0.06 -0.04 -0.03 0 -0.01 0 -0.04 -0.09 -0.03 0.01 -0.06 -0.03 -0.06 -0.02 0.06 -0.07 -0.04 0
C6_B2I 0.01 0.06 -0.01 0.01 0 0.01 0.05 -0.03 -0.03 0.01 0.01 -0.03 0.01 0.05 0.02 0 -0.02 0.03 0.08 -0.01 0.02 0.01 0
C6_B2M -0.01 0.1 0.03 -0.04 -0.03 0 0.02 -0.05 -0.04 -0.02 -0.02 -0.09 -0.03 -0.03 -0.01 0 0 0.07 0.05 -0.06 0.03 0.04 -0.01 0
C6_B2S 0 0.06 -0.02 0.03 0.03 0.03 0.03 0.05 0.03 0.05 0.05 0.03 0.05 0.1 0.06 0.03 0.02 0.07 0.1 -0.03 0.01 -0.01 -0.01 0 0
C6_B2G 0.04 -0.04 0.01 -0.01 0.03 0.03 0 0.03 -0.02 0.02 0.03 0.03 0.02 -0.02 0 0.02 -0.01 -0.01 0.05 -0.01 0.03 -0.02 0.01 -0.02 0.09 0
C6_B2D -0.01 -0.12 -0.04 -0.06 -0.03 -0.04 -0.05 -0.01 -0.09 -0.05 -0.02 -0.01 -0.04 -0.09 -0.04 -0.05 -0.05 -0.04 -0.13 -0.15 -0.1 -0.12 -0.1 -0.1 -0.05 0.02 0
C6_B2R 0.03 -0.05 0.01 -0.04 0.03 0.01 -0.01 -0.01 -0.05 0.04 0.02 0.01 0.04 -0.02 -0.01 0.03 0.01 0.01 0.01 -0.06 0.02 -0.03 0 -0.02 0.09 -0.01 0.02 0
C6_B2J 0.03 0.03 0.01 -0.03 0.05 0.01 0.02 0.06 0.04 0.05 0 0.02 0.04 0.04 0.01 0.03 0.01 -0.01 -0.08 -0.01 -0.06 0 0.01 -0.04 -0.01 0.08 -0.05 0.07 0
C6_B2L -0.02 -0.01 -0.04 -0.03 0 -0.04 -0.01 -0.03 -0.03 -0.07 -0.09 -0.04 -0.04 0 -0.05 -0.01 0.01 -0.01 0.03 0.02 0.1 -0.03 0.01 -0.03 -0.03 -0.06 -0.16 -0.1 -0.01 0
C6_B2O 0 0.02 -0.02 -0.04 0.03 -0.02 -0.02 0.04 0.03 0.02 -0.03 0 0.01 0.05 -0.02 -0.01 -0.03 -0.03 -0.05 0 -0.04 -0.01 0.01 -0.03 0.01 0.04 -0.09 0.03 0.02 -0.01 0
C6_B2B 0 0 0 -0.04 0.04 0 0.01 0.02 -0.01 -0.01 0.01 0.03 -0.01 0 -0.01 0 -0.01 0 0.06 -0.03 -0.02 -0.01 -0.01 -0.04 0.06 0.02 -0.1 -0.02 0.04 -0.07 -0.01 0
C6_B2F 0.03 0 0 -0.04 0.03 0.02 0.02 -0.01 -0.02 0 0 0 0 0 0 0.02 -0.01 0 0.03 -0.04 -0.02 -0.03 0 -0.04 0.05 0 -0.11 0 0.04 -0.09 -0.01 0.04 0
C6_B2P 0.01 -0.02 -0.02 -0.05 0.02 -0.01 -0.01 0 -0.02 0.06 -0.01 0.01 0.03 -0.01 -0.03 0 0 -0.01 0 -0.05 -0.01 -0.03 -0.01 -0.04 0.05 0.02 -0.05 0.09 0.12 -0.07 0.03 -0.03 -0.01 0

Mean of absolute raw residual correlations

## [1] 0.03019161

Visualization of significant misspecifications

For a better overview, here is the residual matrix which marks residual values >.1. Such values can be seen as worrying.

Number of variables

If the matrix contains (p(p+1)/2 - p) = 561 elements (sans diagonal), 28.05 can be significant at alpha = .05

Number of residuals significant at .05 level

## TRUE 
##  198

Number of residuals significant at .001 level

## TRUE 
##   76
C6_B1C C6_B1H C6_B1L C6_B1D C6_B1G C6_B1K C6_B1O C6_B1A C6_B1E C6_B1J C6_B1N C6_B1B C6_B1F C6_B1I C6_B1M C6_B2E C6_B2K C6_B2N C6_B2A C6_B2H C6_B2Q C6_B2C C6_B2I C6_B2M C6_B2S C6_B2G C6_B2D C6_B2R C6_B2J C6_B2L C6_B2O C6_B2B C6_B2F C6_B2P
C6_B1C Diag
C6_B1H . Diag
C6_B1L . . Diag
C6_B1D . . . Diag
C6_B1G . . . . Diag
C6_B1K . . . . . Diag
C6_B1O . . . . . . Diag
C6_B1A . . . . . . . Diag
C6_B1E . . . . . . . >.1 Diag
C6_B1J . . . . . . . . . Diag
C6_B1N . . . . . . . . . . Diag
C6_B1B . . . . . . . . . . . Diag
C6_B1F . . . . . . . . . . . . Diag
C6_B1I . . . . . . . . . . . . . Diag
C6_B1M . . . . . . . . . . . . . . Diag
C6_B2E . . . . . . . . . . . . . . . Diag
C6_B2K . . . . . . . . . . . . . . . . Diag
C6_B2N . . . . . . . . . . . . . . . . . Diag
C6_B2A . . . . . . . . . . . . . . . . . >.1 Diag
C6_B2H . . . . . . . . . . . . . . . . . . . Diag
C6_B2Q . . . . . . . . . . . . . . . . . . . . Diag
C6_B2C . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2I . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2M . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2S . . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2G . . . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2D . >.1 . . . . . . . . . . . . . . . . >.1 >.1 >.1 >.1 >.1 >.1 . . Diag
C6_B2R . . . . . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2J . . . . . . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2L . . . . . . . . . . . . . . . . . . . . . . . . . . >.1 >.1 . Diag
C6_B2O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Diag
C6_B2B . . . . . . . . . . . . . . . . . . . . . . . . . . >.1 . . . . Diag
C6_B2F . . . . . . . . . . . . . . . . . . . . . . . . . . >.1 . . . . . Diag
C6_B2P . . . . . . . . . . . . . . . . . . . . . . . . . . . . >.1 . . . . Diag

Reliability

Internal consistency estimates for individual scales - McDonald’s Omega

The scales show very good overall reliability (especially given the number of items per scale; 3-4). These reliabilities are amongst the highest with respect to countries included in Marsh et al. study.

## Loading required namespace: GPArotation
Reliability.Omega
INSMOT 0.72
EFFPER 0.80
SELFEF 0.86
CEXP 0.78
INTREA 0.85
INTMAT 0.73
COMLRN 0.85
SCVERB 0.74
SCMATH 0.82
SCACAD 0.76
Mean scale reliability 0.79
SD of scale reliabilities 0.05

Measurement invariance

Invariance with respect to gender

The measure shows excellent measurement invariance when fixing loadings and good invariance even when fixing intercepts of items. The instrument thus measures the same constructs equaly well in both genders, i.e., any differences are likely real, not due to how the LVs were measured.

## 
## Measurement invariance models:
## 
## Model 1 : fit.configural
## Model 2 : fit.loadings
## Model 3 : fit.intercepts
## Model 4 : fit.means
## 
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                  Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
## fit.configural  962         6510.6                                  
## fit.loadings    986         6652.6      23.09      24     0.5147    
## fit.intercepts 1044         7008.0                 58               
## fit.means      1054         9893.4     355.78      10     <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Fit measures:
## 
##                cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
## fit.configural      0.958        0.052               NA                 NA
## fit.loadings        0.964        0.048            0.006              0.004
## fit.intercepts      0.956        0.051            0.008              0.003
## fit.means           0.942        0.058            0.014              0.007

Comparing genders on latent means

Constraining loadings and latent intercepts to be equal. Fixing the factor means of the first/reference group (Gender == 1) to zero while estimating the factor means for the other group. These magnitudes equal the difference between the groups.

intercept
INSMOT 0.01
EFFPER -0.05
SELFEF 0.10
CEXP -0.01
INTREA -0.40
INTMAT 0.17
COMLRN 0.10
SCVERB -0.14
SCMATH 0.27
SCACAD 0.05

All the differences in latent means apart from INSMOT, CEXP, and SCACAD are significant. Negative intercept values denote higher mean values for girls and positive values higher mean values for the boys (factors are scaled inversely). The girls show markedly higher interest in reading, boys higher interest in math. Girls report spending more effort and being more perseverant, while boys have higher values of self-eficacy and competitive learning. Girls show higher self-concept in language while the opposite is true for math. Results overall consistent with stereotype threat bias.

Comparing ZS/VG on latent means

Constraining loadings and latent intercepts to be equal. Fixing the factor means of the first/reference group (ZS.VG == ZS) to zero while estimating the factor means for the other group (VG). These magnitudes equal the difference between the groups.

Group 1 = ZS, Group 2 = VG

intercept
INSMOT -0.14
EFFPER -0.11
SELFEF -0.18
CEXP -0.15
INTREA -0.32
INTMAT -0.14
COMLRN -0.08
SCVERB -0.24
SCMATH -0.22
SCACAD -0.17

Children at VG (eight-year academies) show lower means compared to elementary school, i.e., more positive values (factors are scaled inversely) for all factors.

Measurement invariance

Invariance with respect to gender

## 
## Measurement invariance models:
## 
## Model 1 : fit.configural
## Model 2 : fit.loadings
## Model 3 : fit.intercepts
## Model 4 : fit.means
## 
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                  Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
## fit.configural  962         5533.2                                  
## fit.loadings    986         5652.7      20.59      24     0.6625    
## fit.intercepts 1044         5976.6                 58               
## fit.means      1054         8480.7     329.77      10     <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Fit measures:
## 
##                cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
## fit.configural      0.959        0.051               NA                 NA
## fit.loadings        0.965        0.047            0.005              0.004
## fit.intercepts      0.958        0.050            0.007              0.003
## fit.means           0.943        0.058            0.015              0.008
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                    Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.config_fit 962 432865 434857 7050.7                              
## survey.weak_fit   986 432885 434716 7118.6      48.38      24   0.002264
##                     
## survey.config_fit   
## survey.weak_fit   **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                     Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.weak_fit    986 432885 434716 7118.6                              
## survey.strong_fit 1010 433356 435026 7638.4     362.87      24  < 2.2e-16
##                      
## survey.weak_fit      
## survey.strong_fit ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Measurement invariance

Invariance with respect to SES quintiles

## 
## Measurement invariance models:
## 
## Model 1 : fit.configural
## Model 2 : fit.loadings
## Model 3 : fit.intercepts
## Model 4 : fit.means
## 
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                  Df AIC BIC   Chisq Chisq diff Df diff Pr(>Chisq)    
## fit.configural 2405          6895.6                                  
## fit.loadings   2501          7429.1     76.657      96      0.927    
## fit.intercepts 2733          7412.1                232               
## fit.means      2773         10181.3    278.188      40     <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Fit measures:
## 
##                cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
## fit.configural      0.959        0.050               NA                 NA
## fit.loadings        0.966        0.045            0.007              0.005
## fit.intercepts      0.961        0.047            0.006              0.002
## fit.means           0.950        0.052            0.011              0.006
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                     Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.config_fit 2405 362228 367083 8633.1                              
## survey.weak_fit   2501 362218 366443 8815.2     135.55      96   0.004901
##                     
## survey.config_fit   
## survey.weak_fit   **
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                     Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.weak_fit   2501 362218 366443 8815.2                              
## survey.strong_fit 2597 362304 365899 9092.6     198.93      96  3.561e-09
##                      
## survey.weak_fit      
## survey.strong_fit ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Measurement invariance

Invariance with respect to type of school

## 
## Measurement invariance models:
## 
## Model 1 : fit.configural
## Model 2 : fit.loadings
## Model 3 : fit.intercepts
## Model 4 : fit.means
## 
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                  Df AIC BIC  Chisq Chisq diff Df diff Pr(>Chisq)    
## fit.configural  962         5815.0                                  
## fit.loadings    986         6141.1      60.98      24  4.639e-05 ***
## fit.intercepts 1044         6284.5                 58               
## fit.means      1054         8439.7     295.45      10  < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## Fit measures:
## 
##                cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
## fit.configural      0.960        0.051               NA                 NA
## fit.loadings        0.963        0.048            0.003              0.003
## fit.intercepts      0.959        0.050            0.004              0.001
## fit.means           0.947        0.056            0.012              0.006
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                    Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.config_fit 962 431609 433601 6850.9                              
## survey.weak_fit   986 431715 433546 7005.6     138.98      24  < 2.2e-16
##                      
## survey.config_fit    
## survey.weak_fit   ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## Scaled Chi Square Difference Test (method = "satorra.bentler.2001")
## 
##                     Df    AIC    BIC  Chisq Chisq diff Df diff Pr(>Chisq)
## survey.weak_fit    986 431715 433546 7005.6                              
## survey.strong_fit 1010 432056 433725 7394.4     354.51      24  < 2.2e-16
##                      
## survey.weak_fit      
## survey.strong_fit ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Predictive validity

Achievement measures

Predictive validity of the 10 SAL factors with respect to achievement measures (Language, Math). Fit did not deteriorate due to inclusion of predictive factors of language and math achievement scores.

Model test and approximate fit indices

##          chisq.scaled             df.scaled         pvalue.scaled 
##               650.706                82.000                 0.000 
##            cfi.scaled            tli.scaled          rmsea.scaled 
##                 0.930                 0.927                 0.033 
## rmsea.ci.lower.scaled rmsea.ci.upper.scaled                  srmr 
##                 0.033                 0.034                 0.038 
##                  pnfi                   bic 
##                 0.781            538976.797

Latent correlations

Correlations between the SAL factors and achievement measures (Language, Math)

The achievement scores correlate rather strongly and show positive relationship towards each of the SAL factors. Apart from having highest intercorrelations with other SAL factors, SELFEF shows the highest relative predictive power.

INSMOT EFFPER SELFEF CEXP INTREA INTMAT COMLRN SCVERB SCMATH SCACAD C6_Math_rel_fac C6_lang_rel_fac
INSMOT 1
EFFPER 0.77 1
SELFEF 0.62 0.76 1
CEXP 0.68 0.86 0.93 1
INTREA 0.28 0.36 0.31 0.34 1
INTMAT 0.39 0.5 0.56 0.5 0.22 1
COMLRN 0.5 0.51 0.5 0.53 0.25 0.5 1
SCVERB 0.38 0.47 0.57 0.53 0.34 0.17 0.32 1
SCMATH 0.32 0.39 0.57 0.46 0.12 0.91 0.41 0.17 1
SCACAD 0.52 0.63 0.83 0.75 0.36 0.56 0.53 0.66 0.58 1
C6_Math_rel_fac -0.13 -0.14 -0.3 -0.18 -0.12 -0.27 -0.08 -0.17 -0.4 -0.3 1
C6_lang_rel_fac -0.13 -0.17 -0.25 -0.2 -0.24 -0.07 -0.05 -0.37 -0.16 -0.29 0.6 1

Bayesian analysis of predictive power

For achievement measures

Bayes factor in favor of the alternative hypothesis (BF10) and posterior probability for model parameters (given 1:1 prior odds for H0:Ha)

Bayes factors show whether there is evidence either for Ha (effect present) or H0 (effect absent), i.e., whether the data are more consistent with Ha, H0, or inconclusive. Posterior probability refers to the probability of the parameter not being zero (as oposed to probability of the data under a null). Frequentist approach without specific procedures (like equivalence testing), on the other hand, cannot provide evidence for H0, by definition (the only possible conclusions are H0 being rejected or failed to be rejected). These are Bayes Factors based on model selection / information criteria approach as proposed by Wagenmakers, 2007. Each BF represents the relative evidence in the data favoring alternative hypothesis (parameter freely estimated) over the null (the given parameter fixed to 0). Bayes Factors using BIC approximation implicitly assume unit information prior which makes them rather conservative with regard to the alternative hypothesis.

Parameters a-u refer to covariances between Math achievement (a-j), Language achievement (k-u) and the 10 SAL factors.

Math ~~ aINSMOT Math ~~ bEFFPER Math ~~ cSELFEF Math ~~ dCEXP Math ~~ eINTREA Math ~~ fINTMAT Math ~~ gCOMLRN Math ~~ hSCVERB Math ~~ iSCMATH Math ~~ jSCACAD Language ~~ kINSMOT Language ~~ lEFFPER Language ~~ mSELFEF Language ~~ nCEXP Language ~~ oINTREA Language ~~ pINTMAT Language ~~ qCOMLRN Language ~~ rSCVERB Language ~~ sSCMATH Language ~~ tSCACAD Math ~~ u*Language

BF10 Posterior
a 6.174306e+15 1.00
b 1.007541e+19 1.00
c 7.030041e+98 1.00
d 9.019117e+31 1.00
e 4.081917e+14 1.00
f 1.863388e+78 1.00
g 1.873598e+05 1.00
h 4.183234e+30 1.00
i 4.415524e+194 1.00
j 9.613697e+93 1.00
k 3.345478e+16 1.00
l 2.199564e+28 1.00
m 6.875950e+65 1.00
n 2.878808e+39 1.00
o 6.793993e+64 1.00
p 2.773500e+03 1.00
q 2.780000e+00 0.74
r 3.326750e+151 1.00
s 4.739896e+26 1.00
t 1.175954e+90 1.00
u Inf NaN

BF10 indicates how much likely is the data under Ha as compared to H0. Table above shows that there is almost 100% posterior probability in favor of most of these effects.

Grades

Predictive validity of the 10 SAL factors with respect to grades - Language (averaged Czech and English language grades) and Math. Fit did not deteriorate due to inclusion of predictive factors of language and math achievement scores.

Model test and approximate fit indices

##          chisq.scaled             df.scaled         pvalue.scaled 
##               606.136                80.000                 0.000 
##            cfi.scaled            tli.scaled          rmsea.scaled 
##                 0.933                 0.930                 0.033 
## rmsea.ci.lower.scaled rmsea.ci.upper.scaled                  srmr 
##                 0.032                 0.034                 0.037 
##                  pnfi                   bic 
##                 0.783            450775.989

Latent correlations

Correlations between the SAL factors and achievement measures (Language, Math)

Grades correlate rather strongly (stronger than achievement measures) and show positive relationship towards each of the SAL factors. SCACAD and SELFEF show the highest relative predictive power.

INSMOT EFFPER SELFEF CEXP INTREA INTMAT COMLRN SCVERB SCMATH SCACAD grade_math_fac grade_lang_fac
INSMOT 1
EFFPER 0.77 1
SELFEF 0.61 0.75 1
CEXP 0.68 0.86 0.93 1
INTREA 0.27 0.36 0.31 0.34 1
INTMAT 0.38 0.5 0.56 0.5 0.22 1
COMLRN 0.5 0.51 0.5 0.53 0.25 0.5 1
SCVERB 0.38 0.47 0.57 0.53 0.34 0.16 0.32 1
SCMATH 0.31 0.39 0.57 0.45 0.12 0.91 0.4 0.16 1
SCACAD 0.52 0.63 0.83 0.75 0.36 0.55 0.52 0.66 0.58 1
grade_math_fac 0.21 0.25 0.36 0.28 0.11 0.37 0.16 0.24 0.51 0.38 1
grade_lang_fac 0.26 0.29 0.36 0.31 0.2 0.14 0.13 0.43 0.23 0.41 0.69 1