1 Loading, setting up

2 RQs

How can we measure engagement in online science classes?
What’s it’s latent structure (one versus three dimensions; each dimension as an observed or a latent variable [multiple indicators for each dimension?])?
What are the best indicators of engagement?
How does engagement mediate the relationship between interest and course grade in online science courses?

Things to do:

Choose between measures of engagement
Compare treating engagement as having three dimensions (as a latent variable?)
Possibly modeling each of the three dimensions of engagement as a latent construct

3 Possible Models

3.1 Unidimensional

3.1.1 Aff

aff1. Interest -> posemo -> Final Grade
aff2. Interest -> reversed negemo -> Final Grade
aff3. Interest -> tone -> Final Grade
aff4. Interest -> posemo & reversed negemo -> Final Grade
aff5. Interest -> tone & reversed negemo -> Final Grade
aff6. Interest -> Aff eng as a latent factor -> Final Grade

3.1.2 Beh

beh1. Interest -> time spent -> Final Grade
beh2. Interest -> n discussion posts -> Final Grade
beh3. Interest -> n discussion posts & time spent -> Final Grade

3.1.3 Cog

cog1. Interest -> Cogproc -> Final Grade
cog2. Interest -> Analytic thinking -> Final Grade
cog3. Interest -> Cogproc & analytic thinking -> Final Grade

3.2 Multidimensional

O1. Interest -> Cog | Beh | Aff engagement measured based on unidimensional measurements -> Final Grade

3.3 Interest

I1. CFA for interest

3.4 What goes in the table (make sure to use the robust stats)?

Model name
LL
Chi square (p-value) and df
AIC, BIC, SABIC
RMSEA [CI]
CFI and TLI
SRMR

4 Models

4.1 Interest

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
i1 <- lavaan.survey(m_est, group)
summary(i1, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 25 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         15
## 
##   Number of observations                           622
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      13.424       7.204
##   Degrees of freedom                                 5           5
##   P-value (Chi-square)                           0.020       0.206
##   Scaling correction factor                                  1.863
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1481.939     639.010
##   Degrees of freedom                                10          10
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.994       0.996
##   Tucker-Lewis Index (TLI)                       0.989       0.993
## 
##   Robust Comparative Fit Index (CFI)                         0.997
##   Robust Tucker-Lewis Index (TLI)                            0.994
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -2646.763   -2646.763
##   Loglikelihood unrestricted model (H1)      -2640.051   -2640.051
## 
##   Number of free parameters                         15          15
##   Akaike (AIC)                                5323.526    5323.526
##   Bayesian (BIC)                              5390.020    5390.020
##   Sample-size adjusted Bayesian (BIC)         5342.397    5342.397
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.052       0.027
##   90 Percent Confidence Interval          0.019  0.087       0.000  0.056
##   P-value RMSEA <= 0.05                          0.404       0.897
## 
##   Robust RMSEA                                               0.036
##   90 Percent Confidence Interval                             0.000  0.090
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.014       0.014
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.516    0.789
##     q4                1.034    0.070   14.857    0.000    0.533    0.669
##     q5                0.987    0.073   13.600    0.000    0.509    0.770
##     q8                1.040    0.074   13.974    0.000    0.536    0.837
##     q10               1.258    0.060   20.908    0.000    0.649    0.752
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.350    0.051   85.038    0.000    4.350    6.658
##    .q4                4.349    0.044   99.740    0.000    4.349    5.455
##    .q5                4.297    0.037  116.502    0.000    4.297    6.499
##    .q8                4.359    0.047   92.214    0.000    4.359    6.807
##    .q10               4.178    0.062   67.582    0.000    4.178    4.842
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.161    0.017    9.540    0.000    0.161    0.377
##    .q4                0.351    0.050    6.984    0.000    0.351    0.553
##    .q5                0.178    0.025    7.012    0.000    0.178    0.407
##    .q8                0.123    0.017    7.251    0.000    0.123    0.299
##    .q10               0.323    0.028   11.410    0.000    0.323    0.434
##     pre_int           0.266    0.038    6.933    0.000    1.000    1.000

4.2 Aff

4.2.1 A1. Interest -> posemo -> Final Grade

d$posemo_l <- safe_log(d$posemo_prop)

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    posemo_l ~ pre_int
    final_grade ~ pre_int + posemo_l
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff1 <- lavaan.survey(m_est, group)
summary(aff1, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 72 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           605
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      25.710      16.841
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.019       0.207
##   Scaling correction factor                                  1.527
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1478.272     729.167
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.991       0.995
##   Tucker-Lewis Index (TLI)                       0.986       0.991
## 
##   Robust Comparative Fit Index (CFI)                         0.996
##   Robust Tucker-Lewis Index (TLI)                            0.993
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -5497.160   -5497.160
##   Loglikelihood unrestricted model (H1)      -5484.305   -5484.305
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               11038.320   11038.320
##   Bayesian (BIC)                             11135.235   11135.235
##   Sample-size adjusted Bayesian (BIC)        11065.390   11065.390
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.040       0.022
##   90 Percent Confidence Interval          0.016  0.063       0.000  0.044
##   P-value RMSEA <= 0.05                          0.736       0.986
## 
##   Robust RMSEA                                               0.027
##   90 Percent Confidence Interval                             0.000  0.060
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.019       0.019
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.517    0.791
##     q4                1.036    0.070   14.737    0.000    0.535    0.668
##     q5                0.979    0.069   14.276    0.000    0.506    0.769
##     q8                1.045    0.078   13.475    0.000    0.540    0.844
##     q10               1.237    0.060   20.618    0.000    0.639    0.748
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   posemo_l ~                                                            
##     pre_int          -0.007    0.038   -0.176    0.860   -0.003   -0.009
##   final_grade ~                                                         
##     pre_int           5.920    2.231    2.653    0.008    3.058    0.149
##     posemo_l          5.159    4.164    1.239    0.215    5.159    0.093
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.051   85.599    0.000    4.354    6.663
##    .q4                4.355    0.044   99.693    0.000    4.355    5.435
##    .q5                4.302    0.038  114.674    0.000    4.302    6.543
##    .q8                4.362    0.047   93.244    0.000    4.362    6.817
##    .q10               4.187    0.060   69.935    0.000    4.187    4.899
##    .posemo_l          5.370    0.030  179.131    0.000    5.370   14.539
##    .final_grade      50.318   22.403    2.246    0.025   50.318    2.456
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.160    0.017    9.571    0.000    0.160    0.375
##    .q4                0.356    0.052    6.900    0.000    0.356    0.554
##    .q5                0.177    0.026    6.902    0.000    0.177    0.409
##    .q8                0.118    0.016    7.266    0.000    0.118    0.287
##    .q10               0.322    0.029   11.078    0.000    0.322    0.441
##    .posemo_l          0.136    0.013   10.431    0.000    0.136    1.000
##    .final_grade     406.930   34.290   11.867    0.000  406.930    0.969
##     pre_int           0.267    0.039    6.793    0.000    1.000    1.000

4.2.2 A2. Interest -> reversed negemo -> Final Grade

d$negemo_rev <- d$negemo_prop * -1

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    negemo_rev ~ pre_int
    final_grade ~ pre_int + negemo_rev
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff2 <- lavaan.survey(m_est, group)
summary(aff2, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 71 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           506
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      29.813      18.162
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.005       0.151
##   Scaling correction factor                                  1.641
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1299.706     597.920
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.987       0.991
##   Tucker-Lewis Index (TLI)                       0.979       0.986
## 
##   Robust Comparative Fit Index (CFI)                         0.993
##   Robust Tucker-Lewis Index (TLI)                            0.989
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -7055.780   -7055.780
##   Loglikelihood unrestricted model (H1)      -7040.873   -7040.873
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               14155.560   14155.560
##   Bayesian (BIC)                             14248.544   14248.544
##   Sample-size adjusted Bayesian (BIC)        14178.713   14178.713
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.051       0.028
##   90 Percent Confidence Interval          0.027  0.075       0.000  0.050
##   P-value RMSEA <= 0.05                          0.447       0.950
## 
##   Robust RMSEA                                               0.036
##   90 Percent Confidence Interval                             0.000  0.072
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.021       0.021
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.521    0.784
##     q4                1.054    0.080   13.173    0.000    0.549    0.682
##     q5                1.005    0.079   12.661    0.000    0.524    0.781
##     q8                1.055    0.086   12.303    0.000    0.549    0.844
##     q10               1.269    0.064   19.693    0.000    0.661    0.757
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   negemo_rev ~                                                          
##     pre_int         -11.533    5.527   -2.087    0.037   -6.006   -0.132
##   final_grade ~                                                         
##     pre_int           4.608    2.303    2.001    0.045    2.400    0.115
##     negemo_rev       -0.087    0.040   -2.203    0.028   -0.087   -0.191
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.316    0.060   72.132    0.000    4.316    6.497
##    .q4                4.332    0.050   86.992    0.000    4.332    5.380
##    .q5                4.277    0.041  105.588    0.000    4.277    6.383
##    .q8                4.334    0.056   78.074    0.000    4.334    6.657
##    .q10               4.146    0.069   60.374    0.000    4.146    4.753
##    .negemo_rev      -66.944    4.644  -14.415    0.000  -66.944   -1.467
##    .final_grade      72.232    3.190   22.645    0.000   72.232    3.463
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.170    0.019    8.859    0.000    0.170    0.385
##    .q4                0.347    0.052    6.699    0.000    0.347    0.535
##    .q5                0.175    0.030    5.781    0.000    0.175    0.389
##    .q8                0.122    0.020    6.255    0.000    0.122    0.288
##    .q10               0.324    0.032   10.071    0.000    0.324    0.426
##    .negemo_rev     2046.400  279.542    7.321    0.000 2046.400    0.983
##    .final_grade     410.760   34.163   12.024    0.000  410.760    0.944
##     pre_int           0.271    0.044    6.198    0.000    1.000    1.000

4.2.3 A3. Interest -> tone -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    Tone ~ pre_int
    final_grade ~ pre_int + Tone
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff4 <- lavaan.survey(m_est, group)
summary(aff4, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 71 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           608
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      18.952      11.831
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.125       0.542
##   Scaling correction factor                                  1.602
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1466.953     695.752
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.996       1.000
##   Tucker-Lewis Index (TLI)                       0.993       1.003
## 
##   Robust Comparative Fit Index (CFI)                         1.000
##   Robust Tucker-Lewis Index (TLI)                            1.002
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -7613.255   -7613.255
##   Loglikelihood unrestricted model (H1)      -7603.779   -7603.779
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               15270.510   15270.510
##   Bayesian (BIC)                             15367.534   15367.534
##   Sample-size adjusted Bayesian (BIC)        15297.689   15297.689
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.027       0.000
##   90 Percent Confidence Interval          0.000  0.052       0.000  0.031
##   P-value RMSEA <= 0.05                          0.928       0.999
## 
##   Robust RMSEA                                               0.000
##   90 Percent Confidence Interval                             0.000  0.047
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.015       0.015
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.518    0.791
##     q4                1.027    0.070   14.604    0.000    0.533    0.666
##     q5                0.973    0.071   13.621    0.000    0.504    0.768
##     q8                1.043    0.077   13.588    0.000    0.540    0.840
##     q10               1.236    0.059   20.827    0.000    0.640    0.749
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   Tone ~                                                                
##     pre_int          -2.103    1.093   -1.923    0.054   -1.090   -0.098
##   final_grade ~                                                         
##     pre_int           5.761    2.195    2.625    0.009    2.986    0.144
##     Tone             -0.008    0.158   -0.053    0.958   -0.008   -0.004
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.052   84.141    0.000    4.354    6.645
##    .q4                4.355    0.044   99.530    0.000    4.355    5.443
##    .q5                4.303    0.037  114.938    0.000    4.303    6.551
##    .q8                4.359    0.048   91.020    0.000    4.359    6.773
##    .q10               4.187    0.061   68.867    0.000    4.187    4.897
##    .Tone             65.994    0.806   81.829    0.000   65.994    5.916
##    .final_grade      78.349   10.485    7.472    0.000   78.349    3.779
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.161    0.017    9.538    0.000    0.161    0.374
##    .q4                0.357    0.051    6.947    0.000    0.357    0.557
##    .q5                0.177    0.026    6.872    0.000    0.177    0.410
##    .q8                0.122    0.017    7.041    0.000    0.122    0.295
##    .q10               0.321    0.029   11.010    0.000    0.321    0.439
##    .Tone            123.254   10.714   11.504    0.000  123.254    0.990
##    .final_grade     420.771   33.785   12.454    0.000  420.771    0.979
##     pre_int           0.269    0.040    6.716    0.000    1.000    1.000

4.2.4 A4. Interest -> posemo & reversed negemo -> Final Grade

library(jmRtools)

d$posemo_negemo_rev <- composite_mean_maker(d, posemo_l, negemo_rev)

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    posemo_negemo_rev ~ pre_int
    final_grade ~ pre_int + posemo_negemo_rev
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff3 <- lavaan.survey(m_est, group)
summary(aff3, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 65 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           608
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      26.597      17.241
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.014       0.189
##   Scaling correction factor                                  1.543
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1485.954     709.984
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.991       0.994
##   Tucker-Lewis Index (TLI)                       0.985       0.990
## 
##   Robust Comparative Fit Index (CFI)                         0.995
##   Robust Tucker-Lewis Index (TLI)                            0.993
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -8096.792   -8096.792
##   Loglikelihood unrestricted model (H1)      -8083.494   -8083.494
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               16237.584   16237.584
##   Bayesian (BIC)                             16334.608   16334.608
##   Sample-size adjusted Bayesian (BIC)        16264.763   16264.763
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.041       0.023
##   90 Percent Confidence Interval          0.018  0.064       0.000  0.044
##   P-value RMSEA <= 0.05                          0.707       0.984
## 
##   Robust RMSEA                                               0.029
##   90 Percent Confidence Interval                             0.000  0.061
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.018       0.018
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.518    0.790
##     q4                1.028    0.070   14.620    0.000    0.532    0.665
##     q5                0.975    0.071   13.722    0.000    0.505    0.769
##     q8                1.045    0.077   13.558    0.000    0.541    0.840
##     q10               1.235    0.059   20.845    0.000    0.640    0.748
## 
## Regressions:
##                       Estimate  Std.Err  z-value  P(>|z|)   Std.lv
##   posemo_negemo_rev ~                                             
##     pre_int             -1.852    2.879   -0.643    0.520   -0.959
##   final_grade ~                                                   
##     pre_int              5.540    2.310    2.399    0.016    2.869
##     posemo_negm_rv      -0.131    0.048   -2.716    0.007   -0.131
##   Std.all
##          
##    -0.038
##          
##     0.138
##    -0.158
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.052   84.141    0.000    4.354    6.645
##    .q4                4.355    0.044   99.530    0.000    4.355    5.443
##    .q5                4.303    0.037  114.938    0.000    4.303    6.551
##    .q8                4.359    0.048   91.020    0.000    4.359    6.773
##    .q10               4.187    0.061   68.867    0.000    4.187    4.897
##    .posemo_negm_rv  -24.814    2.948   -8.416    0.000  -24.814   -0.995
##    .final_grade      74.545    1.847   40.367    0.000   74.545    3.596
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.161    0.017    9.596    0.000    0.161    0.375
##    .q4                0.357    0.051    6.945    0.000    0.357    0.558
##    .q5                0.177    0.026    6.896    0.000    0.177    0.409
##    .q8                0.122    0.017    6.974    0.000    0.122    0.294
##    .q10               0.322    0.029   11.055    0.000    0.322    0.440
##    .posemo_negm_rv  621.174   78.012    7.963    0.000  621.174    0.999
##    .final_grade     410.093   30.020   13.660    0.000  410.093    0.954
##     pre_int           0.268    0.040    6.721    0.000    1.000    1.000

4.2.5 A5. Interest -> tone & reversed negemo -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    tone_negemo_rev =~ Tone + negemo_rev
    final_grade ~ pre_int + tone_negemo_rev
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff5 <- lavaan.survey(m_est, group)

## Error in `rownames<-`(`*tmp*`, value = lav_partable_labels(object@ParTable, : attempt to set 'rownames' on an object with no dimensions

summary(aff5, standardized = T, fit.measures = T)

## Error in summary(aff5, standardized = T, fit.measures = T): object 'aff5' not found

4.2.6 A6. Interest -> Aff eng as a latent factor -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    aff_latent =~ Tone + posemo_l + negemo_rev
    final_grade ~ pre_int + aff_latent
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
aff6 <- lavaan.survey(m_est, group)

## Error in .local(object, ...): lavaan ERROR: model did not converge

summary(aff6, standardized = T, fit.measures = T)

## Error in summary(aff6, standardized = T, fit.measures = T): object 'aff6' not found

4.3 Beh

4.3.1 B1. Interest -> time spent -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    ts_60 ~ pre_int
    final_grade ~ pre_int + ts_60
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
beh1 <- lavaan.survey(m_est, group)
summary(beh1, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 65 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           622
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      19.112      12.152
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.120       0.515
##   Scaling correction factor                                  1.573
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1591.110     848.361
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.996       1.000
##   Tucker-Lewis Index (TLI)                       0.994       1.002
## 
##   Robust Comparative Fit Index (CFI)                         1.000
##   Robust Tucker-Lewis Index (TLI)                            1.001
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -8203.629   -8203.629
##   Loglikelihood unrestricted model (H1)      -8194.073   -8194.073
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               16451.258   16451.258
##   Bayesian (BIC)                             16548.783   16548.783
##   Sample-size adjusted Bayesian (BIC)        16478.936   16478.936
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.027       0.000
##   90 Percent Confidence Interval          0.000  0.052       0.000  0.032
##   P-value RMSEA <= 0.05                          0.931       0.999
## 
##   Robust RMSEA                                               0.000
##   90 Percent Confidence Interval                             0.000  0.047
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.014       0.014
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.516    0.789
##     q4                1.035    0.069   14.970    0.000    0.533    0.669
##     q5                0.988    0.072   13.736    0.000    0.509    0.770
##     q8                1.040    0.074   14.066    0.000    0.536    0.837
##     q10               1.259    0.060   20.958    0.000    0.649    0.752
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   ts_60 ~                                                               
##     pre_int           3.787    1.834    2.065    0.039    1.952    0.087
##   final_grade ~                                                         
##     pre_int           5.263    2.085    2.524    0.012    2.713    0.126
##     ts_60             0.341    0.030   11.307    0.000    0.341    0.355
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.350    0.051   85.038    0.000    4.350    6.658
##    .q4                4.349    0.044   99.740    0.000    4.349    5.455
##    .q5                4.297    0.037  116.502    0.000    4.297    6.499
##    .q8                4.359    0.047   92.214    0.000    4.359    6.807
##    .q10               4.178    0.062   67.582    0.000    4.178    4.842
##    .ts_60            31.497    1.818   17.323    0.000   31.497    1.407
##    .final_grade      66.490    1.940   34.266    0.000   66.490    3.085
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.161    0.016    9.863    0.000    0.161    0.378
##    .q4                0.351    0.050    6.989    0.000    0.351    0.552
##    .q5                0.178    0.025    7.070    0.000    0.178    0.406
##    .q8                0.123    0.017    7.362    0.000    0.123    0.299
##    .q10               0.324    0.028   11.399    0.000    0.324    0.435
##    .ts_60           497.514   66.052    7.532    0.000  497.514    0.992
##    .final_grade     395.160   32.930   12.000    0.000  395.160    0.851
##     pre_int           0.266    0.038    6.956    0.000    1.000    1.000

4.3.2 B2. Interest -> n discussion posts -> Final Grade

Double-check that n is number of discussion posts

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    n ~ pre_int
    final_grade ~ pre_int + n
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
beh2 <- lavaan.survey(m_est, group)
summary(beh2, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 66 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           608
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      20.569      12.734
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.082       0.469
##   Scaling correction factor                                  1.615
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1679.733     806.073
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.995       1.000
##   Tucker-Lewis Index (TLI)                       0.993       1.001
## 
##   Robust Comparative Fit Index (CFI)                         1.000
##   Robust Tucker-Lewis Index (TLI)                            1.000
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -7524.551   -7524.551
##   Loglikelihood unrestricted model (H1)      -7514.267   -7514.267
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               15093.103   15093.103
##   Bayesian (BIC)                             15190.126   15190.126
##   Sample-size adjusted Bayesian (BIC)        15120.281   15120.281
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.031       0.000
##   90 Percent Confidence Interval          0.000  0.055       0.000  0.034
##   P-value RMSEA <= 0.05                          0.895       0.999
## 
##   Robust RMSEA                                               0.000
##   90 Percent Confidence Interval                             0.000  0.050
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.014       0.014
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.519    0.792
##     q4                1.027    0.070   14.683    0.000    0.533    0.666
##     q5                0.971    0.070   13.812    0.000    0.504    0.767
##     q8                1.040    0.076   13.695    0.000    0.540    0.839
##     q10               1.234    0.059   21.011    0.000    0.641    0.749
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   n ~                                                                   
##     pre_int           5.094    1.104    4.613    0.000    2.644    0.231
##   final_grade ~                                                         
##     pre_int           0.967    1.691    0.572    0.567    0.502    0.024
##     n                 0.943    0.100    9.389    0.000    0.943    0.522
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.052   84.141    0.000    4.354    6.645
##    .q4                4.355    0.044   99.530    0.000    4.355    5.443
##    .q5                4.303    0.037  114.938    0.000    4.303    6.551
##    .q8                4.359    0.048   91.020    0.000    4.359    6.773
##    .q10               4.187    0.061   68.867    0.000    4.187    4.897
##    .n                22.796    2.163   10.541    0.000   22.796    1.988
##    .final_grade      56.306    2.896   19.443    0.000   56.306    2.716
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.160    0.016    9.751    0.000    0.160    0.372
##    .q4                0.356    0.051    6.942    0.000    0.356    0.556
##    .q5                0.177    0.025    6.973    0.000    0.177    0.411
##    .q8                0.123    0.018    6.965    0.000    0.123    0.297
##    .q10               0.321    0.029   11.120    0.000    0.321    0.439
##    .n               124.554   15.201    8.194    0.000  124.554    0.947
##    .final_grade     310.049   32.182    9.634    0.000  310.049    0.721
##     pre_int           0.269    0.040    6.777    0.000    1.000    1.000

4.3.3 B3. Interest -> n discussion posts & time spent -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    n_ts_60 =~ n + ts_60
    final_grade ~ pre_int + n_ts_60
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
beh4 <- lavaan.survey(m_est, group)

## Error in `rownames<-`(`*tmp*`, value = lav_partable_labels(object@ParTable, : attempt to set 'rownames' on an object with no dimensions

summary(beh4, standardized = T, fit.measures = T)

## Error in summary(beh4, standardized = T, fit.measures = T): object 'beh4' not found

4.4 Cog

4.4.1 C1. Interest -> Cogproc -> Final Grade

d$cogproc_l <- safe_log(d$cogproc_prop)

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    cogproc_l =~ pre_int 
    final_grade ~ pre_int + cogproc_l
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
cog1 <- lavaan.survey(m_est, group)
summary(cog1, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 35 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         19
## 
##   Number of observations                           622
## 
##   Estimator                                         ML
##   Model Fit Test Statistic                      14.637
##   Degrees of freedom                                 8
##   P-value (Chi-square)                           0.067

## Error in TEST[[2]]: subscript out of bounds

4.4.2 C2. Interest -> Analytic thinking -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    Analytic ~ pre_int
    final_grade ~ pre_int + Analytic
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
cog2 <- lavaan.survey(m_est, group)
summary(cog2, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 72 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           608
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      17.736      11.718
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.168       0.551
##   Scaling correction factor                                  1.514
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1476.362     694.646
##   Degrees of freedom                                21          21
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.997       1.000
##   Tucker-Lewis Index (TLI)                       0.995       1.003
## 
##   Robust Comparative Fit Index (CFI)                         1.000
##   Robust Tucker-Lewis Index (TLI)                            1.002
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)              -7659.429   -7659.429
##   Loglikelihood unrestricted model (H1)      -7650.561   -7650.561
## 
##   Number of free parameters                         22          22
##   Akaike (AIC)                               15362.859   15362.859
##   Bayesian (BIC)                             15459.883   15459.883
##   Sample-size adjusted Bayesian (BIC)        15390.038   15390.038
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.024       0.000
##   90 Percent Confidence Interval          0.000  0.050       0.000  0.031
##   P-value RMSEA <= 0.05                          0.948       0.999
## 
##   Robust RMSEA                                               0.000
##   90 Percent Confidence Interval                             0.000  0.045
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.013       0.013
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.518    0.791
##     q4                1.028    0.070   14.625    0.000    0.532    0.665
##     q5                0.975    0.071   13.734    0.000    0.505    0.768
##     q8                1.044    0.077   13.567    0.000    0.541    0.840
##     q10               1.236    0.059   20.863    0.000    0.640    0.748
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   Analytic ~                                                            
##     pre_int           1.363    1.419    0.961    0.337    0.706    0.058
##   final_grade ~                                                         
##     pre_int           5.436    2.179    2.495    0.013    2.816    0.136
##     Analytic          0.254    0.105    2.413    0.016    0.254    0.149
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.052   84.141    0.000    4.354    6.645
##    .q4                4.355    0.044   99.530    0.000    4.355    5.443
##    .q5                4.303    0.037  114.938    0.000    4.303    6.551
##    .q8                4.359    0.048   91.020    0.000    4.359    6.773
##    .q10               4.188    0.061   68.867    0.000    4.188    4.897
##    .Analytic         53.740    1.022   52.584    0.000   53.740    4.426
##    .final_grade      64.161    5.791   11.079    0.000   64.161    3.095
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.161    0.017    9.600    0.000    0.161    0.375
##    .q4                0.357    0.051    6.939    0.000    0.357    0.557
##    .q5                0.177    0.026    6.878    0.000    0.177    0.409
##    .q8                0.122    0.017    6.993    0.000    0.122    0.294
##    .q10               0.322    0.029   11.051    0.000    0.322    0.440
##    .Analytic        146.909   17.320    8.482    0.000  146.909    0.997
##    .final_grade     411.317   29.488   13.949    0.000  411.317    0.957
##     pre_int           0.268    0.040    6.726    0.000    1.000    1.000

4.4.3 C3. Interest -> Cogproc & analytic thinking -> Final Grade

m <-
  '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    cogproc_analytic ~ cogproc_l + Analytic
    final_grade ~ pre_int + cogproc_analytic
    '

m_est <- sem(m, data = d)

## Error in lav_data_full(data = data, group = group, cluster = cluster, : lavaan ERROR: missing observed variables in dataset: cogproc_analytic

group <- svydesign(ids = ~course_ID, data = d)
cog3 <- lavaan.survey(m_est, group)

## found:  q1 q4 q5 q8 q10 Analytic final_grade 
## expected:  q1 q4 q5 q8 q10 cogproc_analytic final_grade cogproc_l Analytic

## Error in lav_samplestats_from_moments(sample.cov = sample.cov, sample.mean = sample.mean, : lavaan ERROR: rownames of covariance matrix do not match the model!
##   found: q1 q4 q5 q8 q10 Analytic final_grade
##   expected: q1 q4 q5 q8 q10 cogproc_analytic final_grade cogproc_l Analytic

summary(cog3, standardized = T, fit.measures = T)

## Error in summary(cog3, standardized = T, fit.measures = T): object 'cog3' not found

4.5 Overall

m <-
    '
    pre_int =~ q1 + q4 + q5 + q8 + q10
    posemo ~ pre_int
    n ~ pre_int
    cogproc ~ pre_int
    final_grade ~ pre_int + cogproc + n + posemo
    posemo ~~ n
    n ~~ cogproc
    posemo ~~ cogproc
    '

m_est <- sem(m, data = d)
group <- svydesign(ids = ~course_ID, data = d)
ove <- lavaan.survey(m_est, group)
summary(ove, standardized = T, fit.measures = T)

## lavaan 0.6-3 ended normally after 117 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         33
## 
##   Number of observations                           608
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      30.972      22.312
##   Degrees of freedom                                21          21
##   P-value (Chi-square)                           0.074       0.382
##   Scaling correction factor                                  1.388
##     for the Satorra-Bentler correction
## 
## Model test baseline model:
## 
##   Minimum Function Test Statistic             1754.456     831.804
##   Degrees of freedom                                36          36
##   P-value                                        0.000       0.000
## 
## User model versus baseline model:
## 
##   Comparative Fit Index (CFI)                    0.994       0.998
##   Tucker-Lewis Index (TLI)                       0.990       0.997
## 
##   Robust Comparative Fit Index (CFI)                         0.999
##   Robust Tucker-Lewis Index (TLI)                            0.998
## 
## Loglikelihood and Information Criteria:
## 
##   Loglikelihood user model (H0)             -10355.491  -10355.491
##   Loglikelihood unrestricted model (H1)     -10340.005  -10340.005
## 
##   Number of free parameters                         33          33
##   Akaike (AIC)                               20776.982   20776.982
##   Bayesian (BIC)                             20922.518   20922.518
##   Sample-size adjusted Bayesian (BIC)        20817.750   20817.750
## 
## Root Mean Square Error of Approximation:
## 
##   RMSEA                                          0.028       0.010
##   90 Percent Confidence Interval          0.000  0.048       0.000  0.033
##   P-value RMSEA <= 0.05                          0.969       1.000
## 
##   Robust RMSEA                                               0.012
##   90 Percent Confidence Interval                             0.000  0.043
## 
## Standardized Root Mean Square Residual:
## 
##   SRMR                                           0.016       0.016
## 
## Parameter Estimates:
## 
##   Information                                 Expected
##   Information saturated (h1) model          Structured
##   Standard Errors                           Robust.sem
## 
## Latent Variables:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   pre_int =~                                                            
##     q1                1.000                               0.520    0.794
##     q4                1.025    0.070   14.598    0.000    0.533    0.666
##     q5                0.965    0.069   13.903    0.000    0.502    0.765
##     q8                1.036    0.075   13.736    0.000    0.539    0.838
##     q10               1.234    0.059   20.779    0.000    0.642    0.751
## 
## Regressions:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##   posemo ~                                                              
##     pre_int          -0.176    0.125   -1.405    0.160   -0.092   -0.047
##   n ~                                                                   
##     pre_int           5.087    1.102    4.614    0.000    2.646    0.231
##   cogproc ~                                                             
##     pre_int          -1.067    0.394   -2.705    0.007   -0.555   -0.168
##   final_grade ~                                                         
##     pre_int           0.894    1.598    0.559    0.576    0.465    0.022
##     cogproc           0.309    0.353    0.875    0.381    0.309    0.049
##     n                 0.969    0.106    9.174    0.000    0.969    0.536
##     posemo           -1.469    0.587   -2.505    0.012   -1.469   -0.139
## 
## Covariances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##  .posemo ~~                                                             
##    .n                 3.394    1.577    2.151    0.031    3.394    0.155
##  .n ~~                                                                  
##    .cogproc           5.555    3.508    1.584    0.113    5.555    0.153
##  .posemo ~~                                                             
##    .cogproc          -0.158    0.400   -0.396    0.692   -0.158   -0.025
## 
## Intercepts:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                4.354    0.052   84.141    0.000    4.354    6.645
##    .q4                4.355    0.044   99.530    0.000    4.355    5.443
##    .q5                4.303    0.037  114.938    0.000    4.303    6.551
##    .q8                4.359    0.048   91.020    0.000    4.359    6.773
##    .q10               4.188    0.061   68.867    0.000    4.188    4.897
##    .posemo            5.070    0.191   26.565    0.000    5.070    2.578
##    .n                22.796    2.163   10.541    0.000   22.796    1.988
##    .cogproc          13.937    0.507   27.498    0.000   13.937    4.219
##    .final_grade      58.851    5.879   10.011    0.000   58.851    2.839
##     pre_int           0.000                               0.000    0.000
## 
## Variances:
##                    Estimate  Std.Err  z-value  P(>|z|)   Std.lv  Std.all
##    .q1                0.159    0.016    9.619    0.000    0.159    0.369
##    .q4                0.356    0.051    6.918    0.000    0.356    0.556
##    .q5                0.179    0.026    7.001    0.000    0.179    0.415
##    .q8                0.123    0.017    7.150    0.000    0.123    0.298
##    .q10               0.319    0.029   11.141    0.000    0.319    0.436
##    .posemo            3.859    0.850    4.537    0.000    3.859    0.998
##    .n               124.542   15.196    8.196    0.000  124.542    0.947
##    .cogproc          10.604    1.890    5.610    0.000   10.604    0.972
##    .final_grade     300.655   31.845    9.441    0.000  300.655    0.700
##     pre_int           0.271    0.040    6.789    0.000    1.000    1.000

5 Fit stats

If the row is blank, the model did not converge

## lavaan 0.6-3 ended normally after 72 iterations
## 
##   Optimization method                           NLMINB
##   Number of free parameters                         22
## 
##   Number of observations                           605
## 
##   Estimator                                         ML      Robust
##   Model Fit Test Statistic                      25.710      16.841
##   Degrees of freedom                                13          13
##   P-value (Chi-square)                           0.019       0.207
##   Scaling correction factor                                  1.527
##     for the Satorra-Bentler correction

do %>% 
  as_tibble() %>% 
  mutate_if(is.factor, as.character) %>% 
  mutate_at(vars(npar:srmr), as.numeric) %>% 
  mutate_if(is.numeric, round, 3) %>% 
  arrange(model) %>%
  mutate(var = str_sub(as.character(model), end = 3)) %>% 
  select(model, var, everything()) %>% 
  group_by(var) %>% 
  gt::gt()

model	npar	chisq	df	cfi	tli	rmsea	rmsea.ci.lower	rmsea.ci.upper	aic	bic	srmr
aff
aff1	22	25.710	13	0.991	0.986	0.040	0.016	0.063	11038.32	11135.24	0.019
aff2	22	29.813	13	0.987	0.979	0.051	0.027	0.075	14155.56	14248.54	0.021
aff3	22	26.597	13	0.991	0.985	0.041	0.018	0.064	16237.58	16334.61	0.018
aff4	22	18.952	13	0.996	0.993	0.027	0.000	0.052	15270.51	15367.53	0.015
aff5	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
aff6	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
beh
beh1	22	19.112	13	0.996	0.994	0.027	0.000	0.052	16451.26	16548.78	0.014
beh2	22	20.569	13	0.995	0.993	0.031	0.000	0.055	15093.10	15190.13	0.014
beh3	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
cog
cog1	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA
cog2	22	17.736	13	0.997	0.995	0.024	0.000	0.050	15362.86	15459.88	0.013
cog3	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA

2019-06-28-MVS-Analysis