nara vs nasjonal prøve

Missing

We handle missing by using FIML in package lavaan. We assume missing is at random, is this tenable?

library(lavaan); library(tidyverse)

This is lavaan 0.6-16
lavaan is FREE software! Please report any bugs.

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.3     ✔ readr     2.1.4
✔ forcats   1.0.0     ✔ stringr   1.5.0
✔ ggplot2   3.4.4     ✔ tibble    3.2.1
✔ lubridate 1.9.2     ✔ tidyr     1.3.0
✔ purrr     1.0.2     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

dd <- haven::read_sav("Dataset Njål.sav") %>% scale()
naniar::vis_miss(dd <- data.frame(dd))

The two reading outcomes are only weakly related!

cor(dd$Nara_read, dd$Skalapoeng)

[1] 0.3189501

qplot(dd$Nara_read, dd$Skalapoeng)+geom_abline(slope=1)

Warning: `qplot()` was deprecated in ggplot2 3.4.0.

Running two models separately

Effect patterns are not at all similar.

m1 <- "Nara_read~ Avkoding+binde+vokabular+Lytte"
f1 <- sem(m1, dd, missing = 'fiml', fixed.x = F)


m2 <- "Skalapoeng~ Avkoding+binde+vokabular+Lytte"
f2 <- sem(m2, dd, missing = 'fiml', fixed.x = F)

parameterestimates(f1) %>% dplyr::filter(op=="~") %>% select(c(1:4,7))

        lhs op       rhs   est pvalue
1 Nara_read  ~  Avkoding 0.032  0.498
2 Nara_read  ~     binde 0.192  0.000
3 Nara_read  ~ vokabular 0.090  0.127
4 Nara_read  ~     Lytte 0.587  0.000

parameterestimates(f2) %>% dplyr::filter(op=="~")%>% select(c(1:4,7))

         lhs op       rhs   est pvalue
1 Skalapoeng  ~  Avkoding 0.291  0.000
2 Skalapoeng  ~     binde 0.190  0.003
3 Skalapoeng  ~ vokabular 0.299  0.000
4 Skalapoeng  ~     Lytte 0.106  0.072

Formal tests. Running both regressions simultaneously

m1 <- "Nara_read~ n_avk*Avkoding+n_bin*binde+n_vok*vokabular+n_Lyt*Lytte"
m2 <- "Skalapoeng~ s_avk*Avkoding+s_bin*binde+s_vok*vokabular+s_Lyt*Lytte"
f <- sem(paste(m1,";", m2), dd, missing = 'fiml', fixed.x = F)
parameterestimates(f) %>% dplyr::filter(op=="~")%>% select(c(1:3,5, 8))

         lhs op       rhs   est pvalue
1  Nara_read  ~  Avkoding 0.033  0.483
2  Nara_read  ~     binde 0.187  0.001
3  Nara_read  ~ vokabular 0.099  0.089
4  Nara_read  ~     Lytte 0.583  0.000
5 Skalapoeng  ~  Avkoding 0.290  0.000
6 Skalapoeng  ~     binde 0.186  0.003
7 Skalapoeng  ~ vokabular 0.300  0.000
8 Skalapoeng  ~     Lytte 0.111  0.057

Testing whether the effects are similar

m1 <- "Nara_read~ n_avk*Avkoding+n_bin*binde+n_vok*vokabular+n_Lyt*Lytte"
m2 <- "Skalapoeng~ s_avk*Avkoding+s_bin*binde+s_vok*vokabular+s_Lyt*Lytte"
f <- sem(paste(m1,";", m2), dd, missing = 'fiml', fixed.x = F)
f.const <- sem(paste(m1,";", m2, ";n_avk==s_avk"), dd, missing = 'fiml', fixed.x = F)
anova(f, f.const)


Chi-Squared Difference Test

        Df    AIC    BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)    
f        0 3596.1 3689.9  0.000                                          
f.const  1 3606.5 3696.9 12.455     12.455 0.21893       1  0.0004169 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

f.const <- sem(paste(m1,";", m2, ";n_bin==s_bin"), dd, missing = 'fiml', fixed.x = F)
anova(f, f.const)


Chi-Squared Difference Test

        Df    AIC    BIC Chisq Chisq diff RMSEA Df diff Pr(>Chisq)
f        0 3596.1 3689.9 0e+00                                    
f.const  1 3594.1 3684.4 1e-04 0.00014691     0       1     0.9903

f.const <- sem(paste(m1,";", m2, ";n_vok==s_vok"), dd, missing = 'fiml', fixed.x = F)
anova(f, f.const)


Chi-Squared Difference Test

        Df    AIC    BIC Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)  
f        0 3596.1 3689.9 0.000                                        
f.const  1 3599.2 3689.6 5.134      5.134 0.13152       1    0.02346 *
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

f.const <- sem(paste(m1,";", m2, ";n_Lyt==s_Lyt"), dd, missing = 'fiml', fixed.x = F)
anova(f, f.const)


Chi-Squared Difference Test

        Df    AIC    BIC  Chisq Chisq diff   RMSEA Df diff Pr(>Chisq)    
f        0 3596.1 3689.9  0.000                                          
f.const  1 3627.6 3718.0 33.557     33.557 0.36908       1   6.92e-09 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Conclusion: binde has the same effect on Nara as on Skalapoeng (when taking the other 3 into account). Vokabular has a quite similar (but the difference is significant) effect. The other two predictors are clearly different in their effect on Nara and on Skalapoeng