UG_CWTK feasibility study survey
Samantha Bouwmeester
3/1/2022
SCWS
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.747. This value indicates insufficient reliability. The scalability coefficient H is 0.198, (0.035). This value indicates a very weak scale, i.e. students can not be reliable ordered based on their total scores on the scws,
Item-analyses
The item scalability coefficients are low. Items 2 and 3 do not discriminate at all between different levels of well-being.
Distribution of total score
The summary of the total score shows that the minimum score is 11 and the maximum score is 44. The median is 31 and the mean is 30.4. The skewness is -0.59 (se= 0.25). This indicates significant skewness: the total scores on the scws are negatively skewed.
Math
Number identification (nid)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.891. This value indicates good reliability. The scalability coefficient H is 0.659, (0.064). This value indicates a very strong scale, i.e. students can be reliable ordered based on their total scores on the nid.
Item-analyses
The item scalability coefficients are all high. Items 1, 4, and 9 have proportions correct larger than .95. These items are too easy.
Distribution of total score
The summary of the total score shows that the minimum score is 1 and the maximum score is 14. The median is 12 and the mean is 11.26. The skewness is -1.27 (se= 0.25). This indicates significant negative skewness.
Quantity discrimination (qds)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.818. This value indicates sufficient reliability. The scalability coefficient H is 0.483, (0.080). This value indicates a strong scale, i.e. students can be reliable ordered based on their total scores on the qds
Item-analyses
The item scalability coefficients are all high. Items 1, and 3 have proportions correct larger than .95. These items are too easy.
Distribution of total score
The summary of the total score shows that the minimum score is 1 and the maximum score is 10. The median is 9 and the mean is 8.4. The skewness is -1.59 (se= 0.25). This indicates significant negative skewness.
Missing numbers (mis)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.844. This value indicates sufficient reliability. The scalability coefficient H is 0.494, (0.052). This value indicates a medium strong scale, i.e. students can be reliable ordered based on their total scores on the mis.
Item-analyses
The item scalability coefficients are moderate to high. Items 2 does not discriminate well between different levels of mis.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 10. The median is 7 and the mean is 6.4. The skewness is -0.52 (se= 0.25). This indicates a just significant negative skewness.
Addition (Add)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.966. This value indicates very good reliability. The scalability coefficient H is 0.711, (0.042). This value indicates a very strong scale, i.e. students can be reliable ordered based on their total scores on the add.
Item-analyses
The item scalability coefficients are high. Only Item 3 does not discriminate well between different levels of add.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 20. The median is 8 and the mean is 10.71. The skewness is 0.15 (se= 0.25). This indicates no significant skewness. However, a relative large percentage has a maximum score of 20 indicating a ceiling effect.
Subtraction (sub)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.977. This value indicates very high reliability. The scalability coefficient H is 0.820, (0.033). This value indicates a very strong scale, i.e. students can be reliable ordered based on their total scores on the sub,
Item-analyses
The item scalability coefficients are all high.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 20. The median is 4 and the mean is 7.85. The skewness is 0.65 (se= 0.25). This indicates significant positive skewness. The histogram shows that there are relatively many students who have a score below 3, but also relatively many students who have a maximum score of 20.
Word Problems (prb)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.761. This value indicates insufficient reliability. The scalability coefficient H is 0.460, (0.069). This value indicates a medium scale, i.e. students can be reliable ordered based on their total scores on the prb.
Item-analyses
The item scalability coefficients are moderate. Items 1 and 4 do not discriminate well between different levels of well-being.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 6. The median is 4 and the mean is 3.98. The skewness is -0.7 (se= 0.25). This indicates significant negative skewness.
Literacy
Letter knowledge (LID)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.927. This value indicates very high reliability. The scalability coefficient H is 0.532, (0.077). This value indicates a strong scale, i.e. students can be reliable ordered based on their total scores on the LID,
Item-analyses
The item scalability coefficients are medium to large. Items 1 and 26 do not discriminate well between different levels of LID. Item 14 has a proportions correct larger than .95. This items is too easy.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 26. The median is 23 and the mean is 21.33. The skewness is -2.01 (se= 0.25). This indicates significant negative skewness.
Phonological Awareness (pho)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.932. This value indicates very high reliability. The scalability coefficient H is 0.762, (0.054). This value indicates a very strong scale, i.e. students can be reliable ordered based on their total scores on the pho.
Item-analyses
The item scalability coefficients are high.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 10. The median is 10 and the mean is 7.79. The skewness is -1.32 (se= 0.25). This indicates significant negative skewness. There seem to be a ceiling effect.
Most used words (muw)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.95. This value indicates very high reliability. The scalability coefficient H is 0.696, (0.043). This value indicates a strong scale, i.e. students can be reliable ordered based on their total scores on the muw.
Item-analyses
The item scalability coefficients are high.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 20. The median is 4 and the mean is 5.7. The skewness is 0.93 (se= 0.25). This indicates significant positive skewness. There is a floor effect.
Decodable words (non)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.954. This value indicates very high reliability. The scalability coefficient H is 0.671, (0.048). This value indicates a strong scale, i.e. students can be reliable ordered based on their total scores on the non,
Item-analyses
The item scalability coefficients are all high. The proportion correct of the items show that the items of this scale are all rather difficult. Most proportions correct are below .25.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 20. The median is 1 and the mean is 3.68. The skewness is 1.58 (se= 0.25). This indicates significant positive skewness. There is a floor effect.
Reading words (rcp)
The histogram shows the number of words read in 60 seconds. The histogram and the summary shows that the number of words read in 60s is very much positively skewed. 65 children have read less than 5 words correctly. 27 children had more than 4 words read correctly.
rcp[rcp==999]<-NA # recode missing values
# For the reliability analysis two people with missing values were removed.
hist(rcp[,1])
summary(rcp[,1])## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.000 0.000 2.000 4.717 6.000 45.000The histogram below shows the distribution of the number of words correctly read in the passage. It is very positively skewed as well. There is floor effect. This is confirmed by the summary statistics, the median is 2, indicating that 50% of the people read 2 or less words correctly.
hist(rcp[,3])
summary(rcp[,3])## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.00 0.00 2.00 11.36 10.00 99.00Reading comprehension (com)
There are no valid observations for reading comprehension.
Listening comprehension (lis)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.508. This value indicates insufficient reliability. The scalability coefficient H is 0.328, (0.081). This value indicates a weak scale, i.e. students can not be reliable ordered based on their total scores on the lis. The item proportions correct show that the items are very difficult. Items 3, 4 and 5 have proportions correct below .1.
Item-analyses
The item scalability coefficients are low to medium. Items 1 and 2 do not discriminate well between different levels of lis.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 3. The median is 0 and the mean is 0.5. The skewness is 1.66 (se= 0.25). This indicates significant positive skewness. There is a floor effect.
Writing (wrt)
Internal consistency and scale quality
The reliability coefficient lambda.2 is 0.821. This value indicates sufficient reliability. The scalability coefficient H is 0.525, (0.063). This value indicates a medium to strong scale, i.e. students can be reliable ordered based on their total scores on the wrt. The item proportions correct show that many items are rather difficult (p<.2). Items 8 and 9 have proportions correct below .1.
Item-analyses
The item scalability coefficients are medium to high Item 2 does not discriminate well between different levels of wrt.
Distribution of total score
The summary of the total score shows that the minimum score is 0 and the maximum score is 10. The median is 1 and the mean is 2.11. The skewness is 1.38 (se= 0.25). This indicates significant positive skewness. There is a floor effect.
Confirmatory Factor Analysis
We would like to do a CFA on all math sub test items. Unfortunately, however, the sample is only 92 and this is too small to do a CFA on such a large number of items. Instead we did a CFA on the total scores of the sub tests. Note, however, that even for this analysis the number of participants is in fact to small.
Math
The correlation matrix of the 6 sub scales show that the size of the correlations differ greatly. There are large correlations between nid and mis, add and sub and prb has strong correlations with all other sub tests.
The results of the CFA shows no fit of the one factor model. The CFI is .732 (must be larger than .95), the RMSEA is .277 (must be smaller than .05) and the SRMR is .149 (must be smaller than .08).
library(lavaan)## This is lavaan 0.6-9
## lavaan is FREE software! Please report any bugs.totalsMath<-data.frame(total_nid,total_qds,total_mis,total_add,total_sub,total_prb)
round(cor(totalsMath),3)## total_nid total_qds total_mis total_add total_sub total_prb
## total_nid 1.000 0.481 0.607 0.298 0.212 0.501
## total_qds 0.481 1.000 0.442 0.387 0.312 0.459
## total_mis 0.607 0.442 1.000 0.401 0.265 0.456
## total_add 0.298 0.387 0.401 1.000 0.808 0.547
## total_sub 0.212 0.312 0.265 0.808 1.000 0.536
## total_prb 0.501 0.459 0.456 0.547 0.536 1.000Math<-'Math =~ total_nid+total_qds+total_mis+total_add+total_sub+total_prb'
res_Math<- cfa(Math, data=totalsMath,std.lv=TRUE)
summary(res_Math,fit.measures=TRUE)## lavaan 0.6-9 ended normally after 25 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 12
##
## Number of observations 92
##
## Model Test User Model:
##
## Test statistic 72.341
## Degrees of freedom 9
## P-value (Chi-square) 0.000
##
## Model Test Baseline Model:
##
## Test statistic 251.439
## Degrees of freedom 15
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.732
## Tucker-Lewis Index (TLI) 0.554
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -1392.394
## Loglikelihood unrestricted model (H1) -1356.224
##
## Akaike (AIC) 2808.789
## Bayesian (BIC) 2839.050
## Sample-size adjusted Bayesian (BIC) 2801.171
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.277
## 90 Percent confidence interval - lower 0.219
## 90 Percent confidence interval - upper 0.337
## P-value RMSEA <= 0.05 0.000
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.149
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## Math =~
## total_nid 1.223 0.322 3.794 0.000
## total_qds 0.983 0.217 4.529 0.000
## total_mis 1.360 0.301 4.514 0.000
## total_add 6.680 0.636 10.502 0.000
## total_sub 6.646 0.701 9.485 0.000
## total_prb 1.206 0.179 6.749 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .total_nid 7.850 1.184 6.630 0.000
## .total_qds 3.426 0.523 6.556 0.000
## .total_mis 6.607 1.008 6.557 0.000
## .total_add 9.650 3.498 2.759 0.006
## .total_sub 17.627 4.153 4.245 0.000
## .total_prb 1.935 0.315 6.153 0.000
## Math 1.000Literacy
The literacy test has 6 sub scales the could be included in the CFA. Reading words had only two items that were measured on a different scale and Sub scale reading comprehension had no valid observations.
The correlation matrix of the 6 sub scales show that the size of the correlations differ considerably. There are large correlations between muw and non, muw and wrt, and wrt and non.
The results of the CFA shows a fit of the one factor model. The CFI is .988 (must be larger than .95), the RMSEA is .055 (must be smaller than .05) and the SRMR is .054 (must be smaller than .08).
library(lavaan)
totalsLit<-data.frame(total_LID,total_pho
,total_muw,total_non,total_lis,total_wrt)
round(cor(totalsLit),3)## total_LID total_pho total_muw total_non total_lis total_wrt
## total_LID 1.000 0.292 0.494 0.388 0.304 0.388
## total_pho 0.292 1.000 0.315 0.330 0.256 0.201
## total_muw 0.494 0.315 1.000 0.797 0.384 0.695
## total_non 0.388 0.330 0.797 1.000 0.274 0.620
## total_lis 0.304 0.256 0.384 0.274 1.000 0.171
## total_wrt 0.388 0.201 0.695 0.620 0.171 1.000Lit<-'Lit =~ total_LID+total_pho+total_muw+total_non+total_lis+total_wrt'
res_Lit<- cfa(Lit, data=totalsLit,std.lv=TRUE)
summary(res_Lit,fit.measures=TRUE)## lavaan 0.6-9 ended normally after 24 iterations
##
## Estimator ML
## Optimization method NLMINB
## Number of model parameters 12
##
## Number of observations 92
##
## Model Test User Model:
##
## Test statistic 11.478
## Degrees of freedom 9
## P-value (Chi-square) 0.244
##
## Model Test Baseline Model:
##
## Test statistic 217.274
## Degrees of freedom 15
## P-value 0.000
##
## User Model versus Baseline Model:
##
## Comparative Fit Index (CFI) 0.988
## Tucker-Lewis Index (TLI) 0.980
##
## Loglikelihood and Information Criteria:
##
## Loglikelihood user model (H0) -1322.301
## Loglikelihood unrestricted model (H1) -1316.562
##
## Akaike (AIC) 2668.602
## Bayesian (BIC) 2698.864
## Sample-size adjusted Bayesian (BIC) 2660.985
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.055
## 90 Percent confidence interval - lower 0.000
## 90 Percent confidence interval - upper 0.136
## P-value RMSEA <= 0.05 0.409
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.054
##
## Parameter Estimates:
##
## Standard errors Standard
## Information Expected
## Information saturated (h1) model Structured
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|)
## Lit =~
## total_LID 2.784 0.541 5.147 0.000
## total_pho 1.081 0.329 3.289 0.001
## total_muw 5.843 0.494 11.836 0.000
## total_non 4.510 0.478 9.434 0.000
## total_lis 0.315 0.084 3.737 0.000
## total_wrt 1.691 0.216 7.813 0.000
##
## Variances:
## Estimate Std.Err z-value P(>|z|)
## .total_LID 21.467 3.248 6.610 0.000
## .total_pho 8.777 1.305 6.724 0.000
## .total_muw 2.791 1.992 1.401 0.161
## .total_non 9.226 1.814 5.085 0.000
## .total_lis 0.564 0.084 6.704 0.000
## .total_wrt 2.608 0.426 6.122 0.000
## Lit 1.000Demographics
## Conditional item response (column) probabilities,
## by outcome variable, for each class (row)
##
## $Mat
## Pr(1) Pr(2)
## class 1: 0.1014 0.8986
## class 2: 0.1047 0.8953
## class 3: 0.3311 0.6689
##
## $Mattress
## Pr(1) Pr(2)
## class 1: 0.2044 0.7956
## class 2: 0.0398 0.9602
## class 3: 0.0000 1.0000
##
## $Cupboard
## Pr(1) Pr(2)
## class 1: 1.0000 0.0000
## class 2: 0.4355 0.5645
## class 3: 0.6252 0.3748
##
## $Bicycle
## Pr(1) Pr(2)
## class 1: 1.0000 0.0000
## class 2: 0.7673 0.2327
## class 3: 1.0000 0.0000
##
## $MobilePhone
## Pr(1) Pr(2)
## class 1: 0.4331 0.5669
## class 2: 0.0000 1.0000
## class 3: 0.3612 0.6388
##
## $Radio_not_phone
## Pr(1) Pr(2)
## class 1: 0.8039 0.1961
## class 2: 0.0000 1.0000
## class 3: 0.8585 0.1415
##
## $Sofa.sets
## Pr(1) Pr(2)
## class 1: 1.0000 0.0000
## class 2: 0.3474 0.6526
## class 3: 0.4772 0.5228
##
## $Solar.Panels.Electricity
## Pr(1) Pr(2)
## class 1: 1.0000 0.0000
## class 2: 0.3415 0.6585
## class 3: 0.5230 0.4770
##
## $Motorcycle
## Pr(1) Pr(2)
## class 1: 0.9789 0.0211
## class 2: 0.7302 0.2698
## class 3: 0.9516 0.0484
##
## $Television
## Pr(1) Pr(2)
## class 1: 1.0000 0.0000
## class 2: 0.7714 0.2286
## class 3: 0.7974 0.2026
##
## $None
## Pr(1) Pr(2)
## class 1: 0.9744 0.0256
## class 2: 1.0000 0.0000
## class 3: 1.0000 0.0000
##
## Estimated class population shares
## 0.4335 0.2865 0.28
##
## Predicted class memberships (by modal posterior prob.)
## 0.4778 0.3 0.2222
##
## =========================================================
## Fit for 3 latent classes:
## =========================================================
## number of observations: 90
## number of estimated parameters: 35
## residual degrees of freedom: 55
## maximum log-likelihood: -384.6583
##
## AIC(3): 839.3166
## BIC(3): 926.8099
## G^2(3): 146.4336 (Likelihood ratio/deviance statistic)
## X^2(3): 434.3908 (Chi-square goodness of fit)
## ## [1] 0.4335266 0.2864755 0.2799979A latent class analysis was done to evaluate whether the household items could be used as a proxy for SES. We expected that certain combinations of household items would indicate the SES level.
Results
The results of a three class latent class showed that class two was the largest class with class size is .56. Class 1 has class size .23 and class 3 has size .21.
The item Mat did not discriminate between classes: all classes have high probability to have a mat. The same was true for item mattress: all classes have high probability to have a mattress. Class one has high probability to have a cupboard, while this probability was very low in class 2 and .23 in class three. Bicycles were not often present, but they were most likely in class 3 and class 1. Class 3 and class 1 had very high probability to have a mobile phone, half of the households that belong to class two has a mobile phone. A radio was most likely to be present in class 1, two-third of the households in class 1 has a radio but it was not likely to be present in households of class 2. A sofa set and solar/panels/electricity were likely to be present in class 1 and to a less degree in class 3. Only households in class 1 sometimes have a motorcycle and a television. The other classes didn’t have these items. ## Conclusion
Class two definitely can be defined as the low SES group. The difference between this class and the other classes is large. When you want to distinguish between a medium and high SES group I would say that class 3 is the medium SES group and class 1 is the high SES group. With respect to the individual items two of the items are not useful to distinguish SES and may be dropped. These items are mat and mattress. And – as a consequence – the option “none of the items” can also be dropped.