Chad validation
Samantha Bouwmeester
1/12/2020
First
Since we have only 100 observations the psychometric analyses must be interpreted with caution.
Cronbach’s alpha
I calculated Cronbach’s alpha. It turned out that Cronbach’s alpha is smaller than .8, indicating an insufficient reliability.
alpha(df,check.keys=TRUE)[[1]][1]## raw_alpha
## 0.5771895itemmean<-round(apply(df,2,mean,na.rm=TRUE),3)
itemsd<-round(apply(df,2,sd,na.rm=TRUE),3)
item.df<-data.frame(itemmean,
itemsd,
alpha(df,check.keys=TRUE)[[3]][1], #n
round(alpha(df,check.keys=TRUE)[[2]][1],2), #item alpha
round(alpha(df,check.keys=TRUE)[[3]][c(2,5)],2)) # item total corr
colnames(item.df)<-c("mean","sd","n","alpha","item total correlation", "cor. item total r")
item.df## mean sd n alpha item total correlation cor. item total r
## Q1 0.900 0.302 100 0.56 0.35 0.29
## Q2 0.840 0.368 100 0.57 0.24 0.14
## Q3 0.950 0.219 100 0.58 0.04 -0.02
## Q4 0.650 0.592 100 0.57 0.30 0.14
## Q5 0.859 0.350 99 0.57 0.26 0.17
## Q6 0.600 0.492 100 0.55 0.40 0.27
## Q7 0.340 0.807 100 0.57 0.39 0.18
## Q8 0.110 0.314 100 0.57 0.27 0.19
## Q9 0.270 0.446 100 0.59 0.10 -0.03
## Q10 0.280 0.451 100 0.58 0.21 0.08
## Q11 0.750 0.435 100 0.56 0.35 0.23
## Q12 0.840 0.368 100 0.58 0.13 0.03
## Q13 0.960 0.315 100 0.58 0.13 0.04
## Q14 0.860 0.349 100 0.57 0.23 0.13
## Q15 0.760 0.429 100 0.54 0.46 0.36
## Q16 0.360 0.482 100 0.54 0.51 0.40
## Q17 0.280 0.451 100 0.54 0.48 0.38
## Q18 0.510 0.502 100 0.55 0.43 0.31
## Q19 0.560 1.166 100 0.62 0.41 0.08
## Q20 0.530 0.502 100 0.55 0.44 0.32
## Q21 0.140 0.349 100 0.56 0.35 0.26
## Q22 0.060 0.239 100 0.56 0.38 0.32
## Q23 0.040 0.197 100 0.57 0.26 0.21
## Q24 0.070 0.256 100 0.57 0.24 0.17
## Q25 0.070 0.256 100 0.57 0.26 0.19
## Q26 0.071 0.258 99 0.57 0.18 0.11ICC
One way to split the candidates is using the quartiles of the total test score, with quartile 1 being the lower scoring 25% of the candidates, and quartile 4 being the highest scoring 25% of the candidates. Quartiles 2 and 3 are the two intermediate quarters of the cohort.
Exploratory FA
Below you see the screeplot of the PCA. It turns out that there are two factors that togheter explain about 22% of the total variance. This is a rather small percentage explained variance indicating that the internal validity of the 26 items is not strong.
The plot shows the factor loadings on the first to factor. the yellow and orange items load relatively strong on one of the two axis. The blue and green items have low factor loadings on both axis. The factor loadings are shown in the table.
##
## Summary of Analysis:
## Estimation Method: ols
## Rotation Type: oblique
## Rotation Criterion: CF-varimax
## Test Statistic: 366.051
## Degrees of Freedom: 250
## Effect numbers of Parameters: 101
## P value for perfect fit: 0
##
## Rotated Factor Loadings:
## F1 F2 F3
## MV1 0.016 0.370 0.057
## MV2 0.093 0.060 -0.062
## MV3 0.103 0.014 0.465
## MV4 0.074 0.124 -0.003
## MV5 -0.164 0.316 0.174
## MV6 0.105 0.264 -0.037
## MV7 0.172 0.168 -0.099
## MV8 0.016 0.185 -0.368
## MV9 -0.040 0.074 0.238
## MV10 0.066 0.204 0.151
## MV11 0.250 0.222 0.201
## MV12 0.050 -0.026 0.482
## MV13 0.078 0.106 0.378
## MV14 -0.100 0.255 0.334
## MV15 -0.051 0.534 0.157
## MV16 -0.005 0.605 0.039
## MV17 0.008 0.562 -0.138
## MV18 0.026 0.458 -0.123
## MV19 0.016 0.130 0.183
## MV20 -0.002 0.536 0.003
## MV21 0.125 0.288 -0.309
## MV22 0.668 0.124 -0.041
## MV23 0.506 0.019 -0.337
## MV24 0.457 0.054 -0.040
## MV25 0.678 -0.045 0.050
## MV26 0.689 -0.117 0.073
##
## Factor Correlations:
## F1 F2 F3
## F1 1.000 0.045 -0.162
## F2 0.045 1.000 0.009
## F3 -0.162 0.009 1.000## Importance of components:
## PC1 PC2 PC3 PC4 PC5 PC6 PC7
## Standard deviation 1.7459 1.6609 1.38168 1.25824 1.21236 1.1425 1.11952
## Proportion of Variance 0.1172 0.1061 0.07342 0.06089 0.05653 0.0502 0.04821
## Cumulative Proportion 0.1172 0.2233 0.29676 0.35765 0.41418 0.4644 0.51259
## PC8 PC9 PC10 PC11 PC12 PC13 PC14
## Standard deviation 1.11473 1.06141 1.02426 0.97649 0.97066 0.94076 0.90292
## Proportion of Variance 0.04779 0.04333 0.04035 0.03667 0.03624 0.03404 0.03136
## Cumulative Proportion 0.56038 0.60371 0.64406 0.68074 0.71697 0.75101 0.78237
## PC15 PC16 PC17 PC18 PC19 PC20 PC21
## Standard deviation 0.8964 0.84316 0.82422 0.7766 0.74079 0.69904 0.63708
## Proportion of Variance 0.0309 0.02734 0.02613 0.0232 0.02111 0.01879 0.01561
## Cumulative Proportion 0.8133 0.84062 0.86674 0.8899 0.91105 0.92984 0.94545
## PC22 PC23 PC24 PC25 PC26
## Standard deviation 0.59009 0.58089 0.54831 0.48698 0.44140
## Proportion of Variance 0.01339 0.01298 0.01156 0.00912 0.00749
## Cumulative Proportion 0.95884 0.97182 0.98339 0.99251 1.00000
