First

Since we have only 70 observations the psychometric analyses must be interpreted with great caution.

Cronbach’s alpha

I calculated Cronbach’s alpha. It turned out that Cronbach’s alpha is smaller than .8, indicating an insufficient reliability.

psych::alpha(df,check.keys=TRUE)[[1]][1]
##  raw_alpha
##  0.6456059
itemmean<-round(apply(df,2,mean,na.rm=TRUE),3)
itemsd<-round(apply(df,2,sd,na.rm=TRUE),3)
item.df<-data.frame(itemmean,
                    itemsd,
                    alpha(df,check.keys=TRUE)[[3]][1], #n
                    round(alpha(df,check.keys=TRUE)[[2]][1],2), #item alpha
                    round(alpha(df,check.keys=TRUE)[[3]][c(2,5)],2)) # item total corr
colnames(item.df)<-c("mean","sd","n","alpha","item total correlation", "cor. item total r")

item.df
##      mean    sd  n alpha item total correlation cor. item total r
## Q1  0.814 0.392 70  0.64                   0.25              0.13
## Q2  0.614 0.490 70  0.62                   0.45              0.32
## Q3  0.814 0.392 70  0.64                   0.28              0.17
## Q4  0.843 0.367 70  0.65                   0.18              0.07
## Q5  0.814 0.392 70  0.64                   0.28              0.17
## Q6  0.414 0.496 70  0.65                   0.28              0.14
## Q7  0.314 0.468 70  0.61                   0.53              0.42
## Q8  0.129 0.337 70  0.63                   0.40              0.31
## Q9  0.500 0.504 70  0.66                   0.15              0.00
## Q10 0.300 0.462 70  0.64                   0.31              0.18
## Q11 0.914 0.282 70  0.64                   0.25              0.17
## Q12 0.657 0.478 70  0.65                   0.26              0.12
## Q13 0.914 0.282 70  0.63                   0.33              0.25
## Q14 0.757 0.432 70  0.64                   0.32              0.20
## Q15 0.929 0.259 70  0.64                   0.26              0.18
## Q16 0.786 0.413 70  0.64                   0.27              0.15
## Q17 0.571 0.498 70  0.62                   0.44              0.31
## Q18 0.814 0.392 70  0.65                   0.14              0.02
## Q19 0.471 0.503 70  0.63                   0.40              0.26
## Q20 0.743 0.440 70  0.64                   0.34              0.22
## Q21 0.343 0.478 70  0.62                   0.48              0.35
## Q22 0.129 0.337 70  0.62                   0.45              0.36
## Q23 0.114 0.320 70  0.62                   0.46              0.38
## Q24 0.057 0.234 70  0.63                   0.39              0.33
## Q25 0.143 0.352 70  0.64                   0.29              0.19
## Q26 0.029 0.168 70  0.64                   0.17              0.12

ICC

One way to split the candidates is using the quartiles of the total test score, with quartile 1 being the lower scoring 25% of the candidates, and quartile 4 being the highest scoring 25% of the candidates. Quartiles 2 and 3 are the two intermediate quarters of the cohort.

Exploratory FA

Below you see the screeplot of the PCA. It turns out that there are two factors that togheter explain about 23% of the total variance. This is a rather small percentage explained variance indicating that the internal validity of the 26 items is not strong.

The plot shows the factor loadings on the first to factor. the yellow and orange items load relatively strong on one of the two axis. The blue and green items have low factor loadings on both axis. The factor loadings are shown in the table.

## 
## Summary of Analysis: 
##    Estimation Method:   ols 
##    Rotation Type:   oblique 
##    Rotation Criterion:   CF-varimax 
##    Test Statistic:   496.572 
##    Degrees of Freedom:   250 
##    Effect numbers of Parameters:   101 
##    P value for perfect fit:   0 
## 
## Rotated Factor Loadings: 
##          F1     F2     F3
## MV1  -0.064  0.483  0.309
## MV2   0.075  0.114  0.500
## MV3   0.128  0.525  0.038
## MV4  -0.031 -0.310 -0.019
## MV5  -0.097 -0.150  0.611
## MV6  -0.130  0.199  0.539
## MV7   0.394  0.013  0.448
## MV8   0.337  0.338  0.250
## MV9  -0.112  0.214  0.076
## MV10  0.071  0.072  0.207
## MV11  0.179 -0.230  0.134
## MV12  0.351  0.436 -0.234
## MV13  0.069 -0.018  0.367
## MV14  0.082  0.139  0.162
## MV15  0.109 -0.448  0.109
## MV16  0.304 -0.092 -0.085
## MV17  0.550 -0.052 -0.158
## MV18  0.131 -0.515 -0.059
## MV19  0.459 -0.234 -0.124
## MV20  0.233 -0.415  0.260
## MV21  0.583  0.081 -0.116
## MV22  0.546 -0.007  0.111
## MV23  0.499  0.057  0.090
## MV24  0.510 -0.127  0.153
## MV25 -0.013 -0.050  0.384
## MV26  0.141  0.066  0.001
## 
## Factor Correlations: 
##        F1     F2    F3
## F1  1.000 -0.022 0.061
## F2 -0.022  1.000 0.067
## F3  0.061  0.067 1.000
## Importance of components:
##                           PC1    PC2     PC3     PC4     PC5     PC6     PC7
## Standard deviation     1.7903 1.6801 1.49892 1.35349 1.30275 1.19116 1.17595
## Proportion of Variance 0.1233 0.1086 0.08641 0.07046 0.06528 0.05457 0.05319
## Cumulative Proportion  0.1233 0.2318 0.31826 0.38872 0.45399 0.50856 0.56175
##                            PC8     PC9    PC10    PC11    PC12    PC13    PC14
## Standard deviation     1.12268 1.09328 1.03676 0.98329 0.94500 0.82910 0.81698
## Proportion of Variance 0.04848 0.04597 0.04134 0.03719 0.03435 0.02644 0.02567
## Cumulative Proportion  0.61023 0.65620 0.69754 0.73473 0.76907 0.79551 0.82118
##                           PC15   PC16    PC17    PC18    PC19    PC20    PC21
## Standard deviation     0.81229 0.7800 0.74725 0.71073 0.69242 0.62098 0.60271
## Proportion of Variance 0.02538 0.0234 0.02148 0.01943 0.01844 0.01483 0.01397
## Cumulative Proportion  0.84656 0.8700 0.89144 0.91087 0.92931 0.94414 0.95811
##                           PC22   PC23   PC24    PC25    PC26
## Standard deviation     0.54616 0.5325 0.4386 0.40289 0.39075
## Proportion of Variance 0.01147 0.0109 0.0074 0.00624 0.00587
## Cumulative Proportion  0.96958 0.9805 0.9879 0.99413 1.00000