Examiner IRT abridged
Mark Sanderson-cimino
2023-04-20
Project summary
Create executive factor score using IRT (MIRT package) for combined TabCat and Examiner
Variables:
Tabcat: Running dots, dot coutning, flanker, set shifting, match UDS: Animal fluency, vegitable fluency, L words, F words
Issues:
We have more fluency data than Tabcat data (see descriptives). Should we restrict data in some way?
Data is pretty good after binning, but there is some skewness. Transforming reduces the skewness and also leads to more bins for several variables. Is it worth it to do this transformation?
Data: Below I provide some data looking at these issues. I ran most analyses with: the full dataset and transformed/restricted data (also reduced to 1+ TabCat)
Outline
- Descriptives (raw data)
- EFA summary
- IRT models and comparisons
- EF factor Associations
Descriptives
Lookup table full data. Restricted data is similar.
## recode_score min_rundots max_rundots min_dotcount max_dotcount min_flanker
## 1 1 0 70 0 4 4000.0
## 2 2 95 140 5 6 4687.5
## 3 3 145 210 7 7 5116.0
## 4 4 215 285 8 9 5620.0
## 5 5 290 390 10 11 6135.0
## 6 6 NA NA 12 13 6680.0
## 7 7 NA NA 14 15 7230.0
## 8 8 NA NA 16 17 7737.7
## 9 9 NA NA 18 19 8290.0
## 10 10 NA NA 20 21 8810.0
## 11 11 NA NA 22 22 NA
## 12 12 NA NA 23 27 NA
## max_flanker min_setshift max_setshift min_match max_match min_animals
## 1 4380.0 0.0 2500.0 0 5 1
## 2 5049.8 2968.8 3780.0 6 10 5
## 3 5592.7 3860.0 4840.0 11 15 8
## 4 6090.0 4889.7 5950.0 16 20 11
## 5 6650.0 6015.2 7017.7 21 25 14
## 6 7180.0 7035.5 8090.0 26 31 18
## 7 7730.0 8113.1 10000.0 32 36 21
## 8 8260.0 NA NA 37 41 24
## 9 8790.0 NA NA 42 46 27
## 10 9820.0 NA NA 47 51 30
## 11 NA NA NA 52 57 34
## 12 NA NA NA 58 68 37
## max_animals min_veg max_veg min_fwords max_fwords min_lwords max_lwords
## 1 4 1 3 1 3 1 3
## 2 7 4 5 4 5 4 5
## 3 10 6 7 6 7 6 7
## 4 13 8 9 8 9 8 9
## 5 17 10 12 10 11 10 11
## 6 20 13 14 12 13 12 14
## 7 23 15 16 14 15 15 16
## 8 26 17 18 16 18 17 18
## 9 29 19 20 19 20 19 20
## 10 33 21 23 21 22 21 22
## 11 36 24 25 23 24 23 24
## 12 43 26 33 25 35 25 40Skewness: |minimum| shows transformation that brings skewness closest to zero
## raw square.root log+10 1/x+100 squared exp |minimum|
## rundots -0.94 -2.59 -3.47 3.42 0.42 NaN 0.42
## dotcount -0.23 -1.77 -1.11 0.52 0.71 12.03 0.23
## flanker -0.76 -0.90 -1.04 1.33 -0.33 NaN 0.33
## setshift -1.08 -2.53 -10.33 16.89 -0.28 NaN 0.28
## match -1.02 -1.84 -2.06 1.57 -0.07 28.90 0.07
## animals 0.15 -0.39 -0.44 0.11 1.14 56.34 0.11
## veg 0.26 -0.29 -0.27 -0.04 1.50 37.22 0.04
## fwords 0.23 -0.32 -0.31 -0.01 1.36 56.05 0.01
## lwords 0.27 -0.29 -0.27 -0.05 1.65 56.99 0.05Lookup table skewed, restricted data.
## recode_score min_rundots max_rundots min_dotcount max_dotcount min_flanker
## 1 1 -152100 -143075 0 4 -80432400
## 2 2 -140000 -131075 5 6 -74459744
## 3 3 -129600 -121475 7 7 -69212181
## 4 4 -119700 -110075 8 9 -63369900
## 5 5 -108000 -101475 10 11 -57893136
## 6 6 -99200 -89600 12 13 -52342800
## 7 7 -87075 -79200 14 15 -46758096
## 8 8 -76475 -68000 16 17 -41227500
## 9 9 -65075 -59075 18 19 -35592400
## 10 10 -56000 -46475 20 21 -30140236
## 11 11 -43200 -36500 22 22 -24304748
## 12 12 -33075 0 23 27 -18816300
## max_flanker min_setshift max_setshift min_match max_match min_animals
## 1 -77248000 -100000000 -93750000 -4624 -4335 1
## 2 -70258944 -91186227 -85100400 -4300 -3999 4
## 3 -63714000 -84555100 -79468227 -3948 -3663 7
## 4 -58794175 -77070268 -69859900 -3600 -3399 10
## 5 -52787879 -69639900 -62681341 -3328 -3024 13
## 6 -46986189 -61952075 -54707100 -2943 -2775 16
## 7 -41642796 -53686531 -47036538 -2688 -2415 18
## 8 -36214800 -46941887 -39471600 -2320 -2124 21
## 9 -30172800 -39362631 -32090864 -2023 -1815 24
## 10 -25029900 -31275900 -24483900 -1708 -1488 27
## 11 -19168300 -24038317 0 -1375 -1143 30
## 12 0 NA NA -1024 0 33
## max_animals min_veg max_veg min_fwords max_fwords min_lwords max_lwords
## 1 3 1 3 1 3 1 3
## 2 6 4 5 4 5 4 5
## 3 9 6 7 6 7 6 7
## 4 12 8 9 8 9 8 9
## 5 15 10 11 10 11 10 11
## 6 17 12 13 12 13 12 13
## 7 20 14 15 14 15 14 15
## 8 23 16 17 16 18 16 17
## 9 26 18 19 19 20 18 19
## 10 29 20 21 21 22 20 21
## 11 32 22 23 23 24 22 23
## 12 38 24 33 25 35 24 28EFA summary
4 factor model was best.
Figure 1: EFA 4 factor model, unrestricted, nontransformed data.
## Figure 2: EFA 4 factor model, restricted, transformed data
IRT model comparisons
##
##
## | | Full | Restrict | Skew |
## |:----------:|:------:|:--------:|:------:|
## | **n** | 4003 | 1323 | 1323 |
## | **LogLik** | -30305 | -10703 | -11670 |
## | **AIC** | 60802 | 21597 | 23559 |
## | **BIC** | 61406 | 22095 | 24124 |
##
## Table: Model comparison##
##
## | | Full | Restrict | Skew |
## |:--------------:|:------:|:--------:|:------:|
## | **rundots_r** | 0.5955 | 0.5839 | 0.5275 |
## | **dotcount_r** | 0.5625 | 0.5488 | 0.5306 |
## | **flanker_r** | 0.7863 | 0.7863 | 0.7613 |
## | **setshift_r** | 0.7661 | 0.7638 | 0.766 |
## | **match_r** | 0.8 | 0.802 | 0.8037 |
## | **animals_r** | 0.8935 | 0.8496 | 0.862 |
## | **veg_r** | 0.832 | 0.7513 | 0.749 |
## | **fwords_r** | 0.7806 | 0.6621 | 0.6662 |
## | **lwords_r** | 0.7932 | 0.6696 | 0.6751 |
##
## Table: EF factor loadings by model##
##
## | | Full | Restrict | Skew |
## |:--------------:|:-----:|:--------:|:-----:|
## | **rundots_r** | 0.355 | 0.341 | 0.278 |
## | **dotcount_r** | 0.316 | 0.301 | 0.282 |
## | **flanker_r** | 0.618 | 0.618 | 0.58 |
## | **setshift_r** | 0.587 | 0.583 | 0.587 |
## | **match_r** | 0.64 | 0.643 | 0.646 |
## | **animals_r** | 0.851 | 0.794 | 0.81 |
## | **veg_r** | 0.773 | 0.677 | 0.676 |
## | **fwords_r** | 0.862 | 0.824 | 0.826 |
## | **lwords_r** | 0.869 | 0.827 | 0.829 |
##
## Table: H2: variance accounted for by variableFactor associations
##
##
## | | gender: 1 vs 0 | as.factor(CDRtot) | age |
## |:------------------------:|:--------------:|:-----------------:|:------:|
## | **rundots_r** | 0.481 | -0.75 | -0.065 |
## | **dotcount_r** | 1.211 | 4.846 | -0.014 |
## | **flanker_r** | -0.386 | 1 | -0.001 |
## | **setshift_r** | 0.616 | 5.455 | -0.011 |
## | **match_r** | 0.832 | -1.333 | -0.059 |
## | **animals_r** | 0.201 | -1 | -0.019 |
## | **veg_r** | 1.224 | -2 | -0.011 |
## | **fwords_r** | 0.092 | -0.5 | 0.026 |
## | **lwords_r** | 0.119 | 1 | 0.024 |
## | **ExecutiveComposite_r** | 0.22 | -0.368 | -0.009 |
##
## Table: Regression Betas; Gender, CDRtot as factors (continued below)
##
##
##
## | | edu | CDRbox | log(traila) | log(trailb) |
## |:------------------------:|:------:|:------:|:-----------:|:-----------:|
## | **rundots_r** | -0.849 | -0.44 | -3.148 | -2.496 |
## | **dotcount_r** | -1.442 | -0.668 | -2.241 | -2.445 |
## | **flanker_r** | -0.972 | -0.497 | -4.052 | -2.164 |
## | **setshift_r** | -1.998 | -0.591 | -3.31 | -2.998 |
## | **match_r** | -2.622 | -0.674 | -3.745 | -3.384 |
## | **animals_r** | -1.861 | -0.569 | -2.755 | -2.388 |
## | **veg_r** | -1.834 | -0.541 | -2.094 | -2.106 |
## | **fwords_r** | -1.469 | -0.444 | -1.859 | -1.599 |
## | **lwords_r** | -1.423 | -0.444 | -1.994 | -1.76 |
## | **ExecutiveComposite_r** | -0.768 | -0.235 | -1.149 | -0.993 |##
##
## | | gender: 1 vs 0 | as.factor(CDRtot) | age |
## |:------------------------:|:--------------:|:-----------------:|:-----:|
## | **rundots_r** | 0.191 | 0.768 | 0 |
## | **dotcount_r** | 0 | 0.055 | 0.213 |
## | **flanker_r** | 0.446 | 0.635 | 0.976 |
## | **setshift_r** | 0.057 | 0.025 | 0.343 |
## | **match_r** | 0 | 0.586 | 0 |
## | **animals_r** | 0.255 | 0.683 | 0.004 |
## | **veg_r** | 0 | 0.425 | 0.093 |
## | **fwords_r** | 0.613 | 0.837 | 0 |
## | **lwords_r** | 0.525 | 0.685 | 0 |
## | **ExecutiveComposite_r** | 0 | 0.631 | 0 |
##
## Table: P values Betas; Gender, CDRtot as factors (continued below)
##
##
##
## | | edu | CDRbox | log(traila) | log(trailb) |
## |:------------------------:|:-----:|:------:|:-----------:|:-----------:|
## | **rundots_r** | 0.027 | 0 | 0 | 0 |
## | **dotcount_r** | 0 | 0 | 0 | 0 |
## | **flanker_r** | 0.077 | 0 | 0 | 0 |
## | **setshift_r** | 0 | 0 | 0 | 0 |
## | **match_r** | 0 | 0 | 0 | 0 |
## | **animals_r** | 0 | 0 | 0 | 0 |
## | **veg_r** | 0 | 0 | 0 | 0 |
## | **fwords_r** | 0 | 0 | 0 | 0 |
## | **lwords_r** | 0 | 0 | 0 | 0 |
## | **ExecutiveComposite_r** | 0 | 0 | 0 | 0 |
