1. Import Data

library(rio)
library(ltm)
## Loading required package: MASS
## Loading required package: msm
## Loading required package: polycor
library(irtoys)
## Loading required package: sm
## Package 'sm', version 2.2-5.6: type help(sm) for summary information
## 
## Attaching package: 'sm'
## The following object is masked from 'package:MASS':
## 
##     muscle
Fulldata <- read.csv("/Users/Lorraine/Desktop/GMA.csv")
Items <- Fulldata[, 2:11]

2. Descriptive Statistics

descript(Items)
## 
## Descriptive statistics for the 'Items' data-set
## 
## Sample:
##  10 items and 323 sample units; 2 missing values
## 
## Proportions for each level of response:
##          0      1   logit
## g1  0.2919 0.7081  0.8861
## g2  0.1331 0.8669  1.8736
## g3  0.0341 0.9659  3.3451
## g4  0.3529 0.6471  0.6061
## g5  0.7678 0.2322 -1.1959
## g6  0.2950 0.7050  0.8711
## g7  0.3437 0.6563  0.6471
## g8  0.5046 0.4954 -0.0186
## g9  0.2663 0.7337  1.0137
## g10 0.5232 0.4768 -0.0929
## 
## Missing responses:
##          g1 g2 g3 g4 g5     g6 g7 g8 g9 g10
## Freq 1.0000  0  0  0  0 1.0000  0  0  0   0
## (%)  0.3096  0  0  0  0 0.3096  0  0  0   0
## 
## 
## Frequencies of total scores:
##      0 1 2  3  4  5  6  7  8  9 10
## Freq 0 1 8 18 36 46 39 59 62 34 20
## 
## 
## Point Biserial correlation with Total Score:
##     Included Excluded
## g1    0.5388   0.3529
## g2    0.2208   0.0563
## g3    0.2602   0.1751
## g4    0.5585   0.3696
## g5    0.3955   0.2041
## g6    0.4460   0.2473
## g7    0.6399   0.4698
## g8    0.5720   0.3730
## g9    0.5031   0.3160
## g10   0.4787   0.2600
## 
## 
## Cronbach's alpha:
##                value
## All Items     0.6162
## Excluding g1  0.5736
## Excluding g2  0.6321
## Excluding g3  0.6131
## Excluding g4  0.5699
## Excluding g5  0.6088
## Excluding g6  0.5997
## Excluding g7  0.5427
## Excluding g8  0.5684
## Excluding g9  0.5837
## Excluding g10 0.5989
## 
## 
## Pairwise Associations:
##    Item i Item j p.value
## 1       3     10   1.000
## 2       2      4   1.000
## 3       2      5   1.000
## 4       5      9   0.889
## 5       2      7   0.803
## 6       3      5   0.762
## 7       2     10   0.743
## 8       3      6   0.741
## 9       2      9   0.697
## 10      1      6   0.595

3. 1PL and 2PL IRT models and thier parameters

ip1 <- rasch(Items, IRT.param = TRUE)
ip2 <- ltm(Items ~ z1, IRT.param = TRUE)
coef(ip1)
##          Dffclt   Dscrmn
## g1  -1.05766217 1.016048
## g2  -2.17852547 1.016048
## g3  -3.73845934 1.016048
## g4  -0.72658486 1.016048
## g5   1.41309750 1.016048
## g6  -1.04013970 1.016048
## g7  -0.77497179 1.016048
## g8   0.01888467 1.016048
## g9  -1.20471513 1.016048
## g10  0.10802189 1.016048
coef(ip2)
##          Dffclt    Dscrmn
## g1  -0.92038215 1.2637361
## g2  -8.00612616 0.2365621
## g3  -3.22464345 1.2409738
## g4  -0.65311075 1.2015796
## g5   1.91255284 0.6865801
## g6  -1.23417470 0.8052272
## g7  -0.53206748 2.1092885
## g8   0.01424363 1.1869533
## g9  -1.11026344 1.1487680
## g10  0.12515117 0.8321169

4. Model fit and item fit

anova(ip1, ip2)
## 
##  Likelihood Ratio Table
##         AIC     BIC  log.Lik   LRT df p.value
## ip1 3432.94 3474.49 -1705.47                 
## ip2 3417.70 3493.25 -1688.85 33.24  9  <0.001
item.fit(ip1)
## 
## Item-Fit Statistics and P-values
## 
## Call:
## rasch(data = Items, IRT.param = TRUE)
## 
## Alternative: Items do not fit the model
## Ability Categories: 10
## 
##         X^2 Pr(>X^2)
## g1  26.7591   0.0002
## g2  45.8921  <0.0001
## g3   9.6313   0.1411
## g4  27.6299   0.0001
## g5  17.4723   0.0077
## g6  11.0674   0.0863
## g7  44.6643  <0.0001
## g8  20.0218   0.0027
## g9  28.6201   0.0001
## g10  8.6323   0.1953
item.fit(ip2)
## 
## Item-Fit Statistics and P-values
## 
## Call:
## ltm(formula = Items ~ z1, IRT.param = TRUE)
## 
## Alternative: Items do not fit the model
## Ability Categories: 10
## 
##         X^2 Pr(>X^2)
## g1  15.3640   0.0524
## g2  10.0587   0.2609
## g3  11.7896   0.1608
## g4  33.2740   0.0001
## g5  19.6620   0.0117
## g6  26.0554    0.001
## g7  28.3427   0.0004
## g8  22.2272   0.0045
## g9  20.0078   0.0103
## g10 20.6160   0.0082

5. ICCs & item information plots for the best fitting model

The 2PL model fits better, chi-square (df = 9) = 33.24, p < 0.001.

plot(ip2, type = "ICC", lty = rep(1:3, each=8), legend = TRUE, cex = .5, cx = "bottomright") 

plot(ip2, type = "IIC", lty = rep(1:3, each=8), legend = TRUE, cex = .5, cx = "topright")

6. Does the 2PL model fit better than the 1PL?

Yes, although the 2PL model has a slightly higher BIC than the 1PL model, it has a slightly lower AIC and the chi-square test (df = 9) was 33.24, p < 0.001, which indicates that the 2PL fits significantly better than the 1PL model. In addition, the item-fit analysis indicated less significantly lack-of-fit items in the 2PL model.

7. Using results from the 2PL model, which 2 items are most discriminating, and which 2 are least discriminating?

The two most discriminating items in the 2PL model are item 7 and item 1, and the two least discriminating items in this model are item 2 and item 5.

8. Using the results for the best fitting model, if you had to choose just 3 items for a test, which items would you choose in each the following situations? a. A general-purpose test that will be used for a wide range of ability. b. A minimum qualification test where you need accuracy at low ability (θ < -1). c. A professional certification test where you need accuracy at high ability (θ > 1)

I would choose item 1, 7, and 8 for a general-purpose test. I’d choose item 3, 6, and 9 for a minimum qualification test. Lastly, I’d choose item 5, 8, and 10 for a professional certification test, although itme 5 is the only one with θ > 1.