Polytomous Rasch Models
Dan Isbell
2/28/2021
This tutorial introduces polytomous Rasch models using the eRm package. We’ll start with the conceptually simpler Rating Scale Model.
RSM
First, we’ll load packages and data.
library(tidyverse)
library(eRm)
eit <- read_csv("EIT_learner_single_scores.csv")The eRm package involves 3 basic steps for doing Rasch analyses: 1. Run the model; this gives basic item parameters 2. Calculate person parameters 3. Calculate item fit statistics
1. Run the model
Run the model using the RSM command.
RSM <- RSM(eit[-1]) #estimate the model
summary(RSM)##
## Results of RSM estimation:
##
## Call: RSM(X = eit[-1])
##
## Conditional log-likelihood: -9148.449
## Number of iterations: 44
## Number of parameters: 32
##
## Item (Category) Difficulty Parameters (eta): with 0.95 CI:
## Estimate Std. Error lower CI upper CI
## I02 -1.434 0.077 -1.585 -1.282
## I03 -0.980 0.071 -1.119 -0.841
## I04 -2.206 0.093 -2.388 -2.024
## I05 -1.315 0.076 -1.463 -1.167
## I06 -0.370 0.066 -0.499 -0.242
## I07 -0.891 0.070 -1.028 -0.754
## I08 -0.649 0.068 -0.782 -0.517
## I09 -0.654 0.068 -0.787 -0.521
## I10 -0.896 0.070 -1.033 -0.759
## I11 0.740 0.066 0.611 0.868
## I12 -0.490 0.066 -0.620 -0.360
## I13 1.487 0.074 1.343 1.632
## I14 0.314 0.064 0.188 0.439
## I15 1.204 0.070 1.067 1.341
## I16 1.171 0.069 1.035 1.307
## I17 1.304 0.071 1.165 1.443
## I18 0.633 0.065 0.506 0.760
## I19 1.328 0.071 1.188 1.468
## I20 0.464 0.064 0.338 0.590
## I21 -0.064 0.064 -0.190 0.062
## I22 1.362 0.072 1.222 1.503
## I23 0.251 0.064 0.126 0.376
## I24 0.389 0.064 0.263 0.514
## I25 0.667 0.065 0.539 0.795
## I26 0.121 0.064 -0.004 0.246
## I27 0.092 0.064 -0.034 0.217
## I28 0.616 0.065 0.489 0.743
## I29 0.138 0.064 0.013 0.263
## I30 -0.522 0.067 -0.652 -0.391
## Cat 2 0.414 0.066 0.286 0.543
## Cat 3 1.766 0.111 1.548 1.984
## Cat 4 3.949 0.171 3.615 4.283
##
## Item Easiness Parameters (beta) with 0.95 CI:
## Estimate Std. Error lower CI upper CI
## beta I01.c1 1.809 0.084 1.644 1.974
## beta I01.c2 3.204 0.174 2.863 3.546
## beta I01.c3 3.662 0.259 3.155 4.169
## beta I01.c4 3.288 0.342 2.618 3.959
## beta I02.c1 1.434 0.077 1.282 1.585
## beta I02.c2 2.453 0.163 2.134 2.772
## beta I02.c3 2.535 0.242 2.059 3.010
## beta I02.c4 1.785 0.323 1.152 2.418
## beta I03.c1 0.980 0.071 0.841 1.119
## beta I03.c2 1.546 0.153 1.247 1.845
## beta I03.c3 1.174 0.229 0.725 1.623
## beta I03.c4 -0.029 0.309 -0.634 0.577
## beta I04.c1 2.206 0.093 2.024 2.388
## beta I04.c2 3.998 0.190 3.625 4.371
## beta I04.c3 4.852 0.281 4.301 5.404
## beta I04.c4 4.875 0.370 4.150 5.600
## beta I05.c1 1.315 0.076 1.167 1.463
## beta I05.c2 2.216 0.160 1.903 2.529
## beta I05.c3 2.180 0.238 1.713 2.647
## beta I05.c4 1.312 0.318 0.688 1.937
## beta I06.c1 0.370 0.066 0.242 0.499
## beta I06.c2 0.327 0.145 0.042 0.611
## beta I06.c3 -0.655 0.221 -1.088 -0.221
## beta I06.c4 -2.467 0.303 -3.062 -1.873
## beta I07.c1 0.891 0.070 0.754 1.028
## beta I07.c2 1.368 0.151 1.072 1.664
## beta I07.c3 0.908 0.227 0.462 1.353
## beta I07.c4 -0.384 0.307 -0.986 0.218
## beta I08.c1 0.649 0.068 0.517 0.782
## beta I08.c2 0.885 0.148 0.595 1.174
## beta I08.c3 0.182 0.223 -0.255 0.620
## beta I08.c4 -1.351 0.304 -1.947 -0.755
## beta I09.c1 0.654 0.068 0.521 0.787
## beta I09.c2 0.894 0.148 0.604 1.183
## beta I09.c3 0.196 0.223 -0.242 0.634
## beta I09.c4 -1.333 0.304 -1.929 -0.737
## beta I10.c1 0.896 0.070 0.759 1.033
## beta I10.c2 1.378 0.151 1.082 1.674
## beta I10.c3 0.922 0.227 0.477 1.368
## beta I10.c4 -0.365 0.307 -0.967 0.238
## beta I11.c1 -0.740 0.066 -0.868 -0.611
## beta I11.c2 -1.894 0.150 -2.187 -1.601
## beta I11.c3 -3.985 0.234 -4.444 -3.526
## beta I11.c4 -6.908 0.329 -7.554 -6.262
## beta I12.c1 0.490 0.066 0.360 0.620
## beta I12.c2 0.566 0.146 0.280 0.852
## beta I12.c3 -0.295 0.222 -0.730 0.139
## beta I12.c4 -1.988 0.303 -2.582 -1.394
## beta I13.c1 -1.487 0.074 -1.632 -1.343
## beta I13.c2 -3.389 0.167 -3.717 -3.061
## beta I13.c3 -6.228 0.264 -6.745 -5.711
## beta I13.c4 -9.899 0.372 -10.628 -9.170
## beta I14.c1 -0.314 0.064 -0.439 -0.188
## beta I14.c2 -1.041 0.145 -1.325 -0.757
## beta I14.c3 -2.706 0.225 -3.147 -2.266
## beta I14.c4 -5.203 0.314 -5.819 -4.588
## beta I15.c1 -1.204 0.070 -1.341 -1.067
## beta I15.c2 -2.823 0.159 -3.135 -2.511
## beta I15.c3 -5.378 0.251 -5.870 -4.887
## beta I15.c4 -8.766 0.354 -9.459 -8.073
## beta I16.c1 -1.171 0.069 -1.307 -1.035
## beta I16.c2 -2.757 0.158 -3.068 -2.447
## beta I16.c3 -5.280 0.249 -5.769 -4.791
## beta I16.c4 -8.635 0.352 -9.324 -7.946
## beta I17.c1 -1.304 0.071 -1.443 -1.165
## beta I17.c2 -3.022 0.162 -3.340 -2.705
## beta I17.c3 -5.678 0.255 -6.178 -5.178
## beta I17.c4 -9.165 0.360 -9.870 -8.460
## beta I18.c1 -0.633 0.065 -0.760 -0.506
## beta I18.c2 -1.680 0.148 -1.970 -1.390
## beta I18.c3 -3.665 0.231 -4.118 -3.211
## beta I18.c4 -6.481 0.325 -7.118 -5.844
## beta I19.c1 -1.328 0.071 -1.468 -1.188
## beta I19.c2 -3.071 0.163 -3.389 -2.752
## beta I19.c3 -5.751 0.256 -6.253 -5.248
## beta I19.c4 -9.262 0.361 -9.970 -8.554
## beta I20.c1 -0.464 0.064 -0.590 -0.338
## beta I20.c2 -1.343 0.146 -1.629 -1.056
## beta I20.c3 -3.159 0.227 -3.604 -2.713
## beta I20.c4 -5.806 0.319 -6.431 -5.181
## beta I21.c1 0.064 0.064 -0.062 0.190
## beta I21.c2 -0.286 0.144 -0.567 -0.004
## beta I21.c3 -1.573 0.221 -2.006 -1.140
## beta I21.c4 -3.692 0.306 -4.291 -3.092
## beta I22.c1 -1.362 0.072 -1.503 -1.222
## beta I22.c2 -3.139 0.164 -3.460 -2.818
## beta I22.c3 -5.853 0.258 -6.358 -5.348
## beta I22.c4 -9.399 0.364 -10.111 -8.686
## beta I23.c1 -0.251 0.064 -0.376 -0.126
## beta I23.c2 -0.916 0.144 -1.199 -0.633
## beta I23.c3 -2.518 0.224 -2.957 -2.080
## beta I23.c4 -4.952 0.312 -5.564 -4.340
## beta I24.c1 -0.389 0.064 -0.514 -0.263
## beta I24.c2 -1.192 0.145 -1.477 -0.907
## beta I24.c3 -2.932 0.226 -3.375 -2.489
## beta I24.c4 -5.504 0.316 -6.124 -4.884
## beta I25.c1 -0.667 0.065 -0.795 -0.539
## beta I25.c2 -1.748 0.148 -2.039 -1.457
## beta I25.c3 -3.767 0.232 -4.222 -3.312
## beta I25.c4 -6.617 0.326 -7.257 -5.977
## beta I26.c1 -0.121 0.064 -0.246 0.004
## beta I26.c2 -0.656 0.144 -0.939 -0.374
## beta I26.c3 -2.129 0.222 -2.565 -1.694
## beta I26.c4 -4.433 0.309 -5.039 -3.827
## beta I27.c1 -0.092 0.064 -0.217 0.034
## beta I27.c2 -0.598 0.144 -0.880 -0.316
## beta I27.c3 -2.041 0.222 -2.476 -1.606
## beta I27.c4 -4.316 0.309 -4.921 -3.711
## beta I28.c1 -0.616 0.065 -0.743 -0.489
## beta I28.c2 -1.646 0.148 -1.936 -1.357
## beta I28.c3 -3.614 0.231 -4.067 -3.161
## beta I28.c4 -6.413 0.324 -7.049 -5.778
## beta I29.c1 -0.138 0.064 -0.263 -0.013
## beta I29.c2 -0.690 0.144 -0.972 -0.408
## beta I29.c3 -2.179 0.222 -2.615 -1.744
## beta I29.c4 -4.500 0.310 -5.107 -3.894
## beta I30.c1 0.522 0.067 0.391 0.652
## beta I30.c2 0.629 0.146 0.342 0.916
## beta I30.c3 -0.201 0.222 -0.636 0.234
## beta I30.c4 -1.862 0.303 -2.457 -1.268For polytomous items, examining the thresholds can be more intuitive than eRm’s default summary output:
thresholds(RSM)##
## Design Matrix Block 1:
## Location Threshold 1 Threshold 2 Threshold 3 Threshold 4
## I01 -0.82204 -1.80929 -1.39504 -0.45768 0.37385
## I02 -0.44626 -1.43351 -1.01926 -0.08190 0.74963
## I03 0.00720 -0.98005 -0.56580 0.37156 1.20309
## I04 -1.21874 -2.20599 -1.79174 -0.85438 -0.02285
## I05 -0.32808 -1.31533 -0.90108 0.03628 0.86781
## I06 0.61680 -0.37045 0.04380 0.98117 1.81269
## I07 0.09604 -0.89122 -0.47696 0.46040 1.29192
## I08 0.33785 -0.64940 -0.23515 0.70222 1.53374
## I09 0.33323 -0.65402 -0.23977 0.69760 1.52912
## I10 0.09115 -0.89610 -0.48185 0.45552 1.28704
## I11 1.72702 0.73977 1.15402 2.09138 2.92291
## I12 0.49700 -0.49026 -0.07600 0.86136 1.69288
## I13 2.47464 1.48739 1.90164 2.83900 3.67053
## I14 1.30078 0.31353 0.72778 1.66514 2.49667
## I15 2.19145 1.20420 1.61845 2.55581 3.38734
## I16 2.15874 1.17149 1.58574 2.52310 3.35463
## I17 2.29130 1.30405 1.71830 2.65566 3.48719
## I18 1.62025 0.63300 1.04725 1.98461 2.81614
## I19 2.31549 1.32824 1.74249 2.67985 3.51138
## I20 1.45148 0.46423 0.87848 1.81585 2.64737
## I21 0.92289 -0.06436 0.34989 1.28726 2.11878
## I22 2.34964 1.36239 1.77664 2.71400 3.54553
## I23 1.23809 0.25084 0.66509 1.60245 2.43398
## I24 1.37605 0.38880 0.80305 1.74042 2.57194
## I25 1.65429 0.66703 1.08129 2.01865 2.85017
## I26 1.10835 0.12110 0.53535 1.47271 2.30424
## I27 1.07898 0.09173 0.50598 1.44334 2.27487
## I28 1.60328 0.61603 1.03028 1.96764 2.79917
## I29 1.12512 0.13787 0.55212 1.48948 2.32101
## I30 0.46554 -0.52171 -0.10746 0.82990 1.661432. Person parameters
Next we’ll estimate person parameters and reliability.
RSM_pp <- person.parameter(RSM) #estimate person parameters
SepRel(RSM_pp)## Separation Reliability: 0.9717This allows us to estimate item fit statistics.
3. Item fit
Using the itemfit() function on an object of person parameters is how we get our conventional fit statistics.
RSM_items <- itemfit(RSM_pp) #estimate item fit statisticsWe can also make a rather nice looking Wright Map with the eRm function plotPImap
plotPImap(RSM)
Partial Credit Model
We’ll go through mostly the same steps to run a PCM.
First, the data:
d <- read_csv("listening data.csv")Next, running the model.
PCM <- PCM(d[-1], sum0 = T)
summary(PCM)##
## Results of PCM estimation:
##
## Call: PCM(X = d[-1], sum0 = T)
##
## Conditional log-likelihood: -12372.38
## Number of iterations: 95
## Number of parameters: 61
##
## Item (Category) Difficulty Parameters (eta): with 0.95 CI:
## Estimate Std. Error lower CI upper CI
## Item_01.c2 -3.262 0.682 -4.598 -1.926
## Item_01.c3 -3.873 0.665 -5.175 -2.570
## Item_02.c1 -2.234 0.341 -2.903 -1.565
## Item_02.c2 -2.557 0.336 -3.215 -1.899
## Item_03.c1 -0.362 0.135 -0.627 -0.097
## Item_03.c2 0.243 0.145 -0.041 0.527
## Item_04.c1 -0.763 0.176 -1.108 -0.418
## Item_04.c2 -0.783 0.176 -1.129 -0.438
## Item_05.c1 -0.675 0.133 -0.935 -0.414
## Item_05.c2 0.731 0.151 0.434 1.027
## Item_05.c3 2.998 0.193 2.620 3.375
## Item_06.c1 -1.461 0.202 -1.858 -1.065
## Item_06.c2 -1.056 0.206 -1.459 -0.652
## Item_07.c1 -2.901 0.576 -4.029 -1.772
## Item_07.c2 -3.858 0.563 -4.963 -2.754
## Item_08.c1 -3.125 0.317 -3.746 -2.504
## Item_08.c2 -1.040 0.324 -1.676 -0.405
## Item_09.c1 0.382 0.103 0.181 0.583
## Item_09.c2 1.871 0.128 1.620 2.122
## Item_10.c1 -0.355 0.108 -0.566 -0.144
## Item_10.c2 2.119 0.149 1.827 2.411
## Item_11.c1 -1.589 0.609 -2.783 -0.395
## Item_11.c2 -2.847 0.560 -3.944 -1.750
## Item_11.c3 -3.176 0.541 -4.237 -2.116
## Item_11.c4 -1.898 0.529 -2.935 -0.861
## Item_12.c1 0.795 0.099 0.600 0.989
## Item_12.c2 2.938 0.146 2.651 3.225
## Item_12.c3 4.060 0.164 3.739 4.380
## Item_13.c1 -1.426 0.164 -1.747 -1.104
## Item_13.c2 0.523 0.185 0.160 0.887
## Item_13.c3 1.894 0.202 1.498 2.290
## Item_13.c4 4.662 0.266 4.141 5.184
## Item_14.c1 -1.002 0.271 -1.533 -0.472
## Item_14.c2 -1.171 0.258 -1.677 -0.665
## Item_14.c3 -0.725 0.259 -1.233 -0.218
## Item_14.c4 0.606 0.269 0.078 1.133
## Item_15.c1 -0.106 0.099 -0.300 0.087
## Item_15.c2 3.214 0.173 2.875 3.552
## Item_16.c1 -0.724 0.226 -1.166 -0.282
## Item_16.c2 -1.040 0.215 -1.461 -0.619
## Item_16.c3 -0.110 0.225 -0.550 0.331
## Item_17.c1 -0.622 0.189 -0.992 -0.251
## Item_17.c2 0.015 0.195 -0.367 0.397
## Item_17.c3 0.292 0.197 -0.095 0.678
## Item_17.c4 2.146 0.222 1.711 2.581
## Item_18.c1 -0.062 0.184 -0.423 0.299
## Item_18.c2 -0.102 0.175 -0.445 0.242
## Item_18.c3 0.702 0.187 0.335 1.068
## Item_18.c4 2.430 0.213 2.013 2.847
## Item_19.c1 -0.692 0.167 -1.018 -0.365
## Item_19.c2 -0.237 0.170 -0.569 0.096
## Item_19.c3 1.462 0.192 1.085 1.838
## Item_19.c4 4.070 0.245 3.590 4.549
## Item_20.c1 -0.226 0.147 -0.513 0.062
## Item_20.c2 0.418 0.154 0.116 0.721
## Item_20.c3 1.878 0.178 1.530 2.227
## Item_20.c4 3.589 0.208 3.182 3.997
## Item_21.c1 0.251 0.110 0.036 0.466
## Item_21.c2 1.341 0.128 1.090 1.591
## Item_22.c1 0.517 0.100 0.321 0.713
## Item_22.c2 2.099 0.128 1.849 2.349
##
## Item Easiness Parameters (beta) with 0.95 CI:
## Estimate Std. Error lower CI upper CI
## beta Item_01.c1 2.185 0.717 0.780 3.590
## beta Item_01.c2 3.262 0.682 1.926 4.598
## beta Item_01.c3 3.873 0.665 2.570 5.175
## beta Item_02.c1 2.234 0.341 1.565 2.903
## beta Item_02.c2 2.557 0.336 1.899 3.215
## beta Item_03.c1 0.362 0.135 0.097 0.627
## beta Item_03.c2 -0.243 0.145 -0.527 0.041
## beta Item_04.c1 0.763 0.176 0.418 1.108
## beta Item_04.c2 0.783 0.176 0.438 1.129
## beta Item_05.c1 0.675 0.133 0.414 0.935
## beta Item_05.c2 -0.731 0.151 -1.027 -0.434
## beta Item_05.c3 -2.998 0.193 -3.375 -2.620
## beta Item_06.c1 1.461 0.202 1.065 1.858
## beta Item_06.c2 1.056 0.206 0.652 1.459
## beta Item_07.c1 2.901 0.576 1.772 4.029
## beta Item_07.c2 3.858 0.563 2.754 4.963
## beta Item_08.c1 3.125 0.317 2.504 3.746
## beta Item_08.c2 1.040 0.324 0.405 1.676
## beta Item_09.c1 -0.382 0.103 -0.583 -0.181
## beta Item_09.c2 -1.871 0.128 -2.122 -1.620
## beta Item_10.c1 0.355 0.108 0.144 0.566
## beta Item_10.c2 -2.119 0.149 -2.411 -1.827
## beta Item_11.c1 1.589 0.609 0.395 2.783
## beta Item_11.c2 2.847 0.560 1.750 3.944
## beta Item_11.c3 3.176 0.541 2.116 4.237
## beta Item_11.c4 1.898 0.529 0.861 2.935
## beta Item_12.c1 -0.795 0.099 -0.989 -0.600
## beta Item_12.c2 -2.938 0.146 -3.225 -2.651
## beta Item_12.c3 -4.060 0.164 -4.380 -3.739
## beta Item_13.c1 1.426 0.164 1.104 1.747
## beta Item_13.c2 -0.523 0.185 -0.887 -0.160
## beta Item_13.c3 -1.894 0.202 -2.290 -1.498
## beta Item_13.c4 -4.662 0.266 -5.184 -4.141
## beta Item_14.c1 1.002 0.271 0.472 1.533
## beta Item_14.c2 1.171 0.258 0.665 1.677
## beta Item_14.c3 0.725 0.259 0.218 1.233
## beta Item_14.c4 -0.606 0.269 -1.133 -0.078
## beta Item_15.c1 0.106 0.099 -0.087 0.300
## beta Item_15.c2 -3.214 0.173 -3.552 -2.875
## beta Item_16.c1 0.724 0.226 0.282 1.166
## beta Item_16.c2 1.040 0.215 0.619 1.461
## beta Item_16.c3 0.110 0.225 -0.331 0.550
## beta Item_17.c1 0.622 0.189 0.251 0.992
## beta Item_17.c2 -0.015 0.195 -0.397 0.367
## beta Item_17.c3 -0.292 0.197 -0.678 0.095
## beta Item_17.c4 -2.146 0.222 -2.581 -1.711
## beta Item_18.c1 0.062 0.184 -0.299 0.423
## beta Item_18.c2 0.102 0.175 -0.242 0.445
## beta Item_18.c3 -0.702 0.187 -1.068 -0.335
## beta Item_18.c4 -2.430 0.213 -2.847 -2.013
## beta Item_19.c1 0.692 0.167 0.365 1.018
## beta Item_19.c2 0.237 0.170 -0.096 0.569
## beta Item_19.c3 -1.462 0.192 -1.838 -1.085
## beta Item_19.c4 -4.070 0.245 -4.549 -3.590
## beta Item_20.c1 0.226 0.147 -0.062 0.513
## beta Item_20.c2 -0.418 0.154 -0.721 -0.116
## beta Item_20.c3 -1.878 0.178 -2.227 -1.530
## beta Item_20.c4 -3.589 0.208 -3.997 -3.182
## beta Item_21.c1 -0.251 0.110 -0.466 -0.036
## beta Item_21.c2 -1.341 0.128 -1.591 -1.090
## beta Item_22.c1 -0.517 0.100 -0.713 -0.321
## beta Item_22.c2 -2.099 0.128 -2.349 -1.849Person parameters, reliability:
PCM_pp <- person.parameter(PCM)
SepRel(PCM_pp)## Separation Reliability: 0.8122Item fit statistics:
PCM_items <- itemfit(PCM_pp)
thresholds(PCM)##
## Design Matrix Block 1:
## Location Threshold 1 Threshold 2 Threshold 3 Threshold 4
## Item_01 -1.29086 -2.18507 -1.07729 -0.61023 NA
## Item_02 -1.27851 -2.23411 -0.32291 NA NA
## Item_03 0.12150 -0.36195 0.60494 NA NA
## Item_04 -0.39163 -0.76291 -0.02035 NA NA
## Item_05 0.99929 -0.67476 1.40535 2.26729 NA
## Item_06 -0.52779 -1.46131 0.40574 NA NA
## Item_07 -1.92924 -2.90066 -0.95783 NA NA
## Item_08 -0.52010 -3.12505 2.08485 NA NA
## Item_09 0.93546 0.38229 1.48863 NA NA
## Item_10 1.05945 -0.35480 2.47369 NA NA
## Item_11 -0.47449 -1.58915 -1.25756 -0.32954 1.27828
## Item_12 1.35332 0.79454 2.14333 1.12209 NA
## Item_13 1.16558 -1.42587 1.94921 1.37058 2.76840
## Item_14 0.15146 -1.00250 -0.16861 0.44573 1.33122
## Item_15 1.60686 -0.10649 3.32021 NA NA
## Item_16 -0.03661 -0.72445 -0.31561 0.93024 NA
## Item_17 0.53657 -0.62167 0.63705 0.27629 1.85460
## Item_18 0.60752 -0.06210 -0.03962 0.80344 1.72836
## Item_19 1.01741 -0.69155 0.45495 1.69836 2.60789
## Item_20 0.89728 -0.22565 0.64405 1.45999 1.71074
## Item_21 0.67030 0.25092 1.08968 NA NA
## Item_22 1.04952 0.51701 1.58203 NA NAAnd a Wright map:
plotPImap(PCM)
Additional Plots - Item Characteristic Curves
The plotICC() function lets you inspect how well the scales for item responses are working.
plotICC(PCM, item.subset = "Item_12")
Additional Plots - Item and Test Information Curves
Information is a concept related to how much we can tell about a person’s ability based on their performance. Importantly, information differs accross the range of person ability, and different items with different difficulties will tell us more about some people and less about others.
Information of items accumulates across the whole test. This lets us consider targeting at the test level. We can plot item and test information curves, and it’s actually information that we use to calculate reliability in item response modeling.
plotINFO(PCM, type = "test")