CoT Standard Setting
Session 2: ICS Construct
Implement Computational Solutions:
Be able to execute (or “read” and trace) a computational solution (e.g., algorithms, pseudo codes, simulations, models) to achieve a certain goal
ICS construct (public URL):
(please copy and paste these links)
https://cr4cr.bear-apps.com/constructs/public/dVkuM7iBOhp1cgPGJQQ6W8EvEuKoRe
Fall 2021 report (requires login):
https://cr4cr.bear-apps.com/table_of_reports?scheduled_activities=141
Fall 2021 (AFE) report (requires login):
https://cr4cr.bear-apps.com/table_of_reports?scheduled_activities=219
BASS page:
1 Construct levels
ICS5: Integrated Relational - Complex:
ICS4: Integrated Relational - Simple:
ICS3: Multi-step Solution:
ICS2: Single-step Solution:
ICS1: Attempting/Partial:
ICS0: Not Evident:
2 Analyses
Concurrent calibration with latent regression
- we stacked three datasets (by matching on common items), which are: (dataset 1) pre, (dataset 2) mid (AFE) and (dataset 3) post.
- we added indicator variables for mid and post as the predictors in the latent regression to allow separate means for pre (reference, mean=0), mid, and post timepoints.
3 Datasets (raw files)
There are n=876 respondents at pre, n=632 respondents at mid (AFE) and n=296 respondents at post. Raw data files are available here:
Scoring
4 Calibration
4.1 Model specification
## [1] "item+item*step"4.2 Latent regression coefficient (gain from pre to post)
## afe
## Main dimension -0.839
## S. errors 0.0544.3 Reliability
## EAP/PV RELIABILITY: 0.7424.4 Variance
## errors
## [1,] 1.012 0.0384.5 Item parameters
The parameter difficulty parameters may be thought of as a kind of “average” item difficulty for a partial credit item. This may be useful, if one wishes to have one indicative difficulty parameter for a partial credit item as a whole. Otherwise, to describe the difficulty of a partial credit item, one needs to describe the difficulties of individual score categories within the item, such as using the Thurstonian thresholds (Wu et al., 2016).
## n_item item est error U.fit U.Low U.High U.T W.fit W.Low W.High
## 1 1 Delivery.01.. -2.184 0.064 1.22 0.91 1.09 4.5 1.13 0.91 1.09
## 2 2 Delivery.02.. -0.689 0.055 1.00 0.91 1.09 0.1 1.01 0.92 1.08
## 3 3 Delivery.02.. 0.398 0.076 1.00 0.91 1.09 -0.1 1.00 0.93 1.07
## 4 4 Delivery.03.. 2.589 0.172 0.84 0.86 1.14 -2.4 1.01 0.77 1.23
## 5 5 Elevator.01.. -0.632 0.079 1.13 0.86 1.14 1.8 1.08 0.88 1.12
## 6 6 Elevator.03.. 0.868 0.059 0.97 0.86 1.14 -0.3 0.98 0.87 1.13
## 7 7 Elevator.04.. 0.010 0.115 1.05 0.85 1.15 0.6 1.05 0.92 1.08
## 8 8 LunchList.0.. 0.964 0.085 0.94 0.90 1.10 -1.1 0.99 0.93 1.07
## 9 9 Market.01d_.. -0.497 0.063 0.99 0.92 1.08 -0.3 1.00 0.95 1.05
## 10 10 Market.02b_.. 0.360 0.042 0.89 0.90 1.10 -2.3 0.89 0.90 1.10
## 11 11 Park.01c_NU.. -0.565 0.049 0.96 0.90 1.10 -0.8 0.98 0.92 1.08
## 12 12 Pizza.01a_F.. -1.864 0.142 0.73 0.86 1.14 -4.1 0.89 0.84 1.16
## 13 13 Pizza.01a_F.. -2.456 0.169 0.71 0.86 1.14 -4.4 0.95 0.77 1.23
## 14 14 Pizza.01a_F.. -1.690 0.136 1.23 0.86 1.14 3.1 1.08 0.85 1.15
## 15 15 Pizza.01a_F.. -2.386 0.165 0.72 0.86 1.14 -4.2 0.95 0.78 1.22
## 16 16 Pizza.01a_F.. -1.130 0.122 0.85 0.86 1.14 -2.2 0.92 0.89 1.11
## 17 17 Pizza.01a_F.. -1.546 0.132 1.14 0.86 1.14 1.9 1.09 0.86 1.14
## 18 18 Pizza.01a_F.. -0.398 0.113 1.24 0.86 1.14 3.2 1.18 0.92 1.08
## 19 19 Pizza.02b_F.. 0.431 0.114 1.22 0.86 1.14 2.9 1.10 0.92 1.08
## 20 20 Pizza.02b_F.. 0.434 0.114 1.17 0.86 1.14 2.3 1.08 0.92 1.08
## 21 21 Pizza.02b_F.. -1.887 0.144 0.80 0.86 1.14 -2.9 0.91 0.83 1.17
## 22 22 Shipping.01.. -1.145 0.067 0.83 0.92 1.08 -4.5 0.89 0.94 1.06
## 23 23 SignUp.01c_.. 0.601 0.064 1.24 0.86 1.14 3.2 1.21 0.87 1.13
## 24 24 SignUp.01d_.. -0.081 0.113 1.00 0.86 1.14 0.0 0.97 0.91 1.09
## 25 25 Sorting.03a.. -0.193 0.112 0.89 0.86 1.14 -1.5 0.92 0.92 1.08
## 26 26 TicTac.03_O.. -0.513 0.042 1.05 0.90 1.10 1.0 1.03 0.90 1.10
## 27 27 Video.01_MC 1.240 0.089 0.77 0.90 1.10 -4.8 0.90 0.92 1.08
## W.T
## 1 2.6
## 2 0.3
## 3 0.0
## 4 0.1
## 5 1.4
## 6 -0.3
## 7 1.3
## 8 -0.2
## 9 0.1
## 10 -2.3
## 11 -0.5
## 12 -1.3
## 13 -0.4
## 14 1.1
## 15 -0.4
## 16 -1.5
## 17 1.3
## 18 4.1
## 19 2.2
## 20 1.8
## 21 -1.0
## 22 -3.9
## 23 3.1
## 24 -0.6
## 25 -2.0
## 26 0.6
## 27 -2.4## W.fit: Weighted fit (Infit)## [,1] [,2]
## [1,] "item1" "Delivery.01a_MC"
## [2,] "item2" "Delivery.02a_MC"
## [3,] "item3" "Delivery.02b_NU"
## [4,] "item4" "Delivery.03c_FB"
## [5,] "item5" "Elevator.01a_NU"
## [6,] "item6" "Elevator.03_OE"
## [7,] "item7" "Elevator.04a_MC"
## [8,] "item8" "LunchList.01_MC"
## [9,] "item9" "Market.01d_MC"
## [10,] "item10" "Market.02b_OE"
## [11,] "item11" "Park.01c_NU.v2"
## [12,] "item12" "Pizza.01a_FB.1"
## [13,] "item13" "Pizza.01a_FB.2"
## [14,] "item14" "Pizza.01a_FB.3"
## [15,] "item15" "Pizza.01a_FB.4"
## [16,] "item16" "Pizza.01a_FB.5"
## [17,] "item17" "Pizza.01a_FB.6"
## [18,] "item18" "Pizza.01a_FB.7"
## [19,] "item19" "Pizza.02b_FB.1"
## [20,] "item20" "Pizza.02b_FB.2"
## [21,] "item21" "Pizza.02b_FB.3"
## [22,] "item22" "Shipping.01_NU"
## [23,] "item23" "SignUp.01c_MC"
## [24,] "item24" "SignUp.01d_MC"
## [25,] "item25" "Sorting.03a_NU"
## [26,] "item26" "TicTac.03_OE"
## [27,] "item27" "Video.01_MC"4.6 Step parameters
Delta is the point at which the probability of being in category k − 1 and category k is equals. This mathematical fact provides an interpretation for the delta (d) parameters (Wu et al., 2016)
## n_item item step est error U.fit U.Low U.High U.T W.fit W.Low
## 1 1 Delivery.01.. 0 NA NA 1.85 0.91 1.09 14.9 1.12 0.77
## 2 1 Delivery.01.. 1 -1.044 0.074 1.10 0.91 1.09 2.0 1.02 0.95
## 3 1 Delivery.01.. 2 1.044 NA 1.16 0.91 1.09 3.3 1.07 0.94
## 4 2 Delivery.02.. 0 NA NA 0.98 0.91 1.09 -0.5 1.01 0.91
## 5 2 Delivery.02.. 1 -1.181 0.066 0.99 0.91 1.09 -0.3 0.99 0.97
## 6 2 Delivery.02.. 2 1.181 NA 1.04 0.91 1.09 1.0 1.00 0.92
## 7 5 Elevator.01.. 0 NA NA 1.09 0.86 1.14 1.3 1.02 0.85
## 8 5 Elevator.01.. 1 -0.579 0.107 1.06 0.86 1.14 0.9 1.02 0.94
## 9 5 Elevator.01.. 2 0.579 NA 1.17 0.86 1.14 2.3 1.09 0.91
## 10 6 Elevator.03.. 0 NA NA 1.15 0.86 1.14 2.0 0.97 0.87
## 11 6 Elevator.03.. 1 -1.095 0.130 0.95 0.86 1.14 -0.7 0.98 0.85
## 12 6 Elevator.03.. 2 -1.457 0.122 0.95 0.86 1.14 -0.7 0.99 0.92
## 13 6 Elevator.03.. 3 0.128 0.142 0.99 0.86 1.14 -0.1 1.04 0.86
## 14 6 Elevator.03.. 4 2.424 NA 0.41 0.86 1.14 -10.7 0.97 0.43
## 15 10 Market.02b_.. 0 NA NA 0.95 0.90 1.10 -1.1 0.94 0.89
## 16 10 Market.02b_.. 1 -1.091 0.089 0.88 0.90 1.10 -2.5 0.97 0.90
## 17 10 Market.02b_.. 2 -1.273 0.080 0.95 0.90 1.10 -1.0 0.97 0.96
## 18 10 Market.02b_.. 3 1.258 0.112 0.70 0.90 1.10 -6.6 0.96 0.86
## 19 10 Market.02b_.. 4 1.105 NA 0.68 0.90 1.10 -7.2 1.03 0.82
## 20 11 Park.01c_NU.. 0 NA NA 1.00 0.90 1.10 0.0 0.97 0.92
## 21 11 Park.01c_NU.. 1 0.489 0.091 0.99 0.90 1.10 -0.3 1.00 0.89
## 22 11 Park.01c_NU.. 2 -0.489 NA 0.94 0.90 1.10 -1.2 1.00 0.92
## 23 23 SignUp.01c_.. 0 NA NA 1.45 0.86 1.14 5.6 1.20 0.88
## 24 23 SignUp.01c_.. 1 -0.283 0.112 1.09 0.86 1.14 1.2 1.02 0.82
## 25 23 SignUp.01c_.. 2 -1.336 0.114 1.13 0.86 1.14 1.7 1.09 0.92
## 26 23 SignUp.01c_.. 3 1.619 NA 1.26 0.86 1.14 3.4 0.96 0.78
## 27 26 TicTac.03_O.. 0 NA NA 0.93 0.90 1.10 -1.3 1.03 0.89
## 28 26 TicTac.03_O.. 1 -0.284 0.078 1.06 0.90 1.10 1.2 1.01 0.90
## 29 26 TicTac.03_O.. 2 0.337 0.096 0.99 0.90 1.10 -0.2 0.99 0.89
## 30 26 TicTac.03_O.. 3 -0.053 NA 1.04 0.90 1.10 0.7 1.04 0.92
## W.High W.T
## 1 1.23 1.0
## 2 1.05 0.8
## 3 1.06 2.3
## 4 1.09 0.1
## 5 1.03 -0.5
## 6 1.08 0.1
## 7 1.15 0.2
## 8 1.06 0.7
## 9 1.09 2.0
## 10 1.13 -0.5
## 11 1.15 -0.2
## 12 1.08 -0.2
## 13 1.14 0.6
## 14 1.57 0.0
## 15 1.11 -1.1
## 16 1.10 -0.6
## 17 1.04 -1.3
## 18 1.14 -0.6
## 19 1.18 0.3
## 20 1.08 -0.8
## 21 1.11 0.0
## 22 1.08 -0.1
## 23 1.12 3.0
## 24 1.18 0.2
## 25 1.08 2.3
## 26 1.22 -0.3
## 27 1.11 0.5
## 28 1.10 0.1
## 29 1.11 -0.1
## 30 1.08 0.94.7 Thurstonian thresholds
For partial credit items, to achieve a score of 2, students would generally need to accomplish more tasks than for achieving a score of 1. To reflect this “cumulative achievement”, the Thurstonian thresholds are sometimes used as indicators of “score difficulties”. The Thurstonian threshold for a score category is defined as the ability at which the probability of achieving that score or higher reaches 0.50.
## [,1] [,2] [,3] [,4]
## Delivery.01a_MC -3.333 -1.034 NA NA
## Delivery.02a_MC -1.953 0.575 NA NA
## Delivery.02b_NU 0.398 NA NA NA
## Delivery.03c_FB 2.589 NA NA NA
## Elevator.01a_NU -1.435 0.171 NA NA
## Elevator.03_OE -0.860 -0.189 1.100 3.382
## Elevator.04a_MC 0.010 NA NA NA
## LunchList.01_MC 0.965 NA NA NA
## Market.01d_MC -0.497 NA NA NA
## Market.02b_OE -1.281 -0.473 1.186 2.006
## Park.01c_NU.v2 -0.867 -0.263 NA NA
## Pizza.01a_FB.1 -1.864 NA NA NA
## Pizza.01a_FB.2 -2.456 NA NA NA
## Pizza.01a_FB.3 -1.689 NA NA NA
## Pizza.01a_FB.4 -2.386 NA NA NA
## Pizza.01a_FB.5 -1.130 NA NA NA
## Pizza.01a_FB.6 -1.546 NA NA NA
## Pizza.01a_FB.7 -0.398 NA NA NA
## Pizza.02b_FB.1 0.432 NA NA NA
## Pizza.02b_FB.2 0.435 NA NA NA
## Pizza.02b_FB.3 -1.887 NA NA NA
## Shipping.01_NU -1.146 NA NA NA
## SignUp.01c_MC -0.522 0.016 2.275 NA
## SignUp.01d_MC -0.081 NA NA NA
## Sorting.03a_NU -0.193 NA NA NA
## TicTac.03_OE -1.224 -0.421 0.137 NA
## Video.01_MC 1.240 NA NA NA## ICS1 ICS2 ICS3 ICS4
## Delivery.01a_MC -3.333 -1.034 NA NA
## Delivery.02a_MC -1.953 0.575 NA NA
## Delivery.02b_NU NA NA 0.398 NA
## Delivery.03c_FB NA NA NA 2.589
## Elevator.01a_NU -1.435 0.171 NA NA
## Elevator.03_OE -0.860 -0.189 1.100 3.382
## Elevator.04a_MC NA 0.010 NA NA
## LunchList.01_MC NA NA 0.965 NA
## Market.01d_MC NA -0.497 NA NA
## Market.02b_OE -1.281 -0.473 1.186 2.006
## Park.01c_NU.v2 -0.867 NA -0.263 NA
## Pizza.01a_FB.1 NA -1.864 NA NA
## Pizza.01a_FB.2 NA -2.456 NA NA
## Pizza.01a_FB.3 NA -1.689 NA NA
## Pizza.01a_FB.4 NA -2.386 NA NA
## Pizza.01a_FB.5 NA -1.130 NA NA
## Pizza.01a_FB.6 NA -1.546 NA NA
## Pizza.01a_FB.7 NA -0.398 NA NA
## Pizza.02b_FB.1 NA NA 0.432 NA
## Pizza.02b_FB.2 NA NA 0.435 NA
## Pizza.02b_FB.3 NA NA -1.887 NA
## Shipping.01_NU NA -1.146 NA NA
## SignUp.01c_MC -0.522 0.016 2.275 NA
## SignUp.01d_MC NA -0.081 NA NA
## Sorting.03a_NU NA -0.193 NA NA
## TicTac.03_OE -1.224 -0.421 0.137 NA
## Video.01_MC NA NA NA 1.240## ICS1 ICS2 ICS3 ICS4
## Min. :-3.3330 Min. :-2.4560 Min. :-1.8870 Min. :1.240
## 1st Qu.:-1.5645 1st Qu.:-1.3460 1st Qu.: 0.2023 1st Qu.:1.815
## Median :-1.2525 Median :-0.4730 Median : 0.4335 Median :2.297
## Mean :-1.4344 Mean :-0.7753 Mean : 0.4778 Mean :2.304
## 3rd Qu.:-0.8652 3rd Qu.:-0.1350 3rd Qu.: 1.0662 3rd Qu.:2.787
## Max. :-0.5220 Max. : 0.5750 Max. : 2.2750 Max. :3.382
## NA's :19 NA's :8 NA's :17 NA's :235 Links to CQ files (please copy/paste these links)
5.2 CTT Item Analysis (ITN) file
5.4 Command file
https://www.dropbox.com/s/hrpuvxtgudpjkni/command_ICS.txt?dl=0
5.8 Parameter file
https://www.dropbox.com/s/hc5ad30radfa1zs/ICS_PARMS.txt?dl=0
6 Item fit graph
7 Wright Map
8 Category Characteristic Curves (CCC)
9 Calibration summary from TAM
## ------------------------------------------------------------
## TAM 3.6-45 (2021-04-22 14:35:52)
## R version 4.0.5 (2021-03-31) x86_64, darwin17.0 | nodename=Permans-MacBook-Pro-4.local | login=root
##
## Date of Analysis: 2023-02-08 13:12:48
## Time difference of 0.1911449 secs
## Computation time: 0.1911449
##
## Multidimensional Item Response Model in TAM
##
## IRT Model: PCM2
## Call:
## TAM::tam.mml(resp = data2[, c(2:28)], Y = data2[, 29], irtmodel = "PCM2",
## verbose = FALSE)
##
## ------------------------------------------------------------
## Number of iterations = 31
## Numeric integration with 21 integration points
##
## Deviance = 22532.86
## Log likelihood = -11266.43
## Number of persons = 1455
## Number of persons used = 1454
## Number of items = 27
## Number of estimated parameters = 43
## Item threshold parameters = 41
## Item slope parameters = 0
## Regression parameters = 1
## Variance/covariance parameters = 1
##
## AIC = 22619 | penalty=86 | AIC=-2*LL + 2*p
## AIC3 = 22662 | penalty=129 | AIC3=-2*LL + 3*p
## BIC = 22846 | penalty=313.13 | BIC=-2*LL + log(n)*p
## aBIC = 22709 | penalty=176.41 | aBIC=-2*LL + log((n-2)/24)*p (adjusted BIC)
## CAIC = 22889 | penalty=356.13 | CAIC=-2*LL + [log(n)+1]*p (consistent AIC)
## AICc = 22622 | penalty=88.68 | AICc=-2*LL + 2*p + 2*p*(p+1)/(n-p-1) (bias corrected AIC)
## GHP = 0.71169 | GHP=( -LL + p ) / (#Persons * #Items) (Gilula-Haberman log penalty)
##
## ------------------------------------------------------------
## EAP Reliability
## [1] 0.749
## ------------------------------------------------------------
## Covariances and Variances
## [,1]
## [1,] 1.012
## ------------------------------------------------------------
## Correlations and Standard Deviations (in the diagonal)
## [,1]
## [1,] 1.006
## ------------------------------------------------------------
## Regression Coefficients
## [,1]
## [1,] 0.00000
## [2,] -0.83904
## ------------------------------------------------------------
## Standardized Coefficients
## parm dim est StdYX StdX StdY
## 1 Intercept 1 0.000 NA NA NA
## 2 Y1 1 -0.839 -0.3781 -0.4108 -0.7723
##
## ** Explained Variance R^2
## [1] 0.143
## ** SD Theta
## [1] 1.0864
## ** SD Predictors
## Intercept Y1
## 0.0000 0.4896
## ------------------------------------------------------------
## Item Parameters -A*Xsi
## item N M xsi.item AXsi_.Cat1 AXsi_.Cat2 AXsi_.Cat3
## 1 Delivery.01a_MC 945 1.567 -2.184 -3.228 -4.367 NA
## 2 Delivery.02a_MC 973 1.074 -0.689 -1.870 -1.377 NA
## 3 Delivery.02b_NU 971 0.332 0.398 0.398 NA NA
## 4 Delivery.03c_FB 393 0.107 2.589 2.589 NA NA
## 5 Elevator.01a_NU 395 1.251 -0.632 -1.210 -1.264 NA
## 6 Elevator.03_OE 380 1.529 0.868 -0.227 -0.816 0.180
## 7 Elevator.04a_MC 365 0.499 0.010 0.010 NA NA
## 8 LunchList.01_MC 772 0.319 0.965 0.965 NA NA
## 9 Market.01d_MC 1255 0.539 -0.496 -0.496 NA NA
## 10 Market.02b_OE 765 1.750 0.360 -0.731 -1.644 -0.026
## 11 Park.01c_NU.v2 847 1.079 -0.565 -0.076 -1.131 NA
## 12 Pizza.01a_FB.1 392 0.824 -1.864 -1.864 NA NA
## 13 Pizza.01a_FB.2 392 0.888 -2.456 -2.456 NA NA
## 14 Pizza.01a_FB.3 392 0.801 -1.690 -1.690 NA NA
## 15 Pizza.01a_FB.4 391 0.882 -2.386 -2.386 NA NA
## 16 Pizza.01a_FB.5 392 0.714 -1.130 -1.130 NA NA
## 17 Pizza.01a_FB.6 392 0.781 -1.546 -1.546 NA NA
## 18 Pizza.01a_FB.7 392 0.577 -0.398 -0.398 NA NA
## 19 Pizza.02b_FB.1 387 0.411 0.431 0.431 NA NA
## 20 Pizza.02b_FB.2 388 0.410 0.434 0.434 NA NA
## 21 Pizza.02b_FB.3 387 0.829 -1.887 -1.887 NA NA
## 22 Shipping.01_NU 1231 0.669 -1.145 -1.145 NA NA
## 23 SignUp.01c_MC 385 1.281 0.601 0.318 -0.417 1.803
## 24 SignUp.01d_MC 384 0.531 -0.081 -0.081 NA NA
## 25 Sorting.03a_NU 387 0.537 -0.193 -0.193 NA NA
## 26 TicTac.03_OE 769 1.901 -0.513 -0.797 -0.973 -1.538
## 27 Video.01_MC 769 0.270 1.240 1.240 NA NA
## AXsi_.Cat4 B.Cat1.Dim1 B.Cat2.Dim1 B.Cat3.Dim1 B.Cat4.Dim1
## 1 NA 1 2 0 0
## 2 NA 1 2 0 0
## 3 NA 1 0 0 0
## 4 NA 1 0 0 0
## 5 NA 1 2 0 0
## 6 3.473 1 2 3 4
## 7 NA 1 0 0 0
## 8 NA 1 0 0 0
## 9 NA 1 0 0 0
## 10 1.439 1 2 3 4
## 11 NA 1 2 0 0
## 12 NA 1 0 0 0
## 13 NA 1 0 0 0
## 14 NA 1 0 0 0
## 15 NA 1 0 0 0
## 16 NA 1 0 0 0
## 17 NA 1 0 0 0
## 18 NA 1 0 0 0
## 19 NA 1 0 0 0
## 20 NA 1 0 0 0
## 21 NA 1 0 0 0
## 22 NA 1 0 0 0
## 23 NA 1 2 3 0
## 24 NA 1 0 0 0
## 25 NA 1 0 0 0
## 26 NA 1 2 3 0
## 27 NA 1 0 0 0
##
## Item Parameters Xsi
## xsi se.xsi
## Delivery.01a_MC -2.183 0.064
## Delivery.02a_MC -0.689 0.055
## Delivery.02b_NU 0.398 0.076
## Delivery.03c_FB 2.589 0.172
## Elevator.01a_NU -0.632 0.079
## Elevator.03_OE 0.868 0.059
## Elevator.04a_MC 0.010 0.115
## LunchList.01_MC 0.965 0.085
## Market.01d_MC -0.496 0.063
## Market.02b_OE 0.360 0.042
## Park.01c_NU.v2 -0.565 0.049
## Pizza.01a_FB.1 -1.864 0.142
## Pizza.01a_FB.2 -2.456 0.169
## Pizza.01a_FB.3 -1.690 0.136
## Pizza.01a_FB.4 -2.386 0.165
## Pizza.01a_FB.5 -1.130 0.122
## Pizza.01a_FB.6 -1.546 0.132
## Pizza.01a_FB.7 -0.398 0.113
## Pizza.02b_FB.1 0.431 0.114
## Pizza.02b_FB.2 0.434 0.114
## Pizza.02b_FB.3 -1.887 0.144
## Shipping.01_NU -1.145 0.067
## SignUp.01c_MC 0.601 0.064
## SignUp.01d_MC -0.081 0.113
## Sorting.03a_NU -0.193 0.112
## TicTac.03_OE -0.513 0.042
## Video.01_MC 1.240 0.089
## Delivery.01a_MC_step1 -1.044 0.074
## Delivery.02a_MC_step1 -1.181 0.066
## Elevator.01a_NU_step1 -0.579 0.107
## Elevator.03_OE_step1 -1.095 0.130
## Elevator.03_OE_step2 -1.457 0.122
## Elevator.03_OE_step3 0.128 0.142
## Market.02b_OE_step1 -1.091 0.089
## Market.02b_OE_step2 -1.273 0.080
## Market.02b_OE_step3 1.258 0.112
## Park.01c_NU.v2_step1 0.489 0.091
## SignUp.01c_MC_step1 -0.283 0.112
## SignUp.01c_MC_step2 -1.336 0.114
## TicTac.03_OE_step1 -0.284 0.078
## TicTac.03_OE_step2 0.337 0.096
##
## Item Parameters in IRT parameterization
## item alpha beta tau.Cat1 tau.Cat2 tau.Cat3 tau.Cat4
## 1 Delivery.01a_MC 1 -2.184 -1.044 1.044 NA NA
## 2 Delivery.02a_MC 1 -0.689 -1.181 1.181 NA NA
## 3 Delivery.02b_NU 1 0.398 NA NA NA NA
## 4 Delivery.03c_FB 1 2.589 NA NA NA NA
## 5 Elevator.01a_NU 1 -0.632 -0.579 0.579 NA NA
## 6 Elevator.03_OE 1 0.868 -1.095 -1.457 0.128 2.425
## 7 Elevator.04a_MC 1 0.010 NA NA NA NA
## 8 LunchList.01_MC 1 0.965 NA NA NA NA
## 9 Market.01d_MC 1 -0.496 NA NA NA NA
## 10 Market.02b_OE 1 0.360 -1.091 -1.273 1.258 1.105
## 11 Park.01c_NU.v2 1 -0.565 0.489 -0.489 NA NA
## 12 Pizza.01a_FB.1 1 -1.864 NA NA NA NA
## 13 Pizza.01a_FB.2 1 -2.456 NA NA NA NA
## 14 Pizza.01a_FB.3 1 -1.690 NA NA NA NA
## 15 Pizza.01a_FB.4 1 -2.386 NA NA NA NA
## 16 Pizza.01a_FB.5 1 -1.130 NA NA NA NA
## 17 Pizza.01a_FB.6 1 -1.546 NA NA NA NA
## 18 Pizza.01a_FB.7 1 -0.398 NA NA NA NA
## 19 Pizza.02b_FB.1 1 0.431 NA NA NA NA
## 20 Pizza.02b_FB.2 1 0.434 NA NA NA NA
## 21 Pizza.02b_FB.3 1 -1.887 NA NA NA NA
## 22 Shipping.01_NU 1 -1.145 NA NA NA NA
## 23 SignUp.01c_MC 1 0.601 -0.283 -1.336 1.619 NA
## 24 SignUp.01d_MC 1 -0.081 NA NA NA NA
## 25 Sorting.03a_NU 1 -0.193 NA NA NA NA
## 26 TicTac.03_OE 1 -0.513 -0.284 0.337 -0.053 NA
## 27 Video.01_MC 1 1.240 NA NA NA NA