ICS unidim
ICS Construct (Fall 2021 and Spring 2022)
Design Computational Solutions:
Identify a problem and develop (creative) solutions by identifying input, output, and algorithms
ICS construct (public URL):
(please copy and paste these links)
https://cr4cr.bear-apps.com/constructs/public/5xXhDYnYmUve9c9DkpCgIMRsh6nCLI
Spring 2022 report (requires login):
https://cr4cr.bear-apps.com/scores_reports?scheduled_activities=267
BASS page:
1 Construct levels
ICS5: Integrated Relational - Complex:
ICS4: Integrated Relational - Simple:
ICS3: Multi-step Solution:
ICS2: Single-step Solution:
ICS1: Attempting/Partial:
ICS0: Not Evident:
2 Analyses
Unidimensional Rasch calibration of the ICS items (pre and post, 3 common items)
3 Datasets (raw files)
There are n=947 respondents at Pretest and n=295 at Posttest. Raw data file is available here:
4 Calibration
## ------------------------------------------------------------
## TAM 4.1-4 (2022-08-28 16:03:54)
## R version 4.2.2 (2022-10-31) x86_64, darwin17.0 | nodename=Permans-MacBook-Pro-4.local | login=root
##
## Date of Analysis: 2023-02-22 15:54:55
## Time difference of 0.220572 secs
## Computation time: 0.220572
##
## Multidimensional Item Response Model in TAM
##
## IRT Model: PCM2
## Call:
## TAM::tam.mml(resp = ics[, c(6:34)], Y = ics[, 5], irtmodel = "PCM2",
## verbose = FALSE)
##
## ------------------------------------------------------------
## Number of iterations = 39
## Numeric integration with 21 integration points
##
## Deviance = 18562.6
## Log likelihood = -9281.3
## Number of persons = 1242
## Number of persons used = 1143
## Number of items = 29
## Number of estimated parameters = 44
## Item threshold parameters = 42
## Item slope parameters = 0
## Regression parameters = 1
## Variance/covariance parameters = 1
##
## AIC = 18651 | penalty=88 | AIC=-2*LL + 2*p
## AIC3 = 18695 | penalty=132 | AIC3=-2*LL + 3*p
## BIC = 18872 | penalty=309.82 | BIC=-2*LL + log(n)*p
## aBIC = 18733 | penalty=169.91 | aBIC=-2*LL + log((n-2)/24)*p (adjusted BIC)
## CAIC = 18916 | penalty=353.82 | CAIC=-2*LL + [log(n)+1]*p (consistent AIC)
## AICc = 18654 | penalty=91.61 | AICc=-2*LL + 2*p + 2*p*(p+1)/(n-p-1) (bias corrected AIC)
## GHP = 0.68392 | GHP=( -LL + p ) / (#Persons * #Items) (Gilula-Haberman log penalty)
##
## ------------------------------------------------------------
## EAP Reliability
## [1] 0.656
## ------------------------------------------------------------
## Covariances and Variances
## [,1]
## [1,] 1.227
## ------------------------------------------------------------
## Correlations and Standard Deviations (in the diagonal)
## [,1]
## [1,] 1.108
## ------------------------------------------------------------
## Regression Coefficients
## [,1]
## [1,] 0.00000
## [2,] 0.34692
## ------------------------------------------------------------
## Standardized Coefficients
## parm dim est StdYX StdX StdY
## 1 Intercept 1 0.0000 NA NA NA
## 2 Y1 1 0.3469 0.1322 0.1477 0.3105
##
## ** Explained Variance R^2
## [1] 0.0175
## ** SD Theta
## [1] 1.1174
## ** SD Predictors
## Intercept Y1
## 0.0000 0.4257
## ------------------------------------------------------------
## Item Parameters -A*Xsi
## item N M xsi.item AXsi_.Cat1 AXsi_.Cat2
## 1 Delivery.01a_MC_F 397 1.474 -1.292 -2.178 -2.584
## 2 Delivery.02a_MC_F 397 0.688 -0.928 -0.928 NA
## 3 Delivery.02b_NU_F 397 0.355 0.807 0.807 NA
## 4 Elevator.01a_NU_F 395 1.251 -0.627 -1.228 -1.255
## 5 Elevator.03_OE_F 380 1.529 0.904 -0.241 -0.815
## 6 Elevator.04a_MC_F 365 0.499 0.028 0.028 NA
## 7 LunchList.01_MC_drag.v2_F 772 0.319 0.988 0.988 NA
## 8 Market.01d_MC_F 768 0.544 -0.192 -0.192 NA
## 9 Market.02b_OE_F 768 1.749 0.380 -0.772 -1.694
## 10 Park.01c_NU_F 388 1.263 -0.500 -0.473 -1.001
## 11 Pizza.01a_FB.1_F 392 0.824 -1.871 -1.871 NA
## 12 Pizza.01a_FB.2_F 392 0.888 -2.469 -2.469 NA
## 13 Pizza.01a_FB.3_F 392 0.801 -1.695 -1.695 NA
## 14 Pizza.01a_FB.4_F 391 0.882 -2.398 -2.398 NA
## 15 Pizza.01a_FB.5_F 392 0.714 -1.129 -1.129 NA
## 16 Pizza.01a_FB.6_F 392 0.781 -1.550 -1.550 NA
## 17 Pizza.01a_FB.7_F 392 0.577 -0.388 -0.388 NA
## 18 Pizza.02b_FB.1_F 387 0.411 0.453 0.453 NA
## 19 Pizza.02b_FB.2_F 388 0.410 0.456 0.456 NA
## 20 Pizza.02b_FB.3_F 387 0.829 -1.893 -1.893 NA
## 21 SignUp.01c_MC_F 385 1.281 0.617 0.278 -0.440
## 22 SignUp.01d_MC_F 384 0.531 -0.081 -0.081 NA
## 23 Sorting.03a_NU_F 387 0.537 -0.179 -0.179 NA
## 24 TicTac.03_OE_F 769 1.901 -0.523 -0.847 -1.034
## 25 Delivery.03c_FB_C 476 0.107 2.679 2.679 NA
## 26 Shipping.01_NU_C 1017 0.731 -1.136 -1.136 NA
## 27 Video.01_MC_drag.v2_C 1013 0.301 1.166 1.166 NA
## 28 LunchList.01_MC_S 195 0.456 0.534 0.534 NA
## 29 Market.02b_FB_S 77 0.481 0.492 0.492 NA
## AXsi_.Cat3 AXsi_.Cat4 B.Cat1.Dim1 B.Cat2.Dim1 B.Cat3.Dim1 B.Cat4.Dim1
## 1 NA NA 1 2 0 0
## 2 NA NA 1 0 0 0
## 3 NA NA 1 0 0 0
## 4 NA NA 1 2 0 0
## 5 0.227 3.617 1 2 3 4
## 6 NA NA 1 0 0 0
## 7 NA NA 1 0 0 0
## 8 NA NA 1 0 0 0
## 9 -0.031 1.520 1 2 3 4
## 10 NA NA 1 2 0 0
## 11 NA NA 1 0 0 0
## 12 NA NA 1 0 0 0
## 13 NA NA 1 0 0 0
## 14 NA NA 1 0 0 0
## 15 NA NA 1 0 0 0
## 16 NA NA 1 0 0 0
## 17 NA NA 1 0 0 0
## 18 NA NA 1 0 0 0
## 19 NA NA 1 0 0 0
## 20 NA NA 1 0 0 0
## 21 1.852 NA 1 2 3 0
## 22 NA NA 1 0 0 0
## 23 NA NA 1 0 0 0
## 24 -1.570 NA 1 2 3 0
## 25 NA NA 1 0 0 0
## 26 NA NA 1 0 0 0
## 27 NA NA 1 0 0 0
## 28 NA NA 1 0 0 0
## 29 NA NA 1 0 0 0
##
## Item Parameters Xsi
## xsi se.xsi
## Delivery.01a_MC_F -1.292 0.091
## Delivery.02a_MC_F -0.928 0.120
## Delivery.02b_NU_F 0.807 0.117
## Elevator.01a_NU_F -0.627 0.080
## Elevator.03_OE_F 0.904 0.060
## Elevator.04a_MC_F 0.028 0.116
## LunchList.01_MC_drag.v2_F 0.988 0.085
## Market.01d_MC_F -0.192 0.081
## Market.02b_OE_F 0.380 0.043
## Park.01c_NU_F -0.500 0.077
## Pizza.01a_FB.1_F -1.871 0.143
## Pizza.01a_FB.2_F -2.469 0.169
## Pizza.01a_FB.3_F -1.695 0.137
## Pizza.01a_FB.4_F -2.398 0.166
## Pizza.01a_FB.5_F -1.129 0.123
## Pizza.01a_FB.6_F -1.550 0.133
## Pizza.01a_FB.7_F -0.388 0.113
## Pizza.02b_FB.1_F 0.453 0.115
## Pizza.02b_FB.2_F 0.456 0.114
## Pizza.02b_FB.3_F -1.893 0.145
## SignUp.01c_MC_F 0.617 0.066
## SignUp.01d_MC_F -0.081 0.114
## Sorting.03a_NU_F -0.179 0.113
## TicTac.03_OE_F -0.523 0.043
## Delivery.03c_FB_C 2.679 0.158
## Shipping.01_NU_C -1.136 0.078
## Video.01_MC_drag.v2_C 1.166 0.076
## LunchList.01_MC_S 0.534 0.162
## Market.02b_FB_S 0.492 0.262
## Delivery.01a_MC_F_step1 -0.886 0.111
## Elevator.01a_NU_F_step1 -0.601 0.107
## Elevator.03_OE_F_step1 -1.145 0.131
## Elevator.03_OE_F_step2 -1.478 0.123
## Elevator.03_OE_F_step3 0.138 0.143
## Market.02b_OE_F_step1 -1.152 0.089
## Market.02b_OE_F_step2 -1.302 0.081
## Market.02b_OE_F_step3 1.284 0.113
## Park.01c_NU_F_step1 0.027 0.122
## SignUp.01c_MC_F_step1 -0.340 0.113
## SignUp.01c_MC_F_step2 -1.336 0.115
## TicTac.03_OE_F_step1 -0.324 0.079
## TicTac.03_OE_F_step2 0.337 0.096
##
## Item Parameters in IRT parameterization
## item alpha beta tau.Cat1 tau.Cat2 tau.Cat3 tau.Cat4
## 1 Delivery.01a_MC_F 1 -1.292 -0.886 0.886 NA NA
## 2 Delivery.02a_MC_F 1 -0.928 NA NA NA NA
## 3 Delivery.02b_NU_F 1 0.807 NA NA NA NA
## 4 Elevator.01a_NU_F 1 -0.627 -0.601 0.601 NA NA
## 5 Elevator.03_OE_F 1 0.904 -1.145 -1.478 0.138 2.486
## 6 Elevator.04a_MC_F 1 0.028 NA NA NA NA
## 7 LunchList.01_MC_drag.v2_F 1 0.988 NA NA NA NA
## 8 Market.01d_MC_F 1 -0.192 NA NA NA NA
## 9 Market.02b_OE_F 1 0.380 -1.152 -1.302 1.284 1.171
## 10 Park.01c_NU_F 1 -0.500 0.027 -0.027 NA NA
## 11 Pizza.01a_FB.1_F 1 -1.871 NA NA NA NA
## 12 Pizza.01a_FB.2_F 1 -2.469 NA NA NA NA
## 13 Pizza.01a_FB.3_F 1 -1.695 NA NA NA NA
## 14 Pizza.01a_FB.4_F 1 -2.398 NA NA NA NA
## 15 Pizza.01a_FB.5_F 1 -1.129 NA NA NA NA
## 16 Pizza.01a_FB.6_F 1 -1.550 NA NA NA NA
## 17 Pizza.01a_FB.7_F 1 -0.388 NA NA NA NA
## 18 Pizza.02b_FB.1_F 1 0.453 NA NA NA NA
## 19 Pizza.02b_FB.2_F 1 0.456 NA NA NA NA
## 20 Pizza.02b_FB.3_F 1 -1.893 NA NA NA NA
## 21 SignUp.01c_MC_F 1 0.617 -0.340 -1.336 1.675 NA
## 22 SignUp.01d_MC_F 1 -0.081 NA NA NA NA
## 23 Sorting.03a_NU_F 1 -0.179 NA NA NA NA
## 24 TicTac.03_OE_F 1 -0.523 -0.324 0.337 -0.013 NA
## 25 Delivery.03c_FB_C 1 2.679 NA NA NA NA
## 26 Shipping.01_NU_C 1 -1.136 NA NA NA NA
## 27 Video.01_MC_drag.v2_C 1 1.166 NA NA NA NA
## 28 LunchList.01_MC_S 1 0.534 NA NA NA NA
## 29 Market.02b_FB_S 1 0.492 NA NA NA NA## Item parameters
## |**********|
## |----------|
## Regression parameters
## |**|
## ||
## |--|## est.Dim1 se.Dim1
## 1 0.0000000 0.0000000
## 2 0.3469154 0.1107519Please see this link below for the calibration of the same dataset in ConQuest:
https://www.dropbox.com/sh/u3sy2wk3zg66vs5/AAA9oZRXvTKNlxNs0rptzqWJa?dl=0
4.1 Model specification
## [1] "item+item*step"4.2 Latent regression coefficient (gain from pre to post)
## hs
## Main dimension 0.348
## S. errors 0.078—> ## Reliability
## EAP/PV RELIABILITY: 0.6254.3 Variance
## errors
## [1,] 1.227 0.0494.4 Item parameters
The parameter difficulty parameters may be thought of as a kind of “average” item difficulty for a partial credit item. This may be useful, if one wishes to have one indicative difficulty parameter for a partial credit item as a whole. Otherwise, to describe the difficulty of a partial credit item, one needs to describe the difficulties of individual score categories within the item, such as using the Thurstonian thresholds (Wu et al., 2016).
## n_item item est error
## 1 1 Delivery.01.. -1.292 0.091
## 2 2 Delivery.02.. -0.928 0.120
## 3 3 Delivery.02.. 0.806 0.117
## 4 4 Elevator.01.. -0.628 0.080
## 5 5 Elevator.03.. 0.904 0.060
## 6 6 Elevator.04.. 0.028 0.116
## 7 7 LunchList.0.. 0.987 0.085
## 8 8 Market.01d_.. -0.192 0.081
## 9 9 Market.02b_.. 0.380 0.043
## 10 10 Park.01c_NU.. -0.501 0.077
## 11 11 Pizza.01a_F.. -1.872 0.143
## 12 12 Pizza.01a_F.. -2.470 0.169
## 13 13 Pizza.01a_F.. -1.696 0.137
## 14 14 Pizza.01a_F.. -2.398 0.166
## 15 15 Pizza.01a_F.. -1.130 0.123
## 16 16 Pizza.01a_F.. -1.551 0.133
## 17 17 Pizza.01a_F.. -0.389 0.113
## 18 18 Pizza.02b_F.. 0.453 0.115
## 19 19 Pizza.02b_F.. 0.456 0.114
## 20 20 Pizza.02b_F.. -1.893 0.145
## 21 21 SignUp.01c_.. 0.617 0.066
## 22 22 SignUp.01d_.. -0.081 0.114
## 23 23 Sorting.03a.. -0.180 0.113
## 24 24 TicTac.03_O.. -0.524 0.043
## 25 25 Delivery.03.. 2.679 0.158
## 26 26 Shipping.01.. -1.136 0.078
## 27 27 Video.01_MC.. 1.166 0.076
## 28 28 LunchList.0.. 0.535 0.162
## 29 29 Market.02b_.. 0.493 0.262## W.fit: Weighted fit (Infit)## [,1] [,2]
## [1,] "item1" "Delivery.01a_MC_F"
## [2,] "item2" "Delivery.02a_MC_F"
## [3,] "item3" "Delivery.02b_NU_F"
## [4,] "item4" "Elevator.01a_NU_F"
## [5,] "item5" "Elevator.03_OE_F"
## [6,] "item6" "Elevator.04a_MC_F"
## [7,] "item7" "LunchList.01_MC_drag.v2_F"
## [8,] "item8" "Market.01d_MC_F"
## [9,] "item9" "Market.02b_OE_F"
## [10,] "item10" "Park.01c_NU_F"
## [11,] "item11" "Pizza.01a_FB.1_F"
## [12,] "item12" "Pizza.01a_FB.2_F"
## [13,] "item13" "Pizza.01a_FB.3_F"
## [14,] "item14" "Pizza.01a_FB.4_F"
## [15,] "item15" "Pizza.01a_FB.5_F"
## [16,] "item16" "Pizza.01a_FB.6_F"
## [17,] "item17" "Pizza.01a_FB.7_F"
## [18,] "item18" "Pizza.02b_FB.1_F"
## [19,] "item19" "Pizza.02b_FB.2_F"
## [20,] "item20" "Pizza.02b_FB.3_F"
## [21,] "item21" "SignUp.01c_MC_F"
## [22,] "item22" "SignUp.01d_MC_F"
## [23,] "item23" "Sorting.03a_NU_F"
## [24,] "item24" "TicTac.03_OE_F"
## [25,] "item25" "Delivery.03c_FB_C"
## [26,] "item26" "Shipping.01_NU_C"
## [27,] "item27" "Video.01_MC_drag.v2_C"
## [28,] "item28" "LunchList.01_MC_S"
## [29,] "item29" "Market.02b_FB_S"4.5 Step parameters
Delta is the point at which the probability of being in category k − 1 and category k is equals. This mathematical fact provides an interpretation for the delta (d) parameters (Wu et al., 2016)
## n_item item step est error
## 1 1 Delivery.01.. 0 NA NA
## 2 1 Delivery.01.. 1 -0.886 0.111
## 3 1 Delivery.01.. 2 0.886 NA
## 4 4 Elevator.01.. 0 NA NA
## 5 4 Elevator.01.. 1 -0.601 0.107
## 6 4 Elevator.01.. 2 0.601 NA
## 7 5 Elevator.03.. 0 NA NA
## 8 5 Elevator.03.. 1 -1.146 0.131
## 9 5 Elevator.03.. 2 -1.478 0.123
## 10 5 Elevator.03.. 3 0.138 0.143
## 11 5 Elevator.03.. 4 2.486 NA
## 12 9 Market.02b_.. 0 NA NA
## 13 9 Market.02b_.. 1 -1.152 0.089
## 14 9 Market.02b_.. 2 -1.302 0.081
## 15 9 Market.02b_.. 3 1.284 0.113
## 16 9 Market.02b_.. 4 1.171 NA
## 17 10 Park.01c_NU.. 0 NA NA
## 18 10 Park.01c_NU.. 1 0.027 0.122
## 19 10 Park.01c_NU.. 2 -0.027 NA
## 20 21 SignUp.01c_.. 0 NA NA
## 21 21 SignUp.01c_.. 1 -0.340 0.113
## 22 21 SignUp.01c_.. 2 -1.336 0.115
## 23 21 SignUp.01c_.. 3 1.675 NA
## 24 24 TicTac.03_O.. 0 NA NA
## 25 24 TicTac.03_O.. 1 -0.324 0.079
## 26 24 TicTac.03_O.. 2 0.337 0.096
## 27 24 TicTac.03_O.. 3 -0.013 NA4.6 Thurstonian thresholds
For partial credit items, to achieve a score of 2, students would generally need to accomplish more tasks than for achieving a score of 1. To reflect this “cumulative achievement”, the Thurstonian thresholds are sometimes used as indicators of “score difficulties”. The Thurstonian threshold for a score category is defined as the ability at which the probability of achieving that score or higher reaches 0.50.
## [,1] [,2] [,3] [,4]
## Delivery.01a_MC_F -2.316 -0.269 NA NA
## Delivery.02a_MC_F -0.928 NA NA NA
## Delivery.02b_NU_F 0.807 NA NA NA
## Elevator.01a_NU_F -1.445 0.189 NA NA
## Elevator.03_OE_F -0.863 -0.178 1.144 3.475
## Elevator.04a_MC_F 0.028 NA NA NA
## LunchList.01_MC_drag.v2_F 0.987 NA NA NA
## Market.01d_MC_F -0.192 NA NA NA
## Market.02b_OE_F -1.312 -0.486 1.237 2.078
## Park.01c_NU_F -0.970 -0.031 NA NA
## Pizza.01a_FB.1_F -1.872 NA NA NA
## Pizza.01a_FB.2_F -2.470 NA NA NA
## Pizza.01a_FB.3_F -1.696 NA NA NA
## Pizza.01a_FB.4_F -2.398 NA NA NA
## Pizza.01a_FB.5_F -1.130 NA NA NA
## Pizza.01a_FB.6_F -1.551 NA NA NA
## Pizza.01a_FB.7_F -0.389 NA NA NA
## Pizza.02b_FB.1_F 0.453 NA NA NA
## Pizza.02b_FB.2_F 0.456 NA NA NA
## Pizza.02b_FB.3_F -1.894 NA NA NA
## SignUp.01c_MC_F -0.540 0.016 2.344 NA
## SignUp.01d_MC_F -0.081 NA NA NA
## Sorting.03a_NU_F -0.180 NA NA NA
## TicTac.03_OE_F -1.259 -0.429 0.147 NA
## Delivery.03c_FB_C 2.679 NA NA NA
## Shipping.01_NU_C -1.136 NA NA NA
## Video.01_MC_drag.v2_C 1.166 NA NA NA
## LunchList.01_MC_S 0.535 NA NA NA
## Market.02b_FB_S 0.493 NA NA NA## ICS1 ICS2 ICS3 ICS4
## Delivery.01a_MC_F -2.316 -0.269 NA NA
## Delivery.02a_MC_F -0.928 NA NA NA
## Delivery.02b_NU_F 0.807 NA NA NA
## Elevator.01a_NU_F -1.445 0.189 NA NA
## Elevator.03_OE_F -0.863 -0.178 1.144 3.475
## Elevator.04a_MC_F 0.028 NA NA NA
## LunchList.01_MC_drag.v2_F 0.987 NA NA NA
## Market.01d_MC_F -0.192 NA NA NA
## Market.02b_OE_F -1.312 -0.486 1.237 2.078
## Park.01c_NU_F -0.970 -0.031 NA NA
## Pizza.01a_FB.1_F -1.872 NA NA NA
## Pizza.01a_FB.2_F -2.470 NA NA NA
## Pizza.01a_FB.3_F -1.696 NA NA NA
## Pizza.01a_FB.4_F -2.398 NA NA NA
## Pizza.01a_FB.5_F -1.130 NA NA NA
## Pizza.01a_FB.6_F -1.551 NA NA NA
## Pizza.01a_FB.7_F -0.389 NA NA NA
## Pizza.02b_FB.1_F 0.453 NA NA NA
## Pizza.02b_FB.2_F 0.456 NA NA NA
## Pizza.02b_FB.3_F -1.894 NA NA NA
## SignUp.01c_MC_F -0.540 0.016 2.344 NA
## SignUp.01d_MC_F -0.081 NA NA NA
## Sorting.03a_NU_F -0.180 NA NA NA
## TicTac.03_OE_F -1.259 -0.429 0.147 NA
## Delivery.03c_FB_C 2.679 NA NA NA
## Shipping.01_NU_C -1.136 NA NA NA
## Video.01_MC_drag.v2_C 1.166 NA NA NA
## LunchList.01_MC_S 0.535 NA NA NA
## Market.02b_FB_S 0.493 NA NA NA## ICS1 ICS2 ICS3 ICS4
## Min. :-2.4700 Min. :-0.4860 Min. :0.1470 Min. :2.078
## 1st Qu.:-1.4450 1st Qu.:-0.3490 1st Qu.:0.8948 1st Qu.:2.427
## Median :-0.8630 Median :-0.1780 Median :1.1905 Median :2.776
## Mean :-0.5868 Mean :-0.1697 Mean :1.2180 Mean :2.776
## 3rd Qu.: 0.4530 3rd Qu.:-0.0075 3rd Qu.:1.5137 3rd Qu.:3.126
## Max. : 2.6790 Max. : 0.1890 Max. :2.3440 Max. :3.475
## NA's :22 NA's :25 NA's :27## X Cat1 Cat2 Cat3 Cat4
## 1 Delivery.01a_MC_F -2.3513489 -0.2762146 NA NA
## 2 Delivery.02a_MC_F NA -0.94125366 NA NA
## 3 Delivery.02b_NU_F NA NA 0.81436157 NA
## 4 Elevator.01a_NU_F -1.4579773 0.19290161 NA NA
## 5 Elevator.03_OE_F -0.8758850 -0.18173218 1.1639099 3.526337
## 6 Elevator.04a_MC_F NA 0.03012085 NA NA
## 7 LunchList.01_MC_drag.v2_F NA NA 0.99856567 NA
## 8 Market.01d_MC_F NA -0.19528198 NA NA
## 9 Market.02b_OE_F -1.3363953 -0.49978638 1.2649841 2.121918
## 10 Park.01c_NU_F -0.9910583 <NA> -0.03689575 NA
## 11 Pizza.01a_FB.1_F NA -1.88058472 NA NA
## 12 Pizza.01a_FB.2_F NA -2.48098755 NA NA
## 13 Pizza.01a_FB.3_F NA -1.70388794 NA NA
## 14 Pizza.01a_FB.4_F NA -2.4090271 NA NA
## 15 Pizza.01a_FB.5_F NA -1.13497925 NA NA
## 16 Pizza.01a_FB.6_F NA -1.55813599 NA NA
## 17 Pizza.01a_FB.7_F NA -0.38919067 NA NA
## 18 Pizza.02b_FB.1_F NA NA 0.45895386 NA
## 19 Pizza.02b_FB.2_F NA NA 0.46188354 NA
## 20 Pizza.02b_FB.3_F NA NA -1.90164185 NA
## 21 SignUp.01c_MC_F -0.5554504 0.0116272 2.3861389 NA
## 22 SignUp.01d_MC_F NA -0.08377075 NA NA
## 23 Sorting.03a_NU_F NA -0.17825317 NA NA
## 24 TicTac.03_OE_F -1.2814636 -0.44009399 0.1474915 NA
## 25 Delivery.03c_FB_C NA NA NA 2.70675659
## 26 Shipping.01_NU_C NA -1.146698 NA NA
## 27 Video.01_MC_drag.v2_C NA NA NA 1.17892456
## 28 LunchList.01_MC_S NA NA 0.54025269 NA
## 29 Market.02b_FB_S NA NA NA 0.50070195 Links to CQ files (please copy/paste these links)
https://www.dropbox.com/sh/u3sy2wk3zg66vs5/AAA9oZRXvTKNlxNs0rptzqWJa?dl=0
6 Item fit graph
7 Wright Map
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
## Warning: NAs introduced by coercion
8 Category Characteristic Curves (CCC)
9 Calibration summary from TAM
## ------------------------------------------------------------
## TAM 4.1-4 (2022-08-28 16:03:54)
## R version 4.2.2 (2022-10-31) x86_64, darwin17.0 | nodename=Permans-MacBook-Pro-4.local | login=root
##
## Date of Analysis: 2023-02-22 15:54:55
## Time difference of 0.220572 secs
## Computation time: 0.220572
##
## Multidimensional Item Response Model in TAM
##
## IRT Model: PCM2
## Call:
## TAM::tam.mml(resp = ics[, c(6:34)], Y = ics[, 5], irtmodel = "PCM2",
## verbose = FALSE)
##
## ------------------------------------------------------------
## Number of iterations = 39
## Numeric integration with 21 integration points
##
## Deviance = 18562.6
## Log likelihood = -9281.3
## Number of persons = 1242
## Number of persons used = 1143
## Number of items = 29
## Number of estimated parameters = 44
## Item threshold parameters = 42
## Item slope parameters = 0
## Regression parameters = 1
## Variance/covariance parameters = 1
##
## AIC = 18651 | penalty=88 | AIC=-2*LL + 2*p
## AIC3 = 18695 | penalty=132 | AIC3=-2*LL + 3*p
## BIC = 18872 | penalty=309.82 | BIC=-2*LL + log(n)*p
## aBIC = 18733 | penalty=169.91 | aBIC=-2*LL + log((n-2)/24)*p (adjusted BIC)
## CAIC = 18916 | penalty=353.82 | CAIC=-2*LL + [log(n)+1]*p (consistent AIC)
## AICc = 18654 | penalty=91.61 | AICc=-2*LL + 2*p + 2*p*(p+1)/(n-p-1) (bias corrected AIC)
## GHP = 0.68392 | GHP=( -LL + p ) / (#Persons * #Items) (Gilula-Haberman log penalty)
##
## ------------------------------------------------------------
## EAP Reliability
## [1] 0.656
## ------------------------------------------------------------
## Covariances and Variances
## [,1]
## [1,] 1.227
## ------------------------------------------------------------
## Correlations and Standard Deviations (in the diagonal)
## [,1]
## [1,] 1.108
## ------------------------------------------------------------
## Regression Coefficients
## [,1]
## [1,] 0.00000
## [2,] 0.34692
## ------------------------------------------------------------
## Standardized Coefficients
## parm dim est StdYX StdX StdY
## 1 Intercept 1 0.0000 NA NA NA
## 2 Y1 1 0.3469 0.1322 0.1477 0.3105
##
## ** Explained Variance R^2
## [1] 0.0175
## ** SD Theta
## [1] 1.1174
## ** SD Predictors
## Intercept Y1
## 0.0000 0.4257
## ------------------------------------------------------------
## Item Parameters -A*Xsi
## item N M xsi.item AXsi_.Cat1 AXsi_.Cat2
## 1 Delivery.01a_MC_F 397 1.474 -1.292 -2.178 -2.584
## 2 Delivery.02a_MC_F 397 0.688 -0.928 -0.928 NA
## 3 Delivery.02b_NU_F 397 0.355 0.807 0.807 NA
## 4 Elevator.01a_NU_F 395 1.251 -0.627 -1.228 -1.255
## 5 Elevator.03_OE_F 380 1.529 0.904 -0.241 -0.815
## 6 Elevator.04a_MC_F 365 0.499 0.028 0.028 NA
## 7 LunchList.01_MC_drag.v2_F 772 0.319 0.988 0.988 NA
## 8 Market.01d_MC_F 768 0.544 -0.192 -0.192 NA
## 9 Market.02b_OE_F 768 1.749 0.380 -0.772 -1.694
## 10 Park.01c_NU_F 388 1.263 -0.500 -0.473 -1.001
## 11 Pizza.01a_FB.1_F 392 0.824 -1.871 -1.871 NA
## 12 Pizza.01a_FB.2_F 392 0.888 -2.469 -2.469 NA
## 13 Pizza.01a_FB.3_F 392 0.801 -1.695 -1.695 NA
## 14 Pizza.01a_FB.4_F 391 0.882 -2.398 -2.398 NA
## 15 Pizza.01a_FB.5_F 392 0.714 -1.129 -1.129 NA
## 16 Pizza.01a_FB.6_F 392 0.781 -1.550 -1.550 NA
## 17 Pizza.01a_FB.7_F 392 0.577 -0.388 -0.388 NA
## 18 Pizza.02b_FB.1_F 387 0.411 0.453 0.453 NA
## 19 Pizza.02b_FB.2_F 388 0.410 0.456 0.456 NA
## 20 Pizza.02b_FB.3_F 387 0.829 -1.893 -1.893 NA
## 21 SignUp.01c_MC_F 385 1.281 0.617 0.278 -0.440
## 22 SignUp.01d_MC_F 384 0.531 -0.081 -0.081 NA
## 23 Sorting.03a_NU_F 387 0.537 -0.179 -0.179 NA
## 24 TicTac.03_OE_F 769 1.901 -0.523 -0.847 -1.034
## 25 Delivery.03c_FB_C 476 0.107 2.679 2.679 NA
## 26 Shipping.01_NU_C 1017 0.731 -1.136 -1.136 NA
## 27 Video.01_MC_drag.v2_C 1013 0.301 1.166 1.166 NA
## 28 LunchList.01_MC_S 195 0.456 0.534 0.534 NA
## 29 Market.02b_FB_S 77 0.481 0.492 0.492 NA
## AXsi_.Cat3 AXsi_.Cat4 B.Cat1.Dim1 B.Cat2.Dim1 B.Cat3.Dim1 B.Cat4.Dim1
## 1 NA NA 1 2 0 0
## 2 NA NA 1 0 0 0
## 3 NA NA 1 0 0 0
## 4 NA NA 1 2 0 0
## 5 0.227 3.617 1 2 3 4
## 6 NA NA 1 0 0 0
## 7 NA NA 1 0 0 0
## 8 NA NA 1 0 0 0
## 9 -0.031 1.520 1 2 3 4
## 10 NA NA 1 2 0 0
## 11 NA NA 1 0 0 0
## 12 NA NA 1 0 0 0
## 13 NA NA 1 0 0 0
## 14 NA NA 1 0 0 0
## 15 NA NA 1 0 0 0
## 16 NA NA 1 0 0 0
## 17 NA NA 1 0 0 0
## 18 NA NA 1 0 0 0
## 19 NA NA 1 0 0 0
## 20 NA NA 1 0 0 0
## 21 1.852 NA 1 2 3 0
## 22 NA NA 1 0 0 0
## 23 NA NA 1 0 0 0
## 24 -1.570 NA 1 2 3 0
## 25 NA NA 1 0 0 0
## 26 NA NA 1 0 0 0
## 27 NA NA 1 0 0 0
## 28 NA NA 1 0 0 0
## 29 NA NA 1 0 0 0
##
## Item Parameters Xsi
## xsi se.xsi
## Delivery.01a_MC_F -1.292 0.091
## Delivery.02a_MC_F -0.928 0.120
## Delivery.02b_NU_F 0.807 0.117
## Elevator.01a_NU_F -0.627 0.080
## Elevator.03_OE_F 0.904 0.060
## Elevator.04a_MC_F 0.028 0.116
## LunchList.01_MC_drag.v2_F 0.988 0.085
## Market.01d_MC_F -0.192 0.081
## Market.02b_OE_F 0.380 0.043
## Park.01c_NU_F -0.500 0.077
## Pizza.01a_FB.1_F -1.871 0.143
## Pizza.01a_FB.2_F -2.469 0.169
## Pizza.01a_FB.3_F -1.695 0.137
## Pizza.01a_FB.4_F -2.398 0.166
## Pizza.01a_FB.5_F -1.129 0.123
## Pizza.01a_FB.6_F -1.550 0.133
## Pizza.01a_FB.7_F -0.388 0.113
## Pizza.02b_FB.1_F 0.453 0.115
## Pizza.02b_FB.2_F 0.456 0.114
## Pizza.02b_FB.3_F -1.893 0.145
## SignUp.01c_MC_F 0.617 0.066
## SignUp.01d_MC_F -0.081 0.114
## Sorting.03a_NU_F -0.179 0.113
## TicTac.03_OE_F -0.523 0.043
## Delivery.03c_FB_C 2.679 0.158
## Shipping.01_NU_C -1.136 0.078
## Video.01_MC_drag.v2_C 1.166 0.076
## LunchList.01_MC_S 0.534 0.162
## Market.02b_FB_S 0.492 0.262
## Delivery.01a_MC_F_step1 -0.886 0.111
## Elevator.01a_NU_F_step1 -0.601 0.107
## Elevator.03_OE_F_step1 -1.145 0.131
## Elevator.03_OE_F_step2 -1.478 0.123
## Elevator.03_OE_F_step3 0.138 0.143
## Market.02b_OE_F_step1 -1.152 0.089
## Market.02b_OE_F_step2 -1.302 0.081
## Market.02b_OE_F_step3 1.284 0.113
## Park.01c_NU_F_step1 0.027 0.122
## SignUp.01c_MC_F_step1 -0.340 0.113
## SignUp.01c_MC_F_step2 -1.336 0.115
## TicTac.03_OE_F_step1 -0.324 0.079
## TicTac.03_OE_F_step2 0.337 0.096
##
## Item Parameters in IRT parameterization
## item alpha beta tau.Cat1 tau.Cat2 tau.Cat3 tau.Cat4
## 1 Delivery.01a_MC_F 1 -1.292 -0.886 0.886 NA NA
## 2 Delivery.02a_MC_F 1 -0.928 NA NA NA NA
## 3 Delivery.02b_NU_F 1 0.807 NA NA NA NA
## 4 Elevator.01a_NU_F 1 -0.627 -0.601 0.601 NA NA
## 5 Elevator.03_OE_F 1 0.904 -1.145 -1.478 0.138 2.486
## 6 Elevator.04a_MC_F 1 0.028 NA NA NA NA
## 7 LunchList.01_MC_drag.v2_F 1 0.988 NA NA NA NA
## 8 Market.01d_MC_F 1 -0.192 NA NA NA NA
## 9 Market.02b_OE_F 1 0.380 -1.152 -1.302 1.284 1.171
## 10 Park.01c_NU_F 1 -0.500 0.027 -0.027 NA NA
## 11 Pizza.01a_FB.1_F 1 -1.871 NA NA NA NA
## 12 Pizza.01a_FB.2_F 1 -2.469 NA NA NA NA
## 13 Pizza.01a_FB.3_F 1 -1.695 NA NA NA NA
## 14 Pizza.01a_FB.4_F 1 -2.398 NA NA NA NA
## 15 Pizza.01a_FB.5_F 1 -1.129 NA NA NA NA
## 16 Pizza.01a_FB.6_F 1 -1.550 NA NA NA NA
## 17 Pizza.01a_FB.7_F 1 -0.388 NA NA NA NA
## 18 Pizza.02b_FB.1_F 1 0.453 NA NA NA NA
## 19 Pizza.02b_FB.2_F 1 0.456 NA NA NA NA
## 20 Pizza.02b_FB.3_F 1 -1.893 NA NA NA NA
## 21 SignUp.01c_MC_F 1 0.617 -0.340 -1.336 1.675 NA
## 22 SignUp.01d_MC_F 1 -0.081 NA NA NA NA
## 23 Sorting.03a_NU_F 1 -0.179 NA NA NA NA
## 24 TicTac.03_OE_F 1 -0.523 -0.324 0.337 -0.013 NA
## 25 Delivery.03c_FB_C 1 2.679 NA NA NA NA
## 26 Shipping.01_NU_C 1 -1.136 NA NA NA NA
## 27 Video.01_MC_drag.v2_C 1 1.166 NA NA NA NA
## 28 LunchList.01_MC_S 1 0.534 NA NA NA NA
## 29 Market.02b_FB_S 1 0.492 NA NA NA NA