Let’s Look at the Item Dataframe

kable(items_diao,
      format = "simple",
      digits = 4)

item	a	b	c	Category
1	0.5451	-0.0889	0.1687	6
2	0.7121	-1.0725	0.2357	1
3	0.8428	-1.1112	0.1714	2
4	1.3758	-0.7056	0.2125	3
5	1.2558	-0.4370	0.1145	4
6	0.9606	0.3751	0.1839	2
7	0.8529	0.9476	0.2643	1
8	0.9207	0.0595	0.2572	6
9	0.9837	0.1803	0.2397	6
10	0.7321	-1.5548	0.1617	6
11	1.0019	-0.7283	0.2355	4
12	0.5758	-1.4657	0.1443	3
13	0.9654	0.6495	0.1086	6
14	0.8119	0.2544	0.1504	3
15	0.7843	-0.7167	0.2600	6
16	0.7172	-0.1214	0.1734	4
17	0.6490	-0.1143	0.1835	2
18	0.9087	0.4232	0.1804	2
19	0.6554	1.7763	0.2007	4
20	0.7186	1.4209	0.2240	3
21	1.0891	0.7325	0.2072	3
22	0.8820	1.0025	0.1614	2
23	0.6695	0.2466	0.2278	5
24	0.5928	0.1486	0.2715	6
25	0.9298	-0.4097	0.2159	3
26	0.8203	-0.2772	0.1335	6
27	0.8816	-0.1754	0.0669	1
28	1.0482	-0.8142	0.1650	1
29	0.7175	-0.6684	0.2889	5
30	0.7853	0.0774	0.2485	1
31	1.1618	-0.6762	0.1865	2
32	0.4492	1.6872	0.1534	2
33	0.8916	-0.0509	0.2472	5
34	0.8347	-1.4267	0.2228	6
35	1.1211	-1.1350	0.1777	4
36	0.6760	0.6482	0.1860	5
37	0.9760	-2.6028	0.2673	5
38	0.7511	-2.1080	0.1449	4
39	0.8457	1.4205	0.2238	5
40	0.5459	0.8194	0.2497	6
41	1.0467	1.4260	0.0846	2
42	0.4109	-0.2468	0.1840	5
43	0.6334	0.2066	0.1099	5
44	1.3368	-0.1457	0.1947	6
45	0.6433	-0.9625	0.2355	1
46	0.8442	0.2006	0.1596	1
47	0.5561	-0.5663	0.1225	2
48	0.6162	0.2480	0.1633	6
49	0.5472	1.3895	0.0595	1
50	0.7372	0.0399	0.1166	6
51	0.7094	-1.4823	0.1634	6
52	1.2726	-0.3798	0.1665	3
53	0.7376	-0.6579	0.1630	2
54	0.8054	-1.9076	0.3076	3
55	0.8754	0.6340	0.1412	3
56	0.7696	1.8275	0.2327	5
57	0.5832	0.1954	0.1548	2
58	0.9945	-1.6699	0.1765	2
59	0.8730	-1.2076	0.1932	1
60	0.6558	-1.1230	0.2211	6
61	0.9022	2.2723	0.3217	6
62	0.9476	0.5569	0.2379	5
63	0.4994	-0.8447	0.2330	5
64	1.2390	-0.1884	0.2775	6
65	1.1568	0.3723	0.1446	5
66	1.1785	-0.7906	0.1436	4
67	0.8349	1.0816	0.1940	5
68	0.5943	0.8151	0.2156	5
69	0.9392	-1.1567	0.1773	1
70	0.3565	0.2104	0.2564	5
71	0.8672	-0.8004	0.1933	3
72	0.8299	0.1429	0.2449	4
73	0.9817	-1.4388	0.1910	5
74	0.8661	0.5819	0.2965	4
75	0.8911	-0.3705	0.1906	6
76	0.7730	0.9238	0.2727	3
77	0.7583	-1.0983	0.1612	6
78	0.6445	0.3327	0.1635	3
79	0.6418	1.0529	0.0728	2
80	0.3861	0.5516	0.1916	6
81	0.7845	0.7270	0.2062	2
82	0.8090	-1.1287	0.2401	6
83	0.2347	0.2801	0.2141	4
84	0.8469	1.4291	0.1995	6
85	0.7474	-2.2093	0.2339	1
86	0.8887	-1.5301	0.1733	2
87	0.8812	0.0618	0.2014	6
88	0.4831	0.1968	0.1238	6
89	0.5258	0.1052	0.1578	6
90	0.9063	-0.7520	0.1144	3
91	0.8603	0.9589	0.1031	5
92	1.1209	-0.3716	0.1774	2
93	0.6332	-0.1757	0.2370	2
94	0.7587	0.2080	0.1956	6
95	0.7386	1.1495	0.0704	2
96	1.1148	1.4297	0.1568	1
97	1.1066	0.4332	0.1588	6
98	1.3721	-0.4357	0.2326	4
99	0.6520	-0.8987	0.2130	3
100	0.6854	1.8462	0.2012	4
101	0.9569	0.8762	0.1594	4
102	0.8478	-0.9354	0.3098	6
103	0.8025	0.7053	0.2149	1
104	1.0874	0.7533	0.1834	4
105	0.8237	0.9991	0.1839	3
106	0.9433	-0.2830	0.1722	4
107	1.1363	-1.0064	0.1611	5
108	1.1141	-0.2329	0.1530	2
109	0.4988	-0.6475	0.1475	3
110	1.0897	0.0519	0.2070	4
111	1.2177	-0.7259	0.1704	4
112	0.8956	-0.0447	0.1508	5
113	0.2975	-0.0159	0.2278	6
114	0.7005	-0.6654	0.1768	3
115	0.7350	-0.2396	0.1711	2
116	0.9546	-0.1184	0.2563	3
117	0.6955	0.4755	0.2105	6
118	0.9686	1.3816	0.1379	1
119	0.7341	-0.1900	0.2683	5
120	1.0056	1.1788	0.2327	4
121	0.9135	0.2073	0.1943	4
122	0.4378	-1.0406	0.1169	1
123	0.6812	-0.6661	0.2197	3
124	0.7546	0.7353	0.1703	4
125	0.6300	0.6098	0.1705	6
126	0.8845	-0.2364	0.1835	4
127	0.6153	-0.0109	0.1930	5
128	0.8063	-1.1690	0.1564	2
129	0.5146	0.1262	0.2210	2
130	0.6053	1.8740	0.2077	6
131	1.3161	-0.0511	0.1737	4
132	0.8443	1.3608	0.1792	5
133	0.8597	-0.3246	0.1466	1
134	0.7254	-0.5389	0.2093	6
135	0.7926	0.1964	0.2526	1
136	0.3661	1.8990	0.1663	5
137	0.6523	-0.5295	0.1322	4
138	0.9618	0.7579	0.1669	2
139	0.5176	-0.4871	0.1401	5
140	0.6560	-0.7045	0.0991	4
141	0.9311	0.5344	0.2052	3
142	1.1784	0.4069	0.2238	1
143	0.6642	-0.2135	0.1996	2
144	0.7397	-0.2650	0.2236	4
145	0.7136	-0.5407	0.2017	3
146	0.7844	0.3855	0.1356	5
147	0.8416	1.5004	0.1314	5
148	0.5888	1.8393	0.0840	4
149	0.7869	1.5623	0.2799	4
150	0.7623	-0.4772	0.2242	1
151	0.9020	0.9018	0.1448	4
152	1.1560	0.4668	0.0522	5
153	0.7501	0.3333	0.1915	4
154	0.6309	1.8388	0.2583	5
155	0.8728	2.3876	0.1832	2
156	0.6657	-0.2515	0.2177	1
157	0.9264	-0.4496	0.2441	6
158	0.6404	0.3761	0.1611	3
159	0.4899	0.2475	0.2104	2
160	0.7355	-0.6182	0.1873	1
161	0.6059	-0.0913	0.2371	6
162	1.0678	1.4791	0.2017	1
163	0.9209	-0.7301	0.1436	1
164	0.9448	-0.7485	0.1692	5
165	0.5343	0.7496	0.2044	6

Set the Stage

To start the automated test assembly process, we need to create some initial specifications. There are example items within the eatATA package (items_diao). The items_diao is a dataframe of 165 items and estimated parameters from a 3-PL IRT model. We won’t have all of our items calibrated in IRT. Many may be in CTT. Thus, the parameters within the dataframe will be different. If we use CTT, then the parameters will be difficulty and corrected point-biserial correlations. We will not have the guessing parameter or item information functions. Thus, the example code below must be amended, producing two types of ATA engines for use cases where we have psychometric information linked to our items. When we don’t have psychometric data, then ATA engine reduces to a content representation problem and becomes an easier problem to solve.

# Theta values
theta_values <- c(-1.5, 0, 1.5)


# Calculate item information for each theta-value
items_diao[, paste0("IIF_", theta_values)] <- calculateIIF(A = items_diao$a, 
                                           B = items_diao$b, 
                                           C = items_diao$c, 
                                           theta = theta_values)

# Specify target values for each theta-value
target_values <- structure(c(5.4, 10, 5.4),
                           names = paste0(theta_values))

Specify What a Successful Test Assembly Looks Like, The Objective Function

The objective function is the goal of the ATA engine. For this example, below we specify that the ATA engine should minimize the difference between the test information function (an additive property of the item information functions) and the target information. The code below specifies a minimax operation which minimizes the maximum difference between generated TIFs and the target TIF.

# Objective Function: minimize maximum difference between the TIF and 
#  the target values

minimaxTif <- combineConstraints(lapply(theta_values, 
                                        function(theta_value) {
  minimaxObjective(
    nForms = 2, 
    itemValues = items_diao[, paste0("IIF_", theta_value)],
    targetValue = target_values[as.character(theta_value)],
    itemIDs = items_diao$item
  )
}))

Specify Various Constraints on the Test Assembly Problem

Above we specified the objective function. Below we want to specify additional constraints on the test assembly problem that relate to the structure of the test and how many forms we want to construct. Observe there are item categories. These categories can related to topics or content domains. The below code specifies 2 Forms, the content categories come from the items_diao dataframe, our functional operator is greater than or equal to, we want to target 9, 9, 7, 9, 9, and 11 items per content category, and the itemIDs come from the items_diao dataframe.

Next, we specify how much overlap between the 2 Forms we want. We have created the “noOverlap” object and use the defaults for the itemUsageConstraint function. This means that we can select items once or not at all. Therefore, there is no overlap in items between Forms. We can also specify items to be in both Forms and create an anchor set or block of items that are common between the Forms. We would specify this by passing the argument “whichItems” and identifying the vector of items to use as the anchor. Finally, we specify that the total test length is 55 items, which corresponds (almost) to the vector above indicating how many items per content category.

# Other constraints
contentConstraints <- itemCategoryConstraint(
  nForms = 2, 
  itemCategories = items_diao$Category,
  operator = ">=",
  targetValues = c(9, 9, 7, 9, 9, 11),
  itemIDs = items_diao$item)

noOverlap <- itemUsageConstraint(
  nForms = 2, 
  itemIDs = items_diao$item)

testLength <- itemsPerFormConstraint(
  nForms = 2, 
  operator = "=", 
  targetValue = 55,
  itemIDs = items_diao$item)

Now We Solve the Test Assembly Problem

Below is the code that will execute the test assembly solvers. We pass a list with the objective function, content constraints, and test structure objects that we created above. The solver is based on linear programming and mixed integer programming. The solver ran for about 15 seconds and produces a message indicating the the solution is feasible, but may not be optimal. This outcome may be quite frequent with sub-optimal item banks. The output is a list of three columns with 1s and 0s for each itemID. Two columns are the Forms and one column is the itemID. 1 indicates that the item was chosen for the Form, 0 means it was not chosen.

# Call GLPK solver
solver_out <- useSolver(list(minimaxTif,
                             contentConstraints,
                             noOverlap,
                             testLength),
                        solver = "GLPK", 
                        timeLimit = 60, 
                        verbose = FALSE)

## The solution is feasible, but may not be optimal

Let’s Inspect the Solution and Append it to Our Item Pool

We should inspect the solution before we publish it as an official test. The output is two dataframes, one for each constructed Form. The dataframe contains the items chosen for each Form and their psychometric and category information. If we are happy with the solution, we can append the item selection (1,0) for each Form to the original items_diao dataframe. At this point, we have construct our two Forms from the item bank based on content category and psychometric information according to our objective function and defined constraints. Assuming we had the code generic enough to run on many different item bank dataframes and the objective function and psychometric information were available, the total time to construct the two Forms would be approximately 20 seconds of run time.

# Inspect solution
inspectSolution(solver_out, items = items_diao, idCol = "item")

## $form_1
##     item          a           b           c Category    IIF_-1.5       IIF_0
## 2      2  0.7120737  -1.0724676  0.23574647        1 0.187765299  0.17725993
## 7      7  0.8529416   0.9476441  0.26425560        1 0.004123528  0.12196416
## 10    10  0.7320588  -1.5547915  0.16167142        6 0.281681543  0.13991547
## 11    11  1.0018750  -0.7282744  0.23552461        4 0.197231339  0.36131112
## 12    12  0.5757638  -1.4657497  0.14433079        3 0.178277016  0.12313041
## 15    15  0.7843303  -0.7167498  0.26000603        6 0.145683161  0.23992301
## 17    17  0.6490310  -0.1143393  0.18346260        2 0.078830756  0.21306754
## 19    19  0.6554314   1.7762835  0.20071128        4 0.002806267  0.04315849
## 23    23  0.6694532   0.2466422  0.22784671        5 0.039795601  0.18834352
## 25    25  0.9297526  -0.4096674  0.21594148        3 0.113897133  0.39689267
## 29    29  0.7174636  -0.6683747  0.28892572        5 0.114987834  0.19945991
## 31    31  1.1618373  -0.6761522  0.18654561        2 0.223417102  0.49876629
## 34    34  0.8346871  -1.4266646  0.22283829        6 0.312794262  0.15664895
## 36    36  0.6759513   0.6482477  0.18600942        5 0.024211758  0.16858263
## 42    42  0.4109429  -0.2468453  0.18398225        5 0.057356868  0.08557638
## 45    45  0.6432903  -0.9624621  0.23554304        1 0.147401754  0.16202497
## 46    46  0.8442388   0.2006491  0.15963336        1 0.045031134  0.34949188
## 47    47  0.5561030  -0.5662843  0.12247742        2 0.125252796  0.17044265
## 51    51  0.7094353  -1.4823428  0.16339601        6 0.260672797  0.14545277
## 59    59  0.8730333  -1.2075993  0.19320309        1 0.326616754  0.21074845
## 60    60  0.6557997  -1.1229940  0.22112434        6 0.173296899  0.15743954
## 70    70  0.3564872   0.2104149  0.25637491        5 0.030637369  0.05266321
## 71    71  0.8671829  -0.8004461  0.19331690        3 0.220257432  0.29754456
## 72    72  0.8299203   0.1429151  0.24488410        4 0.035182113  0.28622710
## 74    74  0.8660711   0.5818857  0.29654184        4 0.008824966  0.18782613
## 76    76  0.7730364   0.9238037  0.27272424        3 0.006310192  0.11560139
## 79    79  0.6418133   1.0529236  0.07279458        2 0.027718266  0.16406299
## 80    80  0.3860953   0.5515907  0.19155506        6 0.032858395  0.06609537
## 81    81  0.7845396   0.7269587  0.20616796        2 0.013057604  0.18237117
## 82    82  0.8089940  -1.1286795  0.24008363        6 0.240618879  0.19725992
## 85    85  0.7473774  -2.2092746  0.23392824        1 0.231970821  0.06548958
## 89    89  0.5258156   0.1051565  0.15781724        6 0.062875047  0.14305951
## 93    93  0.6332213  -0.1757039  0.23702409        2 0.069550770  0.18314102
## 102  102  0.8478397  -0.9354204  0.30978557        6 0.179583203  0.21718889
## 109  109  0.4988280  -0.6474654  0.14749428        3 0.103427142  0.13110169
## 114  114  0.7005453  -0.6654463  0.17684477        3 0.155765979  0.23188215
## 115  115  0.7350316  -0.2395516  0.17112642        2 0.100671334  0.28078686
## 116  116  0.9546083  -0.1183986  0.25630357        3 0.049822414  0.40045954
## 117  117  0.6954776   0.4755188  0.21046516        6 0.027953983  0.18638480
## 119  119  0.7341487  -0.1900076  0.26831004        5 0.065486925  0.23187777
## 122  122  0.4378017  -1.0405517  0.11685802        1 0.102011640  0.10024995
## 124  124  0.7545973   0.7352842  0.17031266        4 0.017388255  0.19159637
## 125  125  0.6300442   0.6097675  0.17049652        6 0.030925482  0.16136760
## 128  128  0.8062514  -1.1689514  0.15636885        2 0.302130769  0.21442854
## 129  129  0.5145828   0.1261935  0.22097280        2 0.048687184  0.11917752
## 135  135  0.7926377   0.1964320  0.25257458        1 0.032633690  0.25084721
## 136  136  0.3660670   1.8989604  0.16632128        5 0.013034110  0.03760666
## 137  137  0.6522686  -0.5295331  0.13218749        4 0.145785947  0.22825165
## 139  139  0.5176028  -0.4870829  0.14006039        5 0.102368874  0.14574232
## 140  140  0.6560199  -0.7045368  0.09906439        4 0.186625136  0.23057552
## 148  148  0.5888375   1.8393191  0.08402353        4 0.008959215  0.07090187
## 153  153  0.7501147   0.3333416  0.19153456        4 0.035369592  0.24304508
## 154  154  0.6308509   1.8387900  0.25825939        5 0.002190088  0.03203990
## 157  157  0.9263769  -0.4496143  0.24411745        6 0.110983881  0.37001383
## 160  160  0.7355013  -0.6181665  0.18733496        1 0.151985850  0.25268656
## Sum 4452 38.7920821 -10.3618671 11.02720503     <NA> 5.994784148 10.57915495
##        IIF_1.5
## 2   0.04513363
## 7   0.29570355
## 10  0.02764407
## 11  0.04743133
## 12  0.04010254
## 15  0.06103306
## 17  0.11896432
## 19  0.19032523
## 23  0.14806753
## 25  0.08611980
## 29  0.06417948
## 31  0.04188159
## 34  0.02375928
## 36  0.19945231
## 42  0.06646315
## 45  0.05343013
## 46  0.19619796
## 47  0.08384183
## 51  0.03145816
## 59  0.03072743
## 60  0.04627538
## 70  0.05265016
## 71  0.05487890
## 72  0.16241807
## 74  0.23129585
## 76  0.24201693
## 79  0.24903447
## 80  0.07177520
## 81  0.25488571
## 82  0.03644445
## 85  0.01088512
## 89  0.11162903
## 93  0.10329686
## 102 0.04003575
## 109 0.07179087
## 114 0.07550183
## 115 0.11642107
## 116 0.12096887
## 117 0.18363676
## 119 0.10650963
## 122 0.05477716
## 124 0.25453870
## 125 0.17929794
## 128 0.03867463
## 129 0.09935047
## 135 0.16326534
## 136 0.06551670
## 137 0.09074181
## 139 0.08206896
## 140 0.08066961
## 148 0.19943481
## 153 0.18941360
## 154 0.15049503
## 157 0.07856098
## 160 0.07738618
## Sum 5.99845919
## 
## $form_2
##     item          a           b           c Category     IIF_-1.5        IIF_0
## 1      1  0.5451317 -0.08886849  0.16874715        6 7.365592e-02  0.154206546
## 3      3  0.8427645 -1.11124980  0.17141925        2 3.031511e-01  0.230901288
## 16    16  0.7172287 -0.12136333  0.17339638        4 8.420296e-02  0.265796276
## 20    20  0.7186465  1.42093504  0.22398215        3 3.454402e-03  0.064966691
## 24    24  0.5928138  0.14857070  0.27148424        6 4.083757e-02  0.139844624
## 28    28  1.0482391 -0.81419252  0.16500326        1 2.987609e-01  0.392668120
## 30    30  0.7852769  0.07738795  0.24846123        1 4.261897e-02  0.261815399
## 32    32  0.4491527  1.68721139  0.15343355        2 1.331077e-02  0.053727096
## 33    33  0.8916021 -0.05090641  0.24724434        5 4.831669e-02  0.351298848
## 37    37  0.9759502 -2.60275520  0.26730164        5 2.304513e-01  0.026075789
## 38    38  0.7510756 -2.10795028  0.14486710        4 2.820791e-01  0.082071447
## 40    40  0.5459201  0.81942834  0.24973544        6 1.913406e-02  0.091427469
## 43    43  0.6333612  0.20655156  0.10990036        5 7.233634e-02  0.224055526
## 48    48  0.6162020  0.24800679  0.16327744        6 5.413247e-02  0.186275934
## 49    49  0.5471755  1.38946730  0.05953001        1 2.588406e-02  0.113037017
## 53    53  0.7375985 -0.65790060  0.16296118        2 1.716110e-01  0.260243344
## 54    54  0.8053586 -1.90760309  0.30762326        3 2.554742e-01  0.080860457
## 56    56  0.7696217  1.82751255  0.23268550        5 8.624359e-04  0.028489265
## 57    57  0.5832024  0.19538004  0.15484367        2 6.002559e-02  0.173201355
## 61    61  0.9022153  2.27233316  0.32170746        6 4.604021e-05  0.004004111
## 63    63  0.4994277 -0.84465387  0.23299969        5 9.104753e-02  0.109424599
## 68    68  0.5942900  0.81506686  0.21563052        5 1.984571e-02  0.113805697
## 69    69  0.9391892 -1.15670002  0.17732545        1 3.725229e-01  0.240087356
## 73    73  0.9816789 -1.43878603  0.19097476        5 4.637429e-01  0.168736415
## 77    77  0.7582504 -1.09832247  0.16123222        6 2.566317e-01  0.210800807
## 78    78  0.6444879  0.33267668  0.16354345        3 4.724255e-02  0.196577816
## 83    83  0.2347116  0.28010785  0.21414117        4 1.925448e-02  0.025157199
## 86    86  0.8886972 -1.53005897  0.17326934        2 4.045244e-01  0.152190344
## 88    88  0.4831155  0.19681827  0.12379970        6 6.281556e-02  0.128146496
## 92    92  1.1208505 -0.37162135  0.17742828        2 1.105958e-01  0.607266679
## 99    99  0.6520086 -0.89874820  0.21295880        3 1.532235e-01  0.176562526
## 100  100  0.6854355  1.84624140  0.20119303        4 1.931046e-03  0.037093832
## 103  103  0.8024614  0.70529712  0.21486880        1 1.223901e-02  0.187060892
## 106  106  0.9433105 -0.28299484  0.17220256        4 1.047944e-01  0.455846141
## 110  110  1.0897285  0.05190133  0.20704177        4 2.946060e-02  0.552771070
## 113  113  0.2974700 -0.01594790  0.22784164        6 2.901657e-02  0.040265108
## 123  123  0.6811525 -0.66607503  0.21967972        3 1.325053e-01  0.205368257
## 126  126  0.8844605 -0.23637745  0.18351686        4 9.381649e-02  0.396231827
## 127  127  0.6152624 -0.01087976  0.19298547        5 6.622242e-02  0.185347355
## 130  130  0.6052676  1.87403562  0.20774895        6 3.188025e-03  0.038273117
## 133  133  0.8597210 -0.32456241  0.14661285        1 1.294044e-01  0.394980633
## 134  134  0.7254051 -0.53891829  0.20933600        6 1.279675e-01  0.243484697
## 143  143  0.6642006 -0.21352353  0.19955776        2 8.462461e-02  0.217375271
## 144  144  0.7397371 -0.26500730  0.22359265        4 8.605336e-02  0.257358102
## 145  145  0.7135898 -0.54073042  0.20173132        3 1.294409e-01  0.239189250
## 149  149  0.7869260  1.56229277  0.27991175        4 1.162609e-03  0.038686806
## 150  150  0.7623060 -0.47718474  0.22423855        1 1.171709e-01  0.264492694
## 155  155  0.8728499  2.38762954  0.18321422        2 9.360474e-05  0.006701664
## 156  156  0.6657217 -0.25152802  0.21769041        1 8.328206e-02  0.210945105
## 158  158  0.6403522  0.37612494  0.16114385        3 4.506327e-02  0.191834688
## 159  159  0.4898548  0.24754152  0.21037678        2 4.417267e-02  0.107626878
## 161  161  0.6059048 -0.09129972  0.23709358        6 6.188204e-02  0.166069992
## 163  163  0.9208685 -0.73013259  0.14357431        1 2.517150e-01  0.367912516
## 164  164  0.9447859 -0.74846468  0.16917688        5 2.425955e-01  0.362408019
## 165  165  0.5342723  0.74960968  0.20440464        6 2.586800e-02  0.104318477
## Sum 4916 39.7862889 -0.47717893 10.87964234     <NA> 5.985463e+00 10.585364927
##         IIF_1.5
## 1   0.104305397
## 3   0.038349427
## 16  0.128227055
## 20  0.240066941
## 24  0.112304598
## 28  0.041439391
## 30  0.146244713
## 32  0.104301256
## 33  0.134214516
## 37  0.002225213
## 38  0.013625820
## 40  0.129284787
## 43  0.160101981
## 48  0.147041684
## 49  0.192560071
## 53  0.076432049
## 54  0.011958400
## 56  0.231179262
## 57  0.134619555
## 61  0.139492027
## 63  0.056659705
## 68  0.160640385
## 69  0.029238567
## 73  0.016436614
## 77  0.045410578
## 78  0.163572949
## 83  0.026066269
## 86  0.018967742
## 88  0.107803086
## 92  0.079428310
## 99  0.058273895
## 100 0.200133510
## 103 0.257249006
## 106 0.108068626
## 110 0.160754478
## 113 0.038674906
## 123 0.071559802
## 126 0.116131279
## 127 0.120234330
## 130 0.154850059
## 133 0.109622211
## 134 0.081883600
## 143 0.109398683
## 144 0.105966845
## 145 0.082669089
## 149 0.246593116
## 150 0.085137476
## 155 0.177987640
## 156 0.103525717
## 158 0.166708225
## 159 0.098224186
## 161 0.105356039
## 163 0.059952102
## 164 0.054645405
## 165 0.132689880
## Sum 5.998488453

# Append solution to item pool
items_diao_out <- appendSolution(solver_out, items = items_diao, idCol = "item")

Let’s Actually See What the Solution Looks Like

If we wanted to see the TIFs for the two Forms we can execute the code below. This will be an easier way to inspect the solution. Observe, the two TIFs overlap quite a bit except for regions in the upper proficiency range.

# Plot results
thetas <- seq(-4, 4, 0.05)
IIFsPlot <- with(items_diao, 
                 calculateIIF(A = a, 
                              B = b, 
                              C = c, 
                              theta = thetas))

TIFs <- t(IIFsPlot) %*% as.matrix(solver_out$item_matrix)

plot(0, 0, type = "n", ylim = c(0, max(TIFs)), xlim = range(thetas), 
     xlab = "Theta - Ability",
     ylab = "Test Information")
abline(v = theta_values, lty = 3, col = "gray50")
matlines(thetas, TIFs, lwd = 3)
legend("topright", paste0("Test form ", 1:2), lwd = 3, lty = 1:2, col = 1:2)

ATA Example for a High Stakes Test

Stephen Gunter, Ph.D.

2023-03-09