Introduction to the MST setup

item pool

We have 122 items:

The basic MST structure

We consider 6 modules organized as follows

Parameters for difficulty levels

In the above structure, we have set the following parameters.

Start difficulty level : 0
Difficulty increase : 1
Difficulty decrease : 1

If we change these, to, e.g., start = -1, increase = 0.8 and decrease=0.4 we get something maybe more suitable for our purposes? That is, more focused on the lower skill levels:

From theoretical to specific MST, based on difficulties defined

We need next (after the theoretical difficulties have been defined) to search for a certain number of items at each difficulty level.

Specify number of items in each module, and pick items

Let us for instance set the number of items in the start module to 6 and that the five other modules will have 6, 5, 6, 6 items each. We then look in our itempool for items (with as high discrimination as possible) that correspond to the theoretical difficulty values described in the above figure. The result is the following MST whose items are as follows

Specify the cutoff values

Next we need the cutoff values for routing. For simplicity we set them to be at least half of the items correctly answered. That is, for the start module we need 4 correct answers to move upwards. For the other modules we need 4 correct answers to move upwards.

Probability calculations for the fully specified MST

We have now fully specified a MST by setting the following parameters + Theoretical difficulty level for each module + How much to decrease and increase the difficulty level when branching + The number of items in each module + The cutoff values for branching

When this is done, we can exactly (no need to simulate) calculate the probability of a student of ability \(\theta\) to end up in the risk, low, med, and high categories. For instance, in our specific setup, the following table describes how the different categories have different probabilities when ability \(\theta\) changes:

theta	risk	low	med	high
-2.2	89.9	10.1	0	0
-1.6	51.8	46.8	1.4	0
-1	7.6	67.6	24.8	0
-0.4	0.1	20.6	77	2.2
0.2	0	0.8	74.8	24.4
0.8	0	0	40.1	59.9
1.4	0	0	14.3	85.7

We can also calculate the overall proportions for the entire student population:

risk	low	med	high
6.9	17.9	46.6	28.3

Investigating different parameter setups

Setup 1

Setup: + Start \(\theta=\)-1 + Increments: down 0.4 and up 0.8 + Number of items 7, 5, 5, 5, 5, 5 + Branching cutoffs 4, 3, 3, 3, 3, 3

Which results in the following MST

with probabilities across \(\theta\)

and total probabilities

risk	low	med	high
4	16.1	51.2	28.5

Setup 2

Setup: + Start \(\theta=\)-1.5 + Increments: down 0.2 and up 0.8 + Number of items 7, 5, 5, 5, 5, 5 + Branching cutoffs 4, 3, 3, 3, 3, 3