1. Conditioning Variables for Marginal Maximum Likelihood Estimation

Under Marginal Maximum Likelihood Estimation, we have the additional assumption that ability thetas are normally distributed. Where the population of interest is split into subgroups, for example males and females, the assumption of normality is applied by assuming that each of the two groups has its own underlying normal distribution.

1.1 Categorical conditioning variables

It is obviously important that we get the population assumption right, otherwise our estimates of the ability of the population will be incorrect. Therefore, in our data, we need to correctly categorise sub-population groups.

When this is done, the population mean becomes the grand mean, and \(\alpha\) becomes the coefficient for the associated dummy-coded regressor.

When these specifications are applied to the model, plausible values drawn from it will account for the different prior population distribution patterns.

We can use simulation to illustrate two different distributional patterns for male and female subpopulations.

####################### Generate response data with male and female distributions #######################
rm(list = ls())                                               # clears global environment
library(TAM)                                                  # loads IRT package
set.seed(6778)                                                # sets seed so all this can be replicated
N <- 3000                                                     # sets number of students
theta <- c(rnorm(N/2,mean=0,sd=1.5),rnorm(N/2,mean=.5,sd=1))  # defines real male-female ability distributions
I <- 20                                                       # defines number of items
delta <- seq( -2 , 2 , len=I )                                # defines item difficulty vector
p1 <- plogis( outer( theta , delta , "-" ) )                  # creates probability (P) matrix
resp <- 1 * ( p1 > matrix( runif( N*I ) , nrow=N , ncol=I ) ) # generates actual item response data
colnames(resp) <- paste("I" , 1:I, sep="")                    # adds column names
gender <- rep(1:2,each=N/2)                                   # generates categorical variables

#################### Account for the different male-female ability distributions ########################
mod1 <- tam.mml(resp, group=gender)   # group is gender (it could include a large number of categories)
summary(mod1)
PV <- tam.pv(mod1)
pv1 <- PV$pv$PV1.Dim1
G1 <- pv1[gender==1]  # male
G2 <- pv1[gender==2]  # female
mean(G1)
mean(G2)
sd(G1)
sd(G2)

1.1.1 Results for conditioning variables when specifying categoric group

The results for the model (PVs) that includes the specification of the conditioning variables is as follows:

————————————————————
EAP Reliability
[1] 0.845
————————————————————
Covariances and Variances
Group1 Group2
2.350 1.058
————————————————————
Correlations and Standard Deviations (in the diagonal)
Group1 Group2
1.533 1.029
————————————————————
Regression Coefficients
[,1] [1,] 0.00000
[2,] 0.52928
*————————————————————

The main results (summary) outputted above, pertaining to the population of interest, note the standard deviation for males (Group 1) as SD= 1.533, and the standard deviation for females (Group 2) as SD=1.029. These values being almost identical to the real mean and standard deviations. Also, the regression coefficient is 0.529 for the second group, illustrating that the correct difference in mean theta values between males and females has been correctly captured and specified in the model.

When we focus on the plausible values, we also get similar results with the male group mean and standard deviation as M = 0.01, and SD = 0.56, and the females as M = 0.53 and SD = 1.04.

1.1.2 Results for conditioning variables when NOT specifying categoric group

Using the same response data above [resp], we can run the model again, but this time we do not specify the two groups of interest.

mod1 <- tam.mml(resp, group=gender)

Results are as follows:

————————————————————
EAP Reliability
[1] 0.84
————————————————————
Covariances and Variances
[,1] [1,] 1.747
————————————————————
Correlations and Standard Deviations (in the diagonal)
[,1] [1,] 1.322
————————————————————
Regression Coefficients
[,1] [1,] 0
*————————————————————

Obviously, the summary produces no no group-related output as the two gender groups are not specified in the model. To note, though, the overall SD is 1.322 (between the male and female SD of 1.5 and 1.0, respectively).
In terms of results for the plausible values, we get the following. For males, the results are M = -0.22 and SD = 1.47. For females, M = 0.23 and SD = 1.15. These estimates do not correctly capture the prior known ability distributions of these two groups.

1.2 Continuous conditioning variables

When the conditioning variables are categoric (e.g., male & female), both the mean and standard deviations of the respective subgroups can be accounted for and modelled. However, it is possible to model continuous regressors like student age, for example.

We can also model binary variables (such as gender) as continuous regressors. We may do this as follows:

####################### Generate response data with male and female distributions #######################
rm(list = ls())                                               # clears global environment
library(TAM)                                                  # loads IRT package
set.seed(6778)                                                # sets seed so all this can be replicated
N <- 3000                                                     # sets number of students
theta <- c(rnorm(N/2,mean=0,sd=1.5),rnorm(N/2,mean=.5,sd=1))  # defines real male-female ability distributions
I <- 20                                                       # defines number of items
delta <- seq( -2 , 2 , len=I )                                # defines item difficulty vector
p1 <- plogis( outer( theta , delta , "-" ) )                  # creates probability (P) matrix
resp <- 1 * ( p1 > matrix( runif( N*I ) , nrow=N , ncol=I ) ) # generates actual item response data
colnames(resp) <- paste("I" , 1:I, sep="")                    # adds column names
gender <- rep(1:2,each=N/2)                                   # generates categorical variables

########### Account for the different male-female ability distributions (means  not variance) #############
mod1R <- tam.mml(resp, Y=gender)  # Y is the gender variable (could be a matrix of many variables)
summary(mod1R)
PV <- tam.pv(mod1R)
pv1 <- PV$pv$PV1.Dim1
G1R <- pv1[gender==1]
G2R <- pv1[gender==2]
mean(G1R) # male
mean(G2R) # female
sd(G1R)   # male
sd(G2R)   # female

The results are as follows:

————————————————————
EAP Reliability
[1] 0.842
————————————————————
Covariances and Variances
[,1] [1,] 1.67
————————————————————
Correlations and Standard Deviations (in the diagonal)
[,1] [1,] 1.292
————————————————————
Regression Coefficients
[,1] [1,] 0.0000
[2,] 0.5423
*————————————————————
Standardized Coefficients
parm dim est StdYX StdX StdY
1 Intercept 1 0.0000 NA NA NA
2 Y1 1 0.5423 0.2054 0.2712 0.4107

** Explained Variance R^2
[1] 0.0422
** SD Theta
[1] 1.3205
** SD Predictors
Intercept Y1
0.0000 0.5001

In this instance, only one variance coefficient is returned. In this case the standard deviation was 1.292, almost half way between the two genders’ true standard deviation. Although, the regression coefficient for females (group 2) was 0.5423, in accordance with the true difference. This means that one unit shift in gender (from 0 to 1) is associated with a 0.5423 unit shift in ability thetas.

In this instance, the plausible values for males were well aligned at M = 0.55 and SD = 1.08, and for the females at M = 1.46 and SD = 1.14.

Summary

To summarise, we use the following models for the following purposes:

  1. TAM::tam.mml(resp, group=gender):
    To account for different mean and SDs of subgroups when population parameters are of research interest (categoric regressors).

  2. TAM::tam.mml(resp, Y=gender):
    To account for the linear effect of covariates such as SES and age. Gender is modelled here as a binary covariate.

  3. formulaA <- ~ item+mode
    TAM::mod2 <- TAM::tam.mml.mfr( resp= resp, facets= as.data.frame(mode), formulaA = formulaA ):
    To assess the influence of the effect of factors related to items such as test delivery mode and marker identity. In this instance the item and its specific item facet is modelled as an item and a difficulty estimate is provided for each respective item-facet.

  4. formulaA <- ~ item+mode+mode*item
    TAM::mod3 <- TAM::tam.mml.mfr( resp= resp , facets= as.data.frame(mode) , formulaA = formulaA ):
    To assess the interaction effect between items and facets. It may be that a particular questino delivered in a particular way may be causing some bias. In this instance, z-scores are generated and associated p values a ascertained. Bonferroni correction probably should be applied.

END