This document allows a social work researcher to replicate the IRT analysis presented in our recent article “Item Response Theory and Confirmatory Factor Analysis: Complementary Approaches for Scale Development”. Running the “Isolation-R-Markdown.Rmd” file in RStudio will produce the results we presented in that article. You are encouraged to refer to the article for a full discussion of each of these results.

We load the R mirt library and read the data file as follows:

# Use the library command to load libraries
library(mirt)
# Read a csv data file and define our isolation scale items
data <- read.csv("isolation.csv", header = TRUE, sep=",")
isolation <-(data[,1:5])

Using mirt, We define a graded response model and fit a full information maximum likelihood fitting function. We also request fit indexes from the M2 family of GRM limited information indexes.

# We define a graded response model 
mod1 <- (mirt(isolation, 1, verbose = FALSE, itemtype = 'graded', SE = TRUE))
options(digits = 2)
# M2 is a family of limited information model fit indexes useful for GRMs
M2(mod1, type = "C2") 
##       M2 df    p RMSEA RMSEA_5 RMSEA_95 SRMSR  TLI CFI
## stats  8  5 0.16 0.031       0    0.069  0.03 0.99   1

Using various recommended interpretation frameworks for fit indexes, we conclude we have a plausible GRM.

We check the fit of each item:

# Item S_X2 fit indexes 
itemfit(mod1, fit_stats = "S_X2")
##     item S_X2 df.S_X2 RMSEA.S_X2 p.S_X2
## 1 Item.1   31      10      0.058  0.001
## 2 Item.2   16       8      0.041  0.038
## 3 Item.3   11       9      0.019  0.267
## 4 Item.4   12       9      0.023  0.217
## 5 Item.5    7       8      0.000  0.540

Based on RMSEA values, items seem to have a good fit.

Item-trait relationship details:

Item relationships with the latent trait can be expressed using two different parameterizations — IRT based parameters and factor analysis parameters. They are mathematically related.

IRT item parameters are shown below —– a-parameters are referred to as slope or discrimination parameters and are measures of the strength of the relationship between a item and the latent variable; b-parameters are referred to as location parameters and are interpreted as the point on theta where a respondent has a .5 probability of responding to a category or higher.

# IRT parameters
coef(mod1, IRTpars = TRUE, simplify = TRUE)
## $items
##          a     b1  b2
## Item.1 1.6  0.728 2.3
## Item.2 2.2  0.115 1.6
## Item.3 1.4 -0.995 1.7
## Item.4 1.9 -0.169 1.4
## Item.5 2.2  0.025 1.6
## 
## $means
## F1 
##  0 
## 
## $cov
##    F1
## F1  1

Factor analysis parameters are shown next — F1 parameters are factor loadings and can be interpreted as a strength of the relationship between an item and the latent variable; h2 parameters are squared values of factor loadings and represent variance accounted for in an item by the latent trait.

# FA parameters
summary(mod1)
##           F1    h2
## Item.1 0.682 0.466
## Item.2 0.796 0.634
## Item.3 0.631 0.399
## Item.4 0.749 0.561
## Item.5 0.798 0.636
## 
## SS loadings:  2.7 
## Proportion Var:  0.54 
## 
## Factor correlations: 
## 
##    F1
## F1  1

A strength of IRT is the ability to examine item and scale characteristics using various plots. Graphs are particularly helpful in determining how items and the scale operate along the isolation continuum of scores (theta).

Category characteristic curves:

bwtheme <- standard.theme(color=FALSE)
plot(mod1, type='trace', which.item = c(1,2,3,4,5), facet_items=T, as.table = TRUE,
           auto.key=list(points=F, lines=T, columns=3, space = 'top', cex = .8),
           par.settings=bwtheme,
           theta_lim = c(-3, 3), main="")

The trace lines show how the probability of responding to a category for an item rises and then falls as theta increases.

Item information curves:

plot(mod1, type='infotrace', which.item = c(1,2,3,4,5), facet_items=F, as.table = TRUE,
           auto.key=list(points=F, lines=T, columns=1, space = 'right', cex = .8), 
           par.settings=bwtheme,
           theta_lim = c(-3, 3), main="")

Information is a statistical concept that measures the strength of the relationship between an item and the latent trait. Higher a-parameters indicate higher information.

Scale information and standard errors:

bwtheme <- standard.theme(color=FALSE)
plotinse <- plot(mod1, type = 'infoSE', theta_lim = c(-3,3),main = "", par.settings=bwtheme)
plotinse$legend$right$args$label <- expression(paste("Standard Error ",(theta)))
plotinse$legend$left$args$label <- expression(paste("Information ",(theta)))
plotinse

Item information can be summed to form a test (scale) information function where higher values indicate a stronger relationship between the scale and the latent trait. Conditional standard errors are measures of the precision of score estimates. Information and standard errors are mathematically related with higher information values giving rise to smaller standard errors.

Conditional reliability:

plot(mod1, type = 'rxx', theta_lim = c(-3,3), main = "", par.settings=bwtheme)

Conditional reliability, information, and standard errors also are mathematically related with higher information and lower standard errors giving rise to higher conditional reliability.

Estimated thetas and expected test (scale) scores:

plottcc <- plot(mod1, type = 'score', theta_lim = c(-3,3), main = "", par.settings=bwtheme)
plottcc$xlab <- expression(paste("Model-based Estimated Score  ",(theta),))
plottcc$ylab <- expression(paste("Expected True Score"))
plottcc

This plot shows the relationship between estimated theta scores and estimated true scores. These estimated true scores are model-based meaning that they are generated using item parameters and are expressed in the original scale metric.