dMACS provides the correct magnitudes for bias due to non-invariance; dMACS_Signed provides the correct directions for the effects but its magnitudes are only correct in certain cases. Here, I provide a function that provides dMACS of the correct magnitude and direction, dubbed dMACS_True. Below, instead of writing “MACS” a bunch of times in the formula below, I just write \(d_{T}\) for dMACS_True, d for dMACS, and \(d_{S}\) for dMACS_Signed.
\[d_T=\begin{cases} & d \text{ if } d_S \geq 0 \\ & d \times -1 \text{ if } d_S < 0 \end{cases}\]
dMACS_True <- function(fit.cfa, group1, group2) {
nitems <- lavaan::lavInspect(fit.cfa, what = "rsquare") %>%
.[[1]] %>%
names(.) %>%
length(.)
cfa_minmax <- function(fit.cfa) {
dt <- lavaan::inspect(fit.cfa, what = "data")
latentMin <- min(dt[[1]]) - 1
latentMax <- max(dt[[1]]) + 1
out <- cbind(as.numeric(latentMin), as.numeric(latentMax))
return(out)}
reference_load <- lavaan::inspect(fit.cfa, what = "est") %>%
.[[1]] %>%
.$lambda
focal_load <- lavaan::inspect(fit.cfa, what = "est") %>%
.[[2]] %>%
.$lambda
reference_intrcp <- lavaan::inspect(fit.cfa, what = "est") %>%
.[[1]] %>%
.$nu
focal_intrcp <- lavaan::inspect(fit.cfa, what = "est") %>%
.[[2]] %>%
.$nu
pool.sd <- function(fit.cfa) {
cfa.se <- lavaan::lavInspect(fit.cfa, what = "se")
cfa.n <- lavaan::lavInspect(fit.cfa, what = "nobs")
l <- list()
test <- lavaan::lavInspect(fit.cfa, what = "rsquare") %>%
.[[1]] %>%
names(.) %>%
length(.)
for (i in 1:test) {
grp1 <- cfa.se[[group1]]$nu[i] * sqrt(cfa.n[1])
grp2 <- cfa.se[[group2]]$nu[i] * sqrt(cfa.n[2])
numerator <- ((cfa.n[1] - 1) * grp1 + (cfa.n[2] - 1) * grp2)
denominator <- (cfa.n[1] - 1) + (cfa.n[2] - 1)
pooled.sd <- numerator / denominator
l[[paste("item", i)]] <- pooled.sd}
result <- matrix(unlist(l), nrow = test, byrow = TRUE)
return(result)}
pld_sd <- pool.sd(fit.cfa)
fcl_lt_vrnc <- lavaan::inspect(fit.cfa, what = "est") %>%
.[[2]] %>%
.$psi
l <- list()
rowlab <- c()
for (i in c(1:nitems)) {
focal.fn <- function(x) {
mpr <- focal_intrcp[i] + focal_load[i] * x
return(mpr)}
reference.fn <- function(x) {
mpr <- reference_intrcp[i] + reference_load[i] * x
return(mpr)}
diff.fn <- function(x, i = i) {
d <- ((reference.fn(x) - focal.fn(x))^2) * dnorm(x, mean = 0, sd = sqrt(fcl_lt_vrnc))
return(d)}
dMACS <- round((1 / pld_sd[i]) * sqrt(integrate(diff.fn,
lower = cfa_minmax(fit.cfa)[, 1],
upper = cfa_minmax(fit.cfa)[, 2]
)$value), 3)
l[[length(l) + 1]] <- dMACS
rowlab[[length(rowlab) + 1]] <- paste("Item", i)}
m <- matrix(unlist(l), nrow = nitems, dimnames = list(rowlab, "dMACS")) #Magnitude
l <- list()
rowlab <- c()
for (i in c(1:nitems)) {
focal.fn <- function(x) {
mpr <- focal_intrcp[i] + focal_load[i] * x
return(mpr)}
reference.fn <- function(x) {
mpr <- reference_intrcp[i] + reference_load[i] * x
return(mpr)}
diff.fn <- function(x, i = i) {
d <- ((reference.fn(x) - focal.fn(x))) * dnorm(x, mean = 0, sd = sqrt(fcl_lt_vrnc))
return(d)}
dMACS <- round((1 / pld_sd[i]) * integrate(diff.fn,
lower = cfa_minmax(fit.cfa)[, 1],
upper = cfa_minmax(fit.cfa)[, 2])$value, 3)
l[[length(l) + 1]] <- dMACS
rowlab[[length(rowlab) + 1]] <- paste("Item", i)}
d <- matrix(unlist(l), nrow = nitems, dimnames = list(rowlab, "dMACS_True")) #Direction
D <- ifelse(d < 0, -1, 1) #Recode d to a matrix of signs
H = D * m #Multiply magnitudes by signs
return(H)}I previously (https://rpubs.com/JLLJ/dMACSSigned) looked at neuroticism as an example. Here, I look at that again.
NeurotModel <- '
LatentNeuroticism =~ Vulnerability + Immoderation + SelfConsciousness + Depression + Anger + Anxiety'
NeurUSA <- cfa(NeurotModel, data = USAIP, group = "SEXO")
dUSA <- dMACS(NeurUSA, group1 = "MALE", group2 = "FEMALE")/6; dUSA; sum(dUSA)## dMACS
## Item 1 0.029333333
## Item 2 0.023833333
## Item 3 0.011166667
## Item 4 0.028000000
## Item 5 0.024833333
## Item 6 0.004833333## [1] 0.122dUSAS <- dMACS_Signed(NeurUSA, group1 = "MALE", group2 = "FEMALE")/6; dUSAS; sum(dUSAS)## dMACS_Signed
## Item 1 0.02933333
## Item 2 -0.01666667
## Item 3 -0.01050000
## Item 4 0.02650000
## Item 5 -0.01933333
## Item 6 0.00400000## [1] 0.01333333dUSAT <- dMACS_True(NeurUSA, group1 = "MALE", group2 = "FEMALE")/6; dUSAT; sum(dUSAT)## dMACS_True
## Item 1 0.029333333
## Item 2 -0.023833333
## Item 3 -0.011166667
## Item 4 0.028000000
## Item 5 -0.024833333
## Item 6 0.004833333## [1] 0.002333333As I said before, dMACS_Signed makes it apparent why we need to go beyond dMACS: because cancellation occurs. The total amount of bias indicated by dMACS was 0.122 d, but it was 0.013 d with the signed version of dMACS. This, however, is wrong, because dMACS_Signed is unsquared and thus not really the magnitude of the effects. The only time they correspond is when loadings are unbiased.
Per Nye et al. (2019), “For example, when the group-specific [IRF] lines cross, the squared effect size can be used to quantify the magnitude of the effect, and the unsquared effect size can be used to determine the direction of the effect (i.e., which group the measure is biased against) and the extent to which it will cancel out. In contrast, in situations where the group-specific lines reflecting predicted item scores do not cross (i.e., when there are intercept differences but not loading differences between groups), the values of these two indices will be nearly identical” (p. 13). This is echoed by Gunn, Grimm & Edwards (2020), who wrote, for their own effect size, “If there is no cancellation (e.g., the IRFs do not cross where the effect size is evaluated), then \(SDI_2\) will be equal to \(UDI_2\); however, dMACS will not be equal to dMACS_Signed unless the loading is invariant” (p. 3).
dMACS_True is a simple combination of dMACS and dMACS_Signed that yields both the correct magnitudes and the correct directions of bias. The sum of this, provided above, differs from both dMACS and dMACS_Signed. In this case, it is smaller than dMACS because there is cancellation, but it is also smaller than dMACS_Signed because the degree of cancellation was greater than indicated by unsquared effects. For aggregate bias, dMACS_Signed^2 can be close, but specific values are off, so it’s better to just compute dMACS_True.
Nye, C. D., Bradburn, J., Olenick, J., Bialko, C., & Drasgow, F. (2019). How Big Are My Effects? Examining the Magnitude of Effect Sizes in Studies of Measurement Equivalence. Organizational Research Methods, 22(3), 678–709. https://doi.org/10.1177/1094428118761122
Gunn, H. J., Grimm, K. J., & Edwards, M. C. (2020). Evaluation of Six Effect Size Measures of Measurement Non-Invariance for Continuous Outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(4), 503–514. https://doi.org/10.1080/10705511.2019.1689507