Homework 7.5 - Measurement Invariance
Allie Choate
3/11/2019
Measurement Invariance
Packages Needed For This Assignment:
library(lavaan)
library(semTools)
library(ltm)
library(mirt)
library(psych)
library(ggplot2)Part 1: Measurement Invariance
Download the tab-delimited dataset “HW7.txt” file. This data includes raw responses for the eight-item measure of “Work and Family Conflict” across Male and Female workers. Variables are listed as follows: item1-item8, and group (0 = Male; 1 = Female).
- Suppose item1-item4 loads on “Work to Family Conflict” factor & item5-item8 loads on “Family to Work Conflict” factor.
All Work and Family Conflict items are polytomous, five Likert scale choices.
- Conduct measurement invariance testing treating the data as categorical to investigate measurement invariance of the measure between Male and Female workers following Millsap and Yun-Tein’s (2004) procedure.
- Report and discuss CFA model fit of a two factor with four indicators each group. You should provide a summary table for the measurement invariance tests.
- At which level does the measurement invariance hold? Provide evidence. If measurement invariance does not hold at a certain level, show the evidence.
Import the data:
mi.data<- read.csv("HW7.csv", header=T)
str(mi.data)## 'data.frame': 1000 obs. of 9 variables:
## $ Gender: int 1 1 1 1 1 1 1 1 1 1 ...
## $ x1 : int 2 3 1 3 2 2 1 3 1 2 ...
## $ x2 : int 3 3 3 3 2 0 3 2 1 3 ...
## $ x3 : int 2 2 1 2 2 1 1 2 4 1 ...
## $ x4 : int 2 2 3 3 3 1 1 3 3 2 ...
## $ x5 : int 1 2 4 4 2 1 4 2 2 3 ...
## $ x6 : int 1 2 3 3 3 2 2 2 2 2 ...
## $ x7 : int 1 0 2 3 2 1 2 2 2 2 ...
## $ x8 : int 0 2 2 4 4 0 1 2 1 3 ...As we can see above, R read in all values as integers. To run a categorical MI test, we need to tell R that we want to treat these values as ordered/categorical. To do so, a number of ways can be employed. I find that creating a quick loop to set all variables in the dataset to ‘ordered’ is the easiest.
Convert data to all categorical & specify grouping variable
for(i in 1:ncol(mi.data)){
mi.data[,i] <- as.ordered(mi.data[,i])
}
str(mi.data) #check data structure to make sure it worked correctly## 'data.frame': 1000 obs. of 9 variables:
## $ Gender: Ord.factor w/ 2 levels "1"<"2": 1 1 1 1 1 1 1 1 1 1 ...
## $ x1 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 3 4 2 4 3 3 2 4 2 3 ...
## $ x2 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 4 4 4 4 3 1 4 3 2 4 ...
## $ x3 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 3 3 2 3 3 2 2 3 5 2 ...
## $ x4 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 3 3 4 4 4 2 2 4 4 3 ...
## $ x5 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 2 3 5 5 3 2 5 3 3 4 ...
## $ x6 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 2 3 4 4 4 3 3 3 3 3 ...
## $ x7 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 2 1 3 4 3 2 3 3 3 3 ...
## $ x8 : Ord.factor w/ 5 levels "0"<"1"<"2"<"3"<..: 1 3 3 5 5 1 2 3 2 4 ...To make it easier for MI testing, let’s also re-label gender with actual labels instead of numbers:
mi.data$Gender<- factor(mi.data$Gender, levels=c(1,2), labels=c("male", "female"))
Now we should be good to go for running our two-factor model and MI tests.
Running the Two-Factor Model:
Specify the model:
factor.model<- ' WFC =~ x1 + x2 + x3 + x4
FWC =~ x5 + x6 + x7 + x8'Physically run the model and evaluate fit:
- Remember to specify how you want your model to be identified if you want something other than the default in lavaan!
- Also, because we are not treating the variables as continous, it is benefical to change the estimator method. Lavaan defaults to maximum-likelihood, but something like ‘WLSMV’ is more suitable in this case for ordered variables.
factor.fit<- cfa(factor.model, data=mi.data, std.lv=TRUE, group="Gender", estimator="WLSMV")
summary(factor.fit, standardized=TRUE, fit.measures=T)## lavaan 0.6-3 ended normally after 13 iterations
##
## Optimization method NLMINB
## Number of free parameters 82
##
## Number of observations per group
## male 500
## female 500
##
## Estimator DWLS Robust
## Model Fit Test Statistic 32.098 52.132
## Degrees of freedom 38 38
## P-value (Chi-square) 0.738 0.063
## Scaling correction factor 0.678
## Shift parameter for each group:
## male 2.403
## female 2.403
## for simple second-order correction (Mplus variant)
##
## Chi-square for each group:
##
## male 16.822 27.205
## female 15.277 24.927
##
## Model test baseline model:
##
## Minimum Function Test Statistic 5013.628 3302.993
## Degrees of freedom 56 56
## P-value 0.000 0.000
##
## User model versus baseline model:
##
## Comparative Fit Index (CFI) 1.000 0.996
## Tucker-Lewis Index (TLI) 1.002 0.994
##
## Robust Comparative Fit Index (CFI) NA
## Robust Tucker-Lewis Index (TLI) NA
##
## Root Mean Square Error of Approximation:
##
## RMSEA 0.000 0.027
## 90 Percent Confidence Interval 0.000 0.024 0.000 0.044
## P-value RMSEA <= 0.05 1.000 0.989
##
## Robust RMSEA NA
## 90 Percent Confidence Interval 0.000 NA
##
## Standardized Root Mean Square Residual:
##
## SRMR 0.030 0.030
##
## Parameter Estimates:
##
## Information Expected
## Information saturated (h1) model Unstructured
## Standard Errors Robust.sem
##
##
## Group 1 [male]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## WFC =~
## x1 0.716 0.036 19.694 0.000 0.716 0.716
## x2 0.684 0.038 17.821 0.000 0.684 0.684
## x3 0.662 0.037 17.829 0.000 0.662 0.662
## x4 0.713 0.036 19.859 0.000 0.713 0.713
## FWC =~
## x5 0.677 0.036 18.705 0.000 0.677 0.677
## x6 0.649 0.039 16.706 0.000 0.649 0.649
## x7 0.751 0.033 22.453 0.000 0.751 0.751
## x8 0.655 0.040 16.283 0.000 0.655 0.655
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## WFC ~~
## FWC 0.475 0.048 9.973 0.000 0.475 0.475
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .x1 0.000 0.000 0.000
## .x2 0.000 0.000 0.000
## .x3 0.000 0.000 0.000
## .x4 0.000 0.000 0.000
## .x5 0.000 0.000 0.000
## .x6 0.000 0.000 0.000
## .x7 0.000 0.000 0.000
## .x8 0.000 0.000 0.000
## WFC 0.000 0.000 0.000
## FWC 0.000 0.000 0.000
##
## Thresholds:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## x1|t1 -1.506 0.087 -17.388 0.000 -1.506 -1.506
## x1|t2 -0.429 0.058 -7.392 0.000 -0.429 -0.429
## x1|t3 0.595 0.060 9.935 0.000 0.595 0.595
## x1|t4 1.626 0.093 17.405 0.000 1.626 1.626
## x2|t1 -1.538 0.088 -17.414 0.000 -1.538 -1.538
## x2|t2 -0.445 0.058 -7.657 0.000 -0.445 -0.445
## x2|t3 0.451 0.058 7.745 0.000 0.451 0.451
## x2|t4 1.589 0.091 17.423 0.000 1.589 1.589
## x3|t1 -1.506 0.087 -17.388 0.000 -1.506 -1.506
## x3|t2 -0.559 0.059 -9.413 0.000 -0.559 -0.559
## x3|t3 0.524 0.059 8.888 0.000 0.524 0.524
## x3|t4 1.685 0.097 17.334 0.000 1.685 1.685
## x4|t1 -1.353 0.079 -17.040 0.000 -1.353 -1.353
## x4|t2 -0.507 0.059 -8.625 0.000 -0.507 -0.507
## x4|t3 0.662 0.061 10.886 0.000 0.662 0.662
## x4|t4 1.728 0.100 17.252 0.000 1.728 1.728
## x5|t1 -1.538 0.088 -17.414 0.000 -1.538 -1.538
## x5|t2 -0.423 0.058 -7.304 0.000 -0.423 -0.423
## x5|t3 0.637 0.060 10.541 0.000 0.637 0.637
## x5|t4 1.589 0.091 17.423 0.000 1.589 1.589
## x6|t1 -1.433 0.083 -17.267 0.000 -1.433 -1.433
## x6|t2 -0.530 0.059 -8.975 0.000 -0.530 -0.530
## x6|t3 0.553 0.059 9.325 0.000 0.553 0.553
## x6|t4 1.751 0.102 17.198 0.000 1.751 1.751
## x7|t1 -1.538 0.088 -17.414 0.000 -1.538 -1.538
## x7|t2 -0.524 0.059 -8.888 0.000 -0.524 -0.524
## x7|t3 0.650 0.061 10.714 0.000 0.650 0.650
## x7|t4 1.665 0.096 17.364 0.000 1.665 1.665
## x8|t1 -1.476 0.085 -17.348 0.000 -1.476 -1.476
## x8|t2 -0.418 0.058 -7.215 0.000 -0.418 -0.418
## x8|t3 0.473 0.058 8.097 0.000 0.473 0.473
## x8|t4 1.447 0.084 17.297 0.000 1.447 1.447
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .x1 0.487 0.487 0.487
## .x2 0.532 0.532 0.532
## .x3 0.562 0.562 0.562
## .x4 0.492 0.492 0.492
## .x5 0.541 0.541 0.541
## .x6 0.579 0.579 0.579
## .x7 0.436 0.436 0.436
## .x8 0.571 0.571 0.571
## WFC 1.000 1.000 1.000
## FWC 1.000 1.000 1.000
##
## Scales y*:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## x1 1.000 1.000 1.000
## x2 1.000 1.000 1.000
## x3 1.000 1.000 1.000
## x4 1.000 1.000 1.000
## x5 1.000 1.000 1.000
## x6 1.000 1.000 1.000
## x7 1.000 1.000 1.000
## x8 1.000 1.000 1.000
##
##
## Group 2 [female]:
##
## Latent Variables:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## WFC =~
## x1 0.756 0.029 26.232 0.000 0.756 0.756
## x2 0.671 0.034 19.486 0.000 0.671 0.671
## x3 0.701 0.034 20.865 0.000 0.701 0.701
## x4 0.694 0.033 20.723 0.000 0.694 0.694
## FWC =~
## x5 0.668 0.038 17.626 0.000 0.668 0.668
## x6 0.786 0.029 27.172 0.000 0.786 0.786
## x7 0.661 0.036 18.569 0.000 0.661 0.661
## x8 0.749 0.031 23.838 0.000 0.749 0.749
##
## Covariances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## WFC ~~
## FWC 0.542 0.044 12.450 0.000 0.542 0.542
##
## Intercepts:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .x1 0.000 0.000 0.000
## .x2 0.000 0.000 0.000
## .x3 0.000 0.000 0.000
## .x4 0.000 0.000 0.000
## .x5 0.000 0.000 0.000
## .x6 0.000 0.000 0.000
## .x7 0.000 0.000 0.000
## .x8 0.000 0.000 0.000
## WFC 0.000 0.000 0.000
## FWC 0.000 0.000 0.000
##
## Thresholds:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## x1|t1 -1.071 0.070 -15.409 0.000 -1.071 -1.071
## x1|t2 -0.015 0.056 -0.268 0.789 -0.015 -0.015
## x1|t3 0.893 0.065 13.726 0.000 0.893 0.893
## x1|t4 1.943 0.118 16.474 0.000 1.943 1.943
## x2|t1 -1.089 0.070 -15.551 0.000 -1.089 -1.089
## x2|t2 -0.228 0.057 -4.018 0.000 -0.228 -0.228
## x2|t3 0.806 0.063 12.746 0.000 0.806 0.806
## x2|t4 1.881 0.112 16.758 0.000 1.881 1.881
## x3|t1 -1.227 0.074 -16.466 0.000 -1.227 -1.227
## x3|t2 -0.116 0.056 -2.055 0.040 -0.116 -0.116
## x3|t3 1.089 0.070 15.551 0.000 1.089 1.089
## x3|t4 2.054 0.129 15.866 0.000 2.054 2.054
## x4|t1 -1.165 0.072 -16.093 0.000 -1.165 -1.165
## x4|t2 -0.141 0.056 -2.501 0.012 -0.141 -0.141
## x4|t3 0.900 0.065 13.806 0.000 0.900 0.900
## x4|t4 1.852 0.110 16.873 0.000 1.852 1.852
## x5|t1 -1.645 0.095 -17.388 0.000 -1.645 -1.645
## x5|t2 -0.656 0.061 -10.800 0.000 -0.656 -0.656
## x5|t3 0.353 0.057 6.152 0.000 0.353 0.353
## x5|t4 1.293 0.077 16.800 0.000 1.293 1.293
## x6|t1 -1.685 0.097 -17.334 0.000 -1.685 -1.685
## x6|t2 -0.519 0.059 -8.800 0.000 -0.519 -0.519
## x6|t3 0.440 0.058 7.568 0.000 0.440 0.440
## x6|t4 1.353 0.079 17.040 0.000 1.353 1.353
## x7|t1 -1.476 0.085 -17.348 0.000 -1.476 -1.476
## x7|t2 -0.706 0.062 -11.484 0.000 -0.706 -0.706
## x7|t3 0.385 0.058 6.684 0.000 0.385 0.385
## x7|t4 1.353 0.079 17.040 0.000 1.353 1.353
## x8|t1 -1.572 0.090 -17.424 0.000 -1.572 -1.572
## x8|t2 -0.643 0.061 -10.627 0.000 -0.643 -0.643
## x8|t3 0.445 0.058 7.657 0.000 0.445 0.445
## x8|t4 1.293 0.077 16.800 0.000 1.293 1.293
##
## Variances:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## .x1 0.428 0.428 0.428
## .x2 0.550 0.550 0.550
## .x3 0.509 0.509 0.509
## .x4 0.518 0.518 0.518
## .x5 0.554 0.554 0.554
## .x6 0.383 0.383 0.383
## .x7 0.563 0.563 0.563
## .x8 0.439 0.439 0.439
## WFC 1.000 1.000 1.000
## FWC 1.000 1.000 1.000
##
## Scales y*:
## Estimate Std.Err z-value P(>|z|) Std.lv Std.all
## x1 1.000 1.000 1.000
## x2 1.000 1.000 1.000
## x3 1.000 1.000 1.000
## x4 1.000 1.000 1.000
## x5 1.000 1.000 1.000
## x6 1.000 1.000 1.000
## x7 1.000 1.000 1.000
## x8 1.000 1.000 1.000Specify the configural baseline model for measurement invariance testing:
Realistically, we already have this model specified from when we ran the correlated two-factor model, but just in case the code for the model is provided again below.
mi.config<- ' WFC =~ x1 + x2 + x3 + x4
FWC =~ x5 + x6 + x7 + x8'fit_configural <- cfa(mi.config, mi.data, estimator="WLSMV", group = "Gender")
fitMeasures(fit_configural, c("rmsea","cfi","tli","rni","rfi","ifi","srmr","gfi"))## rmsea cfi tli rni rfi ifi srmr gfi
## 0.000 1.000 1.002 1.001 0.991 1.001 0.030 0.998library("semPlot")
layout(t(1:2))
semPaths(fit_configural, "mod", "est", ask=FALSE, layout = "tree",
levels = c(1,2,4,5), edge.color = "black", reorder = FALSE,
manifests = paste0("x",1:8), mar = c(2,2,4,2))
Configural measurement invariance holds.
Running Tests of MI treating variables as categorical:
For categorical measurement invariance testing, we can simply use a modified version of the measurement invariance command from the previous homework in the package ‘SemTools.’
measurementInvarianceCat(model = mi.config, data=mi.data, group="Gender")## Warning: The measurementInvarianceCat function is deprecated, and it will
## cease to be included in future versions of semTools. See help('semTools-
## deprecated) for details.##
## Measurement invariance models:
##
## Model 1 : fit.configural
## Model 2 : fit.loadings
## Model 3 : fit.thresholds
## Model 4 : fit.means
##
## Scaled Chi Square Difference Test (method = "satorra.2000")
##
## Df AIC BIC Chisq Chisq diff Df diff Pr(>Chisq)
## fit.configural 38 32.098
## fit.loadings 44 37.303 5.383 6 0.495695
## fit.thresholds 66 69.721 43.005 22 0.004712 **
## fit.means 68 312.630 53.070 2 2.992e-12 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
##
## Fit measures:
##
## cfi.scaled rmsea.scaled cfi.scaled.delta rmsea.scaled.delta
## fit.configural 0.996 0.027 NA NA
## fit.loadings 0.996 0.024 0.000 0.003
## fit.thresholds 0.988 0.034 0.008 0.010
## fit.means 0.907 0.094 0.081 0.060Results of Measurement Invariance
Based on the Chi-Squared Likelihood Ratio Test, measurement invariance held a the metric level (i.e., equivalence of factor loadings between groups being compared), but did not hold at any subsequent levels. However, by evaluating fit indices, results are bit more vague. For change in CFI at the scalar level, it looks like the threshold of a change of ‘.01’ or less is observed for both the CFI/RMSEA at the scalar level. Looking back at the results from the Chi-Squared LR test, we also see that this was flagged at the .01 level for scalar invariance. In this case, it could be resonable to conclude that scalar invariance holds (i.e., equivalence of factor loadings and intercepts across groups), especially as chi-square LR tests can be overly sensitive with larger sample sizes.
- Strict invariance, however, does not hold, and appears that residuals are not equal across groups based on both the LR test and evaluation of the change in alternative fit indices.