Structural Equation Model

author: angelayuan
date: Sep 12, 2015

SEM Theory and Development

What is theory?

  • At one level, a theory can be thought of as an explanation of why variables are correlated (or not correlated)
  • A necessary but insufficient condition for the validity of a theory would be that the relationships among variables are consistent with the propositions of the theory

SEM is based on the observations that

  • Every theory implies a set of correlations: Expected variance-covariance matrix \(\sum(\theta)\)

  • If the theory is valid, then the theory should be able to explain or reproduce the patterns of correlations found in the empirical data: Observed variance-covariance matrix \(\sum\)

H0: Expected variance-covariance matrix = Observed variance-covariance matrix

  • Reject H0: one could reasonably conclude that the theory was incorrect
  • Fail to reject H0: Finding the expected pattern of correlations would not imply that the theory is right, only that it is plausible. There might be other theories that would result in the same pattern of correlations
  • E.g. Original theory: Rejection -> self-esteem -> materialism; Alternative theory: materialism -> self-esteem -> rejection

The Key SEM techniques

1. Path Analysis (PA)

Example

  • Topic: Exercise, Hardiness, Fitness, Stress, and Illness within a sample of 373 students
  • Hypotheses
      1. Exercise and Hardiness -> Fitness
      1. Exercise, Hardiness, and Fitness -> Stress
      1. Exercise, Hardiness, Fitness, and Stress -> Illness
  • Technique
    • Canonical Regression: F and S cannot be represented in multiple regression as both a predictor and a criterion in the same analysis
    • Path Analysis: a single measure of each theoretical variable and the researcher has prior hypotheses about causal relations among these variables

Reticular Action Modeling (RAM)

alt text

Variables

  • Observed variables vs. unobserved variables; Manifest variables (indicator) vs. latent variables
    • Circles or ellipses represent unobserved and latent variables; rectangles represent observed variables
  • Exogenous variables vs. endogenous variables
    • Every edogenous variable has a disturbance/error term
    • Error in Fitness represents variance in fitness unexplained by its direct causes (i.e. Exercise and Hardiness)
    • Disturbance is unmeasured/latent/unobserved exogenous variable

More Explanations

  • The causes of exogenous variables are not represented in path models, they are typically considered free to vary and covary. So every exogenous variable has a variance associated with it

  • The unidirection arrow (->) is direct effect. Statistical estimates of direct effects are path coefficients (like regression coefficients in multiple regression), they control for correlations among multiple causes

  • Indirect effect/ mediator effect: involving one or more intervening variables. A total of 9 indirect effectsare in the above RAM figure

  • Hypothesized spurious association. Fitness and Stress have two common causes, Exercise and Hardiness. The path from Fitness -> Stress controls for these common causes. If the path is zero, the observed correlation between Fitness and Stress is spurious

  • Disturbances represent all causes of an endogenous variable that are omitted from the structural model. E.g., disturbance for Stress represents the combined effects of all the omitted causes of Stress, such as gene, diet, work etc.

  • Disturbance is unobserved exogenous variables. The constant (1.0) for disturbance -> endogenous variables represent the assignment of a scale to each unobserved exogenous variable

2. Confirmatory Factor Analysis (CFA)

Example

  • Topic: The Visual Similes Test II
    • Affective form: 30 visual stimuli items that depict scenes related to survival. Predation and disaster
    • Cognitive form: 25 visual stimuli items that required abstractions in order to interpret them. American flag
    • 10 different item parcels, 5 for each form
  • CFA measurement model of the VST II

alt text

Explanation

  • The unidirectional arrow (->) from factors to indicators is called factor loading in CFA (path coefficients in path analysis). Factor loading is also regression coefficients

  • Measurement error (disturbance in path analysis) represents unique variance. Measurement errors represent all sources of residual variation in indicator scores that is not explained by the factors. Two sources:
    • random error (score unreliability)
    • all sources of systematic variance not due to the factors (particular measurement method effect)

Hypothetical constructs

  • Hypothetical constructs (e.g. happiness, self-esteem) are not directly observable. They can be only inferred or measured indirectly through observed variables, also called indicators

Single indicator is not good

  • Reliability: indicators are not generally free from random error
  • Validity: not all of the systematic part of an indicator’s variance may reflect the construct (an item may measure something else)

Multiple indicator

  • Scores across a set of measures tend to be more reliable
  • Multiple indicators may each assess a somewhat different facet of the construct, which enhances score validity
  • Path analysis assumes single-indicator measurement. So that it is crucial that a single indicator is reliable and valid

CFA Results

  • Convergent validity: all indicators have relatively high standardized loading on that factor
  • Discriminant validity: estimated correlations between the factors are not excessively high
  • If estimated correlations between the factors > .85, then the 10 indicators can hardly be said to measure two different construct

Exploratory Factor Analysis (EFA) vs. CFA

  • EFA
    • Developed to test theories of intelligence
    • IT is not considered a member of SEM family
    • It doesn’t require a priori hypotheses about how indicators are related to underlying factors or even the number of factors
    • Researcher has little direct influence on the correspondence between indicators and factors
  • CFA
    • Analyzes a priori measurement models
    • Both the number of factors and their correspondence to the indicators are explicitly specified

3. Structural Regression Models (Full Model)

SR model

  • An SR model is the synthesis of a structural model and a measurement model; it is a hybrid model
  • SR/full model is like conducting a PA but among factors (latent variables) instead of observed variables

Example

  • Hypothesis

alt text

  • SR Model

alt text

Summary of Key SEM Techniques

CFA

  • Tests whether the indicators indeed seem to measure factors, and estimates only unanalyzed associations among factors, not direct causal effects

PA

  • Presumed causal effects can be specified and tested in PA, but only analyzes observed variables, not latent variables

SR

  • Has a structural component and a measurement component, combines features of both CFA and PA

All these core kinds of models have just a covariance structure, which includes a structural model or a measurement model (or both)

Iterative basic steps of SEM

1. Model Specification

SEM is inherently a confirmatory technique (a priori specification of a model). Researcher’s hypotheses are expressed in the form of a structural equation model

  • Drawing a diagram of a model
  • Model can be described as a series of equations
  • Equations define the model’s parameters, which correspond to presumed relations

The fundamental hypothesis for SEM is

  • Expected variance-covariance matrix = Observed variance-covariance matrix
  • we need to specify expected variance-covariance matrix
  • During specification, we make claims about causation

The conditions necessary for causal inference in SEM Meeting these conditions for causal inference is more a matter of study design than of statistical technique

  • Association
  • Isolation (the inclusion of all relevant predictors)
  • The establishment of causal direction

More

  • Three assumptions underlie path diagrams
    • All of the proposed causal relations are linear
    • Represent all the causal relations between the variables. Specify the relationships that do exist, and specify those that do not exist
    • Causal closure: All causes of the variables in the model are represented in the model. Any variable thought to cause two or more variables in the model should in itself be part of the model. (failure often leads to inflated results)
  • Factor analysis can be think of as a path diagram
    • Reponse to an individual item is a function of (1) the trait that the item is measuring, and (2) error
  • Converting the path diagram to structural equations
    • A simple path (X -> Y) represents the direct relationship between two variables
    • A compound path (X -> Y -> Z) consists of two or more simple paths
    • The value of a compound path is the product of all the simple paths constituting the compound path
    • The correlation between any two variables is the sum of the simple and compound paths linking the two variables
    • An example
      • Path diagram

      alt text

      • correlation matrix
    ##      A    B    C
## A 1.00          
## B 0.50 1.00     
## C 0.65 0.70 1.00
    Compute the path coefficients: c=0.5; a+cb=0.65; b+ca=0.70 => a=0.40, b=0.50, c=0.50 (These numbers are standardized partial regression coefficients or beta weights)

2. Model Identification

Summary

  • In general, issues of identification deal with whether a unique solution for the model (or its component parameters) can be obtained
  • It is theoretically possible for the computer to derive a unique estimate of every model parameter
  • Different types of structural equation models must meet certain requirements in order to be identified
  • Models and/or parameters may be underidentified, just-identified, or overidentified
  • A necessary, but insufficient, condition for the identification of a structural equation model is that one cannot estimate more parameters than there are unique elements in the covariance matrix: \(K\times(K+1)/2\)

Type of identification

  • Just-identified or saturated model
    • The number of structural equations composing the model exactly equals the number of unknowns
    • Only one unique solution is obtained, and the model always provides a perfect fit to the data (regression)
  • Underidentified model
    • The number of unknowns exceeds the number of equations
    • e.g. X+Y=26 no unique solution
  • Overidentified model
    • The number of equations exceeds the number of unknowns
    • A number of unique solutions. Find the solution that provides the best fit to the data.
    • e.g. a+b=6; 2a+b=10; 3a+b=12 Unable to find values of a and b that satisfy all three formulas; have to find values that is as close as possible
    • A fitting criterion: Find values of a and b that are positive and yield totals such that the sum of the squared differences between the observations (6, 10, 12) and these totals is as small as possible
  • It is important to make sure that the model is identified or overidentified! Or else we cannot find the set of values for parameters that minimize the fitting criterion

Restrictions for overidentified models

  • First, researchers assign a direction to parameters
    • Recursiveness is a sufficient condition for model identification
    • recursiveness
      • disturbances are uncorrelateed
      • all causal effects are unidirectional
      • e.g. Y1 -> Y2 -> Y1; Y1 -> Y2 -> Y3 -> Y1
  • Second, researchers set some parameters to be fixed to a predetermined value
    • Typically, values of specific parameters are set to zero

alt text

3. Model Estimation

How the program know when the correct answer is obtained?

  • In general, the user specifies a fitting criterionthat the program tries to minimize. When repeated iterations fail to minimize the fitting criterion, the program grinds to a halt and reports the last solution it estimated
  • The goal of the iterative estimation procedure is to minimize the fitting function specified by the user

Three common fitting criteria

tr= trace(sum of the diagonal elements)
S = covariance matrix implied by the model
C = actual covariance matrix
ln = natural log
| | the determinant of a matrix

  • Ordinary Least Squares (OLS)
    • \(OLS = tr(S-C)^{2}\)
  • Generalized Least Squares (GLS)
    • \(GLS = 0.5tr[(S-C)S^{-1}]^{2}\)
  • Maximum Likelihood (ML)
    • \(ML = ln|C| - ln|S| + trSC^{-1} - m\)

Choice of Estimators

  • Most widely used type of estimation is maximum likelihood, followed by generalized least squares
  • For maximum likelihood estimation: a large sample and are willing to assume multivariate normality
  • For GLS: a large sample but are not willing to assume multivariate normality
  • OLS is used in the multiple regression

Difference between OLS and ML, GLS

  • OLS is a partial information technique. Each path value in the path diagram would be estimated independently of the others
  • ML and GLS are full information techniques. One estimates all the path values simultaneously

Choice of Data

  • A covariance matrix has measures of covariance in the off-diagonal positions, with measures of variance in the main diagonal
  • A correlation matrix is simply a standardized covariance matrix, with 1s in the main diagonal
    • Remove important information about the scale of measurement of individual variables from the data

Covariance matrix or correlation matrix?

  • Theoretically, if one is concerned only with the pattern of relationships among variables, a correlation matrix is an appropriate choice
    • May result in more conservative estimates of parameter significance
  • Use of the covariance matrix is strongly recommended in virtually all instances
    • Structural equation models are not always scale free, thus a model that fits the correlation matrix may not fit the covariance matrix

Sample Size

  • Structural equation modeling is very much a large sample technique
  • In general, it seems that a sample size of at least 200 observations would be an appropriate minimum

4. Testing model fit

What does “model fit” mean?

  • The absolute fit of the model to the data
  • The fit of a model to the data relative to other models
  • The degree of parsimonious fit of the model relative to other models

Evaluate model fit

  • How well the model as a whole explains the data

Two types of statistical tests used in SEM

  • Tests of the model
    • e.g. \(\chi^{2}\) likelihood test for goodness of fit
  • Tests of the specific parameterscomposing the model
    • Assess whether parameters predicted to be nonzero in the structural equation model are in fact significantly different from zero

Types of model fit

  • Absolute fit: Concerned with the ability of the model to reproduce the actual correlation/covariance matrix
    • A nonsignificant \(\chi^{2}\) indicates that the model fits the data (df = 0.5q(q+1)-k; sample size should be large than 200)
    • RMR (Root Mean Squared Residual): The closer to zero, the more fit (0.05 good fit)
    • RMSEA (Root Mean Squared Error of Approximation): The closer to zero, the more fit (0.10 good, 0.05 very good, 0.01 outstanding)
    • GFI (goodness-of-fit index): 0.9 good
    • AGFI (adjusted goodness-of-fit index): 0.9 good. If GFI and AGFI are conflict, it indicates that model contains nonsignificant path.
    • \(\chi^{2}/df < 5\): good
    • Note, \(R^{2}\) (the coefficient of determination of the model) are measures of variance accounted for, rather than measures of model fit. It is quite possible to have a well-fitting model that explains only a modest amount of variance in the endogenous variables

  • Comparative fit: Concerned with comparing two or more competing models to assess which provides the better fit to the data; Tests of absolute fit compare the theoretical model against the just-identified model, while tests of comparative fit compare against a model that is known a priori to provides a poor fit to the data;The most common baseline model is the “null”or “independent”model (Specify no relationships between the variables composing the model)

    • NFI (normed fit index): \((\chi^{2}_{indep} - \chi^{2}_{model})/\chi^{2}_{indep}\), 0<NFI<1 (0.9 good)
    • NNFI (nonnormed fit index): \((\chi^{2}_{indep} - df_{indep}/df_{model} \chi^{2}_{model})/(\chi^{2}_{indep} - df_{model})\), (0.9 good)
    • CFI (comparative fit index): \(1 - [(\chi^{2}_{model} - df_{model})/() \chi^{2}_{indep})- df_{indep})]\), (0.9 good)
    • RFI (relative fit index): \((\chi^{2}_{indep} - df_{model}) - [df_{indep} - (df_{model}/n)]/(\chi^{2}_{indep} - df_{indep}/n)\), (0.9 good)
    • ECVI (expected value of the cross-validation index) >0: Smaller values indicate better-fitting models
  • Parsimonious fit: compared with nested model
    • PNFI (parsimonious normed fit index): \((df_{model}/df_{indep})*NFI\), the closer to 1 the better
    • PGFI (parsimonious goodness-of-fit index): \(1-(P/N)*GFI\), the closer to 1 the better
    • AIC (Akaike Information Criterion): \(\chi^{2}_{model} - 2df_{model}\)
    • CAIC (Consistent Akaike Information Criterion): \(\chi^{2}_{model} - (lnN + 1)df_{model}\), the smaller the better

Nested Model Comparisons

  • A nested relationship exists between two models if one can obtain the model with the fewest number of free parameters by constraining some or all of the parameters in the model with the largest number of free parameters. That is, the model with the fewest parameters is a subset of the model with more parameters
  • When two models stand in a nested sequence, the difference between the two may be directly tested with the \(\chi^{2}_{difference}\) test
  • When models differ in more than one parameter, the \(\chi^{2}_{difference}\) test is a useful omnibus test of the additional parameters that can be followed up by the tests of specific parameters
  • The \(\chi^{2}_{difference}\) test is valid only when the models stand in nested sequence
  • The key test of whether Model A is nested in Model B is whether all the relationships constituting Model A exist in Model B
  • The degrees of freedom for the \(\chi^{2}_{difference}\) test should always equal the number of additional paths contained in the more complex model
  • More complex(moreparameters)=better fit (smaller \(\chi^{2}\); smaller df)
  • More simple (less parameters)= worse fit=(bigger \(\chi^{2}\); bigger df)

Results in testing structural equation models

  • Consideration of the individual parameters composing the model is important for assessing the accuracy of the model
  • The parameter tests are not tests of model fit
  • Two likely results
    • a proposed model fits the data even though some parameters are nonsignificant
    • a proposed model fits the data but some of the specified parameters are significant and opposite in direction to that predicted
    • In either case, the researcher’s theory is disconfirmed even though the model may provide a good absolute fit to the data

Interpret the parameter estimates

  • Concern for overall model fit is sometimes so great that little attention is paid to whether estimates of parameters are actually meaningful

Consider equivalent models - Different configuration of hypothesized relations - Offers a competing account of the data

5. Model Respecification

If the model you specified does not fit too well, try respecification - If necessary, respecify the model and evaluate the fit of revised model to the same data - Respecification should be guided by the researcher’s hypotheses - Respecification typically consists of one of two forms of model modification + First, theory-trimming approach: deleting nonsignificant paths from the model to improve model fit. (deleting path: increase df, simplify model, increase \(\chi^{2}\)) + Second, theory-building approach: for each parameter in the model that is set to zero, calculate the decrease in the model \(\chi^{2}\) that would be obtained from estimating that parameter. The amount of change in the model \(\chi^{2}\) is refered to as the modification index for that parameter. (adding path: decrease df, complex model, decrease \(\chi^{2}\))

Problems with specification searches

  • Because specification searches are conducted post hoc and are empirically rather than theoretically derived, model modifications based on such searches must be validated on an independent sample

  • Post hoc modifications to a model should always be (1) identified as such and (2) replicated in another sample

Modification indices

  • Commonly, we would estimate any parameter that is associated with a modification index greater than 5.0
  • This rough guideline should be used with caution
    • First, such specification searches are purely exploratory in nature. Parameters added on the basis of the modification indices (or, deleted on the basis of significance tests) may be reflecting sample-specific variance
    • Second, both theory trimming and theory building are based on a large number of statistical tests, with a corresponding inflation of type I error rates. Moreover, the tests may be misleadingin that adding a parameter to a model based on the modification indices may change the value of parameters already in the model
    • Third, even when the modification indices are greater than 5.0, the improvement in model fit obtained from freeing parameters may be trivial to the model as a whole

Toward a Strategy for Assessing Model Fit

  • First, the focus of assessing model fit almost invariably should be on comparing the fit of competing and theoretically plausible models
  • Second, rather than relying on modification indices and parameter tests to guide the development of models, researchers should be prepared a priori to identify and test the sources of ambiguity in their models
  • Third, use multiple measures of fit
  • Finally, “model fit”does not equate to “truth”or “validity”

How to conduct SEM in R

Reference

  • Learning material: introduction to SEM.
  • Zhou, X.Y. (2008). Core SEM Techniques and Software.