library(pacman); p_load(lavaan, psych)
FITM <- c("chisq", "df", "nPar", "cfi", "rmsea", "rmsea.ci.lower", "rmsea.ci.upper")
describe(GrowthMindsetData)describe(Control)describe(Treatment)Burgoyne et al. (2020) demonstrated that a brief mindset intervention altered the genetic and environmental influences on measured growth mindset. With their twin data, they were able to see that an experimental growth mindset effect increased the genetic variance in their measure. This meant that the growth mindset variable they measured no longer meant the same thing as it did in their pre-treatment measurement or in their control group, because the influences on it had changed.
This allows for a direct test of whether a major assumption of measurement invariance testing holds up: that measurement invariance testing assesses whether an instrument measures the same thing in different groups. Theoretically, mindset as a construct should be the same when measured before or after an intervention, if the intervention works to change mindset. But if it supplies new sources of variance or moderates existing ones, then the construct technically has a different identity because it reflects the effects of different causes or causes acting differently.
Accordingly, the mindset variable in Burgoyne et al.’s (2020) study should be psychometrically biased to the extent it was altered by the intervention, which they verified had an impact of altering response etiology.
This method involves treating mindset as but one part of a larger construct, like conscientiousness. In this case, the latent variable I will be modeling was dubbed “Self-Determination” in Burgoyne et al. (2018). This latent variable was expanded to include an additional variable (challenge-seeking behavior) in 2020, but the inclusion or exclusion of this variable doesn’t qualitatively change the results below, and keeping it out enables an exact replication with Burgoyne et al.’s (2018) pilot study data.
If the meaning of mindset and mindset alone among indicators changed heterogeneously, the loading on it should not be invariant in the post-treatment condition, but it should be in pre-treatment. If the meaning of mindset changed homogeneously, then the intercept should not be invariant. A mixture of types of effects is certainly possible, and low power could also lead to both types of effects being detected or not detected erroneously. Moreover, if there are changes in the other indicators, that could lead to a number of possible effects, with the most likely being a shift in the loadings if one part is more affected than others.
The control and treatment group pre-test means (SDs) were 3.42 (1.13) and 3.38 (1.19) for mindset, and after treatment, they were 3.49 (1.12) and 3.68 (1.22), so I assume the effect was homogeneous with respect to the impact on the mindset measure. I cannot speak strongly to changes in the other self-determination measures, but linear moderation of additive genetic influences was only significant for the case of the mindset indicator, and not the self-determination composite, so I suspect that was impacted more strongly, as one might plausibly suspect given the intervention was targeted at mindset.
Because this is a latent variable model with three correlated indicators, the initial model fits obviously don’t mean anything, but presenting them is perfunctory.
OverPre <- '
SD =~ MindpreFIN + GritTOTpre + AlcpreFIN'
OverPost <- '
SD =~ GritTOTpos + MindposFIN + AlcposFIN'
PreFitC <- sem(OverPre, Control, std.lv = T, estimator = "DWLS")
PreFitT <- sem(OverPre, Treatment, std.lv = T, estimator = "DWLS")
PostFitC <- sem(OverPost, Control, std.lv = T, estimator = "DWLS")
PostFitT <- sem(OverPost, Treatment, std.lv = T, estimator = "DWLS")
round(cbind("Control, Pre" = fitMeasures(PreFitC, FITM),
"Treatment, Pre" = fitMeasures(PreFitT, FITM),
"Control, Post" = fitMeasures(PostFitC, FITM),
"Treatment, Post" = fitMeasures(PostFitT, FITM)), 3)## Control, Pre Treatment, Pre Control, Post Treatment, Post
## chisq 0 0 0 0
## df 0 0 0 0
## npar 6 6 6 6
## cfi 1 1 1 1
## rmsea 0 0 0 0
## rmsea.ci.lower 0 0 0 0
## rmsea.ci.upper 0 0 0 0Let’s now check invariance for the pre-treatment condition. This model should be completely invariant, since the only thing differentiating the groups at this point was randomization.
PreFitConfigural <- sem(OverPre, GrowthMindsetData, std.lv = T, estimator = "DWLS", group = "Condition")
PreFitMetric <- sem(OverPre, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = "loadings")
PreFitScalar <- sem(OverPre, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"))
PreFitStrict <- sem(OverPre, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"))
PreFitLVs <- sem(OverPre, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"))
PreFitMean <- sem(OverPre, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "means"))
round(cbind(Configural = fitMeasures(PreFitConfigural, FITM),
Metric = fitMeasures(PreFitMetric, FITM),
Scalar = fitMeasures(PreFitScalar, FITM),
Strict = fitMeasures(PreFitStrict, FITM),
"Latent Variances" = fitMeasures(PreFitLVs, FITM),
"Latent Mean" = fitMeasures(PreFitMean, FITM)), 3)## Configural Metric Scalar Strict Latent Variances Latent Mean
## chisq 0 1.068 1.606 6.770 6.845 7.548
## df 0 2.000 4.000 7.000 8.000 9.000
## npar 18 16.000 14.000 11.000 10.000 9.000
## cfi 1 1.000 1.000 1.000 1.000 1.000
## rmsea 0 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.lower 0 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0 0.057 0.032 0.042 0.036 0.034PostFitConfigural <- sem(OverPost, GrowthMindsetData, std.lv = T, estimator = "DWLS", group = "Condition")
PostFitMetric <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = "loadings")
"Metric Invariance"## [1] "Metric Invariance"1 - pchisq(fitMeasures(PostFitMetric, "chisq"), 2) #Marginally significant## chisq
## 0.053PostFitMetricP <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = "loadings",
group.partial = c("SD =~ MindposFIN"))
PostFitScalarV1 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("SD =~ MindposFIN"))
PostFitScalarV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"))
"Mindset Loading Freed"## [1] "Mindset Loading Freed"1 - pchisq(fitMeasures(PostFitScalarV1, "chisq") - fitMeasures(PostFitMetricP, "chisq"), 2); "Mindset Loading Not Freed"## chisq
## 0.016## [1] "Mindset Loading Not Freed"1 - pchisq(fitMeasures(PostFitScalarV2, "chisq") - fitMeasures(PostFitMetric, "chisq"), 2)## chisq
## 0.02PostFitScalarPV1 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("SD =~ MindposFIN", "MindposFIN ~ 1"))
PostFitScalarPV1G <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("SD =~ MindposFIN", "GritTOTpos ~ 1"))
PostFitScalarPV1L <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("SD =~ MindposFIN", "AlcposFIN ~ 1"))
PostFitScalarPV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("MindposFIN ~ 1"))
PostFitScalarPV2G <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("GritTOTpos ~ 1"))
PostFitScalarPV2L <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = c("AlcposFIN ~ 1"))
"Mindset Intercept Freed"## [1] "Mindset Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV1, "chisq") - fitMeasures(PostFitMetricP, "chisq"), 1); "Grit Intercept Freed"## chisq
## 0.896## [1] "Grit Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV1G, "chisq") - fitMeasures(PostFitMetricP, "chisq"), 1); "Locus of Control Intercept Freed"## chisq
## 0.005## [1] "Locus of Control Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV1L, "chisq") - fitMeasures(PostFitMetricP, "chisq"), 1); "Mindset Intercept Freed"## chisq
## 0.005## [1] "Mindset Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV2, "chisq") - fitMeasures(PostFitMetric, "chisq"), 1); "Grit Intercept Freed"## chisq
## 0.891## [1] "Grit Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV2G, "chisq") - fitMeasures(PostFitMetric, "chisq"), 1); "Locus of Control Intercept Freed"## chisq
## 0.008## [1] "Locus of Control Intercept Freed"1 - pchisq(fitMeasures(PostFitScalarPV2L, "chisq") - fitMeasures(PostFitMetric, "chisq"), 1)## chisq
## 0.008PostFitStrictV1 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = c("SD =~ MindposFIN", "MindposFIN ~ 1"))
PostFitStrictV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = c("MindposFIN ~ 1"))
PostFitStrictPV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = c("MindposFIN ~ 1", "MindposFIN ~~ MindposFIN"))
PostFitStrictPV2G <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = c("MindposFIN ~ 1", "GritTOTpos ~~ GritTOTpos"))
PostFitStrictPV2L <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = c("MindposFIN ~ 1", "AlcposFIN ~~ AlcposFIN"))
"Mindset Residual Freed"## [1] "Mindset Residual Freed"1 - pchisq(fitMeasures(PostFitStrictPV2, "chisq") - fitMeasures(PostFitScalarPV2, "chisq"), 2); "Grit Residual Freed"## chisq
## 0.395## [1] "Grit Residual Freed"1 - pchisq(fitMeasures(PostFitStrictPV2G, "chisq") - fitMeasures(PostFitScalarPV2, "chisq"), 2); "Locus of Control Residual Freed"## chisq
## 0.033## [1] "Locus of Control Residual Freed"1 - pchisq(fitMeasures(PostFitStrictPV2L, "chisq") - fitMeasures(PostFitScalarPV2, "chisq"), 2)## chisq
## 0.017PostFitLVsV1 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"),
group.partial = c("SD =~ MindposFIN", "MindposFIN ~ 1"))
PostFitLVsV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"),
group.partial = c("MindposFIN ~ 1", "MindposFIN ~~ MindposFIN"))
PostFitMeanV1 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "means"),
group.partial = c("SD =~ MindposFIN", "MindposFIN ~ 1"))
PostFitMeanV2 <- sem(OverPost, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "means"),
group.partial = c("MindposFIN ~ 1", "MindposFIN ~~ MindposFIN"))round(cbind(Configural = fitMeasures(PostFitConfigural, FITM),
Metric = fitMeasures(PostFitMetric, FITM),
"Partial Metric" = fitMeasures(PostFitMetricP, FITM),
"Scalar V1" = fitMeasures(PostFitScalarV1, FITM),
"Scalar V2" = fitMeasures(PostFitScalarV2, FITM),
"Partial Scalar V1" = fitMeasures(PostFitScalarPV1, FITM),
"Partial Scalar V2" = fitMeasures(PostFitScalarPV2, FITM),
"Strict V1" = fitMeasures(PostFitStrictV1, FITM),
"Strict V2" = fitMeasures(PostFitStrictV2, FITM),
"Partial Strict V2" = fitMeasures(PostFitStrictPV2, FITM),
"Latent Variances V1" = fitMeasures(PostFitLVsV1, FITM),
"Latent Variances V2" = fitMeasures(PostFitLVsV2, FITM),
"Latent Mean V1" = fitMeasures(PostFitMeanV1, FITM),
"Latent Mean V2" = fitMeasures(PostFitMeanV2, FITM)), 3)## Configural Metric Partial Metric Scalar V1 Scalar V2
## chisq 0 5.868 0.033 8.316 13.681
## df 0 2.000 1.000 3.000 4.000
## npar 18 16.000 17.000 15.000 14.000
## cfi 1 0.994 1.000 0.992 0.986
## rmsea 0 0.048 0.000 0.046 0.054
## rmsea.ci.lower 0 0.000 0.000 0.009 0.024
## rmsea.ci.upper 0 0.095 0.051 0.085 0.086
## Partial Scalar V1 Partial Scalar V2 Strict V1 Strict V2
## chisq 0.05 5.886 1.918 14.266
## df 2.00 3.000 5.000 6.000
## npar 16.00 15.000 13.000 12.000
## cfi 1.00 0.996 1.000 0.988
## rmsea 0.00 0.034 0.000 0.041
## rmsea.ci.lower 0.00 0.000 0.000 0.013
## rmsea.ci.upper 0.00 0.075 0.026 0.068
## Partial Strict V2 Latent Variances V1 Latent Variances V2
## chisq 7.744 3.576 7.792
## df 5.000 6.000 6.000
## npar 13.000 12.000 12.000
## cfi 0.996 1.000 0.997
## rmsea 0.026 0.000 0.019
## rmsea.ci.lower 0.000 0.000 0.000
## rmsea.ci.upper 0.059 0.033 0.051
## Latent Mean V1 Latent Mean V2
## chisq 4.114 8.329
## df 7.000 7.000
## npar 11.000 11.000
## cfi 1.000 0.998
## rmsea 0.000 0.015
## rmsea.ci.lower 0.000 0.000
## rmsea.ci.upper 0.029 0.047So, regardless of the choice of freeing or not freeing the loading for mindset, the best option is to free the intercept for mindset. Freeing the other intercepts seems to have no effect, and after the intercept is freed, the model fits well in each subsequent stage of testing. From this, we can conclude that it is not the self-determination construct whose meaning changed, but the mindset indicator that was the target of the intervention that did.
This suggests that pre- and post-measured self-determination should have non-invariant indicator covariances for mindset, but not for the latent variable itself, net of this covariance.
This may require freeing the post-treatment mindset loading, because in the configuration where the mindset loading wasn’t freed, its residual variance had to be freed. But, this really means that the residual covariances ought to differ and the evidence that the loading needs to be freed is weaker than the evidence that the residual does, if only one has to be selected, which is likely, if only for power reasons. Even though the power for such a test might be very poor, but we can still try it.
This initial fit is, again, not useful, but presented for perfunctory reasons.
Over <- '
SDPre =~ MindpreFIN + GritTOTpre + AlcpreFIN
SDPost =~ GritTOTpos + MindposFIN + AlcposFIN
SDPre ~~ SDPost
MindpreFIN ~~ MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN'
FitC <- sem(Over, Control, std.lv = T, estimator = "DWLS")
FitT <- sem(Over, Treatment, std.lv = T, estimator = "DWLS")
round(cbind(Control = fitMeasures(FitC, FITM),
Treatment = fitMeasures(FitT, FITM)), 3)## Control Treatment
## chisq 0.359 0.336
## df 5.000 5.000
## npar 16.000 16.000
## cfi 1.000 1.000
## rmsea 0.000 0.000
## rmsea.ci.lower 0.000 0.000
## rmsea.ci.upper 0.000 0.000FitConfigural <- sem(Over, GrowthMindsetData, std.lv = T, estimator = "DWLS", group = "Condition")
FitMetric <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = "loadings")
FitMetricP <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = "loadings",
group.partial = "SDPost =~ MindposFIN")
FitScalar <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts"),
group.partial = "SDPost =~ MindposFIN")
FitStrict <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = "SDPost =~ MindposFIN")
FitLVs <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"),
group.partial = "SDPost =~ MindposFIN")
FitOCVs <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances"),
group.partial = "SDPost =~ MindposFIN")
FitLCVs <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances", "lv.covariances"),
group.partial = "SDPost =~ MindposFIN")
FitMean <- sem(Over, GrowthMindsetData, std.lv = F, estimator = "DWLS", group = "Condition",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances", "lv.covariances", "means"),
group.partial = "SDPost =~ MindposFIN")
round(cbind(Configural = fitMeasures(FitConfigural, FITM),
Metric = fitMeasures(FitMetric, FITM),
"Partial Metric" = fitMeasures(FitMetricP, FITM),
Scalar = fitMeasures(FitScalar, FITM),
Strict = fitMeasures(FitStrict, FITM),
"Latent Variances" = fitMeasures(FitLVs, FITM),
"Residual Covariances" = fitMeasures(FitOCVs, FITM),
"Latent Covariances" = fitMeasures(FitLCVs, FITM),
"Latent Mean" = fitMeasures(FitMean, FITM)), 3)## Configural Metric Partial Metric Scalar Strict Latent Variances
## chisq 0.695 14.686 5.645 14.462 22.399 23.667
## df 10.000 14.000 13.000 17.000 23.000 25.000
## npar 44.000 40.000 41.000 37.000 31.000 29.000
## cfi 1.000 1.000 1.000 1.000 1.000 1.000
## rmsea 0.000 0.008 0.000 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.035 0.000 0.027 0.028 0.026
## Residual Covariances Latent Covariances Latent Mean
## chisq 24.790 26.220 29.000
## df 28.000 29.000 31.000
## npar 26.000 25.000 23.000
## cfi 1.000 1.000 1.000
## rmsea 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000
## rmsea.ci.upper 0.023 0.023 0.024This method may have been underpowered to detect a difference in covariances. All that was confirmed was a difference in the loading of post-treatment mindset. It’s unclear what this means beyond that the interpretation of the measures changed, seemingly heterogeneously. There might have also been an effect on the intercept for mindset, but the result was marginally-significant. Perhaps the lack of an effect on the intercept was due to controlling for the stable part of mindset (from the pre-treatment phase), resulting in more limited power.
A simple way to test invariance after an experiment, even without a control group, is to do so longitudinally.
For the control group, it should be possible, albeit with the caveat that people generally become less interested in doing things the second time around, so there might be something different, but if that’s the case, the effect shouldn’t be relegated to the mindset measure alone unless it is particularly boring or something. To make sure we don’t miss an effect on mindset because of the choice of indicator variable, we should rearrange the indicator order.
LongConfigural <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ 1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ 1
MindposFIN ~ 1
GritTOTpre ~~ GritTOTpre
AlcpreFIN ~~ AlcpreFIN
MindpreFIN ~~ MindpreFIN
GritTOTpos ~~ GritTOTpos
AlcposFIN ~~ AlcposFIN
MindposFIN ~~ MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongMetric <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ 1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ 1
MindposFIN ~ 1
GritTOTpre ~~ GritTOTpre
AlcpreFIN ~~ AlcpreFIN
MindpreFIN ~~ MindpreFIN
GritTOTpos ~~ GritTOTpos
AlcposFIN ~~ AlcposFIN
MindposFIN ~~ MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongScalar <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ i3*1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ i3*1
GritTOTpre ~~ GritTOTpre
AlcpreFIN ~~ AlcpreFIN
MindpreFIN ~~ MindpreFIN
GritTOTpos ~~ GritTOTpos
AlcposFIN ~~ AlcposFIN
MindposFIN ~~ MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongScalarP <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ 1
GritTOTpre ~~ GritTOTpre
AlcpreFIN ~~ AlcpreFIN
MindpreFIN ~~ MindpreFIN
GritTOTpos ~~ GritTOTpos
AlcposFIN ~~ AlcposFIN
MindposFIN ~~ MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongStrict <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ i3*1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ i3*1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongStrictPS <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ 1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ SDPost
SDPre ~~ SDPost'
LongLVs <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ i3*1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ i3*1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ 1*SDPost
SDPre ~~ SDPost'
LongLVsPS <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ 1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 1
SDPre ~~ 1*SDPre
SDPost ~~ 1*SDPost
SDPre ~~ SDPost'
LongMeans <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ i3*1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ i3*1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 0*1
SDPre ~~ 1*SDPre
SDPost ~~ 1*SDPost
SDPre ~~ SDPost'
LongMeansPS <- '
SDPre =~ NA*GritTOTpre + l1*AlcpreFIN + l2*MindpreFIN
SDPost =~ NA*GritTOTpos + l1*AlcposFIN + l2*MindposFIN
GritTOTpre ~ i1*1
AlcpreFIN ~ i2*1
MindpreFIN ~ 1
GritTOTpos ~ i1*1
AlcposFIN ~ i2*1
MindposFIN ~ 1
GritTOTpre ~~ u1*GritTOTpre
AlcpreFIN ~~ u2*AlcpreFIN
MindpreFIN ~~ u3*MindpreFIN
GritTOTpos ~~ u1*GritTOTpos
AlcposFIN ~~ u2*AlcposFIN
MindposFIN ~~ u3*MindposFIN
GritTOTpre ~~ GritTOTpos
AlcpreFIN ~~ AlcposFIN
MindpreFIN ~~ MindposFIN
SDPre ~ 0*1
SDPost ~ 0*1
SDPre ~~ 1*SDPre
SDPost ~~ 1*SDPost
SDPre ~~ SDPost'
FitConLong <- sem(LongConfigural, Control, estimator = "DWLS")
FitMetLong <- sem(LongMetric, Control, estimator = "DWLS")
FitScaLong <- sem(LongScalar, Control, estimator = "DWLS")
FitResLong <- sem(LongStrict, Control, estimator = "DWLS")
FitLVsLong <- sem(LongLVs, Control, estimator = "DWLS")
FitMeansLong <- sem(LongMeans, Control, estimator = "DWLS")
round(cbind(Configural = fitMeasures(FitConLong, FITM),
Metric = fitMeasures(FitMetLong, FITM),
Scalar = fitMeasures(FitScaLong, FITM),
Strict = fitMeasures(FitResLong, FITM),
"Latent Variances" = fitMeasures(FitLVsLong, FITM),
"Latent Means" = fitMeasures(FitMeansLong, FITM)), 3)## Configural Metric Scalar Strict Latent Variances Latent Means
## chisq 0.359 0.846 2.337 2.551 2.574 4.287
## df 5.000 6.000 8.000 11.000 12.000 13.000
## npar 22.000 21.000 19.000 16.000 15.000 14.000
## cfi 1.000 1.000 1.000 1.000 1.000 1.000
## rmsea 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.000 0.000 0.000 0.000 0.000FitConLong <- sem(LongConfigural, Treatment, estimator = "DWLS")
FitMetLong <- sem(LongMetric, Treatment, estimator = "DWLS")
FitScaLong <- sem(LongScalar, Treatment, estimator = "DWLS")
FitScaLongP <- sem(LongScalarP, Treatment, estimator = "DWLS")
FitResLong <- sem(LongStrictPS, Treatment, estimator = "DWLS")
FitLVsLong <- sem(LongLVsPS, Treatment, estimator = "DWLS")
FitMeansLong <- sem(LongMeansPS, Treatment, estimator = "DWLS")
round(cbind(Configural = fitMeasures(FitConLong, FITM),
Metric = fitMeasures(FitMetLong, FITM),
Scalar = fitMeasures(FitScaLong, FITM),
"Partial Scalar" = fitMeasures(FitScaLongP, FITM),
Strict = fitMeasures(FitResLong, FITM),
"Latent Variances" = fitMeasures(FitLVsLong, FITM),
"Latent Means" = fitMeasures(FitMeansLong, FITM)), 3)## Configural Metric Scalar Partial Scalar Strict Latent Variances
## chisq 0.336 2.355 19.700 2.790 3.862 4.013
## df 5.000 6.000 8.000 7.000 10.000 11.000
## npar 22.000 21.000 19.000 20.000 17.000 16.000
## cfi 1.000 1.000 0.995 1.000 1.000 1.000
## rmsea 0.000 0.000 0.042 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.019 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.022 0.066 0.018 0.000 0.000
## Latent Means
## chisq 9.253
## df 12.000
## npar 15.000
## cfi 1.000
## rmsea 0.000
## rmsea.ci.lower 0.000
## rmsea.ci.upper 0.028Longitudinal invariance is completely tenable for the control group, but not for the treatment group. In the treatment group, the intercept for mindset differs between pre- and post-treatment, and the latent mean also changed. Since this method is more powerful for detecting differences in interpretation over time, it also suggests that the earlier result regarding a possible difference in the loadings should have, instead, been regarded as a difference in the intercepts.
An additional way to check whether the mindset indicator’s meaning has changed is to regress it on something like g, but this could be problematic, because post-treatment indicators might be polluted by the retesting effect, leading to non-invariance. But, the authors considered this for the Shipley-2 vocabulary test and administered different items in the pre- and post-treatment. This helps to minimize retesting effects, but it means that vocabulary, as an indicator of g, might not have the same loadings, intercepts, residuals, etc. because it is, in fact, a different indicator. For the Shipley-2 block design test, the authors did not use different forms, polluting it with the retest effect.
Since this affected both groups equally, we shouldn’t be at a loss though, we’re just almost-certainly underpowered and there are incredibly many ways to do this, with different assumptions, and I don’t wish to go through them all since they don’t actually show us much new information.
Burgoyne et al. (2018) conducted a pilot study with the same self-determination measures. The sample for this study was much smaller and the intervention effect appeared more diverse (which could be due to sampling error), but we can still try to use it to replicate this effect.
describe(PilotMindsetData)PreFitConfigural <- sem(OverPre, PilotMindsetData, std.lv = T, estimator = "DWLS", group = "cond")
PreFitMetric <- sem(OverPre, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = "loadings")
PreFitScalar <- sem(OverPre, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts"))
PreFitStrict <- sem(OverPre, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals"))
PreFitLVs <- sem(OverPre, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"))
PreFitMean <- sem(OverPre, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "means"))
round(cbind(Configural = fitMeasures(PreFitConfigural, FITM),
Metric = fitMeasures(PreFitMetric, FITM),
Scalar = fitMeasures(PreFitScalar, FITM),
Strict = fitMeasures(PreFitStrict, FITM),
"Latent Variances" = fitMeasures(PreFitLVs, FITM),
"Latent Mean" = fitMeasures(PreFitMean, FITM)), 3)## Configural Metric Scalar Strict Latent Variances Latent Mean
## chisq 0 1.910 4.322 6.186 8.781 8.866
## df 0 2.000 4.000 7.000 8.000 9.000
## npar 18 16.000 14.000 11.000 10.000 9.000
## cfi 1 1.000 0.998 1.000 0.996 1.000
## rmsea 0 0.000 0.018 0.000 0.020 0.000
## rmsea.ci.lower 0 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0 0.125 0.100 0.073 0.080 0.071PostFitConfigural <- sem(OverPost, PilotMindsetData, std.lv = T, estimator = "DWLS", group = "cond")
PostFitMetric <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = "loadings")
PostFitScalar <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts"))
PostFitScalarP <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts"),
group.partial = "MindposFIN ~ 1")
PostFitStrict <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = "MindposFIN ~ 1")
PostFitLVs <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"),
group.partial = "MindposFIN ~ 1")
PostFitMean <- sem(OverPost, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "means"),
group.partial = "MindposFIN ~ 1")
round(cbind(Configural = fitMeasures(PostFitConfigural, FITM),
Metric = fitMeasures(PostFitMetric, FITM),
Scalar = fitMeasures(PostFitScalar, FITM),
"Partial Scalar" = fitMeasures(PostFitScalarP, FITM),
Strict = fitMeasures(PostFitStrict, FITM),
"Latent Variances" = fitMeasures(PostFitLVs, FITM),
"Latent Mean" = fitMeasures(PostFitMean, FITM)), 3)## Configural Metric Scalar Partial Scalar Strict Latent Variances
## chisq 0 0.221 16.732 0.458 4.127 5.319
## df 0 2.000 4.000 3.000 6.000 7.000
## npar 18 16.000 14.000 15.000 12.000 11.000
## cfi 1 1.000 0.930 1.000 1.000 1.000
## rmsea 0 0.000 0.114 0.000 0.000 0.000
## rmsea.ci.lower 0 0.000 0.061 0.000 0.000 0.000
## rmsea.ci.upper 0 0.057 0.174 0.033 0.067 0.066
## Latent Mean
## chisq 6.926
## df 8.000
## npar 10.000
## cfi 1.000
## rmsea 0.000
## rmsea.ci.lower 0.000
## rmsea.ci.upper 0.068FitConfigural <- sem(Over, PilotMindsetData, std.lv = T, estimator = "DWLS", group = "cond")
FitMetric <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = "loadings")
FitScalar <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts"))
FitScalarP <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts"),
group.partial = "MindposFIN ~ 1")
FitStrict <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals"),
group.partial = "MindposFIN ~ 1")
FitLVs <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances"),
group.partial = "MindposFIN ~ 1")
FitOCVs <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances"),
group.partial = "MindposFIN ~ 1")
FitOCVsP <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances"),
group.partial = c("MindposFIN ~ 1", "MindpreFIN ~~ MindposFIN"))
FitLCVs <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances", "lv.covariances"),
group.partial = c("MindposFIN ~ 1", "MindpreFIN ~~ MindposFIN"))
FitMean <- sem(Over, PilotMindsetData, std.lv = F, estimator = "DWLS", group = "cond",
group.equal = c("loadings", "intercepts", "residuals", "lv.variances", "residual.covariances", "means"),
group.partial = c("MindposFIN ~ 1", "MindpreFIN ~~ MindposFIN"))
round(cbind(Configural = fitMeasures(FitConfigural, FITM),
Metric = fitMeasures(FitMetric, FITM),
Scalar = fitMeasures(FitScalar, FITM),
"Partial Scalar" = fitMeasures(FitScalarP, FITM),
Strict = fitMeasures(FitStrict, FITM),
"Latent Variances" = fitMeasures(FitLVs, FITM),
"Residual Covariances" = fitMeasures(FitOCVs, FITM),
"Partial Residual Covariances" = fitMeasures(FitOCVsP, FITM),
"Latent Covariances" = fitMeasures(FitLCVs, FITM),
"Latent Mean" = fitMeasures(FitMean, FITM)), 3)## Configural Metric Scalar Partial Scalar Strict Latent Variances
## chisq 0.832 3.726 22.708 6.464 11.883 15.644
## df 10.000 14.000 18.000 17.000 23.000 25.000
## npar 44.000 40.000 36.000 37.000 31.000 29.000
## cfi 1.000 1.000 0.997 1.000 1.000 1.000
## rmsea 0.000 0.000 0.033 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.000 0.069 0.000 0.000 0.017
## Residual Covariances Partial Residual Covariances
## chisq 24.751 17.685
## df 28.000 27.000
## npar 26.000 27.000
## cfi 1.000 1.000
## rmsea 0.000 0.000
## rmsea.ci.lower 0.000 0.000
## rmsea.ci.upper 0.042 0.020
## Latent Covariances Latent Mean
## chisq 20.525 19.274
## df 28.000 29.000
## npar 26.000 25.000
## cfi 1.000 1.000
## rmsea 0.000 0.000
## rmsea.ci.lower 0.000 0.000
## rmsea.ci.upper 0.029 0.019FitConLong <- sem(LongConfigural, Control, estimator = "DWLS")
FitMetLong <- sem(LongMetric, Control, estimator = "DWLS")
FitScaLong <- sem(LongScalar, Control, estimator = "DWLS")
FitResLong <- sem(LongStrict, Control, estimator = "DWLS")
FitLVsLong <- sem(LongLVs, Control, estimator = "DWLS")
FitMeansLong <- sem(LongMeans, Control, estimator = "DWLS")
round(cbind(Configural = fitMeasures(FitConLong, FITM),
Metric = fitMeasures(FitMetLong, FITM),
Scalar = fitMeasures(FitScaLong, FITM),
Strict = fitMeasures(FitResLong, FITM),
"Latent Variances" = fitMeasures(FitLVsLong, FITM),
"Latent Means" = fitMeasures(FitMeansLong, FITM)), 3)## Configural Metric Scalar Strict Latent Variances Latent Means
## chisq 0.125 0.903 1.517 4.880 5.077 5.088
## df 5.000 6.000 8.000 11.000 12.000 13.000
## npar 22.000 21.000 19.000 16.000 15.000 14.000
## cfi 1.000 1.000 1.000 1.000 1.000 1.000
## rmsea 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.000 0.000 0.016 0.000 0.000FitConLong <- sem(LongConfigural, Treatment, estimator = "DWLS")
FitMetLong <- sem(LongMetric, Treatment, estimator = "DWLS")
FitScaLong <- sem(LongScalar, Treatment, estimator = "DWLS")
FitScaLongP <- sem(LongScalarP, Treatment, estimator = "DWLS")
FitResLong <- sem(LongStrictPS, Treatment, estimator = "DWLS")
FitLVsLong <- sem(LongLVsPS, Treatment, estimator = "DWLS")
FitMeansLong <- sem(LongMeansPS, Treatment, estimator = "DWLS")
round(cbind(Configural = fitMeasures(FitConLong, FITM),
Metric = fitMeasures(FitMetLong, FITM),
Scalar = fitMeasures(FitScaLong, FITM),
"Partial Scalar" = fitMeasures(FitScaLongP, FITM),
Strict = fitMeasures(FitResLong, FITM),
"Latent Variances" = fitMeasures(FitLVsLong, FITM),
"Latent Means" = fitMeasures(FitMeansLong, FITM)), 3)## Configural Metric Scalar Partial Scalar Strict Latent Variances
## chisq 0.707 0.987 6.513 1.056 2.205 3.595
## df 5.000 6.000 8.000 7.000 10.000 11.000
## npar 22.000 21.000 19.000 20.000 17.000 16.000
## cfi 1.000 1.000 1.000 1.000 1.000 1.000
## rmsea 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.lower 0.000 0.000 0.000 0.000 0.000 0.000
## rmsea.ci.upper 0.000 0.000 0.066 0.000 0.000 0.000
## Latent Means
## chisq 5.299
## df 12.000
## npar 15.000
## cfi 1.000
## rmsea 0.000
## rmsea.ci.lower 0.000
## rmsea.ci.upper 0.008So, just as in Burgoyne et al.’s (2020) large twin study, their MTurk-based pilot study from 2018 also showed that the intercept for mindset was biased due to a homogeneous effect that changed the meaning of the level of the variable. However, when testing longitudinal invariance, the effect was only marginal (p = 0.063).
Measurement invariance testing works. When the causes of variation in an outcome are explicitly modeled and we see that the A, C, E, or any combination thereof are modified by an intervention, and thus the causes of variation in that outcome verifiably differ, measurement invariance testing shows they do, as expected.
If that is true, the result we expected was invariance between the control group in the pre and post sessions, invariance between the control group and the treatment in the pre session, and no invariance with respect to the treatment group in the post session.
That is exactly what was found: there was correspondence in the mindset measure between the control groups before and after the treatment condition occurred; the same was not true for the treatment group. The two groups were also comparable beforehand, but not after, exactly as one would predict from measurement invariance testing working, and the biometric results Burgoyne et al. (2020) achieved.
I would like to thank Alexander Burgoyne for supplying this twin data and for having made his original pilot study data public in the first place.
Burgoyne, A. P., Carroll, S., Clark, D. A., Hambrick, D. Z., Plaisance, K. S., Klump, K. L., & Burt, S. A. (2020). Can a brief intervention alter genetic and environmental influences on psychological traits? An experimental behavioral genetics approach. Learning and Motivation, 72, 101683. https://doi.org/10.1016/j.lmot.2020.101683
Burgoyne, A. P., Hambrick, D. Z., Moser, J. S., & Burt, S. A. (2018). Analysis of a mindset intervention. Journal of Research in Personality, 77, 21–30. https://doi.org/10.1016/j.jrp.2018.09.004
Pilot Data: https://osf.io/atdnm/ Pilot Data Archive: https://web.archive.org/web/20230117213713/https://osf.io/atdnm/
sessionInfo()## R version 4.3.1 (2023-06-16 ucrt)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 19045)
##
## Matrix products: default
##
##
## locale:
## [1] LC_COLLATE=English_United States.utf8
## [2] LC_CTYPE=English_United States.utf8
## [3] LC_MONETARY=English_United States.utf8
## [4] LC_NUMERIC=C
## [5] LC_TIME=English_United States.utf8
##
## time zone: America/Chicago
## tzcode source: internal
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] psych_2.3.6 lavaan_0.6-16 pacman_0.5.1 dplyr_1.1.2 haven_2.5.3
##
## loaded via a namespace (and not attached):
## [1] jsonlite_1.8.7 compiler_4.3.1 tidyselect_1.2.0 parallel_4.3.1
## [5] jquerylib_0.1.4 yaml_2.3.7 fastmap_1.1.1 lattice_0.21-8
## [9] pbivnorm_0.6.0 readr_2.1.4 R6_2.5.1 generics_0.1.3
## [13] knitr_1.43 MASS_7.3-60 forcats_1.0.0 tibble_3.2.1
## [17] bslib_0.5.0 pillar_1.9.0 tzdb_0.4.0 rlang_1.1.1
## [21] utf8_1.2.3 cachem_1.0.8 xfun_0.39 quadprog_1.5-8
## [25] sass_0.4.7 cli_3.6.1 magrittr_2.0.3 grid_4.3.1
## [29] digest_0.6.33 rstudioapi_0.15.0 hms_1.1.3 nlme_3.1-162
## [33] lifecycle_1.0.3 vctrs_0.6.3 mnormt_2.1.1 evaluate_0.21
## [37] glue_1.6.2 stats4_4.3.1 fansi_1.0.4 rmarkdown_2.23
## [41] tools_4.3.1 pkgconfig_2.0.3 htmltools_0.5.5