Start Using the neg.esm()
Function
From the negligible
R package
Introduction
What is the purpose/goal of
neg.esm()?
The purpose of the neg.esm function is to evaluate whether substantial mediation (negligible direct effect) is present via negligible effect (equivalence) testing and David Kenny’s method for assessing full mediation.
What is the theory behind
neg.esm()?
Substantial mediation testing is a negligible effect testing based procedure that evaluates whether a variable is able to explain a considerable (i.e., substantial) proportion of the variability in the relationship between a predictor and an outcome. It is important to point out that this is distinct from assessing the statistical significance of the indirect effect, where the focus is only on the indirect effect (and no interest is paid to the distribution of variability across the indirect and direct effects). With the proposed method, we can estimate the equivalence of \(b_{YX}\) and \(b_{MX}b_{YM.X}\) from the direct effect of X upon Y, \(b_{YX.M}\) (Preacher & Kelley, 2011). This can be evaluated by demonstrating that the (1 − 2α)% confidence interval for \(b_{YX.M}\) falls within the negligible effect interval from −δ to δ, where δ represents a pre-specified difference between the coefficients \(b_{YX}\) and \(b_{MX}b_{YM.X}\) (or equivalently the magnitude of \(b_{YX.M}\) that would be considered meaningless within the framework of the study.
For more details regarding the methods, please see:
Beribisky, N., Mara, C. A., & Cribbie, R. A. (2020). An equivalence testing approach for evaluating substantial mediation. The Quantitative Methods for Psychology, 16(5), 424–441. https://doi.org/10.20982/tqmp.16.4.p424
https://davidakenny.net/cm/mediate.htm
Null and Alternate Hypotheses of the Procedure
The null hypothesis specifies that the total effect of X on Y is non-negligibly (i.e., meaningfully) larger in magnitude than the indirect of X on Y, through M. [This is actually assessed by evaluating whether the effect of X on Y, controlling for M (i.e., the ‘direct effect’), falls at or beyond the bounds of the negligible effect (equivalence) interval.] The alternate hypothesis states that the the effect of X on Y, controlling for M (i.e., the ‘direct effect’), falls within the negligible effect (equivalence) interval.
\(H_{01}: b^*_{YX.M} \le ei_L\)
\(H_{02}: b^*_{YX.M} \ge ei_U\)
\(H_{11}: b^*_{YX.M} \gt ei_L\)
\(H_{12}: b^*_{YX.M} \lt ei_U\)
Using neg.esm()
The basic set-up of the function looks like this:
neg.esm(X, Y, M, alpha = 0.05, minc = 0.15, eil = -0.15, eiu = 0.15, nboot = 1000L, data = NULL, plot = TRUE, saveplot = FALSE, seed = NA)
Required arguments (no default)
X - independent/predictor variable
Y - dependent/outcome variable
M - mediating variable
Optional arguments (has a default)
alpha - nominal Type I error rate. The default is .05, but any value can be used (e.g., .01, .10, .06)
minc - minimum standardized regression coefficient for the total effect (with a default of .15 in absolute value)
eil/eiu - lower and upper negligible effect bounds (standardized units), respectively, for the direct effect (c’ = c - ab). The default is (eil = -.15, eiu = .15)
nboot - number of bootstrap samples used to compute the confidence interval for the indirect effect (ab = c-c’). The default is 1000
data - the name of the R dataset which contains the variables
plot - whether a plot of the effect, along with the proportional distance, should be produced. The default is TRUE.
saveplot - whether the plot should be saved, the default is FALSE.
seed - seed number used for the bootstrap samples, if NA a random seed is used. The default is NA.
Examples
Example 1
Let’s look at an example using the mtcars dataset from R. We want to know if the weight (wt) of a vehicle mediates the relationship between the horsepower (hp) and the miles per gallon (mpg).
hp -> wt -> mpg
[1] "mpg" "cyl" "disp" "hp" "drat" "wt" "qsec" "vs" "am" "gear"
[11] "carb"Let’s start by running a traditional mediation analysis:
library(mediation)
med_model <- lm(wt ~ hp, data = d)
tot_model <- lm(mpg ~ hp + wt,
data = d)
res <- mediate(model.m = med_model,
model.y = tot_model,
treat = 'hp',
mediator = 'wt',
boot = TRUE, sims = 50)
summary(res)
Causal Mediation Analysis
Nonparametric Bootstrap Confidence Intervals with the Percentile Method
Estimate 95% CI Lower 95% CI Upper p-value
ACME -0.0365 -0.0684 -0.02 <2e-16 ***
ADE -0.0318 -0.0542 -0.02 <2e-16 ***
Total Effect -0.0682 -0.1020 -0.05 <2e-16 ***
Prop. Mediated 0.5343 0.4084 0.71 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Sample Size Used: 32
Simulations: 50 We see that both the indirect (ACME) and direct (ADE) effects are statistically significant, and that 54% of the total effect is mediated.
Now, let’s apply the neg.esm function to see if we can reject the null hypothesis that the mediation effect is non-negligible. We will use all defaults, with the exception that we will only use 10 bootstrap samples (to reduce the time of the analysis) and we will specify the dataset explicitly.
Bootstrapping may take several minutes
Bootstrapping may take several minutes
Bootstrapping may take several minutes
*** ***
Proportion Mediated: 0.534
95% CI for Proportion Mediated: (0.401, 1.035)
Completely Standardized Indirect Effect (CSIE): 0.172
95% CI for CSIE: (0.06, 0.316)
*** Equivalence Testing Method for Substantial Mediation (ESM) ***
Number of Bootstrap Iterations: 10(random seed = 8465)
Indirect Effect: -0.415
Correlation between X and Y (must be greater in magnitude than 0.15): -0.776
90% CI on Direct Effect: (-0.581, -0.276)
Equivalence Interval: Lower = -0.15; Upper = 0.15
Decision from the ESM: The null hypothesis that the direct effect (difference between the total and indirect effect) is non-negligible cannot be rejected. Substantial Mediation CANNOT be concluded. Be sure to interpret the magnitude (and precision) of the effect size.
**********************
*** Kenny Method for Full Mediation ***
ab/c (must be greater in magnitude than .80): 0.535
Correlation between X and Y (must be greater in magnitude than .2): -0.776
Decision from Kenny Procedure: Full Mediation CANNOT be concluded
**********************
*** Proportional Distance (the proportional distance from the effect of interest to the equivalence interval of the same sign) ***
Proportional Distance (PD): -2.41
95% CI for PD: (-3.988, 0.5)
**********************
Note that coefficients are now in standardized units. The total effect was large enough to conduct the analysis [r(hp, mpg) = -.78], however the direct effect was non-negligible (the confidence interval on the direct effect did not fall completely within the negligible effect interval) and thus substantial mediation cannot be concluded. The proportional distance highlights that the direct effect is 241% of the distance from 0 to the lower bound of the negligible effect interval (-.15). Using the Kenny approach to evaluating full mediation, we cannot conclude that full mediation has occurred.
Example 2
In this example, we look at whether pretest anxiety (Beck Anxiety Inventory) substantially mediates the relationship between pretest perfectionistic cognitions (Perfectionism Cognititions Inventory) and pretest depression (Centre for Epidemiological Studies of Depression). We are using the dataset perfectionism from the negligible package. In this example, we use \(\alpha\) = .10 and set the seed for replicability.
library(negligible)
neg.esm(pcipre.total,cesdpre.total,baipre.total, data=perfectionism, nboot=50,alpha=.10,seed=116)Bootstrapping may take several minutes
Bootstrapping may take several minutes
Bootstrapping may take several minutes
*** ***
Proportion Mediated: 1.364
90% CI for Proportion Mediated: (0.749, 9.722)
Completely Standardized Indirect Effect (CSIE): 0.058
90% CI for CSIE: (0.007, 0.165)
*** Equivalence Testing Method for Substantial Mediation (ESM) ***
Number of Bootstrap Iterations: 50(random seed = 116)
Indirect Effect: 0.241
Correlation between X and Y (must be greater in magnitude than 0.15): 0.177
80% CI on Direct Effect: (-0.127, 0.032)
Equivalence Interval: Lower = -0.15; Upper = 0.15
Decision from the ESM: The null hypothesis that the direct effect (difference between the total and indirect effect) is non-negligible can be rejected. Substantial Mediation CAN be concluded. Be sure to interpret the magnitude (and precision) of the effect size.
**********************
*** Kenny Method for Full Mediation ***
ab/c (must be greater in magnitude than .80): 1.363
Correlation between X and Y (must be greater in magnitude than .2): 0.177
Decision from Kenny Procedure: Full Mediation CANNOT be concluded
**********************
*** Proportional Distance (the proportional distance from the effect of interest to the equivalence interval of the same sign) ***
Proportional Distance (PD): -0.429
90% CI for PD: (-1.213, 0.471)
**********************
In this example, the confidence interval for the direct effect (\(b_{YX.M}\)) of perfectionstic cognitions on depression, controlling for anxiety (-.13, .03), falls completely within the negligible effect interval (-.15, .15) and therefore we can reject the null hypothesis that the direct effect is non-negligible. For the Kenny approach, although the proportion mediated is large, the minimum correlation between X and Y (perfectionstic cognitions and depression) was not met (i.e., there was not a large enough relationship between X and Y to discuss the proportion mediated). The small X-Y correlation, and the proportion mediated > 1, raise concerns regarding the validity of the substantial mediation analysis as well.
Extractable Elements
A number of elements of the output can be extracted, including:
minc: Minimum correlation between X and Y for a valid negligible effect (equivalence) test
corxy: Sample correlation between the IV (X) and DV (Y)
dir_eff: Sample standardized direct effect between the IV (X) and DV (Y) after controlling for the mediator (M)
eiL: Lower bound of the negligible effect (equivalence) interval
eiU: Upper bound of the negligible effect (equivalence) interval
cil: Lower bound of the 1-2*alpha CI for the standardized direct effect of X on Y
ciu: Upper bound of the 1-2*alpha CI for the standardized direct effect of X on Y
PD: Proportional distance (PD)
cilpd: Lower bound of the 1-alpha CI for the PD
ciupd: Upper bound of the 1-alpha CI for the PD
ab_par: Standardized indirect effect
abdivc_k: Proportion mediated: Standardized indirect effect divided by the standardized total effect
alpha: Nominal Type I error rate