Start Using the neg.twoindmeans()
Function
From the negligible
R package
Introduction
What is the purpose/goal of
neg.twoindmeans()?
The purpose of the neg.twoindmeans function is to test whether the difference between the means of two independent populations is negligibly different (i.e., practically equivalent), where negligible represents the smallest meaningful effect size (MMES) determined a priori by a researcher via an equivalence interval (EI). For example, imagine that you want to test whether the depression scores of participants in an online intervention condition are equivalent to scores for those in a traditional intervention condition at post-assessment. To address such research questions (e.g., mean group similarity), you could use the neg.twoindmeans function.
What is the theory behind
neg.twoindmeans()?
The user specifies either the data associated with the first and second groups/populations (v1, v2, both should be continuous) or specifies the Independent Variable/Predictor (iv, should be a factor) and the Dependent Variable (dv, should be continuous). A ‘data’ statement can be used if the variables are stored in an R dataset.
The user must also specify the lower and upper bounds of the negligible effect (equivalence) interval. These are specified in the original units of the outcome variable.
Null and Alternate Hypotheses of the Procedure
\(H_{01}\): \(\mu_{1}\)- \(\mu_{2}\) \(\geqslant\) \(\delta\); \(H_{02}\): \(\mu_{1}\)- \(\mu_{2}\) \(\leqslant\) -\(\delta\) (The pair of null hypotheses specify that the difference between the independent means in the population exceed the equivalence interval; i.e., that the effect is nonnegligible, either at the upper bound or the lower bound).
\(H_{a1}\): \(\mu_{1}\)- \(\mu_{2}\) \(<\) \(\delta\); \(H_{a2}\): \(\mu_{1}\)- \(\mu_{2}\) \(>\) -\(\delta\) (the pair of alternative hypotheses specifies that the difference between the independent means in the population does not exceed the equivalence interval; i.e., it is negligible at either the upper or lower bound. In other words, that the effect is completely contained within the equivalence interval).
Using neg.twoindmeans()
neg.twoindmeans(v1 = NULL, v2 = NULL, dv = NULL, iv = NULL, eiL, eiU, varequiv = FALSE, normality = FALSE, tr = 0.2, nboot = 500, alpha = 0.05, plot = TRUE, saveplot = FALSE, data=NULL, …)
Function Set-up:
Required arguments (no default)
eiL: the lower bound of the equivalence interval
eiU: the upper bound of the equivalence interval
Optional arguments (has a default)
v1: data for Group 1 (if dv and iv are omitted)
v2: data for Group 2 (if dv and iv are omitted)
dv: dependent variable (if v1 and v2 are omitted)
iv: dichotomous predictor/independent variable (if v1 and v2 are omitted)
varequiv: are the population variances assumed to be equal? Population variances are assumed to be unequal if normality=FALSE
normality: are the population variances (and hence the residuals) assumed to be normally distributed?
tr: proportion of trimming from each tail (relevant if normality = FALSE)
tr: proportion of trimming from each tail (relevant if normality = FALSE)
nboot: number of bootstrap samples for calculating confidence intervals
alpha: nominal type I error rate
plot: should a plot of the results be produced?
saveplot: should the plot be saved?
data: dataset containing v1/v2 or iv/dv
Examples
Example 1
Let’s look at an example using the ToothGrowth dataset from R. Our outcome variable is Guinea pig tooth (odontoblast) length and our predictor is Vitamin C Supplement (Orange Juice vs Ascorbic Acid). In this example, we want to know if the difference in the population means of tooth length across vitamin C Supplement (Orange Juice vs Ascorbic Acid) is negligible (i.e., are the population means of tooth length across both groups are equivalent).
len supp dose
1 4.2 VC 0.5
2 11.5 VC 0.5
3 7.3 VC 0.5
4 5.8 VC 0.5
5 6.4 VC 0.5
6 10.0 VC 0.5# Let's compare Guinea pig tooth (odontoblast) length across two groups:
# Group 1) Took Vitamin C via Orange Juice
# Group 2) Took Vitamin C via Ascorbic Acid
# IV = supp (Vitamin C Supplement)
# DV = len (Tooth length)
# Negligible Effect Interval: {-3, 3}Now, let’s apply the neg.twoindmeans function to see if we can reject the null hypothesis that the two independent sample means are non-negligibly different.
library(negligible)
set.seed(1000)
neg.twoindmeans(iv=supp,dv=len,data=ToothGrowth,eiL=-3,eiU=3) # run neg.twoindmeans using the vectors supp and len for the iv and dv arguments, respectively---- Equivalence of Two Independent Groups----
Test Statistic:
Schuirmann-Yuen Test of the Equivalence of Two Independent Groups
**********************
Levels of the IV: OJ, VC
Group Means: 20.66333, 16.96333
Group Trimmed Means: 21.70556, 16.58333
Group SDs: 6.605561, 8.266029
Group MADs: 5.48562, 9.26625
**********************
Standardized Mean Difference (SMD): 0.6393216
95% CI for SMD: (0.08963287, 1.203361)
**********************
Equivalence Interval: Lower = -3 , Upper = 3
**********************
Mean Difference (MD): 5.122222
90% CI for MD: (0.7988889, 8.556667)
95% CI for MD: (0.3605556, 9.428333)
**********************
Proportional Distance (PD): 1.707407
95% CI for PD: (0.1201852, 3.142778)
**********************
TOST Test Statistics:
Ho: mu1-mu2>=eiU:
t = 0.9476839 (df = 33.49113), p = 0.82496
Ho: mu1-mu2<=eiL:
t = 3.627 (df = 33.49113), p = 0.00047
**********************
NHST Decision:
The null hypothesis that the difference between the means exceeds the equivalence interval cannot be rejected. A negligible difference in means cannot be concluded. Be sure to interpret the magnitude of the effect size.
**********************
Example 2
In the next example, we are calling the variables from the dataframe ToothGrowth.
set.seed(1000)
neg.twoindmeans(iv=ToothGrowth$supp,dv=ToothGrowth$len,eiL=-3,eiU=3) # run neg.twoindmeans omitting the data argument and instead calling the dataframe$variable as the arguments for iv and dv (e.g., iv=ToothGrowth$supp,dv=ToothGrowth$len)---- Equivalence of Two Independent Groups----
Test Statistic:
Schuirmann-Yuen Test of the Equivalence of Two Independent Groups
**********************
Levels of the IV: OJ, VC
Group Means: 20.66333, 16.96333
Group Trimmed Means: 21.70556, 16.58333
Group SDs: 6.605561, 8.266029
Group MADs: 5.48562, 9.26625
**********************
Standardized Mean Difference (SMD): 0.6393216
95% CI for SMD: (0.08963287, 1.203361)
**********************
Equivalence Interval: Lower = -3 , Upper = 3
**********************
Mean Difference (MD): 5.122222
90% CI for MD: (0.7988889, 8.556667)
95% CI for MD: (0.3605556, 9.428333)
**********************
Proportional Distance (PD): 1.707407
95% CI for PD: (0.1201852, 3.142778)
**********************
TOST Test Statistics:
Ho: mu1-mu2>=eiU:
t = 0.9476839 (df = 33.49113), p = 0.82496
Ho: mu1-mu2<=eiL:
t = 3.627 (df = 33.49113), p = 0.00047
**********************
NHST Decision:
The null hypothesis that the difference between the means exceeds the equivalence interval cannot be rejected. A negligible difference in means cannot be concluded. Be sure to interpret the magnitude of the effect size.
**********************
Interpretation of the Results
Note that because ‘varequiv=FALSE’ and ‘normality=FALSE’, the test produced is the Schuirmann-Yuen test (a negiglble effect test based on the Welch test with trimmed means); this test is robust to violations of the normality and variance homogeneity assumptions. Because of this, you will notice that the df is actually not a whole number since it is computed from a more complex formula than n1+n2-2. See:
van Wieringen K, & Cribbie R.A. (2014). Evaluating clinical significance: Incorporating robust statistics with normative comparison tests. British Journal of Mathematical and Statistical Psychology, 67(2), 213-30. doi: https://doi.org/10.1111/bmsp.12015.
The populations were not statistically equivalent on the measure of tooth length with EI = {-3, 3} and a mean difference of 5.12; in other words, the 100(1-2*alpha) CI on the mean difference is not completely contained within the EI.
Interpretation of the Proportional Distance (PD)
The PD for this test is 1.71 [(5.12-0)/|3-0|)] with a 95% CI of [0.12, 3.21]. This suggests that the observed mean difference of 5.12 is approximately 171% of the distance away from 0, beyond the upper bound. For more information regarding the PD see: https://osf.io/preprints/psyarxiv/5buz9/
Extractable Elements
A number of elements of the output can be extracted, including:
meanx: Sample mean of the first population/group.
meany: Sample mean of the second population/group.
trmeanx: Sample trimmed mean of the first population/group.
trmeany: Sample trimmed mean of the second population/group.
sdx: Sample standard deviation of the first population/group.
sdy: Sample standard deviation of the second population/group.
madx: Sample median absolute deviation of the first population/group.
mady: Sample median absolute deviation of the second population/group.
eiL: Lower bound of the negligible effect (equivalence) interval.
eiU: Upper bound of the negligible effect (equivalence) interval.
effsizeraw: Simple difference in the means (or medians if normality = FALSE)
cilraw2: Lower bound of the 1-alpha CI for the raw mean difference.
ciuraw2: Upper bound of the 1-alpha CI for the raw mean difference.
cilraw: Lower bound of the 1-2*alpha CI for the raw mean difference.
ciuraw: Upper bound of the 1-2*alpha CI for the raw mean difference.
effsized: Standardized mean (or median if normality = FALSE) difference.
cild: Lower bound of the 1-alpha CI for the standardized mean (or median if normality = FALSE) difference.
ciud: Upper bound of the 1-alpha CI for the standardized mean (or median if normality = FALSE) difference.
effsizepd: Proportional distance statistic.
cilpd: Lower bound of the 1-alpha CI for the proportional distance statistic.
ciupd: Upper bound of the 1-alpha CI for the proportional distance statistic.
t1: First t-statistic from the TOST procedure.
t2: Second t-statistic from the TOST procedure.
df1: Degrees of freedom for the first t-statistic from the TOST procedure.
df2: Degrees of freedom for the second t-statistic from the TOST procedure.
p1: p value associated with the first t-statistic from the TOST procedure.
p2: p value associated with the second t-statistic from the TOST procedure.
alpha: Nominal Type I error rate