Start Using the neg.paired()
Function
From the negligible
R package
Introduction
What is the purpose/goal of
neg.paired()?
The purpose of the neg.paired function is to test whether the difference between the means of two dependent populations is negligibly different (i.e., practically equivalent), where negligible represents the smallest meaningful effect size (MMES) determined a priori by a researcher via an equivalence interval (EI). For example, if a researcher wanted to test whether the GPA of students before vs after a year of implementing a novel note-taking method are approximately equivalent (e.g., paired group mean similarity), they could use the neg.paired function.
What is the theory behind
neg.paired()?
The user specifies either the data associated with the first and second groups/populations (var1, var2, both should be continuous) or specifies the Independent Variable/Predictor (group, should be a factor) and the Dependent Variable (outcome, should be continuous). A ‘data’ statement can be used if the variables are stored in an R dataset.
The user must also specify the lower and upper bounds of the negligible effect (equivalence) interval. These are specified in the original units of the outcome variable.
Null and Alternate Hypotheses of the Procedure
\(H_{01}\): \(\mu_{1}\)- \(\mu_{2}\) \(\geqslant\) \(\delta\); \(H_{02}\): \(\mu_{1}\)- \(\mu_{2}\) \(\leqslant\) -\(\delta\) (The pair of null hypotheses specify that the difference between the dependent means in the population exceed the equivalence interval; i.e., that the effect is nonnegligible, either at the upper bound or the lower bound).
\(H_{a1}\): \(\mu_{1}\)- \(\mu_{2}\) \(<\) \(\delta\); \(H_{a2}\): \(\mu_{1}\)- \(\mu_{2}\) \(>\) -\(\delta\) (the pair of alternative hypotheses specifies that the difference between the dependent means in the population does not exceed the equivalence interval; i.e., it is negligible at either the upper or lower bound. In other words, that the effect is completely contained within the equivalence interval).
Using neg.paired()
neg.paired(var1 = NULL, var2 = NULL, outcome = NULL, group = NULL, ID = NULL, eil, eiu, normality = TRUE, nboot = 10000, alpha = 0.05, plot = TRUE, saveplot = FALSE, data = NULL, seed = NA, …)
Function Set-up:
Required arguments (no default)
eil: the lower bound of the equivalence interval
eiu: the upper bound of the equivalence interval
Optional arguments (has a default)
var1: data for Group 1 (if this is the outcome variable, group and ID are omitted)
var2: data for Group 2 (if this is the outcome, group and ID are omitted)
outcome: the dependent Variable (if var1 and var2 are omitted)
group: dichotomous predictor/independent Variable (if var1 and var2 are omitted)
ID: participant ID (if var1 and var2 are omitted)
data: the dataset containing var1/var2 or outcome/group/ID
normality: are the population variances (and hence the residuals) assumed to be normally distributed?
nboot: the number of bootstrap samples for calculating CIs
alpha: Nominal Type I Error rate
plot: should a plot of the results be produced?
saveplot: should the plot be saved?
seed: seed number for reproducing results from simulated data
Examples
Example 1
This example uses simulated wide-form data. We want to know whether the mean GPA (GPApre) at the beginning of the academic year is equivalent to the mean GPA at the end of the academic year (GPApost).
library(negligible)
#simulate data
ID<-rep(1:20) # create identifiers ranging from 1 to 20 and store in a vector ID
GPApre<-rnorm(20) # simulate 20 cases that are random normally distributed to represent GPA at time 1 and store in a vector GPApre
GPApost<-rnorm(20) # simulate 20 cases that are random normally distributed to represent GPA at time 2 and store in a vector GPApost
d<-data.frame(ID, GPApre, GPApost) # save ID, GPApre, and GPApost in a data frame, d
head(d) # examine the first 6 rows of the dataframe ID GPApre GPApost
1 1 -0.1034926 1.8015927
2 2 1.2810380 0.7529291
3 3 0.4379488 1.1174428
4 4 -1.9598908 0.5866857
5 5 -0.8127425 -1.4895469
6 6 1.9666859 -0.1400081Now, let’s apply the neg.paired function to see if we can reject the null hypothesis that the two dependent sample means are non-negligibly different.
neg.paired(var1=GPApre,var2=GPApost,eil=-1,eiu=1,plot=TRUE,
data=d) # run neg.paired using the vectors GPApre and GPApost for the var1 and var2 arguments, respectively---- Evaluating the Equivalence of the Central Tendencies of Paired Samples----
Random Seed = 7590
Test Statistic:
Two One-Sided Test of Equivalence for Paired-Samples (TOST-P)
**********************
Means: 0.05362021, 0.1658774
Medians: 0.2013027, 0.2912124
SDs: 1.09474, 1.087204
MADs: 0.5959531, 1.042148
**********************
Equivalence Interval: Lower = -1 , Upper = 1
**********************
Effect Sizes: Effect Size Measures use the Ordinary Mean and Standard Deviation
Standardized Mean Difference (SMD): -0.0948182
90% CI for SMD:(-0.6445298, 0.4246743)
Raw Mean Difference (MD): -0.1122571
90% CI for MD:(-0.6617145, 0.4407505)
95% CI for MD:(-0.7707042, 0.5488483)
**********************
Proportional Distance (PD): -0.1122571
95% CI for PD:(-0.7707042, 0.5488483)
**********************
TOST Test Statistics:
Ho: μ1 - μ2 ≥ eiu:
t = -4.095055 (df = 19), p = 0.00031
Ho: μ1 - μ2 ≤ eil:
t = 3.268449 (df = 19), p = 0.00202
**********************
NHST Decision:
The null hypothesis that the difference between the means exceeds the equivalence interval can be rejected. Be sure to interpret the magnitude of the effect size.
**********************
Example 2
In the next example, we are calling the variables from the dataframe d.
neg.paired(var1=d$GPApre,var2=d$GPApost,eil=-1,eiu=1,plot=TRUE) # run neg.paired omitting the data= argumetn and instead calling the dataframe$variable as the arguments for var1 and var2 (e.g., var1=d$GPApre, var2=d$GPApost)---- Evaluating the Equivalence of the Central Tendencies of Paired Samples----
Random Seed = 5694
Test Statistic:
Two One-Sided Test of Equivalence for Paired-Samples (TOST-P)
**********************
Means: 0.05362021, 0.1658774
Medians: 0.2013027, 0.2912124
SDs: 1.09474, 1.087204
MADs: 0.5959531, 1.042148
**********************
Equivalence Interval: Lower = -1 , Upper = 1
**********************
Effect Sizes: Effect Size Measures use the Ordinary Mean and Standard Deviation
Standardized Mean Difference (SMD): -0.0948182
90% CI for SMD:(-0.6562735, 0.4282284)
Raw Mean Difference (MD): -0.1122571
90% CI for MD:(-0.6726862, 0.4475355)
95% CI for MD:(-0.7755828, 0.5496184)
**********************
Proportional Distance (PD): -0.1122571
95% CI for PD:(-0.7755828, 0.5496184)
**********************
TOST Test Statistics:
Ho: μ1 - μ2 ≥ eiu:
t = -4.095055 (df = 19), p = 0.00031
Ho: μ1 - μ2 ≤ eil:
t = 3.268449 (df = 19), p = 0.00202
**********************
NHST Decision:
The null hypothesis that the difference between the means exceeds the equivalence interval can be rejected. Be sure to interpret the magnitude of the effect size.
**********************
Extractable Elements
A number of elements of the output can be extracted, including:
meanx: Sample mean of the first population/group.
meany: Sample mean of the second population/group.
medx: Sample median of the first population/group.
medy: Sample median second population/group.
sdx: Sample standard deviation of the first population/group.
sdy: Sample standard deviation of the second population/group.
madx: Sample median absolute deviation of the first population/group.
mady: Sample median absolute deviation of the second population/group.
eil: Lower bound of the negligible effect (equivalence) interval.
eiu: Upper bound of the negligible effect (equivalence) interval.
effsizeraw: Simple difference in the means (or medians if normality = FALSE)
cilraw2: Lower bound of the 1-alpha CI for the raw mean difference.
ciuraw2: Upper bound of the 1-alpha CI for the raw mean difference.
cilraw: Lower bound of the 1-2*alpha CI for the raw mean difference.
ciuraw: Upper bound of the 1-2*alpha CI for the raw mean difference.
effsized: Standardized mean (or median if normality = FALSE) difference.
cild: Lower bound of the 1-alpha CI for the standardized mean (or median if normality = FALSE) difference.
ciud: Upper bound of the 1-alpha CI for the standardized mean (or median if normality = FALSE) difference.
effsizepd: Proportional distance statistic.
cilpd: Lower bound of the 1-alpha CI for the proportional distance statistic.
ciupd: Upper bound of the 1-alpha CI for the proportional distance statistic.
t1: First t-statistic from the TOST procedure.
t1: Second t-statistic from the TOST procedure.
df1: Degrees of freedom for the first t-statistic from the TOST procedure.
df2: Degrees of freedom for the second t-statistic from the TOST procedure.
pval1: p value associated with the first t-statistic from the TOST procedure.
pval2: p value associated with the second t-statistic from the TOST procedure.
alpha: Nominal Type I error rate
seed: Seed number