Start Using the neg.cor()
Function
From the negligible
R package
Introduction
What is the purpose/goal of
neg.cor()?
The function neg.cor can help us determine whether two variables have a negligible association with one another. To see how it’s used, let’s first load the negligible package (if you don’t have it installed, you will have to install it first). Let’s load the psych package too, for some descriptives.
What is the theory behind
neg.cor()?
The function will compare the confidence interval around a correlation to something called an equivalence interval. If the confidence interval around the correlation is completely contained within the equivalence interval. we have evidence to say that the correlation between the variables is so small that we can call it unimportant, meaningless, etc.
We are going to be using the dataset mtcars that is already pre-loaded in R. We can investigate whether the weight of a car (in 1000lbs) and its 1/4 mile time are negligibly correlated with one another. Let’s take some descriptives (make sure nothing looks odd, the variables are continuous, etc.) and plot their correlation.
vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 32 3.22 0.98 3.33 3.15 0.77 1.51 5.42 3.91 0.42 -0.02 0.17 vars n mean sd median trimmed mad min max range skew kurtosis se
X1 1 32 17.85 1.79 17.71 17.83 1.42 14.5 22.9 8.4 0.37 0.34 0.32
Null and Alternate Hypotheses of the Procedure
\(H_{01}: eiL \ge \rho\) | \(H_{02}: eiU \le \rho\)
\(H_{11}: eiL < \rho\) | \(H_{12}: eiU > \rho\)
Using neg.cor()
Now let’s use the function. To do this, we will need to set an equivalence interval (in our function, this is going to be using the eiL and eiU arguments). This is going to be the smallest correlation we would consider to be important. Ideally, this would be informed by theory, but for our first example below, let’s go with 0.4. This means that anything larger than 0.4 is going to be too important of an association to deem negligible. Further, this also means that anything smaller than -0.4 is going to be too important of an association to be deemed negligible. The arguments we need are as follows:
Required arguments (no default)
v1: the first variable we are interested in comparing,
v2: the second variable we are interested in comparing,
eiU: the top of our equivalence interval (0.4 in our first example),
eiL: the bottom of our equivalence interval (-0.4 in our first example),
Optional arguments (has a default)
alpha: the optional argument for the alpha level (.05),
data: the optional argument for the data frame, and
seed: the optional argument so that you can reproduce the results (for all of the examples, we will go with seed = 1, so if you rerun this, you should get the same values in your output).
Examples
Example 1
---- Equivalence Based Test of Lack of Association with Resampling ----
Random Seed = 1
Pearson's r: -0.1747159
Equivalence Interval: Lower = -0.4 , Upper = 0.4
# of Resamples: 10000
Bootstrapped 90% Confidence Interval (5th Percentile, 95th Percentile):(-0.354694,-0.01043371)
The null hypothesis that the correlation between v1 and v2 falls outside of the equivalence interval can be rejected. A negligible association CAN be concluded. Be sure to interpret the magnitude (and precision) of the effect size.
**********************
Proportional Distance (PD): -0.4367897
0.95 % CI for PD: ( -0.9971597 , 0.04958426 )
**********************
Let’s interpet the output. -0.17 is the correlation between weight of
the car and time to 1/4 mile. The equivalence interval is -0.4 to 0.4 as
we had defined before. The confidence interval (computed through 1000
boostraps) is -0.35 to -0.01. Note the relationship between each bound
of the confidence interval and the corresponding bound of the
equivalence interval: the upper bound of the confidence interval is
within the equivalence interval (-.01 is smaller than 0.40) and the
lower bound of the confidence interval is within the equivalence
interval (-0.35 is larger than -0.4). Because the confidence interval is
completely contained within the equivalence interval, we can reject the
null hypothesis that the variables are not negligibly associated (i.e.,
we have evidence of a negligible association). We can see this is stated
in the text output as well is in the second part of the plot where the
confidence interval is within the two * symbols denoting the equivalence
interval.
We also get some output for the proportional distance, an effect size used for equivalence testing. Our proportional distance is -0.44 with a 95% confidence interval of (-1.00, 0.05). This suggests that the observed correlation of -.17 is at approximately 44% of the distance away from 0 to the lower bound.
Example 2
There were multiple ways we could have run this function. Let’s rerun it, but this time, we won’t use the $ and instead put variable names into v1 and v2, the dataset name will go into the data argument (that we did not use previously). The results below are identical to what we had above (note this is also because we set the same seed).
---- Equivalence Based Test of Lack of Association with Resampling ----
Random Seed = 1
Pearson's r: -0.1747159
Equivalence Interval: Lower = -0.4 , Upper = 0.4
# of Resamples: 10000
Bootstrapped 90% Confidence Interval (5th Percentile, 95th Percentile):(-0.354694,-0.01043371)
The null hypothesis that the correlation between v1 and v2 falls outside of the equivalence interval can be rejected. A negligible association CAN be concluded. Be sure to interpret the magnitude (and precision) of the effect size.
**********************
Proportional Distance (PD): -0.4367897
0.95 % CI for PD: ( -0.9971597 , 0.04958426 )
**********************
Example 3
For completeness, let’s demonstrate what a failure to detect a negligible association would look like. Let’s change our equivalence interval but keep everything else the same as we had it before. Let’s make our equivalence interval (-0.3, 0.3) instead of (-0.4, 0.4). Everything else will stay the same.
---- Equivalence Based Test of Lack of Association with Resampling ----
Random Seed = 1
Pearson's r: -0.1747159
Equivalence Interval: Lower = -0.3 , Upper = 0.3
# of Resamples: 10000
Bootstrapped 90% Confidence Interval (5th Percentile, 95th Percentile):(-0.354694,-0.01043371)
The null hypothesis that the correlation between v1 and v2 falls outside of the equivalence interval cannot be rejected. A negligible association CANNOT be concluded. Be sure to interpret the magnitude (and precision) of the effect size.
**********************
Proportional Distance (PD): -0.5823863
0.95 % CI for PD: ( -1.329546 , 0.06611235 )
**********************
Let’s look at the output. Our correlation between time to 1/4 mile and weight hasn’t changed (this makes sense because we are still looking at the relationship between the same two variables) and neither has our 90% bootstrapped confidence interval (we might have seen some slight changes in the confidence interval had we not kept our seed set to 1).
Our test decision, however, is different than before. Even though the upper bound of the confidence interval (-0.01) is within the equivalence interval, the lower bound of the confidence interval (-0.35) is not (-0.35 is smaller than -0.30).
For an illustration of this, look at the figure output above. In the bottom part of the figure, you can see that the -0.35 is outside of the range defined by the equivalence interval (the equivalence interval is defined by the stars, *). Because we need both parts of the confidence interval to be completely contained in the equivalence interval, we fail to reject the null hypothesis and fail to detect a negligible association, as can be seen in the decision.
The proportional distance is also different here, because the equivalence interval has changed.
Extractable Elements
A number of elements of the output can be extracted, including:
corxy The correlation between the two variables
q1 Lower bound of the confidence interval for the correlation
q2 Upper bound of the confidence interval for the correlation
eiL Lower bound of the negligible effect (equivalence) interval
eiU Upper bound of the negligible effect (equivalence) interval
decis_rs NHST decision
PD Proportional distance
CIPLL Lower bound of the 1-alpha CI for the PD
CIPDU Upper bound of the 1-alpha CI for the PD
alpha Nominal Type I error rate