Comparing Correlation Values

In our meeting you asked whether correlation values can be compared statistically. For example, in your Figure 2 of your results you show a very slight negative correlation between BDI-II and mean accuracy in the rule-based condition (r(172) = -0.05 in my analysis), and a very weak correlation between BDI-II and mean accuracy in the information-integration condition (r(172) = 0.003 in my analysis).

Your question to me was whether we can test if the correlation of r = -0.05 is significantly different from a correlation of 0.003. You can do this using a test called Fisher’s z-transformation.

Fisher’s Z-Transformation

This statistic allows you to perform a test of significance for the difference between two correlation values. In our experiment, each correlation comes from the same group of participants, so the test is based on dependent groups.

We want to test the difference between two correlations. For example, for BDI and mean accuracy, we want to compare the following correlations

  • The correlation between BDI and RB-accuracy
  • The correlation between BDI and II-accuracy.

These two correlations are said to be overlapping, because they each share one variable (i.e, BDI). To compare these two correlations, we need four pieces of information:

  • The correlation coefficient (i.e., the r-value) between BDI and RB-accuracy
  • The correlation coefficient between BDI and II-accuracy
  • The correlation coefficient between RB-accuracy and II-accuracy
  • The number of participants.

Once we have these three pieces of information, we can run our test. In your data, you already have reported the first two correlation coefficients: They are -0.05 and 0.003, respectively. We just now need to find the correlation between RB-accuracy and II-accuracy (spoiler alert: It’s r = 0.255).

Doing the Test in R

Here’s how to do the test in R using the cocor package. (Note you will need to install it!)

library(cocor)

# declare the required values. 
# Note I have just typed these in manually, so it can be prone
# to error.
n_subjects <- 173
cor_bdi_rb <- -0.05
cor_bdi_ii <- 0.003
cor_rb_ii <- 0.255

# perform the correlation comparison test
bdi_accuracy_comparison <- cocor.dep.groups.overlap(r.jk = cor_bdi_rb, 
                                                    r.jh = cor_bdi_ii, 
                                                    r.kh = cor_rb_ii, 
                                                    n = n_subjects)

This stores quite a lot of information in the bdi_accuracy_comparison object. In order to extract the relevant statistics out, we do the following:

# extract the value for Fisher's Z
z_bdi_accuracy <- bdi_accuracy_comparison@pearson1898$statistic

# extract the p-value for this test
p_bdi_accuracy <- bdi_accuracy_comparison@pearson1898$p.value
z_bdi_accuracy
[1] -0.5718351
p_bdi_accuracy
[1] 0.5674337

If the p-value is below 0.05, you can state that the correlations are significantly different from each other.

Reporting the Result

Once you have your test result, you will want to report it. I’ve flagged in your draft results where you would add this reporting.

Fisher’s Z-Transformation test showed that these correlations were not signficantly different from each other, Z = -0.57, p = .567.