Summary

In this demo, I use a published dataset of 13 studies examining the relation between BCG vaccine and Tuberculosis test outcomes. The data are aggregated and an average size effect, log relative risk, is calculated. Then I fit three different models to examin effects of different study-level variables on the study outcome followed by some diagnostic tests.

Methods

Dataset and Code

The dataset is originally from (Berkey et al. 1995) and is accessed through the pachake metafor (Viechtbauer 2010). It consist of 13 studies, for each study it records the author(s), entries of a 2x2 table of the study participients, the location, the allocation method and the year the study was coducted. Diagram 1 shows the entries of the 2 x 2 table and the corresponding clinical meaning.

Statistical Methods

The log relative risk, as an outcome, is first calculated for each study. Three different models are used; a random-effect model is first fit to obtain an average relative risk of the 13 studies. Then, two mixed-effect models (Hedges LV 1985) are used to explore the potential effect of the study location and the allocation method on the outcome.

Software

This demo study is conducted in R environment (R Core Team 2016). Package metafor (Viechtbauer 2010) is used mainly to obtain the data and fit the models. R markdown (Xie 2016) is used to generate this report.

Results

Vaccination with BCG reduces relative risk of tuberculosis

The aggregated data of 13 studies show reduced relative risk of tuberculosis in the vaccinated group compared to the non-vaccinated, 0.49 on average with a 95% confidence interval [ 0.34 to 0.7 ]. 92.22% of the variability in the estimates is explained by the hererogeneity of the studies. This is confirmed by using Cochrane Q test for hetergeneity (Cochran 1985) which tested significant, p-value < 0.0001.

Figure 1 Forst graph of average and individual relative risk of the 13 studies. Relative risk estimates with 95% confidence intervals are listed along with a visual presentation. Squares on the left side of the dashed line mean lower relative risk, with the horizontal lines covering the corresponding intervals. Numbers of individuals tested positive or negative are listed along with the corresponding study by author(s) and year of publicatioin.

Vaccination with BCG is less beneficial in areas near the equator

Here I fit a mixed-effects model using the year the study was conducted and the location in terms of the absolute latitude as potential variables to explain the variability. The new model account for a big portion of the heterogeniety, 64.63 %, the rest could be due to other variables not included in the data.

The location of the study seems to affect the outcomes with -0.03 decrease in the log relative risk with every degree of latitude getting near the equator and with a 95 % confidence interval -0.05 to -0.01. However, the year of the studies doesn’t seem to affect the outcomes.

Figure 2 Relative risk decrease with the increased absolute latitued. The plot showes absolute latitude on the x-axis and the relative risk on the y-axis. Points represent individual studies sized by the weight (the inverse of the sampling variablility), the solid line represents the linear trend and the two dashed lines represents the bounderies of the 95 % confidence interval.

The allocation method doesn’t affect the trials outcomes

We fit a third model to examine the effect of the allocation method; random, systematic or alternative on the study outcome. The allocation method seems not to affect the relative risk estimates and both residual heterogeneity (Q test for residual heterogeneity p-value is 0) and coefficients test confirmed so (Q test for moderators p-value is 0.41).

Diagnostics

Publication Bias

To examin the possibel bias in the publications we plot the standard error against the log relative risk for the random-effects models, left panal and the standard error against the residuals for the mixed-effects model, right panal. We find a less acurrate estimates of the random-effects model as shown by the dispersion of the points in the graph and confirmed by the funnel plot assymetry test, p-value 0.19. While the mixed-effects model accounts better for the variability as shown by the symmetry of the residuals around the center.

Figure 3 Diagnostic plots of random and mixed-effects models. Two funnel plots showing the standared error on the y-axis and the log relative risk, left, and the residual value, right, on the x-axis. Points represent the individual studies.

Influence of individual studies

Left panel, showes that the amount of heterogeneity when fitting the mixed-effects model with deleting each study once (Belsley, Kuh, and Welsch 1980). Study 4 seems to have a little effect of the heterogenity of the model while studies 7 and 13 have a bigger effect. However, cook’s distance (Cook and Weisberg 1982) plot, right panel, showes that only study 4 has an influence on the model fit.

Figure 4 Influence of individual studies on the heterogeneity and the model fitting. The plots show individual studies on the x-axis and the tau^2 value when, left, and the distance, right, of the mixed-effect models. Dashed lines represent the average values of each parameter.

Conclusion

To sum, the aggregated data drawn from 13 separet studies show that individuals vaccinated by BCG have less risk of testing positive for tuberculosis. Studies conducted in areas near the equator showed weaker relation between the vaccine and the test while the year and the allocation methods of the studies don’t affect the outcomes.

References

Belsley, David A., Edwin Kuh, and Roy E. Welsch. 1980. Regression Diagnostics. Wiley Series in Probability and Statistics. Hoboken, NJ, USA: John Wiley & Sons, Inc. doi:10.1002/0471725153.

Berkey, C S, D C Hoaglin, F Mosteller, and G A Colditz. 1995. “A random-effects regression model for meta-analysis.” Statistics in Medicine 14 (4): 395–411. http://www.ncbi.nlm.nih.gov/pubmed/7746979.

Cochran, WG. 1985. “Some Methods for Strengthening the Common \(\chi\)2 Tests.” Biometrics 10 (4): 417–51.

Cook, R. Dennis., and Sanford Weisberg. 1982. Residuals and influence in regression. Chapman; Hall.

Hedges LV, Olkin I. 1985. Statistical Methods for Meta-Analysis. San Diego, CA: Academic Press.

R Core Team. 2016. “R: A language and environment for statistical computing.” Vienna, Austria: R Foundation for Statistical Computing. https://www.r-project.org/.

Viechtbauer, Wolfgang. 2010. “Conducting Meta-Analyses in R with the metafor Package.” JSS Journal of Statistical Software 36 (3). http://www.jstatsoft.org/.

Xie, Yihui. 2016. “knitr: A General-Purpose Package for Dynamic Report Generation in R.”