csp-770-meta-analysis

1. Calculate a summary (global, aggregated) effect size across multiple studies of the same phenomenon
2. Estimate the precision (heterogeneity) of the aggregated effect
   2.1 - If the various ES vary a lot, it is a sign they may be not be comparable.
3. If there is a high level heterogeneity
   3.1 Attempt to characterize source of heterogeniety
   3.2 Covariates may explain the heterogeneity
   

Choice of ES depends on research question

• For each study you will need
  1. Sample size of each group (study)
  2. The average change of each group
  3. The standard deviation of change scores for each group

• Depending on the package, you calculate standardized effect sizes and then analyze or provide just the raw data

• When you have multiple related effects in a study you can aggregate taking their correlation into account

  • See Borenstein, Hedges, Higgins, & Rothstein (BHHR; 2009)
  • The MAd package provides the function agg to do this

• The summary effect is a weighted average of all effects
  • The weights are the inverse of the ES variance across studies
    
    • Variance is a largely a function of sample size
    • Thus larger studies make larger contributions to the effect
• The summary effect can be calculated as a:
  • fixed effect: Results do not generalize beyond the sample of studies
  • random effect: Results do generalize to all such studies

Some heterogeneity is to be expected, how much is too much?

• Q is the chi-squared statistic evaluating chance variation
• p < .0x indicates that variance of ESs is not random
• \( I^2 \) is a measure of the variance associated with heterogeneity
  • 0% to 40%: may not be important
  • 30% to 60%: may represent problematic heterogeneity
  • 50% to 90%: may represent significant problems
  • 75% to 100%: considerable heterogeneity

Fit a fixed effects model

Fixed effects ( Mantel-Haenszel ) meta-analysis
Call: meta.MH(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, 
    names = name, data = cochrane)
------------------------------------
               OR (lower  95% upper)
Auckland     0.58    0.38       0.89
Block        0.16    0.02       1.45
Doran        0.25    0.07       0.81
Gamsu        0.70    0.34       1.45
Morrison     0.35    0.09       1.41
Papageorgiou 0.14    0.02       1.16
Tauesch      1.02    0.37       2.77
------------------------------------
Mantel-Haenszel OR =0.53 95% CI ( 0.39,0.73 )
Test for heterogeneity: X^2( 6 ) = 6.9 ( p-value 0.3303 )

Fit a random effects model

Random effects ( DerSimonian-Laird ) meta-analysis
Call: meta.DSL(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, 
    names = name, data = cochrane)
------------------------------------
               OR (lower  95% upper)
Auckland     0.58    0.38       0.89
Block        0.16    0.02       1.45
Doran        0.25    0.07       0.81
Gamsu        0.70    0.34       1.45
Morrison     0.35    0.09       1.41
Papageorgiou 0.14    0.02       1.16
Tauesch      1.02    0.37       2.77
------------------------------------
SummaryOR= 0.53  95% CI ( 0.37,0.78 )
Test for heterogeneity: X^2( 6 ) = 6.86 ( p-value 0.334 )
Estimated random effects variance: 0.03 

What is the difference between the fixed and random effects model?

Mean Change in two-condition RCT

Mean Change in two-condition RCTs

Results of Meta-Analysis

Funnel Plot

Forrest Plot

Test of Funnel Plot Assymetry

Linear regression test of funnel plot asymmetry

t = -0.337, df = 6, p = 0.747

Bias Estimate

	bias	se	slope
bias	-0.64	1.9	0.57

The Study Data

Results of Meta-Analysis (without covariates)

Funnel plot with heterogeneity

Results of Meta-Analysis (WITH covariates)

Control for different doses

[1] "Tau^2 = 0.000"

[1] "I^2:  = 0.000"

[1] "H^2:  = 1.000"

[1] "Test for Residual Heterogeneity"

[1] "Q = 5.22, d.f = 6, p-val = 0.5160"

[1] "Test for Moderators Heterogeneity"

[1] "Q = 19.39, d.f = 1, p-val = 0.0000"

Control for different doses