Data analysis for Hayden
This meta-analysis output data was generated by Katie Robinson. The analysis includes 344 effect sizes
Overall effect size and a description of key statistical measures (e.g., I² statistic, Egger’s test) and their relevance in assessing heterogeneity and publication bias.
Metaregression to explore and quantify the relationship between moderators and effect sizes.
#Data table
Meta-analysis in METAFOR
This is a multilevel analysis of the OVERALL effects. It provides a comparison to check against later. We will not use these results.
##
## Multivariate Meta-Analysis Model (k = 344; method: REML)
##
## logLik Deviance AIC BIC AICc
## -438.3953 876.7906 882.7906 894.3038 882.8614
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.6857 0.8281 72 no study_id
## sigma^2.2 0.3130 0.5595 344 no study_id/es_id
##
## Test for Heterogeneity:
## Q(df = 343) = 3471.1484, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## 0.6386 0.1070 5.9711 <.0001 0.4290 0.8483 ***
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Overall \(I^2\)
The \(I^2\) statistic measures the percentage of variation across studies in a meta-analysis that is due to heterogeneity rather than chance. It helps assess the extent to which the results of the studies are consistent, with higher values indicating greater variability in study outcomes.
OR
\(I^2\) is the percentage of variability in the effect sizes which is not caused by sampling error.
## [1] 95.39122Between studies I2 and between effect sizes E1
- The first value represents the between-study heterogeneity (I²_between), which quantifies how much of the variation in effect sizes is due to differences between studies.
- The second value represents the within-study (or within-cluster) heterogeneity (I²_within), which quantifies variability within studies or clusters of effect sizes.
## [1] 65.49438 29.89684Publication bias
Results that are statistically non-significant or unfavourable are less likely to be published than statistically significant results, and hence are less easily identified by systematic reviews This leads to syntheses over-estimating or under-estimating the true effects.
Eggers regression result
Egger’s test assesses the presence of publication bias in a meta-analysis by evaluating the asymmetry of the funnel plot.
##
## Regression Test for Funnel Plot Asymmetry
##
## Model: mixed-effects meta-regression model
## Predictor: standard error
##
## Test for Funnel Plot Asymmetry: z = 12.3208, p < .0001
## Limit Estimate (as sei -> 0): b = -0.6481 (CI: -0.8781, -0.4181)Fail safe N
##
## Fail-safe N Calculation Using the Rosenthal Approach
##
## Observed Significance Level: <.0001
## Target Significance Level: 0.05
##
## Fail-safe N: 185829Funnel plot
MetaSEM overall results
This is where we combined effect sizes using a structural equation modelling approach to multilevel meta-analysis. The main advantage of this approach is that it is not limited by the assumption of independence (i.e., effect sizes are nested within studies), and multiple effect sizes can be included from each study. Unconditional mixed-effects models using maximum likelihood estimation were conducted to calculate the overall pooled effect size. For each pooled effect size, 95% likelihood-based CIs were calculated. All analyses were conducted using the metaSEM package in R Version 2024.12.1+563.
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, data = data)
##
## 95% confidence intervals: z statistic approximation (robust=FALSE)
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Intercept 0.638416 0.106138 0.430389 0.846443 6.0150 1.799e-09 ***
## Tau2_2 0.312811 0.039053 0.236268 0.389355 8.0098 1.110e-15 ***
## Tau2_3 0.672387 0.145024 0.388145 0.956629 4.6364 3.546e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Heterogeneity indices (based on the estimated Tau2):
## Estimate
## I2_2 (Typical v: Q statistic) 0.3027
## I2_3 (Typical v: Q statistic) 0.6506
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 3
## Degrees of freedom: 341
## -2 log likelihood: 879.9901
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Meta regression results
Study design
This works out how many studies in each variable
## # A tibble: 2 × 2
## study_design distinct_points
## <dbl> <int>
## 1 1 36
## 2 2 36This works out how many effect sizes in each variable
1 = RCT, 2 = Quasiexperimental
##
## 1 2
## 143 201This performs the meta regression in study design
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(rct,
## quasi), data = data, intercept.constraints = 0, intervals.type = "LB",
## I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.41606 NA 0.12727 0.70557 NA NA
## Slope_2 0.85864 NA 0.56997 1.14904 NA NA
## Tau2_2 0.31370 NA 0.24544 0.40106 NA NA
## Tau2_3 0.61894 NA 0.40375 0.95904 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.31281 0.6724
## Tau2 (with predictors) 0.31370 0.6189
## R2 0.00000 0.0795
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 4
## Degrees of freedom: 340
## -2 log likelihood: 875.476
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Population age
This works out how many studies in each variable
## # A tibble: 2 × 2
## age distinct_points
## <dbl> <int>
## 1 1 15
## 2 2 57This works out how many effect sizes in each variable
1 = Adolescents, 2 = Children
##
## 1 2
## 62 282This performs the meta regression in population age
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(adolescent,
## child), data = data, intercept.constraints = 0, intervals.type = "LB",
## I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.82676 NA 0.36279 1.30303 NA NA
## Slope_2 0.59039 NA 0.35203 0.82700 NA NA
## Tau2_2 0.31253 NA 0.24459 0.39946 NA NA
## Tau2_3 0.67259 NA 0.44424 1.03376 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.31281127 0.6724
## Tau2 (with predictors) 0.31253156 0.6726
## R2 0.00089419 0.0000
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 4
## Degrees of freedom: 340
## -2 log likelihood: 879.1867
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Sex
This works out how many studies in each variable
## # A tibble: 4 × 2
## gender distinct_points
## <dbl> <int>
## 1 1 6
## 2 2 12
## 3 3 44
## 4 4 10This works out how many effect sizes in each variable
1 = Boys, 2 = Girls, 3 = Both, 4 = Non-binary, 5=Boys and girls and non-binary
##
## 1 2 3 4
## 21 69 210 44This performs the meta regression in population age
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(female,
## male, mixed, nr), data = data, intercept.constraints = 0,
## intervals.type = "LB", I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.503998 NA -0.241528 1.252137 NA NA
## Slope_2 0.606245 NA 0.092702 1.118516 NA NA
## Slope_3 0.644313 NA 0.374756 0.915484 NA NA
## Slope_4 0.729273 NA 0.164437 1.304010 NA NA
## Tau2_2 0.312832 NA 0.244783 0.399924 NA NA
## Tau2_3 0.670918 NA 0.442021 1.032484 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.31281 0.6724
## Tau2 (with predictors) 0.31283 0.6709
## R2 0.00000 0.0022
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 6
## Degrees of freedom: 338
## -2 log likelihood: 879.7429
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Type of MSA
This works out how many studies in each variable
## # A tibble: 4 × 2
## type_of_MSA distinct_points
## <dbl> <int>
## 1 1 8
## 2 2 52
## 3 3 4
## 4 4 10This works out how many effect sizes in each variable
1 = Body weight , 2 = Combined, 3 = Plyometric, 4=Traditional
##
## 1 2 3 4
## 29 230 12 73This performs the meta regression in population age
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(bw,
## combined, plyometric, tm), data = data, intercept.constraints = 0,
## intervals.type = "LB", I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.24651 NA -0.37831 0.87379 NA NA
## Slope_2 0.64772 NA 0.40986 0.88721 NA NA
## Slope_3 0.98539 NA 0.30141 1.67192 NA NA
## Slope_4 0.79653 NA 0.30635 1.28801 NA NA
## Tau2_2 0.31323 NA 0.24512 0.40040 NA NA
## Tau2_3 0.64049 NA 0.42035 0.98798 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.31281 0.6724
## Tau2 (with predictors) 0.31323 0.6405
## R2 0.00000 0.0474
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 6
## Degrees of freedom: 338
## -2 log likelihood: 877.2806
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Facilitator
This works out how many studies in each variable
## # A tibble: 3 × 2
## facilitator distinct_points
## <dbl> <int>
## 1 1 61
## 2 2 9
## 3 3 2This works out how many effect sizes in each variable
1 = teacher/School Satff, 2 = External provider , 3 = both
##
## 1 2 3
## 273 65 6This performs the meta regression in population age
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(teacher,
## external, both), data = data, intercept.constraints = 0,
## intervals.type = "LB", I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.61593 NA 0.38858 0.84330 NA NA
## Slope_2 0.50849 NA -0.07391 1.09526 NA NA
## Slope_3 2.11712 NA 0.78177 3.50800 NA NA
## Tau2_2 0.31095 NA 0.24340 0.39734 NA NA
## Tau2_3 0.65580 NA 0.43673 0.99918 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.3128113 0.6724
## Tau2 (with predictors) 0.3109453 0.6558
## R2 0.0059651 0.0247
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 5
## Degrees of freedom: 339
## -2 log likelihood: 875.0313
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)Body region
This works out how many studies in each variable
## # A tibble: 4 × 2
## body_region distinct_points
## <dbl> <int>
## 1 1 46
## 2 2 25
## 3 3 59
## 4 4 7This works out how many effect sizes in each variable
1 = Upper, 2 = Core 3 = lower, 4 = combined
##
## 1 2 3 4
## 125 39 158 22This performs the meta regression in population age
##
## Call:
## meta3L(y = d, v = sampling_variance, cluster = study_id, x = cbind(upper,
## core, lower, combined), data = data, intercept.constraints = 0,
## intervals.type = "LB", I2 = "I2q", R2 = TRUE)
##
## 95% confidence intervals: Likelihood-based statistic
## Coefficients:
## Estimate Std.Error lbound ubound z value Pr(>|z|)
## Slope_1 0.53448 NA 0.30281 0.76714 NA NA
## Slope_2 0.94765 NA 0.64681 1.24827 NA NA
## Slope_3 0.63603 NA 0.41400 0.86140 NA NA
## Slope_4 0.55542 NA 0.11588 0.99449 NA NA
## Tau2_2 0.30093 NA 0.23471 0.38555 NA NA
## Tau2_3 0.64637 NA 0.42451 0.99905 NA NA
##
## Q statistic on the homogeneity of effect sizes: 3471.148
## Degrees of freedom of the Q statistic: 343
## P value of the Q statistic: 0
##
## Explained variances (R2):
## Level 2 Level 3
## Tau2 (no predictor) 0.312811 0.6724
## Tau2 (with predictors) 0.300926 0.6464
## R2 0.037996 0.0387
##
## Number of studies (or clusters): 72
## Number of observed statistics: 344
## Number of estimated parameters: 6
## Degrees of freedom: 338
## -2 log likelihood: 869.741
## OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
## Other values may indicate problems.)