This is the supplementary analytic output for the paper Stress regulation via being in nature and social support in adults: a meta-analysis

It reports detailed results for all models reported in the paper. The analytic R script by which this html report was generated can be found on the project’s OSF page at: https://osf.io/6wpav/


Brief information about the methods used in the analysis:

RMA results with model-based SEs k = number of studies; sqrt in “Variance components” = tau, the standard deviation of true effects; estimate in “Model results” = naive MA estimate

RVE SEs with Satterthwaite small-sample correction Estimate based on a multilevel RE model with constant sampling correlation model (CHE - correlated hierarchical effects - working model) (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Interpretation of naive-meta-analysis should be based on these estimates.

Prediction interval Shows the expected range of true effects in similar studies. As an approximation, in 95% of cases the true effect in a new published study can be expected to fall between PI LB and PI UB. Note that these are non-adjusted estimates. An unbiased newly conducted study will more likely fall in an interval centered around bias-adjusted estimate with a wider CI width.

Heterogeneity Tau can be interpreted as the total amount of heterogeneity in the true effects. I^2$ represents the ratio of true heterogeneity to total variance across the observed effect estimates. Estimates calculated by two approaches are reported. This is followed by separate estimates of between- and within-cluster heterogeneity and estimated intra-class correlation of underlying true effects.

Proportion of significant results What proportion of effects were statistically at the alpha level of .05.

ES-precision correlation Kendalls’s correlation between the ES and precision.

4/3PSM Applies a permutation-based, step-function 4-parameter selection model (one-tailed p-value steps = c(.025, .5, 1)). Falls back to 3-parameter selection model if at least one of the three p-value intervals contains less than 5 p-values. For this meta-analysis, we applied 3-parameter selection model by default as there were only 11 independent effects in the opposite direction overall (6%), causing the estimates to be unstable across iterations. pvalue = p-value testing H0 that the effect is zero. ciLB and ciUB are lower and upper bound of the CI. k = number of studies. steps = 3 means that the 4PSM was applied, 2 means that the 3PSM was applied. We also ran two sensitivity analyses of the selection model, the Vevea & Woods (2005) step function model with a priori defined selection weights and the Robust Bayesian Meta-analysis model employing the model-averaging approach (Bartoš & Maier, 2020).

PET-PEESE Estimated effect size of an infinitely precise study. Using 4/3PSM as the conditional estimator instead of PET (can be changed to PET). If the PET-PEESE estimate is in the opposite direction, the effect can be regarded nil. By default (can be changed to PET), the function employs a modified sample-size based estimator (see https://www.jepusto.com/pet-peese-performance/). It also uses the same RVE sandwich-type based estimator in a CHE (correlated hierarchical effects) working model with the identical random effects structure as the primary (naive) meta-analytic model.

We report results for both, PET and PEESE, with the first reported one being the primary (based on the conditional estimator).

WAAP-WLS The combined WAAP-WLS estimator (weighted average of the adequately powered - weighted least squares) tries to identify studies that are adequately powered to detect the meta-analytic effect. If there is less than two such studies, the method falls back to the WLS estimator (Stanley & Doucouliagos, 2015). If there are at least two adequately powered studies, WAAP returns a WLS estimate based on effects from only those studies.

type = 1: WAAP estimate, 2: WLS estimate. kAdequate = number of adequately powered studies

p-uniform P-uniform* is a selection model conceptually similar to p-curve. It makes use of the fact that p-values follow a uniform distribution at the true effect size while it includes also nonsignificant effect sizes. Permutation-based version of p-uniform method, the so-called p-uniform* (van Aert, van Assen, 2021).

p-curve Permutation-based p-curve method. Output should be self-explanatory. For more info see p-curve.com

Power for detecting SESOI and bias-corrected parameter estimates Estimates of the statistical power for detecting a smallest effect sizes of interest equal to .20, .50, and .70 in SD units (Cohen’s d). A sort of a thought experiment, we also assumed that population true values equal the bias-corrected estimates (4/3PSM or PET-PEESE) and computed power for those.

Handling of dependencies in bias-correction methods To handle dependencies among the effects, the 4PSM, p-curve, p-uniform are implemented using a permutation-based procedure, randomly selecting only one focal effect (i.e., excluding those which were not coded as being focal) from a single study and iterating nIterations times. Lastly, the procedure selects the result with the median value of the ES estimate (4PSM, p-uniform) or median z-score of the full p-curve (p-curve).

Descriptives

Publication year

Publication year for Being in nature

## from   to 
## 1993 2021

Publication year for Social support

## from   to 
## 1997 2021

Sample sizes

N of effects for Being in nature

## [1] 54

N of effects for Social support

## [1] 18

N of studies for Being in nature

## [1] 16

N of studies for Social support

## [1] 13

N of papers for Being in nature

## [1] 15

N of papers for Social support

## [1] 13

Median N across all the ES eligible for meta-analysis for Being in nature

## [1] 52.5

Median N across all the ES eligible for meta-analysis for Social support

## [1] 186

Total meta-analytic N for Being in nature

## [1] 1697

Total meta-analytic N for Social support

## [1] 3787

Mean gender ratio (percent female) for Being in nature

##     Mean       SD 
## 49.90571 32.49089

Mean gender ratio (percent female) for Social support

##     Mean       SD 
## 45.09786 38.42652

Weighted mean age of included samples for Being in nature

## [1] 29.54122

Weighted mean age of included samples for Social support

## [1] 47.71785

Meta-analysis results

Being in nature

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 54; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1128  0.3358     16     no         study 
## sigma^2.2  0.0853  0.2920     54     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 53) = 361.3608, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval    ci.lb    ci.ub 
##  -0.4211  0.1084  -3.8848  0.0001  -0.6335  -0.2086  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate    SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt   -0.421 0.109  -3.86 14.3      0.00166   **
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt   -0.421 0.109 14.3       -0.654       -0.188
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -1.397     0.555 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.4450121                    85.7479248 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    94.6700000                    48.8300000 
##  Within-cluster heterogeneity                           ICC 
##                    36.9200000                     0.5700000 
## 
## $`Proportion of significant results`
## [1] 0.4444444
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] -0.2634521
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
## -0.598  0.215 -2.776  0.006 -1.020 -0.176 16.000  2.000 
## 
## $`Publication bias`$`Vevea & Woods SM`
##                      est    se zvalue pvalue   ciLB   ciUB  k steps
## moderateSelection -0.307 0.123 -2.500  0.012 -0.548 -0.066 16     9
## severeSelection   -0.181 0.131 -1.378  0.168 -0.437  0.076 16     9
## extremeSelection  -0.009 0.150 -0.058  0.954 -0.303  0.286 16     9
## 
## $`Publication bias`$`Robust BMA`
## Model-averaged estimates:
##                     Mean Median  0.025  0.975
## mu                -0.423 -0.421 -0.573 -0.283
## tau                0.434  0.429  0.340  0.547
## omega[0,0.025]     1.000  1.000  1.000  1.000
## omega[0.025,0.05]  0.991  1.000  0.856  1.000
## omega[0.05,0.5]    0.967  1.000  0.511  1.000
## omega[0.5,0.95]    0.964  1.000  0.471  1.000
## omega[0.95,0.975]  0.965  1.000  0.471  1.000
## omega[0.975,1]     0.967  1.000  0.471  1.000
## PET                0.017  0.000  0.000  0.279
## PEESE              0.014  0.000  0.000  0.000
## (Estimated publication weights omega correspond to one-sided p-values.)
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.546          0.336         -1.628          0.126         -1.266 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.173         -0.437          0.198         -2.209          0.044 
##           ciLB           ciUB 
##         -0.861         -0.013 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate std.error statistic            p.value   conf.low
## 1 WAAP-WLS   b0 -0.5931473 0.0742671 0.0742671 0.0000000001175175 -0.7421082
##    conf.high type kAdequate
## 1 -0.4441863    2         0
## 
## $`Publication bias`$`p-uniform*`
##           est          ciLB          ciUB        pvalue 
## -0.6797468091 -1.0265737657 -0.3180098564  0.0002537374 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 16 
## - Total number of p<0.05 studies included into the analysis: k = 4 (25%) 
## - Total number of studies with p<0.025: k = 3 (18.75%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.312 -6.681     0 -7.962     0
## Flatness test           0.740  5.691     1  7.767     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 99% (99%-99%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20 
##                                         0.174 
## Median power for detecting a SESOI of d = .50 
##                                         0.718 
## Median power for detecting a SESOI of d = .70 
##                                         0.944 
## Median power for detecting PET-PEESE estimate 
##                                         0.791 
##    Median power for detecting 4/3PSM estimate 
##                                         0.858

Social support

## $`RMA results with model-based SEs`
## 
## Multivariate Meta-Analysis Model (k = 18; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0229  0.1512     13     no         study 
## sigma^2.2  0.0034  0.0581     18     no  study/result 
## 
## Test for Heterogeneity:
## Q(df = 17) = 96.6541, p-val < .0001
## 
## Model Results:
## 
## estimate      se     zval    pval    ci.lb    ci.ub 
##  -0.1394  0.0499  -2.7926  0.0052  -0.2372  -0.0416  ** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE SEs with Satterthwaite small-sample correction`
## $`RVE SEs with Satterthwaite small-sample correction`$test
##     Coef. Estimate     SE t-stat d.f. p-val (Satt) Sig.
## 1 intrcpt   -0.139 0.0498   -2.8 11.6       0.0166    *
## 
## $`RVE SEs with Satterthwaite small-sample correction`$CIs
##      Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 intrcpt   -0.139 0.0498 11.6       -0.248      -0.0304
## 
## 
## $`Prediction interval`
## 95% PI LB 95% PI UB 
##    -0.509     0.230 
## 
## $Heterogeneity
##                           Tau                           I^2 
##                     0.1620018                    84.0673537 
##                 Jackson's I^2 Between-cluster heterogeneity 
##                    89.2400000                    73.2500000 
##  Within-cluster heterogeneity                           ICC 
##                    10.8200000                     0.8700000 
## 
## $`Proportion of significant results`
## [1] 0.5555556
## 
## $`Publication bias`
## $`Publication bias`$`ES-precision correlation`
## [1] 0.2679739
## 
## $`Publication bias`$`4/3PSM`
##    est     se zvalue pvalue   ciLB   ciUB      k  steps 
## -0.101  0.078 -1.295  0.195 -0.255  0.052 13.000  2.000 
## 
## $`Publication bias`$`Vevea & Woods SM`
##                      est    se zvalue pvalue   ciLB  ciUB  k steps
## moderateSelection -0.089 0.050 -1.780  0.075 -0.187 0.009 13     9
## severeSelection   -0.040 0.054 -0.748  0.455 -0.146 0.065 13     9
## extremeSelection   0.025 0.063  0.390  0.697 -0.099 0.149 13     9
## 
## $`Publication bias`$`Robust BMA`
## Model-averaged estimates:
##                     Mean Median  0.025  0.975
## mu                -0.210 -0.206 -0.409  0.000
## tau                0.165  0.158  0.100  0.271
## omega[0,0.025]     1.000  1.000  1.000  1.000
## omega[0.025,0.05]  0.985  1.000  0.745  1.000
## omega[0.05,0.5]    0.964  1.000  0.529  1.000
## omega[0.5,0.95]    0.938  1.000  0.171  1.000
## omega[0.95,0.975]  0.939  1.000  0.171  1.000
## omega[0.975,1]     0.945  1.000  0.171  1.000
## PET                0.324  0.000  0.000  2.845
## PEESE              2.967  0.000  0.000 17.954
## (Estimated publication weights omega correspond to one-sided p-values.)
## 
## $`Publication bias`$`PET-PEESE`
##   PET estimate             se         zvalue         pvalue           ciLB 
##         -0.105          0.138         -0.760          0.463         -0.410 
##           ciUB PEESE estimate             se         zvalue         pvalue 
##          0.199         -0.132          0.081         -1.627          0.132 
##           ciLB           ciUB 
##         -0.311          0.047 
## 
## $`Publication bias`$`WAAP-WLS`
##     method term   estimate  std.error  statistic      p.value   conf.low
## 1 WAAP-WLS   b0 -0.1916986 0.04090339 0.04090339 0.0002121461 -0.2779972
##   conf.high type kAdequate
## 1   -0.1054    2         0
## 
## $`Publication bias`$`p-uniform*`
##          est         ciLB         ciUB       pvalue 
## -0.140845675 -0.273948842 -0.004857412  0.042311322 
## 
## $`Publication bias`$`p-curve`
## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 13 
## - Total number of p<0.05 studies included into the analysis: k = 7 (53.85%) 
## - Total number of studies with p<0.025: k = 7 (53.85%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.008 -8.365     0 -7.623     0
## Flatness test           1.000  6.105     1  7.076     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 99% (95.4%-99%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no 
## 
## 
## $`Power for detecting SESOI and bias-corrected parameter estimates`
## Median power for detecting a SESOI of d = .20 
##                                         0.486 
## Median power for detecting a SESOI of d = .50 
##                                         0.998 
## Median power for detecting a SESOI of d = .70 
##                                         1.000 
## Median power for detecting PET-PEESE estimate 
##                                         0.173 
##    Median power for detecting 4/3PSM estimate 
##                                         0.163

Plots

Contour enhanced funnel plot

Being in nature

Social support

Forest plots

Being in nature

Social support

p-curve plots

Being in nature

Social support

PET-PEESE plots

Using the sqrt(2/n) and 2/n terms instead of SE and var for PET and PEESE, respectively since modified sample-size based estimator was implemented (see https://www.jepusto.com/pet-peese-performance/).

Being in nature

Social support

Risk of bias assessment

Risk of bias for Being in nature

Risk of bias for Social support

Moderator analysis for the proportion of females

## $Nature
## $Nature$`Proportion of females`
## [1] 50
## 
## $Nature$`Model results`
##        Coef. Estimate      SE t-stat p-val (z) Sig.
## 1 percFemale -0.00609 0.00162  -3.77    <0.001  ***
## 
## 
## $Social
## $Social$`Proportion of females`
## [1] 43.45
## 
## $Social$`Model results`
##        Coef. Estimate      SE t-stat p-val (z) Sig.
## 1 percFemale -0.00159 0.00105   -1.5     0.133

Subgroup analysis for the type of the comparison group

## $Nature
## $Nature[[1]]
## 
##  1  2 
## 10 44 
## 
## $Nature$`Model results`
##                          Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(comparisonGroupType)1    0.150 0.0251   5.98    <0.001  ***
## 2 factor(comparisonGroupType)2   -0.547 0.1011  -5.41    <0.001  ***
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  44.8      1     1.48 0.0438   *
## 
## 
## $Social
## $Social[[1]]
## 
##  0  1 
## 16  2 
## 
## $Social$`Model results`
##                          Coef. Estimate     SE t-stat p-val (z) Sig.
## 1 factor(comparisonGroupType)0   -0.172 0.0496  -3.46    <0.001  ***
## 2 factor(comparisonGroupType)1    0.127 0.0905   1.40      0.16     
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  8.39      1     1.24 0.172

Subgroup analysis for the population type

## $Nature
## $Nature[[1]]
## 
##  1  2 
## 29 24 
## 
## $Nature$`Model results`
##                     Coef. Estimate    SE t-stat p-val (z) Sig.
## 1 factor(populationType)1   -0.357 0.137  -2.61   0.00897   **
## 2 factor(populationType)2   -0.506 0.166  -3.05   0.00226   **
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.479      1     10.6 0.504    
## 
## 
## $Social
## $Social[[1]]
## 
## 1 2 3 
## 6 6 6 
## 
## $Social$`Model results`
##                     Coef. Estimate     SE  t-stat p-val (z) Sig.
## 1 factor(populationType)1 -0.25094 0.0712 -3.5223    <0.001  ***
## 2 factor(populationType)2 -0.13547 0.0613 -2.2091    0.0272    *
## 3 factor(populationType)3 -0.00389 0.1172 -0.0332    0.9735     
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  1.53      2     5.96 0.292

Subgroup analysis for type of exposure

## [[1]]
## 
##  1  2  3  4 
## 23  6 18  7 
## 
## $`Model results`
## 
## Multivariate Meta-Analysis Model (k = 54; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.1410  0.3755     16     no         study 
## sigma^2.2  0.0870  0.2949     54     no  study/result 
## 
## Test for Residual Heterogeneity:
## QE(df = 50) = 347.4456, p-val < .0001
## 
## Test of Moderators (coefficients 1:4):
## QM(df = 4) = 13.5188, p-val = 0.0090
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb    ci.ub 
## factor(typeOfExposure)1   -0.4093  0.1759  -2.3271  0.0200  -0.7540  -0.0646  * 
## factor(typeOfExposure)2   -0.4339  0.3031  -1.4313  0.1523  -1.0279   0.1602    
## factor(typeOfExposure)3   -0.5594  0.2610  -2.1434  0.0321  -1.0709  -0.0479  * 
## factor(typeOfExposure)4   -0.3072  0.2541  -1.2087  0.2268  -0.8052   0.1909    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.503      3     4.23 0.699

Subgroup analysis for type of social support

## [[1]]
## 
##  0  1  2  3 
## 14  2  1  1 
## 
## $`Model results`
## 
## Multivariate Meta-Analysis Model (k = 18; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0233  0.1526     13     no         study 
## sigma^2.2  0.0033  0.0575     18     no  study/result 
## 
## Test for Residual Heterogeneity:
## QE(df = 14) = 88.1991, p-val < .0001
## 
## Test of Moderators (coefficients 1:4):
## QM(df = 4) = 11.2540, p-val = 0.0239
## 
## Model Results:
## 
##                               estimate      se     zval    pval    ci.lb 
## factor(typeOfSocialSupport)0   -0.1347  0.0561  -2.3986  0.0165  -0.2447 
## factor(typeOfSocialSupport)1   -0.3006  0.1803  -1.6671  0.0955  -0.6541 
## factor(typeOfSocialSupport)2    0.2371  0.2395   0.9899  0.3222  -0.2323 
## factor(typeOfSocialSupport)3   -0.2360  0.1788  -1.3196  0.1870  -0.5865 
##                                 ci.ub 
## factor(typeOfSocialSupport)0  -0.0246  * 
## factor(typeOfSocialSupport)1   0.0528  . 
## factor(typeOfSocialSupport)2   0.7065    
## factor(typeOfSocialSupport)3   0.1145    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  3751      2      1.7 <0.001 ***

Subgroup analysis for source of social support

## [[1]]
## 
##  0  1 
## 17  1 
## 
## $`Model results`
## 
## Multivariate Meta-Analysis Model (k = 18; method: REML)
## 
## Variance Components:
## 
##             estim    sqrt  nlvls  fixed        factor 
## sigma^2.1  0.0201  0.1418     13     no         study 
## sigma^2.2  0.0033  0.0575     18     no  study/result 
## 
## Test for Residual Heterogeneity:
## QE(df = 16) = 91.1265, p-val < .0001
## 
## Test of Moderators (coefficients 1:2):
## QM(df = 2) = 11.4904, p-val = 0.0032
## 
## Model Results:
## 
##                         estimate      se     zval    pval    ci.lb    ci.ub 
## factor(supportSource)0   -0.1573  0.0487  -3.2331  0.0012  -0.2526  -0.0619  ** 
## factor(supportSource)1    0.2371  0.2327   1.0187  0.3084  -0.2191   0.6932     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  65.7      1     10.8 <0.001 ***

Excluding effects based on inconsistent means or SDs

Only for Being in nature, since there were 0 inconsistent means or SDs for Social support studies.

## $Nature
## $Nature$`Count of GRIM/GRIMMER inconsistencies`
## 
## FALSE  TRUE 
##    15    10 
## 
## $Nature$`Model results`
##                                                   Coef Estimate    SE d.f.
## 1 factor(as.logical(inconsistenciesCountGRIMMER))FALSE   -0.557 0.157  Inf
## 2  factor(as.logical(inconsistenciesCountGRIMMER))TRUE   -0.715 0.168  Inf
##   Lower 95% CI Upper 95% CI
## 1       -0.865       -0.249
## 2       -1.044       -0.387
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  0.94      1     2.18 0.427

Excluding effects due to a high risk of bias

## $Nature
## $Nature$RoB
## 
## FALSE  TRUE 
##    42    12 
## 
## $Nature$`Model results`
##                                             Coef Estimate    SE d.f.
## 1 factor(robOverall > acceptableRiskOfBias)FALSE   -0.367 0.120  Inf
## 2  factor(robOverall > acceptableRiskOfBias)TRUE   -0.597 0.261  Inf
##   Lower 95% CI Upper 95% CI
## 1       -0.602      -0.1322
## 2       -1.110      -0.0846
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.638      1     3.34 0.477    
## 
## 
## $Social
## $Social$RoB
## 
## FALSE  TRUE 
##     1    17 
## 
## $Social$`Model results`
##                                             Coef Estimate     SE d.f.
## 1 factor(robOverall > acceptableRiskOfBias)FALSE    0.237 0.0000  Inf
## 2  factor(robOverall > acceptableRiskOfBias)TRUE   -0.157 0.0486  Inf
##   Lower 95% CI Upper 95% CI
## 1        0.237        0.237
## 2       -0.253       -0.062
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  65.7      1     10.8 <0.001 ***

Comparison of strategies

Model without covariates

## $`Model results`
##                Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(strategy)1   -0.701 0.203  Inf       -1.099      -0.3032
## 2 factor(strategy)2   -0.158 0.102  Inf       -0.358       0.0406
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  5.39      1     10.5 0.0415   *

Model with covariates

Controlling for design-related factors that are prognostic w.r.t. the effect sizes (i.e., might vary across moderator categories)

## $`Model results`
##                  Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1   factor(strategy)1 -0.73444 1.944  Inf       -4.545        3.076
## 2   factor(strategy)2 -1.16232 2.025  Inf       -5.131        2.806
## 3      researchDesign  0.12006 0.317  Inf       -0.500        0.741
## 4      populationType  0.00769 0.246  Inf       -0.474        0.489
## 5 comparisonGroupType -0.37602 0.440  Inf       -1.238        0.486
## 6           published -0.08972 0.349  Inf       -0.773        0.594
## 7          robOverall  0.23530 0.317  Inf       -0.386        0.856
## 
## $`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.263      1     5.15 0.629

Numerical inconsistencies in reported p-values

Being in nature

How many results were analyzed

## [1] 112

How many papers reported results in APA format

## [1] 8

How many statcheck errors

## 
##      FALSE       TRUE 
## 0.96428571 0.03571429

What proportion of statcheck errors affected the decision

## TRUE 
##  0.5

How many papers contained statcheck errors

## [1] 0.5

Social support

How many results were analyzed

## [1] 20

How many papers reported results in APA format

## [1] 3

How many statcheck errors

## 
## FALSE  TRUE 
##  0.85  0.15

What proportion of statcheck errors affected the decision

##      TRUE 
## 0.6666667

How many papers contained statcheck errors

## [1] 0.6666667

Moderator/sensitivity analyses

The below reported meta-regressions are all implemented as a multivariate RVE-based models using the CHE working model (Pustejovsky & Tipton, 2020; https://osf.io/preprints/metaarxiv/vyfcj/). Testing of contrasts is carried out using a robust Wald-type test testing the equality of estimates across levels of the moderator.

Published status

## $Nature
## $Nature[[1]]
## 
##  0  1 
##  8 46 
## 
## $Nature$`Model results`
##                 Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0  -0.0333 0.212  Inf       -0.449        0.383
## 2 factor(published)1  -0.4796 0.114  Inf       -0.702       -0.257
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  3.44      1     1.32 0.266    
## 
## 
## $Social
## $Social[[1]]
## 
##  0  1 
##  7 11 
## 
## $Social$`Model results`
##                 Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 factor(published)0  -0.3150 0.0268  Inf       -0.367      -0.2625
## 2 factor(published)1  -0.0887 0.0547  Inf       -0.196       0.0185
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  13.8      1     3.33 0.0284   *

Excluding effects from non-randomized designs

## $Nature
## $Nature[[1]]
## 
##  1  2  3 
##  9  4 41 
## 
## $Nature$`Model results`
##                               Coef Estimate     SE d.f. Lower 95% CI
## 1 factor(researchDesign == 1)FALSE    -0.46 0.1435  Inf       -0.741
## 2  factor(researchDesign == 1)TRUE    -0.31 0.0984  Inf       -0.503
##   Upper 95% CI
## 1       -0.178
## 2       -0.117
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ  0.74      1     6.76 0.419    
## 
## 
## $Social
## $Social[[1]]
## 
##  1  3  4 
##  1  1 16 
## 
## $Social$`Model results`
##                               Coef Estimate     SE d.f. Lower 95% CI
## 1 factor(researchDesign == 1)FALSE   -0.157 0.0486  Inf       -0.253
## 2  factor(researchDesign == 1)TRUE    0.237 0.0000  Inf        0.237
##   Upper 95% CI
## 1       -0.062
## 2        0.237
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  65.7      1     10.8 <0.001 ***

Subgroup analysis for stress vs affective consequences

## $Nature
## $Nature$`Number of included effects per category`
## 
##  1  2 
## 48  6 
## 
## $Nature$`Model results`
##                          Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 as.factor(stressAffective)1   -0.421 0.109  Inf       -0.634      -0.2078
## 2 as.factor(stressAffective)2   -0.429 0.169  Inf       -0.760      -0.0969
## 
## $Nature$`RVE Wald test`
##  test   Fstat df_num df_denom p_val sig
##   HTZ 0.00305      1     5.05 0.958    
## 
## 
## $Social
## $Social$`Number of included effects per category`
## 
##  1  2 
## 14  4 
## 
## $Social$`Model results`
##                          Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 as.factor(stressAffective)1   -0.109 0.0531  Inf       -0.214     -0.00534
## 2 as.factor(stressAffective)2   -0.262 0.0712  Inf       -0.402     -0.12254
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ   4.3      1     2.65 0.142

Subgroup analysis for stress vs affective consequences

## $Nature
## $Nature$`Number of included effects per category`
## 
##  1  3 
## 28 20 
## 
## $Nature$`Model results`
##                            Coef Estimate    SE d.f. Lower 95% CI Upper 95% CI
## 1 as.factor(stressCompRecoded)1   -0.488 0.156  Inf       -0.794      -0.1821
## 2 as.factor(stressCompRecoded)3   -0.306 0.149  Inf       -0.599      -0.0138
## 
## $Nature$`RVE Wald test`
##  test Fstat df_num df_denom p_val sig
##   HTZ 0.609      1     5.43 0.468    
## 
## 
## $Social
## $Social$`Number of included effects per category`
## 
##  1  3 
## 13  1 
## 
## $Social$`Model results`
##                            Coef Estimate     SE d.f. Lower 95% CI Upper 95% CI
## 1 as.factor(stressCompRecoded)1  -0.1184 0.0575  Inf      -0.2311     -0.00567
## 2 as.factor(stressCompRecoded)3   0.0527 0.0000  Inf       0.0527      0.05270
## 
## $Social$`RVE Wald test`
##  test Fstat df_num df_denom  p_val sig
##   HTZ  8.85      1     9.63 0.0145   *

Forest plots

Being in nature

Social support

## R version 4.1.1 (2021-08-10)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur 10.16
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] sk_SK.UTF-8/sk_SK.UTF-8/sk_SK.UTF-8/C/sk_SK.UTF-8/sk_SK.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] gplots_3.1.1       RoBMA_2.1.0        robvis_0.3.0       poibin_1.5        
##  [5] ddpcr_1.15         clubSandwich_0.5.3 weightr_2.0.2      scales_1.2.0      
##  [9] magrittr_2.0.3     multcomp_1.4-17    TH.data_1.1-0      MASS_7.3-54       
## [13] survival_3.2-11    mvtnorm_1.1-3      Amelia_1.8.0       Rcpp_1.0.9        
## [17] pwr_1.3-0          lmerTest_3.1-3     kableExtra_1.3.4   puniform_0.2.4    
## [21] knitr_1.36         lme4_1.1-27.1      esc_0.5.1          dmetar_0.0.9000   
## [25] psychmeta_2.6.0    meta_5.0-0         metafor_3.0-2      Matrix_1.3-4      
## [29] psych_2.1.9        forcats_0.5.1      stringr_1.4.0      dplyr_1.0.9       
## [33] purrr_0.3.4        readr_2.0.2        tidyr_1.1.4        tibble_3.1.8      
## [37] ggplot2_3.3.5      tidyverse_1.3.1    reshape_0.8.8      car_3.0-11        
## [41] carData_3.0-4     
## 
## loaded via a namespace (and not attached):
##   [1] readxl_1.3.1         backports_1.2.1      systemfonts_1.0.3   
##   [4] plyr_1.8.7           splines_4.1.1        digest_0.6.29       
##   [7] htmltools_0.5.2      fansi_1.0.3          cluster_2.1.2       
##  [10] tzdb_0.1.2           openxlsx_4.2.4       modelr_0.1.8        
##  [13] vroom_1.5.5          sandwich_3.0-1       svglite_2.0.0       
##  [16] prettyunits_1.1.1    colorspace_2.0-3     rvest_1.0.1         
##  [19] ggrepel_0.9.1        haven_2.4.3          xfun_0.26           
##  [22] crayon_1.5.1         jsonlite_1.8.0       zoo_1.8-9           
##  [25] glue_1.6.2           gtable_0.3.0         webshot_0.5.2       
##  [28] kernlab_0.9-29       prabclus_2.3-2       DEoptimR_1.0-9      
##  [31] abind_1.4-5          DBI_1.1.1            viridisLite_0.4.0   
##  [34] progress_1.2.2       magic_1.5-9          tmvnsim_1.0-2       
##  [37] bit_4.0.4            foreign_0.8-81       BayesTools_0.1.2    
##  [40] mclust_5.4.7         stats4_4.1.1         netmeta_2.0-0       
##  [43] httr_1.4.2           fpc_2.2-9            runjags_2.2.0-2     
##  [46] modeltools_0.2-23    ellipsis_0.3.2       farver_2.1.0        
##  [49] pkgconfig_2.0.3      flexmix_2.3-17       nnet_7.3-16         
##  [52] sass_0.4.2           dbplyr_2.1.1         utf8_1.2.2          
##  [55] labeling_0.4.2       tidyselect_1.1.2     rlang_1.0.4         
##  [58] munsell_0.5.0        cellranger_1.1.0     tools_4.1.1         
##  [61] cli_3.3.0            generics_0.1.3       broom_0.7.9         
##  [64] mathjaxr_1.4-0       evaluate_0.14        fastmap_1.1.0       
##  [67] yaml_2.2.1           bit64_4.0.5          fs_1.5.2            
##  [70] zip_2.2.0            robustbase_0.93-9    caTools_1.18.2      
##  [73] nlme_3.1-152         xml2_1.3.2           compiler_4.1.1      
##  [76] rstudioapi_0.13      curl_4.3.2           reprex_2.0.1        
##  [79] bslib_0.3.1          stringi_1.7.8        highr_0.9           
##  [82] Brobdingnag_1.2-6    lattice_0.20-44      nloptr_1.2.2.2      
##  [85] vctrs_0.4.1          CompQuadForm_1.4.3   pillar_1.8.0        
##  [88] lifecycle_1.0.1      bridgesampling_1.1-2 jquerylib_0.1.4     
##  [91] bitops_1.0-7         data.table_1.14.2    R6_2.5.1            
##  [94] MuMIn_1.43.17        KernSmooth_2.23-20   gridExtra_2.3       
##  [97] rio_0.5.27           rjags_4-12           codetools_0.2-18    
## [100] gtools_3.9.2         boot_1.3-28          assertthat_0.2.1    
## [103] withr_2.5.0          mnormt_2.0.2         diptest_0.76-0      
## [106] parallel_4.1.1       hms_1.1.1            grid_4.1.1          
## [109] coda_0.19-4          class_7.3-19         minqa_1.2.4         
## [112] rmarkdown_2.11       numDeriv_2016.8-1.1  lubridate_1.8.0