Multimedia Meta-analysis Scripts

---

1 Calculating effect sizes

Notes on clustering corrections:

• ICCs decided as 0.175 based on cluster size and grade level values from Table 3 here.
• Table 4 in this paper reports very similar ICCs.
• Also in above paper: “Though there is no agreed upon delineation of what constitutes a large ICC, several authors have noted that school level variation often ranges from .05 to .20 (Muthén & Satorra, 1989; Spybrook, Raudenbush, Congdon, & Martinez, 2009). Hox (2010) has put forth that ICCs of .05, .10, and .15 can be viewed as small, moderate, or large estimates.”

Study and measure overview [221 total measures across 119 studies]:

---

Effect size and its variance for all 221 study measures:

---

1.1 Summarizing moderator counts

Code-related:

---

Language comprehension:

---

Reading comprehension:

---

1.2 Moderator correlations

Values (click arrows to scroll):

Plot:

---

2 Synthesizing effect sizes

2.1 Code-related

2.1.1 Effects and forest plots

Overall:

## 
## Random-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1186 (SE = 0.0295)
## tau (square root of estimated tau^2 value):      0.3444
## I^2 (total heterogeneity / total variability):   75.65%
## H^2 (total variability / sampling variability):  4.11
## 
## Test for Heterogeneity:
## Q(df = 85) = 234.4579, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3336  0.0486  6.8618  <.0001  0.2383  0.4289  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed proximal:

## 
## Random-Effects Model (k = 12; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0680 (SE = 0.0584)
## tau (square root of estimated tau^2 value):      0.2608
## I^2 (total heterogeneity / total variability):   57.59%
## H^2 (total variability / sampling variability):  2.36
## 
## Test for Heterogeneity:
## Q(df = 11) = 24.4048, p-val = 0.0111
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3944  0.1105  3.5681  0.0004  0.1778  0.6110  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed distal:

## 
## Random-Effects Model (k = 29; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.2955 (SE = 0.1092)
## tau (square root of estimated tau^2 value):      0.5436
## I^2 (total heterogeneity / total variability):   83.89%
## H^2 (total variability / sampling variability):  6.21
## 
## Test for Heterogeneity:
## Q(df = 28) = 114.3533, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.4874  0.1204  4.0477  <.0001  0.2514  0.7234  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed (all):

## 
## Random-Effects Model (k = 41; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.2115 (SE = 0.0695)
## tau (square root of estimated tau^2 value):      0.4599
## I^2 (total heterogeneity / total variability):   81.24%
## H^2 (total variability / sampling variability):  5.33
## 
## Test for Heterogeneity:
## Q(df = 40) = 139.5480, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.4620  0.0887  5.2093  <.0001  0.2882  0.6359  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Standardized:

## 
## Random-Effects Model (k = 45; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0400 (SE = 0.0188)
## tau (square root of estimated tau^2 value):      0.2001
## I^2 (total heterogeneity / total variability):   54.42%
## H^2 (total variability / sampling variability):  2.19
## 
## Test for Heterogeneity:
## Q(df = 44) = 87.5757, p-val = 0.0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2277  0.0470  4.8430  <.0001  0.1355  0.3198  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.1.2 Variation with Imputed ICC

---

2.2 Reading Comprehension

2.2.1 Effects and forest plots

Overall:

## 
## Random-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0352 (SE = 0.0128)
## tau (square root of estimated tau^2 value):      0.1877
## I^2 (total heterogeneity / total variability):   69.82%
## H^2 (total variability / sampling variability):  3.31
## 
## Test for Heterogeneity:
## Q(df = 64) = 209.6213, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2315  0.0359  6.4425  <.0001  0.1610  0.3019  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed (Distal):

## 
## Random-Effects Model (k = 20; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0348 (SE = 0.0224)
## tau (square root of estimated tau^2 value):      0.1865
## I^2 (total heterogeneity / total variability):   59.16%
## H^2 (total variability / sampling variability):  2.45
## 
## Test for Heterogeneity:
## Q(df = 19) = 39.8178, p-val = 0.0035
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3724  0.0627  5.9371  <.0001  0.2495  0.4954  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Standardized:

## 
## Random-Effects Model (k = 45; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0150 (SE = 0.0094)
## tau (square root of estimated tau^2 value):      0.1226
## I^2 (total heterogeneity / total variability):   47.63%
## H^2 (total variability / sampling variability):  1.91
## 
## Test for Heterogeneity:
## Q(df = 44) = 86.2474, p-val = 0.0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.1447  0.0353  4.1001  <.0001  0.0755  0.2139  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.2.2 Variation with Imputed ICC

---

2.3 Language-focused

2.3.1 Effects and forest plots

Overall:

## 
## Random-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0664 (SE = 0.0207)
## tau (square root of estimated tau^2 value):      0.2578
## I^2 (total heterogeneity / total variability):   68.66%
## H^2 (total variability / sampling variability):  3.19
## 
## Test for Heterogeneity:
## Q(df = 63) = 160.6184, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2984  0.0443  6.7439  <.0001  0.2117  0.3852  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed proximal:

## 
## Random-Effects Model (k = 29; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1362 (SE = 0.0574)
## tau (square root of estimated tau^2 value):      0.3691
## I^2 (total heterogeneity / total variability):   73.04%
## H^2 (total variability / sampling variability):  3.71
## 
## Test for Heterogeneity:
## Q(df = 28) = 85.4050, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.5799  0.0883  6.5668  <.0001  0.4068  0.7529  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed distal:

## 
## Random-Effects Model (k = 7; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0168 (SE = 0.0231)
## tau (square root of estimated tau^2 value):      0.1296
## I^2 (total heterogeneity / total variability):   44.14%
## H^2 (total variability / sampling variability):  1.79
## 
## Test for Heterogeneity:
## Q(df = 6) = 12.4326, p-val = 0.0530
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub     
##   0.2102  0.0772  2.7207  0.0065  0.0588  0.3615  ** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed (all):

## 
## Random-Effects Model (k = 36; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1264 (SE = 0.0457)
## tau (square root of estimated tau^2 value):      0.3555
## I^2 (total heterogeneity / total variability):   77.24%
## H^2 (total variability / sampling variability):  4.39
## 
## Test for Heterogeneity:
## Q(df = 35) = 118.2319, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.4924  0.0747  6.5893  <.0001  0.3459  0.6388  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Standardized:

## 
## Random-Effects Model (k = 28; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0 (SE = 0.0045)
## tau (square root of estimated tau^2 value):      0
## I^2 (total heterogeneity / total variability):   0.00%
## H^2 (total variability / sampling variability):  1.00
## 
## Test for Heterogeneity:
## Q(df = 27) = 11.0995, p-val = 0.9970
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.1176  0.0295  3.9818  <.0001  0.0597  0.1755  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

2.3.2 Variation with Imputed ICC

---

2.4 Writing Proficiency

Overall:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Researcher-Developed distal:

## 
## Random-Effects Model (k = 4; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1079 (SE = 0.1855)
## tau (square root of estimated tau^2 value):      0.3285
## I^2 (total heterogeneity / total variability):   48.02%
## H^2 (total variability / sampling variability):  1.92
## 
## Test for Heterogeneity:
## Q(df = 3) = 5.9435, p-val = 0.1144
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   1.0469  0.2383  4.3930  <.0001  0.5798  1.5140  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Standardized:

## 
## Random-Effects Model (k = 2; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0 (SE = 0.1676)
## tau (square root of estimated tau^2 value):      0
## I^2 (total heterogeneity / total variability):   0.00%
## H^2 (total variability / sampling variability):  1.00
## 
## Test for Heterogeneity:
## Q(df = 1) = 0.0105, p-val = 0.9184
## 
## Model Results:
## 
## estimate      se    zval    pval    ci.lb   ci.ub    
##   0.3420  0.2362  1.4482  0.1476  -0.1209  0.8049    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

2.5 Verifying effect sizes using RVE (Robust Variance Estimation)

As the dependency in our meta-analysis results from multiple measures nested within the same studies, we use the hierarchical effects weights to fit our model.

Code-related:

## RVE: Hierarchical Effects Model with Small-Sample Corrections 
## 
## Model: ES ~ 1 
## 
## Number of clusters = 65 
## Number of outcomes = 86 (min = 1 , mean = 1.32 , median = 1 , max = 3 )
## Omega.sq = 0 
## Tau.sq = 0.06845214 
## 
##                Estimate StdErr t-value  dfs     P(|t|>) 95% CI.L 95% CI.U Sig
## 1 X.Intercept.    0.317 0.0516    6.14 45.4 0.000000185    0.213    0.421 ***
## ---
## Signif. codes: < .01 *** < .05 ** < .10 *
## ---
## Note: If df < 4, do not trust the results

Reading comprehension:

## RVE: Hierarchical Effects Model with Small-Sample Corrections 
## 
## Model: ES ~ 1 
## 
## Number of clusters = 52 
## Number of outcomes = 65 (min = 1 , mean = 1.25 , median = 1 , max = 2 )
## Omega.sq = 0 
## Tau.sq = 0.03473362 
## 
##                Estimate StdErr t-value  dfs     P(|t|>) 95% CI.L 95% CI.U Sig
## 1 X.Intercept.    0.231 0.0378    6.11 31.8 0.000000805    0.154    0.308 ***
## ---
## Signif. codes: < .01 *** < .05 ** < .10 *
## ---
## Note: If df < 4, do not trust the results

Language focused:

## RVE: Hierarchical Effects Model with Small-Sample Corrections 
## 
## Model: ES ~ 1 
## 
## Number of clusters = 53 
## Number of outcomes = 64 (min = 1 , mean = 1.21 , median = 1 , max = 3 )
## Omega.sq = 0 
## Tau.sq = 0.0491561 
## 
##                Estimate StdErr t-value  dfs     P(|t|>) 95% CI.L 95% CI.U Sig
## 1 X.Intercept.    0.292 0.0398    7.32 32.4 0.000000024     0.21    0.373 ***
## ---
## Signif. codes: < .01 *** < .05 ** < .10 *
## ---
## Note: If df < 4, do not trust the results

Writing proficiency (estimated using a correlation effects model because each of the 6 studies reports a single writing measure):

## RVE: Correlated Effects Model with Small-Sample Corrections 
## 
## Model: ES ~ 1 
## 
## Number of studies = 6 
## Number of outcomes = 6 (min = 1 , mean = 1 , median = 1 , max = 1 )
## Rho = 0.8 
## I.sq = 55.64952 
## Tau.sq = 0.1429986 
## 
##                Estimate StdErr t-value  dfs P(|t|>) 95% CI.L 95% CI.U Sig
## 1 X.Intercept.    0.814  0.208    3.92 4.87  0.0118    0.276     1.35  **
## ---
## Signif. codes: < .01 *** < .05 ** < .10 *
## ---
## Note: If df < 4, do not trust the results

---

3 Moderator analyses

Descriptive summary of moderator variables used:

Data summary

Name	Piped data
Number of rows	221
Number of columns	17
_______________________
Column type frequency:
factor	7
numeric	10
________________________
Group variables	None

Variable type: factor

skim_variable	n_missing	complete_rate	ordered	n_unique	top_counts
grade	0	1	FALSE	2	K-2: 117, 3-5: 104
pedagogy	0	1	FALSE	3	Beh: 105, Cog: 101, Con: 15
instructional_context	0	1	FALSE	2	Ind: 144, Oth: 77
typetech	0	1	FALSE	3	CAI: 188, Oth: 22, ER: 11
design	0	1	FALSE	2	RCT: 165, QED: 56
control	0	1	FALSE	2	BAU: 176, ALT: 45
avghrs_categories	0	1	FALSE	3	Med: 94, Low: 66, Hig: 61

Variable type: numeric

skim_variable	n_missing	complete_rate	mean	sd	p0	p25	p50	p75	p100	hist
outside_us	0	1	0.62	0.49	0.0	0	1	1	1	▅▁▁▁▇
low_income_included	0	1	0.37	0.48	0.0	0	0	1	1	▇▁▁▁▅
second_lang_included	0	1	0.21	0.41	0.0	0	0	0	1	▇▁▁▁▂
stu_disability_included	0	1	0.20	0.40	0.0	0	0	0	1	▇▁▁▁▂
tech_plus1	0	1	0.46	0.50	0.0	0	0	1	1	▇▁▁▁▇
gamification	0	1	0.46	0.50	0.0	0	0	1	1	▇▁▁▁▇
adaptivity	0	1	0.33	0.47	0.0	0	0	1	1	▇▁▁▁▅
feedback	0	1	0.76	0.43	0.0	1	1	1	1	▂▁▁▁▇
teach_tr	0	1	0.62	0.49	0.0	0	1	1	1	▅▁▁▁▇
avghrs	0	1	21.24	22.37	0.3	6	15	25	120	▇▂▁▁▁

---

3.1 Comparing Across Outcomes (Code-related/ReadingComp/LanguageComp/Writing)

## 
## Mixed-Effects Model (k = 221; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0691 (SE = 0.0119)
## tau (square root of estimated tau^2 value):             0.2628
## I^2 (residual heterogeneity / unaccounted variability): 73.13%
## H^2 (unaccounted variability / sampling variability):   3.72
## R^2 (amount of heterogeneity accounted for):            4.63%
## 
## Test for Residual Heterogeneity:
## QE(df = 217) = 615.9715, p-val < .0001
## 
## Test of Moderators (coefficients 2:4):
## QM(df = 3) = 10.2522, p-val = 0.0165
## 
## Model Results:
## 
##                       estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt                 0.3175  0.0412   7.7067  <.0001   0.2368  0.3982  *** 
## outcomeReading Comp    -0.0778  0.0604  -1.2889  0.1974  -0.1961  0.0405      
## outcomeLanguage Comp   -0.0182  0.0609  -0.2987  0.7651  -0.1375  0.1011      
## outcomeWriting          0.4864  0.1805   2.6940  0.0071   0.1325  0.8403   ** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

3.2 Code-related

Measure type (Custom distal, Custom proximal, Standardized):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1166 (SE = 0.0296)
## tau (square root of estimated tau^2 value):             0.3415
## I^2 (residual heterogeneity / unaccounted variability): 73.99%
## H^2 (unaccounted variability / sampling variability):   3.85
## R^2 (amount of heterogeneity accounted for):            1.66%
## 
## Test for Residual Heterogeneity:
## QE(df = 83) = 226.3338, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 4.0866, p-val = 0.1296
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt    0.2470  0.0647  3.8172  0.0001   0.1202  0.3739  *** 
## typeDRP    0.1648  0.1452  1.1351  0.2563  -0.1198  0.4495      
## typeDRD    0.2087  0.1091  1.9134  0.0557  -0.0051  0.4224    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Location outside of the U.S.:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1219 (SE = 0.0304)
## tau (square root of estimated tau^2 value):             0.3492
## I^2 (residual heterogeneity / unaccounted variability): 75.54%
## H^2 (unaccounted variability / sampling variability):   4.09
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 233.5823, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0562, p-val = 0.8127
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.3119  0.1069  2.9179  0.0035   0.1024  0.5214  ** 
## outside_us    0.0285  0.1203  0.2370  0.8127  -0.2073  0.2643     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students from low-income backgrounds included:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1134 (SE = 0.0288)
## tau (square root of estimated tau^2 value):             0.3368
## I^2 (residual heterogeneity / unaccounted variability): 74.21%
## H^2 (unaccounted variability / sampling variability):   3.88
## R^2 (amount of heterogeneity accounted for):            4.35%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 232.5303, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 5.7069, p-val = 0.0169
## 
## Model Results:
## 
##                      estimate      se    zval    pval   ci.lb   ci.ub      
## intrcpt                0.2646  0.0557  4.7529  <.0001  0.1555  0.3738  *** 
## low_income_included    0.2612  0.1093  2.3889  0.0169  0.0469  0.4755    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students learning a second or other language included:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1204 (SE = 0.0301)
## tau (square root of estimated tau^2 value):             0.3470
## I^2 (residual heterogeneity / unaccounted variability): 75.33%
## H^2 (unaccounted variability / sampling variability):   4.05
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 232.0458, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4833, p-val = 0.4869
## 
## Model Results:
## 
##                       estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt                 0.3513  0.0548   6.4096  <.0001   0.2439  0.4587  *** 
## second_lang_included   -0.0841  0.1210  -0.6952  0.4869  -0.3212  0.1530      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students with disabilities included:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1214 (SE = 0.0302)
## tau (square root of estimated tau^2 value):             0.3484
## I^2 (residual heterogeneity / unaccounted variability): 76.08%
## H^2 (unaccounted variability / sampling variability):   4.18
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 231.8414, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0071, p-val = 0.9327
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                    0.3318  0.0570  5.8179  <.0001   0.2200  0.4435  *** 
## stu_disability_included    0.0094  0.1114  0.0845  0.9327  -0.2089  0.2277      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Grade level K-2 vs. 3-5:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1205 (SE = 0.0301)
## tau (square root of estimated tau^2 value):             0.3471
## I^2 (residual heterogeneity / unaccounted variability): 75.37%
## H^2 (unaccounted variability / sampling variability):   4.06
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 232.1693, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6227, p-val = 0.4300
## 
## Model Results:
## 
##           estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt     0.2688  0.0961  2.7978  0.0051   0.0805  0.4571  ** 
## gradeK-2    0.0881  0.1116  0.7891  0.4300  -0.1306  0.3067     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial, Cognitive, Constructivist:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1195 (SE = 0.0301)
## tau (square root of estimated tau^2 value):             0.3456
## I^2 (residual heterogeneity / unaccounted variability): 75.21%
## H^2 (unaccounted variability / sampling variability):   4.03
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 83) = 223.7778, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 1.3081, p-val = 0.5199
## 
## Model Results:
## 
##                         estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt                   0.3510  0.0604   5.8069  <.0001   0.2325  0.4694  *** 
## pedagogyCognitive        -0.0909  0.1099  -0.8276  0.4079  -0.3062  0.1244      
## pedagogyConstructivist    0.1256  0.1968   0.6382  0.5233  -0.2601  0.5113      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1211 (SE = 0.0302)
## tau (square root of estimated tau^2 value):             0.3480
## I^2 (residual heterogeneity / unaccounted variability): 75.45%
## H^2 (unaccounted variability / sampling variability):   4.07
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 234.3417, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0855, p-val = 0.7700
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt       0.3224  0.0634  5.0870  <.0001   0.1982  0.4466  *** 
## tech_plus1    0.0292  0.0998  0.2924  0.7700  -0.1664  0.2247      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Type of technology (CAI, ER, Other):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1196 (SE = 0.0299)
## tau (square root of estimated tau^2 value):             0.3459
## I^2 (residual heterogeneity / unaccounted variability): 76.07%
## H^2 (unaccounted variability / sampling variability):   4.18
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 83) = 228.1085, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 0.6367, p-val = 0.7274
## 
## Model Results:
## 
##                estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt          0.3242  0.0506   6.4045  <.0001   0.2250  0.4234  *** 
## typetechER       0.1595  0.2013   0.7927  0.4280  -0.2349  0.5540      
## typetechOther   -0.0353  0.4993  -0.0708  0.9436  -1.0140  0.9433      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Gamification Discussed (yes/no):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1219 (SE = 0.0303)
## tau (square root of estimated tau^2 value):             0.3492
## I^2 (residual heterogeneity / unaccounted variability): 75.59%
## H^2 (unaccounted variability / sampling variability):   4.10
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 234.4231, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0370, p-val = 0.8474
## 
## Model Results:
## 
##               estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt         0.3458  0.0772   4.4801  <.0001   0.1945  0.4971  *** 
## gamification   -0.0192  0.1000  -0.1924  0.8474  -0.2152  0.1767      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Adaptivity Discussed (yes/no):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1215 (SE = 0.0302)
## tau (square root of estimated tau^2 value):             0.3485
## I^2 (residual heterogeneity / unaccounted variability): 75.95%
## H^2 (unaccounted variability / sampling variability):   4.16
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 233.8737, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.3438, p-val = 0.5576
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.3590  0.0646   5.5532  <.0001   0.2323  0.4857  *** 
## adaptivity   -0.0581  0.0991  -0.5863  0.5576  -0.2523  0.1361      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Feedback Discussed (yes/no):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1191 (SE = 0.0298)
## tau (square root of estimated tau^2 value):             0.3451
## I^2 (residual heterogeneity / unaccounted variability): 75.35%
## H^2 (unaccounted variability / sampling variability):   4.06
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 228.1074, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4144, p-val = 0.5197
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt     0.3927  0.1037   3.7853  0.0002   0.1894  0.5960  *** 
## feedback   -0.0756  0.1175  -0.6437  0.5197  -0.3059  0.1546      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Instructional Context:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1210 (SE = 0.0302)
## tau (square root of estimated tau^2 value):             0.3478
## I^2 (residual heterogeneity / unaccounted variability): 75.42%
## H^2 (unaccounted variability / sampling variability):   4.07
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 234.1957, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.2620, p-val = 0.6087
## 
## Model Results:
## 
##                                             estimate      se    zval    pval 
## intrcpt                                       0.3168  0.0595  5.3241  <.0001 
## instructional_contextOther than individual    0.0535  0.1046  0.5119  0.6087 
##                                               ci.lb   ci.ub      
## intrcpt                                      0.2002  0.4335  *** 
## instructional_contextOther than individual  -0.1514  0.2585      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Teacher Training Discussed:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1222 (SE = 0.0304)
## tau (square root of estimated tau^2 value):             0.3496
## I^2 (residual heterogeneity / unaccounted variability): 75.94%
## H^2 (unaccounted variability / sampling variability):   4.16
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 233.0076, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1807, p-val = 0.6708
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt     0.3560  0.0706   5.0440  <.0001   0.2177  0.4943  *** 
## teach_tr   -0.0418  0.0982  -0.4250  0.6708  -0.2343  0.1508      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Design (RCT/QED):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1135 (SE = 0.0288)
## tau (square root of estimated tau^2 value):             0.3369
## I^2 (residual heterogeneity / unaccounted variability): 74.07%
## H^2 (unaccounted variability / sampling variability):   3.86
## R^2 (amount of heterogeneity accounted for):            4.32%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 226.7412, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.8046, p-val = 0.0511
## 
## Model Results:
## 
##            estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt      0.1103  0.1235  0.8931  0.3718  -0.1318  0.3524    
## designRCT    0.2614  0.1340  1.9505  0.0511  -0.0013  0.5241  . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (BAU/ALT):

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1195 (SE = 0.0299)
## tau (square root of estimated tau^2 value):             0.3456
## I^2 (residual heterogeneity / unaccounted variability): 75.11%
## H^2 (unaccounted variability / sampling variability):   4.02
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 229.5485, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.5508, p-val = 0.4580
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.2779  0.0897  3.0963  0.0020   0.1020  0.4538  ** 
## controlBAU    0.0793  0.1069  0.7422  0.4580  -0.1301  0.2888     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Duration:

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1226 (SE = 0.0305)
## tau (square root of estimated tau^2 value):             0.3502
## I^2 (residual heterogeneity / unaccounted variability): 75.40%
## H^2 (unaccounted variability / sampling variability):   4.07
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 233.9864, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.1755, p-val = 0.1402
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt    0.2694  0.0661  4.0765  <.0001   0.1399  0.3989  *** 
## avghrs     0.0029  0.0019  1.4750  0.1402  -0.0009  0.0067      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

3.3 Reading Comprehension

Measure type (Custom distal, Standardized):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0205 (SE = 0.0092)
## tau (square root of estimated tau^2 value):             0.1431
## I^2 (residual heterogeneity / unaccounted variability): 52.89%
## H^2 (unaccounted variability / sampling variability):   2.12
## R^2 (amount of heterogeneity accounted for):            41.89%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 126.0652, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 10.9205, p-val = 0.0010
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt    0.3698  0.0541   6.8336  <.0001   0.2637   0.4759  *** 
## typeCSN   -0.2184  0.0661  -3.3046  0.0010  -0.3480  -0.0889  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Location outside of the U.S.:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0362 (SE = 0.0132)
## tau (square root of estimated tau^2 value):             0.1902
## I^2 (residual heterogeneity / unaccounted variability): 69.26%
## H^2 (unaccounted variability / sampling variability):   3.25
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 206.0605, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1336, p-val = 0.7147
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.2393  0.0416   5.7584  <.0001   0.1578  0.3207  *** 
## outside_us   -0.0309  0.0846  -0.3655  0.7147  -0.1967  0.1348      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students from low-income backgrounds included:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0357 (SE = 0.0131)
## tau (square root of estimated tau^2 value):             0.1888
## I^2 (residual heterogeneity / unaccounted variability): 67.60%
## H^2 (unaccounted variability / sampling variability):   3.09
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 199.9647, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.5817, p-val = 0.4457
## 
## Model Results:
## 
##                      estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                0.1957  0.0594  3.2966  0.0010   0.0793  0.3120  *** 
## low_income_included    0.0570  0.0747  0.7627  0.4457  -0.0894  0.2034      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students learning a second or other language included:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0342 (SE = 0.0127)
## tau (square root of estimated tau^2 value):             0.1848
## I^2 (residual heterogeneity / unaccounted variability): 68.50%
## H^2 (unaccounted variability / sampling variability):   3.17
## R^2 (amount of heterogeneity accounted for):            3.06%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 190.1544, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6640, p-val = 0.4151
## 
## Model Results:
## 
##                       estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                 0.2191  0.0385  5.6926  <.0001   0.1437  0.2946  *** 
## second_lang_included    0.0826  0.1014  0.8149  0.4151  -0.1161  0.2814      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students with disabilities included:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0352 (SE = 0.0129)
## tau (square root of estimated tau^2 value):             0.1876
## I^2 (residual heterogeneity / unaccounted variability): 69.41%
## H^2 (unaccounted variability / sampling variability):   3.27
## R^2 (amount of heterogeneity accounted for):            0.18%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 209.3354, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.1498, p-val = 0.2836
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub 
## intrcpt                    0.2446  0.0379   6.4463  <.0001   0.1702  0.3189 
## stu_disability_included   -0.1261  0.1176  -1.0723  0.2836  -0.3565  0.1044 
##                              
## intrcpt                  *** 
## stu_disability_included      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Grade level K-2 vs. 3-5:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0335 (SE = 0.0126)
## tau (square root of estimated tau^2 value):             0.1830
## I^2 (residual heterogeneity / unaccounted variability): 59.07%
## H^2 (unaccounted variability / sampling variability):   2.44
## R^2 (amount of heterogeneity accounted for):            5.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 133.5489, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.8862, p-val = 0.3465
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt     0.2523  0.0422   5.9830  <.0001   0.1696  0.3349  *** 
## gradeK-2   -0.0731  0.0777  -0.9414  0.3465  -0.2253  0.0791      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial, Cognitive, Constructivist:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0367 (SE = 0.0133)
## tau (square root of estimated tau^2 value):             0.1915
## I^2 (residual heterogeneity / unaccounted variability): 69.33%
## H^2 (unaccounted variability / sampling variability):   3.26
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 195.0055, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0213, p-val = 0.8839
## 
## Model Results:
## 
##                    estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt              0.2215  0.0807  2.7437  0.0061   0.0633  0.3797  ** 
## pedagogyCognitive    0.0132  0.0904  0.1461  0.8839  -0.1640  0.1904     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0273 (SE = 0.0110)
## tau (square root of estimated tau^2 value):             0.1652
## I^2 (residual heterogeneity / unaccounted variability): 56.81%
## H^2 (unaccounted variability / sampling variability):   2.32
## R^2 (amount of heterogeneity accounted for):            22.56%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 126.1855, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 5.7610, p-val = 0.0164
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt       0.2969  0.0441   6.7239  <.0001   0.2103   0.3834  *** 
## tech_plus1   -0.1625  0.0677  -2.4002  0.0164  -0.2953  -0.0298    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Type of technology (CAI, ER, Other):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0359 (SE = 0.0130)
## tau (square root of estimated tau^2 value):             0.1894
## I^2 (residual heterogeneity / unaccounted variability): 70.24%
## H^2 (unaccounted variability / sampling variability):   3.36
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 209.4629, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.7222, p-val = 0.3954
## 
## Model Results:
## 
##                estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt          0.2390  0.0371   6.4388  <.0001   0.1663  0.3118  *** 
## typetechOther   -0.1358  0.1598  -0.8498  0.3954  -0.4490  0.1774      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Gamification Discussed (yes/no):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0364 (SE = 0.0133)
## tau (square root of estimated tau^2 value):             0.1908
## I^2 (residual heterogeneity / unaccounted variability): 68.12%
## H^2 (unaccounted variability / sampling variability):   3.14
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 198.8224, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.5879, p-val = 0.4432
## 
## Model Results:
## 
##               estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt         0.2474  0.0415   5.9582  <.0001   0.1660  0.3288  *** 
## gamification   -0.0654  0.0852  -0.7668  0.4432  -0.2324  0.1017      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Adaptivity Discussed (yes/no):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0370 (SE = 0.0134)
## tau (square root of estimated tau^2 value):             0.1924
## I^2 (residual heterogeneity / unaccounted variability): 68.40%
## H^2 (unaccounted variability / sampling variability):   3.16
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 192.8520, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1109, p-val = 0.7392
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.2399  0.0433   5.5403  <.0001   0.1551  0.3248  *** 
## adaptivity   -0.0267  0.0801  -0.3330  0.7392  -0.1837  0.1304      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Feedback Discussed (yes/no):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0343 (SE = 0.0128)
## tau (square root of estimated tau^2 value):             0.1852
## I^2 (residual heterogeneity / unaccounted variability): 67.94%
## H^2 (unaccounted variability / sampling variability):   3.12
## R^2 (amount of heterogeneity accounted for):            2.66%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 208.5025, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.9066, p-val = 0.1673
## 
## Model Results:
## 
##           estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt     0.1060  0.0974  1.0886  0.2763  -0.0848  0.2968    
## feedback    0.1445  0.1046  1.3808  0.1673  -0.0606  0.3495    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Instructional Context:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0255 (SE = 0.0106)
## tau (square root of estimated tau^2 value):             0.1596
## I^2 (residual heterogeneity / unaccounted variability): 53.78%
## H^2 (unaccounted variability / sampling variability):   2.16
## R^2 (amount of heterogeneity accounted for):            27.69%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 117.2455, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 5.6659, p-val = 0.0173
## 
## Model Results:
## 
##                                             estimate      se     zval    pval 
## intrcpt                                       0.2851  0.0410   6.9518  <.0001 
## instructional_contextOther than individual   -0.1632  0.0686  -2.3803  0.0173 
##                                               ci.lb    ci.ub      
## intrcpt                                      0.2047   0.3655  *** 
## instructional_contextOther than individual  -0.2977  -0.0288    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Teacher Training Discussed:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0360 (SE = 0.0132)
## tau (square root of estimated tau^2 value):             0.1897
## I^2 (residual heterogeneity / unaccounted variability): 67.47%
## H^2 (unaccounted variability / sampling variability):   3.07
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 200.4652, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4551, p-val = 0.4999
## 
## Model Results:
## 
##           estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt     0.1857  0.0773  2.4022  0.0163   0.0342  0.3371  * 
## teach_tr    0.0590  0.0874  0.6746  0.4999  -0.1124  0.2304    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Design (RCT/QED):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0360 (SE = 0.0131)
## tau (square root of estimated tau^2 value):             0.1897
## I^2 (residual heterogeneity / unaccounted variability): 69.00%
## H^2 (unaccounted variability / sampling variability):   3.23
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 194.2245, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1660, p-val = 0.6837
## 
## Model Results:
## 
##            estimate      se     zval    pval    ci.lb   ci.ub     
## intrcpt      0.2618  0.0821   3.1884  0.0014   0.1009  0.4227  ** 
## designRCT   -0.0373  0.0914  -0.4075  0.6837  -0.2165  0.1420     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (BAU/ALT):

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0338 (SE = 0.0128)
## tau (square root of estimated tau^2 value):             0.1839
## I^2 (residual heterogeneity / unaccounted variability): 56.96%
## H^2 (unaccounted variability / sampling variability):   2.32
## R^2 (amount of heterogeneity accounted for):            4.01%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 131.2204, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.3862, p-val = 0.1224
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt       0.0799  0.1040  0.7689  0.4419  -0.1238  0.2837    
## controlBAU    0.1709  0.1106  1.5447  0.1224  -0.0459  0.3877    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Duration:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0313 (SE = 0.0121)
## tau (square root of estimated tau^2 value):             0.1769
## I^2 (residual heterogeneity / unaccounted variability): 55.48%
## H^2 (unaccounted variability / sampling variability):   2.25
## R^2 (amount of heterogeneity accounted for):            11.17%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 125.2354, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.3154, p-val = 0.1281
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.2858  0.0506   5.6438  <.0001   0.1866  0.3851  *** 
## avghrs    -0.0019  0.0012  -1.5216  0.1281  -0.0043  0.0005      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

3.4 Language-focused

Measure type (Custom distal, Custom proximal, Standardized):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0221 (SE = 0.0108)
## tau (square root of estimated tau^2 value):             0.1487
## I^2 (residual heterogeneity / unaccounted variability): 40.50%
## H^2 (unaccounted variability / sampling variability):   1.68
## R^2 (amount of heterogeneity accounted for):            66.72%
## 
## Test for Residual Heterogeneity:
## QE(df = 61) = 108.9371, p-val = 0.0002
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 34.2387, p-val < .0001
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt    0.1072  0.0471  2.2748  0.0229   0.0148  0.1996    * 
## typeLRP    0.4204  0.0725  5.7974  <.0001   0.2783  0.5625  *** 
## typeLRD    0.1061  0.0954  1.1123  0.2660  -0.0808  0.2930      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Location outside of the U.S.:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0696 (SE = 0.0216)
## tau (square root of estimated tau^2 value):             0.2638
## I^2 (residual heterogeneity / unaccounted variability): 68.94%
## H^2 (unaccounted variability / sampling variability):   3.22
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 160.4784, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0005, p-val = 0.9814
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt       0.2983  0.0685  4.3547  <.0001   0.1640  0.4325  *** 
## outside_us    0.0021  0.0907  0.0234  0.9814  -0.1757  0.1799      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students from low-income backgrounds included:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0678 (SE = 0.0212)
## tau (square root of estimated tau^2 value):             0.2604
## I^2 (residual heterogeneity / unaccounted variability): 68.36%
## H^2 (unaccounted variability / sampling variability):   3.16
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 160.5543, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4650, p-val = 0.4953
## 
## Model Results:
## 
##                      estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                0.2741  0.0575  4.7703  <.0001   0.1615  0.3868  *** 
## low_income_included    0.0620  0.0909  0.6819  0.4953  -0.1162  0.2402      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students learning a second or other language included:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0697 (SE = 0.0216)
## tau (square root of estimated tau^2 value):             0.2641
## I^2 (residual heterogeneity / unaccounted variability): 69.47%
## H^2 (unaccounted variability / sampling variability):   3.28
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 160.3842, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1988, p-val = 0.6557
## 
## Model Results:
## 
##                       estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt                 0.3116  0.0524   5.9466  <.0001   0.2089  0.4142  *** 
## second_lang_included   -0.0454  0.1019  -0.4459  0.6557  -0.2451  0.1542      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students with disabilities included:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0669 (SE = 0.0210)
## tau (square root of estimated tau^2 value):             0.2586
## I^2 (residual heterogeneity / unaccounted variability): 68.72%
## H^2 (unaccounted variability / sampling variability):   3.20
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 156.5131, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.5365, p-val = 0.4639
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                    0.2864  0.0474  6.0461  <.0001   0.1935  0.3792  *** 
## stu_disability_included    0.0987  0.1347  0.7324  0.4639  -0.1654  0.3627      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Grade level K-2 vs. 3-5:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0694 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2634
## I^2 (residual heterogeneity / unaccounted variability): 68.87%
## H^2 (unaccounted variability / sampling variability):   3.21
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 160.2927, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1834, p-val = 0.6684
## 
## Model Results:
## 
##           estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt     0.2815  0.0614  4.5878  <.0001   0.1612  0.4017  *** 
## gradeK-2    0.0385  0.0899  0.4283  0.6684  -0.1377  0.2148      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial, Cognitive, Constructivist:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0399 (SE = 0.0151)
## tau (square root of estimated tau^2 value):             0.1997
## I^2 (residual heterogeneity / unaccounted variability): 56.13%
## H^2 (unaccounted variability / sampling variability):   2.28
## R^2 (amount of heterogeneity accounted for):            39.98%
## 
## Test for Residual Heterogeneity:
## QE(df = 61) = 131.1784, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 15.4400, p-val = 0.0004
## 
## Model Results:
## 
##                         estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                   0.1915  0.0602  3.1798  0.0015   0.0735  0.3096   ** 
## pedagogyCognitive         0.0835  0.0809  1.0319  0.3021  -0.0751  0.2422      
## pedagogyConstructivist    0.5219  0.1332  3.9176  <.0001   0.2608  0.7830  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0630 (SE = 0.0202)
## tau (square root of estimated tau^2 value):             0.2511
## I^2 (residual heterogeneity / unaccounted variability): 66.78%
## H^2 (unaccounted variability / sampling variability):   3.01
## R^2 (amount of heterogeneity accounted for):            5.13%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 153.7979, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.7076, p-val = 0.0300
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt       0.3818  0.0584   6.5342  <.0001   0.2673   0.4963  *** 
## tech_plus1   -0.1901  0.0876  -2.1697  0.0300  -0.3618  -0.0184    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Type of technology (CAI, ER, Other):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0344 (SE = 0.0137)
## tau (square root of estimated tau^2 value):             0.1855
## I^2 (residual heterogeneity / unaccounted variability): 53.17%
## H^2 (unaccounted variability / sampling variability):   2.14
## R^2 (amount of heterogeneity accounted for):            48.23%
## 
## Test for Residual Heterogeneity:
## QE(df = 61) = 123.2826, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 21.7791, p-val < .0001
## 
## Model Results:
## 
##                estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt          0.2045  0.0428  4.7781  <.0001   0.1206  0.2883  *** 
## typetechER       0.6574  0.1460  4.5013  <.0001   0.3712  0.9437  *** 
## typetechOther    0.1696  0.0927  1.8291  0.0674  -0.0121  0.3514    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Gamification Discussed (yes/no):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0670 (SE = 0.0210)
## tau (square root of estimated tau^2 value):             0.2588
## I^2 (residual heterogeneity / unaccounted variability): 68.09%
## H^2 (unaccounted variability / sampling variability):   3.13
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 159.4316, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.9403, p-val = 0.0864
## 
## Model Results:
## 
##               estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt         0.3591  0.0567   6.3360  <.0001   0.2480  0.4702  *** 
## gamification   -0.1562  0.0911  -1.7147  0.0864  -0.3347  0.0223    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Adaptivity Discussed (yes/no):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0634 (SE = 0.0203)
## tau (square root of estimated tau^2 value):             0.2518
## I^2 (residual heterogeneity / unaccounted variability): 66.52%
## H^2 (unaccounted variability / sampling variability):   2.99
## R^2 (amount of heterogeneity accounted for):            4.54%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 153.4762, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.8555, p-val = 0.0911
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.3301  0.0477   6.9171  <.0001   0.2365  0.4236  *** 
## adaptivity   -0.1989  0.1177  -1.6898  0.0911  -0.4295  0.0318    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Feedback Discussed (yes/no):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0599 (SE = 0.0194)
## tau (square root of estimated tau^2 value):             0.2447
## I^2 (residual heterogeneity / unaccounted variability): 65.58%
## H^2 (unaccounted variability / sampling variability):   2.91
## R^2 (amount of heterogeneity accounted for):            9.90%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 155.7679, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.9735, p-val = 0.0462
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt     0.4102  0.0715   5.7351  <.0001   0.2700   0.5504  *** 
## feedback   -0.1781  0.0893  -1.9934  0.0462  -0.3532  -0.0030    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Instructional Context:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0670 (SE = 0.0210)
## tau (square root of estimated tau^2 value):             0.2588
## I^2 (residual heterogeneity / unaccounted variability): 68.51%
## H^2 (unaccounted variability / sampling variability):   3.18
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 158.6085, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.0349, p-val = 0.1537
## 
## Model Results:
## 
##                                             estimate      se     zval    pval 
## intrcpt                                       0.3442  0.0547   6.2960  <.0001 
## instructional_contextOther than individual   -0.1335  0.0936  -1.4265  0.1537 
##                                               ci.lb   ci.ub      
## intrcpt                                      0.2370  0.4513  *** 
## instructional_contextOther than individual  -0.3168  0.0499      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Teacher Training Discussed:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0692 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2630
## I^2 (residual heterogeneity / unaccounted variability): 69.17%
## H^2 (unaccounted variability / sampling variability):   3.24
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 159.5572, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.6301, p-val = 0.2017
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt     0.3734  0.0733   5.0936  <.0001   0.2297  0.5171  *** 
## teach_tr   -0.1183  0.0926  -1.2768  0.2017  -0.2998  0.0633      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Design (RCT/QED):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0694 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2635
## I^2 (residual heterogeneity / unaccounted variability): 68.78%
## H^2 (unaccounted variability / sampling variability):   3.20
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 159.4851, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1673, p-val = 0.6825
## 
## Model Results:
## 
##            estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt      0.2729  0.0789  3.4580  0.0005   0.1182  0.4276  *** 
## designRCT    0.0392  0.0959  0.4091  0.6825  -0.1488  0.2273      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (BAU/ALT):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0692 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2631
## I^2 (residual heterogeneity / unaccounted variability): 68.57%
## H^2 (unaccounted variability / sampling variability):   3.18
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 160.5183, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.2508, p-val = 0.6165
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.2563  0.0971  2.6396  0.0083   0.0660  0.4465  ** 
## controlBAU    0.0548  0.1094  0.5008  0.6165  -0.1597  0.2693     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Duration:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0627 (SE = 0.0201)
## tau (square root of estimated tau^2 value):             0.2503
## I^2 (residual heterogeneity / unaccounted variability): 67.01%
## H^2 (unaccounted variability / sampling variability):   3.03
## R^2 (amount of heterogeneity accounted for):            5.69%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 153.6947, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.7262, p-val = 0.0987
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.3663  0.0604   6.0672  <.0001   0.2480  0.4846  *** 
## avghrs    -0.0042  0.0025  -1.6511  0.0987  -0.0092  0.0008    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

3.5 Writing

Measure type (Custom distal, Standardized):

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0249 (SE = 0.0991)
## tau (square root of estimated tau^2 value):             0.1577
## I^2 (residual heterogeneity / unaccounted variability): 17.51%
## H^2 (unaccounted variability / sampling variability):   1.21
## R^2 (amount of heterogeneity accounted for):            83.10%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 5.9540, p-val = 0.2026
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.4549, p-val = 0.0348
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt     1.0224  0.1864   5.4835  <.0001   0.6569   1.3878  *** 
## typeWPSN   -0.6793  0.3218  -2.1107  0.0348  -1.3101  -0.0485    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Location outside of the U.S.:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2015 (SE = 0.2308)
## tau (square root of estimated tau^2 value):             0.4489
## I^2 (residual heterogeneity / unaccounted variability): 62.71%
## H^2 (unaccounted variability / sampling variability):   2.68
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 10.0606, p-val = 0.0394
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.3019, p-val = 0.5827
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt       1.0894  0.5435   2.0044  0.0450   0.0241  2.1546  * 
## outside_us   -0.3301  0.6007  -0.5495  0.5827  -1.5074  0.8473    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students from low-income backgrounds included:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2015 (SE = 0.2308)
## tau (square root of estimated tau^2 value):             0.4489
## I^2 (residual heterogeneity / unaccounted variability): 62.71%
## H^2 (unaccounted variability / sampling variability):   2.68
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 10.0606, p-val = 0.0394
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.3019, p-val = 0.5827
## 
## Model Results:
## 
##                      estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt                0.7593  0.2558  2.9686  0.0030   0.2580  1.2607  ** 
## low_income_included    0.3301  0.6007  0.5495  0.5827  -0.8473  1.5074     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students learning a second or other language included:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2015 (SE = 0.2308)
## tau (square root of estimated tau^2 value):             0.4489
## I^2 (residual heterogeneity / unaccounted variability): 62.71%
## H^2 (unaccounted variability / sampling variability):   2.68
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 10.0606, p-val = 0.0394
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.3019, p-val = 0.5827
## 
## Model Results:
## 
##                       estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt                 0.7593  0.2558  2.9686  0.0030   0.2580  1.2607  ** 
## second_lang_included    0.3301  0.6007  0.5495  0.5827  -0.8473  1.5074     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Students with disabilities included:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Grade level K-2 vs. 3-5:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0559 (SE = 0.1216)
## tau (square root of estimated tau^2 value):             0.2364
## I^2 (residual heterogeneity / unaccounted variability): 32.48%
## H^2 (unaccounted variability / sampling variability):   1.48
## R^2 (amount of heterogeneity accounted for):            62.02%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 6.4850, p-val = 0.1657
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.4652, p-val = 0.0627
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt     1.1556  0.2546   4.5398  <.0001   0.6567  1.6545  *** 
## gradeK-2   -0.6339  0.3406  -1.8615  0.0627  -1.3014  0.0335    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial, Cognitive, Constructivist:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2459 (SE = 0.2667)
## tau (square root of estimated tau^2 value):             0.4959
## I^2 (residual heterogeneity / unaccounted variability): 65.91%
## H^2 (unaccounted variability / sampling variability):   2.93
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 11.2734, p-val = 0.0237
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0035, p-val = 0.9525
## 
## Model Results:
## 
##                    estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt              0.7919  0.5648  1.4021  0.1609  -0.3151  1.8989    
## pedagogyCognitive    0.0374  0.6281  0.0595  0.9525  -1.1937  1.2685    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Type of technology (CAI, ER, Other):

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2459 (SE = 0.2667)
## tau (square root of estimated tau^2 value):             0.4959
## I^2 (residual heterogeneity / unaccounted variability): 65.91%
## H^2 (unaccounted variability / sampling variability):   2.93
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 11.2734, p-val = 0.0237
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0035, p-val = 0.9525
## 
## Model Results:
## 
##                estimate      se     zval    pval    ci.lb   ci.ub     
## intrcpt          0.8293  0.2748   3.0177  0.0025   0.2907  1.3680  ** 
## typetechOther   -0.0374  0.6281  -0.0595  0.9525  -1.2685  1.1937     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Gamification Discussed (yes/no):

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0249 (SE = 0.0991)
## tau (square root of estimated tau^2 value):             0.1577
## I^2 (residual heterogeneity / unaccounted variability): 17.51%
## H^2 (unaccounted variability / sampling variability):   1.21
## R^2 (amount of heterogeneity accounted for):            83.10%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 5.9540, p-val = 0.2026
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.4549, p-val = 0.0348
## 
## Model Results:
## 
##               estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt         1.0224  0.1864   5.4835  <.0001   0.6569   1.3878  *** 
## gamification   -0.6793  0.3218  -2.1107  0.0348  -1.3101  -0.0485    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Adaptivity Discussed (yes/no):

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Feedback Discussed (yes/no):

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2459 (SE = 0.2667)
## tau (square root of estimated tau^2 value):             0.4959
## I^2 (residual heterogeneity / unaccounted variability): 65.91%
## H^2 (unaccounted variability / sampling variability):   2.93
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 11.2734, p-val = 0.0237
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0035, p-val = 0.9525
## 
## Model Results:
## 
##           estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt     0.7919  0.5648  1.4021  0.1609  -0.3151  1.8989    
## feedback    0.0374  0.6281  0.0595  0.9525  -1.1937  1.2685    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Instructional Context:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Teacher Training Discussed:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0091 (SE = 0.0789)
## tau (square root of estimated tau^2 value):             0.0954
## I^2 (residual heterogeneity / unaccounted variability): 7.99%
## H^2 (unaccounted variability / sampling variability):   1.09
## R^2 (amount of heterogeneity accounted for):            93.81%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 4.0406, p-val = 0.4005
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 6.8981, p-val = 0.0086
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt     1.9036  0.4472   4.2562  <.0001   1.0270   2.7802  *** 
## teach_tr   -1.2390  0.4717  -2.6264  0.0086  -2.1636  -0.3144   ** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Design (RCT/QED):

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0249 (SE = 0.0991)
## tau (square root of estimated tau^2 value):             0.1577
## I^2 (residual heterogeneity / unaccounted variability): 17.51%
## H^2 (unaccounted variability / sampling variability):   1.21
## R^2 (amount of heterogeneity accounted for):            83.10%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 5.9540, p-val = 0.2026
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.4549, p-val = 0.0348
## 
## Model Results:
## 
##            estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt      0.3431  0.2623  1.3078  0.1909  -0.1711  0.8572    
## designRCT    0.6793  0.3218  2.1107  0.0348   0.0485  1.3101  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (BAU/ALT):

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1472 (SE = 0.1675)
## tau (square root of estimated tau^2 value):      0.3836
## I^2 (total heterogeneity / total variability):   56.36%
## H^2 (total variability / sampling variability):  2.29
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.2738, p-val = 0.0462
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.8144  0.2106  3.8678  0.0001  0.4017  1.2271  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Duration:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2139 (SE = 0.2328)
## tau (square root of estimated tau^2 value):             0.4625
## I^2 (residual heterogeneity / unaccounted variability): 65.73%
## H^2 (unaccounted variability / sampling variability):   2.92
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 11.0571, p-val = 0.0259
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4211, p-val = 0.5164
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt    0.5679  0.4546  1.2493  0.2116  -0.3231  1.4589    
## avghrs     0.0096  0.0147  0.6489  0.5164  -0.0193  0.0384    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4 Additional Moderator Analysis

4.1 Pairwise comparison by outcome type

Code-related

DRD vs DRP [No statistically significant difference]:

## 
## Mixed-Effects Model (k = 41; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2205 (SE = 0.0728)
## tau (square root of estimated tau^2 value):             0.4695
## I^2 (residual heterogeneity / unaccounted variability): 80.12%
## H^2 (unaccounted variability / sampling variability):   5.03
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 39) = 138.7581, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0683, p-val = 0.7938
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.4789  0.1082   4.4249  <.0001   0.2668  0.6911  *** 
## typeDRP   -0.0509  0.1949  -0.2613  0.7938  -0.4329  0.3311      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

DRD vs DSN [Statistically significant difference of -0.2741, p = 0.0055]:

## 
## Mixed-Effects Model (k = 74; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1263 (SE = 0.0336)
## tau (square root of estimated tau^2 value):             0.3554
## I^2 (residual heterogeneity / unaccounted variability): 76.00%
## H^2 (unaccounted variability / sampling variability):   4.17
## R^2 (amount of heterogeneity accounted for):            3.61%
## 
## Test for Residual Heterogeneity:
## QE(df = 72) = 201.9290, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.5466, p-val = 0.0597
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.4589  0.0900   5.1014  <.0001   0.2826  0.6352  *** 
## typeDSN   -0.2107  0.1119  -1.8832  0.0597  -0.4299  0.0086    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

DSN vs DRP [No statistically significant difference]:

## 
## Mixed-Effects Model (k = 57; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0465 (SE = 0.0189)
## tau (square root of estimated tau^2 value):             0.2156
## I^2 (residual heterogeneity / unaccounted variability): 56.41%
## H^2 (unaccounted variability / sampling variability):   2.29
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 55) = 111.9805, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.8076, p-val = 0.1788
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.3802  0.0999   3.8056  0.0001   0.1844  0.5760  *** 
## typeDSN   -0.1496  0.1112  -1.3445  0.1788  -0.3676  0.0685      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Reading Comprehension

CRD vs CSN [Statistically significant difference of -0.1806, p = 0.0193]:

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0205 (SE = 0.0092)
## tau (square root of estimated tau^2 value):             0.1431
## I^2 (residual heterogeneity / unaccounted variability): 52.89%
## H^2 (unaccounted variability / sampling variability):   2.12
## R^2 (amount of heterogeneity accounted for):            41.89%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 126.0652, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 10.9205, p-val = 0.0010
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt    0.3698  0.0541   6.8336  <.0001   0.2637   0.4759  *** 
## typeCSN   -0.2184  0.0661  -3.3046  0.0010  -0.3480  -0.0889  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Language-focused

LRD vs LRP [No statistically significant difference]:

## 
## Mixed-Effects Model (k = 36; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1049 (SE = 0.0408)
## tau (square root of estimated tau^2 value):             0.3238
## I^2 (residual heterogeneity / unaccounted variability): 72.14%
## H^2 (unaccounted variability / sampling variability):   3.59
## R^2 (amount of heterogeneity accounted for):            17.03%
## 
## Test for Residual Heterogeneity:
## QE(df = 34) = 97.8376, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.4927, p-val = 0.0340
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt    0.2305  0.1395  1.6526  0.0984  -0.0429  0.5038  . 
## typeLRP    0.3419  0.1613  2.1196  0.0340   0.0257  0.6581  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

LRD vs LSN [No statistically significant difference]:

## 
## Mixed-Effects Model (k = 35; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0 (SE = 0.0041)
## tau (square root of estimated tau^2 value):             0
## I^2 (residual heterogeneity / unaccounted variability): 0.00%
## H^2 (unaccounted variability / sampling variability):   1.00
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 33) = 23.5321, p-val = 0.8880
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.1974, p-val = 0.2738
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.1811  0.0500   3.6229  0.0003   0.0831  0.2791  *** 
## typeLSN   -0.0635  0.0581  -1.0943  0.2738  -0.1773  0.0503      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

LSN vs LRP [Statistically significant difference of -0.4423, p < 0.0001]:

## 
## Mixed-Effects Model (k = 57; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0231 (SE = 0.0121)
## tau (square root of estimated tau^2 value):             0.1521
## I^2 (residual heterogeneity / unaccounted variability): 40.16%
## H^2 (unaccounted variability / sampling variability):   1.67
## R^2 (amount of heterogeneity accounted for):            70.26%
## 
## Test for Residual Heterogeneity:
## QE(df = 55) = 96.5044, p-val = 0.0005
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 33.1884, p-val < .0001
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt    0.5288  0.0556   9.5176  <.0001   0.4199   0.6377  *** 
## typeLSN   -0.4218  0.0732  -5.7609  <.0001  -0.5652  -0.2783  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Writing

WPRD vs WPSN [Statistica;;y significant difference of -0.5852, p = 0.0294]:

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0249 (SE = 0.0991)
## tau (square root of estimated tau^2 value):             0.1577
## I^2 (residual heterogeneity / unaccounted variability): 17.51%
## H^2 (unaccounted variability / sampling variability):   1.21
## R^2 (amount of heterogeneity accounted for):            83.10%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 5.9540, p-val = 0.2026
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.4549, p-val = 0.0348
## 
## Model Results:
## 
##           estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt     1.0224  0.1864   5.4835  <.0001   0.6569   1.3878  *** 
## typeWPSN   -0.6793  0.3218  -2.1107  0.0348  -1.3101  -0.0485    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4.2 Time categories

Code-related

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1206 (SE = 0.0303)
## tau (square root of estimated tau^2 value):             0.3473
## I^2 (residual heterogeneity / unaccounted variability): 75.13%
## H^2 (unaccounted variability / sampling variability):   4.02
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 83) = 233.9699, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 5.5310, p-val = 0.0629
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt                    0.2256  0.0881  2.5610  0.0104   0.0530  0.3983  * 
## avghrs_categoriesMedium    0.0734  0.1153  0.6368  0.5242  -0.1525  0.2994    
## avghrs_categoriesHigh      0.2952  0.1302  2.2671  0.0234   0.0400  0.5505  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Reading Comprehension

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0368 (SE = 0.0135)
## tau (square root of estimated tau^2 value):             0.1919
## I^2 (residual heterogeneity / unaccounted variability): 63.31%
## H^2 (unaccounted variability / sampling variability):   2.73
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 147.2687, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 0.1495, p-val = 0.9280
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                    0.2577  0.1157   2.2265  0.0260   0.0309  0.4846  * 
## avghrs_categoriesMedium   -0.0186  0.1260  -0.1478  0.8825  -0.2656  0.2284    
## avghrs_categoriesHigh     -0.0427  0.1303  -0.3273  0.7434  -0.2981  0.2128    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Language-focused

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0696 (SE = 0.0218)
## tau (square root of estimated tau^2 value):             0.2638
## I^2 (residual heterogeneity / unaccounted variability): 68.82%
## H^2 (unaccounted variability / sampling variability):   3.21
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 61) = 154.7279, p-val < .0001
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 1.1883, p-val = 0.5520
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub 
## intrcpt                    0.3598  0.0713   5.0469  <.0001   0.2201  0.4996 
## avghrs_categoriesMedium   -0.1012  0.1024  -0.9878  0.3233  -0.3020  0.0996 
## avghrs_categoriesHigh     -0.0982  0.1175  -0.8351  0.4037  -0.3285  0.1322 
##                              
## intrcpt                  *** 
## avghrs_categoriesMedium      
## avghrs_categoriesHigh        
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Writing

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.3453 (SE = 0.3890)
## tau (square root of estimated tau^2 value):             0.5876
## I^2 (residual heterogeneity / unaccounted variability): 72.88%
## H^2 (unaccounted variability / sampling variability):   3.69
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 3) = 10.0455, p-val = 0.0182
## 
## Test of Moderators (coefficients 2:3):
## QM(df = 2) = 0.2204, p-val = 0.8956
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                    1.0894  0.6627   1.6437  0.1002  -0.2096  2.3883    
## avghrs_categoriesMedium   -0.3821  0.8170  -0.4678  0.6400  -1.9834  1.2191    
## avghrs_categoriesHigh     -0.2740  0.7750  -0.3536  0.7237  -1.7931  1.2450    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4.3 Time Continuous

Code-related

## 
## Mixed-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1226 (SE = 0.0305)
## tau (square root of estimated tau^2 value):             0.3502
## I^2 (residual heterogeneity / unaccounted variability): 75.40%
## H^2 (unaccounted variability / sampling variability):   4.07
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 84) = 233.9864, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.1755, p-val = 0.1402
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt    0.2694  0.0661  4.0765  <.0001   0.1399  0.3989  *** 
## avghrs     0.0029  0.0019  1.4750  0.1402  -0.0009  0.0067      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Reading Comprehension

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0313 (SE = 0.0121)
## tau (square root of estimated tau^2 value):             0.1769
## I^2 (residual heterogeneity / unaccounted variability): 55.48%
## H^2 (unaccounted variability / sampling variability):   2.25
## R^2 (amount of heterogeneity accounted for):            11.17%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 125.2354, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.3154, p-val = 0.1281
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.2858  0.0506   5.6438  <.0001   0.1866  0.3851  *** 
## avghrs    -0.0019  0.0012  -1.5216  0.1281  -0.0043  0.0005      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Language-focused

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0627 (SE = 0.0201)
## tau (square root of estimated tau^2 value):             0.2503
## I^2 (residual heterogeneity / unaccounted variability): 67.01%
## H^2 (unaccounted variability / sampling variability):   3.03
## R^2 (amount of heterogeneity accounted for):            5.69%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 153.6947, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 2.7262, p-val = 0.0987
## 
## Model Results:
## 
##          estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt    0.3663  0.0604   6.0672  <.0001   0.2480  0.4846  *** 
## avghrs    -0.0042  0.0025  -1.6511  0.0987  -0.0092  0.0008    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Writing

## 
## Mixed-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.2139 (SE = 0.2328)
## tau (square root of estimated tau^2 value):             0.4625
## I^2 (residual heterogeneity / unaccounted variability): 65.73%
## H^2 (unaccounted variability / sampling variability):   2.92
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 4) = 11.0571, p-val = 0.0259
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4211, p-val = 0.5164
## 
## Model Results:
## 
##          estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt    0.5679  0.4546  1.2493  0.2116  -0.3231  1.4589    
## avghrs     0.0096  0.0147  0.6489  0.5164  -0.0193  0.0384    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4.4 Time categories pairwise contrasts

Code-related

Low vs Medium:

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0332 (SE = 0.0158)
## tau (square root of estimated tau^2 value):             0.1822
## I^2 (residual heterogeneity / unaccounted variability): 43.85%
## H^2 (unaccounted variability / sampling variability):   1.78
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 102.7765, p-val = 0.0009
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.4273, p-val = 0.5133
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                    0.2176  0.0634  3.4307  0.0006   0.0933  0.3419  *** 
## avghrs_categoriesMedium    0.0534  0.0816  0.6537  0.5133  -0.1066  0.2133      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Medium vs High:

## 
## Mixed-Effects Model (k = 60; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1784 (SE = 0.0485)
## tau (square root of estimated tau^2 value):             0.4224
## I^2 (residual heterogeneity / unaccounted variability): 83.80%
## H^2 (unaccounted variability / sampling variability):   6.17
## R^2 (amount of heterogeneity accounted for):            0.81%
## 
## Test for Residual Heterogeneity:
## QE(df = 58) = 203.2912, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.0709, p-val = 0.0797
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub 
## intrcpt                    0.5494  0.1101   4.9912  <.0001   0.3336  0.7651 
## avghrs_categoriesMedium   -0.2437  0.1391  -1.7524  0.0797  -0.5162  0.0289 
##                              
## intrcpt                  *** 
## avghrs_categoriesMedium    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

High vs Low:

## 
## Mixed-Effects Model (k = 48; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1692 (SE = 0.0515)
## tau (square root of estimated tau^2 value):             0.4113
## I^2 (residual heterogeneity / unaccounted variability): 78.91%
## H^2 (unaccounted variability / sampling variability):   4.74
## R^2 (amount of heterogeneity accounted for):            3.86%
## 
## Test for Residual Heterogeneity:
## QE(df = 46) = 161.8721, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.7456, p-val = 0.0294
## 
## Model Results:
## 
##                        estimate      se    zval    pval   ci.lb   ci.ub    
## intrcpt                  0.2271  0.0987  2.3014  0.0214  0.0337  0.4205  * 
## avghrs_categoriesHigh    0.3186  0.1463  2.1784  0.0294  0.0320  0.6053  * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Reading Comprehension

Low vs Medium:

## 
## Mixed-Effects Model (k = 42; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0359 (SE = 0.0169)
## tau (square root of estimated tau^2 value):             0.1894
## I^2 (residual heterogeneity / unaccounted variability): 56.36%
## H^2 (unaccounted variability / sampling variability):   2.29
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 40) = 85.9264, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0226, p-val = 0.8806
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                    0.2578  0.1152   2.2371  0.0253   0.0319  0.4837  * 
## avghrs_categoriesMedium   -0.0188  0.1254  -0.1502  0.8806  -0.2646  0.2270    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Medium vs High:

## 
## Mixed-Effects Model (k = 55; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0410 (SE = 0.0149)
## tau (square root of estimated tau^2 value):             0.2024
## I^2 (residual heterogeneity / unaccounted variability): 68.43%
## H^2 (unaccounted variability / sampling variability):   3.17
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 53) = 141.9545, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0703, p-val = 0.7909
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                    0.2182  0.0619  3.5277  0.0004   0.0970  0.3395  *** 
## avghrs_categoriesMedium    0.0213  0.0805  0.2651  0.7909  -0.1364  0.1791      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

High vs Low:

## 
## Mixed-Effects Model (k = 33; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0316 (SE = 0.0186)
## tau (square root of estimated tau^2 value):             0.1779
## I^2 (residual heterogeneity / unaccounted variability): 61.51%
## H^2 (unaccounted variability / sampling variability):   2.60
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 31) = 66.6566, p-val = 0.0002
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1423, p-val = 0.7060
## 
## Model Results:
## 
##                        estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                  0.2582  0.1129   2.2865  0.0222   0.0369  0.4796  * 
## avghrs_categoriesHigh   -0.0478  0.1266  -0.3772  0.7060  -0.2960  0.2004    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Language-focused

Low vs Medium:

## 
## Mixed-Effects Model (k = 51; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0742 (SE = 0.0261)
## tau (square root of estimated tau^2 value):             0.2725
## I^2 (residual heterogeneity / unaccounted variability): 69.40%
## H^2 (unaccounted variability / sampling variability):   3.27
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 49) = 126.3989, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.9536, p-val = 0.3288
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub 
## intrcpt                    0.3612  0.0726   4.9744  <.0001   0.2189  0.5036 
## avghrs_categoriesMedium   -0.1021  0.1045  -0.9765  0.3288  -0.3070  0.1028 
##                              
## intrcpt                  *** 
## avghrs_categoriesMedium      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Medium vs High:

## 
## Mixed-Effects Model (k = 35; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0488 (SE = 0.0211)
## tau (square root of estimated tau^2 value):             0.2209
## I^2 (residual heterogeneity / unaccounted variability): 68.23%
## H^2 (unaccounted variability / sampling variability):   3.15
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 33) = 81.1598, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0054, p-val = 0.9417
## 
## Model Results:
## 
##                          estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt                    0.2481  0.0833  2.9794  0.0029   0.0849  0.4114  ** 
## avghrs_categoriesMedium    0.0078  0.1059  0.0732  0.9417  -0.1998  0.2153     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

High vs Low:

## 
## Mixed-Effects Model (k = 42; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0870 (SE = 0.0328)
## tau (square root of estimated tau^2 value):             0.2950
## I^2 (residual heterogeneity / unaccounted variability): 66.69%
## H^2 (unaccounted variability / sampling variability):   3.00
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 40) = 101.8970, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.5608, p-val = 0.4539
## 
## Model Results:
## 
##                        estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt                  0.3647  0.0761   4.7941  <.0001   0.2156  0.5138  *** 
## avghrs_categoriesHigh   -0.0947  0.1264  -0.7489  0.4539  -0.3424  0.1531      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Writing

Low vs Medium:

## 
## Mixed-Effects Model (k = 3; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0 (SE = 0.1624)
## tau (square root of estimated tau^2 value):             0
## I^2 (residual heterogeneity / unaccounted variability): 0.00%
## H^2 (unaccounted variability / sampling variability):   1.00
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 1) = 0.1510, p-val = 0.6975
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.8852, p-val = 0.3468
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub 
## intrcpt                    1.0894  0.3065   3.5546  0.0004   0.4887  1.6901 
## avghrs_categoriesMedium   -0.3567  0.3792  -0.9408  0.3468  -1.0999  0.3864 
##                              
## intrcpt                  *** 
## avghrs_categoriesMedium      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Medium vs High:

## 
## Mixed-Effects Model (k = 5; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.3453 (SE = 0.3890)
## tau (square root of estimated tau^2 value):             0.5876
## I^2 (residual heterogeneity / unaccounted variability): 72.88%
## H^2 (unaccounted variability / sampling variability):   3.69
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 3) = 10.0455, p-val = 0.0182
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0300, p-val = 0.8625
## 
## Model Results:
## 
##                          estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                    0.8153  0.4018   2.0291  0.0424   0.0278  1.6029  * 
## avghrs_categoriesMedium   -0.1081  0.6242  -0.1732  0.8625  -1.3315  1.1154    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

High vs Low:

## 
## Mixed-Effects Model (k = 4; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.6176 (SE = 0.7586)
## tau (square root of estimated tau^2 value):             0.7859
## I^2 (residual heterogeneity / unaccounted variability): 81.89%
## H^2 (unaccounted variability / sampling variability):   5.52
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 2) = 9.8944, p-val = 0.0071
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0679, p-val = 0.7944
## 
## Model Results:
## 
##                        estimate      se     zval    pval    ci.lb   ci.ub    
## intrcpt                  1.0894  0.8435   1.2915  0.1965  -0.5639  2.7427    
## avghrs_categoriesHigh   -0.2559  0.9819  -0.2606  0.7944  -2.1805  1.6686    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4.5 Multiple Meta Regressions

Code-related

	Model 1	Model 2	Model 3	Model 4	Model 5
intercept	0.247***	0.054	0.057	0.008	-0.081
	(0.065)	(0.103)	(0.108)	(0.139)	(0.144)
typeDRP	0.165	0.296*	0.198	0.206	0.251
	(0.145)	(0.148)	(0.163)	(0.165)	(0.164)
typeDRD	0.209	0.298**	0.256*	0.255*	0.267*
	(0.109)	(0.111)	(0.112)	(0.113)	(0.112)
avghrs_categoriesMedium		0.093	0.167	0.177	0.196
		(0.112)	(0.120)	(0.122)	(0.121)
avghrs_categoriesHigh		0.407**	0.508***	0.508***	0.453**
		(0.133)	(0.143)	(0.145)	(0.145)
pedagogyCognitive			-0.176	-0.178	-0.168
			(0.117)	(0.118)	(0.116)
pedagogyConstructivist			0.212	0.197	0.180
			(0.222)	(0.226)	(0.223)
gradeK-2				0.062	0.087
				(0.112)	(0.111)
low_income_included					0.245*
					(0.112)
i.squared	73.995	71.881	70.473	70.557	69.267
h.squared	3.845	3.556	3.387	3.396	3.254
tau.squared	0.117	0.110	0.105	0.108	0.104
tau.squared.se	0.030	0.029	0.028	0.029	0.029
cochran.qe	226.334	221.434	208.781	208.752	206.555
p.value.cochran.qe	0.000	0.000	0.000	0.000	0.000
cochran.qm	4.087	14.308	17.734	17.847	22.887
p.value.cochran.qm	0.130	0.006	0.007	0.013	0.004
DF Resid.	83	81	79	78	77
Log Likelihood	-57.530	-52.281	-50.503	-50.250	-47.774
Deviance	115.060	104.562	101.005	100.500	95.547
AIC	123.060	116.562	117.005	118.500	115.547
BIC	132.736	130.929	135.961	139.711	138.985
AICc	123.573	117.697	119.062	121.147	118.880
nobs	86	86	86	86	86
***p < 0.001; **p < 0.01; *p < 0.05

New table [Dec 2023]:

	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
intercept	0.247***	0.054	0.057	0.041	-0.010	-0.089
	(0.065)	(0.103)	(0.108)	(0.110)	(0.142)	(0.146)
typeDRP	0.165	0.296*	0.198	0.214	0.223	0.260
	(0.145)	(0.148)	(0.163)	(0.164)	(0.166)	(0.166)
typeDRD	0.209	0.298**	0.256*	0.258*	0.257*	0.268*
	(0.109)	(0.111)	(0.112)	(0.111)	(0.113)	(0.112)
avghrs_categoriesMedium		0.093	0.167	0.167	0.177	0.196
		(0.112)	(0.120)	(0.119)	(0.122)	(0.121)
avghrs_categoriesHigh		0.407**	0.508***	0.485***	0.484**	0.442**
		(0.133)	(0.143)	(0.147)	(0.149)	(0.149)
pedagogyCognitive			-0.176	-0.230	-0.233	-0.199
			(0.117)	(0.143)	(0.145)	(0.145)
pedagogyConstructivist			0.212	0.200	0.184	0.172
			(0.222)	(0.222)	(0.226)	(0.225)
tech_plus1				0.087	0.088	0.050
				(0.136)	(0.137)	(0.138)
gradeK-2					0.064	0.088
					(0.111)	(0.111)
low_income_included						0.240*
						(0.113)
i.squared	73.995	71.881	70.473	70.105	70.228	69.281
h.squared	3.845	3.556	3.387	3.345	3.359	3.255
tau.squared	0.117	0.110	0.105	0.103	0.106	0.104
tau.squared.se	0.030	0.029	0.028	0.028	0.029	0.029
cochran.qe	226.334	221.434	208.781	200.961	200.956	199.287
p.value.cochran.qe	0.000	0.000	0.000	0.000	0.000	0.000
cochran.qm	4.087	14.308	17.734	18.252	18.351	22.991
p.value.cochran.qm	0.130	0.006	0.007	0.011	0.019	0.006
DF Resid.	83	81	79	78	77	76
Log Likelihood	-57.530	-52.281	-50.503	-50.132	-49.876	-47.536
Deviance	115.060	104.562	101.005	100.264	99.752	95.072
AIC	123.060	116.562	117.005	118.264	119.752	117.072
BIC	132.736	130.929	135.961	139.474	143.190	142.710
AICc	123.573	117.697	119.062	120.911	123.085	121.197
nobs	86	86	86	86	86	86
***p < 0.001; **p < 0.01; *p < 0.05

Reading Comprehension

	Model 1	Model 2	Model 3	Model 4	Model 5
intercept	0.151***	0.142	0.152	0.179	0.150
	(0.038)	(0.114)	(0.117)	(0.120)	(0.130)
typeCRD	0.218***	0.228**	0.233**	0.229**	0.229**
	(0.066)	(0.072)	(0.073)	(0.072)	(0.073)
avghrs_categoriesMedium		-0.005	0.010	0.011	0.013
		(0.116)	(0.123)	(0.122)	(0.123)
avghrs_categoriesHigh		0.029	0.051	0.075	0.071
		(0.123)	(0.135)	(0.137)	(0.138)
pedagogyCognitive			-0.035	-0.056	-0.057
			(0.090)	(0.092)	(0.093)
gradeK-2				-0.066	-0.052
				(0.076)	(0.080)
low_income_included					0.041
					(0.070)
i.squared	52.895	51.306	50.820	46.403	46.695
h.squared	2.123	2.054	2.033	1.866	1.876
tau.squared	0.020	0.023	0.023	0.021	0.023
tau.squared.se	0.009	0.010	0.010	0.010	0.011
cochran.qe	126.065	119.472	114.995	105.008	105.002
p.value.cochran.qe	0.000	0.000	0.000	0.000	0.000
cochran.qm	10.921	10.376	10.577	11.765	11.680
p.value.cochran.qm	0.001	0.016	0.032	0.038	0.070
DF Resid.	63	61	60	59	58
Log Likelihood	-14.437	-14.996	-15.158	-15.191	-15.449
Deviance	28.873	29.993	30.316	30.383	30.898
AIC	34.873	39.993	42.316	44.383	46.898
BIC	41.303	50.547	54.882	58.925	63.382
AICc	35.280	41.084	43.901	46.579	49.837
nobs	65	65	65	65	65
***p < 0.001; **p < 0.01; *p < 0.05

New table [Dec 2023]:

	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
intercept	0.151***	0.142	0.152	0.235	0.238	0.208
	(0.038)	(0.114)	(0.117)	(0.121)	(0.123)	(0.133)
typeCRD	0.218***	0.228**	0.233**	0.204**	0.203**	0.203**
	(0.066)	(0.072)	(0.073)	(0.072)	(0.073)	(0.074)
avghrs_categoriesMedium		-0.005	0.010	-0.010	-0.010	-0.008
		(0.116)	(0.123)	(0.120)	(0.121)	(0.122)
avghrs_categoriesHigh		0.029	0.051	0.101	0.105	0.101
		(0.123)	(0.135)	(0.134)	(0.136)	(0.137)
pedagogyCognitive			-0.035	-0.058	-0.060	-0.061
			(0.090)	(0.088)	(0.090)	(0.091)
tech_plus1				-0.152*	-0.146	-0.148
				(0.074)	(0.079)	(0.080)
gradeK-2					-0.017	-0.002
					(0.080)	(0.084)
low_income_included						0.043
						(0.069)
i.squared	52.895	51.306	50.820	45.677	44.660	44.810
h.squared	2.123	2.054	2.033	1.841	1.807	1.812
tau.squared	0.020	0.023	0.023	0.019	0.020	0.021
tau.squared.se	0.009	0.010	0.010	0.009	0.010	0.010
cochran.qe	126.065	119.472	114.995	102.880	99.902	99.851
p.value.cochran.qe	0.000	0.000	0.000	0.000	0.001	0.000
cochran.qm	10.921	10.376	10.577	16.060	15.677	15.630
p.value.cochran.qm	0.001	0.016	0.032	0.007	0.016	0.029
DF Resid.	63	61	60	59	58	57
Log Likelihood	-14.437	-14.996	-15.158	-13.460	-13.818	-14.068
Deviance	28.873	29.993	30.316	26.920	27.636	28.136
AIC	34.873	39.993	42.316	40.920	43.636	46.136
BIC	41.303	50.547	54.882	55.463	60.120	64.524
AICc	35.280	41.084	43.901	43.116	46.575	49.966
nobs	65	65	65	65	65	65
***p < 0.001; **p < 0.01; *p < 0.05

Language-focused

	Model 1	Model 2	Model 3	Model 4	Model 5
intercept	0.107*	-0.098	-0.207*	-0.214*	-0.253*
	(0.047)	(0.100)	(0.092)	(0.095)	(0.108)
typeLRP	0.420***	0.563***	0.485***	0.492***	0.504***
	(0.073)	(0.095)	(0.079)	(0.081)	(0.082)
typeLRD	0.106	0.235*	0.200*	0.206*	0.219*
	(0.095)	(0.113)	(0.090)	(0.093)	(0.095)
avghrs_categoriesMedium		0.186*	0.256**	0.258**	0.266**
		(0.092)	(0.080)	(0.086)	(0.087)
avghrs_categoriesHigh		0.252*	0.249*	0.256*	0.270*
		(0.114)	(0.103)	(0.106)	(0.108)
pedagogyCognitive			0.121	0.120	0.119
			(0.065)	(0.067)	(0.067)
pedagogyConstructivist			0.476***	0.474***	0.456***
			(0.115)	(0.119)	(0.121)
gradeK-2				0.007	0.026
				(0.061)	(0.066)
low_income_included					0.049
					(0.066)
i.squared	40.504	42.082	20.601	22.456	23.126
h.squared	1.681	1.727	1.259	1.290	1.301
tau.squared	0.022	0.025	0.009	0.011	0.011
tau.squared.se	0.011	0.012	0.008	0.008	0.009
cochran.qe	108.937	106.272	82.487	82.314	81.718
p.value.cochran.qe	0.000	0.000	0.015	0.013	0.011
cochran.qm	34.239	38.572	64.494	63.008	62.986
p.value.cochran.qm	0.000	0.000	0.000	0.000	0.000
DF Resid.	61	59	57	56	55
Log Likelihood	-21.301	-19.345	-12.522	-13.083	-13.297
Deviance	42.603	38.689	25.044	26.165	26.595
AIC	50.603	50.689	41.044	44.165	46.595
BIC	59.046	63.155	57.388	62.393	66.668
AICc	51.317	52.305	44.044	48.078	51.595
nobs	64	64	64	64	64
***p < 0.001; **p < 0.01; *p < 0.05

New table [Dec 2023]:

	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
intercept	0.107*	-0.098	-0.207*	-0.191	-0.199	-0.238*
	(0.047)	(0.100)	(0.092)	(0.102)	(0.105)	(0.114)
typeLRP	0.420***	0.563***	0.485***	0.481***	0.488***	0.496***
	(0.073)	(0.095)	(0.079)	(0.084)	(0.086)	(0.087)
typeLRD	0.106	0.235*	0.200*	0.195*	0.200*	0.211*
	(0.095)	(0.113)	(0.090)	(0.095)	(0.099)	(0.100)
avghrs_categoriesMedium		0.186*	0.256**	0.256**	0.254**	0.260**
		(0.092)	(0.080)	(0.083)	(0.089)	(0.090)
avghrs_categoriesHigh		0.252*	0.249*	0.249*	0.254*	0.266*
		(0.114)	(0.103)	(0.107)	(0.110)	(0.112)
pedagogyCognitive			0.121	0.129	0.131	0.133
			(0.065)	(0.069)	(0.071)	(0.072)
pedagogyConstructivist			0.476***	0.468***	0.461***	0.435***
			(0.115)	(0.118)	(0.122)	(0.126)
tech_plus1				-0.036	-0.041	-0.057
				(0.064)	(0.067)	(0.070)
gradeK-2					0.017	0.043
					(0.063)	(0.070)
low_income_included						0.063
						(0.070)
i.squared	40.504	42.082	20.601	22.651	24.933	25.467
h.squared	1.681	1.727	1.259	1.293	1.332	1.342
tau.squared	0.022	0.025	0.009	0.011	0.013	0.013
tau.squared.se	0.011	0.012	0.008	0.008	0.009	0.010
cochran.qe	108.937	106.272	82.487	82.392	82.229	81.432
p.value.cochran.qe	0.000	0.000	0.015	0.012	0.010	0.009
cochran.qm	34.239	38.572	64.494	63.177	61.391	61.737
p.value.cochran.qm	0.000	0.000	0.000	0.000	0.000	0.000
DF Resid.	61	59	57	56	55	54
Log Likelihood	-21.301	-19.345	-12.522	-12.948	-13.466	-13.527
Deviance	42.603	38.689	25.044	25.896	26.931	27.053
AIC	50.603	50.689	41.044	43.896	46.931	49.053
BIC	59.046	63.155	57.388	62.124	67.005	70.932
AICc	51.317	52.305	44.044	47.809	51.931	55.339
nobs	64	64	64	64	64	64
***p < 0.001; **p < 0.01; *p < 0.05

---

4.6 Pedagogy pairwise contrasts

Code-related

Pedagogical Orientation: Behavorial vs Cognitive

## 
## Mixed-Effects Model (k = 80; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1217 (SE = 0.0314)
## tau (square root of estimated tau^2 value):             0.3488
## I^2 (residual heterogeneity / unaccounted variability): 76.14%
## H^2 (unaccounted variability / sampling variability):   4.19
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 78) = 213.1435, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6764, p-val = 0.4108
## 
## Model Results:
## 
##                    estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt              0.3515  0.0608   5.7827  <.0001   0.2323  0.4706  *** 
## pedagogyCognitive   -0.0909  0.1106  -0.8224  0.4108  -0.3077  0.1258      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial vs Constructivist

## 
## Mixed-Effects Model (k = 66; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1437 (SE = 0.0415)
## tau (square root of estimated tau^2 value):             0.3791
## I^2 (residual heterogeneity / unaccounted variability): 72.25%
## H^2 (unaccounted variability / sampling variability):   3.60
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 64) = 169.5401, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.3271, p-val = 0.5674
## 
## Model Results:
## 
##                         estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt                   0.3559  0.0640  5.5573  <.0001   0.2304  0.4814  *** 
## pedagogyConstructivist    0.1193  0.2086  0.5719  0.5674  -0.2895  0.5280      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Cognitive vs Constructivist

## 
## Mixed-Effects Model (k = 26; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0674 (SE = 0.0318)
## tau (square root of estimated tau^2 value):             0.2596
## I^2 (residual heterogeneity / unaccounted variability): 73.96%
## H^2 (unaccounted variability / sampling variability):   3.84
## R^2 (amount of heterogeneity accounted for):            12.69%
## 
## Test for Residual Heterogeneity:
## QE(df = 24) = 64.8721, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.7923, p-val = 0.1807
## 
## Model Results:
## 
##                    estimate      se     zval    pval    ci.lb   ci.ub     
## intrcpt              0.4807  0.1601   3.0025  0.0027   0.1669  0.7945  ** 
## pedagogyCognitive   -0.2363  0.1765  -1.3388  0.1807  -0.5823  0.1097     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Language-focused

Pedagogical Orientation: Behavorial vs Cognitive

## 
## Mixed-Effects Model (k = 56; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0314 (SE = 0.0135)
## tau (square root of estimated tau^2 value):             0.1773
## I^2 (residual heterogeneity / unaccounted variability): 51.85%
## H^2 (unaccounted variability / sampling variability):   2.08
## R^2 (amount of heterogeneity accounted for):            3.54%
## 
## Test for Residual Heterogeneity:
## QE(df = 54) = 106.6570, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.2482, p-val = 0.2639
## 
## Model Results:
## 
##                    estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt              0.1876  0.0561  3.3415  0.0008   0.0776  0.2976  *** 
## pedagogyCognitive    0.0847  0.0758  1.1172  0.2639  -0.0639  0.2333      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Behavorial vs Constructivist

## 
## Mixed-Effects Model (k = 33; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0060 (SE = 0.0074)
## tau (square root of estimated tau^2 value):             0.0771
## I^2 (residual heterogeneity / unaccounted variability): 17.80%
## H^2 (unaccounted variability / sampling variability):   1.22
## R^2 (amount of heterogeneity accounted for):            91.81%
## 
## Test for Residual Heterogeneity:
## QE(df = 31) = 54.9354, p-val = 0.0051
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 25.2766, p-val < .0001
## 
## Model Results:
## 
##                         estimate      se    zval    pval   ci.lb   ci.ub      
## intrcpt                   0.1757  0.0381  4.6085  <.0001  0.1010  0.2504  *** 
## pedagogyConstructivist    0.5080  0.1010  5.0276  <.0001  0.3100  0.7061  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Pedagogical Orientation: Cognitive vs Constructivist

## 
## Mixed-Effects Model (k = 39; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0815 (SE = 0.0316)
## tau (square root of estimated tau^2 value):             0.2854
## I^2 (residual heterogeneity / unaccounted variability): 67.30%
## H^2 (unaccounted variability / sampling variability):   3.06
## R^2 (amount of heterogeneity accounted for):            25.43%
## 
## Test for Residual Heterogeneity:
## QE(df = 37) = 100.7645, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 8.3456, p-val = 0.0039
## 
## Model Results:
## 
##                    estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt              0.7343  0.1418   5.1796  <.0001   0.4565   1.0122  *** 
## pedagogyCognitive   -0.4526  0.1567  -2.8889  0.0039  -0.7596  -0.1455   ** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

4.7 Tech type pairwise contrasts

Code-related

Tech type: CAI vs ER

## 
## Mixed-Effects Model (k = 84; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1208 (SE = 0.0302)
## tau (square root of estimated tau^2 value):             0.3476
## I^2 (residual heterogeneity / unaccounted variability): 76.44%
## H^2 (unaccounted variability / sampling variability):   4.24
## R^2 (amount of heterogeneity accounted for):            0.14%
## 
## Test for Residual Heterogeneity:
## QE(df = 82) = 228.0506, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6217, p-val = 0.4304
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt       0.3245  0.0508  6.3897  <.0001   0.2250  0.4240  *** 
## typetechER    0.1592  0.2019  0.7885  0.4304  -0.2365  0.5549      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: CAI vs Other

## 
## Mixed-Effects Model (k = 81; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1203 (SE = 0.0308)
## tau (square root of estimated tau^2 value):             0.3469
## I^2 (residual heterogeneity / unaccounted variability): 76.57%
## H^2 (unaccounted variability / sampling variability):   4.27
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 79) = 217.5073, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0051, p-val = 0.9433
## 
## Model Results:
## 
##                estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt          0.3244  0.0507   6.3961  <.0001   0.2250  0.4238  *** 
## typetechOther   -0.0355  0.4997  -0.0711  0.9433  -1.0148  0.9438      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: ER vs Other

## 
## Mixed-Effects Model (k = 7; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0977 (SE = 0.1164)
## tau (square root of estimated tau^2 value):             0.3126
## I^2 (residual heterogeneity / unaccounted variability): 54.73%
## H^2 (unaccounted variability / sampling variability):   2.21
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 5) = 10.6592, p-val = 0.0586
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1422, p-val = 0.7061
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt       0.2891  0.4855  0.5955  0.5515  -0.6625  1.2408    
## typetechER    0.1957  0.5189  0.3771  0.7061  -0.8213  1.2127    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Language focused

Tech type: CAI vs ER

## 
## Mixed-Effects Model (k = 50; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0281 (SE = 0.0132)
## tau (square root of estimated tau^2 value):             0.1677
## I^2 (residual heterogeneity / unaccounted variability): 50.76%
## H^2 (unaccounted variability / sampling variability):   2.03
## R^2 (amount of heterogeneity accounted for):            55.84%
## 
## Test for Residual Heterogeneity:
## QE(df = 48) = 94.7378, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 21.4966, p-val < .0001
## 
## Model Results:
## 
##             estimate      se    zval    pval   ci.lb   ci.ub      
## intrcpt       0.2023  0.0405  4.9961  <.0001  0.1230  0.2817  *** 
## typetechER    0.6544  0.1411  4.6364  <.0001  0.3778  0.9310  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: CAI vs Other

## 
## Mixed-Effects Model (k = 58; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0284 (SE = 0.0124)
## tau (square root of estimated tau^2 value):             0.1686
## I^2 (residual heterogeneity / unaccounted variability): 49.76%
## H^2 (unaccounted variability / sampling variability):   1.99
## R^2 (amount of heterogeneity accounted for):            17.34%
## 
## Test for Residual Heterogeneity:
## QE(df = 56) = 106.7119, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 3.6617, p-val = 0.0557
## 
## Model Results:
## 
##                estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt          0.2024  0.0406  4.9845  <.0001   0.1228  0.2820  *** 
## typetechOther    0.1700  0.0889  1.9136  0.0557  -0.0041  0.3442    . 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: ER vs Other

## 
## Mixed-Effects Model (k = 20; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1011 (SE = 0.0566)
## tau (square root of estimated tau^2 value):             0.3180
## I^2 (residual heterogeneity / unaccounted variability): 64.03%
## H^2 (unaccounted variability / sampling variability):   2.78
## R^2 (amount of heterogeneity accounted for):            33.15%
## 
## Test for Residual Heterogeneity:
## QE(df = 18) = 45.1155, p-val = 0.0004
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 6.1142, p-val = 0.0134
## 
## Model Results:
## 
##             estimate      se    zval    pval   ci.lb   ci.ub      
## intrcpt       0.3850  0.1108  3.4754  0.0005  0.1679  0.6022  *** 
## typetechER    0.5196  0.2101  2.4727  0.0134  0.1077  0.9315    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

Reading Comprehension

Tech type: CAI vs ER

## 
## Random-Effects Model (k = 60; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0371 (SE = 0.0135)
## tau (square root of estimated tau^2 value):      0.1925
## I^2 (total heterogeneity / total variability):   72.01%
## H^2 (total variability / sampling variability):  3.57
## 
## Test for Heterogeneity:
## Q(df = 59) = 206.1196, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2396  0.0375  6.3921  <.0001  0.1661  0.3130  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: CAI vs Other

## 
## Mixed-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0359 (SE = 0.0130)
## tau (square root of estimated tau^2 value):             0.1894
## I^2 (residual heterogeneity / unaccounted variability): 70.24%
## H^2 (unaccounted variability / sampling variability):   3.36
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 63) = 209.4629, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.7222, p-val = 0.3954
## 
## Model Results:
## 
##                estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt          0.2390  0.0371   6.4388  <.0001   0.1663  0.3118  *** 
## typetechOther   -0.1358  0.1598  -0.8498  0.3954  -0.4490  0.1774      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Tech type: ER vs Other

## 
## Random-Effects Model (k = 5; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0 (SE = 0.0554)
## tau (square root of estimated tau^2 value):      0
## I^2 (total heterogeneity / total variability):   0.00%
## H^2 (total variability / sampling variability):  1.00
## 
## Test for Heterogeneity:
## Q(df = 4) = 3.3433, p-val = 0.5021
## 
## Model Results:
## 
## estimate      se    zval    pval    ci.lb   ci.ub    
##   0.0993  0.1268  0.7832  0.4335  -0.1493  0.3479    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

5 Publication Bias

5.1 Funnel Plots

Code-related:

Reading Comprehension:

Language-focused:

Writing:

5.2 Egger’s Tests

Code-related:

## 
## Regression Test for Funnel Plot Asymmetry
## 
## Model:     mixed-effects meta-regression model
## Predictor: standard error
## 
## Test for Funnel Plot Asymmetry: z = 2.7592, p = 0.0058

Reading Comprehension:

## 
## Regression Test for Funnel Plot Asymmetry
## 
## Model:     mixed-effects meta-regression model
## Predictor: standard error
## 
## Test for Funnel Plot Asymmetry: z = 1.8285, p = 0.0675

Language-focused:

## 
## Regression Test for Funnel Plot Asymmetry
## 
## Model:     mixed-effects meta-regression model
## Predictor: standard error
## 
## Test for Funnel Plot Asymmetry: z = 1.7297, p = 0.0837

Writing:

## 
## Regression Test for Funnel Plot Asymmetry
## 
## Model:     mixed-effects meta-regression model
## Predictor: standard error
## 
## Test for Funnel Plot Asymmetry: z = 1.1436, p = 0.2528

5.3 P-curves

Code-related:

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 86 
## - Total number of p<0.05 studies included into the analysis: k = 22 (25.58%) 
## - Total number of studies with p<0.025: k = 22 (25.58%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull zHalf pHalf
## Right-skewness test         0 -9.077     0 -6.50     0
## Flatness test               1  5.248     1  7.74     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 88% (76.1%-94.4%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

Reading Comprehension:

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 65 
## - Total number of p<0.05 studies included into the analysis: k = 18 (27.69%) 
## - Total number of studies with p<0.025: k = 15 (23.08%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.004 -5.957 0.000 -4.796     0
## Flatness test           0.924  2.774 0.997  6.164     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 73% (50.9%-87.6%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

Language-focused:

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 64 
## - Total number of p<0.05 studies included into the analysis: k = 21 (32.81%) 
## - Total number of studies with p<0.025: k = 14 (21.88%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.095 -7.119     0 -8.407     0
## Flatness test           0.393  3.604     1  7.642     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 79% (60.6%-89.8%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

Writing:

## P-curve analysis 
##  ----------------------- 
## - Total number of provided studies: k = 6 
## - Total number of p<0.05 studies included into the analysis: k = 3 (50%) 
## - Total number of studies with p<0.025: k = 3 (50%) 
##    
## Results 
##  ----------------------- 
##                     pBinomial  zFull pFull  zHalf pHalf
## Right-skewness test     0.125 -4.264 0.000 -3.775     0
## Flatness test           1.000  2.699 0.997  3.355     1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.   
## Power Estimate: 94% (66.4%-99%)
##    
## Evidential value 
##  ----------------------- 
## - Evidential value present: yes 
## - Evidential value absent/inadequate: no

---

6 Sensitivity analyses

6.1 Exclude high selection risk studies [Keep RCTs only]

Code-related:

## 
## Random-Effects Model (k = 72; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1391 (SE = 0.0359)
## tau (square root of estimated tau^2 value):      0.3729
## I^2 (total heterogeneity / total variability):   78.67%
## H^2 (total variability / sampling variability):  4.69
## 
## Test for Heterogeneity:
## Q(df = 71) = 216.6770, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3782  0.0556  6.7968  <.0001  0.2691  0.4872  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Reading Comprehension:

## 
## Random-Effects Model (k = 48; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0452 (SE = 0.0165)
## tau (square root of estimated tau^2 value):      0.2125
## I^2 (total heterogeneity / total variability):   77.72%
## H^2 (total variability / sampling variability):  4.49
## 
## Test for Heterogeneity:
## Q(df = 47) = 183.7304, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2276  0.0432  5.2673  <.0001  0.1429  0.3123  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Language-focused:

## 
## Random-Effects Model (k = 41; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0545 (SE = 0.0215)
## tau (square root of estimated tau^2 value):      0.2335
## I^2 (total heterogeneity / total variability):   66.33%
## H^2 (total variability / sampling variability):  2.97
## 
## Test for Heterogeneity:
## Q(df = 40) = 105.9878, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3056  0.0505  6.0538  <.0001  0.2067  0.4046  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Writing:

## 
## Random-Effects Model (k = 4; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1079 (SE = 0.1855)
## tau (square root of estimated tau^2 value):      0.3285
## I^2 (total heterogeneity / total variability):   48.02%
## H^2 (total variability / sampling variability):  1.92
## 
## Test for Heterogeneity:
## Q(df = 3) = 5.9435, p-val = 0.1144
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   1.0469  0.2383  4.3930  <.0001  0.5798  1.5140  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

6.2 Constrain outliers by Winsorizing

Code-related:

## 
## Random-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0651 (SE = 0.0197)
## tau (square root of estimated tau^2 value):      0.2552
## I^2 (total heterogeneity / total variability):   63.05%
## H^2 (total variability / sampling variability):  2.71
## 
## Test for Heterogeneity:
## Q(df = 85) = 183.9515, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3113  0.0405  7.6822  <.0001  0.2318  0.3907  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Reading Comprehension:

## 
## Random-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0224 (SE = 0.0096)
## tau (square root of estimated tau^2 value):      0.1496
## I^2 (total heterogeneity / total variability):   59.48%
## H^2 (total variability / sampling variability):  2.47
## 
## Test for Heterogeneity:
## Q(df = 64) = 176.8840, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2259  0.0318  7.1091  <.0001  0.1636  0.2882  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Language-focused:

## 
## Random-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0455 (SE = 0.0160)
## tau (square root of estimated tau^2 value):      0.2132
## I^2 (total heterogeneity / total variability):   59.98%
## H^2 (total variability / sampling variability):  2.50
## 
## Test for Heterogeneity:
## Q(df = 63) = 133.8790, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2817  0.0395  7.1274  <.0001  0.2043  0.3592  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Writing:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0812 (SE = 0.1243)
## tau (square root of estimated tau^2 value):      0.2850
## I^2 (total heterogeneity / total variability):   41.62%
## H^2 (total variability / sampling variability):  1.71
## 
## Test for Heterogeneity:
## Q(df = 5) = 8.9670, p-val = 0.1104
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.7837  0.1817  4.3130  <.0001  0.4275  1.1398  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

6.3 Changing within study correlations (when aggregating effect sizes within a study)

6.3.1 Code-related

r = 0:

## 
## Random-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1461 (SE = 0.0300)
## tau (square root of estimated tau^2 value):      0.3822
## I^2 (total heterogeneity / total variability):   89.13%
## H^2 (total variability / sampling variability):  9.20
## 
## Test for Heterogeneity:
## Q(df = 85) = 406.2613, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3465  0.0485  7.1384  <.0001  0.2514  0.4417  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.4:

## 
## Random-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1237 (SE = 0.0297)
## tau (square root of estimated tau^2 value):      0.3517
## I^2 (total heterogeneity / total variability):   77.76%
## H^2 (total variability / sampling variability):  4.50
## 
## Test for Heterogeneity:
## Q(df = 85) = 250.0048, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3362  0.0486  6.9113  <.0001  0.2409  0.4316  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.8:

## 
## Random-Effects Model (k = 86; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.1045 (SE = 0.0287)
## tau (square root of estimated tau^2 value):      0.3232
## I^2 (total heterogeneity / total variability):   69.67%
## H^2 (total variability / sampling variability):  3.30
## 
## Test for Heterogeneity:
## Q(df = 85) = 201.4134, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3257  0.0484  6.7271  <.0001  0.2308  0.4206  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

6.3.2 Reading Comprehension

r = 0:

## 
## Random-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0627 (SE = 0.0177)
## tau (square root of estimated tau^2 value):      0.2504
## I^2 (total heterogeneity / total variability):   93.85%
## H^2 (total variability / sampling variability):  16.27
## 
## Test for Heterogeneity:
## Q(df = 64) = 1045.4788, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2570  0.0413  6.2247  <.0001  0.1761  0.3379  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.4:

## 
## Random-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0396 (SE = 0.0136)
## tau (square root of estimated tau^2 value):      0.1989
## I^2 (total heterogeneity / total variability):   73.64%
## H^2 (total variability / sampling variability):  3.79
## 
## Test for Heterogeneity:
## Q(df = 64) = 234.0208, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2365  0.0369  6.4153  <.0001  0.1642  0.3087  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.8:

## 
## Random-Effects Model (k = 65; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0252 (SE = 0.0108)
## tau (square root of estimated tau^2 value):      0.1586
## I^2 (total heterogeneity / total variability):   58.96%
## H^2 (total variability / sampling variability):  2.44
## 
## Test for Heterogeneity:
## Q(df = 64) = 168.9712, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2178  0.0336  6.4857  <.0001  0.1519  0.2836  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

6.3.3 Language-focused

r = 0:

## 
## Random-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0890 (SE = 0.0237)
## tau (square root of estimated tau^2 value):      0.2983
## I^2 (total heterogeneity / total variability):   80.27%
## H^2 (total variability / sampling variability):  5.07
## 
## Test for Heterogeneity:
## Q(df = 63) = 230.2889, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3109  0.0470  6.6185  <.0001  0.2188  0.4030  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.4:

## 
## Random-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0707 (SE = 0.0213)
## tau (square root of estimated tau^2 value):      0.2659
## I^2 (total heterogeneity / total variability):   70.82%
## H^2 (total variability / sampling variability):  3.43
## 
## Test for Heterogeneity:
## Q(df = 63) = 168.8412, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.3010  0.0448  6.7172  <.0001  0.2132  0.3889  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.8:

## 
## Random-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.0544 (SE = 0.0187)
## tau (square root of estimated tau^2 value):      0.2332
## I^2 (total heterogeneity / total variability):   62.03%
## H^2 (total variability / sampling variability):  2.63
## 
## Test for Heterogeneity:
## Q(df = 63) = 142.7276, p-val < .0001
## 
## Model Results:
## 
## estimate      se    zval    pval   ci.lb   ci.ub      
##   0.2903  0.0425  6.8281  <.0001  0.2070  0.3736  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

6.3.4 Writing

r = 0:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.20 (SE = 0.18)
## tau (square root of estimated tau^2 value):      0.45
## I^2 (total heterogeneity / total variability):   71.35%
## H^2 (total variability / sampling variability):  3.49
## 
## Test for Heterogeneity:
## Q(df = 5) = 15.58, p-val < .01
## 
## Model Results:
## 
## estimate    se  zval  pval  ci.lb  ci.ub      
##     0.85  0.22  3.86  <.01   0.42   1.28  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.4:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.16 (SE = 0.17)
## tau (square root of estimated tau^2 value):      0.40
## I^2 (total heterogeneity / total variability):   59.18%
## H^2 (total variability / sampling variability):  2.45
## 
## Test for Heterogeneity:
## Q(df = 5) = 11.88, p-val = 0.04
## 
## Model Results:
## 
## estimate    se  zval  pval  ci.lb  ci.ub      
##     0.82  0.21  3.87  <.01   0.40   1.24  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

r = 0.8:

## 
## Random-Effects Model (k = 6; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of total heterogeneity): 0.12 (SE = 0.16)
## tau (square root of estimated tau^2 value):      0.34
## I^2 (total heterogeneity / total variability):   47.89%
## H^2 (total variability / sampling variability):  1.92
## 
## Test for Heterogeneity:
## Q(df = 5) = 9.84, p-val = 0.08
## 
## Model Results:
## 
## estimate    se  zval  pval  ci.lb  ci.ub      
##     0.79  0.20  3.88  <.01   0.39   1.19  *** 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

---

7 New moderator results [June 2023]

Code-related:

	Model 1	Model 2	Model 3
intercept	0.261**	0.054	0.057
	(0.081)	(0.103)	(0.108)
typeDRP	0.127	0.296*	0.198
	(0.167)	(0.148)	(0.163)
typeDRD	0.197	0.298**	0.256*
	(0.115)	(0.111)	(0.112)
pedagogyCognitive	-0.034		-0.176
	(0.115)		(0.117)
pedagogyConstructivist	0.083		0.212
	(0.220)		(0.222)
avghrs_categoriesMedium		0.093	0.167
		(0.112)	(0.120)
avghrs_categoriesHigh		0.407**	0.508***
		(0.133)	(0.143)
i.squared	74.315	71.881	70.473
h.squared	3.893	3.556	3.387
tau.squared	0.121	0.110	0.105
tau.squared.se	0.031	0.029	0.028
cochran.qe	220.014	221.434	208.781
p.value.cochran.qe	0.000	0.000	0.000
cochran.qm	4.277	14.308	17.734
p.value.cochran.qm	0.370	0.006	0.007
DF Resid.	81	81	79
Log Likelihood	-57.179	-52.281	-50.503
Deviance	114.358	104.562	101.005
AIC	126.358	116.562	117.005
BIC	140.725	130.929	135.961
AICc	127.493	117.697	119.062
nobs	86	86	86
***p < 0.001; **p < 0.01; *p < 0.05

Language-focused:

	Model 1	Model 2	Model 3
intercept	0.038	-0.098	-0.207*
	(0.051)	(0.100)	(0.092)
typeLRP	0.353***	0.563***	0.485***
	(0.063)	(0.095)	(0.079)
typeLRD	0.096	0.235*	0.200*
	(0.077)	(0.113)	(0.090)
pedagogyCognitive	0.121*		0.121
	(0.057)		(0.065)
pedagogyConstructivist	0.355***		0.476***
	(0.105)		(0.115)
avghrs_categoriesMedium		0.186*	0.256**
		(0.092)	(0.080)
avghrs_categoriesHigh		0.252*	0.249*
		(0.114)	(0.103)
i.squared	16.221	42.082	20.601
h.squared	1.194	1.727	1.259
tau.squared	0.007	0.025	0.009
tau.squared.se	0.006	0.012	0.008
cochran.qe	92.157	106.272	82.487
p.value.cochran.qe	0.004	0.000	0.015
cochran.qm	57.043	38.572	64.494
p.value.cochran.qm	0.000	0.000	0.000
DF Resid.	59	59	57
Log Likelihood	-16.590	-19.345	-12.522
Deviance	33.180	38.689	25.044
AIC	45.180	50.689	41.044
BIC	57.645	63.155	57.388
AICc	46.796	52.305	44.044
nobs	64	64	64
***p < 0.001; **p < 0.01; *p < 0.05

Reading Comprehension:

	Model 1	Model 2	Model 3
intercept	0.170*	0.142	0.152
	(0.075)	(0.114)	(0.117)
typeCRD	0.219***	0.228**	0.233**
	(0.066)	(0.072)	(0.073)
pedagogyCognitive	-0.023		-0.035
	(0.080)		(0.090)
avghrs_categoriesMedium		-0.005	0.010
		(0.116)	(0.123)
avghrs_categoriesHigh		0.029	0.051
		(0.123)	(0.135)
i.squared	51.519	51.306	50.820
h.squared	2.063	2.054	2.033
tau.squared	0.021	0.023	0.023
tau.squared.se	0.009	0.010	0.010
cochran.qe	116.774	119.472	114.995
p.value.cochran.qe	0.000	0.000	0.000
cochran.qm	10.957	10.376	10.577
p.value.cochran.qm	0.004	0.016	0.032
DF Resid.	62	61	60
Log Likelihood	-14.687	-14.996	-15.158
Deviance	29.374	29.993	30.316
AIC	37.374	39.993	42.316
BIC	45.883	50.547	54.882
AICc	38.076	41.084	43.901
nobs	65	65	65
***p < 0.001; **p < 0.01; *p < 0.05

Association between gamification and pedagogical orientation for 120 studies:

---

8 New moderator results [Dec 2023]

8.1 18: Connor et al. (2019)

This study reports LRP effect sizes.

Control:

## 
## Mixed-Effects Model (k = 62; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0679 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2606
## I^2 (residual heterogeneity / unaccounted variability): 68.47%
## H^2 (unaccounted variability / sampling variability):   3.17
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 60) = 155.4035, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.8031, p-val = 0.3702
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt       0.2075  0.1035  2.0044  0.0450   0.0046  0.4104  * 
## controlBAU    0.1031  0.1151  0.8962  0.3702  -0.1224  0.3286    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Mixed-Effects Model (k = 62; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0625 (SE = 0.0203)
## tau (square root of estimated tau^2 value):             0.2499
## I^2 (residual heterogeneity / unaccounted variability): 66.95%
## H^2 (unaccounted variability / sampling variability):   3.03
## R^2 (amount of heterogeneity accounted for):            5.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 60) = 149.1791, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 4.3006, p-val = 0.0381
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb    ci.ub      
## intrcpt       0.3701  0.0589   6.2887  <.0001   0.2548   0.4855  *** 
## tech_plus1   -0.1839  0.0887  -2.0738  0.0381  -0.3578  -0.0101    * 
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Instructional Context:

## 
## Mixed-Effects Model (k = 62; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0663 (SE = 0.0211)
## tau (square root of estimated tau^2 value):             0.2574
## I^2 (residual heterogeneity / unaccounted variability): 68.65%
## H^2 (unaccounted variability / sampling variability):   3.19
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 60) = 153.7003, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 1.8442, p-val = 0.1745
## 
## Model Results:
## 
##                                             estimate      se     zval    pval 
## intrcpt                                       0.3335  0.0549   6.0697  <.0001 
## instructional_contextOther than individual   -0.1291  0.0951  -1.3580  0.1745 
##                                               ci.lb   ci.ub      
## intrcpt                                      0.2258  0.4412  *** 
## instructional_contextOther than individual  -0.3155  0.0572      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

8.2 76: Segal-Drori et al. (2010)

This study reports DRP effect sizes.

Control:

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1134 (SE = 0.0293)
## tau (square root of estimated tau^2 value):             0.3367
## I^2 (residual heterogeneity / unaccounted variability): 74.41%
## H^2 (unaccounted variability / sampling variability):   3.91
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 213.8921, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6787, p-val = 0.4100
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.2536  0.0912  2.7815  0.0054   0.0749  0.4323  ** 
## controlBAU    0.0889  0.1078  0.8239  0.4100  -0.1225  0.3002     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Technology Only (T) or Technology Plus (T+):

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1162 (SE = 0.0298)
## tau (square root of estimated tau^2 value):             0.3409
## I^2 (residual heterogeneity / unaccounted variability): 74.97%
## H^2 (unaccounted variability / sampling variability):   3.99
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 218.6314, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0077, p-val = 0.9302
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt       0.3143  0.0633  4.9619  <.0001   0.1901  0.4384  *** 
## tech_plus1    0.0088  0.1003  0.0876  0.9302  -0.1877  0.2053      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

8.3 113: Shamir et al. (2012)

This study reports DRD and LRP effect sizes.

Control (code related):

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1134 (SE = 0.0293)
## tau (square root of estimated tau^2 value):             0.3367
## I^2 (residual heterogeneity / unaccounted variability): 74.41%
## H^2 (unaccounted variability / sampling variability):   3.91
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 213.8921, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6787, p-val = 0.4100
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.2536  0.0912  2.7815  0.0054   0.0749  0.4323  ** 
## controlBAU    0.0889  0.1078  0.8239  0.4100  -0.1225  0.3002     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (language focused):

## 
## Mixed-Effects Model (k = 62; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0679 (SE = 0.0215)
## tau (square root of estimated tau^2 value):             0.2606
## I^2 (residual heterogeneity / unaccounted variability): 68.47%
## H^2 (unaccounted variability / sampling variability):   3.17
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 60) = 155.4035, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.8031, p-val = 0.3702
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub    
## intrcpt       0.2075  0.1035  2.0044  0.0450   0.0046  0.4104  * 
## controlBAU    0.1031  0.1151  0.8962  0.3702  -0.1224  0.3286    
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

8.4 81: Solheim et al. (2018)

This study reports DRD and CRD effect sizes.

Control (code related):

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1164 (SE = 0.0298)
## tau (square root of estimated tau^2 value):             0.3412
## I^2 (residual heterogeneity / unaccounted variability): 75.46%
## H^2 (unaccounted variability / sampling variability):   4.08
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 217.9129, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1478, p-val = 0.7006
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.3343  0.0652   5.1243  <.0001   0.2065  0.4622  *** 
## adaptivity   -0.0381  0.0991  -0.3845  0.7006  -0.2324  0.1562      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Control (reading comp):

## 
## Mixed-Effects Model (k = 64; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.0372 (SE = 0.0135)
## tau (square root of estimated tau^2 value):             0.1929
## I^2 (residual heterogeneity / unaccounted variability): 68.84%
## H^2 (unaccounted variability / sampling variability):   3.21
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 62) = 192.6184, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.1378, p-val = 0.7105
## 
## Model Results:
## 
##             estimate      se     zval    pval    ci.lb   ci.ub      
## intrcpt       0.2401  0.0434   5.5343  <.0001   0.1550  0.3251  *** 
## adaptivity   -0.0299  0.0805  -0.3712  0.7105  -0.1877  0.1279      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

8.5 137: Boggio et al. (2023)

This study reports DRD effect sizes.

Control:

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1134 (SE = 0.0293)
## tau (square root of estimated tau^2 value):             0.3367
## I^2 (residual heterogeneity / unaccounted variability): 74.41%
## H^2 (unaccounted variability / sampling variability):   3.91
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 213.8921, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.6787, p-val = 0.4100
## 
## Model Results:
## 
##             estimate      se    zval    pval    ci.lb   ci.ub     
## intrcpt       0.2536  0.0912  2.7815  0.0054   0.0749  0.4323  ** 
## controlBAU    0.0889  0.1078  0.8239  0.4100  -0.1225  0.3002     
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Gamification:

## 
## Mixed-Effects Model (k = 83; tau^2 estimator: REML)
## 
## tau^2 (estimated amount of residual heterogeneity):     0.1162 (SE = 0.0298)
## tau (square root of estimated tau^2 value):             0.3409
## I^2 (residual heterogeneity / unaccounted variability): 74.99%
## H^2 (unaccounted variability / sampling variability):   4.00
## R^2 (amount of heterogeneity accounted for):            0.00%
## 
## Test for Residual Heterogeneity:
## QE(df = 81) = 218.7485, p-val < .0001
## 
## Test of Moderators (coefficient 2):
## QM(df = 1) = 0.0174, p-val = 0.8950
## 
## Model Results:
## 
##               estimate      se    zval    pval    ci.lb   ci.ub      
## intrcpt         0.3097  0.0786  3.9383  <.0001   0.1556  0.4638  *** 
## gamification    0.0133  0.1007  0.1320  0.8950  -0.1840  0.2106      
## 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1