Bangert-Drowns et al. (1991) の再分析

原著のメタ分析の方針と再分析の方針
- 効果量は実験群と統制群の平均の差を統制群の標準偏差またはプールさせた標準偏差で割ったもの。
- t値やF値から求めたものもある。この場合の方法はGlass, McGaw, and Smith (1981)による。
- 対象者が解答する前にフィードバックの内容を知ることができるか(presearch availability)も調整変数として入れている。
  - → presearch availabilityはフィードバックの効果を大きく左右する。
  - 今回の再分析では対象から外す。

Bangert-Drowns et al. (1991) のTable 1

再分析の方針

Dissertation (1本38ドルはいくらなんでも厳しい), Paper presentation, ERIC Documentは入手困難であったり，全ページが入手できなかったりするため除外
- Journalのみを対象
対象者が解答する前にフィードバックの内容を知ることができるか(presearch availability)はフィードバックの効果を大きく左右する。
- 今回の再分析では対象から外す。
以下の論文はJournalであるが分析から除外
- Peeck and Tillema (1979)
  - Bangert-DrownsはTotal N = 67とあるが，一次研究のTotal Nは32と書いてある
  - Table 1 に結果あり。このnの総和も63でNが合わない
  - 効果の向きも合わない，大きさも合わない
  - Correct answerのfeedbackになっていない
    - 処遇の違いは，自身の答案を返すか，正答だけを返すかである
- Kulhavy, Yekovich, and Dyer (1976)
  - presearch availability が impossible
  - FeedbackがRepeat till correctであり，ジャーナル掲載分では1本しかないため，フィードバックの種別でのメタ分析が出来ないため，今回は除外

一次研究から求めた効果量と分散

Anderson, et al. (1971) Study 1

0% KCR (n=21) と 100% KCR (n=21)のCriterion test correct は57.9, 68.7
Bangert-Drowns では\(d=0.81\)と報告
この値を用いて\(\sigma_{pooled}\)を求めると13.33
これでmesを実行

mes2(57.9, 68.7, 13.33, 21, 21, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = -0.81 [ -1.459 , -0.161 ] 
##   var(d) = 0.103 
##   p-value(d) = 0.016 
##   U3(d) = 20.891 % 
##   CLES(d) = 28.336 % 
##   Cliff's Delta = -0.433 
##  
##  g [ 95 %CI] = -0.795 [ -1.431 , -0.158 ] 
##   var(g) = 0.099 
##   p-value(g) = 0.016 
##   U3(g) = 21.333 % 
##   CLES(g) = 28.703 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = -0.375 [ -0.616 , -0.071 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.018 
##  
##  z [ 95 %CI] = -0.395 [ -0.718 , -0.071 ] 
##   var(z) = 0.026 
##   p-value(z) = 0.018 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 0.23 [ 0.071 , 0.746 ] 
##   p-value(OR) = 0.016 
##  
##  Log OR [ 95 %CI] = -1.47 [ -2.646 , -0.293 ] 
##   var(lOR) = 0.339 
##   p-value(Log OR) = 0.016 
##  
##  Other: 
##  
##  NNT = -6.635 
##  Total N = 42

Anderson, et al. (1971) Study 3

0% KCR (n=26) と 100% KCR (n=24)のCriterion test correct は47.4, 52.8
Bangert-Drowns では\(d=0.61\)と報告
この値を用いて\(\sigma_{pooled}\)を求めると8.85
これでmesを実行

mes2(52.8, 47.4, 8.85, 26, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.61 [ 0.028 , 1.192 ] 
##   var(d) = 0.084 
##   p-value(d) = 0.04 
##   U3(d) = 72.913 % 
##   CLES(d) = 66.693 % 
##   Cliff's Delta = 0.334 
##  
##  g [ 95 %CI] = 0.601 [ 0.028 , 1.174 ] 
##   var(g) = 0.081 
##   p-value(g) = 0.04 
##   U3(g) = 72.594 % 
##   CLES(g) = 66.446 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.292 [ 0.007 , 0.532 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.045 
##  
##  z [ 95 %CI] = 0.3 [ 0.007 , 0.594 ] 
##   var(z) = 0.021 
##   p-value(z) = 0.045 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 3.024 [ 1.052 , 8.695 ] 
##   p-value(OR) = 0.04 
##  
##  Log OR [ 95 %CI] = 1.107 [ 0.051 , 2.163 ] 
##   var(lOR) = 0.276 
##   p-value(Log OR) = 0.04 
##  
##  Other: 
##  
##  NNT = 4.797 
##  Total N = 50

Anderson, et al. (1972) Study 1
- 100% standard (n=24?) と 0% standard (n=24?)のCriterion test correct は27.0, 23.8

Bangert-Drowns では\(d=0.41\)と報告
この値を用いて\(\sigma_{pooled}\)を求めると7.80
これでmesを実行

mes2(27.0, 23.8, 7.80, 24, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.41 [ -0.177 , 0.997 ] 
##   var(d) = 0.085 
##   p-value(d) = 0.166 
##   U3(d) = 65.919 % 
##   CLES(d) = 61.413 % 
##   Cliff's Delta = 0.228 
##  
##  g [ 95 %CI] = 0.404 [ -0.174 , 0.981 ] 
##   var(g) = 0.082 
##   p-value(g) = 0.166 
##   U3(g) = 65.672 % 
##   CLES(g) = 61.231 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.201 [ -0.096 , 0.465 ] 
##   var(r) = 0.019 
##   p-value(r) = 0.178 
##  
##  z [ 95 %CI] = 0.204 [ -0.096 , 0.504 ] 
##   var(z) = 0.022 
##   p-value(z) = 0.178 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 2.105 [ 0.726 , 6.105 ] 
##   p-value(OR) = 0.166 
##  
##  Log OR [ 95 %CI] = 0.744 [ -0.321 , 1.809 ] 
##   var(lOR) = 0.28 
##   p-value(Log OR) = 0.166 
##  
##  Other: 
##  
##  NNT = 7.513 
##  Total N = 48

Farragher and Szabo (1986) Study 1

Bangert-DrownsではN=100と報告されているが，実際には30人×5群なので数字が合わない。
No feedback(n=30, M=41,7, SD=6.19)とConfirmation results(n=30, M=40.2, SD=7.45)を比較すべきと思われる。
これでmesを実行

mes(40.2, 41.7, 7.45, 6.19, 30, 30, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = -0.219 [ -0.737 , 0.299 ] 
##   var(d) = 0.067 
##   p-value(d) = 0.401 
##   U3(d) = 41.332 % 
##   CLES(d) = 43.846 % 
##   Cliff's Delta = -0.123 
##  
##  g [ 95 %CI] = -0.216 [ -0.728 , 0.295 ] 
##   var(g) = 0.065 
##   p-value(g) = 0.401 
##   U3(g) = 41.443 % 
##   CLES(g) = 43.926 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = -0.109 [ -0.358 , 0.155 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.413 
##  
##  z [ 95 %CI] = -0.109 [ -0.374 , 0.156 ] 
##   var(z) = 0.018 
##   p-value(z) = 0.413 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 0.672 [ 0.263 , 1.721 ] 
##   p-value(OR) = 0.401 
##  
##  Log OR [ 95 %CI] = -0.397 [ -1.337 , 0.543 ] 
##   var(lOR) = 0.221 
##   p-value(Log OR) = 0.401 
##  
##  Other: 
##  
##  NNT = -17.995 
##  Total N = 60

Farragher and Szabo (1986) Study 2

Bangert-DrownsではN=100と報告されているが，実際には30人×5群なので数字が合わない。
No feedback(n=30, M=41,7, SD=6.19)とSpecific prescription (n=30, M=44.4, SD=6.33)を比較すべきと思われる。
これでmesを実行

mes(44.4, 41.7, 6.33, 6.19, 30, 30, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.431 [ -0.092 , 0.954 ] 
##   var(d) = 0.068 
##   p-value(d) = 0.104 
##   U3(d) = 66.687 % 
##   CLES(d) = 61.98 % 
##   Cliff's Delta = 0.24 
##  
##  g [ 95 %CI] = 0.426 [ -0.09 , 0.942 ] 
##   var(g) = 0.066 
##   p-value(g) = 0.104 
##   U3(g) = 66.483 % 
##   CLES(g) = 61.829 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.211 [ -0.051 , 0.446 ] 
##   var(r) = 0.015 
##   p-value(r) = 0.112 
##  
##  z [ 95 %CI] = 0.214 [ -0.051 , 0.479 ] 
##   var(z) = 0.018 
##   p-value(z) = 0.112 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 2.186 [ 0.847 , 5.644 ] 
##   p-value(OR) = 0.104 
##  
##  Log OR [ 95 %CI] = 0.782 [ -0.166 , 1.731 ] 
##   var(lOR) = 0.224 
##   p-value(Log OR) = 0.104 
##  
##  Other: 
##  
##  NNT = 7.103 
##  Total N = 60

Karraker (1967)

13ページに結果あり
Wrong responses であることに注意
表にするとこんな感じ

Group N M SD

Knowledge of results 24 3.54 2.53

No knowledge of results 24 6.33 2.82
これでmesを実行

Group	N	M	SD
Knowledge of results	24	3.54	2.53
No knowledge of results	24	6.33	2.82

mes(3.54, 6.33, 2.53, 2.82, 24, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = -1.041 [ -1.661 , -0.422 ] 
##   var(d) = 0.095 
##   p-value(d) = 0.001 
##   U3(d) = 14.883 % 
##   CLES(d) = 23.074 % 
##   Cliff's Delta = -0.539 
##  
##  g [ 95 %CI] = -1.024 [ -1.633 , -0.415 ] 
##   var(g) = 0.092 
##   p-value(g) = 0.001 
##   U3(g) = 15.283 % 
##   CLES(g) = 23.442 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = -0.462 [ -0.664 , -0.197 ] 
##   var(r) = 0.012 
##   p-value(r) = 0.002 
##  
##  z [ 95 %CI] = -0.5 [ -0.8 , -0.2 ] 
##   var(z) = 0.022 
##   p-value(z) = 0.002 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 0.151 [ 0.049 , 0.465 ] 
##   p-value(OR) = 0.001 
##  
##  Log OR [ 95 %CI] = -1.889 [ -3.012 , -0.766 ] 
##   var(lOR) = 0.311 
##   p-value(Log OR) = 0.001 
##  
##  Other: 
##  
##  NNT = -5.877 
##  Total N = 48

Lhyle and Kulhavy (1987) Study 1

321ページに結果あり。この値を用いてmesを実行

mes(13.55, 9.00, 3.83, 3.20, 20, 20, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 1.289 [ 0.586 , 1.993 ] 
##   var(d) = 0.121 
##   p-value(d) = 0.001 
##   U3(d) = 90.135 % 
##   CLES(d) = 81.903 % 
##   Cliff's Delta = 0.638 
##  
##  g [ 95 %CI] = 1.264 [ 0.574 , 1.953 ] 
##   var(g) = 0.116 
##   p-value(g) = 0.001 
##   U3(g) = 89.683 % 
##   CLES(g) = 81.422 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.542 [ 0.267 , 0.735 ] 
##   var(r) = 0.011 
##   p-value(r) = 0.001 
##  
##  z [ 95 %CI] = 0.607 [ 0.274 , 0.94 ] 
##   var(z) = 0.027 
##   p-value(z) = 0.001 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 10.366 [ 2.893 , 37.136 ] 
##   p-value(OR) = 0.001 
##  
##  Log OR [ 95 %CI] = 2.339 [ 1.062 , 3.615 ] 
##   var(lOR) = 0.397 
##   p-value(Log OR) = 0.001 
##  
##  Other: 
##  
##  NNT = 2.115 
##  Total N = 40

Lhyle and Kulhavy (1987) Study 2

322ページに“The group means for criterion corrects were overt-scrambled = 14.25 (SD = 2.51); copy =12.95 (SD = 2.54); control = 10.10 (SD = 3.77).”とある。
Bangert-Drownsはcopyとcontrolを比較していると思われる。scrambledはフィードバックが無秩序に掲載されていて，テキストの中から自分で見つけなければならない（この方が効果が高い）

mes(12.95, 10.10, 2.54, 3.77, 20, 20, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.887 [ 0.216 , 1.558 ] 
##   var(d) = 0.11 
##   p-value(d) = 0.011 
##   U3(d) = 81.236 % 
##   CLES(d) = 73.465 % 
##   Cliff's Delta = 0.469 
##  
##  g [ 95 %CI] = 0.869 [ 0.211 , 1.527 ] 
##   var(g) = 0.106 
##   p-value(g) = 0.011 
##   U3(g) = 80.758 % 
##   CLES(g) = 73.056 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.405 [ 0.097 , 0.643 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.013 
##  
##  z [ 95 %CI] = 0.43 [ 0.097 , 0.763 ] 
##   var(z) = 0.027 
##   p-value(z) = 0.013 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 4.994 [ 1.479 , 16.862 ] 
##   p-value(OR) = 0.011 
##  
##  Log OR [ 95 %CI] = 1.608 [ 0.391 , 2.825 ] 
##   var(lOR) = 0.361 
##   p-value(Log OR) = 0.011 
##  
##  Other: 
##  
##  NNT = 3.145 
##  Total N = 40

Moore and Smith (1961)

Study 2がn=28
Table 2のposttestを比較していると思われる
Experimentが41.286, Controlが43.786, \(t=0.72\)

tes(0.72, 14, 14, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.272 [ -0.508 , 1.053 ] 
##   var(d) = 0.144 
##   p-value(d) = 0.48 
##   U3(d) = 60.724 % 
##   CLES(d) = 57.63 % 
##   Cliff's Delta = 0.153 
##  
##  g [ 95 %CI] = 0.264 [ -0.494 , 1.022 ] 
##   var(g) = 0.136 
##   p-value(g) = 0.48 
##   U3(g) = 60.419 % 
##   CLES(g) = 57.41 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.14 [ -0.264 , 0.502 ] 
##   var(r) = 0.036 
##   p-value(r) = 0.488 
##  
##  z [ 95 %CI] = 0.141 [ -0.27 , 0.552 ] 
##   var(z) = 0.04 
##   p-value(z) = 0.488 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 1.638 [ 0.398 , 6.748 ] 
##   p-value(OR) = 0.48 
##  
##  Log OR [ 95 %CI] = 0.494 [ -0.922 , 1.909 ] 
##   var(lOR) = 0.474 
##   p-value(Log OR) = 0.48 
##  
##  Other: 
##  
##  NNT = 11.833 
##  Total N = 28

Morgan and Morgan (1935)

Table 2に結果，Total N = 87で合致
Initial test (one week later)を比較していると思われる
Table 3に1E-1Cとあり，D/S.E. diffが1.21とあり，Bangert-Drownsはこれを引いているを思われる。
実際にmesを実行すると以下の通りとなる

mes(18.07, 16.12, 7.47, 7.49, 42, 45, level=95, cer=0.2, dig=2, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.26 [ -0.17 , 0.69 ] 
##   var(d) = 0.05 
##   p-value(d) = 0.23 
##   U3(d) = 60.28 % 
##   CLES(d) = 57.31 % 
##   Cliff's Delta = 0.15 
##  
##  g [ 95 %CI] = 0.26 [ -0.17 , 0.68 ] 
##   var(g) = 0.05 
##   p-value(g) = 0.23 
##   U3(g) = 60.19 % 
##   CLES(g) = 57.25 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.13 [ -0.09 , 0.33 ] 
##   var(r) = 0.01 
##   p-value(r) = 0.24 
##  
##  z [ 95 %CI] = 0.13 [ -0.09 , 0.35 ] 
##   var(z) = 0.01 
##   p-value(z) = 0.24 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 1.6 [ 0.74 , 3.49 ] 
##   p-value(OR) = 0.23 
##  
##  Log OR [ 95 %CI] = 0.47 [ -0.3 , 1.25 ] 
##   var(lOR) = 0.15 
##   p-value(Log OR) = 0.23 
##  
##  Other: 
##  
##  NNT = 12.4 
##  Total N = 87

Newman, Williams, and Hiller (1974)

57ページに表がある。
Bangert-Drownsではn=60になっているので，No foodback (n=27)とImmediate feedback (n=33)のRetestの平均点を比較していると思われる。
mesを実行する

mes(20.33, 21.07, 4.23, 5.75, 33, 27, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = -0.149 [ -0.669 , 0.371 ] 
##   var(d) = 0.068 
##   p-value(d) = 0.569 
##   U3(d) = 44.081 % 
##   CLES(d) = 45.807 % 
##   Cliff's Delta = -0.084 
##  
##  g [ 95 %CI] = -0.147 [ -0.66 , 0.366 ] 
##   var(g) = 0.066 
##   p-value(g) = 0.569 
##   U3(g) = 44.157 % 
##   CLES(g) = 45.861 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = -0.074 [ -0.327 , 0.189 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.578 
##  
##  z [ 95 %CI] = -0.074 [ -0.339 , 0.191 ] 
##   var(z) = 0.018 
##   p-value(z) = 0.578 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 0.763 [ 0.297 , 1.961 ] 
##   p-value(OR) = 0.569 
##  
##  Log OR [ 95 %CI] = -0.27 [ -1.214 , 0.673 ] 
##   var(lOR) = 0.222 
##   p-value(Log OR) = 0.569 
##  
##  Other: 
##  
##  NNT = -25.612 
##  Total N = 60

Peeck, Bosch, van den, and Kreupeling (1985)

307ページのDelayのFactualを使っている。
以下のように計算すると再現される。

((10.29 + 10.25)/2 - 8.94) / 1.95

## [1] 0.6820513

正確に計算するために以下のようにする

n1 = 34; n2 = 28; n3 = 35
d.f.m1 = 10.29; d.f.m2 = 10.25; d.f.m3 = 8.94
d.f.s1 = 1.62; d.f.s2 = 1.66; d.f.s3 = 1.95

fb.sg.pl <- sqrt((n1 * d.f.s1^2 + n2 * d.f.s2^2)/(n1 + n2 - 2))
fb.m <- (10.29 + 10.25) /2

mes(fb.m, d.f.m3, fb.sg.pl, d.f.s3, n1+n2, n3, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.75 [ 0.317 , 1.184 ] 
##   var(d) = 0.048 
##   p-value(d) = 0.001 
##   U3(d) = 77.349 % 
##   CLES(d) = 70.215 % 
##   Cliff's Delta = 0.404 
##  
##  g [ 95 %CI] = 0.744 [ 0.315 , 1.174 ] 
##   var(g) = 0.047 
##   p-value(g) = 0.001 
##   U3(g) = 77.17 % 
##   CLES(g) = 70.069 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.339 [ 0.147 , 0.506 ] 
##   var(r) = 0.008 
##   p-value(r) = 0.001 
##  
##  z [ 95 %CI] = 0.353 [ 0.148 , 0.558 ] 
##   var(z) = 0.011 
##   p-value(z) = 0.001 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 3.9 [ 1.778 , 8.556 ] 
##   p-value(OR) = 0.001 
##  
##  Log OR [ 95 %CI] = 1.361 [ 0.575 , 2.147 ] 
##   var(lOR) = 0.157 
##   p-value(Log OR) = 0.001 
##  
##  Other: 
##  
##  NNT = 3.793 
##  Total N = 97

Roper (1977) Study 1

3群ともn=12なのだがBangert-Drownsではn=24とn=25になっている
46ページに素点が載っているので，それを使って再計算する

nofb <- c(2,7,7,17,4,19,21,14,7,8,7,4)
rw   <- c(11,11,5,19,14,11,11,18,10,5,10,12)
ca   <- c(18,12,10,19,11,20,9,13,15,16,18,14)

m.nofb <- mean(nofb); m.rw   <- mean(rw); m.ca   <- mean(ca)
sd.nofb <- sd(nofb); sd.rw   <- sd(rw); sd.ca   <- sd(ca)
n.nofb  <- 12; n.rw    <- 12; n.ca    <- 12

# Correct answer
mes(m.ca, m.nofb, sd.ca, sd.nofb, n.ca, n.nofb, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.932 [ 0.041 , 1.824 ] 
##   var(d) = 0.185 
##   p-value(d) = 0.041 
##   U3(d) = 82.444 % 
##   CLES(d) = 74.515 % 
##   Cliff's Delta = 0.49 
##  
##  g [ 95 %CI] = 0.9 [ 0.04 , 1.761 ] 
##   var(g) = 0.172 
##   p-value(g) = 0.041 
##   U3(g) = 81.601 % 
##   CLES(g) = 73.78 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.423 [ -0.002 , 0.718 ] 
##   var(r) = 0.026 
##   p-value(r) = 0.051 
##  
##  z [ 95 %CI] = 0.451 [ -0.002 , 0.903 ] 
##   var(z) = 0.048 
##   p-value(z) = 0.051 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 5.426 [ 1.077 , 27.334 ] 
##   p-value(OR) = 0.041 
##  
##  Log OR [ 95 %CI] = 1.691 [ 0.074 , 3.308 ] 
##   var(lOR) = 0.608 
##   p-value(Log OR) = 0.041 
##  
##  Other: 
##  
##  NNT = 2.975 
##  Total N = 24

Roper (1977) Study 2

# Right/wrong
mes(m.rw, m.nofb, sd.rw, sd.nofb, n.rw, n.nofb, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.31 [ -0.542 , 1.161 ] 
##   var(d) = 0.169 
##   p-value(d) = 0.459 
##   U3(d) = 62.16 % 
##   CLES(d) = 58.667 % 
##   Cliff's Delta = 0.173 
##  
##  g [ 95 %CI] = 0.299 [ -0.523 , 1.121 ] 
##   var(g) = 0.157 
##   p-value(g) = 0.459 
##   U3(g) = 61.754 % 
##   CLES(g) = 58.373 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.153 [ -0.29 , 0.542 ] 
##   var(r) = 0.039 
##   p-value(r) = 0.487 
##  
##  z [ 95 %CI] = 0.154 [ -0.298 , 0.607 ] 
##   var(z) = 0.048 
##   p-value(z) = 0.487 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 1.754 [ 0.374 , 8.22 ] 
##   p-value(OR) = 0.459 
##  
##  Log OR [ 95 %CI] = 0.562 [ -0.983 , 2.107 ] 
##   var(lOR) = 0.555 
##   p-value(Log OR) = 0.459 
##  
##  Other: 
##  
##  NNT = 10.268 
##  Total N = 24

Sassenrath and Gaverick(1965) Study 1

“there were 129 subjects in the control-no feedback group, 97 subjects in the group which checked their answers on the board, 116 subjects in the group which instructors discussed the questions, and 145 subjects in the group which looked up the answers in the book.”とある。
control vs check answersがcorrect answer (n=226)と思われる。
指標はRetentionかと思われる。

mes(33.89, 30.36, 6.48, 5.13, 97, 129, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.614 [ 0.343 , 0.885 ] 
##   var(d) = 0.019 
##   p-value(d) = 0 
##   U3(d) = 73.045 % 
##   CLES(d) = 66.796 % 
##   Cliff's Delta = 0.336 
##  
##  g [ 95 %CI] = 0.612 [ 0.342 , 0.882 ] 
##   var(g) = 0.019 
##   p-value(g) = 0 
##   U3(g) = 72.977 % 
##   CLES(g) = 66.743 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.291 [ 0.166 , 0.407 ] 
##   var(r) = 0.004 
##   p-value(r) = 0 
##  
##  z [ 95 %CI] = 0.299 [ 0.168 , 0.431 ] 
##   var(z) = 0.004 
##   p-value(z) = 0 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 3.047 [ 1.864 , 4.979 ] 
##   p-value(OR) = 0 
##  
##  Log OR [ 95 %CI] = 1.114 [ 0.623 , 1.605 ] 
##   var(lOR) = 0.062 
##   p-value(Log OR) = 0 
##  
##  Other: 
##  
##  NNT = 4.761 
##  Total N = 226

Sassenrath and Gaverick(1965) Study 2

“there were 129 subjects in the control-no feedback group, 97 subjects in the group which checked their answers on the board, 116 subjects in the group which instructors discussed the questions, and 145 subjects in the group which looked up the answers in the book.”とある。
control vs look-up (n=274)と思われる。
指標はRetentionかと思われる。

mes(32.60, 30.36, 5.59, 5.13, 145, 129, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.416 [ 0.176 , 0.657 ] 
##   var(d) = 0.015 
##   p-value(d) = 0.001 
##   U3(d) = 66.147 % 
##   CLES(d) = 61.581 % 
##   Cliff's Delta = 0.232 
##  
##  g [ 95 %CI] = 0.415 [ 0.175 , 0.656 ] 
##   var(g) = 0.015 
##   p-value(g) = 0.001 
##   U3(g) = 66.105 % 
##   CLES(g) = 61.55 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.204 [ 0.087 , 0.315 ] 
##   var(r) = 0.003 
##   p-value(r) = 0.001 
##  
##  z [ 95 %CI] = 0.206 [ 0.087 , 0.326 ] 
##   var(z) = 0.004 
##   p-value(z) = 0.001 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 2.128 [ 1.375 , 3.294 ] 
##   p-value(OR) = 0.001 
##  
##  Log OR [ 95 %CI] = 0.755 [ 0.319 , 1.192 ] 
##   var(lOR) = 0.049 
##   p-value(Log OR) = 0.001 
##  
##  Other: 
##  
##  NNT = 7.387 
##  Total N = 274

Tait, Hartley, and Anderson (1973)

Experiment 2がn=52
167ページのTable 1のCriterion testのAdjusted Postを見ると，Noneが0.43，Passiveが0.59, positiveが0.59
“The planned comparison between the feedback groups and the no feedback group on adjusted mean criterion post-test score proved significant, t = 2.74, df = 49, P < 01.”とある。これをtesで計算すると以下の通り

tes(2.74, 18, 34, level = 95, cer = 0.2, dig = 3, verbose = TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.799 [ 0.192 , 1.405 ] 
##   var(d) = 0.091 
##   p-value(d) = 0.011 
##   U3(d) = 78.776 % 
##   CLES(d) = 71.388 % 
##   Cliff's Delta = 0.428 
##  
##  g [ 95 %CI] = 0.787 [ 0.19 , 1.384 ] 
##   var(g) = 0.088 
##   p-value(g) = 0.011 
##   U3(g) = 78.426 % 
##   CLES(g) = 71.098 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.361 [ 0.091 , 0.582 ] 
##   var(r) = 0.015 
##   p-value(r) = 0.011 
##  
##  z [ 95 %CI] = 0.378 [ 0.091 , 0.665 ] 
##   var(z) = 0.02 
##   p-value(z) = 0.011 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 4.257 [ 1.418 , 12.785 ] 
##   p-value(OR) = 0.011 
##  
##  Log OR [ 95 %CI] = 1.449 [ 0.349 , 2.548 ] 
##   var(lOR) = 0.3 
##   p-value(Log OR) = 0.011 
##  
##  Other: 
##  
##  NNT = 3.535 
##  Total N = 52

ただし，Bangert-DrownsではExplanationがd=0.49であると示している。この結果はいくら何でも解離が大きい。
“The linear regression of criterion post-test performance on criterion pre-test performance was plotted for the three feedback conoitions. The slopes of the lines for the feedback and no feedback conditions were significantly different, t=2.23, df=46, P<.O5.”という結果をtesで計算してみる。

tes(2.23, 18, 34, level = 95, cer = 0.2, dig = 3, verbose = TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.65 [ 0.051 , 1.249 ] 
##   var(d) = 0.089 
##   p-value(d) = 0.034 
##   U3(d) = 74.216 % 
##   CLES(d) = 67.711 % 
##   Cliff's Delta = 0.354 
##  
##  g [ 95 %CI] = 0.64 [ 0.05 , 1.231 ] 
##   var(g) = 0.086 
##   p-value(g) = 0.034 
##   U3(g) = 73.899 % 
##   CLES(g) = 67.462 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.301 [ 0.023 , 0.535 ] 
##   var(r) = 0.016 
##   p-value(r) = 0.035 
##  
##  z [ 95 %CI] = 0.31 [ 0.023 , 0.597 ] 
##   var(z) = 0.02 
##   p-value(z) = 0.035 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 3.251 [ 1.096 , 9.641 ] 
##   p-value(OR) = 0.034 
##  
##  Log OR [ 95 %CI] = 1.179 [ 0.092 , 2.266 ] 
##   var(lOR) = 0.293 
##   p-value(Log OR) = 0.034 
##  
##  Other: 
##  
##  NNT = 4.464 
##  Total N = 52

この結果も値が大きすぎる。
167ページのTable 1のCriterion testのAdjusted Postを見ると，Noneが0.43，Passiveが0.59, positiveが0.59ということから，フィードバックなしと比較したありの差得点は0.16である。
\(d=\frac{M1-M2}{\sigma_{pooled}}\)ということを踏まえると，\(0.49=\frac{0.16}{\sigma_{pooled}}\)となり，\(\sigma_{pooled}\)は0.32となる。これでmes2を使って計算

mes2(0.59, 0.43, 0.32, 18, 34, level = 95, cer = 0.2, dig = 3, verbose = TRUE, id=NULL, data=NULL)

## Mean Differences ES: 
##  
##  d [ 95 %CI] = 0.5 [ -0.094 , 1.094 ] 
##   var(d) = 0.087 
##   p-value(d) = 0.097 
##   U3(d) = 69.146 % 
##   CLES(d) = 63.816 % 
##   Cliff's Delta = 0.276 
##  
##  g [ 95 %CI] = 0.492 [ -0.092 , 1.077 ] 
##   var(g) = 0.085 
##   p-value(g) = 0.097 
##   U3(g) = 68.88 % 
##   CLES(g) = 63.616 % 
##  
##  Correlation ES: 
##  
##  r [ 95 %CI] = 0.231 [ -0.051 , 0.48 ] 
##   var(r) = 0.017 
##   p-value(r) = 0.105 
##  
##  z [ 95 %CI] = 0.236 [ -0.051 , 0.523 ] 
##   var(z) = 0.02 
##   p-value(z) = 0.105 
##  
##  Odds Ratio ES: 
##  
##  OR [ 95 %CI] = 2.477 [ 0.844 , 7.27 ] 
##   p-value(OR) = 0.097 
##  
##  Log OR [ 95 %CI] = 0.907 [ -0.17 , 1.984 ] 
##   var(lOR) = 0.287 
##   p-value(Log OR) = 0.097 
##  
##  Other: 
##  
##  NNT = 6.013 
##  Total N = 52

Remarks

このパッケージでは「伝えるための心理統計p56」のHedges’gの公式をCohen’s d(mean difference)と呼ぶ（http://monge.tec.fukuoka-u.ac.jp/r_analysis/effect_size_compute_es01.html）
var(d)の平方根が効果量の標準誤差となる。

再計算の結果を表にまとめる

# t01
t01res <- mes2(68.7, 57.9, 13.33, 21, 21, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef01 <- t01res[c("N.total", "d", "var.d")]; rownames(ef01) <- c("ef01")

# t02
t02res <- mes2(52.8, 47.4, 8.85, 26, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef02 <- t02res[c("N.total", "d", "var.d")]; rownames(ef02) <- c("ef02")

#t03
t03res <- mes2(27.0, 23.8, 7.80, 24, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef03 <- t03res[c("N.total", "d", "var.d")]; rownames(ef03) <- c("ef03")

#t04
t04res <- mes(40.2, 41.7, 7.45, 6.19, 30, 30, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef04 <- t04res[c("N.total", "d", "var.d")]; rownames(ef04) <- c("ef04")

#t05
t05res <- mes(44.4, 41.7, 6.33, 6.19, 30, 30, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef05 <- t05res[c("N.total", "d", "var.d")]; rownames(ef05) <- c("ef05")

#t06
t06res <- mes(6.33, 3.54, 2.82, 2.53, 24, 24, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef06 <- t06res[c("N.total", "d", "var.d")]; rownames(ef06) <- c("ef06")

#t07
t07res <- mes(13.55, 9.00, 3.83, 3.20, 20, 20, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef07 <- t07res[c("N.total", "d", "var.d")]; rownames(ef07) <- c("ef07")

#t08
t08res <- mes(12.95, 10.10, 2.54, 3.77, 20, 20, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef08 <- t08res[c("N.total", "d", "var.d")]; rownames(ef08) <- c("ef08")

#t09
t09res <- tes(-0.72, 14, 14, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef09 <- t09res[c("N.total", "d", "var.d")]; rownames(ef09) <- c("ef09")

#t10
t10res <- mes(18.07, 16.12, 7.47, 7.49, 42, 45, level=95, cer=0.2, dig=2, verbose=TRUE, id=NULL, data=NULL)
ef10 <- t10res[c("N.total", "d", "var.d")]; rownames(ef10) <- c("ef10")

#t11
t11res <- mes(20.33, 21.07, 4.23, 5.75, 33, 27, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef11 <- t11res[c("N.total", "d", "var.d")]; rownames(ef11) <- c("ef11")

#t12
n1 = 34; n2 = 28; n3 = 35
d.f.m1 = 10.29; d.f.m2 = 10.25; d.f.m3 = 8.94
d.f.s1 = 1.62; d.f.s2 = 1.66; d.f.s3 = 1.95
fb.sg.pl <- sqrt((n1 * d.f.s1^2 + n2 * d.f.s2^2)/(n1 + n2 - 2))
fb.m <- (10.29 + 10.25) /2
t12res <- mes(fb.m, d.f.m3, fb.sg.pl, d.f.s3, n1+n2, n3, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef12 <- t12res[c("N.total", "d", "var.d")]; rownames(ef12) <- c("ef12")

#t13
nofb <- c(2,7,7,17,4,19,21,14,7,8,7,4)
rw   <- c(11,11,5,19,14,11,11,18,10,5,10,12)
ca   <- c(18,12,10,19,11,20,9,13,15,16,18,14)
m.nofb <- mean(nofb); m.rw   <- mean(rw); m.ca   <- mean(ca)
sd.nofb <- sd(nofb); sd.rw   <- sd(rw); sd.ca   <- sd(ca)
n.nofb  <- 12; n.rw    <- 12; n.ca    <- 12
t13res <- mes(m.ca, m.nofb, sd.ca, sd.nofb, n.ca, n.nofb, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef13 <- t13res[c("N.total", "d", "var.d")]; rownames(ef13) <- c("ef13")

#t14
nofb <- c(2,7,7,17,4,19,21,14,7,8,7,4)
rw   <- c(11,11,5,19,14,11,11,18,10,5,10,12)
ca   <- c(18,12,10,19,11,20,9,13,15,16,18,14)
m.nofb <- mean(nofb); m.rw   <- mean(rw); m.ca   <- mean(ca)
sd.nofb <- sd(nofb); sd.rw   <- sd(rw); sd.ca   <- sd(ca)
n.nofb  <- 12; n.rw    <- 12; n.ca    <- 12
t14res <- mes(m.rw, m.nofb, sd.rw, sd.nofb, n.rw, n.nofb, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef14 <- t14res[c("N.total", "d", "var.d")]; rownames(ef14) <- c("ef14")

#t15
t15res <- mes(33.89, 30.36, 6.48, 5.13, 97, 129, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef15 <- t15res[c("N.total", "d", "var.d")]; rownames(ef15) <- c("ef15")

#t16
t16res <- mes(32.60, 30.36, 5.59, 5.13, 145, 129, level=95, cer=0.2, dig=3, verbose=TRUE, id=NULL, data=NULL)
ef16 <- t16res[c("N.total", "d", "var.d")]; rownames(ef16) <- c("ef16")

#t17
t17res <- mes2(0.59, 0.43, 0.32, 18, 34, level = 95, cer = 0.2, dig = 3, verbose = TRUE, id=NULL, data=NULL)
ef17 <- t17res[c("N.total", "d", "var.d")]; rownames(ef17) <- c("ef17")

mod <- rbind(ef01, ef02, ef03, ef04, ef05, ef06, ef07, ef08, ef09, ef10,
             ef11, ef12, ef13, ef14, ef15, ef16, ef17)

mod$SE <- sqrt(mod$var.d)

再分析用データの作成

DT::datatable(mgdata)

続きは以下のURL

http://rpubs.com/koyo/BD1991bayesmeta