pretest

Load Data

cg = c(6, 6, 7, 5, 5, 5, 11, 6, 7) # = number of errors
tg = c(9, 3, 2, 9, 9, 6, 9, 3, 4, 6) # = number of errors

The control group has n=9, the test group has n=10.

Test for Normality

Visuals seem to indicate no normal distribution for the groups, but normal distribution for the total sample, indicating that the samples are possibly to small.

shapiro.test(tg)
## 
##  Shapiro-Wilk normality test
## 
## data:  tg
## W = 0.84122, p-value = 0.04563
shapiro.test(cg)
## 
##  Shapiro-Wilk normality test
## 
## data:  cg
## W = 0.74224, p-value = 0.004407
shapiro.test(c(cg,tg))
## 
##  Shapiro-Wilk normality test
## 
## data:  c(cg, tg)
## W = 0.95542, p-value = 0.486

which is confirmed by the Shapiro Wilk test for each groups independently (normality assumption is rejected, because the p-values are significant).

Interestingly, together, they are normally distributed, which is encouraging – mind it this is the pre-test, so there would be no influence of the intervention.

t-test to check for significant difference of CG / TG

t.test(tg,cg)
## 
##  Welch Two Sample t-test
## 
## data:  tg and cg
## t = -0.40334, df = 15.63, p-value = 0.6922
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -2.784881  1.895992
## sample estimates:
## mean of x mean of y 
##  6.000000  6.444444
t.test(19-tg,19-cg)
## 
##  Welch Two Sample t-test
## 
## data:  19 - tg and 19 - cg
## t = 0.40334, df = 15.63, p-value = 0.6922
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.895992  2.784881
## sample estimates:
## mean of x mean of y 
##  13.00000  12.55556

This is confirmed by the t-test: the p-value is non significant, thus the effect size t is not statistically significant (and small anyways).

pre-test results for the underline test

cgt = c(5, 7, 5, 10, 4, 10, 8, 3, 6)
tgt = c(7, 1, 2, 2, 8, 4, 2, 2, 3, 4)
shapiro.test(cgt) 
## 
##  Shapiro-Wilk normality test
## 
## data:  cgt
## W = 0.93459, p-value = 0.5262
shapiro.test(tgt) # p-value = 0.04176: H0 accepted: NOT normal distributed
## 
##  Shapiro-Wilk normality test
## 
## data:  tgt
## W = 0.83801, p-value = 0.04176
shapiro.test(c(cgt, tgt)) # p-value = 0.1539: reject H0, so NOT normally distributed
## 
##  Shapiro-Wilk normality test
## 
## data:  c(cgt, tgt)
## W = 0.92722, p-value = 0.1539
t.test(cgt, tgt) # meaningless, because not normal distributed
## 
##  Welch Two Sample t-test
## 
## data:  cgt and tgt
## t = 2.648, df = 16.423, p-value = 0.01726
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.5921519 5.2967370
## sample estimates:
## mean of x mean of y 
##  6.444444  3.500000
mean(cgt)
## [1] 6.444444
mean(tgt)
## [1] 3.5

Normality test (Shapiro Wilk) indicates that control group data is normally distributed (H0 rejected, because p-value = 0.5262). The test group data is just about not normally distributed (H0 accepted, p-value=0.1539), thus a parametric test is unlikely to apply.

The t-test results are thus not valid, and instead a non-parametric test would be needed (limiting insights on effect size).

post-test

post-test vocabulary errors: data

cgp = c(9, 9, 8, 5, 5, 1, 14, 7, 7)
tgp = c(9, 4, 3, 6, 3, 7, 7, 5, 4, 6)

#qcgp = c(3, 1, 0, 8,   0,  0,  2,  1,  6,  6,  2,  4,  5,  6,  8,  5,  1,  5,  2,  4)
#qtgp = c(9, 1, 0, 9, 0,    0,  9,  1,  5,  9,  0,  1,  9,  9,  9,  9,  0,  2,  9,  9)
#qcgp_norm = qcgp / length(cgp)
#qtgp_norm = qtgp / length(tgp)

post-test vocab errors: normality

Shapiro Wilk indicates that both samples are normally distributed (p-values of 0.722 and 0.555, respectively). Incidentally, full sample of control group and test group is also normally distributed (p-value=0.394).

post-test vocab errors: t-test

The mean difference shows 1.82 errors difference less for the test group. The t-test results, however, are not significant.

t.test(cgp, tgp, var.equal = FALSE)
## 
##  Welch Two Sample t-test
## 
## data:  cgp and tgp
## t = 1.361, df = 12.131, p-value = 0.1983
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -1.091542  4.735986
## sample estimates:
## mean of x mean of y 
##  7.222222  5.400000

The reported t-value is higher than the relevant one in the student’s t-table (for 12 degrees of freedom this would be 2.179), thus the null hypothesis is rejected, and the alternative assumed (true difference in means is not equal to 0).

Despite the promising means difference, this is not assumed statistically significant for an alpha level of 0.05 (it would be on an alpha level of 0.2, so there is definitely a trend).

Post-test: Text comprehension and fill the gaps

text comprehension

cgtc = c(6,6,7,7, 6,7,4,3,7)
tgtc = c(3,4,6,6,5,5,6,6,6,6)
shapiro.test(cgtc) # non normal dist
## 
##  Shapiro-Wilk normality test
## 
## data:  cgtc
## W = 0.77626, p-value = 0.01091
shapiro.test(tgtc) # non normal dist
## 
##  Shapiro-Wilk normality test
## 
## data:  tgtc
## W = 0.73048, p-value = 0.002065
t.test(cgtc, tgtc, var.equal=F) # non significant
## 
##  Welch Two Sample t-test
## 
## data:  cgtc and tgtc
## t = 1, df = 14.53, p-value = 0.3337
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.669846  1.847624
## sample estimates:
## mean of x mean of y 
##  5.888889  5.300000
cgfig = c(6,6,6,6,6,6,6,6,6)
tgfig = c(6,5,6,6,5,5,5,5,5,6)
#shapiro.test(cgfig) # all values same = non normal dist
shapiro.test(tgfig) # non normal dist
## 
##  Shapiro-Wilk normality test
## 
## data:  tgfig
## W = 0.64049, p-value = 0.0001687
t.test(cgfig, tgfig, var.equal=F) # does not apply (would be significant)
## 
##  Welch Two Sample t-test
## 
## data:  cgfig and tgfig
## t = 3.6742, df = 9, p-value = 0.005121
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.2305913 0.9694087
## sample estimates:
## mean of x mean of y 
##       6.0       5.4