t-test for a mean against a given value
data = c(21.75, 18.0875, 18.75, 23.5, 14.125, 16.75, 11.125, 11.125, 14.875, 15.5, 20.875,
17.125, 19.075, 25.125, 27.75, 29.825, 17.825, 28.375, 22.625, 28.75, 27, 12.825,
26, 32.825, 25.375, 24.825, 25.825, 15.625, 26.825, 24.625, 26.625, 19.625)
t.test(data, mu=24.1)##
## One Sample t-test
##
## data: data
## t = -2.4289, df = 31, p-value = 0.02114
## alternative hypothesis: true mean is not equal to 24.1
## 95 percent confidence interval:
## 19.48568 23.69792
## sample estimates:
## mean of x
## 21.5918# The above command tests the hypothesis that the population mean of MyVar is 45 and reports the p-value.
# The command for the t test above also reports the 95% confidence interval by default. We can change the confidence interval level by specifying conf.level. For example:
t.test(data, mu=24.1, conf.level=0.99)##
## One Sample t-test
##
## data: data
## t = -2.4289, df = 31, p-value = 0.02114
## alternative hypothesis: true mean is not equal to 24.1
## 99 percent confidence interval:
## 18.75814 24.42545
## sample estimates:
## mean of x
## 21.5918# The above command returns the 99% confidence interval for the sample mean as well as the P-value for a test of mu=45.Two-sample t significance test for a difference between two means and confidence interval for a difference between two means
Assume that in MyData, we have a variable group that identifies which observations are treated (group==”Treat”) and which observations are controls (group==”Control”). The outcome variable we are interested in is MyVar.
The command below tests whether the population mean difference of MyVar between treatment and control is different from 0, i.e. if MyVar is different in the treated vs. control groups on average:
t.test(subset(MyData,group=="Treat")$MyVar , subset(MyDate,group=="Control")$MyVar)The command reports the p-value for the test, and the 95% confidence interval for the difference between the two means. The “subset” instructs R to take only observations from the data where group==”Treat” in one case, and where group==”Control” in the other case. We can change the level for the confidence interval to 99% by specifying conf.level:
t.test(subset(MyData,group=="Treat")$MyVar,subset(MyData,group=="Control")$MyVar, conf.level=0.99)We first need to install the package that pwr:
library(pwr)The package pwr asks you to guess Cohen’s effect size (also called Cohen’s d) in the population. See here for a nice intuitive discussion: http://staff.bath.ac.uk/pssiw/stats2/page2/page14/page14.html. In the terminology of effect size, 0 is no effect, and this is the hypothesis we want to be able to reject. Cohen’s effect size d is the expected effect divided by the expected standard deviation. For example, if the effect is 20 (e.g. you expect the intervention to increase test scores by 20 points) and the standard deviation is 40, Cohen’s effect size is 20/40=0.5. For guidelines, effects are generally considered to be: • Small if the absolute value of d is 0.1 or less (i.e. -0.1< d <0.1) • Medium if the absolute value of d is between 0.1 and 0.5 • Large if the absolute value of d is above 0.5 Generally, it is wise to assume that your intervention will have a small effect, because this will ensure that you take a big enough sample to detect the effect.
Power calculation for a one-sample t-test
pwr.t.test(n = 1000, d = 0.1, sig.level = 0.05, power = NULL, type = c("one.sample"))##
## One-sample t test power calculation
##
## n = 1000
## d = 0.1
## sig.level = 0.05
## power = 0.8847891
## alternative = two.sided# The command above returns the power (power=NULL tells the command that we want it to find out what the power is) for a sample of n=1000, Cohen’s effect size of d=0.1, and significance level of 5% sig.level=0.05, with the type being specified as “one.sample”.You can experiment with alternative values to see how the power changes. This command can also be used to find out the sample size required for a given power this way:
pwr.t.test(n = NULL, d = 0.1, sig.level = 0.05, power = 0.8, type = c("one.sample"))##
## One-sample t test power calculation
##
## n = 786.8089
## d = 0.1
## sig.level = 0.05
## power = 0.8
## alternative = two.sided# The above command tells us what sample size we need for a power of 0.8, a Cohen’s effect size of d=0.1, and significance level of 5% sig.level=0.05.Power calculation for two samples mean difference
pwr.t2n.test(n1 = 800, n2= 500, d =0.15 , sig.level = 0.05, power = NULL)##
## t test power calculation
##
## n1 = 800
## n2 = 500
## d = 0.15
## sig.level = 0.05
## power = 0.7483383
## alternative = two.sided#The command above tells us what the power is for samples of n1=800 (first sample), n2=500 (second sample), a Cohen’s effect size of d=0.15, and significance level of 5% sig.level=0.05.As in the one sample case, we can also find out the sample size needed (either sample 1 or sample 2) for a given power.
pwr.t2n.test(n1 = 800, n2= NULL, d =0.15 , sig.level = 0.05, power = 0.8)##
## t test power calculation
##
## n1 = 800
## n2 = 620.0491
## d = 0.15
## sig.level = 0.05
## power = 0.8
## alternative = two.sided# The above command tells us the sample size for the second sample that is needed for an 80% power given that the first sample is 800, we have a Cohen’s effect size of d=0.15, and significance level of 5% sig.level=0.05