Intro to Statistics with R: Student’s T-test
Cristian Gulisano, Sarah Mbiki
Introduction to t-tests
Polling liberals and conservatives
You are responsible for conducting polls to understand voters’ preferences for a particular political candidate.
In the first poll, you want to understand how preferences vary between liberals and conservatives. You ask a group of liberals and a group of conservatives to each rate their likelihood of voting for the candidate.
Which test should you use to determine if there’s a significant difference in preferences between these two groups?
- Single sample t-test
- Dependent t-test
- Independent t-test
More political polls
In the previous exercise, you decided to conduct an independent t-test to compare liberals and convervatives, since you were comparing two independent samples at a single point in time. The poll was so successful that you decide to conduct two more.
In the second poll, you want to understand the effect of a campaign speech on voters’ preferences. You ask a single group of voters to rate their likelihood of voting for the candidate before the speech and again after the speech.
In the third poll, you want to understand if voters from a particular neighborhood are likely to vote differently when compared to the overall voting population. You poll voters from this neighborhood and compare the results to a recent and trustworthy national poll.
Which tests should you use for the second and third polls, respectively?
- Dependent t-test, independent t-test
- Independent t-test, single sample t-test
- Single sample t-test, dependent t-test
- Dependent t-test, single sample t-test
Significance tests
Which of the following are commonly referred to as significance tests?
- z-tests
- t-tests
- p-values
- Options 1 & 2 above
- None of the above
What’s a summary statistic?
A sampling distribution is a hypthotical distribution of a summary statistic from…
- Multiple samples of different sizes, each from the same underlying population.
- Multiple samples of the same size, each from the same underlying population.
- Multiple samples of different sizes, each from a different underlying population.
- Multiple samples of the same size, each from a different underlying population.
- A single sample from the underlying population.
Single sample t-tests
You want to know how a group of people from a particular geographic region perform on a well-known test of intelligence. In particular, you are interested in finding out whether or not this group scores significantly higher than the overall population on an IQ test. This is a form of Null Hypothesis Significance Testing (NHST), where the null hypothesis is that there’s no difference between this group and the overall population.
Using a random sample of 10 subjects from the group, you find that their mean IQ is 1.5 standard errors higher than the mean IQ of the population. What do you conclude (using a significance level of 5%) regarding the group’s intelligence? (Recall that the critical value for the z-distribution at a significance level of 5% is 1.96.)
- There is not enough information to make a conclusion.
- The group’s IQ is not significantly different from the mean IQ of the population.
- The group’s IQ is significantly different from the mean IQ of the population.
- The group’s IQ is equal to the mean IQ of the population.
Understanding the t-distribution
When performing a t-test, you first calculate your t-statistic using the familiar formula: \[ t=\frac {X - M}{SE} \]
\(X\) is the observed value, \(M\) is the expected value under the null hypothesis (or population mean), and \(SE\) is the standard error. Once you’ve computed the t-statistic, you then compare it to the so-called critical value, which comes from the relevant t-distribution.
The shape of a t-distribution, and thus the critical value, is determined entirely by its degrees of freedom. To demonstrate this, let’s draw some density plots for t-distributions using different degrees of freedom.
Instructions
- Create a vector
xthat contains a sequence of length 100 between -4 and 4. See?seqfor help. - Use
dt()to generate t-distributions with 4, 6, 8, 10, and 12 degrees of freedom (in that order). The first argument todt()is the vector of values at which to evalute the t-distribution (xfrom above) and the second argument (df) is the degrees of freedom. - Plot each of the t-distributions. Once the inital
plot()is created, you’ll uselines()to plot each additional distribution. The two arguments tolines()are the same as the first two arguments toplot(), except that you’ll have to substitute the appropriate y-values. Use the color black for 4 degrees of freedom, red for 6, orange for 8, green for 10, and blue for 12. - Add a
legend()to your plot. The legend should be situated at the top right corner of your plot and should have the title “t-distributions”. This is done by setting the first argument to"topright"and thetitleargument to"t-distributions".
# Generate a vector of 100 values between -4 and 4
x <- seq(-4, 4, length = 100)
# Simulate the t-distribution
y_1 <- dt(x, df = 4)
y_2 <- dt(x, df = 6)
y_3 <- dt(x, df = 8)
y_4 <- dt(x, df = 10)
y_5 <- dt(x, df = 12)
# Plot the t-distributions
plot(x, y_1, type = "l", lwd = 2, xlab = "t-value", ylab = "Density",
main = "Comparison of t-distributions", col = "black")
lines(x, y_2, col = "red")
lines(x, y_3, col = "orange")
lines(x, y_4, col = "green")
lines(x, y_5, col = "blue")
# Add a legend
legend("topright", c("df = 4", "df = 6", "df = 8", "df = 10", "df = 12"),
col = c("black", "red", "orange", "green", "blue"),
title = "t-distributions", lty = 1)
The working memory dataset
In the following exercises, you will conduct a dependent (or paired) t-test on the “working memory” dataset. This dataset consists of the intelligence scores for subjects before and after training, as well as for a control group. Our goal is to assess whether intelligence training results in significantly different intelligence scores for the individuals.
The observations of individuals before and after training are two samples from the same group at different points in time, which calls for a dependent t-test. This will test whether or not the difference in mean intelligence scores before and after training are significant.
The working memory dataset has been loaded into your workspace as the object wm. It contains the data for both the group who received training and the group who did not.
Instructions
- Print
wmto the console to get a feel for the data - Create a subset of
wmthat includes only the training group and store the result inwm_t. A value of1in thetraincolumn indicates that a subject received training, while a value of0indicates that they did not. - View summary statistics for
wm_twith thedescribe()function. - Use the
boxplot()function to create a boxplot of thepreandpostcolumn ofwm_t. Give the x-axis the label “Pre- and Post-Training” and the y-axis the label “Intelligence Score”.
# Take a look at the dataset
wm cond pre post gain train
1 t08 8 9 1 1
2 t08 8 10 2 1
3 t08 8 8 0 1
4 t08 8 7 -1 1
5 t08 9 11 2 1
6 t08 9 10 1 1
7 t08 9 12 3 1
8 t08 9 11 2 1
9 t08 10 11 1 1
10 t08 10 11 1 1
11 t08 10 9 -1 1
12 t08 10 12 2 1
13 t08 11 12 1 1
14 t08 11 14 3 1
15 t08 11 14 3 1
16 t08 12 12 0 1
17 t08 12 13 1 1
18 t08 12 13 1 1
19 t08 12 14 2 1
20 t08 12 15 3 1
21 t12 8 10 2 1
22 t12 8 10 2 1
23 t12 9 13 4 1
24 t12 9 11 2 1
25 t12 9 11 2 1
26 t12 9 14 5 1
27 t12 10 12 2 1
28 t12 10 14 4 1
29 t12 10 12 2 1
30 t12 10 12 2 1
31 t12 11 11 0 1
32 t12 11 13 2 1
33 t12 11 15 4 1
34 t12 12 14 2 1
35 t12 12 14 2 1
36 t12 12 17 5 1
37 t12 12 14 2 1
38 t12 8 12 4 1
39 t12 8 10 2 1
40 t12 9 11 2 1
41 t17 9 13 4 1
42 t17 9 13 4 1
43 t17 9 16 7 1
44 t17 10 13 3 1
45 t17 10 16 6 1
46 t17 10 14 4 1
47 t17 10 14 4 1
48 t17 11 14 3 1
49 t17 11 14 3 1
50 t17 11 17 6 1
51 t17 12 16 4 1
52 t17 12 16 4 1
53 t17 12 19 7 1
54 t17 12 15 3 1
55 t17 8 14 6 1
56 t17 8 12 4 1
57 t17 9 13 4 1
58 t17 9 12 3 1
59 t17 9 12 3 1
60 t17 9 15 6 1
61 t19 10 17 7 1
62 t19 10 15 5 1
63 t19 10 19 9 1
64 t19 10 16 6 1
65 t19 11 15 4 1
66 t19 11 18 7 1
67 t19 11 16 5 1
68 t19 12 15 3 1
69 t19 12 18 6 1
70 t19 12 16 4 1
71 t19 12 19 7 1
72 t19 8 13 5 1
73 t19 8 17 9 1
74 t19 9 15 6 1
75 t19 9 13 4 1
76 t19 9 16 7 1
77 t19 9 14 5 1
78 t19 10 13 3 1
79 t19 10 16 6 1
80 t19 10 14 4 1
81 control 8 9 1 0
82 control 8 10 2 0
83 control 8 8 0 0
84 control 8 7 -1 0
85 control 9 11 2 0
86 control 9 10 1 0
87 control 9 12 3 0
88 control 9 11 2 0
89 control 10 11 1 0
90 control 10 11 1 0
91 control 10 9 -1 0
92 control 10 12 2 0
93 control 11 12 1 0
94 control 11 14 3 0
95 control 11 14 3 0
96 control 12 12 0 0
97 control 12 13 1 0
98 control 12 13 1 0
99 control 12 14 2 0
100 control 12 15 3 0
101 control 8 10 2 0
102 control 8 10 2 0
103 control 9 13 4 0
104 control 9 11 2 0
105 control 9 11 2 0
106 control 9 14 5 0
107 control 10 12 2 0
108 control 10 14 4 0
109 control 10 12 2 0
110 control 10 12 2 0
111 control 11 11 0 0
112 control 11 13 2 0
113 control 11 15 4 0
114 control 12 14 2 0
115 control 12 14 2 0
116 control 12 17 5 0
117 control 12 14 2 0
118 control 8 12 4 0
119 control 8 10 2 0
120 control 9 11 2 0# Create training subset of wm
wm_t <- subset(wm, wm$train == "1")
# Summary statistics
describe(wm_t)wm_t
5 Variables 80 Observations
---------------------------------------------------------------------------
cond
n missing distinct
80 0 4
Value t08 t12 t17 t19
Frequency 20 20 20 20
Proportion 0.25 0.25 0.25 0.25
---------------------------------------------------------------------------
pre
n missing distinct Info Mean Gmd
80 0 5 0.955 10.03 1.551
Value 8 9 10 11 12
Frequency 12 20 19 12 17
Proportion 0.150 0.250 0.238 0.150 0.212
---------------------------------------------------------------------------
post
n missing distinct Info Mean Gmd .05 .10
80 0 13 0.984 13.51 2.87 9.95 10.00
.25 .50 .75 .90 .95
12.00 14.00 15.00 17.00 18.00
Value 7 8 9 10 11 12 13 14 15 16
Frequency 1 1 2 5 8 11 11 15 8 9
Proportion 0.012 0.012 0.025 0.062 0.100 0.138 0.138 0.188 0.100 0.112
Value 17 18 19
Frequency 4 2 3
Proportion 0.050 0.025 0.038
---------------------------------------------------------------------------
gain
n missing distinct Info Mean Gmd .05 .10
80 0 10 0.975 3.487 2.415 0.0 1.0
.25 .50 .75 .90 .95
2.0 3.0 5.0 6.1 7.0
Value -1 0 1 2 3 4 5 6 7 9
Frequency 2 3 7 18 12 16 6 8 6 2
Proportion 0.025 0.038 0.088 0.225 0.150 0.200 0.075 0.100 0.075 0.025
---------------------------------------------------------------------------
train
n missing distinct Info Mean Gmd
80 0 1 0 1 0
Value 1
Frequency 80
Proportion 1
---------------------------------------------------------------------------# Create a boxplot with pre- and post-training groups
boxplot(wm_t$pre, wm_t$post, main = "Boxplot",
xlab = "Pre- and Post-Training", ylab = "Intelligence Score",
col = c("red", "green"))
Perform a dependent t-test
Conducting a dependent t-test, also known as a paired t-test, requires the following steps:
- Define null and alternative hypotheses
- Decide significance level \(\alpha\)
- Compute observed t-value
- Find critical value
- Compare observed value to critical value
We’re performing a Null Hypothesis Significance Test (NHST), so our null hypothesis is that there’s no effect (i.e. training has no impact on intelligence scores). The alternative hypothesis is that training results in signficantly different intelligence scores. We’ll use a significance level of 0.05, which is very common in statistics. That takes care of the first two steps!
In this exercise, we’ll focus on computing the observed t-value, which is computed as follows: \[ t = \frac {\bar{x}_{D}}{s_{D}/\sqrt{n}}\] \(n\) is just the sample size, or the number of individuals in our sample. \(\bar{x}_{D}\) is the mean of the difference scores, or sum of the difference scores divided by the sample size. Finally, \({s}_{D}\) is the standard deviation of the difference scores: \[ {s}_{D}=\sqrt{ \frac{\sum{({x}_{D}}-\bar{x}_D)^2}{n-1}} \]
In the formula for \({s}_D\), \({x}_D\) are the individual difference scores and should not be confused with \(\bar{x}_D\), which is the mean of the difference scores.
Instructions
- Use the code provided to assign the sample size to
n. - Calculate the mean of the difference scores by summing up the differences with
sum()and dividing byn. The differences are contained in thegaincolumn ofwm_t. - Compute the standard deviation of the difference scores as defined above. Use
nandmean_diffin your calculation and be careful with your brackets! Save the result tosd_diff. - Compute the observed t-value by combining
mean_diff,sd_diff, andn. Store the result int_obs.
## The training subset, wm_t, is available in your workspace
# Define the sample size
n <- nrow(wm_t)
# Mean of the difference scores
mean_diff <- sum(wm_t$gain) / n
# Standard deviation of the difference scores
sd_diff <- sqrt(sum((wm_t$gain - mean_diff)^2) / (n - 1))
# Observed t-value
t_obs <- mean_diff / (sd_diff / sqrt(n))
# Print observed t-value
t_obs[1] 14.49238Perform a dependent t-test(2)
Now that we’ve determined our null and alternative hypotheses, decided on a significance level, and computed our observed t-value, all that remains is to calculate the critical value for this test and compare it to our observed t-value. This will tell us whether we have sufficient evidence to reject our null hypothesis. We’ll even go one step further and compute an effect size with Cohen’s d!
The critical value is the point on the relevant t-distribution that determines whether the value we observed is extreme enough to warrant rejecting the null hypothesis. Recall that a t-distribution is defined by its degrees of freedom, which in turn is equal to the sample size minus 1. In this example, we have 80 subjects so the relevant t-distribution has 79 degrees of freedom.
We’re performing a two-tailed t-test in this situation since we care about detecting a significant effect in either the positive or negative direction. In other words, we want to know if training significantly increases or decreases intelligence, however, given that our observed t-value is positive (14.49) the right-hand is the only relevant value here.
Furthermore, since our desired significance level (i.e. alpha) is 0.05, our critical value is the point on our t-distribution at which 0.025 (0.05 / 2) of its total area of 1 is to the right and thus 0.975 (1 - 0.025) of its total area is to the left.
This point is called the 0.975 quantile and is computed for a t-distrbution in R using the qt() function.
Instructions
- Compute the right critical value and store the result in
t_crit. Useqt()with two arguments (in order): - The desired area to the left of the critical value (i.e. quantile value)
- The relevant degrees of freedom for our t-distribution
- Print the critical value to the console
- Print the observed t-value (
t_obs) to compare with the critical value - Compute Cohen’s d, which is the mean of the difference scores divided by the standard deviation of the differences scores.
mean_diffandsd_difffrom the previous exercise are still available in your workspace. Save the result tocohens_d. - Print
cohens_dto the console.
## The variables from the previous exercise are preloaded
# Compute the critical value
t_crit <- qt(0.975, df = 79)
# Print the critical value
t_crit[1] 1.99045# Print the observed t-value to compare
t_obs[1] 14.49238# Compute Cohen's d
cohens_d <- mean_diff / sd_diff
# View Cohen's d
cohens_d[1] 1.620297Compare the observed and critical t-values
The variables t_obs and t_crit are still loaded in your workspace. Is the difference in intelligence scores before and after training significantly different from zero?
- Yes, the difference is statistically significant.
- No, the difference is not statistically significant.
- We don’t have enough information to draw a conclusion.
Interpreting the effect size
The variable cohens_d is still loaded in your workspace. You’ve determined that the difference pre- and post-training is statistically significant at \(\alpha=0.05\). How would you characterize the effect?
- The effect is very strong.
- The effect is quite weak.
- We need more information to say whether the effect is strong or weak.
Let R do the dirty work
Thanks to the handy t.test() function in R, it’s not necessary to manually compute the t-statistic every time you want to do a t-test. Similarly, the cohensD() function from the lsr package automates the process of computing Cohen’s d. We’ll continue with the working memory example to illustrate.
In the case of our dependent t-test, we need to specify these arguments to t.test():
x: Column ofwm_tcontaining post-training intelligence scoresy: Column ofwm_tcontaining pre-training intelligence scorespaired: Whether we’re doing a dependent (i.e. paired) t-test or independent t-test. In this example, it’sTRUE
Note that t.test() carries out a two-sided t-test by default, which is exactly what we want here, but this behavior can be altered with the alternative argument. See ?t.test for more info.
For cohensD(), we’ll need to specify three arguments:
x: Column ofwm_tcontaining post-training intelligence scoresy: Column ofwm_tcontaining pre-training intelligence scoresmethod: Version of Cohen’s d to compute, which should be"paired"in this case
If you’re successful, you’ll get the same values you got when you did everything “by hand” in the previous exercises.
## The lsr package has been loaded for you
# Apply the t.test function
t.test(wm_t$post, wm_t$pre, paired = TRUE)
Paired t-test
data: wm_t$post and wm_t$pre
t = 14.492, df = 79, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
3.008511 3.966489
sample estimates:
mean of the differences
3.4875 # Calculate Cohen's d
cohensD(wm_t$post, wm_t$pre, method = "paired")[1] 1.620297Pop quiz!
You are interested in the effect of training on a particular test of performance. You take a group of 100 people and find that the group had an average gain in performance of 10 points after training compared to before training. Additionally, you find that the standard error of the individual differences is equal to 5. What is the value of the t-statistic?
- 1
- 20
- 2
- There is not enough information to answer the question.
Independent t-tests
What’s an independent t-test?
An independent t-test is appropriate when you want to compare the the means for two independent groups. True or false?
- True
- False
What’s a t-value?
Which option best describes a t-value?
- An expected value divided by standard error
- An observed value minus an expected value, divided by standard deviation
- An observed value minus an expected value, divided by standard error
- An expected value divided by standard deviation
- None of the above
Preliminary statistics
For independent t-tests you will revisit the working memory dataset from the previous chapter. In this dataset, subjects were randomly assigned to four different training groups that trained for 8, 12, 17 and 19 days, respectively (column cond in wm_t).
The scores for each subject were recorded before and after undergoing the training regime. You are interested in determining if there is a significant difference in the gain in intelligence scores between groups that trained for different lengths of time.
Instructions
The wm_t dataset has been loaded into your workspace.
- Print
wm_tto the console to refresh your memory of its contents. - Create a subset of the data for each training group and save each in a variable called
wm_t##, where##is08,12,17or19. - View summary statistics for each subset using the
describe()function from thepsychpackage, which has been loaded for you. Thegainrow tells you the gain in intelligence test scores before and after training. - Create a boxplot using the
ggplot2package to visualize differences in the group means. This code has been provided for you, so you just need to run it as-is. - Use
leveneTest()to perform Levene’s test for homogeneity of variance. This will check if all groups contain an equivalent amount of variance, which is a necessary precondition for the use of pooled standard deviation in the t-statistic for independent groups. Note that the function uses a formula interface, so passwm_t$gainto the left of~andwm_t$condto the right.
# View the wm_t dataset
wm_t cond pre post gain train
1 t08 8 9 1 1
2 t08 8 10 2 1
3 t08 8 8 0 1
4 t08 8 7 -1 1
5 t08 9 11 2 1
6 t08 9 10 1 1
7 t08 9 12 3 1
8 t08 9 11 2 1
9 t08 10 11 1 1
10 t08 10 11 1 1
11 t08 10 9 -1 1
12 t08 10 12 2 1
13 t08 11 12 1 1
14 t08 11 14 3 1
15 t08 11 14 3 1
16 t08 12 12 0 1
17 t08 12 13 1 1
18 t08 12 13 1 1
19 t08 12 14 2 1
20 t08 12 15 3 1
21 t12 8 10 2 1
22 t12 8 10 2 1
23 t12 9 13 4 1
24 t12 9 11 2 1
25 t12 9 11 2 1
26 t12 9 14 5 1
27 t12 10 12 2 1
28 t12 10 14 4 1
29 t12 10 12 2 1
30 t12 10 12 2 1
31 t12 11 11 0 1
32 t12 11 13 2 1
33 t12 11 15 4 1
34 t12 12 14 2 1
35 t12 12 14 2 1
36 t12 12 17 5 1
37 t12 12 14 2 1
38 t12 8 12 4 1
39 t12 8 10 2 1
40 t12 9 11 2 1
41 t17 9 13 4 1
42 t17 9 13 4 1
43 t17 9 16 7 1
44 t17 10 13 3 1
45 t17 10 16 6 1
46 t17 10 14 4 1
47 t17 10 14 4 1
48 t17 11 14 3 1
49 t17 11 14 3 1
50 t17 11 17 6 1
51 t17 12 16 4 1
52 t17 12 16 4 1
53 t17 12 19 7 1
54 t17 12 15 3 1
55 t17 8 14 6 1
56 t17 8 12 4 1
57 t17 9 13 4 1
58 t17 9 12 3 1
59 t17 9 12 3 1
60 t17 9 15 6 1
61 t19 10 17 7 1
62 t19 10 15 5 1
63 t19 10 19 9 1
64 t19 10 16 6 1
65 t19 11 15 4 1
66 t19 11 18 7 1
67 t19 11 16 5 1
68 t19 12 15 3 1
69 t19 12 18 6 1
70 t19 12 16 4 1
71 t19 12 19 7 1
72 t19 8 13 5 1
73 t19 8 17 9 1
74 t19 9 15 6 1
75 t19 9 13 4 1
76 t19 9 16 7 1
77 t19 9 14 5 1
78 t19 10 13 3 1
79 t19 10 16 6 1
80 t19 10 14 4 1# Create subsets for each training time
wm_t08 <- subset(wm_t, wm_t$cond == "t08")
wm_t12 <- subset(wm_t, wm_t$cond == "t12")
wm_t17 <- subset(wm_t, wm_t$cond == "t17")
wm_t19 <- subset(wm_t, wm_t$cond == "t19")
# Summary statistics for the change in training scores before and after training
describe(wm_t08)wm_t08
5 Variables 20 Observations
---------------------------------------------------------------------------
cond
n missing distinct value
20 0 1 t08
Value t08
Frequency 20
Proportion 1
---------------------------------------------------------------------------
pre
n missing distinct Info Mean Gmd
20 0 5 0.959 10.05 1.742
Value 8 9 10 11 12
Frequency 4 4 4 3 5
Proportion 0.20 0.20 0.20 0.15 0.25
---------------------------------------------------------------------------
post
n missing distinct Info Mean Gmd
20 0 9 0.98 11.4 2.474
Value 7 8 9 10 11 12 13 14 15
Frequency 1 1 2 2 4 4 2 3 1
Proportion 0.05 0.05 0.10 0.10 0.20 0.20 0.10 0.15 0.05
---------------------------------------------------------------------------
gain
n missing distinct Info Mean Gmd
20 0 5 0.934 1.35 1.384
Value -1 0 1 2 3
Frequency 2 2 7 5 4
Proportion 0.10 0.10 0.35 0.25 0.20
---------------------------------------------------------------------------
train
n missing distinct Info Mean Gmd
20 0 1 0 1 0
Value 1
Frequency 20
Proportion 1
---------------------------------------------------------------------------describe(wm_t12)wm_t12
5 Variables 20 Observations
---------------------------------------------------------------------------
cond
n missing distinct value
20 0 1 t12
Value t12
Frequency 20
Proportion 1
---------------------------------------------------------------------------
pre
n missing distinct Info Mean Gmd
20 0 5 0.959 9.9 1.674
Value 8 9 10 11 12
Frequency 4 5 4 3 4
Proportion 0.20 0.25 0.20 0.15 0.20
---------------------------------------------------------------------------
post
n missing distinct Info Mean Gmd
20 0 7 0.966 12.5 2.137
Value 10 11 12 13 14 15 17
Frequency 3 4 4 2 5 1 1
Proportion 0.15 0.20 0.20 0.10 0.25 0.05 0.05
---------------------------------------------------------------------------
gain
n missing distinct Info Mean Gmd
20 0 4 0.718 2.6 1.274
Value 0 2 4 5
Frequency 1 13 4 2
Proportion 0.05 0.65 0.20 0.10
---------------------------------------------------------------------------
train
n missing distinct Info Mean Gmd
20 0 1 0 1 0
Value 1
Frequency 20
Proportion 1
---------------------------------------------------------------------------describe(wm_t17)wm_t17
5 Variables 20 Observations
---------------------------------------------------------------------------
cond
n missing distinct value
20 0 1 t17
Value t17
Frequency 20
Proportion 1
---------------------------------------------------------------------------
pre
n missing distinct Info Mean Gmd
20 0 5 0.939 10 1.526
Value 8 9 10 11 12
Frequency 2 7 4 3 4
Proportion 0.10 0.35 0.20 0.15 0.20
---------------------------------------------------------------------------
post
n missing distinct Info Mean Gmd
20 0 7 0.966 14.4 2.084
Value 12 13 14 15 16 17 19
Frequency 3 4 5 2 4 1 1
Proportion 0.15 0.20 0.25 0.10 0.20 0.05 0.05
---------------------------------------------------------------------------
gain
n missing distinct Info Mean Gmd
20 0 4 0.902 4.4 1.516
Value 3 4 6 7
Frequency 6 8 4 2
Proportion 0.3 0.4 0.2 0.1
---------------------------------------------------------------------------
train
n missing distinct Info Mean Gmd
20 0 1 0 1 0
Value 1
Frequency 20
Proportion 1
---------------------------------------------------------------------------describe(wm_t19)wm_t19
5 Variables 20 Observations
---------------------------------------------------------------------------
cond
n missing distinct value
20 0 1 t19
Value t19
Frequency 20
Proportion 1
---------------------------------------------------------------------------
pre
n missing distinct Info Mean Gmd
20 0 5 0.939 10.15 1.447
Value 8 9 10 11 12
Frequency 2 4 7 3 4
Proportion 0.10 0.20 0.35 0.15 0.20
---------------------------------------------------------------------------
post
n missing distinct Info Mean Gmd
20 0 7 0.971 15.75 2.153
Value 13 14 15 16 17 18 19
Frequency 3 2 4 5 2 2 2
Proportion 0.15 0.10 0.20 0.25 0.10 0.10 0.10
---------------------------------------------------------------------------
gain
n missing distinct Info Mean Gmd
20 0 6 0.968 5.6 1.979
Value 3 4 5 6 7 9
Frequency 2 4 4 4 4 2
Proportion 0.1 0.2 0.2 0.2 0.2 0.1
---------------------------------------------------------------------------
train
n missing distinct Info Mean Gmd
20 0 1 0 1 0
Value 1
Frequency 20
Proportion 1
---------------------------------------------------------------------------# Create a boxplot of the different training times
ggplot(wm_t, aes(x = cond, y = gain, fill = cond)) + geom_boxplot()
# Levene's test
leveneTest(wm_t$gain ~ wm_t$cond)Levene's Test for Homogeneity of Variance (center = median)
Df F value Pr(>F)
group 3 1.3134 0.2763
76 Interpreting Levene’s test
The results from Levene’s test conducted in the previous exercise are available in your workspace in the object levene_test. Do the data violate the homogeneity of variance assumption?
- Yes, there is a significant difference between the group variances.
- No, there is not a significant difference between the group variances.
Perform an independent t-test
In this exercise, you’ll manually perform an independent t-test the same way you did for the dependent t-test in the previous chapter. Continuing with the working memory example, our null hypothesis is that the difference in intelligence score gain between the group that trained for 8 days and the group that trained for 19 days is equal to zero. If our observed t-value is sufficiently large, we can reject the null in favor of the alternative hypothesis, which would imply a significant difference in intelligence gain between the two training groups.
Calculation of the observed t-value for an independent t-test is similar to the dependent t-test, but involves slightly different formulas. The t-value is now \[t=\frac{( \bar{x}_1 - \bar{x}_2)}{se_p} \] where \(\bar{x}_1\) and \(\bar{x}_2\) are the mean intelligence gains for group 1 and group 2, respectively. \(se_p\) is the pooled standard error, which is equivalent to \[se_p=\sqrt{\frac{var_1}{n_1}+\frac{var_2}{n_2}} \] where \(var_1\) and \(var_2\) are the variances and \(n_1\) and \(n_2\) are the sample sizes of groups 1 and 2, respectively.
Instructions
The subsets of data for both 8 and 19 days of training, wm_t08 and wm_t19, are available in your workspace. Recall that the gain column contains the gain in intelligence score before and after training for each subject.
- Compute the mean intelligence score gain for each of the two training groups and store the results in
mean_t08andmean_t19. Use themean()function to do this. - Use the objects
mean_t08andmean_t19to find the difference in means. Subtract the lowest mean from the highest mean and store the result inmean_diff. - Determine the number of participants in each sample using the code provided for you. Nothing to change here.
- Calculate the degrees of freedom for the relevant t-distribution. The formula for degrees of freedom in an independent t-test is \(n1 + n2 - 2\). Use the sample sizes you created above and save the result to
df. - Create
var_t08andvar_t19by applying thevar()function to the intelligence gain for each of the two training groups. These objects represent the variance for each group. - Compute the pooled standard error
se_pooledusing the formula outlined above. You will need to use thesqrt()function in addition to the other variables you’ve created in this exercise. Mind your brackets!
# Calculate mean difference by subtracting the gain for t08 by the gain for t19
mean_t08 <- mean(wm_t08$gain)
mean_t19 <- mean(wm_t19$gain)
mean_diff <- (mean_t19 - mean_t08)
# Calculate degrees of freedom
n_t08 <- dim(wm_t08)[1]
n_t19 <- dim(wm_t19)[1]
df <- n_t08 + n_t19 - 2
# Calculate the pooled standard error
var_t08 <- (sum((wm_t08$gain - mean_t08)^2))/(n_t08 - 1)
var_t19 <- (sum((wm_t19$gain - mean_t19)^2))/(n_t19 - 1)
se_pooled <- sqrt((var_t08/n_t08) + (var_t19/n_t19))Perform an independent t-test(2)
In this exercise, we’ll finish computing the observed t-value. Then we will determine the p-value using the relevant t-distribution. If you recall, in the last chapter we calculated the critical value. The p-value is simply an alternative approach to hypothesis testing and determining the significance of your results. It’s good to practice both! Finally, we will finish by calculating effect size via Cohen’s d.
Recall from the previous exercise, the formula for the t-value is \[ t=\frac{\bar{x}_1 - \bar{x}_2}{se_p}\] where \(\bar{x}_1\) and \(\bar{x}_2\) are the mean intelligence gains for group 1 and group 2, respectively, and \(se_p\) is the pooled standard error.
The formula for Cohen’s d for independent t-tests is \[ d = \frac{\bar{x}_1 - \bar{x}_2}{sd_p}\] where \(sd_p\) is the pooled standard deviation, which in turn is equal to \[sd_p=\frac{sd_1 + sd_2}{2} \] where \(sd_1\) and \(sd_2\) are the standard deviations of the first and second groups, respectively.
Instructions
- Calculate the observed t-value according to the formula above. Use
mean_diffandse_pooled, which are still loaded in your workspace from the last exercise, and store the result int_value. - Compute the p-value using
pt(). The first argument is the t-value you just computed and the second argument is the degrees of freedom (stored indffrom last exercise). We multiply the result by 2 since this is a two-sided hypothesis test. - Calculate the standard deviation of the intelligence gain for the 8-day and 19-day training groups using the
sd()function. - Calculate the pooled standard deviation using the formula outlined above and the objects created in the previous instruction.
- Calculate
cohens_dusing the formula provided above. All you need ismean_diffandpooled_sd.
# All variables from the previous exercises are preloaded in your workspace
# Type ls() to see them
# Calculate the t-value
t_value <- mean_diff/se_pooled
# Calculate p-value
#two-tail test, 0.05/2 = 0.025
p_value <- 2*(1-pt(t_value,df = df))
# Calculate Cohen's d
sd_t08 <- sd(wm_t08$gain)
sd_t19 <- sd(wm_t19$gain)
pooled_sd <- (sd_t08 + sd_t19) / 2
cohens_d <- mean_diff/pooled_sdInterpreting the p-value
Is the difference in mean intelligence gains between groups statistically significant at a significance level (i.e. \(\alpha\)) of 0.05? The variable p_value is still loaded in your workspace.
- Yes, the difference is statistically significant.
- No, the difference is not statistically significant.
- I do not have enough information to answer the question.
Interpreting the effect size
Is the effect of training on intelligence gains particularly strong? Check out the variable cohens_d, which is still loaded in your workspace.
- Yes, it is very strong.
- Yes, but only moderately so.
- No, it is very weak.
- There is no training effect.
Independent t-tests, the easy way
As you saw in the last chapter, the t.test() function makes it easy to perform t-tests. This applies for independent t-tests in addition to dependent t-tests.
For an independent t-test, we specify these arguments to t.test():
x: Column of data containing the first group’s intelligence scoresy: Column of data containing the second group’s intelligence scoresvar.equal: Whether to assume the variance is equal in both groups. We do make this assumption here, so it should beTRUE
Note that we don’t have to use the paired argument for an independent t-test, since this is the default behavior of the t.test() function. Check out ?t.test for more info.
To compute Cohen’s d quickly, use cohensD() with three arguments:
x: Column of data containing the first group’s intelligence scoresy: Column of data containing the second group’s intelligence scoresmethod: Version of Cohen’s d to compute, which should be"pooled"in this case
You should get the same values you got “by hand” in the previous exercises.
Instructions
The wm_t08 and wm_t19 datasets have been loaded into your workspace.
- Apply
t.test()to test whether there’s a significant difference between groups who trained for 19 days verses those who trained for only 8 days. Recall that intelligence score gains are contained in thegaincolumn of each data frame. Don’t save the result. - Apply
cohensD()to compute Cohen’s d. Use the same columns as inputs forxandy. Don’t save the result.
## The subsets wm_t08 and wm_t19 are preloaded in your workspace
# Conduct an independent t-test
t.test(wm_t19$gain, wm_t08$gain,var.equal = TRUE)
Two Sample t-test
data: wm_t19$gain and wm_t08$gain
t = 8.9677, df = 38, p-value = 6.443e-11
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
3.290588 5.209412
sample estimates:
mean of x mean of y
5.60 1.35 # Calculate Cohen's d
cohensD(wm_t19$gain, wm_t08$gain, method = 'pooled')[1] 2.835822Understanding Cohen’s d
Which of the following statements is not true?
- Cohen’s d is unbiased by sample size.
- Cohen’s d measures significance based on a predetermined significance level.
- Cohen’s d provides a standardized difference between two means.
- Cohen’s d is calculated by subtracting one group mean from the other, then dividing by the pooled standard deviation.
How many degrees of freedom
Let’s say you’ve just completed a study comparing some measure of interest between two groups: one with 15 participants and the other with 13. What are the degrees of freedom for an independent t-test comparing these two groups?
- 25
- 28
- 26
- 27