library(afex) # to run the ANOVA and plot results
library(psych) # for the describe() command
library(ggplot2) # to visualize our results
library(expss) # for the cross_cases() command
library(car) # for the leveneTest() command
library(emmeans) # for posthoc tests
library(effsize) # for the cohen.d() command
library(apaTables) # to create our correlation table
library(kableExtra) # to create our correlation table
library(sjPlot) # to visualize our resultsAI Experiment Analysis
Loading Libraries
Importing Data
# # import your AI results dataset
d <- read.csv(file="Data/final results.csv", header=T)State Your Hypotheses & Chosen Tests
I predict that participants in the exercise category will cause higher scores on the brief resiliency scale. I also predict participants with more hours of sleep will predict higher scores on the brief resiliency scale. I have chosen to run a t-Test for both hypothesis. # Check Your Variables This is just basic variable checking that is used across all HW assignments.
# # to view stats for all variables
describe(d) vars n mean sd median trimmed mad min max range skew
id 1 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
identity* 2 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
consent* 3 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
age 4 100 40.17 12.18 35.0 38.45 4.45 21 79 58 1.32
race* 5 100 4.44 1.87 6.0 4.58 0.00 1 7 6 -0.49
gender 6 100 1.92 0.27 2.0 2.00 0.00 1 2 1 -3.05
manip_out* 7 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
survey1 8 100 30.96 3.01 31.0 30.91 4.45 25 38 13 0.10
survey2 9 100 6.59 0.49 7.0 6.61 0.00 6 7 1 -0.36
ai_manip* 10 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
Condition 11 100 1.50 0.50 1.5 1.50 0.74 1 2 1 0.00
kurtosis se
id -1.24 2.90
identity* -1.24 2.90
consent* -1.24 2.90
age 1.32 1.22
race* -1.45 0.19
gender 7.38 0.03
manip_out* -1.24 2.90
survey1 -0.80 0.30
survey2 -1.89 0.05
ai_manip* -1.24 2.90
Condition -2.02 0.05#
# # we'll use the describeBy() command to view skew and kurtosis across our IVs
describeBy(d, group = "Condition")
Descriptive statistics by group
Condition: 1
vars n mean sd median trimmed mad min max range skew kurtosis
id 1 50 25.50 14.58 25.5 25.50 18.53 1 50 49 0.00 -1.27
identity 2 50 48.62 29.79 46.5 48.25 43.00 4 98 94 0.07 -1.47
consent 3 50 46.32 29.09 41.0 45.38 34.10 1 99 98 0.24 -1.22
age 4 50 40.46 12.54 35.0 38.35 4.45 21 79 58 1.46 1.50
race 5 50 4.46 1.83 6.0 4.60 0.00 1 6 5 -0.50 -1.56
gender 6 50 1.90 0.30 2.0 2.00 0.00 1 2 1 -2.59 4.79
manip_out 7 50 69.68 14.87 69.5 69.50 18.53 45 96 51 0.06 -1.21
survey1 8 50 30.98 3.08 31.0 30.92 2.97 25 38 13 0.11 -0.72
survey2 9 50 6.58 0.50 7.0 6.60 0.00 6 7 1 -0.31 -1.94
ai_manip 10 50 49.64 27.58 47.0 49.02 37.06 6 98 92 0.18 -1.24
Condition 11 50 1.00 0.00 1.0 1.00 0.00 1 1 0 NaN NaN
se
id 2.06
identity 4.21
consent 4.11
age 1.77
race 0.26
gender 0.04
manip_out 2.10
survey1 0.44
survey2 0.07
ai_manip 3.90
Condition 0.00
------------------------------------------------------------
Condition: 2
vars n mean sd median trimmed mad min max range skew kurtosis
id 1 50 75.50 14.58 75.5 75.50 18.53 51 100 49 0.00 -1.27
identity 2 50 52.38 28.39 54.0 52.80 28.91 1 100 99 -0.06 -1.03
consent 3 50 54.68 28.61 57.0 55.62 35.58 2 100 98 -0.24 -1.15
age 4 50 39.88 11.93 35.0 38.58 4.45 21 78 57 1.10 0.82
race 5 50 4.42 1.92 6.0 4.55 1.48 1 7 6 -0.46 -1.43
gender 6 50 1.94 0.24 2.0 2.00 0.00 1 2 1 -3.59 11.15
manip_out 7 50 31.32 26.99 25.5 26.62 18.53 1 100 99 1.45 1.24
survey1 8 50 30.94 2.97 31.0 30.90 4.45 25 37 12 0.09 -0.99
survey2 9 50 6.60 0.49 7.0 6.62 0.00 6 7 1 -0.40 -1.88
ai_manip 10 50 51.36 30.63 56.0 51.80 37.06 1 100 99 -0.15 -1.29
Condition 11 50 2.00 0.00 2.0 2.00 0.00 2 2 0 NaN NaN
se
id 2.06
identity 4.01
consent 4.05
age 1.69
race 0.27
gender 0.03
manip_out 3.82
survey1 0.42
survey2 0.07
ai_manip 4.33
Condition 0.00#
# # also use histograms and scatterplots to examine your continuous variables
hist(d$survey1)
hist(d$survey2)
plot(d$survey1, d$survey2)
#
# # and table() and cross_cases() to examine your categorical variables
# # you may not need the cross_cases code
table(d$Condition)
1 2
50 50 #
# # and boxplot to examine any categorical variables with continuous variables
boxplot(d$survey1~d$Condition)
boxplot(d$survey2~d$Condition)
#
# #convert any categorical variables to factors
d$Condition <- as.factor(d$Condition)Check Your Assumptions
t-Test Assumptions
- Data values must be independent (independent t-test only) (confirmed by data report)
- Data obtained via a random sample (confirmed by data report)
- IV must have two levels (will check below)
- Dependent variable must be normally distributed (will check below. if issues, note and proceed)
- Variances of the two groups must be approximately equal, aka ‘homogeneity of variance’. Lacking this makes our results inaccurate (will check below - this really only applies to Student’s t-test, but we’ll check it anyway)
Checking IV levels
# # preview the levels and counts for your IV
table(d$Condition, useNA = "always")
1 2 <NA>
50 50 0 #
# # note that the table() output shows you exactly how the levels of your variable are written. when recoding, make sure you are spelling them exactly as they appear
#
# # to drop levels from your variable
# # this subsets the data and says that any participant who is coded as 'BAD' should be removed
# d <- subset(d, IV != "BAD")
#
# table(d$iv, useNA = "always")
#
# # to combine levels
# # this says that where any participant is coded as 'BAD' it should be replaced by 'GOOD'
# d$iv_rc[d$iv == "BAD"] <- "GOOD"
#
# table(d$iv, useNA = "always")
#
# # check your variable types
# str(d)
#
# # make sure that your IV is recognized as a factor by R
# # if you created a new _rc variable make sure to use that one instead
# d$iv <- as.factor(d$iv)Testing Homogeneity of Variance with Levene’s Test
We can test whether the variances of our two groups are equal using Levene’s test. The null hypothesis is that the variance between the two groups is equal, which is the result we want. So when running Levene’s test we’re hoping for a non-significant result!
# # use the leveneTest() command from the car package to test homogeneity of variance
# # uses the same 'formula' setup that we'll use for our t-test: formula is y~x, where y is our DV and x is our IV
leveneTest(survey1~Condition, data = d)Levene's Test for Homogeneity of Variance (center = median)
Df F value Pr(>F)
group 1 0.014 0.9062
98 t-Test Assumptions
- Data values must be independent (independent t-test only) (confirmed by data report)
- Data obtained via a random sample (confirmed by data report)
- IV must have two levels (will check below)
- Dependent variable must be normally distributed (will check below. if issues, note and proceed)
- Variances of the two groups must be approximately equal, aka ‘homogeneity of variance’. Lacking this makes our results inaccurate (will check below - this really only applies to Student’s t-test, but we’ll check it anyway)
Checking IV levels
# # preview the levels and counts for your IV
table(d$Condition, useNA = "always")
1 2 <NA>
50 50 0 #
# # note that the table() output shows you exactly how the levels of your variable are written. when recoding, make sure you are spelling them exactly as they appear
#
# # to drop levels from your variable
# # this subsets the data and says that any participant who is coded as 'BAD' should be removed
# d <- subset(d, IV != "BAD")
#
# table(d$iv, useNA = "always")
#
# # to combine levels
# # this says that where any participant is coded as 'BAD' it should be replaced by 'GOOD'
# d$iv_rc[d$iv == "BAD"] <- "GOOD"
#
#table(d$iv, useNA = "always")
#
# # check your variable types
#str(d)
#
# # make sure that your IV is recognized as a factor by R
# # if you created a new _rc variable make sure to use that one instead
#d$iv <- as.factor(d$iv)Testing Homogeneity of Variance with Levene’s Test
We can test whether the variances of our two groups are equal using Levene’s test. The null hypothesis is that the variance between the two groups is equal, which is the result we want. So when running Levene’s test we’re hoping for a non-significant result!
# # use the leveneTest() command from the car package to test homogeneity of variance
# # uses the same 'formula' setup that we'll use for our t-test: formula is y~x, where y is our DV and x is our IV
leveneTest(survey2~Condition, data = d)Levene's Test for Homogeneity of Variance (center = median)
Df F value Pr(>F)
group 1 0.0405 0.8409
98 Issues with My Data
One issue with the data would be that AI had a difficult time estimate average hours slept per night per participant, which influences the results and testing for the second hypothesis.
Run Your Analysis
Run a t-Test
# # very simple! we specify the dataframe alongside the variables instead of having a separate argument for the dataframe like we did for leveneTest()
t_output <- t.test(d$survey1~d$Condition)View Test Output
t_output
Welch Two Sample t-test
data: d$survey1 by d$Condition
t = 0.066151, df = 97.858, p-value = 0.9474
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
-1.159981 1.239981
sample estimates:
mean in group 1 mean in group 2
30.98 30.94 Calculate Cohen’s d
# # once again, we use our formula to calculate cohen's d
d_output <- cohen.d(d$survey1~d$Condition)View Effect Size
- Trivial: < .2
- Small: between .2 and .5
- Medium: between .5 and .8
- Large: > .8
d_output
Cohen's d
d estimate: 0.01323023 (negligible)
95 percent confidence interval:
lower upper
-0.3836676 0.4101281 Run a t-Test
# # very simple! we specify the dataframe alongside the variables instead of having a separate argument for the dataframe like we did for leveneTest()
t_output <- t.test(d$survey2~d$Condition)View Test Output
t_output
Welch Two Sample t-test
data: d$survey2 by d$Condition
t = -0.20132, df = 97.995, p-value = 0.8409
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
-0.2171466 0.1771466
sample estimates:
mean in group 1 mean in group 2
6.58 6.60 Calculate Cohen’s d
# # once again, we use our formula to calculate cohen's d
d_output <- cohen.d(d$survey2~d$Condition)View Effect Size
- Trivial: < .2
- Small: between .2 and .5
- Medium: between .5 and .8
- Large: > .8
d_output
Cohen's d
d estimate: -0.04026381 (negligible)
95 percent confidence interval:
lower upper
-0.4371975 0.3566699 Write Up Results
t-Test
The first hypothesis that participants in the exercise category will cause higher scores on the Brief Resiliency Scale was tested using an independent t-test. Homogeneity of variance was tested using Levene’s Test and found a non-significant result (p = .014). As a result, Welch’s two sample t-Test was used. The data met all the assumptions of a t-test. The results found were not significant, t(97.99)= -.20 , p = .840 , d= -0.04, 95% [-0.21,0.17]. (refer to Figure 1) The effect size was negligible. The second hypothesis that participants with more hours of sleep will predict higher scores on the Brief Resiliency Scale was tested using an independent t-test. Homogeneity of variance was tested using Levene’s Test and found a significant result (p = .040). After such a result, further testing would not be needed. For the sake of the experiment, a Welch’s two-sample t-Test was used. The results found were not significant t(97.86)= .07 , p = .947 , d= 0.01, 95% [-0.38,0.41]. (refer to Figure 2) The effect size was negligible.
References
Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic.