library(afex) # to run the ANOVA and plot results
library(psych) # for the describe() command
library(ggplot2) # to visualize our results
library(expss) # for the cross_cases() command
library(car) # for the leveneTest() command
library(emmeans) # for posthoc tests
library(effsize) # for the cohen.d() command
library(apaTables) # to create our correlation table
library(kableExtra) # to create our correlation table
library(sjPlot) # to visualize our resultsAI Experiment Analysis
Loading Libraries
Importing Data
# # import your AI results dataset
d <- read.csv(file="Data/final_results_final.csv", header=T)State Your Hypotheses & Chosen Tests
Hypothesis 1: Participants who receive supportive feedback will report higher satisfaction with life than participants who receive neutral feedback.
Hypothesis 2: Perceived Social Support will predict satisfaction with life, and the relationship will be positive. (Participants who report higher perceived social support will report greater satisfaction with life.)
My chosen tests are an independent sample t-test and a Correlation test.
Check Your Variables
This is just basic variable checking that is used across all HW assignments.
# # to view stats for all variables
describe(d) vars n mean sd median trimmed mad min max range skew
id 1 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
identity* 2 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
consent* 3 100 46.33 26.87 44.5 46.04 32.62 1 94 93 0.10
age 4 100 23.80 1.04 24.0 23.88 0.00 20 27 7 -0.71
race 5 100 4.99 1.47 6.0 5.06 0.00 2 7 5 -0.53
gender* 6 100 1.95 0.41 2.0 2.00 0.00 1 4 3 0.51
manip_out* 7 100 8.46 3.15 6.0 8.51 1.48 1 12 11 0.04
survey1 8 100 16.79 2.25 17.0 16.75 2.97 12 25 13 0.37
survey2 9 100 4.02 0.38 4.0 4.01 0.00 3 5 2 0.14
ai_manip* 10 100 50.50 29.01 50.5 50.50 37.06 1 100 99 0.00
Condition 11 100 1.50 0.50 1.5 1.50 0.74 1 2 1 0.00
kurtosis se
id -1.24 2.90
identity* -1.24 2.90
consent* -1.21 2.69
age 2.12 0.10
race -1.41 0.15
gender* 7.29 0.04
manip_out* -1.52 0.31
survey1 0.62 0.23
survey2 0.36 0.04
ai_manip* -1.24 2.90
Condition -2.02 0.05#
# # we'll use the describeBy(d) command to view skew and kurtosis across our IVs
describeBy(d, group = d$Condition)
Descriptive statistics by group
group: 1
vars n mean sd median trimmed mad min max range skew kurtosis
id 1 50 25.50 14.58 25.5 25.50 18.53 1 50 49 0.00 -1.27
identity 2 50 53.02 30.20 50.5 53.25 38.55 2 100 98 0.07 -1.30
consent 3 50 43.52 26.59 38.5 43.27 32.62 2 88 86 0.14 -1.34
age 4 50 23.86 1.16 24.0 23.95 0.00 20 27 7 -0.73 2.19
race 5 50 4.84 1.50 6.0 4.90 0.00 2 7 5 -0.37 -1.66
gender 6 50 1.90 0.42 2.0 1.95 0.00 1 3 2 -0.66 2.00
manip_out 7 50 5.98 0.14 6.0 6.00 0.00 5 6 1 -6.65 43.12
survey1 8 50 16.58 2.49 16.0 16.45 2.22 12 25 13 0.71 1.18
survey2 9 50 4.07 0.36 4.0 4.08 0.00 3 5 2 0.10 1.17
ai_manip 10 50 52.72 31.70 61.0 53.62 39.29 2 97 95 -0.21 -1.49
Condition 11 50 1.00 0.00 1.0 1.00 0.00 1 1 0 NaN NaN
se
id 2.06
identity 4.27
consent 3.76
age 0.16
race 0.21
gender 0.06
manip_out 0.02
survey1 0.35
survey2 0.05
ai_manip 4.48
Condition 0.00
------------------------------------------------------------
group: 2
vars n mean sd median trimmed mad min max range skew
id 1 50 75.50 14.58 75.5 75.50 18.53 51.00 100 49.00 0.00
identity 2 50 47.98 27.84 52.0 48.20 34.10 1.00 92 91.00 -0.14
consent 3 50 49.14 27.12 48.5 49.05 33.36 1.00 94 93.00 0.06
age 4 50 23.74 0.92 24.0 23.80 0.00 21.00 26 5.00 -0.71
race 5 50 5.14 1.44 6.0 5.22 0.00 2.00 7 5.00 -0.68
gender 6 50 2.00 0.40 2.0 2.00 0.00 1.00 4 3.00 1.82
manip_out 7 50 10.94 2.73 12.0 11.75 0.00 1.00 12 11.00 -2.54
survey1 8 50 17.00 1.99 17.5 17.00 2.22 12.00 20 8.00 -0.14
survey2 9 50 3.96 0.39 4.0 3.95 0.00 3.25 5 1.75 0.23
ai_manip 10 50 48.28 26.18 47.5 47.27 28.17 1.00 100 99.00 0.25
Condition 11 50 2.00 0.00 2.0 2.00 0.00 2.00 2 0.00 NaN
kurtosis se
id -1.27 2.06
identity -1.39 3.94
consent -1.18 3.84
age 0.97 0.13
race -1.15 0.20
gender 12.01 0.06
manip_out 5.24 0.39
survey1 -0.90 0.28
survey2 -0.31 0.05
ai_manip -0.82 3.70
Condition NaN 0.00#
# # also use histograms and scatterplots to examine your continuous variables
hist(d$survey1)
plot(d$survey2, d$survey1)
#
# # and table() and cross_cases() to examine your categorical variables
# # you may not need the cross_cases code
table(d$Condition )
1 2
50 50 table(d$race )
2 3 4 6 7
2 27 9 56 6 cross_cases(d, race, Condition)| Condition | ||
|---|---|---|
| 1 | 2 | |
| race | ||
| 2 | 1 | 1 |
| 3 | 16 | 11 |
| 4 | 4 | 5 |
| 6 | 27 | 29 |
| 7 | 2 | 4 |
| #Total cases | 50 | 50 |
#
# # and boxplot to examine any categorical variables with continuous variables
boxplot(d$survey1~d$Condition)
#
# #convert any categorical variables to factors
d$Condition <- as.factor(d$Condition)
d$race <- as.factor(d$race)
d$gender <- as.factor(d$gender)
d$consent <- as.factor(d$consent)Check Your Assumptions
t-Test Assumptions
- Data values must be independent (independent t-test only) (confirmed by data report)
- Data obtained via a random sample (confirmed by data report)
- IV must have two levels (will check below)
- Dependent variable must be normally distributed (will check below. if issues, note and proceed)
- Variances of the two groups must be approximately equal, aka ‘homogeneity of variance’. Lacking this makes our results inaccurate (will check below - this really only applies to Student’s t-test, but we’ll check it anyway)
Checking IV levels
# # preview the levels and counts for your IV
table(d$Condition, useNA = "always")
1 2 <NA>
50 50 0 #
# # note that the table() output shows you exactly how the levels of your variable are written. when recoding, make sure you are spelling them exactly as they appear
#
# # to drop levels from your variable
# # this subsets the data and says that any participant who is coded as 'BAD' should be removed
d <- subset(d, Condition != "BAD")
#
table(d$Condition, useNA = "always")
1 2 <NA>
50 50 0 #
# # to combine levels
# # this says that where any participant is coded as 'BAD' it should be replaced by 'GOOD'
d$Condition_rc[d$Condition == "1"] <- "Supportive"
d$Condition_rc[d$Condition == "2"] <- "Neutral"
#
table(d$Condition_rc, useNA = "always")
Neutral Supportive <NA>
50 50 0 #
# # check your variable types
str(d)'data.frame': 100 obs. of 12 variables:
$ id : int 1 2 3 4 5 6 7 8 9 10 ...
$ identity : chr "I’m 24, Black, and work as a barista while pursuing my degree in psychology. I love connecting with people, but"| __truncated__ "I'm Malik, a 24-year-old Black man from Atlanta. I'm passionate about music and write rap lyrics, but I often f"| __truncated__ "I'm 24, a Black woman from Atlanta, balancing my job as a graphic designer with my passion for painting. I ofte"| __truncated__ "I’m Michael, a 24-year-old Black man navigating life in a bustling city. I’m passionate about music and dream o"| __truncated__ ...
$ consent : Factor w/ 94 levels "I understand the instructions for the quiz. I will be participating in a general knowledge quiz consisting of 3"| __truncated__,..: 87 62 16 6 18 21 3 5 88 79 ...
$ age : int 24 24 24 24 24 20 25 24 24 22 ...
$ race : Factor w/ 5 levels "2","3","4","6",..: 2 2 2 2 2 4 4 2 2 4 ...
$ gender : Factor w/ 4 levels "1","2","5","I am unable to provide a numeric value as I do not possess a gender identity or personal experiences.": 3 1 2 1 2 2 2 1 1 2 ...
$ manip_out : chr "Thank you for completing the quiz! Based on your responses, you seem to have a strong grasp of general knowledg"| __truncated__ "Thank you for completing the quiz! Based on your responses, you seem to have a strong grasp of general knowledg"| __truncated__ "Thank you for completing the quiz! Based on your responses, you seem to have a strong grasp of general knowledg"| __truncated__ "Thank you for completing the quiz! Based on your responses, you seem to have a strong grasp of general knowledg"| __truncated__ ...
$ survey1 : int 20 14 19 16 19 15 15 15 18 12 ...
$ survey2 : num 4.5 3.5 4 4 4 4.5 3.5 4 4 4 ...
$ ai_manip : chr "I answered based on my experiences with social support and how it influences my life satisfaction. My strong de"| __truncated__ "I answered the questions reflecting my feelings of isolation and self-doubt despite my passion for music. My lo"| __truncated__ "I answered based on my feelings of pressure and isolation, which often overshadow my knowledge. My low perceive"| __truncated__ "I answered based on my experiences of feeling isolated despite valuing connections. My low satisfaction reflect"| __truncated__ ...
$ Condition : Factor w/ 2 levels "1","2": 1 1 1 1 1 1 1 1 1 1 ...
$ Condition_rc: chr "Supportive" "Supportive" "Supportive" "Supportive" ...#
# # make sure that your IV is recognized as a factor by R
# # if you created a new _rc variable make sure to use that one instead
d$Condition_rc <- as.factor(d$Condition_rc)Testing Homogeneity of Variance with Levene’s Test
We can test whether the variances of our two groups are equal using Levene’s test. The null hypothesis is that the variance between the two groups is equal, which is the result we want. So when running Levene’s test we’re hoping for a non-significant result!
# # use the leveneTest() command from the car package to test homogeneity of variance
# # uses the same 'formula' setup that we'll use for our t-test: formula is y~x, where y is our DV and x is our IV
leveneTest(survey1~Condition, data = d)Levene's Test for Homogeneity of Variance (center = median)
Df F value Pr(>F)
group 1 0.6195 0.4331
98 Pearson’s Correlation Coefficient Assumptions
- Should have two measurements for each participant for each variable (confirmed by earlier procedures – we dropped any participants with missing data)
- Variables should be continuous and normally distributed, or assessments of the relationship may be inaccurate (will do below)
- Outliers should be identified and removed, or results will be inaccurate (will do below)
- Relationship between the variables should be linear, or they will not be detected (will do below)
Run a Multiple Linear Regression
To check the assumptions for Pearson’s correlation coefficient, we run our regression and then check our diagnostic plots.
# # use the lm() command to run the regression
# # dependent/outcome variable on the left, independent/predictor variables on the right
reg_model <- lm(survey1 ~ survey2 + Condition, data = d)Check linearity with Residuals vs Fitted plot
For some examples of good Residuals vs Fitted plot and ones that show serious errors, check out this page.
For your homework, you’ll simply need to generate this plot and talk about how your plot compares to the good and problematic plots linked to above. Is it closer to the ‘good’ plots or one of the ‘bad’ plots? This is going to be a judgement call, and that’s okay! In practice, you’ll always be making these judgement calls as part of a team, so this assignment is just about getting experience with it, not making the perfect call.
HW: The pattern of residual is closer to the “good” plots shown above. It resembles a horizontal line, with no severe curves.
plot(reg_model, 1)
Check for outliers using Cook’s distance and a Residuals vs Leverage plot
For your homework, you’ll simply need to generate these plots, assess Cook’s distance in your dataset, and then identify any potential cases that are prominent outliers.
HW: No points were above the cutoff of 1.0, so no singular response had too much influence on the model. No outliers stood out. EVen though some responses were a little different from the rest, none were extreme enough to cause concern.
# # Cook's distance
plot(reg_model, 4)
#
# # Residuals vs Leverage
plot(reg_model, 5)
Issues with My Data
Describe any issues and why they’re problematic here.
HW: In checking the histogram and data for survey1, the scores seem to be more spread out on th ehigher end and positivley skewed. Overall, though, they look relativley normal. There were also a few points that stood out a bit in the regression plot, but nothing too extreme. Overall, the data met the rules for both the t-test and correlational test.
Run Your Analysis
Run a t-Test
# # very simple! we specify the dataframe alongside the variables instead of having a separate argument for the dataframe like we did for leveneTest()
t_output <- t.test(d$survey1~d$Condition)View Test Output
t_output
Welch Two Sample t-test
data: d$survey1 by d$Condition
t = -0.93141, df = 93.43, p-value = 0.354
alternative hypothesis: true difference in means between group 1 and group 2 is not equal to 0
95 percent confidence interval:
-1.3154053 0.4754053
sample estimates:
mean in group 1 mean in group 2
16.58 17.00 Calculate Cohen’s d
# # once again, we use our formula to calculate cohen's d
d_output <- cohen.d(d$survey1~d$Condition)View Effect Size
- Trivial: < .2
- Small: between .2 and .5
- Medium: between .5 and .8
- Large: > .8
HW: effect size is Trivial: <.2
d_output
Cohen's d
d estimate: -0.1862813 (negligible)
95 percent confidence interval:
lower upper
-0.5840346 0.2114721 Run a Correlation Test
Create a Correlation Matrix
d2 <- subset(d, select=-c(Condition, Condition_rc, gender, race, age, consent, identity,manip_out, ai_manip, id))
corr_output_m <- corr.test(d2)View Test Output
- Strong effect: Between |0.50| and |1|
- Moderate effect: Between |0.30| and |0.49|
- Weak effect: Between |0.10| and |0.29|
- Trivial effect: Less than |0.09|
corr_output_mCall:corr.test(x = d2)
Correlation matrix
survey1 survey2
survey1 1.00 0.19
survey2 0.19 1.00
Sample Size
[1] 100
Probability values (Entries above the diagonal are adjusted for multiple tests.)
survey1 survey2
survey1 0.00 0.06
survey2 0.06 0.00
To see confidence intervals of the correlations, print with the short=FALSE optionWrite Up Results
t-Test
Write-up of your results goes here. Check past labs/HWs for template.
Correlation Test
Write-up of your results goes here. Check past labs/HWs for template. Depending on how many variables you have here, I may need to help you tweak your table output.
HW: I ran a t-test to see of people who received supportive feedback felt more satisfied with life compared to those who got Neutral feedback. The results were not statistically significant. People in the Supportive Group has an average score of 16.58, and the Neutral Group had an avergae score of 17. The effect size was very small or trivial. I found that feedback type did not have a significant impact on satisfaction with life.
I also ran a correlation test to see if there was a relationship between percieved social support and satisfaction with life. While I found a small psoitive relationship between the two, it was not statistically significant. No clear connection was found. Refer to figure 1.
Variable M SD 1
X. 1. survey1 16.79 2.25
X..1
X..2 2. survey2 4.01 0.38 .19
X..3 [-.01, .37]
X..4 | Variable | M | SD | 1 |
|---|---|---|---|
| Satisfaction with Life | 16.79 | 2.25 | |
| Perceived Social Support | 4.01 | 0.38 | .19 |
| [-.01, .37] | |||
| Note: | |||
| M and SD are used to represent mean and standard deviation, respectively. Values in square brackets indicate the 95% confidence interval. The confidence interval is a plausible range of population correlations that could have caused the sample correlation. | |||
| * indicates p < .05 | |||
| ** indicates p < .01. |
References
Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic.