Running a One-Way & Two-Way ANOVA HW
Timothy Viles
2026-06-15
1 Loading Libraries
#install.packages("afex")
#install.packages("emmeans")
#install.packages("ggbeeswarm")
#install.packages("expss")
library(psych) # for the describe() command
library(ggplot2) # to visualize our results
library(expss) # for the cross_cases() command
library(car) # for the leveneTest() command
library(afex) # to run the ANOVA
library(ggbeeswarm) # to run plot results
library(emmeans) # for posthoc tests2 Importing Data
# For HW, import the project dataset you cleaned previously this will be the dataset you'll use throughout the rest of the semester
d <- read.csv(file="C:/Users/Timmy Viles/OneDrive - Indiana University/Lab in Social Psych PSYP421/research/Final paper/data/projectdata.csv", header=T)
# new code! this adds a column with a number for each row. It will make it easier if we need to drop outliers later
d$row_id <- 1:nrow(d)Note: For your HW, you will choose to run EITHER a one-way ANOVA (a single IV with 3 or more levels) OR a two-way/factorial ANOVA (two IVs with 2 or 3 levels each). You will need to specify your hypothesis and customize your code based on the choice you make. We will run BOTH versions of the test in the lab for illustrative purposes.
3 One-Way ANOVA
3.1 State Your Hypothesis
There will be a significant difference in openness to experience by people’s sexual orientation, between heterosexual, bisexual, and homosexual individuals. Specifically, bisexual and homosexual individuals will report higher levels of openness to experience than heterosexual individuals.
3.2 Check Your Variables
# you only need to check the variables you're using in the current analysis
str(d)## 'data.frame': 689 obs. of 8 variables:
## $ X : int 520 2814 3146 3295 717 6056 4753 5365 2044 1965 ...
## $ gender : chr "female" "male" "female" "male" ...
## $ sexual_orientation: chr "Prefer not to say" "Heterosexual/Straight" "Heterosexual/Straight" "Heterosexual/Straight" ...
## $ big5_open : num 3.67 4.33 5.67 6 5.67 ...
## $ rse : num 2.6 3.1 3.7 3 3 3 4 3.8 2.5 4 ...
## $ phq : num 1.56 1.44 1.11 1.33 1.44 ...
## $ support : num 2.83 3 4 4 3.67 ...
## $ row_id : int 1 2 3 4 5 6 7 8 9 10 ...# make our categorical variables of interest "factors"
# because we'll use our newly created row ID variable for this analysis, so make sure it's coded as a factor, too.
d$sexual_orientation <- as.factor(d$sexual_orientation)
d$X <- as.factor(d$X)
# check that our categorical variables of interest are now factors
str(d)## 'data.frame': 689 obs. of 8 variables:
## $ X : Factor w/ 689 levels "20","30","81",..: 28 165 180 193 38 368 279 312 113 108 ...
## $ gender : chr "female" "male" "female" "male" ...
## $ sexual_orientation: Factor w/ 6 levels "Asexual","Bi",..: 6 4 4 4 1 6 4 4 4 4 ...
## $ big5_open : num 3.67 4.33 5.67 6 5.67 ...
## $ rse : num 2.6 3.1 3.7 3 3 3 4 3.8 2.5 4 ...
## $ phq : num 1.56 1.44 1.11 1.33 1.44 ...
## $ support : num 2.83 3 4 4 3.67 ...
## $ row_id : int 1 2 3 4 5 6 7 8 9 10 ...# check our DV skew and kurtosis
describe(d$big5_open)## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 689 5.25 1.13 5.33 5.35 0.99 1 7 6 -0.82 0.73 0.04# we'll use the describeBy() command to view our DV's skew and kurtosis across our IVs' levels
describeBy(d$big5_open, group = d$sexual_orientation)##
## Descriptive statistics by group
## group: Asexual
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 20 5.58 0.84 5.5 5.56 0.99 4.33 7 2.67 0.17 -1.3 0.19
## ------------------------------------------------------------
## group: Bi
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 107 5.4 1.22 5.67 5.56 0.99 1 7 6 -1.14 1.39 0.12
## ------------------------------------------------------------
## group: Gay/Lesbian
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 38 5.42 1.06 5.5 5.49 1.24 2.33 7 4.67 -0.64 0.05 0.17
## ------------------------------------------------------------
## group: Heterosexual/Straight
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 443 5.18 1.13 5.33 5.26 0.99 1 7 6 -0.77 0.62 0.05
## ------------------------------------------------------------
## group: I use another term
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 19 5.21 1.13 5.33 5.24 1.48 3 7 4 -0.41 -1.02 0.26
## ------------------------------------------------------------
## group: Prefer not to say
## vars n mean sd median trimmed mad min max range skew kurtosis se
## X1 1 62 5.33 1.09 5.33 5.43 0.99 2 7 5 -0.82 0.39 0.14# also use histograms to examine your continuous variable
hist(d$big5_open)
3.3 Check Your Assumptions
3.3.1 ANOVA Assumptions
- DV should be normally distributed across levels of the IV (we checked previously using “describeBy” function)
- All levels of the IVs should have an equal number of cases and there should be no empty cells. Cells with low numbers decrease the power of the test (which increases chance of Type II error)
- Homogeneity of variance should be confirmed (using Levene’s Test)
- Outliers should be identified and removed – we will actually remove them this time!
- If you have confirmed everything above, the sampling distribution should be normal.
3.3.2 Check levels of IVs
table(d$sexual_orientation)##
## Asexual Bi Gay/Lesbian
## 20 107 38
## Heterosexual/Straight I use another term Prefer not to say
## 443 19 62# REMEMBER your test's level of POWER is determined by your SMALLEST subsample3.3.3 Check for outliers using Cook’s distance and Residuals VS Leverage plot
3.3.3.1 Run a Regression to get both outlier plots
# use this commented out section below ONLY IF if you need to remove outliers
# to drop a single outlier, use this code:
#d <- subset(d, X!=c(1108)) # NOTE: Participant 1108 is not truly problematic in the Lab, but we are going to removed them for demonstration.
# to drop multiple outliers, use this code:
# d <- subset(d, row_id!=c(1108) & row_id!=c(602))
# use the lm() command to run the regression
# formula is y~x, where y is our DV, x is our IV
reg_model <- lm(big5_open~sexual_orientation, data = d) 3.3.3.2 Check for outliers
# Cook's distance
plot(reg_model, 4)
# Residuals VS Leverage
plot(reg_model, 5)
# IF you find outliers, go back up to line 114 or 117 and remove it/them, then re-run the reg_model code3.3.4 Check homogeneity of variance
# use the leveneTest() command from the car package to test homogeneity of variance
# uses the 'formula' setup: formula is y~x, where y is our DV and x is our IV
leveneTest(big5_open~sexual_orientation, data = d)## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 5 0.3711 0.8686
## 683table(d$sexual_orientation)##
## Asexual Bi Gay/Lesbian
## 20 107 38
## Heterosexual/Straight I use another term Prefer not to say
## 443 19 62#dropping extra groups from data
d_anova <- subset(d, sexual_orientation %in% c("Heterosexual/Straight", "Bi", "Gay/Lesbian"))
d_anova$sexual_orientation <- droplevels(as.factor(d_anova$sexual_orientation))
table(d_anova$sexual_orientation)##
## Bi Gay/Lesbian Heterosexual/Straight
## 107 38 4433.3.5 Issues with My Data
Levene’s test for homogeneity of variance was not significant for our three-level sexual orientation variable with the One-Way ANOVA. Therefore, the assumption of equal variances was met. Our cell sizes were somewhat unbalanced across sexual orientation groups, which may reduce statistical power and increase the likelihood of a Type II error. We did not identify or remove any outliers for the One-Way ANOVA. Because the hypothesis focused specifically on heterosexual, bisexual, and gay/lesbian individuals, participants who selected asexual, “I use another term,” or “prefer not to say” were excluded from the ANOVA analysis. This allowed the analysis to directly test the predicted group differences.
3.4 Run a One-Way ANOVA
aov_model <- aov_ez(data = d_anova,
id = "X",
between = c("sexual_orientation"),
dv = "big5_open",
anova_table = list(es = "pes"))## Contrasts set to contr.sum for the following variables: sexual_orientation3.5 View One-Way Output
nice(aov_model)## Anova Table (Type 3 tests)
##
## Response: big5_open
## Effect df MSE F pes p.value
## 1 sexual_orientation 2, 585 1.30 2.15 .007 .118
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '+' 0.1 ' ' 1ANOVA Effect Size [partial eta-squared] cutoffs from Cohen (1988): * η^2 < 0.01 indicates a trivial effect * η^2 >= 0.01 indicates a small effect * η^2 >= 0.06 indicates a medium effect * η^2 >= 0.14 indicates a large effect
3.6 Visualize One-Way Results
afex_plot(aov_model, x = "sexual_orientation")
3.7 Run One-Way Posthoc Tests
Remember: We ONLY run posthoc IF the ANOVA test is SIGNIFICANT! E.g., only run the posthoc tests on pet type if there is a main effect for pet type
emmeans(aov_model, specs="sexual_orientation")## sexual_orientation emmean SE df lower.CL upper.CL
## Bi 5.40 0.1100 585 5.19 5.62
## Gay/Lesbian 5.42 0.1850 585 5.06 5.78
## Heterosexual/Straight 5.18 0.0541 585 5.08 5.29
##
## Confidence level used: 0.95pairs(emmeans(aov_model, specs="sexual_orientation", adjust="tukey"))## contrast estimate SE df t.ratio p.value
## Bi - (Gay/Lesbian) -0.0192 0.215 585 -0.089 0.9956
## Bi - (Heterosexual/Straight) 0.2205 0.123 585 1.797 0.1714
## (Gay/Lesbian) - (Heterosexual/Straight) 0.2397 0.193 585 1.245 0.4273
##
## P value adjustment: tukey method for comparing a family of 3 estimates3.8 Write Up One-Way ANOVA Results
To test our hypothesis that there will be a significant difference in openness to experience based on sexual orientation, we used a one-way ANOVA. Specifically, we predicted that bisexual and gay/lesbian individuals would report higher levels of openness to experience compared to heterosexual individuals. Our data was unbalanced, with more heterosexual participants than bisexual and gay/lesbian participants. This may reduce the power of the analysis and increase the chance of a Type II error.
Levene’s test for homogeneity of variance was not significant (p = .8686), indicating that the assumption of equal variances was met. We examined potential outliers using Cook’s distance and Residuals vs. Leverage plots and did not identify any extreme cases requiring removal.
We found a not significant effect of sexual orientation on openness to experience, F(2, 585) = 2.15, p = .118, ηp² = .007. This indicates that there were no significant differences in openness to experience between heterosexual, bisexual, and gay/lesbian individuals. The effect size was trivial, suggesting that sexual orientation explained very little variation in openness to experience.
References
Cohen J. (1988). Statistical Power Analysis for the Behavioral Sciences. New York, NY: Routledge Academic.