Summary & Results

The following is an analysis of the effects of three educational software treatments on a student’s healthy lifestyle score. For this analysis, fifteen 10 years old and fifteen 12 years old were randomly assigned to one of the three treatments. After finishing the program, all students completed a common exam on healthy lifestyle. A two-way analysis of variance (ANOVA) was performed to analyze the effect of educational software and age on healthy lifestyle score. Results of the two-way ANOVA shows that there was a significant interaction effect between age and treatment (F(2, 24) = 10.31, p < .001; ηp2 = 0.46) indicating that treatment depends on student age. Main effects analysis showed that age & treatment did not have a significant effect on healthy lifestyle score (F(1, 24) = 2.82, p = 0.106), (F(1, 24) = 2.82, p = 0.106).

Results from a Tukey’s post-hoc analysis indicates that there are statistically significant differences between healthy lifestyle scores between 10 years old:Arthur's Adventure-12 years old:A Day in School (p < 0.05), 12 years old:Arthur's Adventure-10 years old:Arthur's Adventure (p < 0.05), and 12 years old:Healthy Lifestyle-12 years old:Arthur's Adventure (p <0.05).

Descriptive Statistics

df %>% 
  ggplot(aes(x = trt, y = score, fill = age))+
  geom_boxplot()+
  theme_minimal()+
  labs(x = "Treatment", y = "Score", fill = "Age", title = "Score Distributions by Treatment & Age")+
  scale_fill_manual(values = severance_palette("Dinner"))+
  theme(legend.position="bottom")

df %>% 
  group_by(age, trt) %>% 
  summarize(count = n(), mean = mean(score), sd = sd(score)) %>% 
  ggplot(aes(x = trt, y = mean))+
  geom_point(aes(color = age), size = 7)+
  geom_text(aes(label = mean), nudge_x = .2)+
  theme_minimal()+
  labs(x = "Treatment", y = "Score", color = "Age", title = "Average Scores by Treatment & Age")+
  scale_color_manual(values = severance_palette("Dinner"))+
  theme(legend.position="bottom")

df %>% 
  group_by(age, trt) %>% 
  summarize(count = n(), mean = mean(score), sd = sd(score)) %>% 
  kable(align = 'llccc', col.names = c('Age','Treatment','Count','Mean','Standard Deviation'), caption = "Descriptive Statistics", digits = 2)
Descriptive Statistics
Age Treatment Count Mean Standard Deviation
10 years old A Day in School 5 23.8 3.56
10 years old Arthur’s Adventure 5 30.6 4.10
10 years old Healthy Lifestyle 5 23.4 4.83
12 years old A Day in School 5 22.6 3.65
12 years old Arthur’s Adventure 5 19.4 2.97
12 years old Healthy Lifestyle 5 28.4 4.72

Two-Way Analysis of Variance

fit1 <- aov(data = df, formula = score ~ age + trt + age*trt)

tidy(fit1) %>% 
  kable(col.names = c("", "DF","Sum of Squares","Mean Square","F","P Value"), align = "lccccc", digits = 3)
DF Sum of Squares Mean Square F P Value
age 1 45.633 45.633 2.817 0.106
trt 2 37.800 18.900 1.167 0.328
age:trt 2 334.067 167.033 10.311 0.001
Residuals 24 388.800 16.200

Tukey post-hoc multiple comparisons of means: 95% family-wise confidence level

Tukey Comparison Dotplot

TukeyHSD(fit1,conf.level = .95) %>% 
  tidy() %>%
  mutate(sig = as_factor(case_when(adj.p.value <= 0.05 ~ 'Significant',
                         adj.p.value > 0.05 ~ 'Not Significant'))) %>% 
  ggplot(aes(x = estimate, y = contrast))+
  geom_errorbar(aes(xmin = conf.low, xmax = conf.high), width = .2)+
  geom_point(aes(color = sig),size = 5)+
  geom_vline(xintercept = 0, color = 'red', alpha = .5, linetype = 'dashed')+
  theme_minimal()+
  labs(y = 'Group Contrast', x = 'Difference in mean levels', 
    caption = '*Difference in mean levels not crossing zero indicates a significant relationship.', 
    title = "95% family-wise confidence level", color = "Significance")+
  scale_color_manual(values = severance_palette("Dinner"))+
   theme(legend.position="bottom")

Tukey Comparison Output

TukeyHSD(fit1,conf.level = .95) %>% 
  tidy() %>% 
  select(-term,-null.value) %>% 
  kable(align = 'lcccc', digits = 3, col.names = c('Group Contrast','Mean Difference','Low CI','High CI','Adjusted P Value'), 
        caption = 'Tukey multiple comparisons of means: 95% family-wise confidence level')
Tukey multiple comparisons of means: 95% family-wise confidence level
Group Contrast Mean Difference Low CI High CI Adjusted P Value
12 years old-10 years old -2.467 -5.500 0.567 0.106
Arthur’s Adventure-A Day in School 1.800 -2.695 6.295 0.584
Healthy Lifestyle-A Day in School 2.700 -1.795 7.195 0.309
Healthy Lifestyle-Arthur’s Adventure 0.900 -3.595 5.395 0.872
12 years old:A Day in School-10 years old:A Day in School -1.200 -9.071 6.671 0.997
10 years old:Arthur’s Adventure-10 years old:A Day in School 6.800 -1.071 14.671 0.119
12 years old:Arthur’s Adventure-10 years old:A Day in School -4.400 -12.271 3.471 0.527
10 years old:Healthy Lifestyle-10 years old:A Day in School -0.400 -8.271 7.471 1.000
12 years old:Healthy Lifestyle-10 years old:A Day in School 4.600 -3.271 12.471 0.480
10 years old:Arthur’s Adventure-12 years old:A Day in School 8.000 0.129 15.871 0.045
12 years old:Arthur’s Adventure-12 years old:A Day in School -3.200 -11.071 4.671 0.804
10 years old:Healthy Lifestyle-12 years old:A Day in School 0.800 -7.071 8.671 1.000
12 years old:Healthy Lifestyle-12 years old:A Day in School 5.800 -2.071 13.671 0.241
12 years old:Arthur’s Adventure-10 years old:Arthur’s Adventure -11.200 -19.071 -3.329 0.002
10 years old:Healthy Lifestyle-10 years old:Arthur’s Adventure -7.200 -15.071 0.671 0.087
12 years old:Healthy Lifestyle-10 years old:Arthur’s Adventure -2.200 -10.071 5.671 0.951
10 years old:Healthy Lifestyle-12 years old:Arthur’s Adventure 4.000 -3.871 11.871 0.624
12 years old:Healthy Lifestyle-12 years old:Arthur’s Adventure 9.000 1.129 16.871 0.019
12 years old:Healthy Lifestyle-10 years old:Healthy Lifestyle 5.000 -2.871 12.871 0.390