title: “Anova” author: “Krystian Opała” date: Published on czwartek 06 styczeń 2022 output: rmdformats::readthedown: highlight: kate toc_float: true toc_depth: 4 —
2-way Anova
Two-way ANOVA test hypotheses - There is no difference in the means of factor A - There is no difference in means of factor B - There is no interaction between factors A and B The alternative hypothesis for cases 1 and 2 is: the means are not equal. The alternative hypothesis for case 3 is: there is an interaction between A and B.
Data
In this report we will the jobsatisfaction dataset [datarium package], which contains the job satisfaction score organized by gender and education levels. In this study, a research wants to evaluate if there is a significant two-way interaction between gender and education_level on explaining the job satisfaction score. An interaction effect occurs when the effect of one independent variable on an outcome variable depends on the level of the other independent variables. If an interaction effect does not exist, main effects could be reported.
Descriptive statistics
Before running a model, you should always plot the data, to check that your assumptions look okay. Here are a couple plots you might generate while analyzing these data:
ggplot(jobsatisfaction, aes(x=score)) +
geom_histogram(bins=10) +
facet_grid(gender ~ education_level) +
theme_classic()
Boxplot, to highlight the group means:
ggplot(jobsatisfaction, aes(y=score, x=education_level, fill = gender)) +
geom_boxplot() +
theme_classic()
The distributions within each cell look pretty wonky, but that’s not particularly surprising given the small sample size (n=58):
xtabs(~ gender + education_level, data = jobsatisfaction)
## education_level
## gender school college university
## male 9 9 10
## female 10 10 10
Compute the mean and the SD (standard deviation) of the score by groups:
jobsatisfaction %>% group_by(gender, education_level) %>% get_summary_stats(score, type="mean_sd")
## # A tibble: 6 x 6
## gender education_level variable n mean sd
## <fct> <fct> <chr> <dbl> <dbl> <dbl>
## 1 male school score 9 5.43 0.364
## 2 male college score 9 6.22 0.34
## 3 male university score 10 9.29 0.445
## 4 female school score 10 5.74 0.474
## 5 female college score 10 6.46 0.475
## 6 female university score 10 8.41 0.938
Assumptions
Outliers
Identify outliers in each cell design:
jobsatisfaction %>%
group_by(gender, education_level) %>%
identify_outliers(score)
## [1] gender education_level id score
## [5] is.outlier is.extreme
## <0 rows> (or 0-length row.names)
Normality
Check normality assumption by analyzing the model residuals. QQ plot and Shapiro-Wilk test of normality are used.
jobsatisfaction %>%
group_by(gender,education_level) %>%
shapiro_test(score)
## # A tibble: 6 x 5
## gender education_level variable statistic p
## <fct> <fct> <chr> <dbl> <dbl>
## 1 male school score 0.980 0.966
## 2 male college score 0.958 0.779
## 3 male university score 0.916 0.323
## 4 female school score 0.963 0.819
## 5 female college score 0.963 0.819
## 6 female university score 0.950 0.674
ggqqplot(jobsatisfaction, "score") + facet_grid(gender~education_level)
Anova
In this example, the effect of education_level is our focal variable, that is our primary concern. It is thought that the effect of education_level will depend on one other factor, gender, which are called a moderator variable
results<-aov(score~gender*education_level, data=jobsatisfaction)
anova(results)
## Analysis of Variance Table
##
## Response: score
## Df Sum Sq Mean Sq F value Pr(>F)
## gender 1 0.5 0.5 1.79 0.1871
## education_level 2 113.7 56.8 187.89 <0.0000000000000002 ***
## gender:education_level 2 4.4 2.2 7.34 0.0016 **
## Residuals 52 15.7 0.3
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
report(results)
## The ANOVA (formula: score ~ gender * education_level) suggests that:
##
## - The main effect of gender is statistically not significant and small (F(1, 52) = 1.79, p = 0.187; Eta2 (partial) = 0.03, 95% CI [0.00, 1.00])
## - The main effect of education_level is statistically significant and large (F(2, 52) = 187.89, p < .001; Eta2 (partial) = 0.88, 95% CI [0.83, 1.00])
## - The interaction between gender and education_level is statistically significant and large (F(2, 52) = 7.34, p = 0.002; Eta2 (partial) = 0.22, 95% CI [0.06, 1.00])
##
## Effect sizes were labelled following Field's (2013) recommendations.
Post-hoc tests
A significant two-way interaction indicates that the impact that one factor (e.g., education_level) has on the outcome variable (e.g., job satisfaction score) depends on the level of the other factor (e.g., gender) (and vice versa). So, you can decompose a significant two-way interaction into: - Simple main effect: run one-way model of the first variable at each level of the second variable, - Simple pairwise comparisons: if the simple main effect is significant, run multiple pairwise comparisons to determine which groups are different. For a non-significant two-way interaction, you need to determine whether you have any statistically significant main effects from the ANOVA output. A significant main effect can be followed up by pairwise comparisons between groups.
Pairwise comparisons
test <- jobsatisfaction %>% tukey_hsd(score~gender*education_level)
test
## # A tibble: 19 x 9
## term group1 group2 null.value estimate conf.low conf.high p.adj
## * <chr> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 gender male female 0 -0.193 -0.483 0.0968 1.87e- 1
## 2 education~ school college 0 0.757 0.327 1.19 2.64e- 4
## 3 education~ school univers~ 0 3.25 2.83 3.68 0
## 4 education~ college univers~ 0 2.49 2.07 2.92 0
## 5 gender:ed~ male:sch~ female:~ 0 0.314 -0.433 1.06 8.13e- 1
## 6 gender:ed~ male:sch~ male:co~ 0 0.797 0.0296 1.56 3.75e- 2
## 7 gender:ed~ male:sch~ female:~ 0 1.04 0.289 1.78 1.92e- 3
## 8 gender:ed~ male:sch~ male:un~ 0 3.87 3.12 4.61 0
## 9 gender:ed~ male:sch~ female:~ 0 2.98 2.23 3.73 0
## 10 gender:ed~ female:s~ male:co~ 0 0.482 -0.265 1.23 4.09e- 1
## 11 gender:ed~ female:s~ female:~ 0 0.722 -0.00575 1.45 5.3 e- 2
## 12 gender:ed~ female:s~ male:un~ 0 3.55 2.82 4.28 0
## 13 gender:ed~ female:s~ female:~ 0 2.67 1.94 3.39 0
## 14 gender:ed~ male:col~ female:~ 0 0.240 -0.508 0.987 9.32e- 1
## 15 gender:ed~ male:col~ male:un~ 0 3.07 2.32 3.82 0
## 16 gender:ed~ male:col~ female:~ 0 2.18 1.43 2.93 1.86e-10
## 17 gender:ed~ female:c~ male:un~ 0 2.83 2.10 3.56 0
## 18 gender:ed~ female:c~ female:~ 0 1.94 1.22 2.67 2.73e- 9
## 19 gender:ed~ male:uni~ female:~ 0 -0.886 -1.61 -0.158 8.75e- 3
## # ... with 1 more variable: p.adj.signif <chr>
jobsatisfaction %>%
pairwise_t_test(score~education_level)
## # A tibble: 3 x 9
## .y. group1 group2 n1 n2 p p.signif p.adj p.adj.signif
## * <chr> <chr> <chr> <int> <int> <dbl> <chr> <dbl> <chr>
## 1 score school college 19 19 3.27e- 4 *** 3.27e- 4 ***
## 2 score school university 19 20 3.43e-23 **** 1.03e-22 ****
## 3 score college university 19 20 3.8 e-18 **** 7.6 e-18 ****
3-Way ANOVA
The three-way ANOVA is an extension of the two-way ANOVA for assessing whether there is an interaction effect between three independent categorical variables on a continuous outcome variable. We’ll use the headache dataset [datarium package], which contains the measures of migraine headache episode pain score in 72 participants treated with three different treatments. The participants include 36 males and 36 females. Males and females were further subdivided into whether they were at low or high risk of migraine. We want to understand how each independent variable (type of treatments, risk of migraine and gender) interact to predict the pain score.
summary.aov = summary(headache[c('gender','risk','treatment','pain_score')]); summary.aov
## gender risk treatment pain_score
## male :36 high:36 X:24 Min. : 63.7
## female:36 low :36 Y:24 1st Qu.: 72.9
## Z:24 Median : 77.9
## Mean : 77.6
## 3rd Qu.: 81.5
## Max. :100.0
Descriptive statistics
ggplot(headache, aes(x=pain_score)) +
geom_histogram(bins=5) +
facet_grid(gender ~ risk ~ treatment) +
theme_gray()
Boxplot, to highlight the group means:
p <- qplot(interaction(treatment,risk), pain_score, data=headache, geom="boxplot")
p + aes(fill=gender)
Compute the mean and the SD (standard deviation) of the score by groups:
data <- headache %>% group_by(gender, treatment, risk) %>% get_summary_stats(pain_score, type="mean_sd"); data
## # A tibble: 12 x 7
## gender risk treatment variable n mean sd
## <fct> <fct> <fct> <chr> <dbl> <dbl> <dbl>
## 1 male high X pain_score 6 92.7 5.12
## 2 male low X pain_score 6 76.1 3.86
## 3 male high Y pain_score 6 82.3 5.00
## 4 male low Y pain_score 6 73.1 4.76
## 5 male high Z pain_score 6 79.7 4.05
## 6 male low Z pain_score 6 74.5 4.89
## 7 female high X pain_score 6 78.9 5.32
## 8 female low X pain_score 6 74.2 3.69
## 9 female high Y pain_score 6 81.2 4.62
## 10 female low Y pain_score 6 68.4 4.08
## 11 female high Z pain_score 6 81.0 3.98
## 12 female low Z pain_score 6 69.8 2.72
Assumptions
There are multiple assumptions for 3 way anova: 1) dependent variable must be continuous - yes 2) independent varibales should contain at least 2 categorical groups - yes 3) normality - checked below 4) no significant outliers - shown below, there is just 4 outliers in female group with high risk, of which just 1 is an extremum 5) dependent variable normally distributed for each combination of the groups of the three independent variables - shown below with use of Shapiro-Wilk test; only for group Females with high risk using treatment X p value for Shapiro-Wilk test is below 0,05 which means the normality assumption is violated for this group 6) homogeneity of variances for each combination of the groups - also checked below with use of Levene test; high p-value indicates that the null hypothesis - compared groups have equal population variances can’t be rejected.
Outliers
headache %>%
group_by(gender, risk, treatment) %>%
identify_outliers(pain_score)
## # A tibble: 4 x 7
## gender risk treatment id pain_score is.outlier is.extreme
## <fct> <fct> <fct> <int> <dbl> <lgl> <lgl>
## 1 female high X 57 68.4 TRUE TRUE
## 2 female high Y 62 73.1 TRUE FALSE
## 3 female high Z 67 75.0 TRUE FALSE
## 4 female high Z 71 87.1 TRUE FALSE
Normality
rtest <- headache %>%
group_by(gender,risk,treatment) %>%
shapiro_test(pain_score)
model <- lm(data=headache, pain_score~gender+treatment+risk)
ggqqplot(residuals(model))
headache %>% ggqqplot("pain_score") + facet_grid(risk~gender~treatment)
Homogeneity of variance
headache %>%
levene_test(pain_score~treatment*gender*risk)
## # A tibble: 1 x 4
## df1 df2 statistic p
## <int> <int> <dbl> <dbl>
## 1 11 60 0.179 0.998
Anova
In this part we compute 3 way Anova. Generated report below explains various interactions between independent variables. As for the F values for this test, the result for all three variables is high - 7.41 which indicates that the result is significant - there is influence of independent variables and the pain scores are not due to chance, and also the p-value is really small.
results<-aov(pain_score~treatment*gender*risk, data=headache)
anova(results)
## Analysis of Variance Table
##
## Response: pain_score
## Df Sum Sq Mean Sq F value Pr(>F)
## treatment 2 283 142 7.32 0.00143 **
## gender 1 313 313 16.20 0.00016 ***
## risk 1 1794 1794 92.70 0.000000000000088 ***
## treatment:gender 2 129 65 3.34 0.04220 *
## treatment:risk 2 28 14 0.71 0.49422
## gender:risk 1 3 3 0.14 0.70849
## treatment:gender:risk 2 287 143 7.41 0.00133 **
## Residuals 60 1161 19
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
report(results)
## The ANOVA (formula: pain_score ~ treatment * gender * risk) suggests that:
##
## - The main effect of treatment is statistically significant and large (F(2, 60) = 7.32, p = 0.001; Eta2 (partial) = 0.20, 95% CI [0.06, 1.00])
## - The main effect of gender is statistically significant and large (F(1, 60) = 16.20, p < .001; Eta2 (partial) = 0.21, 95% CI [0.08, 1.00])
## - The main effect of risk is statistically significant and large (F(1, 60) = 92.70, p < .001; Eta2 (partial) = 0.61, 95% CI [0.48, 1.00])
## - The interaction between treatment and gender is statistically significant and medium (F(2, 60) = 3.34, p = 0.042; Eta2 (partial) = 0.10, 95% CI [2.01e-03, 1.00])
## - The interaction between treatment and risk is statistically not significant and small (F(2, 60) = 0.71, p = 0.494; Eta2 (partial) = 0.02, 95% CI [0.00, 1.00])
## - The interaction between gender and risk is statistically not significant and very small (F(1, 60) = 0.14, p = 0.708; Eta2 (partial) = 2.35e-03, 95% CI [0.00, 1.00])
## - The interaction between treatment, gender and risk is statistically significant and large (F(2, 60) = 7.41, p = 0.001; Eta2 (partial) = 0.20, 95% CI [0.06, 1.00])
##
## Effect sizes were labelled following Field's (2013) recommendations.
Post-hoc tests
If there is a significant 3-way interaction effect, you can decompose it into: - Simple two-way interaction: run two-way interaction at each level of third variable, - Simple simple main effect: run one-way model at each level of second variable, - Simple simple pairwise comparisons: run pairwise or other post-hoc comparisons if necessary.
If you do not have a statistically significant three-way interaction, you need to determine whether you have any statistically significant two-way interaction from the ANOVA output. You can follow up a significant two-way interaction by simple main effects analyses and pairwise comparisons between groups if necessary.
Two-way interactions
Below two-way interactions are formed on risk and treatment variables and gender variable acts as the moderator. At each level of gender we are evaluating interaction risk*treatment on pain_score.
model <- lm(pain_score ~ gender*risk*treatment, data = headache)
headache %>% group_by(gender) %>% anova_test(pain_score ~ risk*treatment, error = model)
## Coefficient covariances computed by hccm()
## Coefficient covariances computed by hccm()
## # A tibble: 6 x 8
## gender Effect DFn DFd F p `p<.05` ges
## * <fct> <chr> <dbl> <dbl> <dbl> <dbl> <chr> <dbl>
## 1 male risk 1 60 50.0 0.00000000187 "*" 0.455
## 2 male treatment 2 60 10.2 0.000157 "*" 0.253
## 3 male risk:treatment 2 60 5.25 0.008 "*" 0.149
## 4 female risk 1 60 42.8 0.000000015 "*" 0.416
## 5 female treatment 2 60 0.482 0.62 "" 0.016
## 6 female risk:treatment 2 60 2.87 0.065 "" 0.087
Main effects
Here is investigation of the effect of treatment on pain_score at every level of risk or investigate the effect of risk at every level of treatment.
We can conclude that there was statistically significant simple main effect on males with high risk, which indicates that treatment has a significant effect on pain_score in males with high risk.
treatment.effect <- headache %>%
group_by(gender, risk) %>%
anova_test(pain_score ~ treatment, error = model)
## Coefficient covariances computed by hccm()
## Coefficient covariances computed by hccm()
## Coefficient covariances computed by hccm()
## Coefficient covariances computed by hccm()
treatment.effect
## # A tibble: 4 x 9
## gender risk Effect DFn DFd F p `p<.05` ges
## * <fct> <fct> <chr> <dbl> <dbl> <dbl> <dbl> <chr> <dbl>
## 1 male high treatment 2 60 14.8 0.0000061 "*" 0.33
## 2 male low treatment 2 60 0.66 0.521 "" 0.022
## 3 female high treatment 2 60 0.52 0.597 "" 0.017
## 4 female low treatment 2 60 2.83 0.067 "" 0.086
Pairwise comparisons
Parwise comparisons + with use of all 3 variables. TukeyHSD test creates a set of confidence intervals on the differences between the means of the levels of a factor. A post-hoc test which calculates p-values for significance between used variables. The most outstanding are rows with p-value equal to 0 where the significance is massive, as it is in case of male X treatment and female treatments as well as male high risk vs low risks in females and males. We can also conclude that pain_score in the same risk groups does not significantly differ.
tukey<-TukeyHSD(results); tukey
## Tukey multiple comparisons of means
## 95% family-wise confidence level
##
## Fit: aov(formula = pain_score ~ treatment * gender * risk, data = headache)
##
## $treatment
## diff lwr upr p adj
## Y-X -4.1986 -7.25 -1.15 0.004
## Z-X -4.2152 -7.27 -1.16 0.004
## Z-Y -0.0166 -3.07 3.04 1.000
##
## $gender
## diff lwr upr p adj
## female-male -4.17 -6.25 -2.1 0
##
## $risk
## diff lwr upr p adj
## low-high -9.98 -12.1 -7.91 0
##
## $`treatment:gender`
## diff lwr upr p adj
## Y:male-X:male -6.655 -11.94 -1.37 0.006
## Z:male-X:male -7.327 -12.61 -2.04 0.002
## X:female-X:male -7.885 -13.17 -2.60 0.001
## Y:female-X:male -9.627 -14.91 -4.34 0.000
## Z:female-X:male -8.988 -14.27 -3.70 0.000
## Z:male-Y:male -0.672 -5.96 4.61 0.999
## X:female-Y:male -1.230 -6.52 4.06 0.983
## Y:female-Y:male -2.972 -8.26 2.31 0.566
## Z:female-Y:male -2.333 -7.62 2.95 0.784
## X:female-Z:male -0.558 -5.84 4.73 1.000
## Y:female-Z:male -2.300 -7.59 2.99 0.794
## Z:female-Z:male -1.661 -6.95 3.63 0.939
## Y:female-X:female -1.742 -7.03 3.54 0.926
## Z:female-X:female -1.103 -6.39 4.18 0.990
## Z:female-Y:female 0.639 -4.65 5.93 0.999
##
## $`treatment:risk`
## diff lwr upr p adj
## Y:high-X:high -4.04 -9.33 1.2428 0.230
## Z:high-X:high -5.44 -10.73 -0.1577 0.040
## X:low-X:high -10.70 -15.98 -5.4116 0.000
## Y:low-X:high -15.05 -20.34 -9.7653 0.000
## Z:low-X:high -13.68 -18.97 -8.3979 0.000
## Z:high-Y:high -1.40 -6.69 3.8858 0.970
## X:low-Y:high -6.65 -11.94 -1.3681 0.006
## Y:low-Y:high -11.01 -16.29 -5.7218 0.000
## Z:low-Y:high -9.64 -14.93 -4.3544 0.000
## X:low-Z:high -5.25 -10.54 0.0324 0.052
## Y:low-Z:high -9.61 -14.89 -4.3213 0.000
## Z:low-Z:high -8.24 -13.53 -2.9539 0.000
## Y:low-X:low -4.35 -9.64 0.9327 0.164
## Z:low-X:low -2.99 -8.27 2.3000 0.561
## Z:low-Y:low 1.37 -3.92 6.6537 0.973
##
## $`gender:risk`
## diff lwr upr p adj
## female:high-male:high -4.56 -8.44 -0.6874 0.015
## male:low-male:high -10.37 -14.25 -6.4971 0.000
## female:low-male:high -14.15 -18.03 -10.2800 0.000
## male:low-female:high -5.81 -9.68 -1.9352 0.001
## female:low-female:high -9.59 -13.47 -5.7181 0.000
## female:low-male:low -3.78 -7.66 0.0916 0.058
##
## $`treatment:gender:risk`
## diff lwr upr p adj
## Y:male:high-X:male:high -10.397 -19.03 -1.763 0.007
## Z:male:high-X:male:high -13.058 -21.69 -4.423 0.000
## X:female:high-X:male:high -13.874 -22.51 -5.239 0.000
## Y:female:high-X:male:high -11.564 -20.20 -2.929 0.001
## Z:female:high-X:male:high -11.704 -20.34 -3.069 0.001
## X:male:low-X:male:high -16.687 -25.32 -8.052 0.000
## Y:male:low-X:male:high -19.600 -28.23 -10.965 0.000
## Z:male:low-X:male:high -18.283 -26.92 -9.648 0.000
## X:female:low-X:male:high -18.583 -27.22 -9.948 0.000
## Y:female:low-X:male:high -24.377 -33.01 -15.742 0.000
## Z:female:low-X:male:high -22.959 -31.59 -14.324 0.000
## Z:male:high-Y:male:high -2.661 -11.30 5.974 0.996
## X:female:high-Y:male:high -3.476 -12.11 5.158 0.965
## Y:female:high-Y:male:high -1.166 -9.80 7.469 1.000
## Z:female:high-Y:male:high -1.306 -9.94 7.328 1.000
## X:male:low-Y:male:high -6.290 -14.92 2.345 0.374
## Y:male:low-Y:male:high -9.203 -17.84 -0.568 0.027
## Z:male:low-Y:male:high -7.886 -16.52 0.749 0.105
## X:female:low-Y:male:high -8.185 -16.82 0.449 0.079
## Y:female:low-Y:male:high -13.980 -22.61 -5.345 0.000
## Z:female:low-Y:male:high -12.562 -21.20 -3.927 0.000
## X:female:high-Z:male:high -0.816 -9.45 7.819 1.000
## Y:female:high-Z:male:high 1.495 -7.14 10.129 1.000
## Z:female:high-Z:male:high 1.354 -7.28 9.989 1.000
## X:male:low-Z:male:high -3.629 -12.26 5.006 0.953
## Y:male:low-Z:male:high -6.542 -15.18 2.093 0.316
## Z:male:low-Z:male:high -5.225 -13.86 3.410 0.653
## X:female:low-Z:male:high -5.525 -14.16 3.110 0.573
## Y:female:low-Z:male:high -11.319 -19.95 -2.684 0.002
## Z:female:low-Z:male:high -9.901 -18.54 -1.266 0.012
## Y:female:high-X:female:high 2.310 -6.32 10.945 0.999
## Z:female:high-X:female:high 2.170 -6.46 10.805 0.999
## X:male:low-X:female:high -2.813 -11.45 5.822 0.993
## Y:male:low-X:female:high -5.726 -14.36 2.909 0.519
## Z:male:low-X:female:high -4.409 -13.04 4.226 0.844
## X:female:low-X:female:high -4.709 -13.34 3.926 0.781
## Y:female:low-X:female:high -10.503 -19.14 -1.868 0.006
## Z:female:low-X:female:high -9.086 -17.72 -0.451 0.031
## Z:female:high-Y:female:high -0.140 -8.77 8.495 1.000
## X:male:low-Y:female:high -5.124 -13.76 3.511 0.680
## Y:male:low-Y:female:high -8.037 -16.67 0.598 0.091
## Z:male:low-Y:female:high -6.719 -15.35 1.915 0.279
## X:female:low-Y:female:high -7.019 -15.65 1.616 0.222
## Y:female:low-Y:female:high -12.813 -21.45 -4.179 0.000
## Z:female:low-Y:female:high -11.396 -20.03 -2.761 0.002
## X:male:low-Z:female:high -4.983 -13.62 3.651 0.716
## Y:male:low-Z:female:high -7.896 -16.53 0.738 0.104
## Z:male:low-Z:female:high -6.579 -15.21 2.056 0.308
## X:female:low-Z:female:high -6.879 -15.51 1.756 0.247
## Y:female:low-Z:female:high -12.673 -21.31 -4.038 0.000
## Z:female:low-Z:female:high -11.256 -19.89 -2.621 0.002
## Y:male:low-X:male:low -2.913 -11.55 5.722 0.991
## Z:male:low-X:male:low -1.596 -10.23 7.039 1.000
## X:female:low-X:male:low -1.896 -10.53 6.739 1.000
## Y:female:low-X:male:low -7.690 -16.32 0.945 0.126
## Z:female:low-X:male:low -6.272 -14.91 2.363 0.378
## Z:male:low-Y:male:low 1.317 -7.32 9.952 1.000
## X:female:low-Y:male:low 1.017 -7.62 9.652 1.000
## Y:female:low-Y:male:low -4.777 -13.41 3.858 0.766
## Z:female:low-Y:male:low -3.359 -11.99 5.276 0.973
## X:female:low-Z:male:low -0.300 -8.93 8.335 1.000
## Y:female:low-Z:male:low -6.094 -14.73 2.541 0.422
## Z:female:low-Z:male:low -4.676 -13.31 3.958 0.789
## Y:female:low-X:female:low -5.794 -14.43 2.840 0.500
## Z:female:low-X:female:low -4.377 -13.01 4.258 0.850
## Z:female:low-Y:female:low 1.418 -7.22 10.052 1.000
