2025-11-09

Introduction to ANOVA

What is ANOVA (Analysis of Variance)

ANOVA is a method to help determine if there is a significant difference between a set of population means using the F-test.

  • One-Way ANOVA: Analyzes the effect of 1 independent variable on 1 dependent variable.
    • Ex: How car weight (independent) affects miles per gallon (dependent).
  • Two-Way ANOVA: Analyzes the effect of 2 independent variables on 1 dependent variable.
    • Ex: How car weight and year of release (independent) affects miles per gallon (dependent).
    • Expands to X-Way ANOVA

Math Behind the F-Test

  • Df: Degrees of Freedom. Number of logically independent values in a dataset. k is the number of groups, and N is the total number of observations.
    • df1 (between groups): k - 1
    • df2 (within groups): N - k
  • Means:
    • Group Means: \(\bar{X}\) for each independent group, X.
    • Grand Mean: \(\bar{X}_{grand}\), the mean for the entire dataset.
  • Sum of Squares:
    • Sum of Squares Between Groups: \(SSB = \Sigma n_i(\bar{X}_i - \bar{X}_{grand})^2\) for each group i
    • Sum of Squares Error or Within Groups: \(SSE = \Sigma (x_i - \bar{X})^2\) for each observation i

Math Behind the F-Test Cont.

  • Mean Square
    • Mean Square Between Groups: \(MSB = {SSB \over df1}\)
    • Mean Square Error: \(MSWE {SSE \over df2}\)
  • F-Coefficient:
    • \(F = {MSB \over MSE}\)
  • Finding \(F_{critical}\) using F-Distribution table:
  • If \(F > F_{critical}\)
  • \(p < \alpha\), and the null hypothesis is rejected.

One-Way ANOVA

Hawks Dataset

##      Month             Day             Year       CaptureTime   ReleaseTime 
##  Min.   : 8.000   Min.   : 1.00   Min.   :1992   11:35  : 14          :842  
##  1st Qu.: 9.000   1st Qu.: 9.00   1st Qu.:1995   13:30  : 14   11:00  :  2  
##  Median :10.000   Median :16.00   Median :1999   11:45  : 13   11:35  :  2  
##  Mean   : 9.843   Mean   :15.74   Mean   :1998   12:10  : 13   12:05  :  2  
##  3rd Qu.:10.000   3rd Qu.:23.00   3rd Qu.:2001   14:00  : 13   12:50  :  2  
##  Max.   :11.000   Max.   :31.00   Max.   :2003   13:05  : 12   13:32  :  2  
##                                                  (Other):829   (Other): 56  
##       BandNumber  Species  Age     Sex          Wing           Weight      
##            :  2   CH: 70   A:224    :576   Min.   : 37.2   Min.   :  56.0  
##  1142-09240:  1   RT:577   I:684   F:174   1st Qu.:202.0   1st Qu.: 185.0  
##  1142-09241:  1   SS:261           M:158   Median :370.0   Median : 970.0  
##  1142-09242:  1                            Mean   :315.6   Mean   : 772.1  
##  1142-18229:  1                            3rd Qu.:390.0   3rd Qu.:1120.0  
##  1142-19209:  1                            Max.   :480.0   Max.   :2030.0  
##  (Other)   :901                            NA's   :1       NA's   :10      
##      Culmen         Hallux            Tail        StandardTail  
##  Min.   : 8.6   Min.   :  9.50   Min.   :119.0   Min.   :115.0  
##  1st Qu.:12.8   1st Qu.: 15.10   1st Qu.:160.0   1st Qu.:162.0  
##  Median :25.5   Median : 29.40   Median :214.0   Median :215.0  
##  Mean   :21.8   Mean   : 26.41   Mean   :198.8   Mean   :199.2  
##  3rd Qu.:27.3   3rd Qu.: 31.40   3rd Qu.:225.0   3rd Qu.:226.0  
##  Max.   :39.2   Max.   :341.40   Max.   :288.0   Max.   :335.0  
##  NA's   :7      NA's   :6                        NA's   :337    
##      Tarsus        WingPitFat        KeelFat           Crop       
##  Min.   :24.70   Min.   :0.0000   Min.   :0.000   Min.   :0.0000  
##  1st Qu.:55.60   1st Qu.:0.0000   1st Qu.:2.000   1st Qu.:0.0000  
##  Median :79.30   Median :1.0000   Median :2.000   Median :0.0000  
##  Mean   :71.95   Mean   :0.7922   Mean   :2.184   Mean   :0.2345  
##  3rd Qu.:87.00   3rd Qu.:1.0000   3rd Qu.:3.000   3rd Qu.:0.2500  
##  Max.   :94.00   Max.   :3.0000   Max.   :4.000   Max.   :5.0000  
##  NA's   :833     NA's   :831      NA's   :341     NA's   :343

One-Way ANOVA: Finding our Variables

Species gives us 3 groups (“CH”, “RT”, and “SS”), making it a good fit for ANOVA. Let’s see how it affects tail length.

  • Null and Alternative Hypothesis:
    • \(H_0: \mu_{ch} = \mu_{rt} = \mu_{ss}\) (There is no significant difference between tail length between hawk species)
    • \(H_1: \mu\) for at least species differs

Plotting Differences:

Finding ANOVA:

We will use the default aov function to perform our one way ANOVA to examine if mean tail length differs across species.

hawks_aov <- aov(Hawks$StandardTail ~ factor(Hawks$Species))
summary(hawks_aov)
##                        Df Sum Sq Mean Sq F value Pr(>F)    
## factor(Hawks$Species)   2 684862  342431    1325 <2e-16 ***
## Residuals             568 146807     258                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 337 observations deleted due to missingness

The p value is very low, which implies that species does impact tail length

Two-Way ANOVA

Two-Way ANOVA: Finding Our Variables

Species is still a good independent variable, but we’ll add sex, and see how both affect weight.

  • Null Hypotheses:
    • \(H_0^1: \mu_{ch} = \mu_{rt} = \mu_{ss}\) (There is no significant difference in weight between hawk species
    • \(H_0^2: \mu_m = \mu_f\) (There is no significant difference in weight between months of the year).
    • \(H_0^3\): There is no interaction effect between Species and Month

Plotting Differences:

Finding ANOVA

We will use the default aov function to perform our one way ANOVA to examine if mean tail length differs across species.

hawks_aov2 <- aov(Hawks$StandardTail ~ factor(Hawks$Species) * factor(Hawks$Sex))
summary(hawks_aov2)
##                                          Df Sum Sq Mean Sq F value Pr(>F)    
## factor(Hawks$Species)                     2 684862  342431 1633.85 <2e-16 ***
## factor(Hawks$Sex)                         2  28697   14349   68.46 <2e-16 ***
## factor(Hawks$Species):factor(Hawks$Sex)   3    113      38    0.18   0.91    
## Residuals                               563 117997     210                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 337 observations deleted due to missingness

The p value is very low for \(H_0^1\) and \(H_0^2\), which implies that species and sex both impact tail length.

However, there the p-value for interaction between species and sex is high, which suggests they do not affect one another.

Compare One-Way and 2-Way for Hawks Tail Size

References: