• What is hypothesis testing? Using statistics to decide whether or not data supports a given hypothesis
• Example: If we have a dataset containing the heights of different people, we can form the hypothesis “The average person is less than 6 feet tall”, and test it using our dataset
• Hypothesis I’ll test: In the iris dataset, petal length differs significantly between species

summary(iris)

##   Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
##  Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
##  1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
##  Median :5.800   Median :3.000   Median :4.350   Median :1.300  
##  Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
##  3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
##  Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
##        Species  
##  setosa    :50  
##  versicolor:50  
##  virginica :50  
##                 
##                 
## 

To get a better feel for the data, visualizing the aspects of it we’re interested in seems like a good place to begin analysis. So, let’s visualize the petal length by species in a few different ways.

ggplot(iris, aes(x = Species, y = Petal.Length, fill = Species)) + geom_boxplot()

ggplot(iris, aes(x = Petal.Length, fill = Species)) + geom_density()

It seems pretty obvious from the graph that species and petal lengths are correlated somehow. Setosa irises seem to have the shortest petal length. Versicolor and virginica irises both have longer petals, but virginicas are the longest. But is this true of any other physical characteristics as well?

• Ok, so there is some obvious clustering of the physical characteristics of irises depending on what species they belong to.

Since we have three different species in the dataset, we’ll use an ANOVA test to determine whether or not petal length varies significantly between species. There’s two values to pay close attention to here - the F statistic and the p value.

The F statistic is computed generally as \(F=\frac{\text{Variability between groups}}{\text{Variability inside group}}\). In this case, the f statistic computation will be \(F=\frac{\text{Variance of petal length between species}}{\text{Variance of petal length within species}}\)

• The numerator - the variance between species - is calculated using \(\frac{SSB}{\text{Number of groups}-1}\)
• The denominator - the variance within species - is calculated using \(\frac{SSW}{\text{Degrees of freedom within a species}}\)
• Where SSB is the sum of squares between groups, and SSW is the sum of squares within groups.

aov_results <- aov(Petal.Length ~ Species, data = iris)
summary(aov_results)

##              Df Sum Sq Mean Sq F value Pr(>F)    
## Species       2  437.1  218.55    1180 <2e-16 ***
## Residuals   147   27.2    0.19                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

With a p-value of <\(2*10^{-16}\), which is significantly below the threshold p value of 0.05, we can conclude that petal length does differ significantly between species.