2024-03-21

K-Means Introduction

  • Clustering algorithm for unlabed Data
  • Aims to optimally partition data into K-Clusters
  • Iteratively moves clusters towards the mean position of all nearby points
  • Has quite a few Variants like K-Medians, however we will focus on the vanilla version that is built into R

K-Means Background

  • Loss Function serves to minimize the distance the “centriod” cluster has to all of its points \[ Loss = \sum_{i=1}^{N} \sum_{j=1}^{K} r_{ij} ||\mathbf{x}_i - \boldsymbol{\mu}_j||^2 \]
  • \(\boldsymbol{\mu}\) represents the centroid.
  • \(\boldsymbol{x}\) is the current data point
  • \(\boldsymbol{r}\) indicates what cluster the variable belongs to 1 or 0
  • Euclidian Distance: \[ ||\mathbf{x}_i - \boldsymbol{\mu}_j||^2 \]

Examining and cleaning Data

  • Data consists of measurements of penguins of different species and islands
  • First we will drop NA values, as well as only look at male penguins for our example for clearer clusters
penguins<- na.omit(penguins)
penguins <- penguins [(penguins$sex=="male"),]
head(penguins[,1:5],5)
## # A tibble: 5 × 5
##   species island    bill_length_mm bill_depth_mm flipper_length_mm
##   <fct>   <fct>              <dbl>         <dbl>             <int>
## 1 Adelie  Torgersen           39.1          18.7               181
## 2 Adelie  Torgersen           39.3          20.6               190
## 3 Adelie  Torgersen           39.2          19.6               195
## 4 Adelie  Torgersen           38.6          21.2               191
## 5 Adelie  Torgersen           34.6          21.1               198

Slide with Plot

  • how will K-means cluster this unlabed species data?

Optimal Num Clusters

  • For many applications, we will not know how many classes our data has
  • So we must first find the optimal number of clusters
avg_dist <- c()
for (i in 2:10)
{
  kmean <- kmeans(penguins[,3:5], i)
  avg_dist <- c(avg_dist, mean(kmean$withinss))
}
x <- 2:10
y <- avg_dist
data <- data.frame(x,y)
plot <- ggplot(data, aes(x, y)) +
  geom_line() + geom_point() +
  labs(title="K-means error by clusters",x="Number of Clusters",
       y = "Average Square Distance")

Optimal Num Clusters

- Here we can see our “Elbow” Point is 3 Clusters. This lines up with our 3 Target Species groups

Applying for Species Prediciton

  • Now we will see if K-means can Identify Adelie, Gentoo, and Chinstrap Penguins

Actual Clusters

  • As seen, K-means did an amazing job at clustering the 3 different species
  • Note: Red points are the final centroid Locations

Assessing K-Means

  • As we can see our clusters were able to successfully match the species

Actual Metrics

  • Lets find our final accuracy

\[ ACC = \frac{Correct (True)}{Total(True+False)} \] - Now plugging in the numbers \[ \frac{155}{163} \times 100\% = 95.1\% \text{ Accuracy} \]