#Loading packages needed for this assignment
library(tidyverse)
library(cluster)
library(FactoMineR)
library(factoextra)
library(patchwork)Activity 4.1 - Hierarchical clustering
SUBMISSION INSTRUCTIONS
- Render to html
- Publish your html
- Submit a link to your published solutions
Question 1
Consider three data sets below. Each data set contains three “clusters” in two dimensions. In the first, the clusters are three convex spheres. (A convex cluster is one where all points in the cluster can be connected with a straight line that does not leave the cluster.) In the second, one cluster is a sphere; one is a ring; and one is a half-moon. In the third, two clusters are spirals and one is a sphere. Our goal is to compare the performance of various hierarchical methods in clustering different cluster shapes.
three_spheres <- read.csv('Data/cluster_data1.csv')
ring_moon_sphere <- read.csv('Data/cluster_data2.csv')
two_spirals_sphere <- read.csv('Data/cluster_data3.csv')plot_cluster_sol <- function(df, method_name, dataset_label) {
d_mat <- dist(df, method = 'euclidean')
ag_res <- agnes(d_mat, method = method_name)
clust_labels <- cutree(ag_res, k = 3)
sil_obj <- silhouette(clust_labels, d_mat)
avg_sil <- mean(sil_obj[, 3]) %>% round(2)
df_plot <- df %>%
mutate(cluster = factor(clust_labels))
ggplot(data = df_plot, aes(x = x, y = y, color = cluster, shape = cluster)) +
geom_point(alpha = 0.8) +
guides(color = 'none', shape = 'none') +
labs(title = paste0(dataset_label, "\n", str_to_title(method_name), " linkage (s̄ = ", avg_sil, ")")) +
theme_classic(base_size = 13)
}
methods <- c('single', 'complete', 'average', 'ward')
plots_d1 <- map(methods, ~plot_cluster_sol(three_spheres, .x, "Three Spheres"))
plots_d2 <- map(methods, ~plot_cluster_sol(ring_moon_sphere, .x, "Ring, Moon, Sphere"))
plots_d3 <- map(methods, ~plot_cluster_sol(two_spirals_sphere, .x, "Two Spirals, Sphere"))
(plots_d1[[1]] + plots_d2[[1]] + plots_d3[[1]]) /
(plots_d1[[2]] + plots_d2[[2]] + plots_d3[[2]]) /
(plots_d1[[3]] + plots_d2[[3]] + plots_d3[[3]]) /
(plots_d1[[4]] + plots_d2[[4]] + plots_d3[[4]])Perform agglomerative clustering with single, complete, average, and Ward linkages. Cut each tree to produce three clusters. Produce 12 scatterplots, one per data set/linkage combination, showing the 3-cluster solution. Title each graph with the linkage used, as well as the average silhouette width (\(\bar s\)) for that clustering solution. Use patchwork to create a nice 4x3 grid of your plots. Play around with larger #| fig-width and #| fig-height values in your R chunk options to make sure your grid is large enough.
Discuss the following:
- Which linkage works best for which scenario?
Three convex spheres (cluster_data1.csv): Complete, average, and Ward linkages all work well, cleanly separating the three spherical clusters into groups of roughly 100 observations each (\(\bar s \approx 0.73\)).
Ring, half-moon, and sphere (cluster_data2.csv): Single linkage method successfully identifies the true non-convex structures (\(\bar s = 0.41\)), separating the outer ring, the inner half-moon, and the sphere (250 points each).
- Does the average silhouette width always do a good job of measuring the quality of the clustering solution?
No. Because silhouette width relies on average Euclidean distance to points within the same cluster (\(a(i)\)) versus the nearest neighboring cluster (\(b(i)\)), it favors compact, convex, spherical clusters.
(Hint: you have a lot of repetitive code to write. You may find it helpful to write a function that takes a data set and a linkage method as arguments, does the clustering and computes average silhouette width, and produces the desired plot.)
Question 2
Consider the data set below on milk content of 25 mammals. The variables have been pre-scaled to z-scores, hence no additional standardizing is necessary. (Data source: Everitt et al. Cluster analysis 4ed)
mammals <- read.csv('Data/mammal_milk.csv') %>%
column_to_rownames('Mammal')A)
Perform agglomerative clustering with single, complete, average, and Ward linkages. Which has the best agglomerative coefficient?
mammal_dist <- dist(mammals, method = 'euclidean')
(mammal_single <- agnes(mammal_dist, method = 'single'))$ac[1] 0.7875718
(mammal_complete <- agnes(mammal_dist, method = 'complete'))$ac[1] 0.8985539
(mammal_average <- agnes(mammal_dist, method = 'average'))$ac[1] 0.8706571
(mammal_ward <- agnes(mammal_dist, method = 'ward'))$ac[1] 0.9413994
Ward’s linkage has the best agglomerative coefficient (\(\text{AC} \approx 0.941\))
B)
Plot a dendrogram of the method with the highest AC. Which mammals cluster together first?
fviz_dend(mammal_ward) +
labs(title = 'Ward linkage dendrogram - Mammal milk data')Deer and Reindeer cluster together first.
C)
If the tree is cut at a height of 4, how many clusters will form? Which cluster will have the fewest mammals, and which mammals will they be?
fviz_dend(mammal_ward, h = 4) +
geom_hline(yintercept = 4, linetype = 2) +
labs(title = 'Ward linkage cut at h = 4')ward_h4 <- cutree(as.hclust(mammal_ward), h = 4)
ward_h4 Bison Buffalo Camel Cat Deer Dog Dolphin
1 2 1 2 3 3 4
Donkey Elephant Fox Guinea pig Hippo Horse Llama
1 1 2 2 1 1 1
Monkey Mule Orangutan Pig Rabbit Rat Reindeer
1 1 1 2 3 3 3
Seal Sheep Whale Zebra
4 2 3 1
table(ward_h4)ward_h4
1 2 3 4
11 6 6 2
Cutting the tree at a height of \(h = 4\) results in 4 clusters.The cluster with the fewest mammals contains only 2 mammals, Dolphin and Seal.
D)
Use WSS and average silhouette method to suggest the optimal number of clusters. Re-create the dendrogram with the cluster memberships indicated.
p_wss <- fviz_nbclust(mammals,
FUNcluster = hcut,
method = 'wss',
diss = mammal_dist,
hc_method = 'ward',
hc_func = 'agnes') +
labs(title = 'Plot of WSS vs k')
p_sil <- fviz_nbclust(mammals,
FUNcluster = hcut,
method = 'silhouette',
diss = mammal_dist,
hc_method = 'ward',
hc_func = 'agnes') +
labs(title = 'Plot of mean silhouette width vs k')
p_wss + p_silfviz_dend(mammal_ward, k = 3) +
labs(title = 'Ward linkage, 3 clusters')E)
Use suitable visualizations, including dimension reduction techniques, to explore the different milk characteristics of the assigned clusters. Discuss.
ward_k3 <- cutree(mammal_ward, k = 3)
mammal_pca <- PCA(mammals, scale.unit = FALSE, graph = FALSE)fviz_pca(mammal_pca,
habillage = factor(ward_k3),
repel = TRUE) +
ggtitle('PCA biplot with 3 Ward clusters') +
guides(color = 'none', shape = 'none')mammal_long <- mammals %>%
rownames_to_column('Mammal') %>%
mutate(ward_k3 = factor(ward_k3)) %>%
pivot_longer(cols = Water:Ash,
names_to = 'variable',
values_to = 'value')
ggplot(data = mammal_long) +
geom_boxplot(aes(x = ward_k3, y = value, fill = ward_k3)) +
labs(fill = 'Cluster', x = 'Cluster', y = 'Z-score',
title = 'Milk composition by cluster') +
facet_wrap(~variable, scales = 'free_y') +
theme_classic()The first two principal components explain 95.3% of the total variance (\(\text{Dim1} = 78.1\%\), \(\text{Dim2} = 17.2\%\)).
Cluster 1 (17 mammals): Positioned on the left side of the biplot, this largest group is characterized by high Water and Lactose content and low Fat, Protein, and Ash levels.
Cluster 2 (2 mammals: Dolphin, Seal): Located in the upper right quadrant of the biplot, these mammals produce highly concentrated, energy dense milk with very high Fat and high Protein, paired with extremely low Water and Lactose and below average Ash.
Cluster 3 (6 mammals: Deer, Dog, Rabbit, Rat, Reindeer, Whale): Located in the lower right quadrant, these mammals have moderately low Water and Lactose, high Protein, moderately high Fat, and high Ash content.