#Loading packages needed for this assignment
library(tidyverse)
library(cluster)
library(factoextra)
library(patchwork)Activity 4.1 - Hierarchical clustering
SUBMISSION INSTRUCTIONS
- Render to html
- Publish your html
- Submit a link to your published solutions
Question 1
Consider three data sets below. Each data set contains three “clusters” in two dimensions. In the first, the clusters are three convex spheres. (A convex cluster is one where all points in the cluster can be connected with a straight line that does not leave the cluster.) In the second, one cluster is a sphere; one is a ring; and one is a half-moon. In the third, two clusters are spirals and one is a sphere. Our goal is to compare the performance of various hierarchical methods in clustering different cluster shapes.
three_spheres <- read.csv('Data/cluster_data1.csv')
ring_moon_sphere <- read.csv('Data/cluster_data2.csv')
two_spirals_sphere <- read.csv('Data/cluster_data3.csv')Perform agglomerative clustering with single, complete, average, and Ward linkages. Cut each tree to produce three clusters. Produce 12 scatterplots, one per data set/linkage combination, showing the 3-cluster solution. Title each graph with the linkage used, as well as the average silhouette width (\(\bar s\)) for that clustering solution. Use patchwork to create a nice 4x3 grid of your plots. Play around with larger #| fig-width and #| fig-height values in your R chunk options to make sure your grid is large enough.
Discuss the following:
- Which linkage works best for which scenario?
- Does the average silhouette width always do a good job of measuring the quality of the clustering solution?
(Hint: you have a lot of repetitive code to write. You may find it helpful to write a function that takes a data set and a linkage method as arguments, does the clustering and computes average silhouette width, and produces the desired plot.)
cluster_plot <- function(data, linkage, data_name, k = 3){
data_numeric <- data %>% dplyr::select(x, y)
data_dist <- dist(data_numeric, method = 'euclidean')
data_agnes <- agnes(data_dist, method = linkage)
clusters <- cutree(data_agnes, k = k)
avg_sil <- (silhouette(x = clusters, dist = data_dist)
%>% summary()
)$avg.width
(data_numeric
%>% mutate(cluster = factor(clusters))
%>% ggplot(aes(x = x, y = y, color = cluster)) +
geom_point() +
guides(color = 'none') +
labs(title = bquote(.(str_to_title(linkage))~'linkage,'~
bar(s) == .(round(avg_sil, 2))),
subtitle = data_name) +
theme_classic(base_size = 14)
)
}
data_list <- list('Three spheres' = three_spheres,
'Ring, moon, sphere' = ring_moon_sphere,
'Two spirals, sphere' = two_spirals_sphere)
linkages <- c('single', 'complete', 'average', 'ward')
plot_list <- (expand_grid(linkage = linkages, data_name = names(data_list))
%>% pmap(function(linkage, data_name)
cluster_plot(data = data_list[[data_name]],
linkage = linkage,
data_name = data_name))
)
wrap_plots(plot_list, ncol = 3)For compact, round clusters like the three spheres, Ward, complete, and average linkage work best. They all recover the correct clusters with a silhouette score of 0.73. Single linkage struggles because it connects points together and combines separate clusters. However, for non-convex shapes single linkage works best because it follows curved clusters. Ward, complete, average is best for compact clusters and single linkage for more unusual shapes, as long as the clusters are well separated.
No, average silhouette width doesn’t always give an good picture of cluster quality. It works as intented for compact clusters like the spheres, but for rings and spirals, it gives the highest scores to incorrect solutions.
Question 2
Consider the data set below on milk content of 25 mammals. The variables have been pre-scaled to z-scores, hence no additional standardizing is necessary. (Data source: Everitt et al. Cluster analysis 4ed)
mammals <- read.csv('Data/mammal_milk.csv') %>%
column_to_rownames('Mammal')A)
Perform agglomerative clustering with single, complete, average, and Ward linkages. Which has the best agglomerative coefficient?
mammal_dist <- dist(mammals, method = 'euclidean')
(mammal_single <- agnes(mammal_dist, method = 'single'))$ac[1] 0.7875718
(mammal_complete <- agnes(mammal_dist, method = 'complete'))$ac[1] 0.8985539
(mammal_average <- agnes(mammal_dist, method = 'average'))$ac[1] 0.8706571
(mammal_ward <- agnes(mammal_dist, method = 'ward'))$ac[1] 0.9413994
Ward linkage (0.9413994) has the best agglomerative coefficient.
B)
Plot a dendrogram of the method with the highest AC. Which mammals cluster together first?
fviz_dend(mammal_ward) +
labs(title = 'Ward')Several mammals cluster very low, but it appears to me as if Deer and Reindeer cluster first, which checks out, especially given the names.
C)
If the tree is cut at a height of 4, how many clusters will form? Which cluster will have the fewest mammals, and which mammals will they be?
4 clusters will form, the smallest cluster will have just Dolphin and Seal grouped.
D)
Use WSS and average silhouette method to suggest the optimal number of clusters. Re-create the dendrogram with the cluster memberships indicated.
p_wss <- fviz_nbclust(mammals,
FUNcluster = hcut,
method = 'wss',
diss = mammal_dist,
hc_method = 'ward',
hc_func = 'agnes') +
labs(title = 'Plot of WSS vs k')
p_sil <- fviz_nbclust(mammals,
FUNcluster = hcut,
method = 'silhouette',
diss = mammal_dist,
hc_method = 'ward',
hc_func = 'agnes') +
labs(title = 'Plot of mean silhouette width vs k')
p_wss + p_sil fviz_dend(mammal_ward, k = 3) +
labs(title = 'Ward w/ 3 clusters')E)
Use suitable visualizations, including dimension reduction techniques, to explore the different milk characteristics of the assigned clusters. Discuss.
library(FactoMineR)
mammal_pca <- PCA(mammals, graph = FALSE)
fviz_pca(mammal_pca,
habillage = factor(cutree(mammal_ward, k = 3)),
repel = TRUE) +
ggtitle('PCA biplot')(mammals
%>% mutate(cluster = factor(cutree(mammal_ward, k = 3)))
%>% pivot_longer(cols = -cluster, names_to = 'variable', values_to = 'value')
%>% ggplot(aes(x = cluster, y = value, fill = cluster)) +
geom_boxplot() +
facet_wrap(~variable, scales = 'free_y') +
labs(x = 'Cluster', y = 'z-score', fill = 'Cluster') +
theme_classic()
)PC1 separates watery milk from nutrient-rich milk and PC2 mainly separating Ash from Fat. Cluster 1 contains mostly mammals with watery, sugary milk, having more Water and Lactose but lower Fat and Protein. Cluster 2 appears to have higher Protein and Ash, with less Water and Lactose. Cluster 3, which is the cluster with just Dolphin and Seal stands out with very high Fat and very low Water and Lactose. The boxplots also show these differences, showing that Water, Lactose, and Fat are the biggest factors separating the clusters. Overall, PC1 separates the watery milk from the richer milk, while PC2 helps distinguish the two nutrient-rich clusters.