Determining the number of clusters
To determine the appropriate number of clusters in k-means
clustering, several methods can be used, including the elbow method, the
average silhouette method, and the gap statistic. In R, the first two
methods can be implemented easily using the fviz_nbclust()
function from the factoextra package, while the gap
statistic is available through the cluster package.
The elbow method evaluates the total within-cluster variation for different values of k. The “elbow” point, where the rate of decrease starts to slow down, suggests a suitable number of clusters.
library(factoextra)
#> Warning: package 'factoextra' was built under R version 4.5.3
#> Loading required package: ggplot2
#> Warning: package 'ggplot2' was built under R version 4.5.3
#> Welcome to factoextra!
#> Want to learn more? See two factoextra-related books at https://www.datanovia.com/en/product/practical-guide-to-principal-component-methods-in-r/
fviz_nbclust(x = wine_stand[,2:14],FUNcluster = kmeans, method = 'wss' )
The silhouette method assesses how well observations are clustered by comparing within-cluster similarity to between-cluster dissimilarity. Higher average silhouette values indicate a better clustering structure.
fviz_nbclust(x = wine_stand[,2:14],FUNcluster = kmeans, method = 'silhouette' ) #Average silhouette method
The gap statistic compares the observed within-cluster dispersion to that expected under a reference (random) distribution of the data. The optimal number of clusters is typically chosen as the smallest k such that the gap statistic is within one standard deviation of its maximum.
library(cluster)
#> Warning: package 'cluster' was built under R version 4.5.3
set.seed(123)
gap_stat <- clusGap(x = wine_stand[,2:14], FUN = kmeans, K.max = 15, nstart = 25, B = 50 )
fviz_gap_stat(gap_stat)
Together, these approaches provide complementary information to help determine an appropriate number of clusters before fitting the final k-means model.
In addition to visual methods such as the elbow plot, silhouette
plot, and gap statistic, the NbClust package provides a
more comprehensive approach for determining the optimal number of
clusters. It evaluates a large number of clustering validity indices
(over 20 different criteria) and uses them to propose a suitable number
of clusters for the data.
Each index assesses clustering quality in a different way, based on measures such as within-cluster compactness and between-cluster separation. The final recommendation is typically based on a majority rule, i.e., the number of clusters that is most frequently suggested across all indices.
The output includes the suggested number of clusters from each index, as well as a summary indicating the most commonly selected solution. This makes NbClust a useful complementary tool when deciding on the number of clusters, especially when different graphical methods give conflicting results.
library(NbClust)
#> Warning: package 'NbClust' was built under R version 4.5.2
set.seed(1234)
nc <- NbClust(wine_stand[,2:14], min.nc=2, max.nc=15, method="kmeans")
#> *** : The Hubert index is a graphical method of determining the number of clusters.
#> In the plot of Hubert index, we seek a significant knee that corresponds to a
#> significant increase of the value of the measure i.e the significant peak in Hubert
#> index second differences plot.
#>
#> *** : The D index is a graphical method of determining the number of clusters.
#> In the plot of D index, we seek a significant knee (the significant peak in Dindex
#> second differences plot) that corresponds to a significant increase of the value of
#> the measure.
#>
#> *******************************************************************
#> * Among all indices:
#> * 2 proposed 2 as the best number of clusters
#> * 19 proposed 3 as the best number of clusters
#> * 1 proposed 14 as the best number of clusters
#> * 1 proposed 15 as the best number of clusters
#>
#> ***** Conclusion *****
#>
#> * According to the majority rule, the best number of clusters is 3
#>
#>
#> *******************************************************************
table(nc$Best.n[1,])
#>
#> 0 1 2 3 14 15
#> 2 1 2 19 1 1
