Activity 4.2 - Kmeans, PAM, and DBSCAN clustering

SUBMISSION INSTRUCTIONS

1. Render to html
2. Publish your html to RPubs
3. Submit a link to your published solutions

Loading required packages:

library(cluster)
library(dbscan)
library(factoextra)
library(tidyverse)
library(patchwork)
library(ggrepel)
library(ggplot2)

Question 1

Reconsider the three data sets below. We will now compare kmeans, PAM, and DBSCAN to cluster these data sets.

three_spheres <- read.csv('/Users/rosagomez/Desktop/DSCI 415/Activities/Data/cluster_data1.csv')
ring_moon_sphere <- read.csv('/Users/rosagomez/Desktop/DSCI 415/Activities/Data/cluster_data2.csv')
two_spirals_sphere <- read.csv('/Users/rosagomez/Desktop/DSCI 415/Activities/Data/cluster_data3.csv')

A)

With kmeans and PAM, we can specify that we want 3 clusters. But recall with DBSCAN we select minPts and eps, and the number of clusters is determined accordingly. Use k-nearest-neighbor distance plots to determine candidate epsilon values for each data set if minPts = 4. Add horizontal line(s) to each plot indicating your selected value(s) of \(\epsilon.\)

kNNdistplot(three_spheres[,1:2], minPts = 4)
abline(h=0.22)

Looks like it’s taking off right between 0.20 and 0.25

kNNdistplot(ring_moon_sphere[,1:2], minPts = 4)
abline(h=0.23)

Looks like it’s taking off between 0.20 and 0.24 possibly.

kNNdistplot(two_spirals_sphere[,1:2], minPts = 4)
abline(h=0.10)

This one has two elbows. choose the first elbow for the K.

B)

Write a function called plot_dbscan_results(df, eps, minPts). This function takes a data frame, epsilon value, and minPts as arguments and does the following:

• Runs DBSCAN on the inputted data frame df, given the eps and minPts values;
• Creates a scatterplot of the data frame with points color-coded by assigned cluster membership. Make sure the title of the plot includes the value of eps and minPts used to create the clusters!!

Using this function, and your candidate eps values from A) as a starting point, implement DBSCAN to correctly identify the 3 cluster shapes in each of the three data sets. You will likely need to revise the eps values until you settle on a “correct” solution.

plot_dbscan_results_1 <- dbscan(three_spheres[,1:2],eps = 0.29, minPts = 4)
three_spheres$dbcluster <- factor(plot_dbscan_results_1$cluster)

p1 <- ggplot(data = three_spheres, aes(x = x, y = y, color=dbcluster)) +
  geom_point() + 
  labs(title= "Three Spheres, eps = 0.29, minPts = 4" , color='Cluster') + 
  theme_classic(base_size = 13) 
p1

plot_dbscan_results_2 <- dbscan(ring_moon_sphere[,1:2],eps = 0.28, minPts = 4)
ring_moon_sphere$dbcluster <- factor(plot_dbscan_results_2$cluster)

p2 <- ggplot(data =ring_moon_sphere, aes(x = x, y = y, color=dbcluster)) +
  geom_point() + 
  labs( title = "Ring Moon Sphere, eps = 0.28, minPts = 4" , color='Cluster') + 
  theme_classic(base_size = 13)
p2

plot_dbscan_results_3 <- dbscan(two_spirals_sphere[,1:2],eps = 0.1, minPts = 4)
two_spirals_sphere$dbcluster <- factor(plot_dbscan_results_3$cluster)

p3 <- ggplot(data = two_spirals_sphere, aes(x = x, y = y, color=dbcluster)) +
  geom_point() + 
  labs(title = "Two Spiral Sphere, eps = 0.1, minPts = 4" , color='Cluster') + 
  theme_classic(base_size = 13)
p3

C)

Compare your DBSCAN solutions to the 3-cluster solutions from k-means and PAM. Use the patchwork package and your function from B) to produce a 3x3 grid of plots: one plot per method/data set combo. Comment on your findings.

par(mfrow=c(1,3))
kNNdistplot(three_spheres[,1:2], minPts = 4)
abline(h=0.105)
kNNdistplot(ring_moon_sphere[,1:2], minPts = 4)
abline(h=0.079)
kNNdistplot(two_spirals_sphere[,1:2], minPts = 4)
abline(h=0.10)

(p1 ) /( p2 ) / (p3 )

DBSCAN solutions are hard to find the Epsilon based off the elbow. I was having a difficult time deciding on where to put the horizontal line. Do I put the line where the elbow is more defined or where it takes off? I first tried where the elbow was more defined, and the Epsilon was too low, it created a lot of different clusters in the 3-cluster solutions from kmeans and pam.

Question 2

In this question we will apply cluster analysis to analyze economic development indicators (WDIs) from the World Bank. The data are all 2020 indicators and include:

• life_expectancy: average life expectancy at birth
• gdp: GDP per capita, in 2015 USD
• co2: CO2 emissions, in metric tons per capita
• fert_rate: annual births per 1000 women
• health: percentage of GDP spent on health care
• imports and exports: imports and exports as a percentage of GDP
• internet and electricity: percentage of population with access to internet and electricity, respectively
• infant_mort: infant mortality rate, infant deaths per 1000 live births
• inflation: consumer price inflation, as annual percentage
• income: annual per-capita income, in 2020 USD

wdi <- read.csv('/Users/rosagomez/Desktop/DSCI 415/Activities/Data/wdi_extract_clean.csv') 
head(wdi)

      country life_expectancy        gdp      co2 fert_rate    health internet
1 Afghanistan        61.45400   527.8346 0.180555     5.145 15.533614  17.0485
2     Albania        77.82400  4437.6535 1.607133     1.371  7.503894  72.2377
3     Algeria        73.25700  4363.6853 3.902928     2.940  5.638317  63.4727
4      Angola        63.11600  2433.3764 0.619139     5.371  3.274885  36.6347
5   Argentina        75.87800 11393.0506 3.764393     1.601 10.450306  85.5144
6     Armenia        73.37561  4032.0904 2.334560     1.700 12.240562  76.5077
  infant_mort electricity  imports inflation  exports    income
1        55.3        97.7 36.28908  5.601888 10.42082  475.7181
2         8.1       100.0 36.97995  1.620887 22.54076 4322.5497
3        20.4        99.7 24.85456  2.415131 15.53520 2689.8725
4        42.3        47.0 27.62749 22.271539 38.31454 1100.2175
5         8.7       100.0 13.59828 42.015095 16.60541 7241.0303
6        10.2       100.0 39.72382  1.211436 29.76499 3617.0320

Focus on using kmeans for this problem.

A)

My claim: 3-5 clusters appear optimal for this data set. Support or refute my claim using appropriate visualizations.

wdi_scaled <- scale(wdi %>% select(life_expectancy:income))

fviz_nbclust(wdi_scaled,
             FUNcluster = kmeans)+
  fviz_nbclust(wdi_scaled,
               FUNcluster = kmeans, 
               method = 'wss')

kmeans_best <- kmeans(wdi_scaled, centers= 4, nstart =10)

With the average silhouette and wss plots, it looks like 3-5 would be the best. The highest number of K in the silhouette is 4, 3-4 is almost the same, however , there’s a big dip at 5. But in the WSS plot, 5 looks like it’s the best, as it is where the dip isn’t as dramatic. I believe 4 would be the best.

B)

Use k-means to identify 4 clusters. Characterize the 4 clusters using a dimension reduction technique. Provide examples of countries that are representative of each cluster. Be thorough.

kNNdistplot(wdi_scaled, minPts = 5)
abline(h = 4.5)

wdi_dbscan <- dbscan(wdi_scaled, eps = 4.5, minPts = 5)

rownames(wdi_scaled) <- wdi$country
wdi_pca <- prcomp(wdi_scaled)

selected_labels <- rownames(wdi_scaled)

fviz_pca(wdi_pca,
         habillage = factor(kmeans_best$cluster),
         label = 'var',
         repel = TRUE) + 
  geom_text_repel(aes(label = selected_labels), size = 3) +
  ggtitle("DBscan cluster results") +
  guides (shapes = 'none') +
  labs (color = 'country', shape = 'Cluster')

Warning: Using `size` aesthetic for lines was deprecated in ggplot2 3.4.0.
ℹ Please use `linewidth` instead.
ℹ The deprecated feature was likely used in the ggpubr package.
  Please report the issue at <https://github.com/kassambara/ggpubr/issues>.

Warning: ggrepel: 111 unlabeled data points (too many overlaps). Consider
increasing max.overlaps

For the first cluster (red circle) cluser such as Nigeria, Mali, Benin have high fertility rates, and infant mortality rates. Because the electricity , health and life expectancy are directly across from those countries it has low electricty, health and life expectancy rates. This is telling me these countries have low economic development.

The second cluster ( green triangle) such as Colombia, Brazil , Argentina have low imports and export rates, have a low GDP and income as well. They also have higher fertility rates, and better health, or upcoming better living conditions. Because they are right in the middle and maybe leaning more towards the first right quadrant, this is telling me that they are growing economically.

Third cluster, blue squares , such as the US, Japan, Norway, Qatar, have high internet, and eletricity accessibility. Also they have high income, same as Switzerland and United Arab Emirates. They also have low fertility rates and low infant mortality rates. They are countries you would want to live in to have a better quality of life.

Cluster 4, purple crosses, these are more of the outliers, such as singapore, ireland, luxembourg, these countries have really high import and export rates.

C)

Remove Ireland, Singapore, and Luxembourg from the data set. Use k-means to find 4 clusters again, with these three countries removed. How do the cluster definitions change?

remove <- c("Ireland", "Singapore", "Luxembourg")
wdi_filtered <- wdi[!wdi$country %in% remove,]

wdi_scaled_filtered <- wdi_filtered %>% select(where (is.numeric)) %>% scale()

rownames (wdi_scaled_filtered ) <- wdi_filtered$country


wdi_pca_filtered <- prcomp(wdi_scaled_filtered)

kmeans_filtered <- kmeans(wdi_scaled_filtered, centers = 4, nstart =25)

selected_labels <- rownames(wdi_scaled_filtered)

fviz_pca(wdi_pca_filtered,
         habillage = factor(kmeans_filtered$cluster),
         label = 'var',
         repel = TRUE) + 
  geom_text_repel(aes(label = selected_labels), size = 3) +
  ggtitle("DBscan cluster results") +
  guides (shapes = 'none') +
  labs (color = 'country', shape = 'Cluster')

Warning: ggrepel: 85 unlabeled data points (too many overlaps). Consider
increasing max.overlaps

Now with the outliers removed, the clusters do change, making the United States an outlier almost. It shifted the United states more towards better health and high income, gdp. It also changed Colombia and Sudan as high inflation countries. The countries with high fertility and high infant mortalitie rates also changed. It shifted the data around.