Homework 4

Question 1

acs <- read.csv("HW3data.csv")
state <- acs$State
data <- select(acs, -State)

a.

pca.sc <- prcomp(data, scale = TRUE)
data_pca <- pca.sc$x[ , 1:7]
row.names(data_pca) <- state

b.

dists <- dist(data_pca)

c.

hclust.sing <- hclust(dists, method = "single")
par(mar = c(.5, 2, .5, 1)) 
plot(hclust.sing, hang = -1, sub = "", main = "")
  abline(h = hclust.sing$height[51 - 7], lty = 2)

d.

Under single linkage, clusters are considered close if just one of their values is close to the value of another cluster. So clusters that merge together only need to share one similar observation, and the distance between those observations must be within our cut height distance. Therefore, clusters that share no observations within the cut height distance will not merge with each other, and so every cluster that has merged into one, must have at least one observation within the cut height distance to another observation in its cluster. Since cluster 1 and 3 obviously didn’t merge that means they share no observations within the cut height distance, which we can check for here.

clus.sing <- cutree(hclust.sing, h = hclust.sing$height[51 - 7])
  table(clus.sing)

clus.sing
 1  2  3  4  5  6  7 
41  1  5  1  1  1  1 

d.mat <- as.matrix(dists)
min(d.mat[which(clus.sing == 1), which(clus.sing == 3)])

[1] 3.851532

The smallest distance between any two observations in cluster 1 and cluster 3 is 3.85 which is greater than our cut height distance of 3.77.

e.

The distance between Arizona and Florida, and Texas and California, are both less than the cut height distance.

d.mat[which(clus.sing == 3), which(clus.sing == 3)]

            Arizona California  Florida   Nevada    Texas
Arizona    0.000000   4.919295 2.837324 3.706277 3.994248
California 4.919295   0.000000 4.725443 7.192576 2.925050
Florida    2.837324   4.725443 0.000000 4.482842 3.770813
Nevada     3.706277   7.192576 4.482842 0.000000 6.559933
Texas      3.994248   2.925050 3.770813 6.559933 0.000000

f. 

Complete linkage has stricter conditions than single linkage. Complete linkage looks at the distance between the two farthest observations in clusters. So instead of merging clusters with one similar observation it merges clusters with all similar observations. This results in a more spaced out dendrogram since clusters must be more similar to merge than under single linkage. Under single linkage clusters were able to merge much quicker.

hclust.com <- hclust(dists, method = "complete")
par(mar = c(.5, 2, .5, 1)) 
plot(hclust.com, hang = -1, sub = "", main = "")
  abline(h = 7.5, lty = 2)

g.

With a cut height of 7.5, we would have 9 clusters. The three states in their own clusters are the District of Columbia, Alaska, and Hawaii. This tells us these states are fairly different than most states.

clus.com <- cutree(hclust.com, h = 7.5)
  table(clus.com)

clus.com
 1  2  3  4  5  6  7  8  9 
 7  1  6 13  1 14  1  4  4 

h.

The maximum distance between observations in cluster 8 and 9 is 7.89 which is greater than our cut height distance. So these clusters can’t and didn’t merge at our cut height.

max(d.mat[which(clus.com == 8), which(clus.com == 9)])

[1] 7.894893

i.

Looking at the distances between all the observations in cluster 8, you’ll notice all distances are less than the cut height distance.

d.mat[which(clus.com == 8), which(clus.com == 8)]

                 Maine Minnesota New Hampshire  Vermont
Maine         0.000000  4.240758      3.901416 2.949826
Minnesota     4.240758  0.000000      1.781645 4.887299
New Hampshire 3.901416  1.781645      0.000000 3.965155
Vermont       2.949826  4.887299      3.965155 0.000000

j.

The function choose 9 clusters. States with the lowest maximum cluster probabilities are the ones the model is least confident about assigning to a single cluster. Their probabilities are more spread out across multiple clusters, meaning they don’t strongly fit into any one cluster and may lie between clusters or have mixed characteristics. The states with the min maxes are Oregon, Maryland, and Maine, so these states didn’t strongly fit into one single cluster as well as the others.

gmm_model <- Mclust(data_pca)
probs <- apply(gmm_model$z, 1, max)
sort(probs)[1:3]

   Oregon  Maryland     Maine 
0.7499619 0.8475613 0.9154546 

k.

There does appear to be an elbow in the k-distance plot around 6.8. We got 1 cluster and 3 noise points. To no surprise, the three outlier states were Alaska, District of Columbia, and Hawaii.

minPts <- 2*ncol(data_pca)
kNNdistplot(data_pca, k = minPts - 1)
  abline(h = 6.8, lty = 2)

db <- dbscan(data_pca, eps = 6.8, minPts = minPts)
  print(db)

DBSCAN clustering for 51 objects.
Parameters: eps = 6.8, minPts = 14
Using euclidean distances and borderpoints = TRUE
The clustering contains 1 cluster(s) and 3 noise points.

 0  1 
 3 48 

Available fields: cluster, eps, minPts, metric, borderPoints

state[db$cluster == 0]

[1] "Alaska"               "District of Columbia" "Hawaii"              

l.

Once again, the three outlier states were Alaska, District of Columbia, and Hawaii.

lof_scores <- lof(data_pca, minPts = 8)
sort(head(order(lof_scores, decreasing = TRUE), n = 3))

[1]  2  9 12

state[c(2,9,12)]

[1] "Alaska"               "District of Columbia" "Hawaii"