Clustering_Analysis
Pin Lyu
2024-03-17
Data Set-up
# Load data
car_info <- read_excel('/Users/pin.lyu/Desktop/BC_Class_Folder/Market_Research/Data/Theltegos data.xlsx')Data Exploration
Basic Statistics
# Data transformation
car <- car_info[,-c(1)]
# Stats summary table
summary(car)## Displacement Moment Horsepower Length Width
## Min. : 996 Min. : 90.0 Min. : 65.0 Min. :3535 Min. :1595
## 1st Qu.:1364 1st Qu.:126.5 1st Qu.: 87.5 1st Qu.:4003 1st Qu.:1719
## Median :1968 Median :230.0 Median :136.0 Median :4315 Median :1769
## Mean :2001 Mean :213.4 Mean :146.7 Mean :4301 Mean :1759
## 3rd Qu.:2445 3rd Qu.:310.0 3rd Qu.:197.5 3rd Qu.:4684 3rd Qu.:1818
## Max. :3498 Max. :353.0 Max. :301.0 Max. :4916 Max. :1855
## Weight Trunk Speed Acceleration
## Min. : 929 Min. :127.0 Min. :154.0 Min. : 5.40
## 1st Qu.:1145 1st Qu.:272.5 1st Qu.:175.5 1st Qu.: 8.20
## Median :1340 Median :320.0 Median :201.0 Median :10.80
## Mean :1341 Mean :376.7 Mean :203.8 Mean :10.44
## 3rd Qu.:1573 3rd Qu.:530.0 3rd Qu.:238.0 3rd Qu.:12.30
## Max. :1660 Max. :588.0 Max. :275.0 Max. :15.10Correlation Matrix
# Confusion matrix
corr <- cor(car)
corrplot(corr, method = 'square', order = 'FPC', tl.col = 'black', addCoef.col = 'black', type = 'lower', diag = FALSE)
Based on the correlation matrix above, it’s evident that, aside from the variable “trunk,” which measures the total volume of a car’s trunk and does not strongly correlate with the other variables, the remaining variables exhibit strong correlations with each other. This is particularly notable with the variable “Acceleration,” where all variables, except “Trunk,” display significant negative correlations. In general, a higher horsepower generated by a vehicle’s engine indicates faster the top speed and acceleration.
Moreover, a vehicle’s displacement refers to the total volume of its cylinders, and higher values for these parameters typically translate to higher horsepower, resulting in faster speed and acceleration. It’s worth noting that “Acceleration” measures the time it takes for a vehicle to go from 0 to 60 mph. Therefore, the lower the value represents less time it takes for that vehicle to get to 60 mph which is a indication of how fast the car is. This is why parameters that contribute to faster acceleration correlate with variables showing a negative correlation, which are colored red in the correlation matrix.
Feature Selection
# Select four core variables in cars
car_select <- car[,c('Moment','Horsepower', 'Length', 'Weight','Acceleration', 'Trunk')]Data Standardization
# Standardize the data frame
std_car <- as.data.frame(scale(car_select))Clustering Analysis
1. Hierarchical Clustering Analysis
# Dissimilarity (distance) matrix
distance <- dist(car_select, method = "euclidean")
# Hierarchical clustering using Complete Linkage
hc1 <- hclust(distance, method = "single" )
# Plot dendrogram
plot(hc1, cex = 0.6, hang = -1, main = "Hierarchical Clustering Dendrogram")
# Check optimal level of clusters (the elbow method)
fviz_nbclust(car_select, FUN = hcut, method = "wss")
# Cut tree into 3 groups
sub_grp <- cutree(hc1, k = 3)
# Show different clusters in dendrogram
plot(hc1, cex = 0.6, main = "Hierarchical Clustering Dendrogram")
rect.hclust(hc1, k = 3, border = 2:5)
# Visualize the clusters
plot <- fviz_cluster(list(data = car_select, cluster = sub_grp))
# Add title
plot + ggtitle('Hierarchical Cluster Plot')
Data points 8 and 9 are kind of outliers this is because these vehicles are the only two sports cars in the data set.
## ANOVA for hierarchical clustering Anlaysis
# Factorize the output for table
sub_grp <- as.factor(sub_grp)
# Run MANOVA on all the variables
anova_hc <- aov(cbind(Moment,
Horsepower,
Length,
Weight,
Acceleration,
Trunk) ~ sub_grp, data = car_select)
# get a summary of anova results
summary(anova_hc)## Response Moment :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 128637 64318 82.015 1.004e-07 ***
## Residuals 12 9411 784
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Horsepower :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 80890 40445 49.308 1.63e-06 ***
## Residuals 12 9843 820
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Length :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 2380381 1190190 42.904 3.411e-06 ***
## Residuals 12 332889 27741
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Weight :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 679841 339920 31.388 1.708e-05 ***
## Residuals 12 129957 10830
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Acceleration :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 101.711 50.855 24.721 5.549e-05 ***
## Residuals 12 24.685 2.057
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Trunk :
## Df Sum Sq Mean Sq F value Pr(>F)
## sub_grp 2 285528 142764 33.11 1.304e-05 ***
## Residuals 12 51741 4312
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1From the results, all p-values of the F-stats are smaller than 0.05. Therefore, all variables differ significantly across the 3 clusters in the hierarchical method. Therefore, we will include all variables in the interpretation of clusters.
2. K-means Clustering Analysis
# K-Means cluster
km = kmeans(car_select, center = 3)
# Plot the cluster
clusplot(car_select, km$cluster,color = T, main = 'K-Means Cluster Plot')
# Cluster centers
center <- km$centers
print(center)## Moment Horsepower Length Weight Acceleration Trunk
## 1 346.5000 298.00000 4328.000 1475.000 5.600000 322.5000
## 2 116.8571 80.42857 3900.286 1115.571 12.957143 249.4286
## 3 281.6667 173.66667 4758.333 1560.167 9.116667 543.1667The average of each group by variable are significantly different from other groups. This suggests that teach group are different in nature.
## ANOVA for K-means clustering Anlaysis
# Factorize the output for table
km$cluster <- as.factor(km$cluster)
# Run MANOVA on all the variables
anova_car_km <- aov(cbind(Moment,
Horsepower,
Length,
Weight,
Acceleration,
Trunk) ~ km$cluster, data = car_select)
# get a summary of anova results
summary(anova_car_km)## Response Moment :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 128637 64318 82.015 1.004e-07 ***
## Residuals 12 9411 784
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Horsepower :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 80890 40445 49.308 1.63e-06 ***
## Residuals 12 9843 820
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Length :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 2380381 1190190 42.904 3.411e-06 ***
## Residuals 12 332889 27741
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Weight :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 679841 339920 31.388 1.708e-05 ***
## Residuals 12 129957 10830
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Acceleration :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 101.711 50.855 24.721 5.549e-05 ***
## Residuals 12 24.685 2.057
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Response Trunk :
## Df Sum Sq Mean Sq F value Pr(>F)
## km$cluster 2 285528 142764 33.11 1.304e-05 ***
## Residuals 12 51741 4312
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1All these variables are statistically significant in the K-means clusters which suggest that each clusters are meaningfully different from each other. With that said, we can say the result of K-means is identical to that of the hierarchical clusters. Therefore, their interpretations would be the same.
Report
In the steps above, I performed two hierarchical and K-means clustering analysis separately. before conducting the clustering analysis, I first exmained the variables in the data. There are, in total, of 9 variables (Columns) that provide information about the characteristics of a car made by a specific manufacturer (Rows). In total, there are 15 different brands of vehicles.
Next, I checked the multicollinearity of these variables. All variables, except for “Trunk”, are heavily correlated with each other either positively or negatively. Therefore, I excluded variables such “Width”, “Speed”, “Displacement”, as they provide duplicate information that other variables already include. “Width” has a correlation level of 0.91 with “Length” which means a extremely strong positive correlation between these two variables. Subsequently, I only selected “Length” to avoid redundancy. Following that, both “speed” and “Displacement” provide information on how fast a car can go. These information can be captured by variables such as “Horsepower” and “Acceleration” more directly. Therefore, I excluded these two variables as well.
Following these preliminary steps, I began the first clustering analysis using hierarchical methods. In this method, I used euclidean distance and single linkage method to group the data based on the selected variables. Then, I conducted an elbow test to determine the most optimal level of clusters that maximize the between distance of each cluster while minimizes the within distance of each cluster. Based on that test, I concluded that the best level of clusters is three. Then I utilized this information to conduct the second analysis using K-means. I specified that the number of clusters is three. after calculate the center mean for each cluster, I was able to conclude that each cluster is significantly different from other groups. Hence, making them a successful clustering analysis. It’s worth to note that both clustering methods returned the same results.
In group one, we have observation 1, 2, 3, 4, 5, 6, 7. Based on the variables used in the analysis, we can see that they all are light weight and have relative low horsepower when comparing to other vehicles in the same data set. And due to their low horsepower, their acceleration time from 0 to 60 mph is also longest among the three clusters of vehicles. Due to these characteristics, we can generalize these vehicles as the “compact vehicles”.
In the second group which only include two observations, 8 and 9. Their horsepower are the highest. Thus, that engine power translate into the most impressive acceleration time in 0 to 60 mps, in just 5.4 seconds and 5.6 seconds respectively. For these reason, I refer this group as “too-fast-to-see vehicles”.
Lastly, in group three, we have observations 10, 11, 12, 13, 14 and 15. These cars are noticeably faster than “compact vehicles” and are much heavier. They are also the lengthiest in the entire data set. And what comes with the length is the ability to offer a spacious trunk for storage. Comparing to other two groups, vehicles from this group also have the most trunk volume. For these reasons, I refers this group as “utility vehicles”.
As mentioned above, the two clustering methods have shown to produce the same results in clusters. Therefore, the interpretation of K-means should be the same as the one described for hierarchical clusters mentioned earlier.
Cluster Characteristics Break-down
| “Compact” | “Too-Fast-To-See” | “Utility” | |
|---|---|---|---|
| Obs | 1, 2, 3, 4, 5, 6, 7 | 8, 9 | 10, 11, 12, 13, 14, 15 |
| Moment | Low | High | Medium |
| Horsepower | Low | High | Medium |
| Length | Shortest | Medium | Longest |
| Weight | Lightest | Medium | Heaviest |
| Acceleration | Slowest | Fastest | Medium |
| Trunk | Low | Medium | High |