HW 11-PCA
Charlie Wiseman
Introduction:
The Automobile Dataset from the UCI Machine Learning Repository contains 205 car samples with 26 attributes, including numerical features like engine displacement, horsepower, curb weight, and miles per gallon (MPG), plus categorical features like car make or fuel type. For PCA, we’ll focus on numerical features, and you can use a categorical feature fuel-type (gas vs. diesel) for visualization.
Source: https://archive.ics.uci.edu/dataset/10/automobile
Download the dataset and import it here
library(tidyverse)
library(broom)
#Set working directory
# Load data, treating "?" as NA
auto_data <- read_csv("automobile.csv", na = "?")Data Preparation:
Clean & filter: Keep 8 numerical columns relevant to car. Features chosen: wheel_base, length, width, curb_weight, engine_size, horsepower, city_mpg, highway_mpg
auto_clean <- auto_data |>
select(fuel_type, wheel_base, length, width, curb_weight, engine_size,
horsepower, city_mpg, highway_mpg) |>
mutate(fuel_type = as.factor(fuel_type)) |> # Ensure fuel_type is a factor
drop_na() # Drop rows with NAs to ensure PCA works# Step 1: Standardize data
numeric_data <- auto_clean |>
select(where(is.numeric)) |>
scale()
# Step 2 +3 : Find Directions of Maximum Variation and rank them
pca_fit <- prcomp(numeric_data)
pca_fit## Standard deviations (1, .., p=8):
## [1] 2.4609703 1.0025191 0.6944628 0.4193810 0.3290435 0.2952943 0.2552724
## [8] 0.1406752
##
## Rotation (n x k) = (8 x 8):
## PC1 PC2 PC3 PC4 PC5
## wheel_base 0.3144622 -0.58325043 -0.1400494 -0.181761340 0.061969778
## length 0.3632918 -0.34403449 -0.1619823 -0.177442376 -0.589394926
## width 0.3628955 -0.25507029 0.1376091 0.809456560 0.293278038
## curb_weight 0.3911193 -0.09103041 0.1278216 -0.143468257 -0.003281696
## engine_size 0.3498786 0.11092441 0.6233518 -0.440961264 0.380154518
## horsepower 0.3324751 0.46636804 0.3311621 0.233287261 -0.553362040
## city_mpg -0.3495267 -0.38328744 0.4566544 0.005625945 -0.063787053
## highway_mpg -0.3596409 -0.30232298 0.4605478 0.103790754 -0.328564753
## PC6 PC7 PC8
## wheel_base 0.70494325 0.03057073 -0.081003965
## length -0.44153531 -0.36223845 0.135568383
## width -0.15608310 -0.13669176 0.005588183
## curb_weight -0.31120431 0.83061977 -0.122595116
## engine_size -0.07765246 -0.35955977 -0.049355129
## horsepower 0.42475591 0.06042025 0.132635371
## city_mpg 0.01025487 0.16044025 0.701742046
## highway_mpg -0.01840879 -0.02366389 -0.668975943pca_fit$sdev^2## [1] 6.05637465 1.00504446 0.48227854 0.17588046 0.10826966 0.08719873 0.06516400
## [8] 0.01978950#Plot the scatter plot with arrows
library(broom)
pca_fit |>
# Add PCs to the original dataset
augment(auto_clean) |>
ggplot(aes(.fittedPC1, .fittedPC2)) +
geom_point(aes(color = fuel_type)) + # Use fuel_type for coloring
# Add the PCA1 and PCA2 arrows
geom_segment(aes(x = -4.9, y = 0, xend = 5, yend = 0),
arrow = arrow(type = "closed", length = unit(0.1, "inches")),
color = "black") + # PC1 arrow
geom_segment(aes(x = 0, y = -2.5, xend = 0, yend = 4),
arrow = arrow(type = "closed", length = unit(0.1, "inches")),
color = "black") + # PC2 arrow
# Add text labels
geom_text(aes(x = 5, y = 0, label = "PC1"),
vjust = -0.5, color = "black") +
geom_text(aes(x = 0, y = 4, label = "PC2"),
hjust = -0.5, color = "black") +
# Customize the plot
xlab("PC1") +
ylab("PC2") +
theme_minimal()## Warning in geom_segment(aes(x = -4.9, y = 0, xend = 5, yend = 0), arrow = arrow(type = "closed", : All aesthetics have length 1, but the data has 203 rows.
## ℹ Please consider using `annotate()` or provide this layer with data containing
## a single row.## Warning in geom_segment(aes(x = 0, y = -2.5, xend = 0, yend = 4), arrow = arrow(type = "closed", : All aesthetics have length 1, but the data has 203 rows.
## ℹ Please consider using `annotate()` or provide this layer with data containing
## a single row.## Warning in geom_text(aes(x = 5, y = 0, label = "PC1"), vjust = -0.5, color = "black"): All aesthetics have length 1, but the data has 203 rows.
## ℹ Please consider using `annotate()` or provide this layer with data containing
## a single row.## Warning in geom_text(aes(x = 0, y = 4, label = "PC2"), hjust = -0.5, color = "black"): All aesthetics have length 1, but the data has 203 rows.
## ℹ Please consider using `annotate()` or provide this layer with data containing
## a single row.
#Plot the rotation arrows
#Rotation Matrix
arrow_style <- arrow(
angle = 20, length = grid::unit(8, "pt"),
ends = "first", type = "closed"
)
pca_fit |>
# extract rotation matrix
tidy(matrix = "rotation") |>
pivot_wider(
names_from = "PC", values_from = "value",
names_prefix = "PC"
) |>
ggplot(aes(PC1, PC2)) +
geom_segment(
xend = 0, yend = 0,
arrow = arrow_style
) +
geom_text(aes(label = column), hjust = 1) +
xlim(-1, 0.5) + ylim(-1, 0.7) +
coord_fixed()+
theme_minimal()
Question 1: What does PC1 show?
Describe the pattern captured by the first principal component (PC1). For example, does it reflect differences in car performance (e.g., horsepower, engine size) or efficiency (e.g., MPG)? Use the rotation plot to support your interpretation.
PC1 reflects a gradient between small, efficient and low-performing cars and large, inefficient and high-performance cars. Small PC1 values mean small width, length, horsepower, and engine size along with high highway and city MPG. High PC1 values mean large vehicles with large, powerful engines and low highway and city MPG. PC1 generally separates vehicles by gross size.
Question 2: What does PC2 show?
Explain what the second principal component (PC2) highlights. For instance, does it separate cars based on size (e.g., length, width) or other characteristics? Refer to the rotation plot for evidence.
PC2 reflects a gradient between fuel-efficient cars with large wheel bases and low-horsepower engines at low values and high horsepower cars with small wheel bases and low fuel efficiency at high values. PC2 might separate more modest cars with high-performance muscle and super cars.
Question 3: What does the analysis reveal about car types?
Based on the PC1 vs. PC2 scatter plot, do gas and diesel cars form distinct clusters? What do the clusters (or overlap) suggest about similarities or differences in their features (e.g., fuel efficiency, performance)? Use the rotation plot to identify key driving features.
Diesel cars have lower horsepower and engine size than most gas-powered cars but larger wheel bases. They cluster at the bottom half of the graph but spread across the left and right. This shows that gas cars generally have higher horsepower, smaller wheel bases and lower mpg than diesel cars. Both fuel types run the gamut of measurements better separated by PC1 such as width, length, and engine size. It is worth noting that gas cars exist with higher PC1 values than any diesel cars, so the graph also shows that the largest cars out there are gas cars.
Question 4: How much variation is explained by PC1 and PC2?
#How much variation is explained by PC1 and PC2?
pca_fit |>
tidy(matrix = "eigenvalues")## # A tibble: 8 × 4
## PC std.dev percent cumulative
## <dbl> <dbl> <dbl> <dbl>
## 1 1 2.46 0.757 0.757
## 2 2 1.00 0.126 0.883
## 3 3 0.694 0.0603 0.943
## 4 4 0.419 0.0220 0.965
## 5 5 0.329 0.0135 0.978
## 6 6 0.295 0.0109 0.989
## 7 7 0.255 0.00815 0.998
## 8 8 0.141 0.00247 1Report the percentage of variation explained by PC1 and PC2 using the PCA output. What does this tell you about how well these components summarize the dataset?
PC1 explains about 76% of variation, and PC2 explains about 13% of variation. Cumulatively, PC1 and PC2 explain about 88% of the variation.
This means that by looking at only the first two principal components, you can account for over three-fourths of the important information in 8 columns of data, making it easier to analyze and interpret.