Build and visualize eigenimagery that accounts for 80% of the variability. Provide full R code and discussion

Eigenimage(Eigenfaces), initially proposed by Sirovich and Kirby in 1987, refer to a set of fundamental features derived through principal component analysis (PCA), which builds upon singular value decomposition (SVD). The purpose of eigenimage is to transform the high-dimensional space of images into a lower-dimensional representation. These eigenigmages serve as a compact basis for expressing image variations. By combining these basic features linearly, one can effectively represent the entire set of n images, where r (the number of eigenimage) is less than n. (r<n)

Singular value decomposition involves identifying three key components for a matrix X of size m x n:

\[X =UΣV^T\]

Where U and V are m x m and n x n orthogonal matrices

Σ is an m x n diagonal matrix with the non-negative singular values σ_1, σ_2, σ_3,…,σ_n arranged in descending order along the diagonal.

Each image is transformed into a single large column vector.Next,the average image is computed and subtracted from each column vector.The mean-subtracted image vectors are arranged horizontally as columns in the matrix X.Singular Value Decomposition (SVD) is performed on the mean-subtracted matrix X leads to Principal Component Analysis (PCA).

The columns of U are the eigenimage.

The singular values provide essential information about the variability and importance of each eigenvector (eigenimage)

Larger singular values correspond to eigenvectors that capture significant variations in the images. Smaller singular values correspond to less important or noise-related variations.

Load the Shoe images

library(jpeg)
library(OpenImageR)
#read jpg files from working directory
shoes_list <- list.files(path='C:/Users/tonyl/OneDrive/Desktop/CUNY/Data 605/Week 4/jpg', pattern = "\\.jpg$", full.names = TRUE)

Create an Array

Create a matrix containing RGB vectors from the images.

# load the images:
jpg = readJPEG(paste0(shoes_list[1]))

# Create an matrix with empty entries.
shoe = matrix(0, length(shoes_list), prod(dim(jpg)))

# Assign image data as RBG vectors into the matrix.

for (i in 1:length(shoes_list)){
  im <- readJPEG(paste0(shoes_list[i]))
  r  <- as.vector(im[,,1])
  g  <- as.vector(im[,,2])
  b  <- as.vector(im[,,3])
  
  shoe[i,] = t(c(r,g,b))
}
# Transpose of the shoe matrix
shoe_t = as.data.frame(t(shoe))

Scale the image

The image data is centered and scaled for creating eigenimage.

scaled_shoes = scale(shoe_t, center = TRUE, scale = TRUE)

Computing the Eigenimage

Eignvalues, eigenvectors and sigma is generated.

# Generate Principal Components
pca_result <- prcomp(scaled_shoes, scale. = TRUE)

# eigenvalues and eigenvectors:
eigenvalues <- pca_result$sdev^2
eigenvectors <- pca_result$rotation

# compute the cumulative variance:
cum_var <- cumsum(eigenvalues) / sum(eigenvalues)
cum_var
##  [1] 0.6833138 0.7824740 0.8353528 0.8629270 0.8825040 0.8996099 0.9144723
##  [8] 0.9271856 0.9374462 0.9472860 0.9561859 0.9647964 0.9732571 0.9804242
## [15] 0.9874038 0.9941511 1.0000000
# find the number of eigenimage needed to explain 80% of the variance:
threshold <- min(which(cum_var > .80))
threshold
## [1] 3
# compute the eigenimage:
# This is exactly what SVD doing, we are using 80% of the variance (Scaling_shoes, which equals to Σ) to reconstruct a Matrix X

scaling_shoes = diag(eigenvalues[1:threshold]^(-1/2)) / (sqrt(nrow(scaled_shoes)-1))
eigen_shoes = scaled_shoes%*%eigenvectors[,1:threshold]%*%scaling_shoes
# Shoe that accounts for 83.5% of variability
imageShow(array(eigen_shoes[,3], dim(jpg)))

Reference: SVD and Eigenfaces. Yue Wang. 02/22/21 https://nextjournal.com/yuewangpl/svd-and-eigenfaces

Stanford Digital Imaging Process lecture https://web.stanford.edu/class/ee368/Handouts/Lectures/2019_Winter/10-EigenImages.pdf

Singular Value Decomposition,Eigenfaces, and 3D Reconstructions. Neil Muller, Lourenc¸o Magaia, B. M. Herbst. https://epubs.siam.org/doi/pdf/10.1137/S0036144501387517

Principal Component Analysis (PCA). Steve Brunton https://www.youtube.com/watch?v=fkf4IBRSeEc&list=PLMrJAkhIeNNSVjnsviglFoY2nXildDCcv&index=22

```