Assignment 4

With the attached data file, build and visualize eigenimagery that accounts for 80% of the variability. Provide full R code and discussion.

Eigenimages are a methodology for using linear algebra to recognize complex patterns like faces and other images. Done correctly, only a limited number of eigenimages are needed to approximate a set of images with small error. A training set is created. Eigenfaces are derived from the training set. Individual pictures are projected onto the newly created eigenspace for those with higher eigenvalues.

This Stanford Digital Imaging Process lecture from an Electrical Engineering course covers the concepts with good detail: https://web.stanford.edu/class/ee368/Handouts/Lectures/2019_Winter/10-EigenImages.pdf

Note that a number of online sources were used for help with this assignment. These are listed at the bottom of this document. Of particular help was Diego Herrero’s Facial Recognition using Eigenfaces.

Load the Golf Shoe Photos

For our purposes, we will load our photos of golf shoes.

#number of files
num = length(list.files(pattern="\\.jpg$"))
#read photos from working directory (where .rmd file is stored by default)
shoes_jpg = list.files(pattern="\\.jpg$")[1:num]

Create an Initial Array

Create an array, using the first shoe picture to set dimensions.

#borrowing from a classmate, load the first image to set reasonable dimensions as an array
shoe_jpg = readJPEG(paste0(shoes_jpg[1]))

Create an empty matrix.

data = matrix(0, num, prod(dim(shoe_jpg)))

Load the Images

Load the image data as RBG vectors into the matrix.

for (i in 1:num){
  im <- readJPEG(paste0(shoes_jpg[i]))
  r  <- as.vector(im[,,1])
  g  <- as.vector(im[,,2])
  b  <- as.vector(im[,,3])
  
  data[i,] = t(c(r,g,b))
}

Create a dataframe with the transpose of the data matrix. Note that I don’t quite understand why the transpose is required here. Without it, the object becomes too large to conduct subsequent steps, and I noticed other examples that used it.

shoes_df = as.data.frame(t(data))
scaled_shoes = scale(shoes_df, center = TRUE, scale = TRUE)

Create Function to View Shoes

Due to the EBImage package no longer being available for R 4.0.1, I could not find an alternative solution to displaying the original shoe plots.

Check Our Variance

We will look at the variance to determine the threshold for variability.

cum_var = cumsum(eigenvalues) / sum(eigenvalues)
cum_var
##  [1] 0.6833138 0.7824740 0.8353528 0.8629269 0.8825039 0.8996099 0.9144723
##  [8] 0.9271856 0.9374462 0.9472859 0.9561859 0.9647964 0.9732571 0.9804242
## [15] 0.9874038 0.9941511 1.0000000

We can see that we exceed 80% variability just using three images.

thres = min(which(cum_var > .80))
thres
## [1] 3

To better examine how quickly this can be achieved, let’s build a graph.

cum_var_df = as.data.frame(cum_var)

ggplot(cum_var_df, aes(x=1:num, y=cum_var)) + geom_line() + geom_point() +
  xlab("# of Eigenfaces") + ylab("Cumulative Variance")

We see that the third eigenface gives us the requested variability.

Create Eigenshoes

Borrowing from the work of others below - I understand some of the methodology but not all.

We’re creating a diagonal matrix based on the number of threshold components (in our case 3) before scaling it. This is our scaling matrix. We then multiply our original scaled dataframe by our first three eigenvectors followed by that diagonal matrix.

scaling_shoes = diag(eigenvalues[1:thres]^(-1/2)) / (sqrt(nrow(scaled_shoes)-1))
eigen_shoes = scaled_shoes%*%eigenvectors[,1:thres]%*%scaling_shoes
eigenimage <- array(eigen_shoes[,3], dim(shoe_jpg))

Now display our third image using the plotting.

imageShow(eigenimage)

Conclusion

Eigenimages are a set of eigenvectors that can be used to recognize images utilizing dimensionality reduction. Images are converted to r, g, and b vectors, which are combined into a matrix. The covariance matrix is computed. From there, the eigenimages are computed.

At a high level, the images of these shoes can have millions of pixels, which would not make for efficient computing. We attempt to represent this set of images by truncating the matrices to lower dimensions.