Problem Set 1

In this problem, we’ll verify using R that SVD and Eigenvalues are related as worked out in the weekly module. Given a 3 x 2 matrix A:

\[A = \left[\begin{array}{cc} 1 & 2 & 3 \\ -1 & 0 & 4 \end{array}\right]\]

write code in R to compute \(X = AA^T\) and \(Y = A^TA\). Then, compute the eigenvalues and eigenvectors of X and Y using the built-in commands in R. Then compute the left-singular, singular values, and right-singular vectors of A using the svd command. Examine the two sets of singular vectors and show that they are indeed eigenvectors of X and Y. In addition, the two non-zero eigenvalues (the 3rd value will be very close to zero, if not zero) of both X and Y are the same and are squares of the non-zero singular values of A.

Solution

Define \(A\), \(A^{T}\), \(X\), and \(Y\):

A <- matrix(c(1,-1,2,0,3,4),nrow=2)
At <- t(A)

X <- A%*%At
Y <- At%*%A

Define the eigenvectors and eigenvalues of \(X\) and \(Y\):

# eigenvalues and eigenvectors of X
evX <- eigen(X)
eValX <- evX$values
eVecX <- evX$vectors

# eigenvalues and eigenvectors of Y
evY <- eigen(Y)
eValY <- evY$values
eVecY <- evY$vectors

Compute the left-singular, singular values, and right-singular vectors of \(A\):

svdA <- svd(A)
leftVals <- svdA$u
singVals <- svdA$d
rightVals <- svdA$v

Examine the two sets of singular vectors and show that they are indeed eigenvectors of \(X\) and \(Y\).

Left singular values of \(A\) are eigenvectors of \(X\):

leftVals
##            [,1]       [,2]
## [1,] -0.6576043 -0.7533635
## [2,] -0.7533635  0.6576043
eVecX
##           [,1]       [,2]
## [1,] 0.6576043 -0.7533635
## [2,] 0.7533635  0.6576043

Right singular values of \(A\) are eigenvectors of \(Y\):

rightVals
##             [,1]       [,2]
## [1,]  0.01856629 -0.6727903
## [2,] -0.25499937 -0.7184510
## [3,] -0.96676296  0.1765824
eVecY
##             [,1]       [,2]       [,3]
## [1,] -0.01856629 -0.6727903  0.7396003
## [2,]  0.25499937 -0.7184510 -0.6471502
## [3,]  0.96676296  0.1765824  0.1849001

The two non-zero eigenvalues (the 3rd value will be very close to zero, if not zero) of both \(X\) and \(Y\) are the same and are squares of the non-zero singular values of \(A\).

singVals
## [1] 5.157693 2.097188
eValX
## [1] 26.601802  4.398198
singVals^2 - eValX
## [1] -3.552714e-15  2.664535e-15
singVals
## [1] 5.157693 2.097188
eValY
## [1] 2.660180e+01 4.398198e+00 1.058982e-16
singVals^2 - eValY[1:2]
## [1] 4.618528e-14 0.000000e+00