HW #4 - Problem Set 1: Verify SVD and Eigenvalues are related
1.setup a \(3\times2\) matrix A, compute \(X=AA^{T}\), \(Y=A^{T}A\), and eigenvalues and eigenvector of both X and Y
A <- matrix(c(1,2,3,-1,0,4),byrow = TRUE,2,3)
A
## [,1] [,2] [,3]
## [1,] 1 2 3
## [2,] -1 0 4
X <- A%*%t(A)
X_eigenvalues <- eigen(X)$values
X_eigenvectors <- eigen(X)$vectors
Y <- t(A)%*%A
Y_eigenvalues <- eigen(Y)$values
Y_eigenvectors <- eigen(Y)$vectors
X
## [,1] [,2]
## [1,] 14 11
## [2,] 11 17
X_eigenvalues
## [1] 26.601802 4.398198
X_eigenvectors
## [,1] [,2]
## [1,] 0.6576043 -0.7533635
## [2,] 0.7533635 0.6576043
Y
## [,1] [,2] [,3]
## [1,] 2 2 -1
## [2,] 2 4 6
## [3,] -1 6 25
Y_eigenvalues
## [1] 2.660180e+01 4.398198e+00 1.058982e-16
Y_eigenvectors
## [,1] [,2] [,3]
## [1,] -0.01856629 -0.6727903 0.7396003
## [2,] 0.25499937 -0.7184510 -0.6471502
## [3,] 0.96676296 0.1765824 0.1849001
2.compute the left-singular, sigular values, and right-singular vectors of A
svd(A)
## $d
## [1] 5.157693 2.097188
##
## $u
## [,1] [,2]
## [1,] -0.6576043 -0.7533635
## [2,] -0.7533635 0.6576043
##
## $v
## [,1] [,2]
## [1,] 0.01856629 -0.6727903
## [2,] -0.25499937 -0.7184510
## [3,] -0.96676296 0.1765824
For a singular value decomposition, \(A=U\sum V^{T}\) where
U and V represent orthogonal unit vectors, or the left-singular and right-singular vectors;
\(\sum\) represents a diagonal matrix.
3.verify: the two sets of sigular vectors are eigenvector of X and Y.
eigenvectors of \(AA^{T}\) is the same as left-sigular vectors:
X_eigenvectors
## [,1] [,2]
## [1,] 0.6576043 -0.7533635
## [2,] 0.7533635 0.6576043
svd(A)$u
## [,1] [,2]
## [1,] -0.6576043 -0.7533635
## [2,] -0.7533635 0.6576043
round(abs(svd(A)$u),7)==round(abs(X_eigenvectors),7)
## [,1] [,2]
## [1,] TRUE TRUE
## [2,] TRUE TRUE
eigenvectors of \(A^{T}A\) is the same as right-singular vectors:
Y_eigenvectors
## [,1] [,2] [,3]
## [1,] -0.01856629 -0.6727903 0.7396003
## [2,] 0.25499937 -0.7184510 -0.6471502
## [3,] 0.96676296 0.1765824 0.1849001
svd(A)$v
## [,1] [,2]
## [1,] 0.01856629 -0.6727903
## [2,] -0.25499937 -0.7184510
## [3,] -0.96676296 0.1765824
round(abs(svd(A)$v))==round(abs(Y_eigenvectors[1:3,1:2]))
## [,1] [,2]
## [1,] TRUE TRUE
## [2,] TRUE TRUE
## [3,] TRUE TRUE
Note: since eigenvectors can be scaled by -1, absolute values are compared.
4.verify: the two non-zero eigenvalues of both X and Y are the same,
X_eigenvalues
## [1] 26.601802 4.398198
Y_eigenvalues
## [1] 2.660180e+01 4.398198e+00 1.058982e-16
round(X_eigenvalues,7)==round(Y_eigenvalues[1:2],7)
## [1] TRUE TRUE
and are squares of the non-zero singular values of A.
svd(A)$d
## [1] 5.157693 2.097188
round(sqrt(X_eigenvalues),7)==round(svd(A)$d,7)
## [1] TRUE TRUE
HW #4 - Problem Set 2: Compute Inverse with Co-factors
1.create a function to compute inverse of a matrix using co-factors.
myinverse <- function(matrixA){
cofactor_matrix <- matrix(,nrow(matrixA),ncol(matrixA))
i <- 1
j <- 1
while (i <= nrow(matrixA)) {
while (j <= ncol(matrixA)) {
factor_ij <- (-1)^(i+j)
submatrix <- matrixA[-i,-j]
cofactor_matrix[i,j] <- factor_ij*det(submatrix)
j <- j+1
}
j <- 1
i <- i+1
}
inverseA <- t(cofactor_matrix)/det(matrixA)
return(inverseA)
}
The function is based on \(A^{-1}=C^{T}/det(A)\) where \(C^{T}\) is the transpose of the co-factor matrix.
2.Examine the function.(assume the matrix is a well-conditioned full-rank square matrix)
testA <- matrix(c(1,3,3,4,6,6,7,7,9,5,3,1,8,3,1,4),byrow = TRUE,4,4)
inverse_A <- myinverse(testA)
testA
## [,1] [,2] [,3] [,4]
## [1,] 1 3 3 4
## [2,] 6 6 7 7
## [3,] 9 5 3 1
## [4,] 8 3 1 4
inverse_A
## [,1] [,2] [,3] [,4]
## [1,] -0.32015810 0.134387352 -0.02371542 0.09090909
## [2,] 0.92885375 -0.525691700 0.32806324 -0.09090909
## [3,] -0.62055336 0.470355731 -0.08300395 -0.18181818
## [4,] 0.09881423 0.007905138 -0.17786561 0.18181818
round(testA%*%inverse_A,7)==diag(nrow(testA))
## [,1] [,2] [,3] [,4]
## [1,] TRUE TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE TRUE
## [4,] TRUE TRUE TRUE TRUE