Problem Set 1

\[A = U_{mxr}\Sigma_{rxr}V^T_{rxn}\]

\[A^TA = V_{nxr}\Sigma^2_{rxr}V^T_{rxn}\]

\[AA^T = U_{mxr}\Sigma^2_{rxr}U^T_{rxm}\]

A <- matrix(c(1,2,3,1,1,1,2,0,1), nrow = 3)

I <- matrix(c(1,0,0,0,1,0,0,0,1), nrow = 3)

#A^TA
ATA <- t(A) %*% A
eigen(ATA)
## $values
## [1] 19.029505  2.801686  0.168809
## 
## $vectors
##           [,1]        [,2]       [,3]
## [1,] 0.8398531 -0.44770719  0.3069284
## [2,] 0.3858246  0.09462844 -0.9177063
## [3,] 0.3818195  0.88915900  0.2522104
#AA^T
AAT <- A %*% t(A)
eigen(AAT)
## $values
## [1] 19.029505  2.801686  0.168809
## 
## $vectors
##            [,1]       [,2]       [,3]
## [1,] -0.4560266  0.8514870 -0.2588620
## [2,] -0.4734978 -0.4784171 -0.7395383
## [3,] -0.7535513 -0.2146786  0.6213481
#Perform the same test on the identity matrix I
#I^TI
ITI <- t(I) %*% I
eigen(ITI)
## $values
## [1] 1 1 1
## 
## $vectors
##      [,1] [,2] [,3]
## [1,]    0    0    1
## [2,]    0    1    0
## [3,]    1    0    0
#BB^T
IIT <- I %*% t(I)
eigen(IIT)
## $values
## [1] 1 1 1
## 
## $vectors
##      [,1] [,2] [,3]
## [1,]    0    0    1
## [2,]    0    1    0
## [3,]    1    0    0

By example, we can see that for this particular 3x3 matrix A, \[A^TA \neq AA^T\]. On the contrary, \[I^TI = II^T\]

  1. Two square matrices A and B will be similar if they are non-singular, having equal non-zero eigenvalues if A is of rank r, but this does not mean their eigenvectors will be the same. We can see in example matrix A above that their eigenvalues are the same but their eigenvectors are not. However, in the example of the identity matrix above, both the eigenvalues and the eigenvectors are equal.

Problem Set 2

#Starting matrix
A <- matrix(c(1,2,3,1,1,1,2,0,1), nrow = 3)
A
##      [,1] [,2] [,3]
## [1,]    1    1    2
## [2,]    2    1    0
## [3,]    3    1    1
#Matrix to eliminate the 2 in the 2nd
E21 <- matrix(c(1,-2,0,0,1,0,0,0,1), nrow = 3)
E21
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]   -2    1    0
## [3,]    0    0    1
#Perform the elimination
E21.A <- E21 %*% A

#Matrix to eliminate the 3 in the 3rd row
E31 = matrix(c(1,0,-3,0,1,0,0,0,1), nrow = 3)
E31
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]   -3    0    1
#Perform the elimination
E31.E21.A <- E31 %*% E21.A

#Matrix to eliminate the -2 in the 3rd row 2nd column
E32 <- matrix(c(1,0,0,0,1,-2,0,0,1), nrow = 3)
E32
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0   -2    1
##Perform the final elimination
U <- E32 %*% E31.E21.A
U
##      [,1] [,2] [,3]
## [1,]    1    1    2
## [2,]    0   -1   -4
## [3,]    0    0    3
#Compute the inverse of the product of E21 E31 and E32
L <- solve(E21) %*% solve(E31) %*% solve(E32)

#Print both matrices to see side by side if they are equal
L %*% U
##      [,1] [,2] [,3]
## [1,]    1    1    2
## [2,]    2    1    0
## [3,]    3    1    1
A
##      [,1] [,2] [,3]
## [1,]    1    1    2
## [2,]    2    1    0
## [3,]    3    1    1
#Verify that the matrices are equal
A == L %*% U
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE

\[A = LU\]