Problem Set 1

  1. Show that \(A^{T}A \neq AA^{T}\) in general. (Proof and demonstration)

Proof:
Suppose, \(A^{T}\) is a \(mxn\) matrix and lets call it \(B\)
So, \(A^{T} = B\) or \(A^{T}A= BA\)
By laws of matrices, \(AB \neq BA\)
\(\therefore A^{T}A \neq AA^{T}\) \(\Box\)

Demonstration

problem <- function(A){
  a_at <- A %*% t(A)
  at_a <- t(A) %*% A
  return(a_at == at_a)
}

problem(matrix(c(1, -2, 3,
                  4, 8, 4,
                  -3, 5, 7), 3))
##       [,1]  [,2]  [,3]
## [1,] FALSE FALSE FALSE
## [2,] FALSE FALSE FALSE
## [3,] FALSE FALSE FALSE
  1. For a special type of square matrix A, we get \(A^{T}A = AA^{T}\) . Under what conditions could this be true?

Under the following conditions \(A^{T}A = AA^{T}\) could be true - if matrix is symmetrical along the diagonal. 

problem(matrix(c(1, 4, 5,
                 4, 2, 6,
                 5, 6, 3), 3))
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE

Problem Set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars.

LU <- function(A){
  L <- diag(1, nrow = nrow(A), ncol = ncol(A))
  U <- A
  for(i in 1:(nrow(A)-1)){
    for(j in (i+1):nrow(A)){
      if(U[i,i] != 0){
        L[j,i] <- U[j,i] / U[i,i]
        U[j,] <- U[j,] - (L[j,i] * U[i,])
      }
    }  
  }
print(L)
print(U)
return(L%*%U)
}
A <- matrix(c(1,4,-3,-2,8,5,3,4,7), 3, 3, byrow = TRUE)

LU(A)
##      [,1] [,2] [,3]
## [1,]    1  0.0    0
## [2,]   -2  1.0    0
## [3,]    3 -0.5    1
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]   -2    8    5
## [3,]    3    4    7