Problem Set 1

  1. Show that A^T A != A A^T in general. (Proof and demonstration.)

Matrix mulitplication is not communicative, as demonstrated below. http://mathworld.wolfram.com/MatrixMultiplication.html

A = matrix(c(1, 2, 3, 1, 1, 1, 2, 3, 4), nrow = 3, ncol = 3)
A
##      [,1] [,2] [,3]
## [1,]    1    1    2
## [2,]    2    1    3
## [3,]    3    1    4
AT = t(A)
AT
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    1    1    1
## [3,]    2    3    4
ATA = AT %*% A
ATA
##      [,1] [,2] [,3]
## [1,]   14    6   20
## [2,]    6    3    9
## [3,]   20    9   29
AAT = A %*% AT
AAT
##      [,1] [,2] [,3]
## [1,]    6    9   12
## [2,]    9   14   19
## [3,]   12   19   26
ATA == AAT
##       [,1]  [,2]  [,3]
## [1,] FALSE FALSE FALSE
## [2,] FALSE FALSE FALSE
## [3,] FALSE FALSE FALSE
  1. For a special type of square matrix A, we get AT A = AAT . Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

Matrix multiplication IS communicative if matrices A and B (or A^T) are symmetrical along the diagonal so that A^T = A. This also means it will only be true for square matrices. For example.

A = matrix(c(1, 2, 3, 2, 1, 2, 3, 2, 1), nrow = 3, ncol = 3)
A
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    2    1    2
## [3,]    3    2    1
AT = t(A)
AT
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    2    1    2
## [3,]    3    2    1
ATA = AT %*% A
ATA
##      [,1] [,2] [,3]
## [1,]   14   10   10
## [2,]   10    9   10
## [3,]   10   10   14
AAT = A %*% AT
AAT
##      [,1] [,2] [,3]
## [1,]   14   10   10
## [2,]   10    9   10
## [3,]   10   10   14
ATA == AAT
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE

Problem Set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars. Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer. You don’t have to worry about permuting rows of A and you can assume that A is less than 5x5, if you need to hard-code any variables in your code. If you doing the entire assignment in R, then please submit only one markdown document for both the problems.

Once again, used the concepts outlined using the youtube video here: https://www.youtube.com/watch?v=f-zQJtkgcOE

factorize <- function(A){

if (nrow(A) == 2){
  value <- -A[2,1] / A[1,1]
  multiply <- matrix(c(1, value, 0, 1), nrow=2, ncol=2) 
  lower_triangular_matrix <- multiply %*% A
  
} else if (nrow(A) == 3){
  value <- (A[,1]/(-1*A[1,1]))
  multiply <- matrix(c(1, value[2], value[3], 0, 1, 0, 0, 0, 1), nrow=3, ncol=3) 
  transformation_1 <- multiply %*% A
  
  value_2 <- -1*(transformation_1[3,2]/transformation_1[2,2])
  multiply_2 <- matrix(c(1, 0, 0, 0, 1, value_2, 0, 0, 1), nrow=3, ncol=3) 
  lower_triangular_matrix <- multiply_2 %*% transformation_1
  
} else if (nrow(A) == 4){
  value <- (A[,1]/(-1*A[1,1]))
  multiply <- matrix(c(1, value[2], value[3], value[4], 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1), nrow=4, ncol=4)
  transformation_1 <- multiply %*% A
  
  value_1 <- (transformation_1[,2]/(-1*transformation_1[2,2]))
  multiply_1 <- matrix(c(1, 0, 0, 0, 0, 1, value_1[3], value_1[4], 0, 0, 1, 0, 0, 0, 0, 1), nrow=4, ncol=4)
  transformation_2 <- round(multiply_1 %*% transformation_1, 1)
  
  value_2 <- round((-1*(transformation_2[4,3])/(transformation_2[3,3])), 1)
  multiply_2 <- matrix(c(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, value_2, 0, 0, 0, 1), nrow=4, ncol=4)
  lower_triangular_matrix <- round(multiply_2 %*% transformation_2, 1) #account for rounding errors
}

  return (list(lower_triangular_matrix = lower_triangular_matrix))
}
test_2 <-  matrix(
    c(4, 6, 8, 13), 
    nrow=2, 
    ncol=2)
factorize(test_2)
## $lower_triangular_matrix
##      [,1] [,2]
## [1,]    4    8
## [2,]    0    1
test_3 <-  matrix(
    c(14, 5, 52, 46, 38, 32, 4, 1, 2, 4, 7, 13), 
    nrow=3, 
    ncol=3)
factorize(test_3)
## $lower_triangular_matrix
##      [,1]     [,2]        [,3]
## [1,]   14 46.00000   4.0000000
## [2,]    0 21.57143  -0.4285714
## [3,]    0  0.00000 -15.6158940
test_4 <-  matrix(
    c(4, 52, 3, 67, 23, 5, 7, 8, 3, 6, 8, 1, 5, 6, 7, 12, 13, 45, 324, 6), 
    nrow=4, 
    ncol=4)
factorize(test_4)
## $lower_triangular_matrix
##      [,1] [,2]  [,3]  [,4]
## [1,]    4   23   3.0   5.0
## [2,]    0 -294 -33.0 -59.0
## [3,]    0    0   6.9   5.3
## [4,]    0    0   0.0   9.3