Problem Set 1

(1) Show that \(A^TA \ne AA^T\) in general. (Proof and demonstration.)

In matrix multiplication, \(AB \ne BA\) does not hold - is not commutative.

Lets first define our matrix A.

A = matrix(c(3,4,1,2), ncol=2, byrow=T)
A
##      [,1] [,2]
## [1,]    3    4
## [2,]    1    2

Define our \(A^T\)

AT = t(A)
AT
##      [,1] [,2]
## [1,]    3    1
## [2,]    4    2

If we times AT by A we get:

ATA = AT %*% A
ATA
##      [,1] [,2]
## [1,]   10   14
## [2,]   14   20

If we times A by AT:

AAT = A %*% AT
AAT
##      [,1] [,2]
## [1,]   25   11
## [2,]   11    5

As you can see, \(A^TA \ne AA^T\).

ATA == AT
##       [,1]  [,2]
## [1,] FALSE FALSE
## [2,] FALSE FALSE

(2) For a special type of square matrix A, we get AT A = AAT. Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

Under normal circumstances, \(A^TA \ne AA^T\). However, there are special cases when \(A^TA = AA^T\). For example, the identity matrix.

A = matrix(c(1,0,0,0,1,0,0,0,1), ncol=3, byrow=T)
A
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

Define our \(A^T\)

AT = t(A)
AT
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

If we times AT by A we get:

ATA = AT %*% A
ATA
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

If we times A by AT:

AAT = A %*% AT
AAT
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

As you can see, for this special case, \(A^TA = AA^T\).

ATA == AT
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE

Problem Set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars. Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer.

fact <- function(A, L, x) {
 
c <- ncol(A)
r <- nrow(A)

if(x >= c){
  return(list(L,x))
}

else {
  for (i in (x+1):r){
    mult <- (-A[i,x]/A[x,x])
    L[i,x] <- round(-mult,2)
    A[i,] <- A[i,] + mult * A[x, ]
  }
  print(A)
}
fact(A, L, x + 1)
}


A <- matrix(c(1, 4, -3, -2, 8, 5, 3, 4, 7), nrow=3, ncol=3, byrow = TRUE)
print(A)
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]   -2    8    5
## [3,]    3    4    7
matx1 <- fact(A, diag(nrow(A)), 1)
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]    0   16   -1
## [3,]    0   -8   16
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5

Refrences: https://www.khanacademy.org/math/precalculus/precalc-matrices/modal/a/properties-of-matrix-multiplication

https://www.youtube.com/watch?v=UlWcofkUDDU&feature=youtu.be