1. Problem set 1

  1. Show that \(A^TA \neq AA^T\) in general. (Proof and demonstration.)
  2. For a special type of square matrix A, we get \(A^TA = AA^T\). Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix). Please typeset your response using LaTeX mode in RStudio. If you do it in paper, please either scan or take a picture of the work and submit it. Please ensure that your image is legible and that your submissions are named using your first initial, lastname, assignment and problem set within the assignment. E.g. LFulton_Assignment2_PS1.png
1 - Let,

\[ A = \left(\begin{array}{cc} a & b\\ c & d \end{array}\right) \] and, \[ A^T = \left(\begin{array}{cc} a & c\\ b & d \end{array}\right) \]

\[ AA^T = \left(\begin{array}{cc} a & b\\ c & d \end{array}\right) \left(\begin{array}{cc} a & c\\ b & d \end{array}\right) = \left(\begin{array}{cc} a^2+b^2 & ac+bd\\ ca+db & c^2+d^2 \end{array}\right) \]

\[ A^TA = \left(\begin{array}{cc} a & c\\ b & d \end{array}\right) \left(\begin{array}{cc} a & b\\ c & d \end{array}\right) = \left(\begin{array}{cc} a^2+b^2 & ab+cd\\ ba+dc & b^2+d^2 \end{array}\right) \] Therefore, \(A^TA \neq AA^T\).

  • e.g.:

\[ A = \left(\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right) \] and, \[ A^T = \left(\begin{array}{cc} 1 & 3\\ 2 & 4 \end{array}\right) \]

\[ AA^T = \left(\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right) \left(\begin{array}{cc} 1 & 3\\ 2 & 4 \end{array}\right) = \left(\begin{array}{cc} 1+4 & 3+8\\ 3+8 & 9+16 \end{array}\right) = \left(\begin{array}{cc} 5 & 11\\ 11 & 16 \end{array}\right) \]

\[ A^TA = \left(\begin{array}{cc} 1 & 3\\ 2 & 4 \end{array}\right) \left(\begin{array}{cc} 1 & 2\\ 3 & 4 \end{array}\right) = \left(\begin{array}{cc} 1+9 & 2+12\\ 2+12 & 4+16 \end{array}\right) = \left(\begin{array}{cc} 10 & 14\\ 14 & 20 \end{array}\right) \]

A <- matrix(c(1, 2, 3, 4), nrow = 2, ncol = 2, byrow = T)
AT <- t(A)

AAT<-A %*% AT
ATA<-AT %*% A

A
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4
AT
##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4
AAT
##      [,1] [,2]
## [1,]    5   11
## [2,]   11   25
ATA
##      [,1] [,2]
## [1,]   10   14
## [2,]   14   20
2 - A square matrix is a matrix with the same number of rows and columns. An n-by-n matrix is known as a square matrix of order n. With The Identity matrix I we can get \(A^TA = AA^T\).

Let, \[ A = \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) \] and, \[ A^T = \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) \]

\[ AA^T = \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) = \left(\begin{array}{cc} a^2+0 & 0+0\\ 0+0 & 0+d^2 \end{array}\right) = \left(\begin{array}{cc} a^2 & 0\\ 0 & d^2 \end{array}\right) \]

\[ A^TA = \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) \left(\begin{array}{cc} a & 0\\ 0 & d \end{array}\right) = \left(\begin{array}{cc} a^2+0 & 0+0\\ 0+0 & 0+d^2 \end{array}\right) = \left(\begin{array}{cc} a^2 & 0\\ 0 & d^2 \end{array}\right) \] Therefore, \(A^TA = AA^T\).

  • e.g.:

\[ A = \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) \] and, \[ A^T = \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) \]

\[ AA^T = \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) = \left(\begin{array}{cc} 1+0 & 0+0\\ 0+0 & 0+16 \end{array}\right) = \left(\begin{array}{cc} 1 & 0\\ 0 & 16 \end{array}\right) \]

\[ A^TA = \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) \left(\begin{array}{cc} 1 & 0\\ 0 & 4 \end{array}\right) = \left(\begin{array}{cc} 1+0 & 0+0\\ 0+0 & 0+16 \end{array}\right) = \left(\begin{array}{cc} 1 & 0\\ 0 & 16 \end{array}\right) \]

A <- matrix(c(1, 0, 0, 4), nrow = 2, ncol = 2, byrow = T)
AT <- t(A)

AAT<-A %*% AT
ATA<-AT %*% A

A
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    4
AT
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    4
AAT
##      [,1] [,2]
## [1,]    1    0
## [2,]    0   16
ATA
##      [,1] [,2]
## [1,]    1    0
## [2,]    0   16

2. Problem set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars.
Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer. Please submit your response in an R Markdown document using our class naming convention, E.g. LFulton_Assignment2_PS2.png
You don’t have to worry about permuting rows of A and you can assume that A is less than 5x5, if you need to hard-code any variables in your code. If you doing the entire assignment in R, then please submit only one markdown document for both the problems.

The goal is to factorize a square matrix A \(m \times m\) into LU: square matrix A = LU

A <- matrix(c(1,2,3,2,1,3,3,2,1), nrow = 3, ncol = 3, byrow = T)

factorization <- function(A){
  if(nrow(A) != ncol(A)){
    print("Have a square matrix A $m \times m$")
    stop("Matrix A should have the number of columns = the number of rows")
  } else {
    rowA <- nrow(A)
    L <- diag(rowA)
    U <- A
    
    for (i in 1:(rowA-1)) {
      for (j in (i+1):rowA) {
        L[j, i] <- U[j, i]/U[i, i]
        U[j, ] <- U[j, ]-L[j, i]*U[i, ]
      }
    }
    LU <- list("L"=L, "U"=U)
  }
}

A
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    2    1    3
## [3,]    3    2    1
LU <- factorization(A)
LU$L
##      [,1]     [,2] [,3]
## [1,]    1 0.000000    0
## [2,]    2 1.000000    0
## [3,]    3 1.333333    1
LU$U
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    0   -3   -3
## [3,]    0    0   -4
L_U_multiplication <- factorization(A)$L %*% factorization(A)$U
L_U_multiplication
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    2    1    3
## [3,]    3    2    1
cat("L_U_multiplication == A \n", L_U_multiplication == A)
## L_U_multiplication == A 
##  TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE