1. Problem Set 1, Part 1

  1. Show that \(A^T A \neq AA^T\) in general. (Proof and demonstration.)

  2. For a special type of square matrix A, we get \(A^T A = AA^T\) Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

library(tidyverse)
library(tinytex)

\[ A = \begin{bmatrix} 1 & 2 & 1 \\ 3 & 2 & 3 \\ \end{bmatrix} \]

A = matrix(c(1,2,1,3,2,3), nrow=2, byrow=TRUE)
A
##      [,1] [,2] [,3]
## [1,]    1    2    1
## [2,]    3    2    3
\(A^TA\) = \[\begin{bmatrix} 1 & 3 \\ 2 & 2 \\ 1 & 3 \end{bmatrix} \times\] \[\begin{bmatrix} 1 & 2 & 1 \\ 3 & 2 & 3 \end{bmatrix}\]

=

\[\begin{bmatrix} 10 & 8 & 10 \\8 & 8 & 8 \\10 & 8 & 10 \end{bmatrix}\] 
t(A) %*% A
##      [,1] [,2] [,3]
## [1,]   10    8   10
## [2,]    8    8    8
## [3,]   10    8   10
\(AA^T\) = \[\begin{bmatrix} 1 & 2 & 1 \\ 3 & 2 & 3 \\ \end{bmatrix} \times\] \[\begin{bmatrix} 1 & 3 \\ 2 & 2 \\ 1 & 3 \end{bmatrix}\]

=

\[\begin{bmatrix} 6 & 10 \\ 10 & 22 \\ \end{bmatrix}\] 
A %*% t(A)
##      [,1] [,2]
## [1,]    6   10
## [2,]   10   22

Therefore:

\(A^TA\) \[\begin{bmatrix} 10 & 8 & 10 \\ 8 & 8 & 8 \\ 10 & 8 & 10 \end{bmatrix} \neq\] \(AA^T\) \[\begin{bmatrix} 6 & 10 \\ 10 & 22 \\ \end{bmatrix}\]

Problem Set 1, Part 2

  1. For a special type of square matrix A, we get \(A^T A = AA^T\) Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

When square matrix A is symmetrical, \(A^T A = AA^T\):

Example:

\[ B = \begin{bmatrix} 1 & 2 & 2\\ 3 & 2 & 1\\ 4 & 2 & 1 \\ \end{bmatrix} \]

B = matrix(c(1,2,2,3,2,1,4,2,1), nrow=3, byrow=TRUE)
B
##      [,1] [,2] [,3]
## [1,]    1    2    2
## [2,]    3    2    1
## [3,]    4    2    1
\(C\) = \(B^TB\) = \[\begin{bmatrix} 1 & 3 & 4\\ 2 & 2 & 2\\ 2 & 2 & 1 \\ \end{bmatrix} \times\] \[\begin{bmatrix} 1 & 2 & 2\\ 3 & 2 & 1\\ 4 & 2 & 1 \\ \end{bmatrix}\]

=

\[\begin{bmatrix} 26 & 16 & 9 \\ 16 & 12 & 8 \\ 9 & 8 & 6 \end{bmatrix}\] 
C <- t(B) %*% B
C
##      [,1] [,2] [,3]
## [1,]   26   16    9
## [2,]   16   12    8
## [3,]    9    8    6
\(B \times B^T\) = \[\begin{bmatrix} 1 & 2 & 2\\ 3 & 2 & 1\\ 4 & 2 & 1 \\ \end{bmatrix} \times\] \[\begin{bmatrix} 1 & 3 & 4\\ 2 & 2 & 2\\ 2 & 2 & 1 \\ \end{bmatrix}\]

=

\[\begin{bmatrix} 9 & 9 & 10 \\ 9 & 14 & 17 \\ 10 & 17 & 21 \end{bmatrix}\] 
B %*% t(B)
##      [,1] [,2] [,3]
## [1,]    9    9   10
## [2,]    9   14   17
## [3,]   10   17   21

\(B^T \times B\)

\[\begin{bmatrix} 1 & 3 & 4\\ 2 & 2 & 2\\ 2 & 2 & 1 \\ \end{bmatrix} \times\] \[\begin{bmatrix} 1 & 2 & 2\\ 3 & 2 & 1\\ 4 & 2 & 1 \\ \end{bmatrix}\]

=

\[\begin{bmatrix} 26 & 16 & 9 \\ 16 & 12 & 8 \\ 9 & 8 & 6 \end{bmatrix}\] 
t(B) %*% B
##      [,1] [,2] [,3]
## [1,]   26   16    9
## [2,]   16   12    8
## [3,]    9    8    6

2. Problem Set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars. Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer. Please submit your response in an R Markdown document using our class naming convention, E.g. LFulton_Assignment2_PS2.png You don’t have to worry about permuting rows of A and you can assume that A is less than 5x5, if you need to hard-code any variables in your code. If you doing the entire assignment in R, then please submit only one markdown document for both the problems.

LU_square_factor <- function(A) {
  # Checking that A is a square matrix
  if (dim(A)[1]!=dim(A)[2]) {
    stop("Input matrix must be square")
  }
  
  U <- A
  n <- dim(A)[1]
  L <- diag(n)
  
  # If the dimension==1, then U=A and L=[1]
  if (n==1) {
    return(list(L,U))
  }
  
  # Looping through the rows and columns of the lower triangle and
  # determining the multiplier for each position, adding it to the L matrix
  for(a in 2:n) {
    for(b in 1:(a-1)) {
      # c = multiplier
      c <- -U[a,b] / U[b,b]
      U[a, ] <- c * U[b, ] + U[a, ]
      L[a,b] <- -c
    }
  }
  return(list(L,U))
}

Example Cases:

# Using 3x3 matrix
A <- matrix(c(1,2,3,6,3,1,2,2,4),nrow=3, byrow=TRUE)
LU <- LU_square_factor(A)
L<-LU[[1]]  
U<-LU[[2]]
A
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    6    3    1
## [3,]    2    2    4
L
##      [,1]      [,2] [,3]
## [1,]    1 0.0000000    0
## [2,]    6 1.0000000    0
## [3,]    2 0.2222222    1
U
##      [,1] [,2]       [,3]
## [1,]    1    2   3.000000
## [2,]    0   -9 -17.000000
## [3,]    0    0   1.777778
A==L%*%U
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE
# Using 2x2 matrix
A <- matrix(c(5,6,2,4), nrow=2, byrow=TRUE)
LU <- LU_square_factor(A)
L<-LU[[1]]  
U<-LU[[2]]
A
##      [,1] [,2]
## [1,]    5    6
## [2,]    2    4
L
##      [,1] [,2]
## [1,]  1.0    0
## [2,]  0.4    1
U
##      [,1] [,2]
## [1,]    5  6.0
## [2,]    0  1.6
A==L%*%U
##      [,1] [,2]
## [1,] TRUE TRUE
## [2,] TRUE TRUE
# Error Message since it is 1-dimensional
#A <- matrix(c(5,6,2), nrow=1, byrow=TRUE)
#LU <- LU_square_factor(A)
#L<-LU[[1]]  
#U<-LU[[2]]
#A

# Error in LU_square_factor(A) : Input matrix must be square
#2. stop("Input matrix must be square")
#1. LU_square_factor(A)