(1) Show that \(A^TA \neq AA^T\) in general. (Proof and demonstration.)

Let Matrix A be \[ \mathbf{A} = \left[\begin{array} {rrr} a & r\\ b & s \end{array}\right]\]

and

\[\mathbf{A^T} = \left[\begin{array} {rrr} a & b\\ r & s\end{array}\right]\] Proof:

\[\mathbf{A^TA} = \left[\begin{array} {rrr} a \times a + b \times b & a \times r + b \times s\\ r \times a + s \times b & r \times r + s \times s \end{array}\right]\]

\[\mathbf{A^TA} = \left[\begin{array} {rrr} a^2 + b^2 & ar + bs\\ ra + sb & r^2 + s^2 \end{array}\right]\]

\[\mathbf{AA^T} = \left[\begin{array} {rrr} a \times a + r \times r & a \times b + r \times s\\ b \times a + s \times r & b \times b + s \times s \end{array}\right]\]

\[\mathbf{AA^T} = \left[\begin{array} {rrr} a^2 + r^2 & ab + rs\\ ab + sr & b^2 + s^2 \end{array}\right]\]

Therefore \(A^TA \neq AA^T\).

For example:

A

A = matrix(c(1, -2, 4, -3), 2, 2, byrow = T)
A
##      [,1] [,2]
## [1,]    1   -2
## [2,]    4   -3

A Transpose

TA = t(A)
TA
##      [,1] [,2]
## [1,]    1    4
## [2,]   -2   -3

A multiplied by A Transpose

A %*% TA
##      [,1] [,2]
## [1,]    5   10
## [2,]   10   25

A Transpose multiplied by A

TA %*% A
##      [,1] [,2]
## [1,]   17  -14
## [2,]  -14   13

(2) For a special type of square matrix A, we get \(A^TA = AA^T\) . Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

Conditions:

A matrix is considered symmetric if it is equal to it’s transpose. If \(A\) is symmetric then \(A = A^T\) so \(A^TA\) should be the same as \(AA^T\)

Proof using rhs

\[AA^T = (AA^T)^T\\\]

\[ = A^T(A^T){^T}\\\]

\[ = A^TA\]

A

A <- matrix(c(3, 2, 4, 2, 6, 2, 4, 2 ,3), 3, 3, byrow = T)
A
##      [,1] [,2] [,3]
## [1,]    3    2    4
## [2,]    2    6    2
## [3,]    4    2    3

A Transpose

TA <- t(A)
TA
##      [,1] [,2] [,3]
## [1,]    3    2    4
## [2,]    2    6    2
## [3,]    4    2    3

A multiplied by A Transpose -> \(AA^T\)

A %*% TA
##      [,1] [,2] [,3]
## [1,]   29   26   28
## [2,]   26   44   26
## [3,]   28   26   29

A Transpose multiplied by A -> \(A^TA\)

TA %*% A
##      [,1] [,2] [,3]
## [1,]   29   26   28
## [2,]   26   44   26
## [3,]   28   26   29

Transpose of \(AA^T\) rhs

t(A %*% TA)
##      [,1] [,2] [,3]
## [1,]   29   26   28
## [2,]   26   44   26
## [3,]   28   26   29

Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer.

LU_D <- function(M){

  m <- dim(M)
  if (m[1] != m[2]){
    print("This is not a square matrix!")
  }
  
  U <- matrix( 0, nrow = nrow(M), ncol=nrow(M) )
  L <- diag(3)
  for(i in 1 : nrow(M)) {
    p1 <- i+1
    m1 <- i-1
    for (j in 1:nrow(M)){
      U[i, j] <- M[i,j]
        if ( m1 > 0 ) {
          for ( k in 1:m1 ) {
            U[i,j] <- U[i,j] - L[i,k] * U[k,j]
          }
        }
      }
    
  if ( p1 <= nrow(M) ) {
    for ( j in p1:nrow(M) ) {
      L[j,i] <- M[j,i]
      if ( m1 > 0 ) {
        for ( k in 1:m1 ) {
          L[j,i] <- L[j,i] - L[j,k] * U[k,i]
        }
      }
      L[j,i] <- L[j,i] / U[i,i]
    }
  }
  }
  
  output <- list(L=L, U=U)
  return(output)
}

Test function

D <- matrix(c(1, 4, -3, -2, 8, 5, 3, 4, 7), 3, 3, byrow = T)
E <- matrix(c(2, 4, -4, 1, -4, 3, -6, -9, 5), 3, 3, byrow = T)

LU_D(D)
## $L
##      [,1] [,2] [,3]
## [1,]    1  0.0    0
## [2,]   -2  1.0    0
## [3,]    3 -0.5    1
## 
## $U
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5
LU_D(E)
## $L
##      [,1] [,2] [,3]
## [1,]  1.0  0.0    0
## [2,]  0.5  1.0    0
## [3,] -3.0 -0.5    1
## 
## $U
##      [,1] [,2] [,3]
## [1,]    2    4 -4.0
## [2,]    0   -6  5.0
## [3,]    0    0 -4.5

Helpful Links:

+LU Decomposition - Shortcut Method

+Trackable R function for LU factorization

+The LU Decomposition of a Matrix Examples 1