Problem Set 1

Part 1: Proof & Demonstration

Show that \[A^TA \neq AA^T \]in general. (Proof and demonstration.)

Proof:

Given a 2x2 matrix, \(A\):

\[ A = \begin{bmatrix} a11 & a12 \\ a21 & a22 \\ \end{bmatrix} \]

For any 2x2 matrix \(A\), the transpose of \(A\) is

\[ A^T = \begin{bmatrix} a11 & a21 \\ a12 & a22 \\ \end{bmatrix} \]

For any 2x2 matrix \(A\) and its transpose, \(A^T\), \(A^TA\) is equal to:

\[ A^TA = \begin{bmatrix} a11 & a21 \\ a12 & a22 \\ \end{bmatrix} \cdot \begin{bmatrix} a11 & a12 \\ a21 & a22 \\ \end{bmatrix} = \begin{bmatrix} a11 \cdot a11 + a21 \cdot a21 & a11 \cdot a12 + a21 \cdot a22\\ a12 \cdot a11 + a22 \cdot a21 & a12 \cdot a12 + a22 \cdot a22\\ \end{bmatrix} = A^TA \]

For any 2x2 matrix \(A\) and its transpose \(A^T\), \(AA^T\) is equal to:

\[ AA^T = \begin{bmatrix} a11 & a12 \\ a21 & a22 \\ \end{bmatrix} \cdot \begin{bmatrix} a11 & a21 \\ a12 & a22 \\ \end{bmatrix} = \begin{bmatrix} a11 \cdot a11 + a12 \cdot a12 & a11 \cdot a21 + a12 \cdot a22 \\ a21 \cdot a11 + a22 \cdot a12 & a22 \cdot a21 + a22 \cdot a22\\ \end{bmatrix} = AA^T \]

We can see from above that \[AA^T \neq A^TA\]. Hence, for any 2x2 matrix \(A\), in general \[AA^T \neq A^TA\]. A similar result occurs with a 3x3 matrix, which will be shown in the following demonstration.

Demonstration:

Let’s consider a 3x3 matrix \(A\):

\[ A = \begin{bmatrix} 3 & 1 & 8\\ 2 & 3 & 3\\ 5 & 7 & 6\\ \end{bmatrix} \]

First, we will use this matrix \(A\) to solve the left side of the above equation.

A <- matrix(c(3,2,5,1,3,7,8,3,6), nrow = 3, ncol = 3)
A
##      [,1] [,2] [,3]
## [1,]    3    1    8
## [2,]    2    3    3
## [3,]    5    7    6

Now, let’s find the transpose of \(A\):

A_T <- t(A)
A_T
##      [,1] [,2] [,3]
## [1,]    3    2    5
## [2,]    1    3    7
## [3,]    8    3    6

Now, we’ll determine \(AA^T\).

A_TA <- A_T %*% A
A_TA
##      [,1] [,2] [,3]
## [1,]   38   44   60
## [2,]   44   59   59
## [3,]   60   59  109

We’ve found the left side of the equation using matrix \(A\), so now we will find the right side.

AA_T <- A %*% A_T
AA_T
##      [,1] [,2] [,3]
## [1,]   74   33   70
## [2,]   33   22   49
## [3,]   70   49  110

Now that we’ve bound both sides of the equation, we can see that the two sides are not equal.

A_TA == AA_T
##       [,1]  [,2]  [,3]
## [1,] FALSE FALSE FALSE
## [2,] FALSE FALSE FALSE
## [3,] FALSE FALSE FALSE

Part 2: Special Square Matrix

For a special type of square matrix \(A\), we get \(A^TA=AA^T\) . Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

Solution:

This can only be true under the condition that \(A\) and \(A^T\) are symmetrical. For example:

\[ A= \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} , A^T = \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} \\ AA^T = \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} \cdot \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 + 1 \cdot 1 & 1 \cdot 1 + 1 \cdot 1\\ 1 \cdot 1 + 1 \cdot 1 & 1 \cdot 1 + 1 \cdot 1\\ \end{bmatrix} \\ A^TA = \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} \cdot \begin{bmatrix} 1 & 1\\ 1 & 1\\ \end{bmatrix} = \begin{bmatrix} 1 \cdot 1 + 1 \cdot 1 & 1 \cdot 1 + 1 \cdot 1\\ 1 \cdot 1 + 1 \cdot 1 & 1 \cdot 1 + 1 \cdot 1\\ \end{bmatrix} \]

Hence, \[AA^T = A^TA\].

Problem Set 2

Write an R function to factorize a square matrix \(A\) into \(LU\) or \(LDU\), whichever you prefer.

LU Decomposition Function

# LU Decomposition Function
LU <- function(x) {
  # check to make sure matrix is square
  if (ncol(x) != nrow(x)) {
    stop("Input matrix must be square.")
  }
  
  # number of rows for the matrices
  n <- nrow(x)
  # identity matrix
  L <- diag(n)
  # initialize U
  U <- x
  
  for (j in 1:(n-1)){
    for (i in (j+1):n){
      # if pivot = 0, move on
      if(U[i,j]==0){
        next
      }
      else{
        # find multiplier
        m <- -(U[i,j] / U[j,j])
        # Gaussian Elimination
        U[i, ] <- m * U[j, ] + U[i, ]
        # add multiplier to L
        L[i, j] <- -m
      }
    }
  }
  # print matrix L
  cat("Matrix L:\n")
  print(L)
  # print matrix U
  cat("\nMatrix U:\n")
  print(U)
}

Testing Function

# provide a matrix A
A = matrix(c(3,6,2,9), nrow = 2, ncol = 2)
print(A)
##      [,1] [,2]
## [1,]    3    2
## [2,]    6    9
LU(A)
## Matrix L:
##      [,1] [,2]
## [1,]    1    0
## [2,]    2    1
## 
## Matrix U:
##      [,1] [,2]
## [1,]    3    2
## [2,]    0    5