Problem Set 1

(1) Show that \(A^{T} A \ne AA^{T}\) in general. (Proof and demonstration.)

If we take matrix \(A\) to be of the type \(m \times n\), then \(A^{T}\) will be of the type \(n \times m\) as by definition, \(A^{T}\) is a flipped version of \(A\) (i.e row 1 of \(A\) becomes column 1 of \(A^{T}\), row 2 of \(A\) becomes column 2 of \(A^{T}\), etc.).

Multiplying \(A^{T}\) (\(n \times m\)) by \(A\) (\(m \times n\)) will result in a square \(n \times n\) matrix. If we reverse the multiplication so that it is \(A\) (\(m \times n\)) by \(A^{T}\) (\(n \times m\)), then we will get an \(m \times m\) matrix.

Therefore, as per the above logic, if \(m \ne n\), the result of \(A^{T} A\) (\(n \times n\)) will always be different to that of \(AA^{T}\) (\(m \times m\)). The resulting matrices will not be equal and so we can assert that \(A^{T} A \ne AA^{T}\).

This is best demonstrated through the use of an example:

# Matrix A.
A <- matrix(c(1, 2, 3, 4, 5, 6, 7, 8), nrow = 2, byrow = TRUE)

# Matrix A transposed.
At <- t(A)

# Display Matrix A.
A
##      [,1] [,2] [,3] [,4]
## [1,]    1    2    3    4
## [2,]    5    6    7    8
# Display Matrix A transposed.
At
##      [,1] [,2]
## [1,]    1    5
## [2,]    2    6
## [3,]    3    7
## [4,]    4    8
# Multiple A x At. This results in a 2 X 2 matrix.
A %*% At
##      [,1] [,2]
## [1,]   30   70
## [2,]   70  174
# Multiple At x A. This results in a 4 x 4 matrix.
At %*% A
##      [,1] [,2] [,3] [,4]
## [1,]   26   32   38   44
## [2,]   32   40   48   56
## [3,]   38   48   58   68
## [4,]   44   56   68   80

Answer: As per the above example, multiplying \(A \times A^{T}\) results in a 2 x 2 matrix, whereas \(A^{T} \times A\) results in a 4 x 4 matrix and thus \(A^{T} A \ne AA^{T}\).

(2) For a special type of square matrix \(A\), we get \(A^{T} A = AA^{T}\). Under what conditions could this be true?

Answer: This is true for symmetric matrices where \(A = A^{T}\).

Again, this is best demonstrated by an example:

# Define Matrix A, and Matrix At.
A <- matrix(c(1, 2, 0, 2, 1, 0, 0, 0, 5), nrow = 3, byrow = TRUE)
At = t(A)

# Display Matrix A.
A
##      [,1] [,2] [,3]
## [1,]    1    2    0
## [2,]    2    1    0
## [3,]    0    0    5
# Display Matrix At.
At
##      [,1] [,2] [,3]
## [1,]    1    2    0
## [2,]    2    1    0
## [3,]    0    0    5
# Confirm that Matrix A is equal to Matrix At.
A == At
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE
# Confirm that AtA = AAt.
(A %*% At) == (At %*% A)
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE

Problem Set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track fights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars.

Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer.

The following function factorizes square matrix A into LU.

# Function that factorizes A into LU.
a_to_lu_factorization <- function(A) {
  # Check that passed matrix 'A' is squared. If not, print an error message and return 'NA'.
  if (dim(A)[1] != dim(A)[2]) {
    print('Only squared matrices are valid. Please ensure you are passing the correct matrix type.')

    return(NA)
  }
  else {
    # Otherwise, continue with calculations and return the expected result.
    L <- matrix(c(rep(0, nrow(A) * nrow(A))), nrow = nrow(A))

    # This is not a neccessary variable assignment, but for the sake of readability,
    # assign the value of passed parameter 'A', to a new variable 'U'.
    U <- A

    for (column_n in 1:(nrow(U))) {
      if (column_n <= nrow(U)) {
        L[column_n:nrow(U), column_n] <- U[column_n:nrow(U), column_n] / U[column_n, column_n]
      }

      if (column_n < nrow(U)) {
        for(k in column_n:(nrow(U) -1)) {
          k <- k + 1

          # Create 0 row values.
          U[k, ] <- U[k, ] + (U[column_n, ] * (-1 * U[k, column_n] / U[column_n, column_n])) 
        }
      }
    }

    # Return list of results.
    return(list(L, U))
  }
}
# Define a sample matrix to pass into our function.
sample_matrix_a <- matrix(c(-3, 1, 2, 6, -5, 2, 9, 5, -6), nrow = 3, byrow = TRUE)

# Run the function.
LU <- a_to_lu_factorization(sample_matrix_a)

# Display the output of the function.
LU
## [[1]]
##      [,1]      [,2] [,3]
## [1,]    1  0.000000    0
## [2,]   -2  1.000000    0
## [3,]   -3 -2.666667    1
## 
## [[2]]
##      [,1] [,2] [,3]
## [1,]   -3    1    2
## [2,]    0   -3    6
## [3,]    0    0   16