DATA605 ASSIGNMENT #2

Problem Set1

1. Show that (A^T)A =/= (A*A^T) in general. (Proof and demonstration.)

Proof:

We know that the number of rows in a resulting multiplied matrix equals the number of rows of the first matrix, and the number of columns of the second matrix.

Alternatively, if the first matrix, matrix A, is a mxn matrix, and its transpose, A^T, is a nxm matrix, then A^TA will result as a nxn matrix. This is because the result will be the number of rows from its transpose, which is n, and the number of columns from matrix A, which is also n. A^TA = nxn matrix

If matrix A is a mxn matrix, and its transpose is an nxm matrix, then AA^T will result as a mxm matrix. This is because the result will be the number of rows from matrix A, which is m, and the number of columns from its transpose, which is also m. AA^T = mxm matrix

nxn =/= nxn and therefore, the two do not equal each other.

Demonstration:

Let’s say, for example, that this is matrix A:

A <- matrix(c(1, 2, 
              3, 4), nrow = 2, ncol = 2)
print(A)
##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4

We can calculate the transpose of A by swtiching its rows and columns like so:

AT <- t(A)
print(AT)
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4

We can then multiply the two to calculate A^T * A, or the transopse of A mulitplied by A, like so:

ATA <- t(A) %*% A

Now we can find A^T*A like so:

AAT <- A %*% t(A)

Here is the result for A^T * A:

print("A^T * A:")
## [1] "A^T * A:"
print(ATA)
##      [,1] [,2]
## [1,]    5   11
## [2,]   11   25

And A*A^T:

print("A * A^T:")
## [1] "A * A^T:"
print(AAT)
##      [,1] [,2]
## [1,]   10   14
## [2,]   14   20

As we can see here, ATA =/= AAT.

2.For a special type of square matrix A, we get (A^T)A = A*A^T . Under what conditions could this be true?

This could be true when both the matrix and its transpose are equal to the same number, so that when multiplied, they also equal the product of the transpose and its matrix. This is also called a symmetrical matrix. The two products result in the matrix being squared, making them both equal to one another.

For example:

Let’s say we have matrix A:

A <- matrix(c(1,1,-1, 
              1,2,0,
              -1,0,5), nrow = 3, ncol = 3)
print(A)
##      [,1] [,2] [,3]
## [1,]    1    1   -1
## [2,]    1    2    0
## [3,]   -1    0    5

And its transpose:

AT <- t(A)
print(AT)
##      [,1] [,2] [,3]
## [1,]    1    1   -1
## [2,]    1    2    0
## [3,]   -1    0    5

We can now calculate A^T * A and its product:

print("A^T * A:")
## [1] "A^T * A:"
print(ATA)
##      [,1] [,2]
## [1,]    5   11
## [2,]   11   25
det(ATA)
## [1] 4

And A * A^T and its product:

print("A * A^T:")
## [1] "A * A^T:"
print(AAT)
##      [,1] [,2]
## [1,]   10   14
## [2,]   14   20
det(AAT)
## [1] 4

In this example, these matrices are both equal to 4.

Problem Set2

1. Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars. Write an R function to factorize a square matrix A into LU or LDU.

Factorizing a square matrix A into LU:

LU <- function(A) {
  n <- nrow(A)
  L <- matrix(0, n, n) #lower
  U <- matrix(0, n, n) #upper
  
  for (i in 1:n) {  # loops through each row
    L[i, i] <- 1 
    
    for (j in i:n) {  #loops for Gaussian elimination, upper and diagonal
      U[i, j] <- A[i, j] - sum(L[i, 1:(i-1)] * U[1:(i-1), j])
      
      if (i < n) { 
        L[(i+1):n, i] <- (A[(i+1):n, i] - L[(i+1):n, 1:i-1] %*% U[1:i-1, i]) / U[i, i]
      }
    }
  }
  
  D <- diag(U)
  U <- U / diag(U)
  
  return(list(L = L, D = diag(D), U = U))
}

Here is a square matrix to factorize into LU:

A <- matrix(c(1, 2, 
              3, 4), nrow = 2, ncol = 2)
print(A)
##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4

LU decomposition allows us to find the lower matrix and the upper matrix, whose product will equal the original square matrix. In this case we have our matrix A:

LU_A<-LU(A)
L <- LU_A$L
U <- LU_A$U
cat("Matrix A:\n")
## Matrix A:
print(A)
##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4

We also have matrix L, which is the lower matrix:

cat("Matrix L:\n")
## Matrix L:
print(L)
##      [,1] [,2]
## [1,]    1    0
## [2,]    2    1

And we also have matrix U, which is the upper matrix:

cat("Matrix U:\n")
## Matrix U:
print(U)
##      [,1] [,2]
## [1,]    1    3
## [2,]    0    1