Assignment 2

Problem Set 1

1. Show that \(A^TA \neq AA^T\) in general. (Proof and demonstration).

The assigned text, A First Course in Linear Algebra, states that matrix multiplication is not commutative. Thus, the order of the operands affects the result of the equation.

Proof:

  1. \(A = \begin{bmatrix}a&b\\c&d\end{bmatrix} \\\) \(A^T = \begin{bmatrix}e&f\\g&h\end{bmatrix} \\\)

  2. \(A^TA = \begin{bmatrix}ea+fc&eb+fd\\ga+hc&gb+hd\end{bmatrix} \\\) \(AA^T = \begin{bmatrix}ae+bg&af+bh\\ce+dg&cf+dh\end{bmatrix}\)

  3. \(A^TA \neq AA^T\)

Demonstration:

# Define variables
A <- matrix(c(1, 2, 1, 2), ncol = 2)
t <- 2
At <- A^t
A %*% At
##      [,1] [,2]
## [1,]    5    5
## [2,]   10   10
# Multiply matricies
AtA <- At %*% A
AAt <- A %*% At

# Logic test for entire system
identical(AtA, AAt)
## [1] FALSE

The identical logical function that tests for exact equality between two objects. We can conclude that \(A^TA\) and \(AA^T\) are not identical. This can further be visualized in the output below:

AtA
##      [,1] [,2]
## [1,]    3    3
## [2,]   12   12
AAt
##      [,1] [,2]
## [1,]    5    5
## [2,]   10   10
AtA == AAt
##       [,1]  [,2]
## [1,] FALSE FALSE
## [2,] FALSE FALSE

2. For a special type of square matrix A, we get \(A^TA = AA^T\). Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

This equation can be true when a matrix is square and symmetric. A matrix is symmetric when the inverse of A equals A.

Proof:

\(A = A^T\) thus \(A^TA = AA^T\) equals \(AA = AA\).

Demonstration:

# Define variables
A <- matrix(c(1, 2, 2, 1), ncol = 2)
t <- 2
At <- A^t
 
# Multiply matricies
AtA <- At %*% A
AAt <- A %*% At

# Logic test 
identical(AtA, AAt)
## [1] TRUE
# Visualize output
AtA
##      [,1] [,2]
## [1,]    9    6
## [2,]    6    9
AAt
##      [,1] [,2]
## [1,]    9    6
## [2,]    6    9

Problem set 2

Matrix factorization is a very important problem. There are supercomputers built just to do matrix factorizations. Every second you are on an airplane, matrices are being factorized. Radars that track flights use a technique called Kalman filtering. At the heart of Kalman Filtering is a Matrix Factorization operation. Kalman Filters are solving linear systems of equations when they track your flight using radars.

Write an R function to factorize a square matrix A into LU or LDU, whichever you prefer.

You don’t have to worry about permuting rows of A and you can assume that A is less than 5x5, if you need to hard-code any variables in your code. If you doing the entire assignment in R, then please submit only one markdown document for both the problems.

Answer:

I utilized the matlib package and the function lu to decompose (factorize) the defined matrix A. The logic test shows that \(A = LU\) in this example.

#Define matrix
A <- matrix(c(1,8,-4,-2), nrow=2, byrow=TRUE)

#Decompose (factorize) matrix A
lu <-LU(A, verbose=T)
## 
## Initial equation:
##    x1 + 8*x2  =  b1 
## -4*x1 - 2*x2  =  b2 
## 
## Lower triangle equation:
##    x1       =  b1 
## -4*x1 + x2  =  b2 
## 
## Upper triangle equation:
## x1 + 8*x2  =  b1 
##     30*x2  =  b2
#Define variables
L <- lu$L
U <- lu$U

#Lower triangle matrix 
L
##      [,1] [,2]
## [1,]    1    0
## [2,]   -4    1
#Upper triangle matrix
U
##      [,1] [,2]
## [1,]    1    8
## [2,]    0   30
#Logical test
A == L %*% U
##      [,1] [,2]
## [1,] TRUE TRUE
## [2,] TRUE TRUE