Problem Set 1

  1. Show that \(A^TA \neq AA^T\) in general. (Proof and demonstration.)

  2. For a special type of square matrix \(A\), we get \(A^TA = AA^T\). Under what conditions could this be true? (Hint: The Identity matrix I is an example of such a matrix).

Solution to Set 1

Proof

Now to prove \(A^TA \neq AA^T\) in general for a square matrix.

A 2x2 square matrix for \(A\): \[ A = \begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} \] With a transpose as \[ A^T = \begin{bmatrix} a & c \\ b & d \\ \end{bmatrix} \]

Matrix multiply \(AA^T\)

\[ AA^T = \begin{bmatrix} a^2+b^2 & ac+bd \\ ac+bd & c^2+d^2 \\ \end{bmatrix} \]

Matrix multiply \(A^TA\)

\[ A^TA = \begin{bmatrix} a^2+c^2 & ab+cd \\ ab+cd & b^2+d^2 \\ \end{bmatrix} \]

Thus

\[ \begin{bmatrix} a^2+b^2 & ac+bd \\ ac+bd & c^2+d^2 \\ \end{bmatrix} \neq \begin{bmatrix} a^2+c^2 & ab+cd \\ ab+cd & b^2+d^2 \\ \end{bmatrix} \]

Demonstration with R
(A = matrix(c(1,2,3,4), 2, 2))
##      [,1] [,2]
## [1,]    1    3
## [2,]    2    4
# transpose matrix A
(tr_A = t(A))
##      [,1] [,2]
## [1,]    1    2
## [2,]    3    4
# multiplication of transpose (A) by matrix A
(AtA <- tr_A %*% A)
##      [,1] [,2]
## [1,]    5   11
## [2,]   11   25
# multiplication of matrix A by transpose (A)
(AAt <- A %*% tr_A)
##      [,1] [,2]
## [1,]   10   14
## [2,]   14   20
# validation check for symmetry
AtA == AAt
##       [,1]  [,2]
## [1,] FALSE FALSE
## [2,] FALSE FALSE

Part 2

Under what conditions is \(AA^T = A^TA\)?

Proof

When matrix \(A\) is symmetric, \(A = A^T\). And then, with substitution \(AA = AA\).

\[ A = \begin{bmatrix} 4 & 8 \\ 8 & 4 \\ \end{bmatrix} \] With a transpose as \[ A^T = \begin{bmatrix} 4 & 8 \\ 8 & 4 \\ \end{bmatrix} \]

Demonstration with R
(A = matrix(c(4, 8, 8, 4), nrow = 2))
##      [,1] [,2]
## [1,]    4    8
## [2,]    8    4
# transpose matrix A
(tr_A = t(A))
##      [,1] [,2]
## [1,]    4    8
## [2,]    8    4
# multiplication of transpose A by matrix A
(AtA <- tr_A %*% A)
##      [,1] [,2]
## [1,]   80   64
## [2,]   64   80
# multiplication of matrix A by transpose A
(AAt <- A %*% tr_A)
##      [,1] [,2]
## [1,]   80   64
## [2,]   64   80
# validation check for symmetry
AtA == AAt
##      [,1] [,2]
## [1,] TRUE TRUE
## [2,] TRUE TRUE

Problem Set 2

Write an R function to factorize a square matrix \(A\) into LU or LDU, whichever you prefer. Please submit your response in an R Markdown document.

You don’t have to worry about permuting rows of \(A\) and you can assume that \(A\) is less than \(5 \times 5\), if you need to hard-code any variables in your code.

Solution

Function lu in R is computing A = PLU, which is equivalent to computing the LU decomposition of matrix A with its rows permuted by the permutation matrix P-1: P-1A = LU.

lu function in R using row pivot based on Stackoverflow website

library(Matrix)
B <- matrix(c(1, 1, 1, 1, 1, 1, -1, -1, 1, -1, -1, 1, 1, -1, 1, -1), 4)
B
##      [,1] [,2] [,3] [,4]
## [1,]    1    1    1    1
## [2,]    1    1   -1   -1
## [3,]    1   -1   -1    1
## [4,]    1   -1    1   -1
rr <- expand(lu(B))
rr
## $L
## 4 x 4 Matrix of class "dtrMatrix" (unitriangular)
##      [,1] [,2] [,3] [,4]
## [1,]    1    .    .    .
## [2,]    1    1    .    .
## [3,]    1    0    1    .
## [4,]    1    1   -1    1
## 
## $U
## 4 x 4 Matrix of class "dtrMatrix"
##      [,1] [,2] [,3] [,4]
## [1,]    1    1    1    1
## [2,]    .   -2   -2    0
## [3,]    .    .   -2   -2
## [4,]    .    .    .   -4
## 
## $P
## 4 x 4 sparse Matrix of class "pMatrix"
##             
## [1,] | . . .
## [2,] . . | .
## [3,] . | . .
## [4,] . . . |

Random matrix with LU factorization

A random n*m matrix with the lu() function based on R Studio Community - LU factorization

set.seed(24)
p = matrix(data = sample(1:16), nrow = 4, ncol = 4)
rr <- expand(lu(p))
rr
## $L
## 4 x 4 Matrix of class "dtrMatrix" (unitriangular)
##      [,1]       [,2]       [,3]       [,4]      
## [1,]  1.0000000          .          .          .
## [2,]  0.1875000  1.0000000          .          .
## [3,]  0.4375000 -0.3333333  1.0000000          .
## [4,]  0.5000000  0.2424242 -0.4659091  1.0000000
## 
## $U
## 4 x 4 Matrix of class "dtrMatrix"
##      [,1]     [,2]     [,3]     [,4]    
## [1,] 16.00000 14.00000  4.00000  5.00000
## [2,]        . 12.37500 11.25000 10.06250
## [3,]        .        .  8.00000 10.16667
## [4,]        .        .        . 12.79735
## 
## $P
## 4 x 4 sparse Matrix of class "pMatrix"
##             
## [1,] . . | .
## [2,] . | . .
## [3,] . . . |
## [4,] | . . .
# Loop thorugh i'th row and j'th column of p
for( i in 1:nrow(p) ){
  for( j in 1:ncol(p) ){
    # This doesn't do anything, but here you can think about how to check
    # where in the matrix you are by checking the relative values of i and j
    p[i,j] = p[i,j]
  }
}

Source: RPubs