Problem Set 1

1. Show that \(AA^T \neq A^TA\) in general (proof and demonstration)

Proof For this we will make use of the case where is A is not square-matrix. That is: # Rows \(\neq\) # Columns

For this proof we will make use an argument of dimensionality. That is that the dimensions of AB \(\neq\) BA.

We know that the dimensions of the Matrix multiplication AB needs to meet the requirement that Columns_A = Rows_B. We also know that the dimensions of resulting product equals [Rows_A, Cols_B]

Following the above if dimensions of A are [a,b], under the condition that \(a \neq b\), then dimensions of \(A^T\) are [b,a]. Therefore dimensions of \(AA^T\) are [a,a], but dimensions of \(A^TA\) are [b,b]. Since we started by saying that \(a \neq b\) we have proven that \(AA^T \neq A^TA\)

Demonstration

\(A = \begin{bmatrix}1 & 2\\ 3 & 4\\ 5 & 6 \end{bmatrix}\)

\(A^T = \begin{bmatrix}1 & 3 & 5\\ 2 & 4 & 6\end{bmatrix}\)

\(AA^T = \begin{bmatrix}5&11&17 \\ 11 & 25 & 39\\ 17 & 39 & 61 \end{bmatrix}\)

\(A^TA = \begin{bmatrix}35&44 \\ 44 & 56 \end{bmatrix}\)

We see above that obviously the matrices are not equal, but for our proof, we see that the dimensions are not equal.

2. For a special type of square matrix A, we get \(A^TA = AA^T\) . Under what conditions could this be true?

Answer

The special case is any diagonal matrix. This is because if A = diagonal matrix, \(A^T = A\) thus \(A^TA = AA^T = AA\)

We can see it with an example

\(A= \begin{bmatrix}5&0&0 \\ 0 & 5 & 0\\ 0 & 0 & 5 \end{bmatrix}\)

\(A^TA = AA^T = \begin{bmatrix}25&0&0 \\ 0 & 25 & 0\\ 0 & 0 & 25 \end{bmatrix}\)

Problem Set 2

Write an R function to factorize a square matrix A into LU. You don’t have to worry about permuting rows of A and you can assume that A is less than 5x5, if you need to hard-code any variables in your code.

Comments
What took me the most time was to attempt something without using for-loops. I am sure there must be a way either using apply or doing some recursion but I gave up and ended up replicating exactly how I solved the problem by hand: by going cell by cell.

Here is the code:

# Matrix we will use for testing
A=matrix(c(1,4,-3,-2,8,5,3,4,7),3,3,byrow=TRUE)

lu.decomp <- function(M){
  rows <- dim(M)[1]
  cols <- dim(M)[2]
  
  if (rows != cols) {
    return(list(0,0))} #If is not square matrix return list of 0's
  else {
    L <- diag(x = 1, rows,cols) # L by definition has 1's in the diagonal
    U <- M # Upper will have its first row EQUAL to input matrix
    for (x in 1:(rows-1)){
      for (y in (x+1):rows){
          factor = U[y,x]/U[x,x] # Is the number used to eliminate
          L[y,x] = factor # We make the lower matrix that value
          U[y,] = U[y,]-factor*U[x,] # We use Gaussian elimination
      }
    }
    return(list("L"=L,"U"=U)) # We return our completed L and U matrices
  }
}

# we output A and the LU of A using our function
A

##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]   -2    8    5
## [3,]    3    4    7

S <- lu.decomp(A)
S

## $L
##      [,1] [,2] [,3]
## [1,]    1  0.0    0
## [2,]   -2  1.0    0
## [3,]    3 -0.5    1
## 
## $U
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5

Now let’s check it with the function lu.decomposition in the package matrixcalc

S2 <- lu.decomposition(A)
S2

## $L
##      [,1] [,2] [,3]
## [1,]    1  0.0    0
## [2,]   -2  1.0    0
## [3,]    3 -0.5    1
## 
## $U
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5

They both provide me the same values for L and U.