Problem Set 1 (A)

Show that \(A^{T}A\)\(\neq\)\(AA^{T}\) in general. (Proof and demonstration.)

General

\(A=\left[ \begin{array}{c} a & b & c \\ d & e & f \end{array} \right]\)

\(A^{T}=\left[ \begin{array}{c} a & d \\ b & e \\ c & f \end{array} \right]\)

\(A^{T}A=\left[ \begin{array}{c} a^{2}+d^{2} & ab+de & ac+df\\ ab+ed & b^{2}+e^{2} & bc+ef \\ ac+df & bc+ef & c^{2}+f^{2} \end{array} \right]\)

\(AA^{T}\left[ \begin{array}{c} a^{2}+b^{2}+c^{2} & ad+be+cf \\ ad+be+cf & d^{2}+e^{2}+f^{2} \end{array} \right]\)

Demo

cat("Matrix A","\n","\n")
## Matrix A 
## 
(A = matrix(c(6,4,24,1,-9,8),
           nrow = 2, byrow = TRUE))
##      [,1] [,2] [,3]
## [1,]    6    4   24
## [2,]    1   -9    8

Next we will transpose the matrix using t{base} and assign it to the variable A_trans

cat("Matrix A Transposed","\n","\n")
## Matrix A Transposed 
## 
(A_trans <- t(A))
##      [,1] [,2]
## [1,]    6    1
## [2,]    4   -9
## [3,]   24    8

As you can see the original 3x2 matrix is converted into a 2x3 matrix when transposed.

Next we will multiply matrix A by its transposed matrix and vice versa.

#transposed A multiplied by A
t(A) %*% A
##      [,1] [,2] [,3]
## [1,]   37   15  152
## [2,]   15   97   24
## [3,]  152   24  640
#A multiplied by transposed A
A %*% t(A)
##      [,1] [,2]
## [1,]  628  162
## [2,]  162  146

Lastly, using identical{base} we will run our last test using the R’s test for exact equality.

identical(t(A) %*% A, A %*% t(A))
## [1] FALSE

As we proved in the previous mutiplication of matrices samples \(A^{T}A\)\(\neq\)\(AA^{T}\) is true.

Problem Set 1 (B)

For a special type of square matrix \(A\), we get \(A^{T}A=AA^{T}\). Under what conditions could this be true? (Hint: The Identity matrix \(I\) is an example of such a matrix).

An Identity Matrix is a square and it has 1s on the diagonal and 0’s everywhere else. It can be calculated using the following formula \(A.A^{-1} = A^{-1}.A=I\). It is important to note that it is not guaranteed that each matrix will have an inverse.

First we will create a 2x2 matrix and then use solve{base} to obtain the inverse of the matrix.

cat("2x2 Matrix A","\n","\n")
## 2x2 Matrix A 
## 
(A <- matrix(c(1,9,9,8),ncol=2))
##      [,1] [,2]
## [1,]    1    9
## [2,]    9    8
identical(t(A) %*% A, A %*% t(A))
## [1] TRUE
cat("\n","Matrix A-1","\n","\n")
## 
##  Matrix A-1 
## 
solve(A)
##            [,1]        [,2]
## [1,] -0.1095890  0.12328767
## [2,]  0.1232877 -0.01369863

Next we will implement \(A.A^{-1} = A^{-1}.A=I\) and create a varible entitled I which will reflect the Identity Matrix

cat("2x2 Matrix A Inverse","\n","\n")
## 2x2 Matrix A Inverse 
## 
(A_inv <- matrix(c(-0.1095890, 0.12328767, 0.1232877, -0.01369863),nrow = 2, byrow = TRUE))
##            [,1]        [,2]
## [1,] -0.1095890  0.12328767
## [2,]  0.1232877 -0.01369863
cat("\n","Identity Matrix","\n","\n")
## 
##  Identity Matrix 
## 
(I <- round(A %*% A_inv)) #use the round function to remove extraneous digtis
##      [,1] [,2]
## [1,]    1    0
## [2,]    0    1

Lastly, we will test to see if \(A^{T}A = AA^{T}\) for the Identity Matrix

identical(t(A) %*% A, A %*% t(A))
## [1] TRUE

Problem Set 2

Write an R function to factorize a square matrix \(A\) into \(LU\) or \(LDU\); whichever you prefer.

Create a function entitled flu which will host 3 separate matrices that will eliminate the following row/column combinations: [2,1],[3,1] and [3,2]

We will multiply these matrices with the 3x3 matrix A in attempts to finding the lower triangular matrix & upper triangular matrix.

flu <- function(A) {
    E21 <- -A[2,1]/A[1,1]
    E21M <- matrix(c(1,0,0,
                     E21,1,0,
                     0,0,1), nrow=3, byrow=TRUE)
    A21 <- E21M %*% A
    
    cat("\n","Elimination E2.1 & Matrix A","\n","\n")
    print(A21)
    
    E31 <- -A21[3,1]/A21[1,1]
    E31M <- matrix(c(1,0,0,
                     0,1,0,
                     E31,0,1), nrow=3, byrow=TRUE)
    A31 <- E31M %*% A21
    
    cat("\n","Elimination E3.1 & Matrix A","\n","\n")
    print(A31)
    
    E32 <- -A31[3,2]/A31[2,2]
    E32M <- matrix(c(1,0,0,
                     0,1,0,
                     0,E32,1), nrow=3, byrow=TRUE)
    
    U <- E32M %*% A31
    
    L  <- solve(E21M) %*% solve(E31M) %*% solve(E32M)
    
    A <- L %*% U

    cat("\n","Lower Triangular Matrix","\n","\n")
    print(L)
    
    cat("\n","Upper Triangular Matrix","\n","\n")
    print(U)
    
    cat("\n","Checking for Validation","\n","\n")
    (L %*% U == A)
}
cat("Matrix A","\n","\n")
## Matrix A 
## 
(A <- matrix(c(1,4,-3,
            -2,8,5,
            3,4,7),nrow=3, byrow=TRUE))
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]   -2    8    5
## [3,]    3    4    7
cat("\n","Factorization & LU","\n","\n")
## 
##  Factorization & LU 
## 
(flu(A))
## 
##  Elimination E2.1 & Matrix A 
##  
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]    0   16   -1
## [3,]    3    4    7
## 
##  Elimination E3.1 & Matrix A 
##  
##      [,1] [,2] [,3]
## [1,]    1    4   -3
## [2,]    0   16   -1
## [3,]    0   -8   16
## 
##  Lower Triangular Matrix 
##  
##      [,1] [,2] [,3]
## [1,]    1  0.0    0
## [2,]   -2  1.0    0
## [3,]    3 -0.5    1
## 
##  Upper Triangular Matrix 
##  
##      [,1] [,2] [,3]
## [1,]    1    4 -3.0
## [2,]    0   16 -1.0
## [3,]    0    0 15.5
## 
##  Checking for Validation 
## 
##      [,1] [,2] [,3]
## [1,] TRUE TRUE TRUE
## [2,] TRUE TRUE TRUE
## [3,] TRUE TRUE TRUE