C1 = c(3,-1,1)
C2 = c(1,0,2)
A = cbind(C1,C2)Let
\[A = U \Sigma V^{T}, \\ A_{32} \\ U_{33} \\ \Sigma_{32} \\ V_{22} \]
Where U is a unitary matrix, Sigma is a diagonal matrix with eigenvalues, and V is the matrix of eigenvectors. We will solve for U, sigma, and V.
t(A) %*% A ## C1 C2
## C1 11 5
## C2 5 5\[ A^{T}A = (U \Sigma V^{T})^{T}U \Sigma V^{T} =V \Sigma^{T} U^{T} U \Sigma V^{T} \] Matrix U has the property \[ U^{T} = U^{-1} \] so \[ A^{T}A = V \Sigma^{T} \Sigma V^{T} \] ###Solve for eigenvalues
V = eigen(t(A) %*% A )$vectors
V## [,1] [,2]
## [1,] -0.8701999 0.4926988
## [2,] -0.4926988 -0.8701999S = matrix(0L, nrow = 3, ncol =2)
o1 = eigen(t(A) %*% A )$values[1]
o2 = eigen(t(A) %*% A )$values[2]
S[1,1] = sqrt(o1)
S[2,2] = sqrt(o2)
S## [,1] [,2]
## [1,] 3.718999 0.000000
## [2,] 0.000000 1.472769
## [3,] 0.000000 0.000000v1 = eigen(t(A) %*% A )$vectors[,1]
v2 = eigen(t(A) %*% A )$vectors[,2]\[|A^{T}A - \lambda I| \] We use this formula to solve for the eigenvalues of sigma and eigenvectors of v. Sigma is a diagonal matrix so \[ A^TA = V \Sigma^{T} \Sigma V^{T} = V \Sigma^{2} V^{T} \]
Recall that \[A = U \Sigma V^{T} \] Right-side multiply by V \[AV = U \Sigma \] Note the ith column of AV \[A \vec{v}_{i} = \sigma_i\vec{u}_i \implies \vec{u}_i= \frac{1}{\sigma_i}A\vec{v}_i\] so
U = matrix(0L, nrow = 3, ncol = 3)
U[,1] = (A %*% V[,1])/sqrt(o1)
U[,2] = (A %*% V[,2])/sqrt(o2)
U## [,1] [,2] [,3]
## [1,] -0.8344446 0.4127576 0
## [2,] 0.2339877 -0.3345391 0
## [3,] -0.4989508 -0.8471805 0Finally, we need to solve for the third column of U. This is the vector that forms the solution to \[A^{T}x=0\] Solving this equation we find that \[ u_13 = -2\\ u_23 = -5 \\ u_33 = 1\]
w=sqrt(30)
U[,3] = c(-2/w,-5/w,1/w)
U## [,1] [,2] [,3]
## [1,] -0.8344446 0.4127576 -0.3651484
## [2,] 0.2339877 -0.3345391 -0.9128709
## [3,] -0.4989508 -0.8471805 0.1825742You can solve for the SVD using the above process, or simply
svd(A)## $d
## [1] 3.718999 1.472769
##
## $u
## [,1] [,2]
## [1,] -0.8344446 0.4127576
## [2,] 0.2339877 -0.3345391
## [3,] -0.4989508 -0.8471805
##
## $v
## [,1] [,2]
## [1,] -0.8701999 0.4926988
## [2,] -0.4926988 -0.8701999For any matrix A, AA^T is a symmetric matrix. That is, \[ (AA^{T})^{T} = (A^{T})^{T}A^{T} = A^{T}A \] For any symmetric matrix, there exists an orthogonal matrix V such that \[ \Sigma^2 = V^{T}A^{T}AV \] #Problem 2 ##[a] ###Create Matrix
R1 = c(1,2,1)
R2 = c(0,1,-1)
A = rbind(R1,R2)Let
\[A = U \Sigma V^{T}, \\ A_{23} \\ U_{22} \\ \Sigma_{23} \\ V_{33} \]
Where U is a unitary matrix, Sigma is a diagonal matrix with eigenvalues, and V is the matrix of eigenvectors. We will solve for U, sigma, and V.
t(A) %*% A ## [,1] [,2] [,3]
## [1,] 1 2 1
## [2,] 2 5 1
## [3,] 1 1 2\[ A^{T}A = (U \Sigma V^{T})^{T}U \Sigma V^{T} =V \Sigma^{T} U^{T} U \Sigma V^{T} \] Matrix U has the property \[ U^{T} = U^{-1} \] so \[ A^{T}A = V \Sigma^{T} \Sigma V^{T} \]
eigen(t(A) %*% A )## eigen() decomposition
## $values
## [1] 6.236068e+00 1.763932e+00 8.881784e-16
##
## $vectors
## [,1] [,2] [,3]
## [1,] -0.3897342 0.1729896 0.9045340
## [2,] -0.8714722 -0.3868166 -0.3015113
## [3,] -0.2977305 0.9057856 -0.3015113V = eigen(t(A) %*% A )$vectors
V## [,1] [,2] [,3]
## [1,] -0.3897342 0.1729896 0.9045340
## [2,] -0.8714722 -0.3868166 -0.3015113
## [3,] -0.2977305 0.9057856 -0.3015113S = matrix(0L, nrow = 2, ncol = 3)
o1 = eigen(t(A) %*% A )$values[1]
o2 = eigen(t(A) %*% A )$values[2]
S[1,1] = sqrt(o1)
S[2,2] = sqrt(o2)
S## [,1] [,2] [,3]
## [1,] 2.497212 0.000000 0
## [2,] 0.000000 1.328131 0v1 = eigen(t(A) %*% A )$vectors[,1]
v2 = eigen(t(A) %*% A )$vectors[,2]\[|A^{T}A - \lambda I| \] We use this formula to solve for the eigenvalues of sigma and eigenvectors of v. Sigma is a diagonal matrix so \[ A^TA = V \Sigma^{T} \Sigma V^{T} = V \Sigma^{2} V^{T} \]
Recall that \[A = U \Sigma V^{T} \] Right-side multiply by V \[AV = U \Sigma \] Note the ith column of AV \[ A \vec{v}_{i} = \sigma_i\vec{u}_i \implies \vec{u}_i= \frac{1}{\sigma_i}A\vec{v}_i \] so
U = matrix(0L, nrow = 2, ncol = 2)
U[,1] = (A %*% V[,1])/sqrt(o1)
U[,2] = (A %*% V[,2])/sqrt(o2)
U## [,1] [,2]
## [1,] -0.9732490 0.2297529
## [2,] -0.2297529 -0.9732490We can express matrix A as the sum of rank 1 matrices as \[ A = \sigma_1 \vec{u}_1\vec{v}_1 + \sigma_2\vec{u}_2\vec{v}_2\] #Could not resolve any further
data = read.csv("C:/Users/Will/OneDrive/Documents/School/375T Predictive Analytics/HW6/preference.csv")
svd(data)## $d
## [1] 12.7477512 8.8129123 1.8485212 0.6406147
##
## $u
## [,1] [,2] [,3] [,4]
## [1,] -0.5094678 0.03289134 0.4183176 0.58116100
## [2,] -0.4826216 0.19072958 -0.1968219 -0.68215598
## [3,] -0.3914029 0.07175148 -0.4478458 0.21871397
## [4,] -0.1821768 -0.67770518 0.5099854 -0.33686770
## [5,] -0.1868867 -0.67372273 -0.5562098 0.18268590
## [6,] -0.5349838 0.21025101 0.1274804 -0.04717331
##
## $v
## [,1] [,2] [,3] [,4]
## [1,] -0.6675005 0.1720407 0.5994796 0.4067792
## [2,] -0.6964677 0.1985441 -0.5523495 -0.4128232
## [3,] -0.1700612 -0.6865523 0.4021622 -0.5813696
## [4,] -0.2011344 -0.6779588 -0.4168930 0.5710666Our interpretation is that custumer preference varies the most with books, pens, movies, and music in that order.