Nathan Cooper
June 25, 2018
‘Eigen’ is, in this context, a German word for Proper.
In these situations the solutions are solved by defining the system as a square matrix that satisfies:
\[ Ax = \lambda x \]
Where we are looking for the null spaces of A where x is non-zero:
\[ (A - \lambda I)x = 0 \] \[ \boxed{det(A - \lambda I) = 0} \]
For 2x2 and 3x3 matrices Determinant can be found by multiplying factors along the diagonals of a matrix, the adding/subtracting terms on different diagonals:
\[ A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \\ \end{bmatrix} \]
Factors along the solid lines are added, factors along the dotted lines are subtracted.
This generates a 3\(^{rd}\) degree polynomial. The eigenvalues are the roots.
The eigenvectors are solved by setting \(\lambda\) equal to the root and solving for the vector x.
\[ I = m*r^2 \]
\[ \textbf{M} = \begin{bmatrix} I_{xx} & I_{xy} & I_{xz} \\ I_{yx} & I_{yy} & I_{yz} \\ I_{zx} & I_{zy} & I_{zz} \\ \end{bmatrix} \]
A system of particles has an Inertia matrix defined by M in the code below. What are the systems ‘preferred’ axes of rotations and there inertia values?
## [,1] [,2] [,3]
## [1,] 1.0 0.5 0.0
## [2,] 0.5 1.0 0.5
## [3,] 0.0 0.5 1.0
## [1] 1.7071068 1.0000000 0.2928932
## [,1] [,2] [,3]
## [1,] 0.5000000 -7.071068e-01 -0.5000000
## [2,] 0.7071068 -1.099065e-15 0.7071068
## [3,] 0.5000000 7.071068e-01 -0.5000000
\[ \lambda_1 = \sqrt{2} \]
\[ x_1 = \begin{bmatrix} \frac{1}{2} & \frac{1}{\sqrt{2}} & \frac{1}{2} \\ \end{bmatrix} \]
\[ \lambda_2 = 1 \]
\[ x_2 = \begin{bmatrix} \frac{-1}{\sqrt{2}} & 0 & \frac{1}{\sqrt2} \\ \end{bmatrix} \]
\[ \lambda_3 = 1 - \frac{1}{\sqrt2} = 0.2928932 \]
\[ x_2 = \begin{bmatrix} \frac{-1}{2} & \frac{1}{\sqrt2} & \frac{-1}{2} \\ \end{bmatrix} \]
The Eigenvalues give the rotational inertia for each axis.
The Eigenvectors define each rotational axis.
Note that intermediate case is unstable. -If you don’t flip your phone perfectly, it’ll ‘wobble’. -Wobbling is picking up some rotation along the other eigenvectors.
I am not responsible for damaged phones. Go to your junk drawer and use an old one :P
The SVD technique for recommender system also uses eigenvectors and eigenvalues.
If we had complete knowledge of our user-item matrix, R, then for the decomposition:
\[ R = U\Sigma V^T \]
If the rows of R are the users, then U is a user-user matrix, i.e., a user-user collaborative filter.
\(\Sigma\) is composed of the Eigenvalues, these act as weights to each user-item.
Note that multiplying any matrix by it’s transpose creates a square matrix, from Eigenvalues/vectors can be computed.
In real-world Applications, R is incomplete and Sparse.
In this regard imputation and normalization are the key to using SVD.
Imputing values for missing data makes a complete matrix. However, the imputed value may not make sense in the rating system.
Normalization forces the imputed values to make sense with regard to the missing value.
Normalization also stores Bias between user-items.
The SVD recalculates the ratings matrix with respect to imputation and normalization.
Predictions are made with the normalization is removed. Bias and missing value context is added back in.