This article is short intro to smoe basic matrix and vector notation and operations in R. This article helps to understand more complex methods which are based on matrix and vector algebra. I am not explaining too much how some of the calculations are made, if you want to know, look this article (I have taken examples from there): A tutorial on Principal Components Analysis.

Vector

Vector is a list of numerical elements:

vec=c(4, 11, 8, 10)

To find vector length we have to square each element, sum the squares and take square root of the results:

#let's define a function for that
veclen=function(vec) {
  sqrt(sum(vec^2))
}
#use it on our vector
veclen(vec)
## [1] 17.34935

Vector addition means adding two vector (add first element of vector 1 to the element of vector 3 and result is a first element of vector 3):

vec1=c(3, 2, 1, -2)
vec2=c(2, -1, 4, 1)
vec3=vec1+vec2
vec3
## [1]  5  1  5 -1

Scalar multiplication means multiplying every component by that real number to yield a new vector:

vec=c(3, 6, 8, 4)
vec*1.5
## [1]  4.5  9.0 12.0  6.0

Inner product (dot product) of a vector is found by multiplying each component in vector 1 by the component in vector 2 in the same position and adding them all together to yield a scalar value. The inner product is only de ned for vectors of the same dimension:

vec1=c(1, 6, 7, 4)
vec2=c(3, 2, 8, 3)
sum(vec1*vec2)
## [1] 83

Orthogonality two vectors are orthogonal to each other if their inner product equals zero. In two-dimensional space this is equivalent to saying that the vectors are perpendicular, or that the only angle between them is a 90 degrees:

#are orthogonal vectors
vec1=c(2, 1, -2, 4)
vec2=c(3, -6, 4, 2)
sum(vec1*vec2)==0
## [1] TRUE

Normal vectors (unit vector) is a vector of length 1. Any vector with an initial length >0 can be normalized by dividing each component in it by the vector’s length:

v=c(2,4,1,2)
#find it's length
veclength=veclen(v)
#normalize it and find its length (must be 1)
veclen(v/veclength)==1
## [1] TRUE
#make function that returns normalized vector
vecnorm=function(vector) {
  vector/veclen(vector)
}
vecnorm(v)
## [1] 0.4 0.8 0.2 0.4

Orthonormal vectors are vectors of unit length that are orthogonal to each other:

u =c(2/5, 1/5, -2/5, 4/5)
v=c(3/sqrt(65), -6/sqrt(65), 4/sqrt(65), 2/sqrt(65))
#are orthonormal because
sum(u*v)==0
## [1] TRUE
veclen(u)==1
## [1] TRUE
veclen(v)==1
## [1] TRUE

Gram-Schmidt Orthonormalization Process is a method for converting a set of vectors into a set of orthonormal vectors. It basically begins by normalizing the first vector under consideration and iteratively rewriting the remaining vectors in terms of themselves minus a multiplication of the already normalized vectors:

mat=matrix(c(1,2,1,0,2,0,2,3,1,1,1,0),nrow=4, ncol=3, byrow = TRUE)
mat
##      [,1] [,2] [,3]
## [1,]    1    2    1
## [2,]    0    2    0
## [3,]    2    3    1
## [4,]    1    1    0
#normalize 1st vector
v1=mat[,1]
v2=mat[,2]
v3=mat[,3]
normv1=vecnorm(v1)
normv2=vecnorm(v2)

vv2=v2-sum(normv1*v2)*normv1
normvv2=vecnorm(vv2)

vv3=v3-sum(normv1*v3)*normv1-sum(normvv2*v3)*normvv2
normvv3=vecnorm(vv3)

#make function to makse orthonormal vectors
orthonorm=function(matrix){
  #first normalise all matrix vectors (except last vector)
  initialNormalized=matrix(ncol=ncol(matrix), nrow=nrow(matrix))
  for(i in 1:(ncol(matrix)-1)) {
    initialNormalized[,i]=vecnorm(matrix[,i])
  }
  #lets make matrix where we add final results
  orthonormMatrix=matrix(ncol=ncol(matrix), nrow=nrow(matrix))
  #first vector is just normalised vector
  orthonormMatrix[,1]=initialNormalized[,1]
  #all other vectors are looped though
  for(i in 2:ncol(matrix)) {
    temp=matrix[,i]
    for(j in 2:i){#and following equation is used to calculate orthonomr vec
      temp=temp-sum(orthonormMatrix[,j-1]*matrix[,i])*orthonormMatrix[,j-1]
    }
    orthonormMatrix[,i]=vecnorm(temp)
  }
  orthonormMatrix
}

#example
orthonorm(mat)
##           [,1]          [,2]          [,3]
## [1,] 0.4082483  2.357023e-01  6.666667e-01
## [2,] 0.0000000  9.428090e-01 -3.333333e-01
## [3,] 0.8164966 -4.186913e-16 -1.850372e-16
## [4,] 0.4082483 -2.357023e-01 -6.666667e-01

Matrix transpose:

a=matrix(c(1,2,3,4,5,6),nrow=2, ncol=3, byrow = TRUE)
a
##      [,1] [,2] [,3]
## [1,]    1    2    3
## [2,]    4    5    6
t(a)
##      [,1] [,2]
## [1,]    1    4
## [2,]    2    5
## [3,]    3    6

Matrix multiplication it is possible to multiply two matrices only when the second matrix has the same number of rows as the first matrix has columns. The resulting matrix has as many rows as the first matrix and as many columns as the second matrix:

a=matrix(c(2,1,4,1,5,2),nrow=2, ncol=3, byrow = TRUE)
b=matrix(c(3,2,-1,4,1,2),nrow=3, ncol=2, byrow = TRUE)
a
##      [,1] [,2] [,3]
## [1,]    2    1    4
## [2,]    1    5    2
b
##      [,1] [,2]
## [1,]    3    2
## [2,]   -1    4
## [3,]    1    2
a %*% b
##      [,1] [,2]
## [1,]    9   16
## [2,]    0   26

Identity matrix is a square matrix with entries on the diagonal equal to 1 and all other entries equal zero. The identity matrix behaves like the number 1 in ordinary multiplication, which mean \(AI=A\):

a=matrix(c(2,4,6,1,3,5),nrow=2, ncol=3, byrow = TRUE)
a
##      [,1] [,2] [,3]
## [1,]    2    4    6
## [2,]    1    3    5
i=diag(3)#this is identity matrix
i
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1
a%*%i
##      [,1] [,2] [,3]
## [1,]    2    4    6
## [2,]    1    3    5

Orthogonal matrix: a matrix A is orthogonal if \(AA^T=A^TA=I\)

a=matrix(c(1,0,0,0,3/5,-4/5,0,4/5,3/5),nrow=3, ncol=3, byrow = TRUE)
a
##      [,1] [,2] [,3]
## [1,]    1  0.0  0.0
## [2,]    0  0.6 -0.8
## [3,]    0  0.8  0.6
#identity matrix
I=diag(3)
I
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1
#a is orthogonal because:
t(a)%*%a
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    1    0
## [3,]    0    0    1

Diagonal matrix \(A\) is a matrix where all the entries \(ai_{ij}\) are 0 when \(i\ne j\):

diag(c(1,2,3))
##      [,1] [,2] [,3]
## [1,]    1    0    0
## [2,]    0    2    0
## [3,]    0    0    3

Determinant is a function of a square matrix that reduces it to a single number:

a=matrix(c(4,1,1,2),nrow=2, ncol=2, byrow = TRUE)
det(a)
## [1] 7
b=matrix(c(-1,4,3,2,6,4,3,-2,8),nrow=3, ncol=3, byrow = TRUE)
b
##      [,1] [,2] [,3]
## [1,]   -1    4    3
## [2,]    2    6    4
## [3,]    3   -2    8
det(b)
## [1] -138

Eigenvectors and eigenvalues: an eigenvector is a nonzero vector that satisfies the equation \(A\vec v=\lambda \vec v\), where \(A\) is a square matrix, \(\lambda\) is a scalar, and \(\vec v\) is the eigenvector:

a=matrix(c(2,1,1,2),nrow=2, ncol=2, byrow = TRUE)
a
##      [,1] [,2]
## [1,]    2    1
## [2,]    1    2
eigen(a)
## $values
## [1] 3 1
## 
## $vectors
##           [,1]       [,2]
## [1,] 0.7071068 -0.7071068
## [2,] 0.7071068  0.7071068