Ch3.1.1 Part 1: Vector and Matrix Operations

Data Science, Matrices and Math

Data science makes use of matrices, vectors and math.

title

Data Science, Matrices and Math

title

title title

Principle Component Analysis

PCA analysis & clustering

Lots of linear algebra (matrices & equations)

PCA & Eigenvectors

Magnitude and direction.

Orthogonal eigenvectors
Math 225 Linear Algebra

Soundwaves & Vectors

Math 362 Fourier Analysis
Record my voice saying “aah”.
Vector x = aah.wav:

library(audio) 
x <-load.wave("aah1sec.wav") 
head(x)

sample rate: 44100Hz, mono, 16-bits
[1] -0.03848384 -0.03360088 -0.02908414 -0.02600177 -0.02514725 -0.02630696

Soundwaves, Vectors & Plots

Graph first twenty values of x and follow the dots!

plot(x[1:20])

title

Soundwaves & Transform Matrices

Multiply by transform matrix F to obtain Fx = c.
The graph of c reveals frequency information.

plot(c[1:500])

title

Linear Algebra & Frequency Transform

Dominant frequency of “aah” wave is \( k = 82 \) Hz.

c(max(c[1:500]), which.max(c[1:500]))

[1]  0.01767538 82.00000000

title

Numerical Analysis & Transform Matrices

Slow: Computing Fx \( \sim O(n^2) \)
Fast: Computing fft(x) \( \sim O(n \ln(n)) \)

(n <- length(x))

[1] 44032

c(n^2, n*log(n))

[1] 1938817024.0     470819.7

Systems of Linear Equations

Linear Systems form basis of many STEM applications.
Math 225 Computational Linear Algebra - take it while you still can!

title

Vectors in Math

Vectors can be written horizontally or vertically.
When type-written, horizontal format is common:

\[ \begin{aligned} \mathbf{u} & = \begin{bmatrix} u_1, & u_2, & \ldots, & u_n \end{bmatrix} \\ \mathbf{v} & = \begin{bmatrix} v_1, & v_2, & \ldots, & v_n \end{bmatrix} \end{aligned} \]

When hand-written, vertical format is common:

\[ \mathbf{u} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix}, ~~ \mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} \]

Vector-Vector Addition

For two vectors \( \mathbf{u} \) and \( \mathbf{v} \),

\[ \mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix} =\begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ \vdots \\ u_n + v_n \end{bmatrix} \]

Example: Vector-Vector Addition

Compute:

\[ \mathbf{u} + \mathbf{v} = \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} + \begin{bmatrix} 4 \\ -4 \\ 3 \\ 1 \end{bmatrix} = ~~? \]

Example: Vector-Vector Addition

Result:

\[ \mathbf{u} + \mathbf{v} = \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} + \begin{bmatrix} 4 \\ -4 \\ 3 \\ 1 \end{bmatrix} =\begin{bmatrix} 7 \\ -3 \\ 1 \\ 1 \end{bmatrix} \]

Example: Plot 1

u <- c(3,1,-2,0)
plot(u)

plot of chunk unnamed-chunk-7

v <- c(4,-4,3,1)
plot(v)

plot of chunk unnamed-chunk-8

Example: Plot 2

u <- c(3,1,-2,0)
plot(u,type="l",col="blue")

plot of chunk unnamed-chunk-9

v <- c(4,-4,3,1)
plot(v,type="l",col="red")

plot of chunk unnamed-chunk-10

Scalar-Vector Multiplication

In science and mathematics, a scalar \( x \) is a number, as opposed to a vector \( \mathbf{u} \).
When multiplying by a scalar, it has the effect of scaling the quantity being multiplied, which how it got its name.
If \( x \) is a scalar, then

\[ x \mathbf{u} = x \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} =\begin{bmatrix} x u_1 \\ x u_2 \\ \vdots \\ x u_n \end{bmatrix} \]

Example: Scalar-Vector Multiplication

Compute:

\[ 4 \mathbf{u} = 4 \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} = ~~? \]

u <- c(3,1,-2,0)
plot(u)

plot of chunk unnamed-chunk-11

Example: Scalar-Vector Multiplication

Result:

\[ 4 \mathbf{u} = 4 \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} =\begin{bmatrix} 12 \\ 4 \\ -8 \\ 0 \end{bmatrix} \]

v <- c(12,4,-8,0)
plot(v)

plot of chunk unnamed-chunk-12

Example: Before & After Plot

plot(u,type="l",col="blue")

plot of chunk unnamed-chunk-13

plot(v,type="l",col="red")

plot of chunk unnamed-chunk-14

Vector-Scalar Addition

If \( x \) is a scalar, then

\[ \mathbf{u} + x = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + x =\begin{bmatrix} u_1 + x \\ u_2 +x \\ \vdots \\ u_n +x \end{bmatrix} \]

Vector-Scalar Addition

Here are some more details:

\[ \begin{aligned} \mathbf{u} + x & = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + x = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + x \begin{bmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{bmatrix} = \begin{bmatrix} u_1 \\ u_2 \\ \vdots \\ u_n \end{bmatrix} + \begin{bmatrix} x \\ x \\ \vdots \\ x \end{bmatrix} \\ \\ & =\begin{bmatrix} u_1 + x \\ u_2 +x \\ \vdots \\ u_n +x \end{bmatrix} \end{aligned} \]

Example: Vector-Vector Addition

Compute:

\[ \mathbf{u} + x = \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} + 4 = ~~? \]

Example: Vector-Vector Addition

Result:

\[ \mathbf{u} + x = \begin{bmatrix} 3 \\ 1 \\ -2 \\ 0 \end{bmatrix} + 4 =\begin{bmatrix} 7 \\ 5 \\ 2 \\ 4 \end{bmatrix} \]

Example: Before & After Plot

u <- c(3,1,-2,0)
plot(u,type="l",col="blue")

plot of chunk unnamed-chunk-15

v <- c(7,5,2,4)
plot(v,type="l",col="red")

plot of chunk unnamed-chunk-16

Vectors in R

In R, vectors are expressed horizontally:

u <- c(3, -1 , 2); v <- c(1, 6, 0); x <- 4
u

[1]  3 -1  2

[1] 1 6 0

Vectors Arithmetic in R

Horizontal uses less space, but harder to see operations.

u <- c(3, -1 , 2); v <- c(1, 6, 0); x <- 4
u + v

[1] 4 5 2

u + x

[1] 7 3 6

Vector Dimension & Compatibility ...

If the lengths of two vectors \( \mathbf{u} \) and \( \mathbf{v} \) are different, then mathematically they cannot be added, because of the dimension incompatibility.

\[ \mathbf{u} + \mathbf{v} = \begin{bmatrix} u_1 \\ u_2 \\ u_3 \end{bmatrix} + \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} = \mathrm{Dimension ~Error} \]

... Except in R

R “recycles” the shorter vector by repeating the pattern in the entries until the correct length is found.
This is basically what happened for \( \mathbf{u} + x \).

u <- c(1,1,1,1); v <- c(1,6)
u + v

[1] 2 7 2 7

Example 1: Recycling

Math Result

\[ \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} + \begin{bmatrix} 1 \\ 6 \end{bmatrix} = \mathrm{~Error} \]

R Result

u <- c(1,1,1,1); v <- c(1,6)
u + v

[1] 2 7 2 7

Example 2: Recycling

Math Result

\[ \begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} + \begin{bmatrix} 3 \\ 6 \end{bmatrix} = \mathrm{~Error} \]

R Result

u <- c(1,2,0); v <- c(3,6)
u + v  = ?

Your Answer Here!

Example 2: Recycling

Math Result

\[ \begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix} + \begin{bmatrix} 3 \\ 6 \end{bmatrix} = \mathrm{~Error} \]

R Result

u <- c(1,2,0); v <- c(3,6)
u + v

[1] 4 8 3

Example 3: Recycling

Math Result

\[ \begin{bmatrix} 0 \\ 2 \\ 1 \end{bmatrix} + \begin{bmatrix} 3 \\ 6 \\ 4 \\ 10 \end{bmatrix} = \mathrm{~Error} \]

R Result

u <- c(0,2,1); v <- c(3,6,4,10)
u + v = ?

Your Answer Here!

Example 3: Recycling

Math Result

\[ \begin{bmatrix} 0 \\ 2 \\ 1 \end{bmatrix} + \begin{bmatrix} 3 \\ 6 \\ 4 \\ 10 \end{bmatrix} = \mathrm{~Error} \]

R Result

u <- c(0,2,1); v <- c(3,6,4,10)
u + v

[1]  3  8  5 10

Recycling in R

Recycling may be as good or better than zero-padding.
Keep pattern information vs reduce computational energy.

u <- c(1,1,1); v <- c(1,6)
u + v

[1] 2 7 2

u <- c(1,1,1); v <- c(1,6,0)
u + v

[1] 2 7 1

Is Recycling Common?

MATLAB

title

Python

title

What's Next

After this, we look at similar operations for matrices.

\[ \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \\ \end{bmatrix} \]

u <- c(2,1,4,1,0,3)
(A <- matrix(u,3))

     [,1] [,2]
[1,]    2    1
[2,]    1    0
[3,]    4    3