Video 1 Vectors

Start in the \((x,y)\) standard coordinate plane. Define the \(i=[1 \quad 0 ]\) and \(j=[0\quad1]\). Note that any linear combination of the two vectors \(ai + bj\) could reach any point on the plane. We call this set of achievable points the span of the two vectors.

What’s the usefulness of vectors? They can be used to represent data, for one thing.

Video 2 Linear combinations, span, and basis vectors

Consider vector \(a = [2 \quad 4]\), which is represented in the coordinate plane as a point 2 points to the right and four points up. We can break \(a\) down into a sum of scaled vectors \(i, j\): \[ [2 \quad 4] = 2i + 4j \]

This applies not just to \(a\), but any vector in the coordinate plane. Thus, \(i\) and \(j\) are commonly known as the basis vectors of the coordinate plane. Basis vectors are defined differently for different spaces, so knowing what the basis vectors of \(a = [2 \quad 4]\) are (i.e. what is being scaled by 2 and 4), is essential for “placing” \(a\).

Original \(i,j\)

Different \(i, j\)

Let’s consider vectors \(a\) and \(b = wb\) , where \(b\) simply a scaled version of \(a\). The span of \(a,b\) is simply a line along \(a\) (or \(b\)). The span of \(a\) is equal to the span of \(b\) is equal to the span of \(a,b\).

We use the term linearly dependent to refer to groups of vectors where the removal of one does not change the span. This implies that the vector that could be removed is simply a linear combination of the other vectors, and thus exists on the span defined by the other vectors. This definition applies to \(a, b\).

Linear independence, then, refers to groups of vectors where the removal of one vector does change the span. This is equivalent to saying that each vector adds a new dimension to the span. Thus, this one vector cannot be expressed as a linear combination of the other vectors.

The basis vector(s) of a vector space, then, are the linearly independent vector(s) that span the full space. In the vector space defined by the linear combinations of \(a, b\), there is only one basis vector.

Video 3 Matrix transformations

We can think of a 2x2 matrix \(T\) as describing with its two columns the results of a transformation on \(i, j\) (defined above) that span the 2d vector space.

\[ T = \begin{bmatrix} 5 & 4 \\ -2 & 1 \end{bmatrix} \]

Where the transformation resulted in \(i\) becoming \([5 \quad -2]\) and \(j\) becoming \([4 \quad 1 ]\). This transformation is linear; the origin is centered at 0 and the grid lines remain parallel.

Given a vector \(v = -1i + 2j = [-1 \quad 2]\) in the original vector space, the transformation applied to (multiplied by) \(v\) results in a new vector just as it did for the basis vectors.

We can find the coordinates of the transformed \(v\) by simply calculating the linear combination of the transformed basis vectors when using the original scalars. Just as \(v = -1i + 2j\),

\[ \text{ new vector } = -1 \cdot [5 \quad -2 ] + 2 \cdot [4 \quad 1] = [-5+8 \quad 2+2] = [3 \quad 4 ] \]

## [1] "Transformation T:"
##      [,1] [,2]
## [1,]    5    4
## [2,]   -2    1
## [1] "-1 * T[,1] + 2 * T[,2]"
## [1] 3 4

This is equivalent to saying that the result of a transformation \(T\) on vector \(v\) (where \(v\) is understood to be a linear combination of basis vectors \(i,j\)) is the linear combination of the transformed basis vectors using the same scalars as in \(v\). The transformed basis vectors are the columns of \(T\). In essence, given transformation matrix \(T\)

\[ T = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \]

and vector \(v = [ x \quad y]\), the transformation \(T\) applied to \(v\) can be expressed as:

\[ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} x \\ y \end {bmatrix} = x\begin{bmatrix} a \\ c \end{bmatrix} + y \begin{bmatrix} b \\ d \end{bmatrix} = \begin{bmatrix} a x + b y \\ c x + d y \end{bmatrix} \]

You saw that right – the operation we’ve been describing is matrix-vector multiplication! To multiply a matrix by a vector (where the matrix is on the left) is to transform a vector in the manner defined by the matrix.

Video 4 Matrix multiplication

Let’s consider the multiplication operation below. What we’re seeing is two transformations applied to a vector. The matrix with \(e,f,g,h\) is first applied, and then the matrix \(a,b,c,d\) is applied to the result of the initial operation.\[ \begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} e& f \\ g &h \end{bmatrix} \begin{bmatrix} x \\ y \end {bmatrix} \]

Transformations are not commutative; \(T_1 T_2 v \ne T_2 T_1 v\). However, they are associative; \(T_1(T_2 T_3) = (T_1 T_2) T_3\).

Video 5 Linear transformations in 3d space

For transformations in 3d space, we’d expect the matrix to have 3 columns and 3 rows; one for each basis vector.

Video 6 Determinants

The determinant of a matrix is how much its transformation scales the area of a 1x1 unit in the original vector space. This factor can be generalized to any shape in the original space.

\[ det \left( \begin{bmatrix} a & b \\ c & d \end{bmatrix} \right) =ad - bc \]

\(a, d\) represent how much an area is scaled in the \(x\) and \(y\) direction.

\(b,c\), which are subtracted, account for diagonal transformations.

A negative determinant represents an inversion. A determinant of 0 represents a reduction in dimensions.

Let’s confirm that negative determinants correspond to inversion and zero determinants represent a reduction.

Inversion (negative determinant)

\[ det \left( \begin{bmatrix} -2 & 0 \\ 0 & 2 \end{bmatrix} \right) = -4 \]

The first basis vector is scaled in the negative direction by 2 units while the second basis vector is scaled by a factor of 2 in the positive direction. The first basis vector, instead of being to the right of the second basis vector, is now to the left of it. This is an inversion that also scales areas in the vector space by 4.

Reduction in dimensions aka singular transformation (zero determinant)

\[ det \left( \begin{bmatrix} 3 & 2 \\ 0 & 0 \end{bmatrix} \right) = 0 \]

This transformation takes the (square) area created by the initial basis vectors. Since this transformation turns \(j\) into a basis vector along the x-axis, the 2 dimensional space is transformed into a one dimensional line. All points now exist along the x-axis. The matrix is linearly dependent. Following the transformation, there is only one basis vector.

Video 7 Inverse matrices, rank, and null space

One of the useful things about matrices is that they can be used to interpret and solve systems of linear equations where the number of equations equals the number of unknowns.

Case in point: \[ \begin{aligned} 2x + 5y + 3z &= -3 \\ 4x + 0y + 8z &= 0 \\ 1x + 3y + 0z &= 2 \end{aligned} \quad \Rightarrow \quad \begin{bmatrix} 2 & 5 & 3 \\ 4 & 0 & 8 \\ 1 & 3 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \\ z \end{bmatrix}= \begin{bmatrix} -3 \\ 0 \\ 2 \end{bmatrix} \]

Whatever the system of linear equation relates to on the right, when cast in matrix form it has a geometric interpretation; the solution vector \(v\) is the vector that, when transformed, becomes \([ -3 \quad 0 \quad 2]\).

How would we solve for this \(v\)? Given that we have the transformation \(T\) and the result \(w\), we could apply the inverse transformation of \(T\), denoted by \(T^{-1}\), to \(w\). It would bring us “back” to \(v\). Formally speaking,

\[ T^{-1}(T(\vec{v})) = \vec{v} \]

Thus, we can solve for \(v\) in the following manner:

\[ \begin{aligned} T \cdot \vec{v} &= \vec{w} \\ T^{-1} \cdot T \cdot \vec{v} &= T^{-1} \cdot \vec{w}\\ \vec{v} &= T^{-1} \cdot \vec{w} \end{aligned} \]

The reason \(T^{-1} \cdot T\) “disappears” is because if one applied a transformation \(T\) and then its inverse \(T^{-1}\) to a vector, they would be back to where they started. Thus, \(T^{-1} \cdot T = I\), where \(I\) is the identity matrix. Multiplying this identity matrix by anything does nothing. This should be evident from its columns (recall that the columns of a matrix are how its basis vectors are transformed). In the case of \(I\), the basis vectors are the default basis vectors for our space.

\[ I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \]

An important caveat regarding the inverse

We cannot assume that for every transformation \(T\), there exists an inverse \(T^{-1}\). If the determinant of a (square) transformation is 0, there exists no inverse. This is because in order to collapse a space into a smaller dimension, more than one vector in the larger dimension corresponds to a single vector in the smaller dimension. For a vector \(w\) in the smaller dimension, there exists no \(T^{-1}\) that, when applied to \(w\), would necessarily give you the \(v\) pre-transformation.

\[ \not\exists\, T^{-1} \;\; \text{such that } T^{-1}(T(\vec{v})) = \vec{v} \quad \forall \vec{v} \]

\[ \exists \, \vec{v}_1 \ne \vec{v}_2 \quad \text{such that} \quad T(\vec{v}_1) = T(\vec{v}_2) \]

\[ \Rightarrow \; T^{-1}(T(\vec{v}_1)) \; \text{is ambiguous — no unique } \vec{v}_1 \]

rank: the number of dimensions in the output of a transformation. A singular transformation \(T_s\) on a vector in an \(n-\)dimensional space has rank \(<n\).

null space: Given a transformation \(T\), the set of vectors \(\vec{v}\) for which \(T \cdot \vec{v} = \vec{0}\), where \(\vec{0}\) is the 0 vector (the origin). For a linear transformation, the null space always contains the origin \([0 \quad 0], [0 \quad 0 \quad 0 ], \text{ etc.}\). In the event that a transformation is singular, the null space will contain more than just \(\vec{0}\), as the number of vectors that collapse onto the origin will be greater than 1. From 3d to 2d, the null space is a plane; from 2d to a line, the null space is a line.

column space : the set of all possible outputs of a transformation – the span of the columns of the transformation matrix.

rank, then, is the number of dimensions in the column space. \(\vec{0}\) is always in the column space.

Video 8 Non-square matrices

\(x \times y\) non-square matrices have the effect of mapping the basis vectors in \(x\) dimensions in a space with \(y\) dimensions. This is consistent with the observation that the columns in a matrix representing a transformation are the transformed basis vectors.

Video 9 Dot products and duality

A dot product of two vectors is the result of the operation

\[ \vec{v} \cdot \vec{w} = \begin{bmatrix} a_1 \\ a_2 \end{bmatrix} \cdot \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = a_1 b_1 + a_2 b_2 \]

Note that the algorithm described above is identical to the result of applying a transformation from 2 dimensions to 1 dimension to a vector:

\[ \begin{bmatrix} a_1 \quad a_2 \end{bmatrix} \cdot \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = a_1 b_1 + a_2 b_2 \]

What’s suggested is that each dot product represents a linear transformation of some kind!

But how do we understand this linear transformation as represented by \(\vec{v}\)?

Here goes: By definition, the dot product is also

  1. (length of \(\vec{v}\)) * (length of projection \(\vec{w}\) on \(\vec{v}\)).

  2. (length of \(\vec{w}\)) * (the length of projection \(\vec{v}\) on \(\vec{w}\)).

Which implies that the dot product of \(\vec{v} \cdot \vec{w} = \vec{w} \cdot \vec{v}\). Order doesn’t matter.

(Recall that a projection of \(\vec{v}\) on \(\vec{w}\) is the vector on a line running through \(\vec{w}\) pointing to where the distance between the tip of \(\vec{v}\) and the line running through \(\vec{w}\) is minimized. This is the location of an orthogonal line segment connecting the tip of \(\vec{v}\) and the line.)

Let’s consider a vector \(\vec{u}\). We can say that its tip pointing to \((u_x, u_y)\) correspond to where basis vectors \(\vec{i}, \vec{j}\) land following some 2d-to-1d transformation that is equivalent to the projection of the two onto \(\vec{u}\). Thus, this transformation/projection can be described as \(\vec{u} = [u_x \quad u_y]\). Furthermore, following the projection, we characterize \(\vec{u}\) as pointing to the unit vector \(\vec{u}\) in the 1st dimension; meaning, it points to coordinate 1 on the “new” number line that is the post-transformation span. Thus, we can think of the length of \(\vec{u}\) as 1.

Given that \(\vec{u}\) describes the location of basis vectors post-transformation, if we apply \(\vec{u}\) to \(\vec{v}\), where \(\vec{v}\) is necessarily a linear combination of basis vectors, then we obtain the projection of \(\vec{v}\) onto \(\vec{u}\). Where it points to on the tick mark is also its length \(||\vec{v}||\).

Since we are working with unit vectors, the projection of \(\vec{v}\) on \(\vec{u}\), in giving us the location of transformed \(\vec{v}\) on the number line, also gives us the product of the length of \(\vec{u}\) and the length of projected \(\vec{v}\).

\[ \vec{u} \cdot \vec{v} = \begin{bmatrix} a_1 \quad a_2 \end{bmatrix} \cdot \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = a_1 b_1 + a_2 b_2 = || \vec{v}|| \cdot 1 \]

Now consider the case where \(\vec{u}\) is not a unit vector. It’s our original \(\vec{w}\) and it has a length that isn’t 1. It’s a scalar \(m\cdot 1\). Owing to linearity, whatever transformation \(\vec{u}\) corresponds to unit length 1, then the scaled transformation \(m \vec{u}\) must correspond to length \(m\). Thus:

\[ m \vec{u} \cdot \vec{v} = \begin{bmatrix} ma_1 \quad ma_2 \end{bmatrix} \cdot \begin{bmatrix} b_1 \\ b_2 \end{bmatrix} = ma_1 b_1 + ma_2 b_2 = m(a_1b_1 + a_2b_2) = m \cdot || \vec{v}|| \]

Dot products can be used to determine whether two vectors \(\vec{v}, \vec{w}\) are pointed in the same (positive), different (negative), or orthogonal (0) directions.

Video 10-11 Cross products

Two dimensional cross products

Consider the cross product \(\vec{v} \times \vec{w}\). The result is a scalar equal to the change to the unit area following the transformation \([\vec{v} \quad \vec{w}]\). As we established, this value is the determinant.

\[ p =\vec{det \left( [\vec{v} \quad \vec{w}] \right)} \]

Note that \(\vec{v} \times \vec{w} = - \vec{w} \times \vec{v}\). In other words, order matters. If \(\vec{v}\) is a clockwise rotation away from \(\vec{w}\), the cross product is positive.

Three dimensional cross products

In three dimensions, which is where cross products usually apply, the cross product is a vector \(\vec{z}\). It points in a direction orthogonal to \(\vec{v}, \vec{w}\), which means \(\vec{z}\) exists in an additional dimension.

Its direction can be determined using the hand rule.

With three dimensions, the definition is a little more involved: \[ \left[ \begin{array}{c} v_1 \\ v_2 \\ v_3 \end{array} \right] \times \left[ \begin{array}{c} w_1 \\ w_2 \\ w_3 \end{array} \right] = \det \left( \left[ \begin{array}{ccc} \hat{i} & v_1 & w_1 \\ \hat{j} & v_2 & w_2 \\ \hat{k} & v_3 & w_3 \end{array} \right] \right) \]

Such that the cross product, \(\vec{p}\), can be computed (if you use the algorithm for computing the determinant for 3-by-3 matrices) as: \[ \vec {P} = \hat{i} \underbrace{(v_2 w_3 - v_3 w_2)}_{\text{Some number}} + \hat{j} \underbrace{(v_3 w_1 - v_1 w_3)}_{\text{Some number}} + \hat{k} \underbrace{(v_1 w_2 - v_2 w_1)}_{\text{Some number}} \]

“But what actually is a cross product?” What’s actually going on in the above definition?

Recall duality – that for every transformation that can be performed on a vector, there exists a corresponding matrix (vector if the transformation is to a number line) that encodes the same transformation, such that taking the dot product gives you the post-transformation vector.

The equation for \(\vec{P}\) we saw above is the same as the solution to the below expression:

\[ \vec{\mathbf{p}} \cdot \begin{bmatrix} p_1 \\ p_2 \\ p_3 \end{bmatrix} \cdot \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \det \left( \begin{bmatrix} x & v_1 & w_1 \\ y & v_2 & w_2 \\ z & v_3 & w_3 \end{bmatrix} \right) \]

We are essentially solving for the vector \(\vec{P}\) that when dotted with any vector \(\vec{u}\), gives the volume of the parallelepiped defined by \(\vec{v}, \vec{w}, \vec{u}\). As we previously established, this vector represents a transformation to the number line.

By the rules of a parallelepiped, if \(\vec{u}\) is the third edge of a parallelepiped (not the \(\vec{v}, \vec{w}\) defining the base), then if \(\vec{u} \cdot \vec{p} = || \vec{u} || * ||\vec{v}|| = V(parallelepiped)\), then \(|| \vec{p} ||\) must be equal to the area of the parallelogram defining the base of the parallelepiped. Additionally, this \(\vec{p}\) must be perpendicular to the \(\vec{v}, \vec{w}\) that define the base.

Video 12 Cramer’s Rule

Consider the setup \(A\vec{v} = \vec{b}\), where \(A\) and \(\vec{b}\) are known. We want to find \(\vec{v}\). In essence, we want to find the input vector such that when transformed by matrix \(A\), we get \(\vec{b}\). Assume \(A\) is a non-singular transformation. We could solve this using \(A^{-1}\). However, there is another way of doing this that relies on representing the coordinates of our input vector using the area of parallelograms.

Consider \(\vec{v}\) with unknown \((x,y)\) coordinates.

We can represent the \(x\) coordinate using the area of the parallelogram created by the basis vector \(\vec{j} = [0 \quad 1]\) and \(\vec{x}\). This area is given by \(det \left( [ \vec{v} \quad \vec{j} ] \right) = x(1) - 0(y) = x\).

We can represent the \(y\) coordinate using the area of the parallelogram created by the basis vector \(\vec{i} = [1 \quad 0 ]\) and \(\vec{v}\). This area is given by \(det \left( [ \vec{i} \quad \vec{v} ] \right) = 1(y) - x(0) = y\).

The general algorithm here is replace column corresponding to basis vector of desired coordinate with input vector.

Note the ordering of the columns in the two matrices we take the determinant of. This ordering matters because it gives us the correct non-signed coordinate.

Consider that the transformation \(A\), being linear, scales all shapes in the original vector space by \(det(A)\), including the two parallelograms we created above!

The “new” parallelograms formed by the transformed basis vectors and \(\vec{b}\) can be computed using the columns of \(A\), which give us the locations of the new basis vectors, and the resulting vector \(\vec{b}\). Thus,

The parallelogram created by the basis vector \(\vec{v}\) and \(\vec{j} = [0 \quad 1]\) (whose area is \(x\)) is transformed to a new parallelogram created by the transformed basis vector \(\vec{b}\) and \(A_{(2)}\) (second column of \(A\)). Its area, which is the area of the original parallelogram scaled by \(det(A)\), is \(det([\vec{b} \quad A_{(2)}])\).

The paralellogram created by the basis vector \(\vec{i} = [1 \quad 0]\) and \(\vec{v}\) (whose area is \(y\)) is transformed to a new parallelogram created by the transformed basis vector \(A_{(1)}\) (first column of \(A\)) and \(\vec{b}\). Its area is \(det([A_{(1)} \quad \vec{b} ])\).

Retain the basis, input/output vector ordering used to calculate the original areas.

Given that both of these new areas are the original areas scaled by \(det(A)\), we can divide these new areas by that scale factor to arrive at our original areas – which, if you remember, are equal to the \(x,y\) coordinates of \(\vec{v}\).

In essence, given \(A \vec{v} = \vec{b}\)

\[v_i = \frac{det([\dotsc \quad A_{(i-1)} \quad b \quad A_{(i+1)} \dotsc ])}{det(A)}\]

Regarding the matrix in the numerator, it takes \(A\) and replaces the \(i\)th column corresponding to the \(i\)th coordinate with \(\vec{b}\). This matrix represents the transformed parallelepiped whose original volume was equal to the \(i\)th coordinate of interest. We divide out the new volume by the change in volume engendered by transformation \(A\) to arrive at the original volume and \(i\)th coordinate of interest.

Video 13 Change of basis

Earlier we considered that for the vector \(\vec{v} = [ 2 \quad 1 ]\), whose values represent a linear combination of basis vectors, its orientation in space was dependent on which basis vectors define that space. Any linear combination of our basis vectors \(\vec{i}, \vec{j} = [1 \quad 0], [0 \quad 1]\) in our space could map to a different location if the same linear combination were applied to Jennifer’s (different) basis vectors.

We need a way to see: a) Jennifer’s coordinates for a vector described using our coordinates b) our coordinates for a vector described using Jennifer’s coordinates

We can first think of a change of basis matrix \(A\) as encoding the location of Jennifer’s basis vectors in our space (that is, described using our coordinates).

\[A = \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix}\]

Her basis vectors, which would look like \([1 \quad 0], [0 \quad 1]\) to her, are \([2 \quad 1], [-1 \quad 1]\) in our coordinates.

\[ \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix} \]

It follows that the operation \(A \vec{v} = \vec{b}\) means \(\vec{v}\) in her coordinates corresponds to \(\vec{b}\) in our coordinates. Recall that \(\vec{v}\) is a linear combination of the relevant basis vectors in a space. If \(A_{(1)}, A_{(2)}\) are her basis vectors (described in our coordinates), and \(\vec{v}\) is the set of weights of the basis vectors, then the result \(\vec{b}\) reconstructs \(\vec{v}\) in our coordinates.

We can see that her basis vector \(\vec{i} = [1 \quad 0 ]\) corresponds to \([2 \quad 1 ]\) in our coordinate system. The same applies to any \([x \quad y ]\) vector.

Let’s say we’d like to reconstruct her coordinates \([-1 \quad 2 ]\) in our space.

\[ \begin{bmatrix} 2 & -1 \\ 1 & 1 \end{bmatrix} \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} -2 - 2 \\ -1 +2 \end{bmatrix} = [-4 \quad 1] \]

What if she wants to reconstruct a vector in our space using her coordinates?

Recall the general setup \(A \vec{v} = \vec{b}\), where \(\vec{b}\) is expressed in our coordinates, and \(\vec{v}\) expressed in hers.

For a given change of basis matrix \(A\), its inverse \(A^{-1}\) reverses this mapping. This change of basis, however, is expressed in her coordinates. Thus, the columns of \(A^{-1}\) are in her coordinates. They are where our basis vectors point to in her space. Thus, by the same logic above, \(A^{-1} \vec{b}\) is a linear combination of our basis vectors (described in her coordinates) that reconstructs \(\vec{b}\) in her space.

From \[ A^{-1} A \vec{v} = A^{-1} \vec{b}\]

we have \[\vec{v} = A^{-1} \vec{b} \]

Example: earlier we said that her basis vector \([1 \quad 0]\) looks like \([2 \quad 1]\) to us; thus, \(A^{-1}[2 \quad 1]\) gives her \([1 \quad 0]\). \[ \vec{x} = \begin{bmatrix} \frac{1}{3} & \frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} \end{bmatrix} [2 \quad 1 ] = [1 \quad 0] \] Similarly, our \([3 \quad 2]\) corresponds to her: \[ \vec{x} = \begin{bmatrix} \frac{1}{3} & \frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} \end{bmatrix} [3 \quad 2 ] = [ \frac{5}{3} \quad \frac{1}{3}] \] So far we’ve expressed vectors in our space as vectors in Jennifer’s space, and vice versa. How do we do the same for transformations?

Consider transformation \(T\) and change of basis matrix A \(A\). \(T\) represents a 90-degree rotation in our space and \(A\) expresses in our coordinates the basis vectors in her space. Our intent is to express the same 90-degree rotation in her coordinates.

\[ T = \begin{bmatrix} 0& 1 \\ -1 & 0 \end{bmatrix} \quad A = \begin{bmatrix} 2 & 1 \\ -1 & 1 \end{bmatrix} \] We start with \(\vec{v}\) in her space. Given \(\vec{v}\), \(A\vec{v} = \vec{b}\); \(\vec{b}\) reconstructs her \(\vec{v}\) in our coordinates.

We rotate \(\vec{b}\) in our space using \(T\).

Then… \[ T A \vec{v} = T \vec{b} \] Since \(T\vec{b}\) is still a vector described in our coordinates, we reconstruct this vector in her coordinates by multiplying by \(A^{-1}\) as we have done above. This gives us the transformed \(\vec{v}\) in her coordinates.

\[ A^{-1} T A \vec{v} = A^{-1} T \vec{b}. \] In this way, we can represent our rotation \(T\) (in our coordinates) on a vector \(\vec{b}\) in our space as a rotation \(A^{-1} T A\) (in her coordinates) on a vector \(\vec{v}\) in her space.

Summarily, Let \(A\) be a change of basis matrix whose columns are a new basis’ vectors described using standard coordinates. This \(A\) maps \(\vec{v}\) in the new basis to \(\vec{b}\) in the standard basis.

By the same token, \(A^{-1}\) is a change of basis matrix whose columns are the standard basis expressed in the new coordinates. This \(A^{-1}\) maps \(\vec{b}\) in the standard basis to \(\vec{v}\) in the new basis.

A useful heuristic: the output of multiplying \(A / A^{-1}\) and a vector will be expressed in the same coordinates as \(A / A^{-1}\)’s columns.

\(A \vec{v}\) is in standard coordinates (as are \(A\)’s columns) and \(A^{-1}\vec{v}\) is in new coordinates (as are \(A^{-1}\)’s columns).

Video 14 Eigenvectors and eigenvalues

Consider transformation matrix \(A\). For a given transformation, there may exist a non-zero \(\vec{v}\) that, when transformed, is stretched or squished by a non-trivial factor of \(\lambda\). Thus, applying a certain transformation \(A\) to a vector \(\vec{v}\) will preserve its span but scale \(\vec{v}\).

For a given \(A\), there could be zero, one, or multiple \(\vec{v}\) that fulfill the below condition. We refer to \(\vec{v}\) as an eigenvector of \(A\), and \(\lambda\) (the scaling factor) as the corresponding eigenvalue to that eigenvector.

\[ A \vec{v} = \lambda \vec{v} \]