Wikinotes

Maintainer: admin

A review of the course course material for MATH 133. Sections are organised based, to some extent, on Anderson's notes (available through WebCT).

Written by @dellsystem for Wikinotes back when it was still on MediaWiki, and converted into Markdown. There are probably lots of formatting errors, please contact her if you notice anything major (or feel free to fix it yourself!).

We can use matrices to solve linear systems
First, put the system in augmented matrix form
Get it in reduced row-echelon form
This is an equivalent, solvable system
THREE POSSIBLE CASES FOR THE NUMBER OF SOLUTIONS:
- No solutions (inconsistent)
  - all zeros in the coefficient matrix
  - non-zero in the column to the right
  - Occurs when you have more independent equations than variables
- Unique solution (consistent)
  - when number of unknowns == number of independent equations
- Infinite solutions (consistent)
  - Occurs when there are more variables than independent equations
  - Make one non-leading variable a parameter
  - And solve using that as a parameter
  - You end up with a vector equation

Switching rows
Multiplying a row by a constant
Adding or subtracting different rows

Row-echelon form:
- Any zero-only rows are at the bottom
- The first non-zero entity in a row is a leading 1
- Leading 1s are to the right of all leading 1s in above rows
Reduced row echelon form:
- Leading 1 is the only non-zero entry in column
- Not always possible, if there is no unique solution
- But it helps you see the solution
Rank: number of leading ones (variables)

A square matrix E is obtained by doing a single elementary row operation to an identity matrix (denoted I). This square matrix E is called the elementary matrix.
An elementary row operation comes in three flavours:
- Type I: Row Replacement (Example: Switching Row 1 with Row 2)
- Type II: Row Multiplication (Example: Multiplying Row 3 by a scalar)
  - Type III: Row Addition/Subtraction (Example: Adding 3*Row 1 to Row 3)
IMPORTANT: Every elementary matrix is invertible.
Any elementary row operation is equivalent to multiplying that matrix on the left by an elementary matrix
an elementary matrix is the identity matrix after having had that same operation performed on it
note that to get a COLUMN operation, you multiply the matrix on the right
thus, any series of row operations can be represented by elementary matrix
this also means that you can get any (invertible) matrix by doing elementary row operations on I
- also, any such matrix is invertible
example: let's say you have the matrix below and you want to write it as the product of elementary matrices.

$$A = \begin{pmatrix} -4 & -2 \\ 5 & 5 \end{pmatrix}$$

So work backwards - row reduce it to the identity matrix.
- (1) Row 1 = Row 1 + Row 2
  - (2) Row 2 = Row 2 - 5 * Row 1
- (3) Row 2 = Row 2 / -10
  - (4) Row 1 = Row 1 - 3 * Row 2

$$\begin{pmatrix} -4 & -2 \\ 5 & 5 \end{pmatrix} \to (1) \to \begin{pmatrix} 1 & 3 \\ 5 & 5 \end{pmatrix} \to (2) \to \begin{pmatrix} 1 & 3 \\ 0 & -10 \end{pmatrix} \to (3) \to \begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix} \to (4) \to \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$$

Now you get the identity matrix. To get the original matrix from the identity, do the reverse:
* (5) Row 1 = Row 1 + 3 * Row 2
* (6) Row 2 = Row 2 * -10
* (7) Row 2 = Row 2 + 5 * Row 1
- (8) Row 1 = Row 1 - Row 2

$$\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \to (5) \to \begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix} \to (6) \to \begin{pmatrix} 1 & 3 \\ 0 & -10 \end{pmatrix} \to (7) \to \begin{pmatrix} 1 & 3 \\ 5 & 5 \end{pmatrix} \to (8) \to \begin{pmatrix} -4 & -2 \\ 5 & 5 \end{pmatrix}$$

So the four elementary matrices that correspond to these row operations are:

$$\begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix} , \begin{pmatrix} 1 & 0 \\ 0 & -10 \end{pmatrix} , \begin{pmatrix} 1 & 0 \\ 5 & 1 \end{pmatrix} , \begin{pmatrix} 1 &-1 \\ 0 & 1 \end{pmatrix}$$

But you have to do it in REVERSE order.

$$\begin{pmatrix} 1 &-1 \\ 0 & 1 \end{pmatrix} \times \begin{pmatrix} 1 & 0 \\ 5 & 1 \end{pmatrix} \times \begin{pmatrix} 1 & 0 \\ 0 & -10 \end{pmatrix} \times \begin{pmatrix} 1 & 3 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} -4 & -2 \\ 5 & 5 \end{pmatrix}$$

If you want the get the inverse of the original matrix using elementary matrices:
- Use the steps you used to get the identity matrix from the original matrix
- So the four elementary matrices that correspond to those row operations are:

$$\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} , \begin{pmatrix} 1 & 0 \\ -5 & 1 \end{pmatrix} , \begin{pmatrix} 1 & 0 \\ 0 & -\frac{1}{10} \end{pmatrix} , \begin{pmatrix} 1 & -3 \\ 0 & 1 \end{pmatrix}$$

And since you have to do them in reverse order:

$$\begin{pmatrix} 1 & -3 \\ 0 & 1 \end{pmatrix} \times \begin{pmatrix} 1 & 0 \\ 0 & -\frac{1}{10} \end{pmatrix} \times \begin{pmatrix} 1 & 0 \\ -5 & 1 \end{pmatrix} \times \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} = \begin{pmatrix} -\frac{1}{2} & -\frac{1}{5} \\ \frac{1}{2} & \frac{2}{5} \end{pmatrix} = A^{-1}$$

Which gives you the inverse matrix. Magic
ANOTHER EXAMPLE, WITH A RECTANGULAR (NON-SQUARE) MATRIX
- Let $A = \begin{pmatrix} 2 & -1 & 3 \\ 0 & 1 & -2 \end{pmatrix}$
  - Let's find a matrix B so that BA is the reduced row-echelon form of A:
  - Here are the steps to get A into reduced row-echelon form:
    - (1) Row 1 = Row 1 + Row 2
    - (2) Divide row 1 by 2
    - RREF of A: $\begin{pmatrix} 1 & 0 & \frac{1}{2} \\ 0 & 1 & -2 \end{pmatrix}$
  - Now let's find the identity matrix we will need to use. Since A is 2 x 3, we need a 2 x 2 identity matrix.
  - So let's perform those same row operations on this identity matrix:
  - $B = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \to (1) \to \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} \to (2) \to \begin{pmatrix} \frac{1}{2} & \frac{1}{2} \\ 0 & 1 \end{pmatrix}$
  - $BA = \begin{pmatrix} \frac{1}{2} & \frac{1}{2} \\ 0 & 1 \end{pmatrix} \times \begin{pmatrix} 2 & -1 & 3 \\ 0 & 1 & -2 \end{pmatrix} = \begin{pmatrix} 1 & 0 & \frac{1}{2} \\ 0 & 1 & -2 \end{pmatrix}$
  - This is indeed the reduced row-echelon matrix. Again, magic

AX = 0
ALWAYS has at least the trivial solution, where all variables == 0
however, we want to know if there are non-trivial solutions too
this will occur if the number of variables > number of independent equations
note that these systems can never be inconsistent because there is always the trivial solution
FINDING BASIC SOLUTIONS
- Let's say you have one equation, -3x2 + x3 + x4 = 0 ==> x4 = 3x2 - x3
- Set x1 = s, x2 = t, x3 = u
- And you can rewrite x4 based on t and u.
- So you have 3 column vectors, multiples of s, t and u
- x1 = 1s + 0t + 0u
- x2 = 0s + 1t + 0u
- x3 = 0s + 0t + 1u
- x4 = 0s + 3t - 1u
- You can rewrite these as vectors

$$\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix} , \begin{pmatrix} 0 \\ 1 \\ 0 \\ 3 \end{pmatrix} , \begin{pmatrix} 0 \\ 0 \\ 1 \\ -1 \end{pmatrix}$$

Note that you can write the basic solution vectors in several ways
- For example, any scalar multiple of the above columns in any order would also be correct

Matrix: number of rows x number of columns
- multiplying two matrices: m x n * n x o
- resultant matrix: m * o
- if you have a 2 x 2 * 2 x 1, you can think of it in the column picture, for example:

$$\begin{pmatrix} 4 & 5 \\ 2 & 1 \end{pmatrix} \times \begin{pmatrix} 2 \\ 4 \end{pmatrix} = 2 \times \begin{pmatrix} 4 \\ 2 \end{pmatrix} + 4 \times \begin{pmatrix} 5 \\ 1 \end{pmatrix} = \begin{pmatrix} 28 \\ 8 \end{pmatrix}$$

there's really not much else to say about this, it all boils down to m x n * n x o ==> m x o

FINDING THE DETERMINANT: FROM FIRST PRINCIPLES
- only for square matrices
- Laplace expansions, for n x n matrix B:
- First get the minor matrix for each element of that row
  - Basically cross out that row and column; resulting elements = minor matrix
  - Find the determinant of that matrix (ad-bc)
- Now get the cofactor ... if row + column == odd, multiply minor det by -1
- Well you know how this works
RULES FOR DETERMINANTS OF SIMILAR MATRICES
- Let A be a n x n matrix, and let c be a scalar
- $\det(A^T) == \det(A)$
- $\det(A^{-1}) == \det(A)^{-1}$
- Multiply one row of A by c$\to c \times \det(A)$
- $\det(cA) == c^n \times det(A)$
- If A has two identical rows/columns: $\to \det(A) == 0$
- If A has a row or column consisting solely of zeros: $\to det(A) == 0$
- If A is a triangular matrix $\to \det(A) ==$ product of diagonal elements
- Switching two rows/columns of A $\to -\det(A)$
- Adding/subtracting different rows does not change $\det(A)$
- $\det(adj(A)) == \det(A)^{n-1}$
- $\det(A \times adj(A)) == \det(A)^n$
- $\det(AB) == \det(A) \times \det(B)$ (assuming AB exists)
  - if BA also exists, $\det(BA) == \det(AB)$
if you can transform a matrix into another using row operations
- you can get its determinant
- this is useful when you don't have the actual values, just the rows/etc

FINDING THE INVERSE
- if a determinant is 0, there is no inverse ... matrix is singular
  - Matrix of cofactors: replace each element with its cofactor (minor det * -1, maybe)
- Adjoint matrix: transpose of cofactor matrix
- Now divide the adjoint by the determinant ... that's the inverse
- You can prove that $A \times adj(A) \times \frac{1}{\det(a)} == I$ for 2x2
  - Just multiply out $A \times adj(A)$ and you get $\det(A) \times I$
- You can ALSO get the inverse using an augmented matrix, put the identity on the right
- And just do row ops until you get the identity on the left
- The resulting matrix on the right is the inverse
RULES FOR INVERSES
- $(AB)^{-1} == B^{-1} \times A^{-1}$
  - proof: $A \times B \times B^{-1} \times A^{-1} = A \times A^{-1} == I$
  - IT MAKES SENSE
- $(A^{T})^{-1} == (A^{-1})^T$
- $(cA^{-1}) = c^{-1} \times A^{-1}$
- I is invertible; $I^{-1} == I$
- if A is invertible so is $A^{-1}$, and $(A^{-1})^{-1} == A$
- If A is invertible so is $A^k$, and $(A^k)^{-1} = (A^{-1})^k$
CRAMER'S RULE
- a way to solve for X if there's a unique solution
- only works when there are n independent equations and n unknowns
- so you have AX = B, and let's say you 3 variables: $x_1, x_2, x_3$
- to get $x_1$, replace the 1st column of A by B
  - $x_1 = \frac{\det(A_1B)}{\det(A)}$
- to get $x_2$, replace the 2nd column of A by B
  - $x_2 = \frac{\det(A_2B)}{\det(A)}$
- and so on for x3

Based on homogenous systems, AX = 0
An eigenvalue is a value of $A \mathbf{X} = \lambda \mathbf{X}$
- where A is a square matrix, X is a vector (eigenvector - we'll solve that later), $\lambda$ is a scalar
so $A \mathbf{X} - \lambda \mathbf{X} == 0$
therefore $(A - \lambda I) \mathbf{X} == 0$
- we need to find the non-trivial solutions for this
- so set $\det(A-\lambda \mathbf{I}) = 0$ so we can't find the inverse to obtain the trivial solution of X
characteristic polynomial: $\det(A-\lambda \mathbf{I}) = 0$
- the roots of that equation ^ are the eigenvalues
- if the polynomial is a quadratic equation:
  - if $b^2-4ac > 0$, 2 eigenvalues
  - if $b^2-4ac = 0$, 1 eigenvalue
  - The number of eigenvalues is equal to the number of rows/columns (0 is a non-trivial possibility)
  - These eigenvalues are the ONLY values of $\lambda$ that make $\det(A-\lambda \mathbf{I}) == 0$
  - This of course implies that $A-\lambda \mathbf{I}$ is not invertible for $\lambda = $eigenvalues
  - And that it is invertible for all other values of $\lambda$
algebraic muliplicity: number of times that eigenvalue shows up in the polynomial
geometric multiplicity: number of linearly independent vectors in the solution for that eigenvalue
- basically, plug that eigenvalue in the original matrix
- row reduce that
- use parameters to get one or more solution vectors
- these are the eigenvectors for this eigenvalue
- how many solution vectors needed? that's the geometric multiplicity
- geometric always <= algebraic multiplicity
trace: sum of elements on main diagonal of matrix (upper left to bottom right)
- applicable for all square matrices
PROPERTIES OF EIGENVALUES
- Let A be a n x n square matrix with n eigenvalues, $\lambda_1, \lambda_2 ... \lambda_n$
- Trace of $A = \lambda_1 + \lambda_2 + ... + \lambda_n$
- Determinant of $A = \lambda_1 \times \lambda_2 \times ... \times \lambda_n$
- Eigenvalues of $A^k = \lambda_1^k, \lambda_2^k ... \lambda_n^k$
EXAMPLE
- Let $A = \begin{pmatrix}1 & 1 & 3 \\ -2 & -1 & -2 \\ -1 & -1 & -3 \end{pmatrix}$
- Find the characteristic polynomial (multiply by -1 to make it easier): $\lambda I - A = \begin{pmatrix} \lambda -1 & -1 & -3 \\ 2 & \lambda + 1 & 2 \\ 1 & 1 & \lambda + 3 \end{pmatrix} $
- $P(\lambda) = (\lambda+2)(\lambda+1)\lambda = 0$ (after simplifying)
- So we find the eigenvalues: $\lambda_1 = -2, \lambda_2 = -1, \lambda_3 = 0$
- For $\lambda_1 = -2, \lambda I - A = \begin{pmatrix} -3 & -1 & -3 \\ 2 & -1 & 2 \\ 1 & 1 & 1 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{pmatrix} \Delta v_1 = \begin{pmatrix}-1 \\ 0 \\ 1 \end{pmatrix}t$
- For $\lambda_2 = -1, \lambda I - A = \begin{pmatrix} -2 & -1 & -3 \\ 2 & 0 & 2 \\ 1 & 1 & 2 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0\end{pmatrix} \Delta v_1 = \begin{pmatrix}-1 \\ -1 \\ 1 \end{pmatrix}t$
- For $\lambda_3 = 0, \lambda I - A = \begin{pmatrix} 1 & -1 & -3 \\ 2 & 1 & 2 \\ 1 & 1 & 3 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & -1 \\ 0 & 1 & 4 \\ 0 & 0 & 0\end{pmatrix} \Delta v_1 = \begin{pmatrix}1 \\ -4 \\ 1 \end{pmatrix}t$
- So we have three eigenvalues, each with geometric and algebraic multiplicity of 1.

a matrix is diagonalisable when a.m. = g.m. for every eigenvalue
only for square matrices obviously
Let D be the diagonal matrix (a multiple of I)
occurs when there exists an invertible matrix P such that $P^{-1}AP = D$
Applications:
- $A^n = PD^nP^{-1}$
- so, useful for finding $A^n$
- since for each element in $D^n$ you just put the element of D to the power of n
HOW DO YOU FIND IT?
- first get the eigenvalues for matrix A
- then find the eigenvector for each eigenvalue
- If you have a 3x3 matrix, you will need 3 eigenvalues (distinct or not, doesn't matter)
- and thus 3 eigenvectors
- fill each element of the diagonal of D with an eigenvalue
- fill each column of P with each eigenvector (in order ^)
- then find the inverse of P (however you want)
- and solve for $A^n$
AN EXAMPLE:
- Let's continue with the matrix we used above, in the [[MATH 133/Review#Eigenvalues|eigenvalues]] section
- The matrix D is given by: $D = \begin{pmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{pmatrix} = \begin{pmatrix} -2 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 0 \end{pmatrix}$
- The matrix P is given by $P = \begin{pmatrix} v_1 \\ v_2 \\ v_3 \end{pmatrix} = \begin{pmatrix} -1 & -1 & 1 \\ 0 & -1 & -4 \\ 1 & 1 & 1 \end{pmatrix}$
- We can then calculate the inverse of P: $P^{-1} = \begin{pmatrix} \frac{3}{2} & 1 & \frac{5}{2} \\ -2 & -1 & -2 \\ \frac{1}{2} & 0 & \frac{1}{2} \end{pmatrix}$

Let u and v be two vectors in $R^3: \mathbf{u} = \begin{pmatrix} x_1 \\ y_1 \\ z_1 \end{pmatrix}, \mathbf{v} = \begin{pmatrix} x_2 \\ y_2 \\ z_2 \end{pmatrix}$

$\mathbf{u} = \mathbf{v}$ iff $x_1 = x_2, y_1 = y_2, z_1 = z_2$ (i.e. the vectors have the same magnitude and direction)
The magnitude of the vector is given by $\begin{Vmatrix} \mathbf{v} \end{Vmatrix} = \sqrt{x_1^2 + y_1^2 + z_1^2}$
$\mathbf{v} = 0$ iff $\begin{Vmatrix} \mathbf{v} \end{Vmatrix} = 0$ (that is, if all elements are zero)
$\begin{Vmatrix} a\mathbf{v} \end{Vmatrix} = a \times \begin{Vmatrix} \mathbf{v} \end{Vmatrix}$ for any scalar a
For u and v to be parallel, they would have to be scalar multiples of each other
Taking u and v as points, the distance vector from u to v is given by: $\overrightarrow{\mathbf{u} \mathbf{v}} = \mathbf{v} - \mathbf{u} = \begin{pmatrix} x_2 - x_1 \\ y_2 - y_1 \\ z_2 - z_1 \end{pmatrix}$
The distance between the points u and v is given by:$\begin{Vmatrix} \overrightarrow{\mathbf{u} \mathbf{v}} \end{Vmatrix} = \sqrt{(x_1 - x_2)^2 + (y_1 - y_2)^2 + (z_1 - z_2)^2}$

Dot-product (or scalar product) of two vectors:
- $\mathbf{u} \bullet \mathbf{v} = \mathbf{u}^T \times \mathbf{v} = (x_1, y_1, z_1) \times \begin{pmatrix} x_2 \\ y_2 \\ z_2 \end{pmatrix} = x_1 x_2 + y_1 y_2 + z_1 z_2$
- Order does not matter; $\mathbf{u} \bullet \mathbf{v} = \mathbf{v} \bullet \mathbf{u}$
- $\mathbf{u} \bullet \mathbf{u} = x_1^2 + y_1^2 + z_1^2 = \begin{Vmatrix} \mathbf{u} \end{Vmatrix}^2$
- (av) · w = a(v · w) = v · (aw) fix formatting later
- v · (u + w) = v · u + v · w
- v · (u − w) = v · u − v · w
Finding the angle between the two vectors u and v:
- Either position them so that they have a common tail, or make them lines
- Let $\phi$ be the angle between the two vectors
- $\cos \phi = \frac{\mathbf{u} \bullet \mathbf{v}}{\begin{Vmatrix}\mathbf{u}\end{Vmatrix} \begin{Vmatrix} \mathbf{v} \end{Vmatrix}}$
- If the two vectors are perpendicular, then $\cos \phi = 0; \Delta \mathbf{u} \bullet \mathbf{v} = 0$
Projecting vectors onto lines
- Let's say you have a vector $\mathbf{d}$ in some random direction, any direction is fine.
- Let the vector $\mathbf{v_1}$ be a vector parallel to d, $\mathbf{v_2}$ be a vector that is perpendicular (orthogonal) to d.
- Now let's say that you have a vector $\mathbf{v}$ in some random direction - again, any will do.
- You want to rewrite $\mathbf{v}$ as the sum of the component vectors $\mathbf{v_1}$ and $\mathbf{v_2}$
  - Note: this is always possible (given that all vectors are non-zero).
  - Remember that if you reverse the direction of a vector, it's still parallel to that vector.
  - The vectors $\mathbf{v_1}$ and $\mathbf{v_2}$ may be zero - this happens if d is parallel to or perpendicular to v.
- Essentially what you are doing is resolving the vector into its components - one component along the vector d, one perpendicular to d.
- Here's the derivation: start with $\begin{Vmatrix} \mathbf{v_1} \end{Vmatrix} = \cos \phi \times \begin{Vmatrix} \mathbf{v1} \end{Vmatrix}$ (trivial - draw a diagram if unsure why)
  - $\begin{Vmatrix} \mathbf{v_1} \end{Vmatrix} = \cos \phi \times \begin{Vmatrix} \mathbf{v} \end{Vmatrix}$ where $\phi$ is the angle between the vectors d and $v$, in the direction d
  - Multiply and divide by $\begin{Vmatrix} \mathbf{d} \end{Vmatrix}$, so you get $\begin{Vmatrix} \mathbf{v_1} \end{Vmatrix} = \frac{\begin{Vmatrix} \mathbf{v} \end{Vmatrix} \begin{Vmatrix} \mathbf{d} \end{Vmatrix} \cos \phi}{\begin{Vmatrix} \mathbf{d} \end{Vmatrix}}$
  - Since the top is just equal to the dot product, as $\begin{Vmatrix} \mathbf{v} \end{Vmatrix} \begin{Vmatrix} \mathbf{d} \end{Vmatrix} \cos \phi = \mathbf{v} \bullet \mathbf{d}$, we can simplify this to $\begin{Vmatrix} \mathbf{v_1} \end{Vmatrix} = \frac{\mathbf{v} \bullet \mathbf{d}} {\begin{Vmatrix} \mathbf{d} \end{Vmatrix}}$
  - Now, we have only the magnitude of $v_1$. To get the vector in the desired direction, multiply it by a unit vector in the direction of d
  - Such a unit vector is found by multiplying by d then dividing by the length of d
  - So we get the formula $\mathbf{v_1} = \bigg( \frac{\mathbf{v} \bullet \mathbf{d}} {\begin{Vmatrix} \mathbf{d} \end{Vmatrix}^2} \bigg) \mathbf{d}$
  - This can also be written as $proj_\mathbf{d}(\mathbf{v})$
- Now, to get $v_2$, we just have to do $\mathbf{v_2} = \mathbf{v} - \mathbf{v_1}$
- EXAMPLE PROJECTION:
  - Let $\mathbf{v} = \begin{pmatrix} -3 \\ 4 \\ 9 \end{pmatrix}, \mathbf{d} = \begin{pmatrix} -1 \\ 3 \\ 0 \end{pmatrix}$ and we want to find $proj_\mathbf{d}(\mathbf{v})$
  - $\mathbf{v} \bullet \mathbf{d} = 15$
  - $\begin{Vmatrix} \mathbf{d} \end{Vmatrix}^2 = \sqrt{10}^2 = 10$
  - $\mathbf{v_1} = \frac{3}{2} \mathbf{d} = \begin{pmatrix} -1.5 \\ 4.5 \\ 0 \end{pmatrix}$
  - $\mathbf{v_2} = \mathbf{v} - \mathbf{v_1} = \begin{pmatrix} -3 \\ 4 \\ 9 \end{pmatrix} - \begin{pmatrix} -1.5 \\ 4.5 \\ 0 \end{pmatrix} = \begin{pmatrix} -1.5 \\ 0.5 \\ 9 \end{pmatrix}$
  - Again, magic!

Let's take P as a point in $R^3$ with coordinates $(x_0, y_0, z_0)$
Position vectors: the position vector is the vector from the origin to point P.
- $\overrightarrow{O}{P} = \begin{pmatrix} x_0 \\ y_0 \\ z_0 \end{pmatrix}$
Vector equations of lines:
- Line passing through two points: Let Q be another point in $R^3$ with coordinates $(x_1, y_1, z_1)$
  - The vector equation of a line passing through those two points would be:
  - $\begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} x_0 \\ y_0 \\ z_0 \end{pmatrix} + \begin{pmatrix} x_1 - x_0 \\ y_1 - y_0 \\ z_1 - z_0 \end{pmatrix}t$
  - This ensures that P is on the line (when t = 0) and that Q is on the line (when t = 1).
- Line passing through one point and parallel to a given direction vector $\mathbf{d} = \begin{pmatrix} a \\ b \\ c \end{pmatrix}$:
  - Let's say we have a line parallel to $\mathbf{d}$ that passes through the point P
  - The vector equation of that line would be given by $P + t \mathbf{d}$
  - Specifically, $\begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} x_0 \\ y_0 \\ z_0 \end{pmatrix} + \begin{pmatrix} a \\ b \\ c \end{pmatrix}t$
- The equivalent scalar equations to the above would be:
  - $x = x_0 + at$
  - $y = y_0 + bt$
  - $z = z_0 + ct$
Shortest distance from a point to a line, with a numerical example:
- L = line through the point (6, 1, -3), with direction vector $\mathbf{d} = \begin{pmatrix} -1 \\ 3 \\ 0 \end{pmatrix}$
- First write the vector equation of the line: $\begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 6 \\ 1 \\ -3 \end{pmatrix} + \begin{pmatrix} -1 \\ 3 \\ 0 \end{pmatrix}t = \begin{pmatrix} 6 - t \\ 1 + 3t \\ -3 \end{pmatrix}$
- The shortest distance from that line to a point at (3, 5, 6) would be a straight line, perpendicular to the line
- Let's find the equation of that line, as the line passing through two points - one point on line L, the other point being the point (3, 5, 6)
- Perpendicular line: $\mathbf{v} = \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} 6 - t - 3 \\ 1 + 3t - 5 \\ -3 - 6 \end{pmatrix} = \begin{pmatrix} 3 - t \\ -4 + 3t \\ -9 \end{pmatrix}$
- Now let's make this proportional to the direction vector $\mathbf{d}$. Solve for t such that the dot-product is zero.
- $\mathbf{v} \bullet \mathbf{d} = \begin{pmatrix} 3 - t \\ -4 + 3t \\ -9 \end{pmatrix} \bullet \begin{pmatrix} -1 \\ 3 \\ 0 \end{pmatrix} = 10t - 15 = 0$
- Therefore $t = 1.5$ so the vector that begins at the line, is perpendicular to the line and ends at the point (3, 5, 6) is: $\mathbf{v} = \begin{pmatrix} 3 - 1.5 \\ -4 + 4.5 \\ -9 \end{pmatrix} = \begin{pmatrix} 1.5 \\ 0.5 \\ -9 \end{pmatrix}$
- The length of this vector is given by its magnitude: $\begin{Vmatrix} \mathbf{v} \end{Vmatrix} = \sqrt{1.5^2 + 0.5^2 + (-9)^2} = \sqrt{83.5}$
- Note: other methods are possible, too; you could solve this using line projection, or by getting a different equation, etc
Shortest distance between two lines in $R^3$:
- The lines have the vector equations $L_1: \begin{pmatrix} 3 - 2t \\ 2 + t \\ 5 - 3t \end{pmatrix}, L_2: \begin{pmatrix} -4 + s \\ 3 - 2s \\ 3s \end{pmatrix}$
- So we first find the equation of a line joining two those lines. $\overrightarrow{L_1 L_2} = \begin{pmatrix} -4 + s - (3 - 2t) \\ 3 - 2s - (2 + t) \\ 3s - (5 - 3t) \end{pmatrix} = \begin{pmatrix} -7 + s + 2t \\ 1 - 2s - t \\ 3s - 5 + 3t \end{pmatrix}$
- Now, we note that this line is perpendicular to the direction vectors of BOTH lines
- We could just find the cross product of both vectors and solve for all variables, but here's an easier way:
  - $\overrightarrow{L_1 L_2} \perp L_1 \to \overrightarrow{L_1 L_2} \bullet L_1 = \begin{pmatrix} -7 + s + 2t \\ 1 - 2s - t \\ 3s - 5 + 3t \end{pmatrix} \bullet \begin{pmatrix} -2 \\ 1 \\ -3 \end{pmatrix} = 30 - 13s - 14t = 0$
  - $\overrightarrow{L_1 L_2} \perp L_2 \to \overrightarrow{L_1 L_2} \bullet L_2 = \begin{pmatrix} -7 + s + 2t \\ 1 - 2s - t \\ 3s - 5 + 3t \end{pmatrix} \bullet \begin{pmatrix} 1 \\ -2 \\ 3 \end{pmatrix} = -24 + 14s + 13t = 0$
  - Solving for s and t we get $s = -2, t = 4$
  - So the point on $L_1$ is $\begin{pmatrix} 3 - 2(4) \\ 2 + (4) \\ 5 - 3(4) \end{pmatrix} = \begin{pmatrix} -5 \\ 6 \\ -7 \end{pmatrix}$ and the point on $L_2$ is $\begin{pmatrix} -4 + (-2) \\ 3 - 2(-2) \\ 3(-2) \end{pmatrix} = \begin{pmatrix} -6 \\ 7 \\ -6 \end{pmatrix}$
  - The line segment joining these points is given by $\begin{pmatrix} -7 + s + 2t \\ 1 - 2s - t \\ 3s - 5 + 3t \end{pmatrix} = \begin{pmatrix} -1 \\ 1 \\ 1 \end{pmatrix}$ which has a length of $\sqrt{(-1)^2 + 1^2 + 1^2} = \sqrt{3}$
  - M A G I C

Let's say we have the point Q $(x_0, y_0, z_0)$ and a vector $\mathbf{n} = \begin{pmatrix} a \\ b \\ c \end{pmatrix}$.
- There is exactly one plane containing the point Q and perpendicular to the vector n. So the vector n is the normal vector to that plane.
- Let any other point on the plane be given by P(x, y, z)
- So the equation of the plane is given by the condition that the distance between a point on the plane and the point Q is perpendicular to n.
- In other words: $(P - Q) \perp \mathbf{n}; \to (P - Q) \bullet \mathbf{n} = 0 \to \begin{pmatrix} x - x_0 \\ y - y_0 \\ z - z_0 \end{pmatrix} \bullet \begin{pmatrix} a \\ b \\ c \end{pmatrix} = 0$
- Therefore, $a(x - x_0) + b(y - y_0) + c(z - z_0) = 0 \to ax + by + cz = d$ (where d is a constant)
THE DISTANCE BETWEEN A POINT AND A PLANE, given a point Q (1, 5, 5) and the plane 2x - y + 3z = 5.
- The shortest distance between a point on the plane P and the point (1, 5, 5) would be along the normal vector to that plane, $\mathbf{n} = \begin{pmatrix} 2 \\ -1 \\ 3 \end{pmatrix}t$.
- So $P - Q = \begin{pmatrix} 1 - x \\ 5 - y \\ 5 - z \end{pmatrix} = \begin{pmatrix} 2 \\ -1 \\ 3 \end{pmatrix}t$
- From this we can get parametric equations for t: $x = 1 - 2t; y = 5 + t; z = 5 - 3t$
- By substituting this into the equation of a plane we can solve for t: $14t = 7 \to t = \frac{1}{2}$
- Now we can get the coordinates of the point closest to point Q: $(x, y, z) \to (0, 5.5, 3.5)$
- So the distance vector from this point P to point Q is $Q - P = \begin{pmatrix} 1 - 0 \\ 5 - 5.5 \\ 5 - 3.5 \end{pmatrix} = \begin{pmatrix} 1 \\ -0.5 \\ 1.5 \end{pmatrix}$
- The length of this vector is $\begin{Vmatrix} Q - P \end{Vmatrix} = \sqrt{1^2 + (-0.5)^2 + 1.5^2} = \sqrt{3.5} = \frac{1}{2} \sqrt{14}$

Let's take two vectors in $R^3: \mathbf{u} = \begin{pmatrix} a \\ b \\ c \end{pmatrix}, \mathbf{v} = \begin{pmatrix} d \\ e \\ f \end{pmatrix}$
The cross product of these two vectors, denoted $\mathbf{u} \times \mathbf{v}$, is also a vector in $R^3$
- To get the cross-product, put those two vectors side by side, like this: $\begin{pmatrix}a & d \\ b & e \\ c & f \end{pmatrix}$
- To get the entry in the first row, cross out the first row and take the determinant of the remaining two rows: $\det \begin{pmatrix} b & e \\ c & f \end{pmatrix}$
- To get the entry in the second row, cross out the second row and take the ''negative'' \determinant of the remaining two rows: $\det \begin{pmatrix} a & d \\ c & f \end{pmatrix}$
- Same deal for the third row.
- A numerical example: $\mathbf{u} = \begin{pmatrix} 2 \\ 1 \\ -3 \end{pmatrix}, \mathbf{v} = \begin{pmatrix} 1 \\ 2 \\ 1 \end{pmatrix}$
- Element in first row: $\det \begin{pmatrix} 1 & 2 \\ -3 & 1 \end{pmatrix} = 1 \times 1 - 2 \times (-3) = 7$
- Element in second row: $-\det \begin{pmatrix} 2 & 1 \\ -3 & 1 \end{pmatrix} = -(1 \times 1 - 1 \times (-3)) = -5$
- Element in third row: $\det \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} = 2 \times 2 - 1 \times 1 = 3$
- So the cross-product is $\begin{pmatrix} 7 \\ -5 \\ 3 \end{pmatrix}$
You could also get the cross-product by writing it as a 3 x 3 matrix, with the top row consisting of the standard basis vectors for $R^3 (i, j, k)$
- Then just take the determinant of that matrix
- $\mathbf{u} \times \mathbf{v} = \det \begin{pmatrix} i & j & k \\ 2 & 1 & -3 \\ 1 & 2 & 1 \end{pmatrix} = i \det \begin{pmatrix} 1 & 2 \\ -3 & 1 \end{pmatrix} - j \det \begin{pmatrix} 2 & 1 \\ -3 & 1 \end{pmatrix} + k \det \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} = 7i - 5k + 3j = \begin{pmatrix} 7 \\ -5 \\ 3 \end{pmatrix}$
Properties of the cross-product:
- If you cross any vector with itself, OR with the zero vector, you get 0
- $\mathbf{u} \times \mathbf{v} = - \mathbf{v} \times \mathbf{u}$
- (av) × w = a(v × w) = v × (aw) where a is a scalar
- Associative: v × (u + w) = (v × u) + (v × w)
- (u + w) × v = (u × v) + (w × v)
- The Lagrange Identity: $ \begin{Vmatrix} u \times v \end{Vmatrix}^2 = \begin{Vmatrix} u \end{Vmatrix}^2 \begin{Vmatrix} v \end{Vmatrix}^2 - (u \bullet v)^2$
- If you have three vectors u, v and w, then $u \bullet (v \times w) = \det \begin{pmatrix} u & v & w \end{pmatrix}$
- Let $\theta$ be the angle between the two vectors:
  - $\begin{Vmatrix} u \times v \end{Vmatrix} = \begin{Vmatrix} u \end{Vmatrix} \begin{Vmatrix} v \end{Vmatrix} \sin \theta$ (incidentally, this also gives the area of the parallelogram defined by the two vectors)
  - From the above it follows that if the vectors are parallel the cross-product is the zero vector (although there are easier ways to determine that)
  - And if the vectors are perpendicular then $\begin{Vmatrix} u \times v \end{Vmatrix} = \begin{Vmatrix} u \end{Vmatrix} \begin{Vmatrix} v \end{Vmatrix}$
  - Example of the area of the parallelogram between two vectors in $R^2$:
  - The vectors defining two adjacent sides of the parallelogram are $u = \begin{pmatrix} 1 \\ 4 \end{pmatrix}, v = \begin{pmatrix} 8 \\ 6 \end{pmatrix}$
  - $ u \times v = \det \begin{pmatrix} 1 & 8 \\ 4 & 6 \end{pmatrix} = 1(6) - (8)(4) = -26$
  - So the area is 26 units squared. Note that the cross-product of two vectors in $R^2$ is a scalar
  - You could also do it by making them both vectors in R^3: $u = \begin{pmatrix} 1 \\ 4 \\ 0 \end{pmatrix}, u = \begin{pmatrix} 8 \\ 6 \\ 0\end{pmatrix}$
  - This would give you the vector $ \begin{pmatrix} 0 \\ 0 \\ -26 \end{pmatrix}$ which clearly has a length of 26 units. So you get the same answer in the end.
What is the main point of the cross-product? Well, it gives you a vector that is perpendicular to both u and v.
- For example, let's say you want to find the equation of a plane that passes through three points: A (6, -1, 1); B (1, 0, 0); C (21, 3, 2).
  - First we need two vectors that are part of the plane. Any of $\overrightarrow{A B}, \overrightarrow{A C}, \overrightarrow{B C}$ or the reverse would work.
  - Let's use $\overrightarrow{A B}$ and $\overrightarrow{A C}$: $\overrightarrow{A B} = \begin{pmatrix} -5 \\ 1 \\ -1 \end{pmatrix}, \overrightarrow{A C} = \begin{pmatrix} 15 \\ 4 \\ 1 \end{pmatrix}$
  - Now, let's find the normal vector for the plane. This vector would be perpendicular to both $\overrightarrow{A B}$ and $\overrightarrow{A C}$, so let's find it using their cross product.
  - $\overrightarrow{A B} \times \overrightarrow{A C} = \begin{pmatrix} -5 \\ 1 \\ -1 \end{pmatrix} \times \begin{pmatrix} 15 \\ 4 \\ 1 \end{pmatrix} = \begin{pmatrix} 5 \\ -10 \\ -35 \end{pmatrix}$
  - Since the equation of a plane can be written based on the normal vector, the equation is $5x - 10y - 35z = k$ where k is a constant.
  - To find k, we can take any point on the plane and substitute in its coordinates to solve for k. Let's use point A.
  - $5(6) - 10(-1) - 35(1) = 5 \to k = 5$
  - So the equation of the plane is $5x - 10y - 35z = 5$ or $x - 2y - 7z = 1$.
- Or, let's say you want to find the equation of a plane that contains a point A and a line.
  - So take two points on that line (any two will do, pick nice and easy numbers to make it easier), name them B and C, and do the steps above.

We can perform transformations on vectors in $R^2$ - rotations, reflections, translations etc
Such transformations can often be performed by multiplying a vector by a 2 by 2 matrix
In function format, $T( \mathbf{v} ) = A \mathbf{v}$ where A is the matrix and T is the transformation function
- Example matrix transformations, on the vector $\mathbf{v} = \begin{pmatrix} x \\ y \end{pmatrix}$
  - The identity matrix I: $\to T( \mathbf{v} ) = \mathbf{v}$ (that is, the matrix does not change)
  - The zero matrix: $\to T( \mathbf{v} ) = \overrightarrow{0}$
  - The matrix $\begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}: T \bigg( \begin{pmatrix} x \\ y \end{pmatrix} \bigg) = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}= \begin{pmatrix} x \\ -y \end{pmatrix}$ which is a reflection across the y-axis
  - The matrix $\begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}: T \bigg( \begin{pmatrix} x \\ y \end{pmatrix} \bigg) = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}= \begin{pmatrix} -y \\ x \end{pmatrix}$ - counterclockwise rotation of 90 degrees about the origin, a special case of the below
  - Counterclockwise rotation about the origin by angle $\theta : \begin{pmatrix} \cos \theta & - \sin \theta \\ \sin \theta & \cos \theta \end{pmatrix}$
  - Projection of a vector onto a line:
    - Let's say you have a line given by y=mx. The direction vector of this line would therefore be $\mathbf{d} = \begin{pmatrix} 1 \\ m \end{pmatrix}$
    - So the transformation T would project vector $\mathbf{v} = \begin{pmatrix} x \\ y \end{pmatrix}$ onto the vector d:
    - $T(v) = proj_d(v) = \bigg( \frac{\mathbf{v} \bullet \mathbf{d}} {\begin{Vmatrix} \mathbf{d} \end{Vmatrix}^2} \bigg) \mathbf{d} = \bigg( \frac{x + ym} {1 + m^2} \bigg) \begin{pmatrix} 1 \\ m \end{pmatrix} = \frac{1}{1+m^2} \begin{pmatrix} x + my \\ mx + m^2y \end{pmatrix} = \frac{1}{1+m^2} \begin{pmatrix} 1 & m \\ m & m^2 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}$
  - On the other hand, if you wanted to reflect a vector across the line y=mx, the matrix transformation would be:
    - $\frac{1}{1+m^2} \begin{pmatrix} 1 - m^2 & 2m \\ 2m & m^2 - 1 \end{pmatrix}$, do the derivation yourself as an exercise (sorry can't be bothered)
Linear transformations: all matrix transformations are linear and vice versa (same thing)
- Must have the following properties:
- $T(c \mathbf{v}) = cT( \mathbf{v}), v \in R^2, c$ is a scalar
- $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T( \mathbf{v}), u \in R^2, v \in R^2$
- Converting into matrix forms from other forms:
  - Let's say you have $T \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 5x + 4y \\ -7x + y \end{pmatrix}$
  - The matrix associated with T would be $\begin{pmatrix} 5 & 4 \\ -7 & 1 \end{pmatrix}$
  - Or, let's say you're given $T \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 3 \\ -2 \end{pmatrix}$ and $T \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} -6 \\ -5 \end{pmatrix}$ ... then the matrix is $\begin{pmatrix} 3 & -6 \\ -2 & -5 \end{pmatrix}$
  - Or, say you're given $T \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 5 \\ 1 \end{pmatrix}, T \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ -3 \end{pmatrix}. T \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 10 \\ 0 \\ 0 \end{pmatrix}$ ... then $T \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 10 \\ 5 \\ -2 \end{pmatrix}$
  - OR, say you're given $T \begin{pmatrix} 1 \\ 2 \end{pmatrix}, T \begin{pmatrix} 2 \\ 3 \end{pmatrix}$
    - And you want to find the matrix. So get $T\begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $T\begin{pmatrix} 0 \\ 1 \end{pmatrix}$ as linear combinations of the basis vectors above.
    - $\begin{pmatrix} 1 \\ 0 \end{pmatrix} = -3 \begin{pmatrix} 1 \\ 2 \end{pmatrix} + 2 \begin{pmatrix} 2 \\ 3 \end{pmatrix}. \begin{pmatrix} 0 \\ 1 \end{pmatrix} = 2 \begin{pmatrix} 1 \\ 2 \end{pmatrix} - 1 \begin{pmatrix} 2 \\ 3 \end{pmatrix}$
    - So $T \begin{pmatrix} 1 \\ 0 \end{pmatrix} = -3 \begin{pmatrix} 2 \\ -5 \end{pmatrix} + 2 \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 0 \\ 23 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \end{pmatrix} = 2 \begin{pmatrix} 2 \\ -5 \end{pmatrix} - 1 \begin{pmatrix} 3 \\ 4 \end{pmatrix} = \begin{pmatrix} 1 \\ -14 \end{pmatrix}$
    - So the matrix is $\begin{pmatrix} 0 & 1 \\ 23 & -14 \end{pmatrix}$
- Things that make a transformation NOT LINEAR:
  - Adding a constant
  - Making something a constant
  - Exponents, powers, logarithms etc
Isometric transformations: where the length of the vector does not change
Effect of a matrix transformation on unit squares:
- Let A be a matrix inducing the linear transformation $T: R^2 \to R^2$
- The image, under the transformation T, of the unit square is a paralellogram with sides T(i) and T(j), and area $\det (A)$

S and T are both transformations; the composite $S \circ T (v) = S(T(v)), v \in R^2$
- So first do T, then take the resulting vector and do S on that
- Obviously $S \circ T (v) \neq T \circ S$ unless otherwise specified
- If S is associated with the matrix A, and T with B, then $S \circ T (v)$ is associated with AB.
- EXAMPLE: composite of rotations:
  - $R_\theta \circ R_\phi = R_\phi \circ R_\theta = R_{\theta + \phi}$
  - $A_\theta A_\phi = \begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix} \begin{pmatrix} \cos \phi & -\sin \phi \\ \sin \phi & \cos \phi \end{pmatrix} = \begin{pmatrix} \cos \theta \cos \phi - \sin \theta \sin \phi & -\cos \theta \sin \phi - \sin \theta \cos \phi \\ \sin \theta \cos \phi + \cos \theta \sin \phi & -\sin \theta \sin \phi + \cos \theta \cos \phi \end{pmatrix}$
  - Since $A_\theta A_\phi = A_{\theta + \phi}$, the above is equal to $\begin{pmatrix} \cos (\theta + \phi) & -\sin (\theta + \phi) \\ \sin (\theta + \phi) & \cos (\theta + \phi) \end{pmatrix}$, which gives us the sum formulae for sine and cosine (trig identities)
  - Pretty cool
- ANOTHER EXAMPLE: show that the composite of two reflections - one across y=x, one across y=0 - is a different rotation depending on their order
  - Let rotation S be across the line y = x. It is induced by the matrix $A = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}$
  - Let rotation T be across the x-axis. It is induced by the matrix $B = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix}$
  - $S \circ T \to AB = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} = \begin{pmatrix} 0 & -1 \\ 1 & 0 \end{pmatrix}$ which is a 90 degree counterclockwise rotation
  - $T \circ S \to BA = \begin{pmatrix} 1 & 0 \\ 0 & -1 \end{pmatrix} \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}$ which is a 270 degree counterclockwise rotation

Inverse transformation undoes the transformation, takes it back to the original state
Equivalent to multiplying by the inverse of the original transformation's matrix
Such a transformation only exists if the matrix is invertible
This is pretty obvious, but let's say transformation T is induced by the matrix $A = \begin{pmatrix} 2 & 0 \\ 0 & -2 \end{pmatrix}$
Then the inverse transformation S would be induced by the matrix $A^{-1} = \begin{pmatrix} \frac{1}{2} & 0 \\ 0 & -\frac{1}{2} \end{pmatrix}$
So $S \circ T = T \circ S = Iv = v$ as $A^{-1}A = AA^{-1} = I$

subspace: a subset U of $R^n$ is called a subspace IF:
- The zero vector belongs to U
- If two vectors X and Y are both in U, then $X + Y \in U$
- If c is a scalar and X is in U, then $cX \in U$
- EXAMPLES:
  - The trivial cases: $R^n$ is a subspace of itself, and the zero vector is a subspace of $R^n$; all other subspaces are called proper subspaces
  - A straight line $L_m$ in $R^2$ through the origin. $L_m$ is the set of all $\begin{pmatrix} x \\ y \end{pmatrix} \in R^2$ such that y = mx
    - Obviously the zero vector (the origin) is part of $L_m$
    - Work out the other proofs as an exercise, or edit them in if you want
  - Same for lines through the origin in $R^3$
  - Planes through the origin in $R^3$ are subspaces of $R^3$:
    - def plane p: given that $n = \begin{pmatrix} a \\ b \\ c \end{pmatrix}$ is the normal vector for the plane, $p \cdot n = 0$
  - Certainly 0 is in the plane, and the other conditions are satisfied too (just prove that they equal 0 etc)
  - Null space of a matrix A: all vectors X where AX = 0. A subspace of R^n.
    - Basically take the matrix A to be part of a homogeneous system, and solve for X; resulting span of vector(s) = null space
    - Obviously this means matrix A has to have more variables than independent equations, otherwise you only get the zero vector
    - (Note that even if you only get the trivial solution - i.e. the zero vector - it's still a subspace, just not a proper one
  - Image space of matrix A: same as column space, later
  - Eigenspace of matrix A: again a subspace of R^n. Basically all the eigenvectors, use them as bases
  - Linear combination: the whole span of all possible linear combinations of vectors OR the intersection of all subspaces containing the vectors
  - If asked if a group of vectors span R^n, for a value of n, check to see if every possible value can be represented by them
    - This occurs if you have n vectors and they are all linearly independent
    - This is obviously true for the basis vectors ... in $R^2$, they would be $\begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\begin{pmatrix} 0 \\ 1 \end{pmatrix}$

For a set of vectors, say 3 vectors in R^3. Can you rewrite any vector as a linear combination of the other two vectors?
- If so, then the set of vectors is linearly dependent. Otherwise, linearly independent
- Obviously, for m vectors in R^n, if m > n then it's definitely linearly dependent
- If m = n, it could be either way
- And if m < n then they still could be linearly independent but either way they're not going to span R^n
- Anything containing the zero vector is not independent
- Any non-zero vector by itself is linearly independent
- The non-zero rows of a row-echelon matrix are independent
- An invertible square matrix times a set of compatible independent vectors results in a linearly independent set
To prove linear independence, you could write it in matrix form, as a homogeneous system
- If you only get the trivial solution, it's linearly independent
- For example, let's say you have 3 vectors in R^3: $\begin{pmatrix} 3 \\ 1 \\ -2 \end{pmatrix}, \begin{pmatrix} -2 \\ -3 \\ -1 \end{pmatrix}, \begin{pmatrix} 1 \\ 3 \\ 3 \end{pmatrix}$
  - The matrix would be $\begin{pmatrix} 3 & -2 & 1 \\ 1 & -3 & 3 \\ -2 & -1 & 3 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$ so there is only the trivial solution, so it's independent
- On the other hand, let's say you have four vectors in $R^3: \begin{pmatrix} 28 \\ -11 \\ -1 \end{pmatrix}, \begin{pmatrix} -4 \\ 2 \\ -2 \end{pmatrix}, \begin{pmatrix} -4 \\ 4 \\ -12 \end{pmatrix}, \begin{pmatrix} -4 \\ 1 \\ 3 \end{pmatrix}$
  - We can see right away that it's not a linearly independent set, as there are 4 vectors and 3 dimensions
  - HOWEVER, it might still span R^3 ... this would occur if there are 3 linearly independent vectors among the four.
  - So let's take the matrix, and row-reduce if. If there is MORE than one row of zeros, then there are not 3 linearly independent vectors among the 4.
  - $R^3: \begin{pmatrix} 28 & -11 & -1 \\ -4 & 2 & -2 \\ -4 & 4 & -12 \\ -4, 1, 3 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & -2 \\ 0 & 1 & -5 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}$
  - So the largest group of linearly independent vectors in this set has 2 vectors, so it does NOT span R^3
  - On the other hand, if the -4 in the last vector was a 4, then the set would have 3 linearly independent vectors, so it would span R^3
Linear independence and matrix invertibility:
- For a square n x n matrix, the following are equivalent:
  - The columns of A are linearly independent and span R^n
  - The rows of A are linearly independent and span R^n
  - A is invertible (i.e. its determinant is not zero)
- This means that if you want to see if 3 vectors in R^3 are linearly independent, you can write them in matrix form and find the determinant
- e.g. $\begin{pmatrix} 1 \\ 1 \\ -1 \end{pmatrix}, \begin{pmatrix} 1 \\ -1 \\ 1 \end{pmatrix} \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \to A = \begin{pmatrix} 1 & 1 & 0 \\ 1 & -1 & 0 \\ -1 & 1 & 1 \end{pmatrix}$
- $\det A = -1 - 1 = -2 \to$ the matrix is invertible, and thus the vectors are linearly independent.

If a vector can be written as a linear combination of other vectors then it's within the span of those vectors
- So the zero vector is always within any span, but that's trivial
- If you have two vectors in $R^2$, and they're not parallel, then you can write any vector in $R^2$ as a linear combination of those two!!!
- Same for 3 vectors in $R^3$, but you have to check that they're linearly independent etc
Basically any n independent vectors span $R^n$, even if they're in a set of >n vectors
If two non-zero vectors are parallel in $R^3$, then their span is equal to the span of just one vector - a straight line through the origin
If two non-zero vectors are not parallel in $R^3$, then their span is a plane through the origin whose normal is the cross product of the vectors

Any independent set in R^n can have up to n vectors
No spanning set for R^n can have less than n vectors
A set is a basis of a subspace in R^n if that set is independent AND spans that subspace
Dimension of a subspace: number of vectors needed in a basis for the subspace, or rank of matrix whose columns are the vectors
- EXAMPLES:
  - Any zero vector in R^n has a dimension of 0
  - Any set of vectors in R^n can have a dimension of n (IF there are n linearly independent vectors among them) or less
  - Given the set of vectors $S = span \left \lbrace \begin{pmatrix} 8 \\ 5 \\ 8 \end{pmatrix}, \begin{pmatrix} -2 \\ 0 \\ -2 \end{pmatrix}, \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}, \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} \right \rbrace$
    - S has a dimension of 2, and any two vectors or multiples thereof can be basis vectors
  - Given the set of vectors $S = span \left \lbrace \begin{pmatrix} 1 \\ -5 \\ -4 \end{pmatrix}, \begin{pmatrix} 1 \\ 1 \\ -4 \end{pmatrix}, \begin{pmatrix} -2 \\ 2 \\ 1 \end{pmatrix}, \begin{pmatrix} -3 \\ 2 \\ 1 \end{pmatrix} \right \rbrace$
    - S has a dimension of 3, and any three linearly independent vectors or multiples/combinations thereof can be basis vectors
  - Given another set of vectors, $S = span \left \lbrace \begin{pmatrix} 1 \\ -5 \\ -4 \end{pmatrix}, \begin{pmatrix} 2 \\ -10 \\ -8 \end{pmatrix}, \begin{pmatrix} 1 \\ 1 \\ -4 \end{pmatrix}, \begin{pmatrix} -2 \\ 2 \\ 1 \end{pmatrix} \right \rbrace$
    - S has a dimension of 3, and again any three linearly independent vectors or multiples/combinations thereof can be basis vectors
    - But this time, the second vector is obviously a multiple of the first vector, so discard that, and take the 1st, 3rd, and 4th as bases
- Basically, you want to see if a linear combination of the vectors can make any vector in R^n
  - If you have 4 vectors, 3 of them linearly independent, then they span R^3; the set as a whole is not a basis, but 3 independent vectors are

For an m x n matrix A:
- The column space - col(A) - is the subspace of R^m spanned by the columns of A; if you reduce the columns, bases are given by the non-zero columns
- The row space - row(A) - is the subspace of R^n spanned by the rows of A; bases given by the non-zero rows in RREF
- The rank is the dimension of the row-space of A which is incidentally equal to the dimension of the column-space of A
For an n x n matrix, it is invertible iff its rank is equal to n.
EXAMPLE: Given the matrix $ \begin{pmatrix} 1 & -1 & 3 & 2 & 1 \\ -2 & 3 & 1 & 1 & 1 \\ -4 & 7 & 9 & 7 & 5 \\ 3 & -4 & 2 & 1 & 0 \end{pmatrix}$, find bases for the row and column spaces, and the rank.
- RREF: $ \begin{pmatrix} 1 & 0 & 10 & 7 & 4 \\ 0 & 1 & 7 & 5 & 3 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}$
- Since the dimension of the rowspace is 2, the rank is 2.
- So two bases for the row space are the first two rows: $\begin{pmatrix}1 & 0 & 10 & 7 & 4\end{pmatrix}^T, \begin{pmatrix} 0 & 1 & 7 & 5 & 3 \end{pmatrix}^T$
- And the bases for the column space are any linearly independent columns in the original matrix
  - We could transpose the original matrix and RREF it or just column-reduce it to get basis vectors
  - But we could just take the two columns that have leading 1s in RREF and use those columns in the original matrix as bases
  - So two bases for the column space in this case are $\begin{pmatrix} 1 & -2 & -4 & 3\end{pmatrix}^T, \begin{pmatrix} -1 & 3 & 7 & -4 \end{pmatrix}^T$

SUMMARY OF EQUIVALENT THINGS

the following are equivalent for a m x n matrix A:
- AX = 0 has only the trivial solution
- The columns of A are independent
- rank(A) = n
- $A^T A$ is an invertible n x n matrix
- MAIN POINT: the columns are a linearly independent set of vectors
and the following are equivalent for a m x n matrix A:
- AX = B has a solution for every column $B \in R^n$
- The columns of A span R^n
- rank(A) = m
- $AA^T$ is an invertible m x m matrix
- MAIN POINT: The columns span $R^n$

Recall that the null space is the set of vectors X for which AX = 0
- EXAMPLE: Given $\begin{pmatrix}1 & -2 & 1 & 1 \\ -1 & 2 & 0 & 1 \\ 2 & -4 & 1 & 0 \end{pmatrix}$; reduce it to $\to \begin{pmatrix} 1 & -2 & 0 & -1 \\ 0 & 0 & 1 & 2 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{pmatrix}$
- Now parameterise it to get the solutions. Let's take $x_2 = s, x_4 = t$.
- So we get $X = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix} = \begin{pmatrix}2s + t \\ s \\ -2t \\ t \end{pmatrix} = \begin{pmatrix} 2 \\ 1 \\ 0 \\ 0 \end{pmatrix}s + \begin{pmatrix} 1 \\ 0 \\ -2 \\ 1 \end{pmatrix}t$
- So the linearly independent vectors $\begin{pmatrix} 2 \\ 1 \\ 0 \\ 0 \end{pmatrix}$ and $\begin{pmatrix} 1 \\ 0 \\ -2 \\ 1 \end{pmatrix}$ are basis vectors for and thus span the null space of A, which has dimension = 2
The image space is just the column space of the matrix, and the dimension is just the rank of the matrix
For an m x n matrix A of rank r, the dimension of the null space is equal to the n - r (number of columns minus rank)
- Example: find the eigenvalues and eigenspace for the matrix $A = \begin{pmatrix} 0 & 1 & 1 \\ 1 & 0 & 1 \\ 1 & 1 & 0 \end{pmatrix}$
- The characteristic polynomial factors to $(x - 2)(x + 1)^2$ so the eigenvectors are $\lambda_1 = 2, \lambda_2 = -1$
- For $\lambda_1, A - \lambda_1 I = \begin{pmatrix} -2 & 1 & 1 \\ 1 & -2 & 1 \\ 1 & 1 & -2 \end{pmatrix} \to \begin{pmatrix} 1 & 0 & -1 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{pmatrix} \to \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}t$
- So the basis for the eigenspace is $\begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}$ and the eigenspace can be described as $span = \left \lbrace \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix} \right \rbrace$ which is of dimension 1 incidentally if anyone cares
- For $\lambda_2, A - \lambda_1 I = \begin{pmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{pmatrix} \to \begin{pmatrix} 1 & 1 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{pmatrix}, x_2 = s, x_3 = t, x_1 = -s - t \to \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} = \begin{pmatrix} -s - t \\ s \\ t \end{pmatrix} \to \begin{pmatrix} -1 \\ 1 \\ 0 \end{pmatrix}s + \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix}t$
- So the bases are $\begin{pmatrix} -1 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix}$ and the eigenspace can be described as $span = \left \lbrace \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} \right \rbrace$ (dimension 2)
- Also, note that $\lambda_1$ has a dimension and thus geometric multiplicity of 1
- And that $\lambda_2$ has a dimension of 2, and an algebraic multiplicity of 2
- So this matrix is diagonalisable

Course review

1Linear equations and matrices¶

1.1Introduction¶

1.2Row operations¶

1.3Row-reducing matrices¶

1.4Elementary matrices¶

1.5Homogeneous systems¶

1.6Matrix multiplication¶

1.7Determinants¶

1.8Inverses¶

2Eigenvalues and diagonalising matrices¶

2.1Eigenvalues¶

2.2Diagonalising matrices¶

3Vector geometry¶

3.1Basics of vectors¶

3.2Dot products and projections¶

3.3Lines¶

3.4Planes¶

3.5The cross product¶

3.6Matrix Transformations of R^2¶

3.7Composites of transformations¶

3.8Inverses of matrix transformations¶

4The vector space R^n¶

4.1The vector space¶

4.2Linear independence¶

4.3Span¶

4.4Dimension and bases¶

4.5Rank, row and column spaces¶

4.6Null space and image space¶