HTSEFP: Problems

Part of the "How to solve every problem" series for MATH 223 problems (as opposed to proofs, the page for which hasn't been transferred over to the beta yet). Created in April 2011 for Loveys' class by @dellsystem, with some contributions from @tahnok, @clarle, and one anonymous user. Unfortunately, as of this writing, the page has never been updated by anyone in Jonsson's class, so if there are new types of problems that students are expected to be familiar with for Jonsson's version of the course they won't yet appear here (but please add them if you can). Still, the information presented here should be timeless, as the methods of solving the problems listed below are unlikely to change.

1Finding the inverse, product of elementary matrices¶

Given a large, complex matrix:

• (a) Express it as a product of elementary matrices
• (b) Find its inverse, and express the inverse as a product of elementary matrices

1.1General solution¶

First, row-reduce the matrix to the identity matrix, using an augmented matrix with the identity matrix on the right. Once the matrix has been row-reduced to the identity, whatever the matrix on the right looks like is the inverse matrix. Make sure you keep track of the row operations that have been done to get it to the identity. To get the inverse matrix as the product of elementary matrices, take the product of each elementary matrix resulting from the aforementioned row operations, in reverse order. To get the original matrix as a product of elementary matrices, take the inverse of the above elementary matrices, and reverse the order again.

In shorthand notation, with $E_1$ being the elementary matrix resulting from the first row operation above, etc:

$A = E^{-1}_1 E^{-1}_2 E^{-1}_3 E^{-1}_4 ... E^{-1}_n$ (i.e. inverses in regular order)

$A^{-1} = E_n ... E_4 E_3 E_2 E_1$ (i.e. not the inverse, unless you mean the order)

1.2Examples¶

• Assignment 2, question 4

2Finding the null, row and column spaces¶

Given a large, complex matrix:

• (a) Find bases for the null space, row space, and column space
• (b) Express each row and column as a linear combination of the vectors in the respective spaces
• (c) Find its rank

2.1General solution¶

First, row-reduce the matrix to RREF. The row space is easily obtained from the RREF matrix - just take the rows with leading ones in them. To get the column space, find the columns in the RREF matrix that contain leading ones, then find the corresponding columns in the original matrix, and take those columns. To find the null space, solve the homogeneous system. (That should be easy enough by now - look at the examples for more.) Expressing each row and column as a linear combination is also easy, you can often just look at the relationship between the rows/columns that you found either in the RREF matrix or through the process of row-reducing. The rank of the matrix is just equal to the number of linearly independent rows (i.e. the number of vectors in your row space).

2.2Examples¶

• Fall 2010 final, question 1
• Winter 2010 final, question 1
• Fall 2007 final, question 1
• Assignment 3, question 2
• Assignment 4, question 1

3Change-of-basis matrix between ordered bases¶

Given two sets of vectors:

• (a) Verify that they are ordered bases
• (b) Find the change-of-basis matrix from one basis to another (and the other way around too)

3.1General solution¶

To show that they are ordered bases, just show that the vectors in each basis are independent. This can be done by representing them as column vectors in a 3x3 matrix and row-reducing to get the identity.

To find the change-of-basis matrix from basis B to basis A, first try to express each vector in basis A as a linear combination of the vectors in basis B. Let $\vec v_1, \vec v_2, \vec v_3$ be the basis vectors of A and let $\vec v_4, \vec v_5, \vec v_6$ be the basis vectors of B. So we try to solve this system of equations:

$$\vec v_1 = \alpha_1 \vec v_4 + \alpha_2 \vec v_5 + \alpha_3 \vec v_6$$

$$\vec v_2 = \alpha_4 \vec v_4 + \alpha_5 \vec v_5 + \alpha_6 \vec v_6$$

$$\vec v_3 = \alpha_7 \vec v_4 + \alpha_8 \vec v_5 + \alpha_9 \vec v_6$$

which in matrix form can be written as

$$\begin{pmatrix} \vec v_1 \\ \vec v_2 \\ \vec v_3 \end{pmatrix} = \begin{pmatrix} \alpha_1 & \alpha_2 & \alpha_3 \\ \alpha_4 & \alpha_5 & \alpha_6 \\ \alpha_7 & \alpha_8 & \alpha_9 \end{pmatrix} \begin{pmatrix} \vec v_4 \\ \vec v_5 \\ \vec v_6 \end{pmatrix}$$

To solve, we use an augmented matrix to represent the basis vectors of A on the left, B on the right. We then row-reduce the left to the identity, then take the transpose of the matrix on the right. That matrix gives you the alphas. The original (non-transposed) matrix is the change of basis matrix. I wrote a proof in my assignment but don't feel like proving it again.

Note that the change of basis matrix from A to B is the inverse of the change of basis matrix from B to A.

3.2Examples¶

• Assignment 4, question 4

4Linear operators: the associated matrix, kernel, imagespace¶

Given a transformation T over the real vector space of polynomials/matrices/regular vectors:

• (a) Verify that it is a linear operator
• (b) Given a basis B, find $[T]_B$
• (c) Find a basis for the kernel and imagespace of T

4.1General solution¶

(a) Show that it preserves vector addition, and scalar multiplication. Easy enough, just make sure that you don't do something silly like $f(x + y)$ or $f(\alpha x)$.

(b) Evaluate T for each vector in the basis, and express the result as a coefficient column vector, in terms of the basis. These column vectors correspond to the basis vectors of T. Note that if you're using a nonstandard basis, you will of course have to express the coefficients in terms of that basis and not the standard basis. See examples.

(c) To find the kernel, find the nullspace, then express it in terms of the vector space being used. To find the image space, just find the columns with leading ones in the RREF matrix and take the corresponding columns in the original matrix, then express it in terms of the vector space being used.

4.2Examples¶

• Fall 2010 final, question 2
• Winter 2010 final, question 2
• Fall 2009 final, question 2
• Winter 2008 final, question 2
• Fall 2007 final, question 4
• Assignment 5, question 1
• Assignment 5, question 2

5Determining if a transformation is linear¶

Given several transformations $T: V \to V$, decide if they are linear transformations or not.

5.1General solution¶

For each transformation, check if it preserves vector addition and scalar multiplication. If not, it's not a linear transformation.

5.2Examples¶

• Assignment 4, question 5

6Finding the non-recursive formula for a sequence¶

Give an explicit, non-recursive formula for something defined recursively.

6.1General solution¶

Let's say you're given the following recursive formula: $x_{n+2} = a x_{n+1} + b x_n$

First set up a matrix in this form to represent the system:

$$\begin{pmatrix} x_{n+2} \\ x_{n+1} \end{pmatrix} = \begin{pmatrix} a & b \\ 1 & 0 \end{pmatrix} \begin{pmatrix} x_{n+1} \\ x_n \end{pmatrix}$$

Then, find the characteristic polynomial, and from that get the eigenvalues: $\lambda_1, \lambda_2$. The formula for $x_n$ can then be magically given by the formula $x_n = \alpha \lambda_1^n + \beta \lambda_2^n$.

We can solve for alpha and beta using $x_0$ and $x_1$ which will be given. That's it.

6.2Examples¶

• Fall 2010 final, question 4
• Winter 2010 final, question 4
• Fall 2009 final, question 5
• Winter 2008 final, question 4
• Assignment 7, question 1
• Assignment 6, question 3 (to an extent)

7Diagonalising matrices¶

Given a diagonalisable matrix A:

• (a) Find its characteristic and minimal polynomals
• (b) Find an invertible matrix P blah blah
• (c) Find A to the power of some large integer

7.1General solution¶

(a) Finding the characteristic polynomial is easy enough, just solve for $\det(A- \lambda I) = \det(\lambda I - A) = 0$ (the two are equivalent, so either one works, although the second form is often nicer as you don't get negative signs in front of your lambdas). To find the minimal polynomial, just remember this - the minimal polynomial will only have repeated roots if the matrix is NOT diagonalisable. (This is just useful to know in general.) As this matrix is diagonalisable, it will not have repeated roots, so get rid of any eigenvalues with algebraic multiplicity > 1 in your characteristic polynomial and you're set.

(b) Find the eigenvalues (from above). Find the associated eigenvectors. Put the eigenvectors, as column vectors, into a matrix - that forms P.

(c) We know that $D = P^{-1}AP$ where D is a diagonal matrix with the eigenvalues of A being the diagonal elements, in the same order as the associated eigenvectors of P. So $A = PDP^{-1}$ and $A^n = P D^n P^{-1}$. To find $D^n$ just raise each diagonal element to the power of n. So to find A to the power of n just multiply out the above matrices, and that's it.

7.2Examples¶

• Fall 2010 final, question 6
• Winter 2008 final, question 3
• Fall 2007 final, question 3
• Assignment 6, question 2

8Semi-diagonalising non-diagonalisable matrices¶

Given a NON-diagonalisable matrix A:

• (a) Find its characteristic and minimal polynomials
• (b) Find A to the power of some large integer (it's doable)

8.1General solution¶

(a) As above, the minimal polynomial will only have repeated roots if it's not diagonalisable. This matrix is not diagonalisable, so the characteristic and minimal polynomials should be the same.

(b) We derived the solution to this in assignment 7. Here's the derivation:

Start with $J = \begin{pmatrix} \lambda & 1 \\ 0 & \lambda \end{pmatrix}$. Now $J^2 = \begin{pmatrix} \lambda^2 & 2 \lambda \\ 0 & \lambda^2 \end{pmatrix}, \, J^3 = \begin{pmatrix} \lambda^3 & 3\lambda^2 \\ 0 & \lambda^3 \end{pmatrix} \,...\,\, J^n = \begin{pmatrix} \lambda^n & n\lambda^{n-1} \\ 0 & \lambda^n \end{pmatrix}$

This may seem irrelevant, but it's not, just wait. Now we find the eigenvalues for matrix A. Let's assume that this is a 2 by 2 matrix and there is only one distinct eigenvalue. We find the associated eigenvector (there can only be one) for that eigenvalue, and put that eigenvector into the matrix P, so that it looks like this (where the eigenvector has entries $\vec v_1, \vec v_2$:

$$P = \begin{pmatrix} \vec v_1 & 1 \\ \vec v_2 & 0 \end{pmatrix}$$

So when we do $P^{-1}AP$ we get the matrix J that we defined above. So calculate J to the power of n. Then $A^n = PJ^nP^{-1}$, so just multiply it out to find A to the power of n.

8.2Examples¶

• Assignment 7, question 2

This hasn't actually appeared on any past exams that I've seen, but it seems like an interesting problem, and who knows, it might show up.

9Defining integral functions as inner products¶

Given, basically, an integral function:

• (a) Show that this defines an inner product space
• (b) Use Cauchy-Schwarz to prove a given inequality, and identify the functions for which equality holds

9.1General solution¶

(a) We just need to show that it respects linearity in the first argument, conjugate symmetry, and positive definiteness.

Linearity: prove that it respects vector addition and scalar multiplication.

Conjugate symmetry: just switch the order around. May be real symmetry.

Positive definiteness: usually for integrals this just involves noting that as long as f(x) is not 0, the integrand is non-negative everywhere and positive for at least some small interval, and so the integral is positive.

(b) Identify the two functions you need to take the inner product of - usually one will be f, the other will be whatever you need to make up what the left side appears to be (hard to explain, take a look at the examples if you want to see how this works). Recall that Cauchy-Schwarz (don't misspell this, k) states that $| \langle \vec v, \vec w \rangle | \le \left \| \vec v \right \| \left \| \vec w \right \|$ with equality only if the two inputs are scalar multiples of each other. So you end up evaluating one integral, and use that to prove the inequality, and then state that equality only holds when one input is a scalar multiple of the other.

9.2Examples¶

• Fall 2010 final, question 9
• Winter 2010 final, question 9
• Fall 2009 final, question 3
• Fall 2007 final, question 7
• Assignment 9, question 1
• Assignment 8, question 5 (to an extent)

10Solving systems of differential equations¶

Give the general solution to a system of differential equations.

10.1General solution¶

We have a pair of differential equations in the form

$$y_1' = ay_1 +by_2$$
$$y_2' = cy_1 + dy_2$$

We can represent this with the coefficient matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$

Diagonalise this matrix - find the eigenvalues, then the eigenvectors, then P, P inverse, and D.

We then make the substitution $\vec z = P^{-1} \vec y$ so $\begin{pmatrix} z_1' \\ z_2' \end{pmatrix} = D\vec z$. Let the diagonal matrix be $\begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix}$. So we get $z_1' = \lambda_1 z_1,\,z_2' = \lambda_2 z_2$. To solve for z, we use euler's constant: $z_1 = c_1 e^{\lambda_1 x}, \, z_2 = c_2 e^{\lambda_2 x}$. Now that we have solved for z, we can multiply it by P on the left to get y. From the resulting matrix you get functions (with arbitrary constants) for $y_1$ and $y_2$ and there you go.

10.2Examples¶

• Winter 2010 final, question 3
• Assignment 7, question 3 (method derived and proved in this assignment)

11Finding the determinant of large, unwieldy matrices¶

Given a large >3x3 square matrix, find its determinant.

11.1General solution¶

Row-reduce it, keeping track of which row operations alter the determinant. Once you get a triangular matrix, just multiply the elements along the main diagonal, and apply any necessary changes due to row operations, and there you go.

11.2Examples¶

• Assignment 7, question 4.

Couldn't find any examples from past exams, but there's an example in the lecture notes from March 7, 2011. Will add link later.

12Finding the determinant of a matrix using other relevant matrices¶

Given three matrices, one of which is a sort of combination of the rows of the other two (whose determinants are known), find the determinant of the other matrix.

12.1General solution¶

Perform row operations / tranposes etc as necessary to get the two matrices whose determinants are known into the form you want them to be. Calculate the determinants of the resulting matrices, then when they differ by only one row - and the sum of those rows is equal to the relevant row in the matrix whose determinant you want to find - add the determinants together. Check out the examples.

12.2Examples¶

• Winter 2010 final, question 7
• Winter 2008 final, question 7
• Fall 2007 final, question 6
• Assignment 8, question 1

13Finding bases for the sum and intersection of two subspaces¶

Given two subspaces of a field, find a basis for each of them, then a basis for their sum and a basis for their intersection.

13.1General solution¶

Put all the basis vectors in a big matrix and row-reduce. A basis for each subspace is given by the largest set of independent vectors from the spanning vectors. To find a basis for their sum, take the column vectors from the original matrix corresponding to the ones containing leading ones in the RREF matrix. To find a basis for their intersection, you need to find a relationship between the column vectors, so that you find a vector (or more than one) that is a linear combination of both vectors from the first subspace (only) and vectors from the second subspace (only). Remember dimsum - $dim(W_1 + W_2) + dim(W_1 \cap W_2) = dim(W_1) + dim(W_2)$.

13.2Examples¶

• Fall 2009 final, question 1
• Assignment 4, question 2
• Assignment 4, question 3

14Determining if subsets are subspaces or not¶

Given several subsets of a vector space, decide if they are subspaces or not

14.1General solution¶

• Check to see if the subset contains the zero vector
• Check to see if it respects vector addition
• See if it respects scalar multiplication

Checking for the zero vector is pretty simple. Checking vector addition is a little more tricky. You have to show that for any 2 arbitrary vectors from the subset the result of adding them is still in the subset. The same thing is true of scalar multiplication. You have to be able to choose any scalar and show that any vector times that scalar is still in the subset.

14.2Examples¶

• Winter 2008 final, question
• Fall 2007 final, question 2
• Assignment 2, question 1
• Assignment 2, question 2
• Assignment 3, question 1

15Finding orthonormal bases for a subspace and its perpendicular¶

Given a subspace W:

• (a) Find an orthonormal basis for both W and $W^{\perp}$
• (b) Find the projection vectors onto both W and $W^{\perp}$

15.1General solution¶

(a) Use the Gram-Schmidt orthogonalisation process to get basis vectors for W that are orthogonal. For example, if there are two vectors in W:

• Set $\vec w_1 = \vec v_1$
• Let $\vec w_2 = \vec v_2 - \frac{\langle v_2, w_1 \rangle}{\langle w_1, w_1 \rangle} \vec w_1$

Solve for the second basis vector, and you get the orthogonal basis for W. Now you just need to normalise it - divide it by its length (which you can find by taking the inner product of it with itself and square rooting that). That gives you the orthonormal basis.

To find an orthonormal basis for $W^{\perp}$, solve the homogeneous equation $Ax = 0$, where A is just the the basis vectors for W turned on their side (so each basis vector becomes a row of A). Row-reduce A to get the basis vectors for the null space. We then take their conjugates (important) and normalise them to get the orthonormal basis for W.

(b) Let $\{ \vec u_1, \vec u_2 \}$ be the basis vectors for W that we found above. (We don't have to take the normalised ones.)

$$Proj_{W} (v) = \frac{\langle v, u_1 \rangle}{\langle u_1, u_1 \rangle} u_1 + \frac{\langle v, u_2 \rangle}{\langle u_2, u_2 \rangle} u_2$$

The projection onto $W^{\perp}$ is just v minus the above projection vector: $Proj_{W^{\perp}}(v) = \vec v - Proj_{W} (v)$.

15.2Examples¶

• Fall 2010 final, question 7
• Winter 2010 final, question 6
• Fall 2009 final, question 6
• Winter 2008 final, question 6
• Fall 2007 final, question 8
• Assignment 9, question 2

Loveys also works out an example (and explains the method) in his notes for inner product spaces 1.

16Finding an orthogonal or unitary matrix to diagonalise¶

Given a real symmetric or Hermitian or whatever matrix:

• (a) Find an orthogonal/unitary matrix P such that $P^TAP$ is diagonal
• (b) Find the diagonal matrix

16.1General solution¶

(a) First, find the eigenvalues and associated eigenvectors for this matrix. Often, you'll be given an eigenvalue and can figure out the other eigenvalue from the trace of the matrix. Once you have the eigenvectors, orthogonalise them via Gram-Schmidt, then normalise them to get the orthogonal (if it's real) or unitary (if it's complex) matrix.

(b) A diagonal matrix with the eigenvalues along the main diagonal. Same order as the eigenvectors in P.

16.2Example¶

• Fall 2010 final, question 8
• Winter 2010 final, question 8
• Fall 2009 final, question 7
• Winter 2008 final, question 8
• Fall 2007 final, question 9
• Assignment 9, question 3

17Determining if a function is an inner product¶

Given several matrices over the real or complex fields, decide if the function defined by $\langle v, w \rangle = v^T A \overline{w} $ is an inner product on the field.

17.1General solution¶

For something to define an inner product, it has to fulfill three conditions: linearity in the first argument, conjugate symmetry, and positive definiteness.

If the matrix is over the real numbers, then linearity is fine for any matrix, but symmetry only occurs when the matrix is symmetric. Positive definiteness occurs if for any non-zero vector v, $\langle \vec v^T A \vec v \rangle$ is positive. We can also determine positive definiteness from the eigenvalues of a real symmetric matrix - such a matrix is positive definite iff all the eigenvalues are positive. So as long as the matrix is symmetric and positive definite, it will result in an inner product over the reals. (If the matrix is the identity matrix, you get the standard inner product.)

If we're working over the complex field, then again linearity is true for any matrix. However, although we could ignore the bar over the w when working over the reals, we have to take it into account over the complex field - if it weren't there, $\vec v^T I \vec w$ wouldn't even give an inner product. So when do we get conjugate symmetry? Well, we want it so that $\langle w, v \rangle = \overline{\langle v, w \rangle}$. That is, we want $\vec w^T A \overline{\vec v} = \overline{\vec v^T T \overline{\vec w}}$. Since the result is essentially a 1 by 1 matrix ("essentially") then it's equal to its own transpose - so $\overline{\vec v^T T \overline{\vec w}} = \vec w^T \overline{A^T \vec v}$. So to get conjugate symmetry we need $A = \overline{A^T}$, meaning the matrix must be equal to its conjugate transpose (i.e. the matrix must be Hermitian). For positive definiteness, we again need all positive eigenvalues.

So, if we're given a real matrix:

• Check if it's symmetric. If yes, continue.
• Check the eigenvalues. If they're all positive real numbers, then you win.

If we're given a complex (I actually almost wrote fake) matrix:

• Check if it's Hermitian (i.e. if its conjugate transpose is equal to itself ... should be easy)
• If so, check the eigenvalues. If they're all positive real numbers, then you win.

17.2Examples¶

• Assignment 8, question 3

I couldn't find a relevant question on any past exams, but this seems like a good candidate for a multiple choice question this year. Loveys explains this pretty well in his notes for inner product spaces 1.

18Quadratic forms¶

Given a "quadratic form" $Q(x, y, z)$ determine the shape of the graph of $Q(x, y, z) = 1$.

18.1General solution¶

From the quadratic form, put the coefficients of the form into the values of a matrix as such:

• The coefficients of the variables with degree 2 (example: $x^2, y^2, z^2$ should go on the diagonal of the matrix.
• The coefficients of the variables that are a combination of each other (for lack of a better term, example: $xy, yz, xz$) should have their value, divided by 2, put into the values adjacent to the diagonal.

To explain it better with an example, say you have the quadratic form $2x^2 + 2xy + 3y^2 + 4xz + 8yz + 5z^2$:

• The coefficients of the variables with degree 2 will go into the diagonal: $\displaystyle B=\begin{pmatrix}2 & \cdots & \cdots\\\vdots & 3 & \cdots\\\vdots & \vdots & 5\\\end{pmatrix}$
• The coefficients of the variables that are combinations, with their value divided by 2, should be put into the values adjacent to the diagonal. For $xy$, that would be $2/2 = 1$; for $xz$, that would be $4/2 = 2$; for $yz$, that would be $8/2 = 4$.
• These will go into the matrix as shown. Note that the matrix will always be symmetric:
  $$B=\begin{pmatrix}2 & 1 & 2\\1 & 3 & 4\\2 & 4 & 5\\\end{pmatrix}$$

After obtaining your matrix, simply calculate your eigenvalues (technically you don't even really need to have to calculate them all, just make sure you know what the signs of your eigenvalues are).

Now use the following table to determine what the shape of your graph will be:

Number of negative eigenvalues	Number of zero eigenvalues	Number of positive eigenvalues	Shape of graph
3 (negative or zero)1	See previous	0	Empty graph
2	0	1	Two sheets
1	1	1	Hyperbolic cylinder
1	0	2	Hyperboloid or one sheet
0	2	1	Two planes
0	1	2	Elliptic cylinder
0	0	3	Ellipsoid

And that's it.

18.2Examples¶

None on actual exams, but the last question of Assignment 9 focuses on these. I highly recommend knowing the shapes of everything given the eigenvalue signs though.