In this lecture, we started chapter 6 (inner product spaces). This topic will not be covered on the midterm, but it will be a big part of the final, so don't neglect it.
1Inner product spaces¶
Definition: an inner product space on $V$ is a function
$$\begin{align} V \times V & \to \mathbb F \\ (u, v) & \mapsto \langle u, v\rangle \end{align}$$
such that the following conditions are satisfied:
- Positive definiteness: $\langle v, v \rangle \geq 0$ for all $v \in V$ with equality if and only if $v = 0$
- Linearity in the first argument: $\langle au+v, w \rangle = a\langle u, w \rangle + \langle v, w \rangle$
- Conjugate symmetry: $\langle v, w \rangle = \overline{\langle w, v \rangle}$
1.1Examples¶
The Euclidean inner product: $V = \mathbb F^n$. Then $\langle (v_1, \ldots, v_n), (w_1, \ldots, w_n)\rangle = v_1 \overline{w_1} + \ldots + v_n\overline{w_n}$ which is basically the dot product of the two vectors, but, like, conjugate. (If $\mathbb F = \mathbb R$ then this is just the standard dot product operation that we all know and love.)
Integration over a particular region: $V = P_m(\mathbb F)$. Then
$$\langle p, q \rangle = \int_0^1 p(x)\overline{q(x)} \,dx$$
As an exercise, verify that this satisfies the 4 conditions above.
1.2Inner products as linear functions¶
For a fixed $w \in V$, we can define the function
$$\begin{align} f_w: V & \to \mathbb F \\ v & \mapsto \langle v, w \rangle \end{align}$$
Let's show that this function is linear:
$$f_w(au+v) = \langle au + v, w \rangle = a\langle u, w \rangle + \langle v, w \rangle = af_w(u) + f_w(u) \checkmark$$
As an exercise, prove the following properties:
- $\langle v, 0 \rangle = \langle 0, v \rangle = 0$
- $\langle u, av + w \rangle = a\langle u, v \rangle + \langle u, w \rangle$
From now on, we assume that $V$ denotes an inner product space - i.e., a vector space equipped with some inner product.
1.3Norms¶
For $v \in V$, the norm of $V$, denoted by $\lVert v \rVert$, is defined as
$$\lVert v \rVert = \sqrt{\langle v, v \rangle}$$
Note that the radicand is always $\geq 0$ due to positive definiteness.
1.3.1Examples¶
$V = \mathbb F^n$, with the Euclidean inner product as defined before:
$$\lVert (z_1, \ldots, z_n) \rVert = \sqrt{|z_1|^2 + \ldots + |z_n|^2}$$
$V = P_3(\mathbb R)$, $p(x) = x^2+1$:
$$\lVert p(x) \rVert = \sqrt{\langle x^2+1, x^2+1 \rangle} = \sqrt{\int_0^1 (x^2+1)^2 \,dx}$$
1.3.2Properties¶
Just one for now. Proof: left as an exercise.
For any $a \in \mathbb F$, $v \in V$:
$$\lVert av \rVert = |a| \lVert v\rVert$$
1.3.3Orthogonality¶
$u, v \in V$ are orthogonal if $\langle u, v \rangle = 0$. (Note that this is a reflexive relation.)
1.3.4The Pythagorean theorem¶
If $\langle u, v \rangle = 0$, then:
$$\lVert u + v \rVert^2 = \lVert u \rVert^2 + \lVert v \rVert^2$$
Proof:
$$\lVert u+v \rVert^2 = \langle u + v, u+v \rangle = \lVert u \rVert^2 + \lVert v \rVert^2 + \underbrace{\langle u, v \rangle}_{= 0} + \underbrace{\langle v, u \rangle}_{\text{also } = 0} = \lVert u \rVert^2 + \lVert v \rVert^2 \; \checkmark$$
which was a nice one-liner.
(To see that $\langle u, v \rangle = \langle v, u \rangle = 0$, just use conjugate symmetry. Proof left as an exercise for the astute reader.)
If we are working in $\mathbb R^n$, then the converse of the theorem holds true as well: $\lVert u+v \rVert^2 = \lVert u\rVert^2 + \lVert v \rVert^2$ implies that $\langle u, v \rangle = \langle v, u \rangle = 0$.
1.3.5The Cauchy-Schwarz inequality¶
$$|\langle u, v \rangle | \leq \lVert u\rVert \cdot \lVert v \rVert \tag{lol}$$
Equality holds when $u = \lambda v$ for some $\lambda \in \mathbb F$ (i.e., when the two vectors are scalar multiples of each other).
Proof: let $u, v \in V$. If $v = 0$, then we have 0 on both sides of the inequality and we're done; equality holds trivially. So let's assume that $v \neq 0$. Consider
$$u = \frac{\langle u, v\rangle}{\lVert v\rVert^2} v + w$$
where $w$ is some vector orthogonal to $v$. By the Pythagorean theorem,
$$\lVert u \rVert^2 = \lVert \frac{\langle u, v\rangle}{\lVert v\rVert^2} v + w \rVert^2 = \frac{|\langle u, v \rangle |^2}{\lVert v \rVert^2} + \underbrace{\lVert w \rVert^2}_{\geq 0} \geq \frac{|\langle u, v\rangle|^2}{\lVert v\rVert^2}$$
Consequently, by multiplying both sides by the denominator, we get
$$|\langle u, v \rangle |^2 \leq \lVert u\rVert^2\cdot \lVert u\rVert^2$$
and if we take the square root of both sides then we get the Cauchy-Schwarz inequality.