Thursday, February 21, 2013

Introduction to inner product spaces

In this lecture, we started chapter 6 (inner product spaces). This topic will not be covered on the midterm, but it will be a big part of the final, so don't neglect it.

1Inner product spaces¶

Definition: an inner product space on $V$ is a function

$$\begin{align} V \times V & \to \mathbb F \\ (u, v) & \mapsto \langle u, v\rangle \end{align}$$

such that the following conditions are satisfied:

1. Positive definiteness: $\langle v, v \rangle \geq 0$ for all $v \in V$ with equality if and only if $v = 0$
2. Linearity in the first argument: $\langle au+v, w \rangle = a\langle u, w \rangle + \langle v, w \rangle$
3. Conjugate symmetry: $\langle v, w \rangle = \overline{\langle w, v \rangle}$

1.1Examples¶

The Euclidean inner product: $V = \mathbb F^n$. Then $\langle (v_1, \ldots, v_n), (w_1, \ldots, w_n)\rangle = v_1 \overline{w_1} + \ldots + v_n\overline{w_n}$ which is basically the dot product of the two vectors, but, like, conjugate. (If $\mathbb F = \mathbb R$ then this is just the standard dot product operation that we all know and love.)

Integration over a particular region: $V = P_m(\mathbb F)$. Then

$$\langle p, q \rangle = \int_0^1 p(x)\overline{q(x)} \,dx$$

As an exercise, verify that this satisfies the 4 conditions above.

1.2Inner products as linear functions¶

For a fixed $w \in V$, we can define the function

$$\begin{align} f_w: V & \to \mathbb F \\ v & \mapsto \langle v, w \rangle \end{align}$$

Let's show that this function is linear:

$$f_w(au+v) = \langle au + v, w \rangle = a\langle u, w \rangle + \langle v, w \rangle = af_w(u) + f_w(u) \checkmark$$

As an exercise, prove the following properties:

1. $\langle v, 0 \rangle = \langle 0, v \rangle = 0$
2. $\langle u, av + w \rangle = a\langle u, v \rangle + \langle u, w \rangle$

From now on, we assume that $V$ denotes an inner product space - i.e., a vector space equipped with some inner product.

1.3Norms¶

For $v \in V$, the norm of $V$, denoted by $\lVert v \rVert$, is defined as

$$\lVert v \rVert = \sqrt{\langle v, v \rangle}$$

Note that the radicand is always $\geq 0$ due to positive definiteness.

1.3.1Examples¶

$V = \mathbb F^n$, with the Euclidean inner product as defined before:

$$\lVert (z_1, \ldots, z_n) \rVert = \sqrt{|z_1|^2 + \ldots + |z_n|^2}$$

$V = P_3(\mathbb R)$, $p(x) = x^2+1$:

$$\lVert p(x) \rVert = \sqrt{\langle x^2+1, x^2+1 \rangle} = \sqrt{\int_0^1 (x^2+1)^2 \,dx}$$

1.3.2Properties¶

Just one for now. Proof: left as an exercise.

For any $a \in \mathbb F$, $v \in V$:

$$\lVert av \rVert = |a| \lVert v\rVert$$

1.3.3Orthogonality¶

$u, v \in V$ are orthogonal if $\langle u, v \rangle = 0$. (Note that this is a reflexive relation.)

1.3.4The Pythagorean theorem¶

If $\langle u, v \rangle = 0$, then:

$$\lVert u + v \rVert^2 = \lVert u \rVert^2 + \lVert v \rVert^2$$

Proof:

$$\lVert u+v \rVert^2 = \langle u + v, u+v \rangle = \lVert u \rVert^2 + \lVert v \rVert^2 + \underbrace{\langle u, v \rangle}_{= 0} + \underbrace{\langle v, u \rangle}_{\text{also } = 0} = \lVert u \rVert^2 + \lVert v \rVert^2 \; \checkmark$$

which was a nice one-liner.

(To see that $\langle u, v \rangle = \langle v, u \rangle = 0$, just use conjugate symmetry. Proof left as an exercise for the astute reader.)

If we are working in $\mathbb R^n$, then the converse of the theorem holds true as well: $\lVert u+v \rVert^2 = \lVert u\rVert^2 + \lVert v \rVert^2$ implies that $\langle u, v \rangle = \langle v, u \rangle = 0$.

1.3.5The Cauchy-Schwarz inequality¶

$$|\langle u, v \rangle | \leq \lVert u\rVert \cdot \lVert v \rVert \tag{lol}$$

This feels familiar.

Equality holds when $u = \lambda v$ for some $\lambda \in \mathbb F$ (i.e., when the two vectors are scalar multiples of each other).

Proof: let $u, v \in V$. If $v = 0$, then we have 0 on both sides of the inequality and we're done; equality holds trivially. So let's assume that $v \neq 0$. Consider

$$u = \frac{\langle u, v\rangle}{\lVert v\rVert^2} v + w$$

where $w$ is some vector orthogonal to $v$. By the Pythagorean theorem,

$$\lVert u \rVert^2 = \lVert \frac{\langle u, v\rangle}{\lVert v\rVert^2} v + w \rVert^2 = \frac{|\langle u, v \rangle |^2}{\lVert v \rVert^2} + \underbrace{\lVert w \rVert^2}_{\geq 0} \geq \frac{|\langle u, v\rangle|^2}{\lVert v\rVert^2}$$

Consequently, by multiplying both sides by the denominator, we get

$$|\langle u, v \rangle |^2 \leq \lVert u\rVert^2\cdot \lVert u\rVert^2$$

and if we take the square root of both sides then we get the Cauchy-Schwarz inequality.