9.3 Orthogonality

Last updated
Save as PDF
Share
1. Share
2. Share
3. Tweet

$Schilling.jpg$
Contributed by Isaiah Lankham, Bruno Nachtergaele, & Anne Schilling
Professor (Mathematics) at University of California, Davis

Using the inner product, we can now define the notion of orthogonality, prove that the Pythagorean theorem holds in any inner product space, and use the Cauchy-Schwarz inequality to prove the triangle inequality. In particular, this will show that $\norm{v}=\sqrt{\inner{v}{v}}$ does indeed define a norm.

Definition 9.3.1. Two vectors $u,v\in V $ are orthogonal (denoted $u\bot v$) if $\inner{u}{v}=0$.

Note that the zero vector is the only vector that is orthogonal to itself. In fact, the zero vector is orthogonal to every vector $v\in V$.

Theorem 9.3.2. (Pythagorean Theorem). If $u,v\in V$, an inner product space, with $u\bot v$, then $\norm{\cdot} $ defined by $\norm{v}:=\sqrt{\inner{v}{v}} $ obeys

\[ \norm{u+v}^2 = \norm{u}^2 + \norm{v}^2. \]

Proof.  Suppose $u,v\in V $ such that $u\bot v$. Then
\begin{equation*}
\begin{split}
    \norm{u+v}^2 &= \inner{u+v}{u+v}
    = \norm{u}^2 + \norm{v}^2 + \inner{u}{v} + \inner{v}{u}\\
    &= \norm{u}^2 + \norm{v}^2.
\end{split}
\end{equation*}

Note that the converse of the Pythagorean Theorem holds for real vector spaces since, in that case, $\inner{u}{v}+\inner{v}{u}=2\mathrm{Re} \inner{u}{v} =0$.

Given two vectors $u,v\in V $ with $v\neq 0$, we can uniquely decompose $u $ into two pieces: one piece parallel to $v $ and one piece orthogonal to $v$. This is called an orthogonal decomposition. More precisely, we have
\begin{equation*}
    u=u_1+u_2,
\end{equation*}
where $u_1=a v $ and $u_2\bot v $ for some scalar $a \in \mathbb{F}$. To obtain such a decomposition, write $u_2=u-u_1=u-av$. Then, for $u_2 $ to be
orthogonal to $v$, we need
\begin{equation*}
    0 = \inner{u-av}{v} = \inner{u}{v} - a\norm{v}^2.
\end{equation*}
Solving for $a $ yields $a=\inner{u}{v}/\norm{v}^2 $ so that
\begin{equation}\label{eq:orthogonal decomp}
    u=\frac{\inner{u}{v}}{\norm{v}^2} v + \left( u-\frac{\inner{u}{v}}{\norm{v}^2} v \right). \tag{9.3.1}
\end{equation}

This decomposition is particularly useful since it allows us to provide a simple proof for the Cauchy-Schwarz inequality.

Theorem 9.3.3 (Cauchy-Schwarz inequality). Given any $u,v\in V$, we have

\[ |\inner{u}{v}| \le \norm{u} \norm{v}. \]

Furthermore, equality holds if and only if $u $ and $v $ are linearly dependent, i.e., are scalar multiples of each other.

Proof. If $v=0$, then both sides of the inequality are zero. Hence, assume that $v\neq 0$, and consider the orthogonal decomposition
\begin{equation*}
    u = \frac{\inner{u}{v}}{\norm{v}^2} v + w
\end{equation*}
where $w\bot v$. By the Pythagorean theorem, we have
\begin{equation*}
    \norm{u}^2 = \left\| \frac{\inner{u}{v}}{\norm{v}^2}v \right\|^2 + \norm{w}^2
    = \frac{|\inner{u}{v}|^2}{\norm{v}^2} + \norm{w}^2 \ge \frac{|\inner{u}{v}|^2}{\norm{v}^2}.
\end{equation*}
Multiplying both sides by $\norm{v}^2 $ and taking the square root then yields the Cauchy-Schwarz inequality.
Note that we get equality in the above arguments if and only if $w=0$. But, by Equation (9.3.1), this means that $u $ and $v $ are linearly dependent.

The Cauchy-Schwarz inequality has many different proofs. Here is another one.

Alternate Proof of Theorem 9.3.3. Given $u,v\in V$, consider the norm square of the vector $u+r e^{i\theta}v:$
\begin{equation*}
    0 \le \norm{u+re^{i\theta}v}^2
    = \norm{u}^2 + r^2 \norm{v}^2 + 2 \mathrm{Re} (r e^{i\theta} \inner{u}{v}).
\end{equation*}
Since $ \inner{u}{v} $ is a complex number, one can choose $\theta $ so that $e^{i \theta} \inner{u}{v} $ is real. Hence, the right hand side is a parabola $a r^2 +b r +c $ with real coefficients. It will lie above the real axis, i.e. $ar^2+br+c\ge 0$, if it does not have any real solutions for $r$. This is the case when the discriminant satisfies $b^2-4ac\le 0$. In our case this means
\begin{equation*}
    4 | \inner{u}{v} |^2 -4 \norm{u}^2 \norm{v}^2\le 0.
\end{equation*}

Moreover, equality only holds if $r $ can be chosen such that $u+r e^{i\theta} v =0$, which means that $u $ and $v $ are scalar multiples.

Now that we have proven the Cauchy-Schwarz inequality, we are finally able to verify the triangle inequality. This is the final step in showing that $\norm{v}=\sqrt{\inner{v}{v}}$ does indeed define a norm. We illustrate the triangle inequality in Figure 9.3.1.

Figure 9.3.1: The Triangle Inequality in $ \mathbb{R^2} $

Theorem 9.3.4. (Triangle Inequality). For all $u,v\in V $ we have

\[ \norm{u+v} \le \norm{u} + \norm{v}. \]

Proof. By a straightforward calculation, we obtain
\begin{equation*}
\begin{split}
    \norm{u+v}^2 &= \inner{u+v}{u+v} = \inner{u}{u} + \inner{v}{v} +\inner{u}{v} + \inner{v}{u}\\
    &= \inner{u}{u} + \inner{v}{v} + \inner{u}{v} +\overline{\inner{u}{v}}
    = \norm{u}^2 + \norm{v}^2 +2 \mathrm{Re} \inner{u}{v}.
\end{split}
\end{equation*}
Note that $\mathrm{Re}\inner{u}{v}\le |\inner{u}{v}| $ so that, using the Cauchy-Schwarz inequality, we obtain
\begin{equation*}
    \norm{u+v}^2 \le \norm{u}^2 + \norm{v}^2 + 2\norm{u}\norm{v} = (\norm{u}+\norm{v})^2.
\end{equation*}
Taking the square root of both sides now gives the triangle inequality.

Remark 9.3.5. Note that equality holds for the triangle inequality if and only if $v=ru $ or $u=rv $ for some $r\ge 0$. Namely, equality in the proof happens only if $\inner{u}{v} =\norm{u}\norm{v}$, which is equivalent to $u $ and $v $ being scalar multiples of one another.

Theorem 9.3.6. (Parallelogram Law). Given any $u,v\in V$, we have
\[ \norm{u+v}^2 + \norm{u-v}^2 = 2(\norm{u}^2 + \norm{v}^2). \]

Proof. By direct calculation,
\begin{equation*}
\begin{split}
    \norm{u+v}^2 + \norm{u-v}^2 &= \inner{u+v}{u+v} + \inner{u-v}{u-v}\\
    &= \norm{u}^2 + \norm{v}^2 + \inner{u}{v} + \inner{v}{u} + \norm{u}^2 + \norm{v}^2
    - \inner{u}{v} -\inner{v}{u}\\
    &=2(\norm{u}^2 + \norm{v}^2).
\end{split}
\end{equation*}

Remark 9.3.7. We illustrate the parallelogram law in Figure 9.3.2.

Figure 9.3.2: The Parallelogram Law in $ \mathbb{R^2} $

Contributors

Both hardbound and softbound versions of this textbook are available online at WorldScientific.com.