8.1E: Orthogonal Complements and Projections Exercises

Exercises for 1

solutions

2

In each case, use the Gram-Schmidt algorithm to convert the given basis $B$ of $V$ into an orthogonal basis.

1. $V = \mathbb{R}^2$, $B = \{(1, -1), (2, 1)\}$
2. $V = \mathbb{R}^2$, $B = \{(2, 1), (1, 2)\}$
3. $V = \mathbb{R}^3$, $B = \{(1, -1, 1), (1, 0, 1), (1, 1, 2)\}$
4. $V = \mathbb{R}^3$, $B = \{(0, 1, 1), (1, 1, 1), (1, -2, 2)\}$

2. $\{(2,1),\frac{3}{5}(-1,2)\}$
3. $\{(0,1,1),(1,0,0),(0,-2,2)\}$

In each case, write $\mathbf{x}$ as the sum of a vector in $U$ and a vector in $U^\perp$.

1. $\mathbf{x} = (1, 5, 7)$, $U = span \;\{(1, -2, 3), (-1, 1, 1)\}$
2. $\mathbf{x} = (2, 1, 6)$, $U = span \;\{(3, -1, 2), (2, 0, -3)\}$

3. $\mathbf{x} = (3, 1, 5, 9)$,
   $U = span \;\{(1, 0, 1, 1), (0, 1, -1, 1), (-2, 0, 1, 1)\}$

4. $\mathbf{x} = (2, 0, 1, 6)$,
   $U = span \;\{(1, 1, 1, 1), (1, 1, -1, -1), (1, -1, 1, -1)\}$

5. $\mathbf{x} = (a, b, c, d)$,
   $U = span \;\{(1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0)\}$

6. $\mathbf{x} = (a, b, c, d)$,
   $U = span \;\{(1, -1, 2, 0), (-1, 1, 1, 1)\}$

2. $\mathbf{x} = \frac{1}{182}(271,-221,1030) + \frac{1}{182}(93,403,62)$
3. $\mathbf{x}= \frac{1}{4}(1, 7, 11, 17) + \frac{1}{4}(7, -7, -7, 7)$
4. $\mathbf{x} = \frac{1}{12}(5a - 5b + c - 3d, -5a + 5b - c + 3d, a - b + 11c + 3d, -3a + 3b + 3c + 3d) + \frac{1}{12}(7a + 5b - c + 3d, 5a + 7b + c - 3d, -a + b + c -3d, 3a - 3b - 3c + 9d)$

Let $\mathbf{x} = (1, -2, 1, 6)$ in $\mathbb{R}^4$, and let $U = span \;\{(2, 1, 3, -4), (1, 2, 0, 1)\}$.

1. Compute $\proj{U}{\mathbf{x}}$.
2. Show that $\{(1, 0, 2, -3), (4, 7, 1, 2)\}$ is another orthogonal basis of $U$.
3. Use the basis in part (b) to compute $\proj{U}{\mathbf{x}}$.

1. $\frac{1}{10}(-9,3,-21,33) = \frac{3}{10}(-3,1,-7,11)$
2. $\frac{1}{70}(-63,21,-147,231) = \frac{3}{10}(-3,1,-7,11)$

In each case, use the Gram-Schmidt algorithm to find an orthogonal basis of the subspace $U$, and find the vector in $U$ closest to $\mathbf{x}$.

1. $U = span \;\{(1, 1, 1), (0, 1, 1)\}$, $\mathbf{x} = (-1, 2, 1)$
2. $U = span \;\{(1, -1, 0), (-1, 0, 1)\}$, $\mathbf{x} = (2, 1, 0)$
3. $U = span \;\{(1, 0, 1, 0), (1, 1, 1, 0), (1, 1, 0, 0)\}$, $\mathbf{x} = (2, 0, -1, 3)$
4. $U = span \;\{(1, -1, 0, 1), (1, 1, 0, 0), (1, 1, 0, 1)\}$, $\mathbf{x} = (2, 0, 3, 1)$

2. $\{(1, -1, 0), \frac{1}{2}(-1, -1, 2)\}$; $\proj{U}{\mathbf{x}} = (1, 0, -1)$
3. $\{(1, -1, 0, 1), (1, 1, 0, 0), \frac{1}{3}(-1, 1, 0, 2)\}$; $\proj{U}{\mathbf{x}} = (2, 0, 0, 1)$

Let $U = span \;\{\mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{k}\}$, $\mathbf{v}_{i}$ in $\mathbb{R}^n$, and let $A$ be the $k \times n$ matrix with the $\mathbf{v}_{i}$ as rows.

1. Show that $U^\perp = \{\mathbf{x} \mid \mathbf{x} \mbox{ in } \mathbb{R}^n, A\mathbf{x}^{T} = \mathbf{0}\}$.

2. Use part (a) to find $U^\perp$ if
   $U = span \;\{(1, -1, 2, 1), (1, 0, -1, 1)\}$.

2. $U^\perp = span \;\{(1, 3, 1, 0), (-1, 0, 0, 1)\}$

[ex:8_1_6]

1. Prove part 1 of Lemma [lem:023783].
2. Prove part 2 of Lemma [lem:023783].

[ex:8_1_7] Let $U$ be a subspace of $\mathbb{R}^n$. If $\mathbf{x}$ in $\mathbb{R}^n$ can be written in any way at all as $\mathbf{x} = \mathbf{p} + \mathbf{q}$ with $\mathbf{p}$ in $U$ and $\mathbf{q}$ in $U^\perp$, show that necessarily $\mathbf{p} = \proj{U}{\mathbf{x}}$.

Let $U$ be a subspace of $\mathbb{R}^n$ and let $\mathbf{x}$ be a vector in $\mathbb{R}^n$. Using Exercise [ex:8_1_7], or otherwise, show that $\mathbf{x}$ is in $U$ if and only if $\mathbf{x} = \proj{U}{\mathbf{x}}$.

Write $\mathbf{p} = \proj{U}{\mathbf{x}}$. Then $\mathbf{p}$ is in $U$ by definition. If $\mathbf{x}$ is $U$, then $\mathbf{x} - \mathbf{p}$ is in $U$. But $\mathbf{x} - \mathbf{p}$ is also in $U^\perp$ by Theorem [thm:023885], so $\mathbf{x} - \mathbf{p}$ is in $U \cap U^\perp = \{\mathbf{0}\}$. Thus $\mathbf{x} = \mathbf{p}$.

Let $U$ be a subspace of $\mathbb{R}^n$.

1. Show that $U^\perp = \mathbb{R}^n$ if and only if $U = \{\mathbf{0}\}$.
2. Show that $U^\perp = \{\mathbf{0}\}$ if and only if $U = \mathbb{R}^n$.

If $U$ is a subspace of $\mathbb{R}^n$, show that $\proj{U}{\mathbf{x}} = \mathbf{x}$ for all $\mathbf{x}$ in $U$.

Let $\{\mathbf{f}_{1}, \mathbf{f}_{2}, \dots , \mathbf{f}_{m}\}$ be an orthonormal basis of $U$. If $\mathbf{x}$ is in $U$ the expansion theorem gives $\mathbf{x} = (\mathbf{x}\bullet \mathbf{f}_{1})\mathbf{f}_{1} + (\mathbf{x}\bullet \mathbf{f}_{2})\mathbf{f}_{2} + \dots + (\mathbf{x}\bullet \mathbf{f}_{m})\mathbf{f}_{m} = \proj{U}{\mathbf{x}}$.

If $U$ is a subspace of $\mathbb{R}^n$, show that $\mathbf{x} = \proj{U}{\mathbf{x}} + \proj{U^\perp}{\mathbf{x}}$ for all $\mathbf{x}$ in $\mathbb{R}^n$.

If $\{\mathbf{f}_{1}, \dots, \mathbf{f}_{n}\}$ is an orthogonal basis of $\mathbb{R}^n$ and $U = span \;\{\mathbf{f}_{1}, \dots, \mathbf{f}_{m}\}$, show that
$U^\perp = span \;\{\mathbf{f}_{m + 1}, \dots, \mathbf{f}_{n}\}$.

[ex:8_1_13] If $U$ is a subspace of $\mathbb{R}^n$, show that $U^{\perp \perp} = U$. [Hint: Show that $U \subseteq U^{\perp \perp}$, then use Theorem [thm:023953] (3) twice.]

If $U$ is a subspace of $\mathbb{R}^n$, show how to find an $n \times n$ matrix $A$ such that $U = \{\mathbf{x} \mid A\mathbf{x} = \mathbf{0}\}$. [Hint: Exercise [ex:8_1_13].]

Let $\{\mathbf{y}_{1}, \mathbf{y}_{2}, \dots, \mathbf{y}_{m}\}$ be a basis of $U^\perp$, and let $A$ be the $n \times n$ matrix with rows $\mathbf{y}^T_1, \mathbf{y}^T_2, \dots, \mathbf{y}^T_m, 0, \dots, 0$. Then $A\mathbf{x} = \mathbf{0}$ if and only if $\mathbf{y}_{i}\bullet \mathbf{x} = 0$ for each $i = 1, 2, \dots, m$; if and only if $\mathbf{x}$ is in $U^{\perp \perp} = U$.

Write $\mathbb{R}^n$ as rows. If $A$ is an $n \times n$ matrix, write its null space as $\func{null }A = \{\mathbf{x} \mbox{ in } \mathbb{R}^n \mid A\mathbf{x}^{T} = \mathbf{0}\}$. Show that:

$\func{null }A = (\func{row }A)^\perp$; $\func{null }A^{T} = (\func{col }A)^\perp$.

If $U$ and $W$ are subspaces, show that $(U + W)^\perp = U^\perp \cap W^\perp$. [See Exercise [ex:5_1_22].]

[ex:8_1_17] Think of $\mathbb{R}^n$ as consisting of rows.

1. Let $E$ be an $n \times n$ matrix, and let
   $U = \{\mathbf{x} E \mid \mathbf{x} \mbox{ in } \mathbb{R}^n\}$. Show that the following are equivalent.

   1. $E^{2} = E = E^{T}$ ($E$ is a projection matrix).
   2. $(\mathbf{x} - \mathbf{x}E)\bullet (\mathbf{y}E) = 0$ for all $\mathbf{x}$ and $\mathbf{y}$ in $\mathbb{R}^n$.
   3. [Hint: For (ii) implies (iii): Write $\mathbf{x} = \mathbf{x}E + (\mathbf{x} - \mathbf{x}E)$ and use the uniqueness argument preceding the definition of $\proj{U}{\mathbf{x}}$. For (iii) implies (ii): $\mathbf{x} - \mathbf{x}E$ is in $U^\perp$ for all $\mathbf{x}$ in $\mathbb{R}^n$.]
2. If $E$ is a projection matrix, show that $I - E$ is also a projection matrix.
3. If $EF = 0 = FE$ and $E$ and $F$ are projection matrices, show that $E + F$ is also a projection matrix.
4. If $A$ is $m \times n$ and $AA^{T}$ is invertible, show that $E = A^{T}(AA^{T})^{-1}A$ is a projection matrix.

4. $E^2 = A^T(AA^T)^{-1}AA^T(AA^T)^{-1}A = A^T(AA^T)^{-1}A = E$

Let $A$ be an $n \times n$ matrix of rank $r$. Show that there is an invertible $n \times n$ matrix $U$ such that $UA$ is a row-echelon matrix with the property that the first $r$ rows are orthogonal. [Hint: Let $R$ be the row-echelon form of $A$, and use the Gram-Schmidt process on the nonzero rows of $R$ from the bottom up. Use Lemma [cor:004537].]

Let $A$ be an $(n - 1) \times n$ matrix with rows $\mathbf{x}_{1}, \mathbf{x}_{2}, \dots, \mathbf{x}_{n-1}$ and let $A_{i}$ denote the
$(n - 1) \times (n - 1)$ matrix obtained from $A$ by deleting column $i$. Define the vector $\mathbf{y}$ in $\mathbb{R}^n$ by

$$\mathbf{y} = \left[ \def\arraycolsep{1.5pt} \begin{array}{ccccc} \det A_{1} & -\det A_{2} & \det A_{3} & \cdots & (-1)^{n+1} \det A_{n} \end{array}\right] \nonumber$$

Show that:

1. $\mathbf{x}_{i}\bullet \mathbf{y} = 0$ for all $i = 1, 2, \dots , n - 1$. [Hint: Write $B_{i} = \left[ \begin{array}{c} x_{i} \\ A \end{array} \right]$ and show that $\det B_{i} = 0$.]
2. $\mathbf{y} \neq \mathbf{0}$ if and only if $\{\mathbf{x}_{1}, \mathbf{x}_{2}, \dots , \mathbf{x}_{n-1}\}$ is linearly independent. [Hint: If some $\det A_{i} \neq 0$, the rows of $A_{i}$ are linearly independent. Conversely, if the $\mathbf{x}_{i}$ are independent, consider $A = UR$ where $R$ is in reduced row-echelon form.]
3. If $\{\mathbf{x}_{1}, \mathbf{x}_{2}, \dots , \mathbf{x}_{n-1}\}$ is linearly independent, use Theorem [thm:023885](3) to show that all solutions to the system of $n - 1$ homogeneous equations

   $$A\mathbf{x}^T = \mathbf{0} \nonumber$$