9.2: Gram-Schmidt Orthogonalization

Suppose that $M$ is an $m$-dimensional subspace with basis

$$\{x_{1}, \cdots, x_{m}\} \nonumber$$

We transform this into an orthonormal basis

$$\{q_{1}, \cdots, q_{m}\} \nonumber$$

for $M$ via

1. Set $y_{1} = x_{1}$ and $q_{1} = \frac{y_{1}}{||y_{1}||}$

2. $y_{2} = x_{2}$ minus the projection of $x_{2}$ onto the line spanned by $q_{1}$.

That is,

$$y_{2} = x_{2}-q_{1}(q_{1}^{T}q_{1})^{-1}q_{1}^{T}x_{2} = x_{2}-q_{1}q_{1}^{T}x_{2} \nonumber$$

Set $q_{2} = \frac{y_{2}}{||y_{2}||}$ and $Q_{2} = \{q_{1}, q_{2}\}$

3. $y_{3} = x_{3}$ minus the projection of $x_{3}$ onto the plane spanned by $q_{1}$ and $q_{2}$. That is,

$$y_{3} = x_{3}-Q_{2}(Q_{2}^{T}Q_{2})^{-1}Q_{2}^{T} x_{3} = x_{3}-q_{1}q_{1}^{T}x_{3} \nonumber$$

Set $q_{3} = \frac{y_{3}}{||y_{3}||}$ and $Q_{3} = \{q_{1}, q_{2}, q_{3}\}$. Continue in this fashion through step (m)

• (m) $y_{m} = x_{m}$ minus its projection onto the subspace spanned by the columns of $Q_{m-1}$

$$y_{m} = x_{m}-Q_{m-1}(Q_{m-1}^{T}Q_{m-1})^{-1}Q_{m-1}^{T} x_{m}x_{m}- \sum_{j=1}^{m-1} q_{j}q_{j}^{T} x_{m} \nonumber$$

Set $q_{m} = \frac{y_{m}}{||y_{m}||}$. To take a simple example, let us orthogonalize the following basis for $\mathbb{R}^3$

$$\begin{array}{ccc} {x_{1} = \begin{pmatrix} {1}\\{0}\\{0} \end{pmatrix}}&{x_{2} = \begin{pmatrix} {1}\\{1}\\{0} \end{pmatrix}}&{x_{3} = \begin{pmatrix} {1}\\{1}\\{1} \end{pmatrix}} \end{array} \nonumber$$

1. $q_{1} = y_{1} = x_{1}$
2. $y_{2} = x_{2}-q_{1}q_{1}^{T}x_{2} = \begin{pmatrix} {0}&{1}&{0} \end{pmatrix}^T$ and so, $q_{2} = y_{2}$
3. $y_{3} = x_{3}-q_{2}q_{2}^{T}x_{3} = \begin{pmatrix} {0}&{0}&{1} \end{pmatrix}^T$ and so, $q_{3} = y_{3}$

We have arrived at

$$\begin{array}{ccc} {q_{1} = \begin{pmatrix} {1}\\{0}\\{0} \end{pmatrix}}&{q_{2} = \begin{pmatrix} {0}\\{1}\\{0} \end{pmatrix}}&{q_{3} = \begin{pmatrix} {0}\\{0}\\{1} \end{pmatrix}} \end{array} \nonumber$$

Once the idea is grasped the actual calculations are best left to a machine. Matlab accomplishes this via the orth command. Its implementation is a bit more sophisticated than a blind run through our steps (1) through (m). As a result, there is no guarantee that it will return the same basis. For example

>>X=[1 1 1;0 1 1 ;0 0 1];

>>Q=orth(X)

Q=

  0.7370  -0.5910  0.3280

  0.5910   0.3280 -0.7370

  0.3280   0.7370  0.5910

This ambiguity does not bother us, for one orthogonal basis is as good as another. Let us put this into practice, via (10.8).