Skip to main content
\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)
Mathematics LibreTexts

9.2: Gram-Schmidt Orthogonalization

  • Page ID
    21855
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)

    Suppose that \(M\) is an \(m\)-dimensional subspace with basis

    \[\{x_{1}, \cdots, x_{m}\} \nonumber\]

    We transform this into an orthonormal basis

    \[\{q_{1}, \cdots, q_{m}\} \nonumber\]

    for \(M\) via

    1. Set \(y_{1} = x_{1}\) and \(q_{1} = \frac{y_{1}}{||y_{1}||}\)

    2. \(y_{2} = x_{2}\) minus the projection of \(x_{2}\) onto the line spanned by \(q_{1}\).

    That is,\[y_{2} = x_{2}-q_{1}(q_{1}^{T}q_{1})^{-1}q_{1}^{T}x_{2} = x_{2}-q_{1}q_{1}^{T}x_{2} \nonumber\]

    Set \(q_{2} = \frac{y_{2}}{||y_{2}||}\) and \(Q_{2} = \{q_{1}, q_{2}\}\)

    3. \(y_{3} = x_{3}\) minus the projection of \(x_{3}\) onto the plane spanned by \(q_{1}\) and \(q_{2}\). That is,

    \[y_{3} = x_{3}-Q_{2}(Q_{2}^{T}Q_{2})^{-1}Q_{2}^{T} x_{3} = x_{3}-q_{1}q_{1}^{T}x_{3} \nonumber\]

    Set \(q_{3} = \frac{y_{3}}{||y_{3}||}\) and \(Q_{3} = \{q_{1}, q_{2}, q_{3}\}\). Continue in this fashion through step (m)

    • (m) \(y_{m} = x_{m}\) minus its projection onto the subspace spanned by the columns of \(Q_{m-1}\)

    \[y_{m} = x_{m}-Q_{m-1}(Q_{m-1}^{T}Q_{m-1})^{-1}Q_{m-1}^{T} x_{m}x_{m}- \sum_{j=1}^{m-1} q_{j}q_{j}^{T} x_{m} \nonumber\]

    Set \(q_{m} = \frac{y_{m}}{||y_{m}||}\). To take a simple example, let us orthogonalize the following basis for \(\mathbb{R}^3\)

    \[\begin{array}{ccc} {x_{1} = \begin{pmatrix} {1}\\{0}\\{0} \end{pmatrix}}&{x_{2} = \begin{pmatrix} {1}\\{1}\\{0} \end{pmatrix}}&{x_{3} = \begin{pmatrix} {1}\\{1}\\{1} \end{pmatrix}} \end{array} \nonumber\]

    1. \(q_{1} = y_{1} = x_{1}\)
    2. \(y_{2} = x_{2}-q_{1}q_{1}^{T}x_{2} = \begin{pmatrix} {0}&{1}&{0} \end{pmatrix}^T\) and so, \(q_{2} = y_{2}\)
    3. \(y_{3} = x_{3}-q_{2}q_{2}^{T}x_{3} = \begin{pmatrix} {0}&{0}&{1} \end{pmatrix}^T\) and so, \(q_{3} = y_{3}\)

    We have arrived at

    \[\begin{array}{ccc} {q_{1} = \begin{pmatrix} {1}\\{0}\\{0} \end{pmatrix}}&{q_{2} = \begin{pmatrix} {0}\\{1}\\{0} \end{pmatrix}}&{q_{3} = \begin{pmatrix} {0}\\{0}\\{1} \end{pmatrix}} \end{array} \nonumber\]

    Once the idea is grasped the actual calculations are best left to a machine. Matlab accomplishes this via the orth command. Its implementation is a bit more sophisticated than a blind run through our steps (1) through (m). As a result, there is no guarantee that it will return the same basis. For example

    >>X=[1 1 1;0 1 1 ;0 0 1];
    
    >>Q=orth(X)
    
    Q=
    
      0.7370  -0.5910  0.3280
    
      0.5910   0.3280 -0.7370
    
      0.3280   0.7370  0.5910

    This ambiguity does not bother us, for one orthogonal basis is as good as another. Let us put this into practice, via (10.8).