3.8: QR Factorization
The Gram-Schmidt process naturally leads to a matrix factorization. Let \(\text{A}\) be an \(m\)-by-\(n\) matrix with \(n\) linearly-independent columns given by \(\{\text{x}_1,\: \text{x}_2,\cdots , \text{x}_n\}\). Following the Gram-Schmidt process, it is always possible to construct an orthornormal basis for the column space of \(\text{A}\), denoted by \(\{\text{q}_1,\: \text{q}_2,\cdots , \text{q}_n\}\). An important feature of this orthonormal basis is that the first \(k\) basis vectors from the orthonormal set span the same vector subspace as the first \(k\) columns of the matrix \(\text{A}\). For some coefficients \(r_{ij}\), we can therefore write
\[\begin{aligned}\text{x}_1&=r_{11}\text{q}_1, \\ \text{x}_2&=r_{12}\text{q}_1+r_{22}\text{q}_2, \\ \text{x}_3&=r_{13}\text{q}_1+r_{23}\text{q}_2+r_{33}\text{q}_3, \\ \vdots&\qquad \vdots \\ \text{x}_n&=r_{1\text{n}}\text{q}_1+r_{2\text{n}}\text{q}_2+\cdots +r_{\text{nn}}\text{q}_{\text{n}};\end{aligned} \nonumber \]
and these equations can be written in matrix form as
\[\left(\begin{array}{ccccc}|&|&|&&| \\ \text{x}_1&\text{x}_2&\text{x}_3&\cdots&\text{x}_{\text{n}} \\ |&|&|&&|\end{array}\right)=\left(\begin{array}{ccccc}|&|&|&&| \\ \text{q}_1&\text{q}_2&\text{q}_3&\cdots&\text{q}_{\text{n}} \\ |&|&|&&|\end{array}\right)\left(\begin{array}{cccc}r_{11}&r_{12}&\cdots &r_{1n} \\ 0&r_{22}&\cdots &r_{2n} \\ \vdots&\vdots&\vdots&\vdots \\ 0&0&\cdots &r_{nn}\end{array}\right).\nonumber \]
This form represents the matrix factorization called the QR factorization, and is usually written as
\[\text{A}=\text{QR},\nonumber \]
where \(\text{Q}\) is an orthogonal matrix and \(\text{R}\) is an upper triangular matrix. The diagonal elements of \(\text{R}\) can also be made non-negative by suitably adjusting the signs of the orthonormal basis vectors.
As a concrete example, we will find the the \(\text{QR}\) factorization of the matrix
\[\text{A}=\left(\begin{array}{cc}1&2\\2&1\end{array}\right)=\left(\begin{array}{cc}|&|\\ \text{a}_1&\text{a}_2 \\ |&|\end{array}\right).\nonumber \]
Applying the Gram-Schmidt process to the column vectors of \(\text{A}\), we have for the unnormalized orthogonal vectors
\[\begin{aligned}\text{q}_1&=\text{a}_1=\left(\begin{array}{c}1\\2\end{array}\right), \\ \text{q}_2&=\text{a}_2-\frac{(\text{q}_1^{\text{T}}\text{a}_2)\text{q}_1}{\text{q}_1^{\text{T}}\text{q}_1}=\left(\begin{array}{c}2\\1\end{array}\right)-\frac{4}{5}\left(\begin{array}{c}1\\2\end{array}\right)=\left(\begin{array}{r}6/5 \\ -3/5\end{array}\right)=\frac{3}{5}\left(\begin{array}{r}2\\-1\end{array}\right),\end{aligned} \nonumber \]
and normalizing, we obtain
\[\text{q}_1=\frac{1}{\sqrt{5}}\left(\begin{array}{c}1\\2\end{array}\right),\quad\text{q}_2=\frac{1}{\sqrt{5}}\left(\begin{array}{r}2\\-1\end{array}\right).\nonumber \]
The projection of the columns of \(\text{A}\) onto the set of orthonormal vectors is given by
\[\text{a}_1=(\text{a}_1^{\text{T}}\text{q}_1)\text{q}_1,\quad\text{a}_2=(\text{a}_2^{\text{T}}\text{q}_1)\text{q}_1+(\text{a}_2^{\text{T}}\text{q}_2)\text{q}_2,\nonumber \]
and with \(r_{ij}=\text{a}_j^{\text{T}}\text{q}_i\), we compute
\[\begin{aligned}r_{11}&=\text{a}_1^{\text{T}}\text{q}_1=\left(\begin{array}{cc}1&2\end{array}\right)\left(\begin{array}{c}1\\2\end{array}\right)\frac{1}{\sqrt{5}}=\sqrt{5}, \\ r_{12}&=\text{a}_2^{\text{T}}\text{q}_1=\left(\begin{array}{cc}2&1\end{array}\right)\left(\begin{array}{c}1\\2\end{array}\right)\frac{1}{\sqrt{5}}=\frac{4\sqrt{5}}{5}, \\ r_{22}&=\text{a}_2^{\text{T}}\text{q}_2=\left(\begin{array}{cc}2&1\end{array}\right)\left(\begin{array}{c}2\\-1\end{array}\right)\frac{1}{\sqrt{5}}=\frac{3\sqrt{5}}{5}.\end{aligned} \nonumber \]
The \(\text{QR}\) factorization of \(\text{A}\) is therefore given
\[\left(\begin{array}{cc}1&2\\2&1\end{array}\right)=\left(\begin{array}{cc}1/\sqrt{5}&2/\sqrt{5} \\ 2/\sqrt{5}&-1/\sqrt{5}\end{array}\right)\left(\begin{array}{cc}\sqrt{5}&4\sqrt{5}/5 \\ 0&3\sqrt{5}/5\end{array}\right).\nonumber \]