2.9: The Rank Theorem
- Page ID
- 70193
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)- Learn to understand and use the rank theorem.
- Picture: the rank theorem.
- Theorem: rank theorem.
- Vocabulary words: rank, nullity.
In this section we present the rank theorem, which is the culmination of all of the work we have done so far.
The reader may have observed a relationship between the column space and the null space of a matrix. In Example 2.6.11 in Section 2.6, the column space and the null space of a \(3\times 2\) matrix are both lines, in \(\mathbb{R}^2 \) and \(\mathbb{R}^3 \text{,}\) respectively:
Figure \(\PageIndex{1}\)
In Example 2.4.6 in Section 2.4, the null space of the \(2\times 3\) matrix
Figure \(\PageIndex{2}\)
In Example 2.4.10 in Section 2.4, the null space of a \(3\times 3\) matrix is a line in \(\mathbb{R}^3 \text{,}\) and the column space is a plane in \(\mathbb{R}^3 \text{:}\)
Figure \(\PageIndex{3}\)
In all examples, the dimension of the column space plus the dimension of the null space is equal to the number of columns of the matrix. This is the content of the rank theorem.
The rank of a matrix \(A\text{,}\) written \(\text{rank}(A)\text{,}\) is the dimension of the column space \(\text{Col}(A)\).
The nullity of a matrix \(A\text{,}\) written \(\text{nullity}(A)\text{,}\) is the dimension of the null space \(\text{Nul}(A)\).
The rank of a matrix \(A\) gives us important information about the solutions to \(Ax=b\). Recall from Note 2.3.6 in Section 2.3 that \(Ax=b\) is consistent exactly when \(b\) is in the span of the columns of \(A\text{,}\) in other words when \(b\) is in the column space of \(A\). Thus, \(\text{rank}(A)\) is the dimension of the set of \(b\) with the property that \(Ax=b\) is consistent.
We know that the rank of \(A\) is equal to the number of pivot columns, Definition 1.2.5 in Section 1.2, (see this Theorem 2.7.1 in Section 2.7), and the nullity of \(A\) is equal to the number of free variables (see this Theorem 2.7.2 in Section 2.7), which is the number of columns without pivots. To summarize:
\[\begin{aligned} \text{rank}(A) = \dim\text{Col}(A) &= \text{the number of columns with pivots}\\ \text{nullity}(A) = \dim\text{Nul}(A) &= \text{the number of free variables}\\ &= \text{the number of columns without pivots} \end{aligned}\]
Clearly
\[ \text{#(columns with pivots)} + \text{#(columns without pivots)} = \text{#(columns)}, \nonumber \]
so we have proved the following theorem.
If \(A\) is a matrix with \(n\) columns, then
\[ \text{rank}(A) + \text{nullity}(A) = n. \nonumber \]
In other words, for any consistent system of linear equations,
\[ \text{(dim of column span)} + \text{(dim of solution set)} = \text{(number of variables)}. \nonumber \]
The rank theorem is really the culmination of this chapter, as it gives a strong relationship between the null space of a matrix (the solution set of \(Ax=0\)) with the column space (the set of vectors \(b\) making \(Ax=b\) consistent), our two primary objects of interest. The more freedom we have in choosing \(x\) the less freedom we have in choosing \(b\) and vice versa.
Here is a concrete example of the rank theorem and the interplay between the degrees of freedom we have in choosing \(x\) and \(b\) in a matrix equation \(Ax=b\).
Consider the matrices
\[A=\left(\begin{array}{ccc}1&0&0\\0&1&0\\0&0&0\end{array}\right)\quad\text{and}\quad B=\left(\begin{array}{ccc}0&0&0\\0&0&0\\0&0&1\end{array}\right).\nonumber\]
If we multiply a vector \((x,y,z)\) in \(\mathbb{R}^3 \) by \(A\) and \(B\) we obtain the vectors \(Ax = (x,y,0)\) and \(Bx = (0,0,z)\). So we can think of multiplication by \(A\) as giving the latitude and longitude of a point in \(\mathbb{R}^3 \) and we can think of multiplication by \(B\) as giving the height of a point in \(\mathbb{R}^3 \). The rank of \(A\) is 2 and the nullity is 1. Similarly, the rank of \(B\) is 1 and the nullity is 2.
These facts have natural interpretations. For the matrix \(A\text{:}\) the set of all latitudes and longitudes in \(\mathbb{R}^3 \) is a plane, and the set of points with the same latitude and longitude in \(\mathbb{R}^3 \) is a line; and for the matrix \(B\text{:}\) the set of all heights in \(\mathbb{R}^3 \) is a line, and the set of points at a given height in \(\mathbb{R}^3 \) is a plane. As the rank theorem tells us, we “trade off” having more choices for \(x\) for having more choices for \(b\text{,}\) and vice versa.
The rank theorem is a prime example of how we use the theory of linear algebra to say something qualitative about a system of equations without ever solving it. This is, in essence, the power of the subject.
Consider the following matrix and its reduced row echelon form:
Figure \(\PageIndex{4}\)
A basis for \(\text{Col}(A)\) is given by the pivot columns:
\[\left\{\left(\begin{array}{c}1\\-2\\2\end{array}\right),\:\left(\begin{array}{c}2\\-3\\4\end{array}\right)\right\},\nonumber\]
so \(\text{rank}(A) = \dim\text{Col}(A) = 2\).
Since there are two free variables \(x_3,x_4\text{,}\) the null space of \(A\) has two vectors (see Theorem 2.7.2 in Section 2.7):
\[\left\{\left(\begin{array}{c}8\\-4\\1\\0\end{array}\right),\:\left(\begin{array}{c}7\\-3\\0\\1\end{array}\right)\right\},\nonumber\]
so \(\text{nullity}(A) = 2\).
In this case, the rank theorem says that \(2 + 2 = 4\text{,}\) where 4 is the number of columns.
