1.7.3: Rank and Nullity
- Page ID
- 186298
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\( \newcommand{\dsum}{\displaystyle\sum\limits} \)
\( \newcommand{\dint}{\displaystyle\int\limits} \)
\( \newcommand{\dlim}{\displaystyle\lim\limits} \)
\( \newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\)
( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\id}{\mathrm{id}}\)
\( \newcommand{\Span}{\mathrm{span}}\)
\( \newcommand{\kernel}{\mathrm{null}\,}\)
\( \newcommand{\range}{\mathrm{range}\,}\)
\( \newcommand{\RealPart}{\mathrm{Re}}\)
\( \newcommand{\ImaginaryPart}{\mathrm{Im}}\)
\( \newcommand{\Argument}{\mathrm{Arg}}\)
\( \newcommand{\norm}[1]{\| #1 \|}\)
\( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\)
\( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\AA}{\unicode[.8,0]{x212B}}\)
\( \newcommand{\vectorA}[1]{\vec{#1}} % arrow\)
\( \newcommand{\vectorAt}[1]{\vec{\text{#1}}} % arrow\)
\( \newcommand{\vectorB}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\( \newcommand{\vectorC}[1]{\textbf{#1}} \)
\( \newcommand{\vectorD}[1]{\overrightarrow{#1}} \)
\( \newcommand{\vectorDt}[1]{\overrightarrow{\text{#1}}} \)
\( \newcommand{\vectE}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash{\mathbf {#1}}}} \)
\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)
\(\newcommand{\longvect}{\overrightarrow}\)
\( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)
\(\newcommand{\avec}{\mathbf a}\) \(\newcommand{\bvec}{\mathbf b}\) \(\newcommand{\cvec}{\mathbf c}\) \(\newcommand{\dvec}{\mathbf d}\) \(\newcommand{\dtil}{\widetilde{\mathbf d}}\) \(\newcommand{\evec}{\mathbf e}\) \(\newcommand{\fvec}{\mathbf f}\) \(\newcommand{\nvec}{\mathbf n}\) \(\newcommand{\pvec}{\mathbf p}\) \(\newcommand{\qvec}{\mathbf q}\) \(\newcommand{\svec}{\mathbf s}\) \(\newcommand{\tvec}{\mathbf t}\) \(\newcommand{\uvec}{\mathbf u}\) \(\newcommand{\vvec}{\mathbf v}\) \(\newcommand{\wvec}{\mathbf w}\) \(\newcommand{\xvec}{\mathbf x}\) \(\newcommand{\yvec}{\mathbf y}\) \(\newcommand{\zvec}{\mathbf z}\) \(\newcommand{\rvec}{\mathbf r}\) \(\newcommand{\mvec}{\mathbf m}\) \(\newcommand{\zerovec}{\mathbf 0}\) \(\newcommand{\onevec}{\mathbf 1}\) \(\newcommand{\real}{\mathbb R}\) \(\newcommand{\twovec}[2]{\left[\begin{array}{r}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\ctwovec}[2]{\left[\begin{array}{c}#1 \\ #2 \end{array}\right]}\) \(\newcommand{\threevec}[3]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\cthreevec}[3]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \end{array}\right]}\) \(\newcommand{\fourvec}[4]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\cfourvec}[4]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \end{array}\right]}\) \(\newcommand{\fivevec}[5]{\left[\begin{array}{r}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\cfivevec}[5]{\left[\begin{array}{c}#1 \\ #2 \\ #3 \\ #4 \\ #5 \\ \end{array}\right]}\) \(\newcommand{\mattwo}[4]{\left[\begin{array}{rr}#1 \amp #2 \\ #3 \amp #4 \\ \end{array}\right]}\) \(\newcommand{\laspan}[1]{\text{Span}\{#1\}}\) \(\newcommand{\bcal}{\cal B}\) \(\newcommand{\ccal}{\cal C}\) \(\newcommand{\scal}{\cal S}\) \(\newcommand{\wcal}{\cal W}\) \(\newcommand{\ecal}{\cal E}\) \(\newcommand{\coords}[2]{\left\{#1\right\}_{#2}}\) \(\newcommand{\gray}[1]{\color{gray}{#1}}\) \(\newcommand{\lgray}[1]{\color{lightgray}{#1}}\) \(\newcommand{\rank}{\operatorname{rank}}\) \(\newcommand{\row}{\text{Row}}\) \(\newcommand{\col}{\text{Col}}\) \(\renewcommand{\row}{\text{Row}}\) \(\newcommand{\nul}{\text{Nul}}\) \(\newcommand{\var}{\text{Var}}\) \(\newcommand{\corr}{\text{corr}}\) \(\newcommand{\len}[1]{\left|#1\right|}\) \(\newcommand{\bbar}{\overline{\bvec}}\) \(\newcommand{\bhat}{\widehat{\bvec}}\) \(\newcommand{\bperp}{\bvec^\perp}\) \(\newcommand{\xhat}{\widehat{\xvec}}\) \(\newcommand{\vhat}{\widehat{\vvec}}\) \(\newcommand{\uhat}{\widehat{\uvec}}\) \(\newcommand{\what}{\widehat{\wvec}}\) \(\newcommand{\Sighat}{\widehat{\Sigma}}\) \(\newcommand{\lt}{<}\) \(\newcommand{\gt}{>}\) \(\newcommand{\amp}{&}\) \(\definecolor{fillinmathshade}{gray}{0.9}\)Since each matrix \(A\) has two important subspaces related to it (the column space and null space), we give the dimensions of those subspaces names.
The rank of a matrix \(A\), written \(\text{rank}(A)\text{,}\) is the dimension of the column space \(\text{Col}(A)\).
The nullity of a matrix \(A\), written \(\text{nullity}(A)\text{,}\) is the dimension of the null space \(\text{Null}(A)\).
The rank of a matrix \(A\) gives us important information about the solutions to \(A\vec{x}=\vec{b}\). Recall that \(A\vec{x}=\vec{b}\) is consistent exactly when \(\vec{b}\) is in the span of the columns of \(A\text{,}\) in other words when \(\vec{b}\) is in the column space of \(A\). Thus, \(\text{rank}(A)\) is the dimension of the set of \(b\) with the property that \(A\vec{x}=\vec{b}\) is consistent.
We know that the rank of \(A\) is equal to the number of pivot columns, and the nullity of \(A\) is equal to the number of free variables, which is the number of columns without pivots. To summarize:
\[\begin{aligned} \text{rank}(A) = \dim\left(\text{Col}(A)\right) &= \text{the number of columns with pivots}\\ \text{nullity}(A) = \dim\left(\text{Null}(A)\right) &= \text{the number of free variables}\\ &= \text{the number of columns without pivots} \end{aligned} \nonumber \]
Clearly
\[ \text{(columns with pivots)} + \text{(columns without pivots)} = \text{(columns)} \nonumber \]
so we have proved the following theorem.
If \(A\) is a matrix with \(n\) columns, then
\[ \text{rank}(A) + \text{nullity}(A) = n \nonumber \]
In other words, for any consistent system of linear equations,
\[ \text{(dim of column span)} + \text{(dim of solution set)} = \text{(number of variables)} \nonumber \]
The rank theorem is really the culmination of this chapter, as it gives a strong relationship between the null space of a matrix (the solution set of \(A\vec{x}=0\)) with the column space (the set of vectors \(\vec{b}\) making \(A\vec{x}=\vec{b}\) consistent), our two primary objects of interest. The more freedom we have in choosing \(\vec{x}\) the less freedom we have in choosing \(\vec{b}\) and vice versa.
Here is a concrete example of the rank theorem and the interplay between the degrees of freedom we have in choosing \(\vec{x}\) and \(\vec{b}\) in a matrix equation \(A\vec{x}=\vec{b}\).
Consider the matrices
\[A=\left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&0\end{array}\right]\quad\text{and}\quad B=\left[\begin{array}{ccc}0&0&0\\0&0&0\\0&0&1\end{array}\right]\nonumber\]
If we multiply a vector \(\left[\begin{array} {c} x\\y\\z \end{array}\right]\) in \(\mathbb{R}^3 \) by \(A\) and \(B\) we obtain the vectors \(A\left[\begin{array} {c} x\\y\\z \end{array}\right] = \left[\begin{array} {c} x\\y\\0 \end{array}\right]\) and \(B\left[\begin{array} {c} x\\y\\z \end{array}\right] = \left[\begin{array} {c} 0\\0\\z\end{array}\right]\). So we can think of multiplication by \(A\) as giving the latitude and longitude of a point in \(\mathbb{R}^3 \) and we can think of multiplication by \(B\) as giving the height of a point in \(\mathbb{R}^3 \). The rank of \(A\) is 2 and the nullity is 1. Similarly, the rank of \(B\) is 1 and the nullity is 2.
These facts have natural interpretations. For the matrix \(A\text{:}\) the set of all latitudes and longitudes in \(\mathbb{R}^3 \) is a plane, and the set of points with the same latitude and longitude in \(\mathbb{R}^3 \) is a line; and for the matrix \(B\text{:}\) the set of all heights in \(\mathbb{R}^3 \) is a line, and the set of points at a given height in \(\mathbb{R}^3 \) is a plane. As the rank theorem tells us, we “trade off” having more choices for \(\vec{x}\) for having more choices for \(\vec{b}\text{,}\) and vice versa.
The rank theorem is a prime example of how we use the theory of linear algebra to say something qualitative about a system of equations without ever solving it. This is, in essence, the power of the subject.
Consider the following matrix and its reduced row echelon form:
Figure \(\PageIndex{4}\)
A basis for \(\text{Col}(A)\) is given by the pivot columns:
\[\left\{\left[\begin{array}{c}1\\-2\\2\end{array}\right],\:\left[\begin{array}{c}2\\-3\\4\end{array}\right]\right\}\nonumber\]
so \(\text{rank}(A) = \dim\text{Col}(A) = 2\).
Since there are two free variables \(x_3,x_4\text{,}\) the null space of \(A\) has two vectors:
\[\left\{\left[\begin{array}{c}8\\-4\\1\\0\end{array}\right],\:\left[\begin{array}{c}7\\-3\\0\\1\end{array}\right]\right\}\nonumber\]
so \(\text{nullity}(A) = 2\).
In this case, the rank theorem says that \(2 + 2 = 4\text{,}\) where 4 is the number of columns.
Consider the system of three equations using five variables \(\left\{ \begin{array} {l} a_1x_1+a_2x_2+a_3x_3+a_4x_4+ a_5x_5 = \alpha \\ b_1x_1+b_2x_2+b_3x_3+b_4x_4+ b_5x_5 = \beta \\ c_1x_1+c_2x_2+c_3x_3+c_4x_4+ c_5x_5 = \gamma \end{array} \right.\)
If we know that this system is consistent for all possible choices of \(\alpha,\beta,\gamma\), how many linearly independent solutions to the homogeneous system of equations below? \[\left\{ \begin{array} {l} a_1x_1+a_2x_2+a_3x_3+a_4x_4+ a_5x_5 = 0 \\ b_1x_1+b_2x_2+b_3x_3+b_4x_4+ b_5x_5 = 0 \\ c_1x_1+c_2x_2+c_3x_3+c_4x_4+ c_5x_5 =0\end{array} \right.\nonumber\]
Solution
We define \(A=\left[ \begin{array} {ccccc} a_1 & a_2 & a_3& a_4 &a_5 \\ b_1 & b_2 & b_3& b_4 &b_5 \\ c_1 & c_2 & c_3& c_4 &c_5 \end{array} \right]\) to make things easier to describe.
Since the system \(A\vec{x}=\vec{b}\) is consistent for all \(\vec{b}=\left[ \begin{array} {c} \alpha \\ \beta \\ \gamma \end{array} \right]\) in \(\mathbb{R}^3\), we know that \(\text{Col}(A)=\mathbb{R}^3\)
That means \(\dim(\text{Col}(A))=\text{rank}(A)=3\), and since it has 5 columns, we know \(\text{nullity}(A)=5-3=2\)
Therefore, any basis for \(\text{Null}(A)\) must contain exactly two linearly independent vectors.
This is exactly the same as the collection of solutions to \(A\vec{x}=\vec{0}\), so our answer is \(\boxed{\text{two linearly independent solutions}}\)