Learning Objectives
- Determine the row space, column space, and null space of a matrix.
- Calculate the rank and nullity of a matrix and understand their relationship through the Rank-Nullity Theorem.
- Verify if a set of vectors forms a basis for the row or column space.
We begin this section with a new definition.
Definition \(\PageIndex{1}\): Row and Column Space
Let \(A\) be an \(m\times n\) matrix. The column space of \(A\), written \(\mathrm{col}(A)\), is the span of the column vectors of \(A\). The row space of \(A\), written \(\mathrm{row}(A)\), is the span of the row vectors of \(A\).
Using the reduced row-echelon form, we can obtain an efficient description of the row and column space of a matrix. Consider the following lemma.
Lemma \(\PageIndex{1}\): Effect of Row Operations on Row Space
Let \(A\) and \(B\) be \(m\times n\) matrices such that \(A\) can be carried to \(B\) by elementary row \(\left[ \mbox{column} \right]\) operations. Then \(\mathrm{row}(A)=\mathrm{row}(B)\) \(\left[\mathrm{col}(A)=\mathrm{col}(B) \right]\).
- Proof
-
We will prove that the above is true for row operations, which can be easily applied to column operations.
Let \(\vec{r}_1, \vec{r}_2, \ldots, \vec{r}_m\) denote the rows of \(A\).
- If \(B\) is obtained from \(A\) by a interchanging two rows of \(A\), then \(A\) and \(B\) have exactly the same rows, so \(\mathrm{row}(B)=\mathrm{row}(A)\).
- Suppose \(p\neq 0\), and suppose that for some \(j\), \(1\leq j\leq m\), \(B\) is obtained from \(A\) by multiplying row \(j\) by \(p\). Then \[\mathrm{row}(B)=\mathrm{span}\{ \vec{r}_1, \ldots, p\vec{r}_{j}, \ldots, \vec{r}_m\}.\nonumber \] Since \[\{ \vec{r}_1, \ldots, p\vec{r}_{j}, \ldots, \vec{r}_m\} \subseteq\mathrm{row}(A),\nonumber \] it follows that \(\mathrm{row}(B)\subseteq\mathrm{row}(A)\). Conversely, since \[\{ \vec{r}_1, \ldots, \vec{r}_m\}\subseteq\mathrm{row}(B),\nonumber \] it follows that \(\mathrm{row}(A)\subseteq\mathrm{row}(B)\). Therefore, \(\mathrm{row}(B)=\mathrm{row}(A)\).
Suppose \(p\neq 0\), and suppose that for some \(i\) and \(j\), \(1\leq i,j\leq m\), \(B\) is obtained from \(A\) by adding \(p\) time row \(j\) to row \(i\). Without loss of generality, we may assume \(i<j\).
Then \[\mathrm{row}(B)=\mathrm{span}\{ \vec{r}_1, \ldots, \vec{r}_{i-1}, \vec{r}_i+p\vec{r}_j, \ldots,\vec{r}_j,\ldots, \vec{r}_m\}.\nonumber \]
Since \[\{ \vec{r}_1, \ldots, \vec{r}_{i-1}, \vec{r}_i+p\vec{r}_{j}, \ldots, \vec{r}_m\} \subseteq\mathrm{row}(A),\nonumber \] it follows that \(\mathrm{row}(B)\subseteq\mathrm{row}(A)\).
Conversely, since \[\{ \vec{r}_1, \ldots, \vec{r}_m\}\subseteq\mathrm{row}(B),\nonumber \] it follows that \(\mathrm{row}(A)\subseteq\mathrm{row}(B)\). Therefore, \(\mathrm{row}(B)=\mathrm{row}(A)\).
Consider the following lemma.
Lemma \(\PageIndex{2}\): Row Space of a reduced row-echelon form Matrix
Let \(A\) be an \(m \times n\) matrix and let \(R\) be its reduced row-echelon form. Then the nonzero rows of \(R\) form a basis of \(\mathrm{row}(R)\), and consequently of \(\mathrm{row}(A)\).
This lemma suggests that we can examine the reduced row-echelon form of a matrix in order to obtain the row space. Consider now the column space. The column space can be obtained by simply saying that it equals the span of all the columns. However, you can often get the column space as the span of fewer columns than this. A variation of the previous lemma provides a solution. Suppose \(A\) is row reduced to its reduced row-echelon form \(R\). Identify the pivot columns of \(R\) (columns which have leading ones), and take the corresponding columns of \(A\). It turns out that this forms a basis of \(\mathrm{col}(A)\).
Before proceeding to an example of this concept, we revisit the definition of rank.
Definition \(\PageIndex{2}\): Rank of a Matrix
Previously, we defined \(\mathrm{rank}(A)\) to be the number of leading entries in the row-echelon form of \(A\). Using an understanding of dimension and row space, we can now define rank as follows: \[\mbox{rank}(A) = \dim(\mathrm{row}(A))\nonumber \]
Consider the following example.
Example \(\PageIndex{1}\): Rank, Column and Row Space
Find the rank of the following matrix and describe the column and row spaces. \[A = \left[ \begin{array}{rrrrr} 1 & 2 & 1 & 3 & 2 \\ 1 & 3 & 6 & 0 & 2 \\ 3 & 7 & 8 & 6 & 6 \end{array} \right]\nonumber \]
Solution
The reduced row-echelon form of \(A\) is \[\left[ \begin{array}{rrrrr} 1 & 0 & -9 & 9 & 2 \\ 0 & 1 & 5 & -3 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] Therefore, the rank is \(2\).
Notice that the first two columns of \(R\) are pivot columns. By the discussion following Lemma \(\PageIndex{2}\), we find the corresponding columns of \(A\), in this case the first two columns. Therefore a basis for \(\mathrm{col}(A)\) is given by \[\left\{\left[ \begin{array}{r} 1 \\ 1 \\ 3 \end{array} \right] , \left[ \begin{array}{r} 2 \\ 3 \\ 7 \end{array} \right] \right\}\nonumber \]
For example, consider the third column of the original matrix. It can be written as a linear combination of the first two columns of the original matrix as follows. \[\left[ \begin{array}{r} 1 \\ 6 \\ 8 \end{array} \right] =-9\left[ \begin{array}{r} 1 \\ 1 \\ 3 \end{array} \right] +5\left[ \begin{array}{r} 2 \\ 3 \\ 7 \end{array} \right]\nonumber \]
What about an efficient description of the row space? By Lemma \(\PageIndex{2}\) we know that the nonzero rows of \(R\) create a basis of \(\mathrm{row}(A)\). For the above matrix, the row space equals \[\mathrm{row}(A) = \mathrm{span} \left\{ \left[ \begin{array}{rrrrr} 1 & 0 & -9 & 9 & 2 \end{array} \right], \left[ \begin{array}{rrrrr} 0 & 1 & 5 & -3 & 0 \end{array} \right] \right\}\nonumber \]
Notice that the column space of \(A\) is given as the span of columns of the original matrix, while the row space of \(A\) is the span of rows of the reduced row-echelon form of \(A\).
Consider another example.
Example \(\PageIndex{2}\): Rank, Column and Row Space
Find the rank of the following matrix and describe the column and row spaces. \[\left[ \begin{array}{rrrrrr} 1 & 2 & 1 & 3 & 2 \\ 1 & 3 & 6 & 0 & 2 \\ 1 & 2 & 1 & 3 & 2 \\ 1 & 3 & 2 & 4 & 0 \end{array} \right]\nonumber \]
Solution
The reduced row-echelon form is \[\left[ \begin{array}{rrrrrr} 1 & 0 & 0 & 0 & \frac{13}{2} \\ 0 & 1 & 0 & 2 & -\frac{5}{2} \\ 0 & 0 & 1 & -1 & \frac{1}{2} \\ 0 & 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] and so the rank is \(3\). The row space is given by \[\mathrm{row}(A) = \mathrm{span} \left\{ \left[ \begin{array}{ccccc} 1 & 0 & 0 & 0 & \frac{13}{2} \end{array} \right], \left[ \begin{array}{rrrrr} 0 & 1 & 0 & 2 & -\frac{5}{2} \end{array} \right] , \left[ \begin{array}{rrrrr} 0 & 0 & 1 & -1 & \frac{1}{2} \end{array} \right] \right\}\nonumber \]
Notice that the first three columns of the reduced row-echelon form are pivot columns. The column space is the span of the first three columns in the original matrix, \[\mathrm{col}(A) = \mathrm{span} \left\{ \left[ \begin{array}{r} 1 \\ 1 \\ 1 \\ 1 \end{array} \right], \; \left[ \begin{array}{r} 2 \\ 3 \\ 2 \\ 3 \end{array} \right] , \; \left[ \begin{array}{r} 1 \\ 6 \\ 1 \\ 2 \end{array} \right] \right\}\nonumber \]
Consider the solution given above for Example \(\PageIndex{2}\), where the rank of \(A\) equals \(3\). Notice that the row space and the column space each had dimension equal to \(3\). It turns out that this is not a coincidence, and this essential result is referred to as the Rank Theorem and is given now. Recall that we defined \(\mathrm{rank}(A) = \mathrm{dim}(\mathrm{row}(A))\).
Theorem \(\PageIndex{1}\): Rank Theorem
Let \(A\) be an \(m \times n\) matrix. Then \(\mathrm{dim}(\mathrm{col} (A))\), the dimension of the column space, is equal to the dimension of the row space, \(\mathrm{dim}(\mathrm{row}(A))\).
The following statements all follow from the Rank Theorem.
Corollary \(\PageIndex{1}\): Results of the Rank Theorem
Let \(A\) be a matrix. Then the following are true:
- \(\mathrm{rank}(A) = \mathrm{rank}(A^T)\).
- For \(A\) of size \(m \times n\), \(\mathrm{rank}(A) \leq m\) and \(\mathrm{rank}(A) \leq n\).
- For \(A\) of size \(n \times n\), \(A\) is invertible if and only if \(\mathrm{rank}(A) = n\).
- For invertible matrices \(B\) and \(C\) of appropriate size, \(\mathrm{rank}(A) = \mathrm{rank}(BA) = \mathrm{rank}(AC)\).
Consider the following example.
Example \(\PageIndex{3}\): Rank of the Transpose
Let \[A = \left[ \begin{array}{rr} 1 & 2 \\ -1 & 1 \end{array} \right]\nonumber \] Find \(\mathrm{rank}(A)\) and \(\mathrm{rank}(A^T)\).
Solution
To find \(\mathrm{rank}(A)\) we first row reduce to find the reduced row-echelon form. \[A = \left[ \begin{array}{rr} 1 & 2 \\ -1 & 1 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array}\right]\nonumber \]
Therefore the rank of \(A\) is \(2\). Now consider \(A^T\) given by \[A^T = \left[ \begin{array}{rr} 1 & -1 \\ 2 & 1 \end{array} \right]\nonumber \] Again we row reduce to find the reduced row-echelon form.
\[\left[ \begin{array}{rr} 1 & -1 \\ 2 & 1 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rr} 1 & 0 \\ 0 & 1 \end{array} \right]\nonumber \]
You can see that \(\mathrm{rank}(A^T) = 2\), the same as \(\mathrm{rank}(A)\).
We now define what is meant by the null space of a general \(m\times n\) matrix.
Definition \(\PageIndex{3}\): Null Space or Kernel of \(A\)
The null space of a matrix \(A\), also referred to as the kernel of \(A\), is defined as follows. \[\mathrm{null} \left( A\right) =\left\{ \vec{x} :A \vec{x} =\vec{0}\right\}\nonumber \]
It can also be referred to using the notation \(\ker \left( A\right)\). Similarly, we can discuss the image of \(A\), denoted by \(\mathrm{im}\left( A\right)\). The image of \(A\) consists of the vectors of \(\mathbb{R}^{m}\) which “get hit” by \(A\). The formal definition is as follows.
Definition \(\PageIndex{4}\): Image of \(A\)
The image of \(A\), written \(\mathrm{im}\left( A\right)\) is given by \[\mathrm{im}\left( A \right) = \left\{ A\vec{x} : \vec{x} \in \mathbb{R}^n \right\}\nonumber \]
Consider \(A\) as a mapping from \(\mathbb{R}^{n}\) to \(\mathbb{R}^{m}\) whose action is given by multiplication. The following diagram displays this scenario. \[\overset{\mathrm{null} \left( A\right) }{\mathbb{R}^{n}}\ \overset{A}{\rightarrow }\ \overset{ \mathrm{im}\left( A\right) }{\mathbb{R}^{m}}\nonumber \] As indicated, \(\mathrm{im}\left( A\right)\) is a subset of \(\mathbb{R}^{m}\) while \(\mathrm{null} \left( A\right)\) is a subset of \(\mathbb{R}^{n}\).
It turns out that the null space and image of \(A\) are both subspaces. Consider the following example.
Example \(\PageIndex{4}\): Null Space
Let \(A\) be an \(m\times n\) matrix. Then the null space of \(A\), \(\mathrm{null}(A)\) is a subspace of \(\mathbb{R}^n\).
Solution
- Since \(A\vec{0}_n=\vec{0}_m\), \(\vec{0}_n\in\mathrm{null}(A)\).
- Let \(\vec{x},\vec{y}\in\mathrm{null}(A)\). Then \(A\vec{x}=\vec{0}_m\) and \(A\vec{y}=\vec{0}_m\), so \[A(\vec{x}+\vec{y})=A\vec{x}+A\vec{y} = \vec{0}_m+\vec{0}_m=\vec{0}_m,\nonumber \] and thus \(\vec{x}+\vec{y}\in\mathrm{null}(A)\).
- Let \(\vec{x}\in\mathrm{null}(A)\) and \(k\in\mathbb{R}\). Then \(A\vec{x}=\vec{0}_m\), so \[A(k\vec{x}) = k(A\vec{x})=k\vec{0}_m=\vec{0}_m,\nonumber \] and thus \(k\vec{x}\in\mathrm{null}(A)\).
Therefore by the subspace test, \(\mathrm{null}(A)\) is a subspace of \(\mathbb{R}^n\).
The proof that \(\mathrm{im}(A)\) is a subspace of \(\mathbb{R}^m\) is similar and is left as an exercise to the reader.
We now wish to find a way to describe \(\mathrm{null}(A)\) for a matrix \(A\). However, finding \(\mathrm{null} \left( A\right)\) is not new! There is just some new terminology being used, as \(\mathrm{null} \left( A\right)\) is simply the solution to the system \(A\vec{x}=\vec{0}\).
Theorem \(\PageIndex{2}\): Basis of null(A)
Let \(A\) be an \(m \times n\) matrix such that \(\mathrm{rank}(A) = r\). Then the system \(A\vec{x}=\vec{0}_m\) has \(n-r\) basic solutions, providing a basis of \(\mathrm{null}(A)\) with \(\dim(\mathrm{null}(A))=n-r\).
Consider the following example.
Example \(\PageIndex{5}\): Null Space of \(A\)
Let \[A=\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & -1 & 1 \\ 2 & 3 & 3 \end{array} \right]\nonumber \] Find \(\mathrm{null} \left( A\right)\) and \(\mathrm{im}\left( A\right)\).
Solution
In order to find \(\mathrm{null} \left( A\right)\), we simply need to solve the equation \(A\vec{x}=\vec{0}\). This is the usual procedure of writing the augmented matrix, finding the reduced row-echelon form and then the solution. The augmented matrix and corresponding reduced row-echelon form are \[\left[ \begin{array}{rrr|r} 1 & 2 & 1 & 0 \\ 0 & -1 & 1 & 0 \\ 2 & 3 & 3 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & 3 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \]
The third column is not a pivot column, and therefore the solution will contain a parameter. The solution to the system \(A\vec{x}=\vec{0}\) is given by \[\left[ \begin{array}{r} -3t \\ t \\ t \end{array} \right] :t\in \mathbb{R}\nonumber \] which can be written as \[t \left[ \begin{array}{r} -3 \\ 1 \\ 1 \end{array} \right] :t\in \mathbb{R}\nonumber \]
Therefore, the null space of \(A\) is all multiples of this vector, which we can write as \[\mathrm{null} (A) = \mathrm{span} \left\{ \left[ \begin{array}{r} -3 \\ 1 \\ 1 \end{array} \right] \right\}\nonumber \]
Finally \(\mathrm{im}\left( A\right)\) is just \(\left\{ A\vec{x} : \vec{x} \in \mathbb{R}^n \right\}\) and hence consists of the span of all columns of \(A\), that is \(\mathrm{im}\left( A\right) = \mathrm{col} (A)\).
Notice from the above calculation that that the first two columns of the reduced row-echelon form are pivot columns. Thus the column space is the span of the first two columns in the original matrix, and we get \[\mathrm{im}\left( A\right) = \mathrm{col}(A) = \mathrm{span} \left\{ \left[ \begin{array}{r} 1 \\ 0 \\ 2 \end{array} \right], \; \left[ \begin{array}{r} 2 \\ -1 \\ 3 \end{array} \right] \right\}\nonumber \]
Here is a larger example, but the method is entirely similar.
Example \(\PageIndex{6}\): Null Space of \(A\)
Let \[A=\left[ \begin{array}{rrrrr} 1 & 2 & 1 & 0 & 1 \\ 2 & -1 & 1 & 3 & 0 \\ 3 & 1 & 2 & 3 & 1 \\ 4 & -2 & 2 & 6 & 0 \end{array} \right]\nonumber \] Find the null space of \(A\).
Solution
To find the null space, we need to solve the equation \(AX=0\). The augmented matrix and corresponding reduced row-echelon form are given by
\[\left[ \begin{array}{rrrrr|r} 1 & 2 & 1 & 0 & 1 & 0 \\ 2 & -1 & 1 & 3 & 0 & 0 \\ 3 & 1 & 2 & 3 & 1 & 0 \\ 4 & -2 & 2 & 6 & 0 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrrrr|r} 1 & 0 & \frac{3}{5} & \frac{6}{5} & \frac{1}{5} & 0 \\ 0 & 1 & \frac{1}{5} & -\frac{3}{5} & \frac{2}{5} & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] It follows that the first two columns are pivot columns, and the next three correspond to parameters. Therefore, \(\mathrm{null} \left( A\right)\) is given by \[\left[ \begin{array}{c} \left( -\frac{3}{5}\right) s +\left( -\frac{6}{5}\right) t+\left( \frac{1}{5}\right) r \\ \left( -\frac{1}{5}\right) s +\left( \frac{3}{5}\right) t +\left( - \frac{2}{5}\right) r \\ s \\ t \\ r \end{array} \right] :s ,t ,r\in \mathbb{R}\text{.}\nonumber \] We write this in the form \[s \left[ \begin{array}{r} -\frac{3}{5} \\ -\frac{1}{5} \\ 1 \\ 0 \\ 0 \end{array} \right] + t \left[ \begin{array}{r} -\frac{6}{5} \\ \frac{3}{5} \\ 0 \\ 1 \\ 0 \end{array} \right] + r \left[ \begin{array}{r} \frac{1}{5} \\ -\frac{2}{5} \\ 0 \\ 0 \\ 1 \end{array} \right] :s , t , r\in \mathbb{R}\text{.}\nonumber \] In other words, the null space of this matrix equals the span of the three vectors above. Thus \[\mathrm{null} \left( A\right) =\mathrm{span}\left\{ \left[ \begin{array}{r} -\frac{3}{5} \\ -\frac{1}{5} \\ 1 \\ 0 \\ 0 \end{array} \right] ,\left[ \begin{array}{r} -\frac{6}{5} \\ \frac{3}{5} \\ 0 \\ 1 \\ 0 \end{array} \right] ,\left[ \begin{array}{r} \frac{1}{5} \\ -\frac{2}{5} \\ 0 \\ 0 \\ 1 \end{array} \right] \right\}\nonumber \]
Notice also that the three vectors above are linearly independent and so the dimension of \(\mathrm{null} \left( A\right)\) is 3. The following is true in general, the number of parameters in the solution of \(AX=0\) equals the dimension of the null space. Recall also that the number of leading ones in the reduced row-echelon form equals the number of pivot columns, which is the rank of the matrix, which is the same as the dimension of either the column or row space.
Before we proceed to an important theorem, we first define what is meant by the nullity of a matrix.
Definition \(\PageIndex{5}\): Nullity
The dimension of the null space of a matrix is called the nullity, denoted \(\dim( \mathrm{null}\left(A\right))\).
From our observation above we can now state an important theorem, sometimes known as the Rank-Nullity Theorem.
Theorem \(\PageIndex{3}\): Rank and Nullity
Let \(A\) be an \(m\times n\) matrix. Then \(\mathrm{rank}\left( A\right) + \dim( \mathrm{null}\left(A\right)) =n\).
Example \(\PageIndex{7}\): Rank and Nullity
Let \[A=\left[ \begin{array}{rrr} 1 & 2 & 1 \\ 0 & -1 & 1 \\ 2 & 3 & 3 \end{array} \right]\nonumber \]
Find \(\mathrm{rank}\left( A\right)\) and \(\dim( \mathrm{null}\left(A\right))\).
Solution
In the above Example \(\PageIndex{5}\) we determined that the reduced row-echelon form of \(A\) is given by \[\left[ \begin{array}{rrr} 1 & 0 & 3 \\ 0 & 1 & -1 \\ 0 & 0 & 0 \end{array} \right]\nonumber \]
Therefore the rank of \(A\) is \(2\). We also determined that the null space of \(A\) is given by \[\mathrm{null} (A) = \mathrm{span} \left\{ \left[ \begin{array}{r} -3 \\ 1 \\ 1 \end{array} \right] \right\}\nonumber \]
Therefore the nullity of \(A\) is \(1\). It follows from Theorem \(\PageIndex{3}\) that \(\mathrm{rank}\left( A\right) + \dim( \mathrm{null}\left(A\right)) = 2 + 1 = 3\), which is the number of columns of \(A\).
We conclude this section with two similar, and important, theorems.
Theorem \(\PageIndex{4}\)
Let \(A\) be an \(m\times n\) matrix. The following are equivalent.
- \(\mathrm{rank}(A)=n\).
- \(\mathrm{row}(A)=\mathbb{R}^n\), i.e., the rows of \(A\) span \(\mathbb{R}^n\).
- The columns of \(A\) are independent in \(\mathbb{R}^m\).
- The \(n\times n\) matrix \(A^TA\) is invertible.
- There exists an \(n\times m\) matrix \(C\) so that \(CA=I_n\).
- If \(A\vec{x}=\vec{0}_m\) for some \(\vec{x}\in\mathbb{R}^n\), then \(\vec{x}=\vec{0}_n\).
Theorem \(\PageIndex{5}\)
Let \(A\) be an \(m\times n\) matrix. The following are equivalent.
- \(\mathrm{rank}(A)=m\).
- \(\mathrm{col}(A)=\mathbb{R}^m\), i.e., the columns of \(A\) span \(\mathbb{R}^m\).
- The rows of \(A\) are independent in \(\mathbb{R}^n\).
- The \(m\times m\) matrix \(AA^T\) is invertible.
- There exists an \(n\times m\) matrix \(C\) so that \(AC=I_m\).
- The system \(A\vec{x}=\vec{b}\) is consistent for every \(\vec{b}\in\mathbb{R}^m\).
