3.1: Column Space

We begin with the simple geometric interpretation of matrix-vector multiplication. Namely, the multiplication of the n-by-1 vector $x$ by the m-by-n matrix $A$ produces a linear combination of the columns of A. More precisely, if $a_{j}$ denotes the jth column of A then

$$Ax = \begin{pmatrix} {a_{1}}&{a_{2}}&{\cdots}&{a_{n}} \end{pmatrix} \begin{pmatrix} {x_{1}}\\ {x_{2}}\\ {\cdots}\\ {x_{n}} \end{pmatrix} \nonumber$$

$$= x_{1}a_{1}+x_{2}a_{2}+\cdots+x_{n}a_{n} \nonumber$$

The picture that I wish to place in your mind's eye is that AxA⁢x lies in the subspace spanned by the columns of $A$. This subspace occurs so frequently that we find it useful to distinguish it with a definition.

Column Space

The column space of the m-by-n matrix $S$ is simply the span of the its columns, i.e. $Ra(S) \equiv \{Sx | x \in \mathbb{R}^{n}\}$ subspace of $\mathcal{R}^{m}$ stands for range in this context.The notation $R_{a}$ stands for range in this context.

Example

Let us examine the matrix:

$$A = \begin{pmatrix} {0}&{1}&{0}&{0}\\ {-1}&{0}&{1}&{0}\\ {0}&{0}&{0}&{1} \end{pmatrix} \nonumber$$

The column space of this matrix is:

$$Ra(A) = \{x_{1} \begin{pmatrix} {0}\\ {-1}\\ {0} \end{pmatrix}+x_{2} \begin{pmatrix} {1}\\ {0}\\ {0} \end{pmatrix}+x_{3} \begin{pmatrix} {0}\\ {1}\\ {0} \end{pmatrix}+x_{4} \begin{pmatrix} {0}\\ {0}\\ {1} \end{pmatrix} | x \in \mathbb{R}^{4}\} \nonumber$$

As the third column is simply a multiple of the first, we may write:

$$Ra(A) = \{x_{1} \begin{pmatrix} {0}\\ {1}\\ {0} \end{pmatrix}+x_{2} \begin{pmatrix} {1}\\ {0}\\ {0} \end{pmatrix}+x_{3} \begin{pmatrix} {0}\\ {0}\\ {1} \end{pmatrix} | x \in \mathbb{R}^{3}\} \nonumber$$

As the three remaining columns are linearly independent we may go no further. In this case, $Ra(⁢A)$ comprises all of $\mathbb{R}^{3}$

Method for Finding a Basis

To determine the basis for $Ra(⁢A)$ (where $A$ is an arbitrary matrix) we must find a way to discard its dependent columns. In the example above, it was easy to see that columns 1 and 3 were colinear. We seek, of course, a more systematic means of uncovering these, and perhaps other less obvious, dependencies. Such dependencies are more easily discerned from the row reduced form. In the reduction of the above problem, we come very easily to the matrix

$$A_{red} = \begin{pmatrix} {-1}&{0}&{1}&{0}\\ {0}&{1}&{0}&{0}\\ {0}&{0}&{0}&{1} \end{pmatrix} \nonumber$$

Once we have done this, we can recognize that the pivot column are the linearly independent columns of $A_{red}$. One now asks how this might help us distinguish the independent columns of A. For, although the rows of $A_{red}$ are linear combinations of the rows of $A$ pay attention to the indices of the pivot columns. In our example, columns $\{1, 2, 4\}$ are the pivot columns of $A_{red}$ and hence the first, second, and fourth columns of $A$ i.e.,

$$\{\begin{pmatrix} {0}\\ {-1}\\ {0} \end{pmatrix}, \begin{pmatrix} {1}\\ {0}\\ {0} \end{pmatrix}, \begin{pmatrix} {0}\\ {0}\\ {1} \end{pmatrix}\} \nonumber$$

comprise a basis for $Ra(A)$:

Definition: A Basis for the Column Space

Suppose $A$ is m-by-n. If columns $\{c_{j} | j = 1, \cdots, r\}$ are the pivot columns of $A_{red}$ then columns $\{c_{j} | j = 1, \cdots, r\}$ of $A$ constitute a basis for $Ra(A)$