3.3: Eigenvalues and Eigenvectors

The world is filled with examples of systems that evolve in time—the weather in a region, the economy of a nation, the diversity of an ecosystem, etc. Describing such systems is difficult in general and various methods have been developed in special cases. In this section we describe one such method, called diagonalization, which is one of the most important techniques in linear algebra. A very fertile example of this procedure is in modelling the growth of the population of an animal species. This has attracted more attention in recent years with the ever increasing awareness that many species are endangered. To motivate the technique, we begin by setting up a simple model of a bird population in which we make assumptions about survival and reproduction rates.

Example \(\PageIndex{1}\)

Consider the evolution of the population of a species of birds. Because the number of males and females are nearly equal, we count only females. We assume that each female remains a juvenile for one year and then becomes an adult, and that only adults have offspring. We make three assumptions about reproduction and survival rates:

1. The number of juvenile females hatched in any year is twice the number of adult females alive the year before (we say the reproduction rate is 2).
2. Half of the adult females in any year survive to the next year (the adult survival rate is \(\frac{1}{2}\)).
3. One quarter of the juvenile females in any year survive into adulthood (the juvenile survival rate is \(\frac{1}{4}\)).

If there were 100 adult females and 40 juvenile females alive initially, compute the population of females \(k\) years later.

Solution

Let \(a_{k}\) and \(j_{k}\) denote, respectively, the number of adult and juvenile females after \(k\) years, so that the total female population is the sum \(a_{k} + j_{k}\). Assumption 1 shows that \(j_{k+1} = 2a_{k}\), while assumptions 2 and 3 show that \(a_{k+1} = \frac{1}{2}a_{k} + \frac{1}{4}j_{k}\). Hence the numbers \(a_{k}\) and \(j_{k}\) in successive years are related by the following equations:

\[\begin{aligned} a_{k+1} & = \frac{1}{2} a_k + \frac{1}{4} j_k \\ j_{k+1} & = 2 a_k \end{aligned} \nonumber \]

If we write \(\mathbf{v}_k = \left[ \begin{array}{c} a_k \\ j_k \end{array}\right]\) and \(A = \left[ \begin{array}{rr} \frac{1}{2} & \frac{1}{4} \\ 2 & 0 \end{array}\right]\) these equations take the matrix form

\[\mathbf{v}_{k+1} = A\mathbf{v}_k, \mbox{ for each } k = 0,1,2,\dots \nonumber \]

Taking \(k = 0\) gives \(\mathbf{v}_{1} = A\mathbf{v}_{0}\), then taking \(k = 1\) gives \(\mathbf{v}_{2} = A\mathbf{v}_{1} = A^{2}\mathbf{v}_{0}\), and taking \(k = 2\) gives \(\mathbf{v}_{3} = A\mathbf{v}_{2} = A^{3}\mathbf{v}_{0}\). Continuing in this way, we get

\[\mathbf{v}_k = A^k \mathbf{v}_0, \mbox{ for each } k=0,1,2, \dots \nonumber \]

Since \(\mathbf{v}_0 = \left[ \begin{array}{c} a_0 \\ j_0 \end{array}\right] = \left[ \begin{array}{c} 100 \\ 40 \end{array}\right]\) is known, finding the population profile \(\mathbf{v}_{k}\) amounts to computing \(A^{k}\) for all \(k \geq 0\). We will complete this calculation in Example 3.5.1 after some new techniques have been developed.

Let \(A\) be a fixed \(n \times n\) matrix. A sequence \(\mathbf{v}_{0}, \mathbf{v}_{1}, \mathbf{v}_{2}, \dots\) of column vectors in \(\mathbb{R}^n\) is called a linear dynamical system¹ if \(\mathbf{v}_{0}\) is known and the other \(\mathbf{v}_{k}\) are determined (as in Example \(\PageIndex{1}\)) by the conditions

\[\mathbf{v}_{k+1} = A\mathbf{v}_k \mbox{ for each } k = 0, 1, 2, \dots \nonumber \]

These conditions are called a matrix recurrence for the vectors \(\mathbf{v}_{k}\). As in Example \(\PageIndex{1}\), they imply that

\[\mathbf{v}_k = A^k \mathbf{v}_0 \mbox{ for all }k \ge 0 \nonumber \]

so finding the columns \(\mathbf{v}_{k}\) amounts to calculating \(A^{k}\) for \(k \geq 0\).

Direct computation of the powers \(A^{k}\) of a square matrix \(A\) can be time-consuming, so we adopt an indirect method that is commonly used. The idea is to first diagonalize the matrix \(A\), that is, to find an invertible matrix \(P\) such that

\[\label{eq:diagonalizeP} P^{-1}AP=D \mbox{ is a diagonal matrix} \]

This works because the powers \(D^{k}\) of the diagonal matrix \(D\) are easy to compute, and Equation \ref{eq:diagonalizeP} enables us to compute powers \(A^{k}\) of the matrix \(A\) in terms of powers \(D^{k}\) of \(D\). Indeed, we can solve Equation \ref{eq:diagonalizeP} for \(A\) to get \(A = PDP^{-1}\). Squaring this gives

\[A^2 = (PDP^{-1})(PDP^{-1}) = PD^2P^{-1} \nonumber \]

Using this we can compute \(A^{3}\) as follows:

\[A^3 = AA^2 = (PDP^{-1})(PD^2P^{-1}) = PD^3P^{-1} \nonumber \]

Continuing in this way we obtain Theorem 3.3.1 (even if \(D\) is not diagonal).

Theorem \(\PageIndex{1}\)

If \(A = PDP^{-1}\) then \(A^{k} = PD^{k}P^{-1}\) for each \(k = 1, 2, \dots\).

Hence computing \(A^{k}\) comes down to finding an invertible matrix \(P\) as in equation Equation \ref{eq:diagonalizeP}. To do this it is necessary to first compute certain numbers (called eigenvalues) associated with the matrix \(A\).

Eigenvalues and Eigenvectors

Theorem \(\PageIndex{1}\): Eigenvalues and Eigenvectors of a Matrix

If \(A\) is an \(n \times n\) matrix, a number \(\lambda\) is called an eigenvalue of \(A\) if

\[A\mathbf{x} = \lambda \mathbf{x} \mbox{ for some column } \mathbf{x} \neq \mathbf{0} \mbox{ in } \mathbb{R}^n \nonumber \]

In this case, \(\mathbf{x}\) is called an eigenvector of \(A\) corresponding to the eigenvalue \(\lambda\), or a \(\lambda\)-eigenvector for short.

Example \(\PageIndex{2}\)

If \(A = \left[ \begin{array}{rr} 3 & 5 \\ 1 & -1 \end{array}\right]\) and \(\mathbf{x} = \left[ \begin{array}{r} 5 \\ 1 \end{array}\right]\) then \(A\mathbf{x} = 4 \mathbf{x}\) so \(\lambda = 4\) is an eigenvalue of \(A\) with corresponding eigenvector \(\mathbf{x}\).

The matrix \(A\) in Example \(\PageIndex{2}\) has another eigenvalue in addition to \(\lambda = 4\). To find it, we develop a general procedure for any \(n \times n\) matrix \(A\).

By definition a number \(\lambda\) is an eigenvalue of the \(n \times n\) matrix \(A\) if and only if \(A\mathbf{x} = \lambda\mathbf{x}\) for some column \(\mathbf{x} \neq \mathbf{0}\). This is equivalent to asking that the homogeneous system

\[(\lambda I - A)\mathbf{x} = \mathbf{0} \nonumber \]

of linear equations has a nontrivial solution \(\mathbf{x} \neq \mathbf{0}\). By Theorem 2.4.5 this happens if and only if the matrix \(\lambda I - A\) is not invertible and this, in turn, holds if and only if the determinant of the coefficient matrix is zero:

\[\det (\lambda I -A)=0 \nonumber \]

This last condition prompts the following definition:

Definition \(\PageIndex{5}\): Characteristic Polynomial of a Matrix

Characteristic Polynomial of a Matrix009024 If \(A\) is an \(n \times n\) matrix, the characteristic polynomial \(c_{A}(x)\) of \(A\) is defined by

\[c_A(x) = \det (xI - A) \nonumber \]

Note that \(c_{A}(x)\) is indeed a polynomial in the variable \(x\), and it has degree \(n\) when \(A\) is an \(n \times n\) matrix (this is illustrated in the examples below). The above discussion shows that a number \(\lambda\) is an eigenvalue of \(A\) if and only if \(c_{A}(\lambda) = 0\), that is if and only if \(\lambda\) is a root of the characteristic polynomial \(c_{A}(x)\). We record these observations in

Theorem \(\PageIndex{2}\)

Let \(A\) be an \(n \times n\) matrix.

1. The eigenvalues \(\lambda\) of \(A\) are the roots of the characteristic polynomial \(c_{A}(x)\) of \(A\).
2. The \(\lambda\)-eigenvectors \(\mathbf{x}\) are the nonzero solutions to the homogeneous system \[(\lambda I - A)\mathbf{x} = \mathbf{0} \nonumber \] of linear equations with \(\lambda I - A\) as coefficient matrix.

In practice, solving the equations in part 2 of Theorem 3.3.2 is a routine application of gaussian elimination, but finding the eigenvalues can be difficult, often requiring computers (see Section 8.5). For now, the examples and exercises will be constructed so that the roots of the characteristic polynomials are relatively easy to find (usually integers). However, the reader should not be misled by this into thinking that eigenvalues are so easily obtained for the matrices that occur in practical applications!

Example \(\PageIndex{3}\)

Find the characteristic polynomial of the matrix \(A = \left[ \begin{array}{rr} 3 & 5 \\ 1 & -1 \end{array}\right]\) discussed in Example \(\PageIndex{2}\), and then find all the eigenvalues and their eigenvectors.

Solution

Since \(xI-A = \left[ \begin{array}{rr} x & 0 \\ 0 & x \end{array}\right] - \left[ \begin{array}{rr} 3 & 5 \\ 1 & -1 \end{array}\right] = \left[ \begin{array}{cc} x - 3 & -5 \\ -1 & x+1 \end{array}\right]\) we get

\[c_A(x) = \det \left[ \begin{array}{cc} x - 3 & -5 \\ -1 & x+1 \end{array}\right] = x^2 - 2x - 8 = (x-4)(x+2) \nonumber \]

Hence, the roots of \(c_{A}(x)\) are \(\lambda_{1} = 4\) and \(\lambda_{2} = -2\), so these are the eigenvalues of \(A\). Note that \(\lambda_{1} = 4\) was the eigenvalue mentioned in Example \(\PageIndex{2}\), but we have found a new one: \(\lambda_{2} = -2\).

To find the eigenvectors corresponding to \(\lambda_{2} = -2\), observe that in this case

\[(\lambda_2 I - A)\mathbf{x} = \left[ \begin{array}{cc} \lambda_2 - 3 & -5 \\ -1 & \lambda_2+1 \end{array}\right] = \left[ \begin{array}{rr} -5 & -5 \\ -1 & -1 \end{array}\right] \nonumber \]

so the general solution to \((\lambda_{2} I - A)\mathbf{x} = \mathbf{0}\) is \(\mathbf{x} = t \left[ \begin{array}{r} -1 \\ 1 \end{array}\right]\) where \(t\) is an arbitrary real number. Hence, the eigenvectors \(\mathbf{x}\) corresponding to \(\lambda\)₂ are \(\mathbf{x} = t \left[ \begin{array}{r} -1 \\ 1 \end{array}\right]\) where \(t \neq 0\) is arbitrary. Similarly, \(\lambda_{1} = 4\) gives rise to the eigenvectors \(\mathbf{x} = t \left[ \begin{array}{r} 5 \\ 1 \end{array}\right], t \neq 0\) which includes the observation in Example \(\PageIndex{2}\).

Note that a square matrix \(A\) has many eigenvectors associated with any given eigenvalue \(\lambda\). In fact every nonzero solution \(\mathbf{x}\) of \((\lambda I - A)\mathbf{x} = \mathbf{0}\) is an eigenvector. Recall that these solutions are all linear combinations of certain basic solutions determined by the gaussian algorithm (see Theorem 1.3.2). Observe that any nonzero multiple of an eigenvector is again an eigenvector,² and such multiples are often more convenient.³ Any set of nonzero multiples of the basic solutions of \((\lambda I - A)\mathbf{x} = \mathbf{0}\) will be called a set of basic eigenvectors corresponding to \(\lambda\).

Example \(\PageIndex{4}\)

Find the characteristic polynomial, eigenvalues, and basic eigenvectors for

\[A = \left[ \begin{array}{rrr} 2 & 0 & 0 \\ 1 & 2 & -1 \\ 1 & 3 & -2 \end{array}\right] \nonumber \]

Solution

Here the characteristic polynomial is given by

\[c_A(x) = \det \left[ \begin{array}{ccc} x-2 & 0 & 0 \\ -1 & x-2 & 1 \\ -1 & -3 & x+2 \end{array}\right] = (x-2)(x-1)(x+1) \nonumber \]

so the eigenvalues are \(\lambda_{1} = 2\), \(\lambda_{2} = 1\), and \(\lambda_{3} = -1\). To find all eigenvectors for \(\lambda_{1} = 2\), compute

\[\lambda_1 I-A = \left[ \begin{array}{ccc} \lambda_1-2 & 0 & 0 \\ -1 & \lambda_1-2 & 1 \\ -1 & -3 & \lambda_1+2 \end{array}\right] = \left[ \begin{array}{rrr} 0 & 0 & 0 \\ -1 & 0 & 1 \\ -1 & -3 & 4 \end{array}\right] \nonumber \]

We want the (nonzero) solutions to \((\lambda_{1}I - A)\mathbf{x} = \mathbf{0}\). The augmented matrix becomes

\[\left[ \begin{array}{rrr|r} 0 & 0 & 0 & 0 \\ -1 & 0 & 1 & 0 \\ -1 & -3 & 4 & 0 \end{array}\right] \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & -1 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right] \nonumber \]

using row operations. Hence, the general solution \(\mathbf{x}\) to \((\lambda_{1}I - A)\mathbf{x} = \mathbf{0}\) is \(\mathbf{x} = t \left[ \begin{array}{r} 1 \\ 1 \\ 1 \end{array}\right]\) where \(t\) is arbitrary, so we can use \(\mathbf{x}_1 = \left[ \begin{array}{r} 1 \\ 1 \\ 1 \end{array}\right]\) as the basic eigenvector corresponding to \(\lambda_{1} = 2\). As the reader can verify, the gaussian algorithm gives basic eigenvectors \(\mathbf{x}_2 = \left[ \begin{array}{r} 0 \\ 1 \\ 1 \end{array}\right]\) and \(\mathbf{x}_3 = \left[ \begin{array}{r} 0 \\ \frac{1}{3} \\ 1 \end{array}\right]\) corresponding to \(\lambda_{2} = 1\) and \(\lambda_{3} = -1\), respectively. Note that to eliminate fractions, we could instead use \(3\mathbf{x}_3 = \left[ \begin{array}{r} 0 \\ 1 \\ 3 \end{array}\right]\) as the basic \(\lambda_{3}\)-eigenvector.

Example \(\PageIndex{5}\)

If \(A\) is a square matrix, show that \(A\) and \(A^{T}\) have the same characteristic polynomial, and hence the same eigenvalues.

Solution

We use the fact that \(xI - A^{T} = (xI - A)^{T}\). Then

\[c_{A^T}(x) = \det \left( xI-A^T \right) = \det \left[ (xI-A)^T \right] = \det (xI-A) = c_A(x) \nonumber \]

by Theorem 3.2.3. Hence \(c_{A^{T}}(x)\) and \(c_{A}(x)\) have the same roots, and so \(A^{T}\) and \(A\) have the same eigenvalues (by Theorem 3.3.2).

The eigenvalues of a matrix need not be distinct. For example, if \(A = \left[ \begin{array}{rr} 1 & 1 \\ 0 & 1 \end{array}\right]\) the characteristic polynomial is \((x - 1)^2\) so the eigenvalue 1 occurs twice. Furthermore, eigenvalues are usually not computed as the roots of the characteristic polynomial. There are iterative, numerical methods (for example the QR-algorithm in Section 8.5) that are much more efficient for large matrices.

A-Invariance

If \(A\) is a \(2 \times 2\) matrix, we can describe the eigenvectors of \(A\) geometrically using the following concept. A line \(L\) through the origin in \(\mathbb{R}^2\) is called \(A\)-invariant if \(A\mathbf{x}\) is in \(L\) whenever \(\mathbf{x}\) is in \(L\). If we think of \(A\) as a linear transformation \(\mathbb{R}^2 \to \mathbb{R}^2\), this asks that \(A\) carries \(L\) into itself, that is the image \(A\mathbf{x}\) of each vector \(\mathbf{x}\) in \(L\) is again in \(L\).

Example \(\PageIndex{6}\)

The \(x\) axis \(L = \left\{ \left[ \begin{array}{r} x \\ 0 \end{array} \right] \mid x \mbox{ in } \mathbb{R} \right\}\) is \(A\)-invariant for any matrix of the form

\[A = \left[ \begin{array}{rr} a & b \\ 0 & c \end{array} \right] \mbox{ because } \left[ \begin{array}{rr} a & b \\ 0 & c \end{array} \right] \left[ \begin{array}{r} x \\ 0 \end{array}\right] = \left[ \begin{array}{r} ax \\ 0 \end{array}\right] \mbox{ is } L \mbox{ for all } \mathbf{x} = \left[ \begin{array}{r} x \\ 0 \end{array}\right] \mbox{ in } L \nonumber \]

To see the connection with eigenvectors, let \(\mathbf{x} \neq \mathbf{0}\) be any nonzero vector in \(\mathbb{R}^2\) and let \(L_{\mathbf{x}}\) denote the unique line through the origin containing \(\mathbf{x}\) (see the diagram).

By the definition of scalar multiplication in Section 2.6, we see that \(L_{\mathbf{x}}\) consists of all scalar multiples of \(\mathbf{x}\), that is

\[L_{\mathbf{x}} = \mathbb{R} \mathbf{x} = \left\{ t\mathbf{x} \mid t \mbox{ in } \mathbb{R} \right\} \nonumber \]

Now suppose that \(\mathbf{x}\) is an eigenvector of \(A\), say \(A\mathbf{x} = \lambda\mathbf{x}\) for some \(\lambda\) in \(\mathbb{R}\). Then if \(t\mathbf{x}\) is in \(L_{\mathbf{x}}\) then

\[A(t\mathbf{x}) = t\left(A\mathbf{x}\right) = t(\lambda \mathbf{x}) = (t\lambda)\mathbf{x} \mbox{ is again in } L_{\mathbf{x}} \nonumber \]

That is, \(L_{\mathbf{x}}\) is \(A\)-invariant. On the other hand, if \(L_{\mathbf{x}}\) is \(A\)-invariant then \(A\mathbf{x}\) is in \(L_{\mathbf{x}}\) (since \(\mathbf{x}\) is in \(L_{\mathbf{x}}\)). Hence \(A\mathbf{x} = t\mathbf{x}\) for some \(t\) in \(\mathbb{R}\), so \(\mathbf{x}\) is an eigenvector for \(A\) (with eigenvalue \(t\)). This proves:

Theorem \(\PageIndex{3}\)

Let \(A\) be a \(2 \times 2\) matrix, let \(\mathbf{x} \neq \mathbf{0}\) be a vector in \(\mathbb{R}^2\), and let \(L_{\mathbf{x}}\) be the line through the origin in \(\mathbb{R}^2\) containing \(\mathbf{x}\). Then

\[\mathbf{x} \mbox{ is an eigenvector of } A \quad \mbox{ if and only if } \quad L_{\mathbf{x}} \mbox{ is } A\mbox{-invariant} \nonumber \]

Example \(\PageIndex{7}\)

1. If \(\theta\) is not a multiple of \(\pi\), show that \(A = \left[ \begin{array}{cc} \cos \theta & - \sin \theta \\ \sin \theta & \cos \theta \end{array}\right]\) has no real eigenvalue.
2. If \(m\) is real show that \(B = \frac{1}{1+m^2} \left[ \begin{array}{cc} 1-m^2 & 2m \\ 2m & m^2 -1 \end{array}\right]\) has a 1 as an eigenvalue.

Solution

1. \(A\) induces rotation about the origin through the angle \(\theta\) (Theorem 2.2.15). Since \(\theta\) is not a multiple of \(\pi\), this shows that no line through the origin is \(A\)-invariant. Hence \(A\) has no eigenvector by Theorem 3.3.3, and so has no eigenvalue.
2. \(B\) induces reflection \(Q_{m}\) in the line through the origin with slope \(m\) by Theorem 2.6.5. If \(\mathbf{x}\) is any nonzero point on this line then it is clear that \(Q_{m}\mathbf{x} = \mathbf{x}\), that is \(Q_{m}\mathbf{x} = 1\mathbf{x}\). Hence 1 is an eigenvalue (with eigenvector \(\mathbf{x}\)).

​​​​If \(\theta = \frac{\pi}{2}\) in Theorem 3.3.7, then \(A = \left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array}\right]\) so \(c_{A}(x) = x^{2} + 1\). This polynomial has no root in \(\mathbb{R}\), so \(A\) has no (real) eigenvalue, and hence no eigenvector. In fact its eigenvalues are the complex numbers \(i\) and \(-i\), with corresponding eigenvectors \(\left[ \begin{array}{r} 1 \\ -i \end{array}\right]\) and \(\left[ \begin{array}{r} 1 \\ i \end{array}\right]\) In other words, \(A\) has eigenvalues and eigenvectors, just not real ones.

Note that every polynomial has complex roots,⁴ so every matrix has complex eigenvalues. While these eigenvalues may very well be real, this suggests that we really should be doing linear algebra over the complex numbers. Indeed, everything we have done (gaussian elimination, matrix algebra, determinants, etc.) works if all the scalars are complex.

---

1. More precisely, this is a linear discrete dynamical system. Many models regard \(\mathbf{v}_{t}\) as a continuous function of the time \(t\), and replace our condition between \(\mathbf{b}_{k+1}\) and \(A\mathbf{v}_k\) with a differential relationship viewed as functions of time.
2. In fact, any nonzero linear combination of \(\lambda\)-eigenvectors is again a \(\lambda\)-eigenvector.
3. Allowing nonzero multiples helps eliminate round-off error when the eigenvectors involve fractions.
4. This is called the Fundamental Theorem of Algebra and was first proved by Gauss in his doctoral dissertation.