4.1: An introduction to Eigenvalues and Eigenvectors
\(\newcommand{\twovec}[2]{\begin{pmatrix} #1 \\ #2 \end{pmatrix} } \) \(\newcommand{\threevec}[3]{\begin{pmatrix} #1 \\ #2 \\ #3 \end{pmatrix} } \) \(\newcommand{\fourvec}[4]{\begin{pmatrix} #1 \\ #2 \\ #3 \\ #4 \end{pmatrix} } \) \(\newcommand{\fivevec}[5]{\begin{pmatrix} #1 \\ #2 \\ #3 \\ #4 \\ #5 \end{pmatrix} } \)
This section introduces the concept of eigenvalues and eigenvectors and offers an example that motivates our interest in them. The point here is to develop an intuitive understanding of eigenvalues and eigenvectors and explain how they can be used to simplify some problems that we have previously encountered. In the rest of this chapter, we will develop this concept into a richer theory and illustrate its use with more meaningful examples.
Preview Activity 4.1.1.
Before we introduce the definition of eigenvectors and eigenvalues, it will be helpful to remember some ideas we have seen previously.
-
Suppose that \(\mathbf v\) is the vector shown in the figure. Sketch the vector \(2\mathbf v\) and the vector \(-\mathbf v\text{.}\)
- State the geometric effect that scalar multiplication has on the vector \(\mathbf v\text{.}\) Then sketch all the vectors of the form \(\lambda \mathbf v\) where \(\lambda\) is a scalar.
-
State the geometric effect of the matrix transformation defined by
\begin{equation*} \left[\begin{array}{rr} 3 & 0 \\ 0 & -1 \\ \end{array}\right]\text{.} \end{equation*}
-
Suppose that \(A\) is a \(2\times 2\) matrix and that \(\mathbf v_1\) and \(\mathbf v_2\) are vectors such that
\begin{equation*} A\mathbf v_1 = 3 \mathbf v_1, \qquad A\mathbf v_2 = -\mathbf v_2\text{.} \end{equation*}
Use the linearity of matrix multiplication to express the following vectors in terms of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
- \(A(4\mathbf v_1)\text{.}\)
- \(A(\mathbf v_1 + \mathbf v_2)\text{.}\)
- \(A(4\mathbf v_1 -3\mathbf v_2)\text{.}\)
- \(A^2\mathbf v_1\text{.}\)
- \(A^2(4\mathbf v_1 - 3\mathbf v_2)\text{.}\)
- \(A^4\mathbf v_1\text{.}\)
A few examples
We will now introduce the definition of eigenvalues and eigenvectors and then look at a few simple examples.
Given a square \(n\times n\) matrix \(A\text{,}\) we say that a nonzero vector \(\mathbf v\) is an eigenvector of \(A\) if there is a scalar \(\lambda\) such that
The scalar \(\lambda\) is called the eigenvalue associated to the eigenvector \(\mathbf v\text{.}\)
At first glance, there is a lot going on in this definition so let's look at an example.
Consider the matrix \(A = \left[\begin{array}{rr} 7 & 6 \\ 6 & -2 \\ \end{array}\right]\) and the vector \(\mathbf v=\twovec{2}{1}\text{.}\) We find that
In other words, \(A\mathbf v = 10\mathbf v\text{,}\) which says that \(\mathbf v\) is an eigenvector of the matrix \(A\) with associated eigenvalue \(\lambda = 10\text{.}\)
Similarly, if \(\mathbf w = \twovec{-1}{2}\text{,}\) we find that
Here again, we have \(A\mathbf w = -5\mathbf w\) showing that \(\mathbf w\) is an eigenvector of \(A\) with associated eigenvalue \(\lambda=-5\text{.}\)
Activity 4.1.2.
This definition has an important geometric interpretation that we will investigate here.
- Suppose that \(\mathbf v\) is a nonzero vector and that \(\lambda\) is a scalar. What is the geometric relationship between \(\mathbf v\) and \(\lambda\mathbf v\text{?}\)
- Let's now consider the eigenvector condition: \(A\mathbf v = \lambda\mathbf v\text{.}\) Here we have two vectors, \(\mathbf v\) and \(A\mathbf v\text{.}\) If \(A\mathbf v = \lambda\mathbf v\text{,}\) what is the geometric relationship between \(\mathbf v\) and \(A\mathbf v\text{?}\)
- The sliders in the diagram below allow you to choose a matrix \(A\). The vector \({\mathbf v}\) is shown in red and may be varied by clicking in the head of the vector. The vector \(A{\mathbf v}\) is then shown in grayChoose the matrix \(A= \left[\begin{array}{rr} 1& 2 \\ 2& 1 \\ \end{array}\right] \text{.}\) Move the vector \(\mathbf v\) so that the eigenvector condition holds. What is the eigenvector \(\ mathbf v\) and what is the associated eigenvalue?
- By algebraically computing \(A\mathbf v\text{,}\) verify that the eigenvector condition holds for the vector \(\mathbf v\) that you found.
- If you multiply the eigenvector \(\mathbf v\) that you found by \(2\text{,}\) do you still have an eigenvector? If so, what is the associated eigenvalue?
- Are you able to find another eigenvector \(\mathbf v\) that is not a scalar multiple of the first one that you found? If so, what is the eigenvector and what is the associated eigenvalue?
- Now consider the matrix \(A = \left[\begin{array}{rr} 2 & 1 \\ 0 & 2 \\ \end{array}\right] \text{.}\) Use the diagram to describe any eigenvectors and associated eigenvalues.
- Finally, consider the matrix \(A = \left[\begin{array}{rr} 0 & -1 \\ 1 & 0 \\ \end{array}\right] \text{.}\) Use the diagram to describe any eigenvectors and associated eigenvalues. What geometric transformation does this matrix perform on vectors? How does this explain the presence of any eigenvectors?
Let's consider the ideas we saw in the activity in some more depth. To be an eigenvector of \(A\text{,}\) the vector \(\mathbf v\) must satisfy \(A\mathbf v = \lambda\mathbf v\) for some scalar \(\lambda\text{.}\) This means that \(\mathbf v\) and \(A\mathbf v\) are scalar multiples of one another, which means they must lie on the same line.
Consider now the matrix \(A = \left[\begin{array}{rr} 1 & 2 \\ 2 & 1 \\ \end{array}\right] \text{.}\) On the left of Figure 4.1.3, we see that \(\mathbf v=\twovec{1}{0}\) is not an eigenvector of \(A\) since the vectors \(\mathbf v\) and \(A\mathbf v\) do not lie on the same line. On the right, however, we see that \(\mathbf v=\twovec{1}{1}\) is an eigenvector. In fact, \(A\mathbf v\) is obtained from \(\mathbf v\) by stretching \(\mathbf v\) by a factor of \(3\text{.}\) Therefore, \(\mathbf v\) is an eigenvector of \(A\) with eigenvalue \(\lambda = 3\text{.}\)
It is not difficult to see that any multiple of \(\twovec{1}{1}\) is also an eigenvector of \(A\) with eigenvalue \(\lambda = 3\text{.}\) Indeed, we will see later that all the eigenvectors associated to a given eigenvalue form a subspace of \(\mathbb R^n\text{.}\)
In Figure 4.1.4, we see that \(\mathbf v=\twovec{-1}{1}\) is also an eigenvector with eigenvalue \(\lambda =-1\text{.}\)
The interactive diagram we used in the activity is meant to convey the fact that the eigenvectors of a matrix \(A\) are special vectors. Most of the time, the vectors \(\mathbf v\) and \(A\mathbf v\) appear somewhat unrelated. For certain vectors, however, \(\mathbf v\) and \(A\mathbf v\) line up with one another. Something important is going on when that happens so we call attention to those vectors by calling them eigenvectors. For these vectors, the operation of multiplying by \(A\) reduces to the much simpler operation of scalar multiplying by \(\lambda\text{.}\) The reason eigenvectors are important is because it is extremely convenient to be able to replace matrix multiplication by scalar multiplication.
Eigen is a German word that can be interpreted as meaning “characteristic”. As we will see, the eigenvectors and eigenvalues of a matrix \(A\) give an important characterization of the matrix.
The usefulness of eigenvalues and eigenvectors
In the next section, we will introduce an algebraic technique for finding the eigenvalues and eigenvectors of a matrix. Before doing that, however, we would like to discuss why eigenvalues and eigenvectors are so useful.
Let's continue looking at the example \(A = \left[\begin{array}{rr} 1 & 2 \\ 2 & 1 \\ \end{array}\right] \text{.}\) We have seen that \(\mathbf v_1 = \twovec{1}{1}\) is an eigenvector with eigenvalue \(\lambda=3\) and \(\mathbf v_2=\twovec{-1}{1}\) is an eigenvector with eigenvalue \(\lambda = -1\text{.}\) This means that \(A\mathbf v_1 = 3\mathbf v_1\) and \(A\mathbf v_2=-\mathbf v_2\text{.}\) By the linearity of matrix multiplication, we can easily understand what happens when we multiply a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\) by \(A\text{:}\)
For instance, if we consider the vector \(\mathbf x=\mathbf v_1-2\mathbf v_2\text{,}\) we find that
as seen in the figure.
In other words, multiplying by \(A\) has the effect of stretching a vector \(\mathbf x\) in the \(\mathbf v_1\) direction by a factor of \(3\) and flipping the \(\mathbf x\) in \(\mathbf v_2\) direction.
We can draw an analogy with the more familiar example of the diagonal matrix \(D=\left[\begin{array}{rr} 3 & 0 \\ 0 & -1 \\ \end{array}\right] \text{.}\) As we have seen, the effect of multiplying a vector \(\mathbf x\) by \(D\) is to stretch \(\mathbf x\) horizontally by a factor of \(3\) and to flip \(\mathbf x\) vertically. This is illustrated in Figure 4.1.5.
The matrix \(A=\left[\begin{array}{rr} 1 & 2 \\ 2 & 1 \\ \end{array}\right]\) has a similar effect when viewed in the basis defined by the eigenvectors \(\mathbf v_1\) and \(\mathbf v_2\text{,}\) as seen in Figure 4.1.6.
In a sense that will be made precise later, having a set of eigenvectors of \(A\) that forms a basis of \(\mathbb R^2\) enables us to think of \(A\) as being equivalent to a diagonal matrix \(D\text{.}\) Of course, as the other examples in the previous activity show, it may not always be possible to form a basis from the eigenvectors of a matrix. For example, the only eigenvectors of the matrix \(\left[\begin{array}{rr} 2 & 1 \\ 0 & 2 \end{array}\right]\text{,}\) which represents a shear, have the form \(\twovec{x}{0}\text{.}\) In this example, we are not able to create a basis for \(\mathbb R^2\) consisting of eigenvectors of the matrix. This is also true for the matrix \(\left[\begin{array}{rr} 0 & -1 \\ 1 & 0 \\ \end{array}\right] \text{,}\) which represents a \(90^\circ\) rotation.
Activity 4.1.3.
Let's consider an example that illustrates how we can put these ideas to use.
Suppose that we work for a car rental company that has two locations, \(P\) and \(Q\text{.}\) When a customer rents a car at one location, they have the option to return it to either location at the end of the day. After doing some market research, we determine:
- 80% of the cars rented at location \(P\) are returned to \(P\) and 20% are returned to \(Q\text{.}\)
- 40% of the cars rented at location \(Q\) are returned to \(Q\) and 60% are returned to \(P\text{.}\)
- Suppose that there are 1000 cars at location \(P\) and no cars at location \(Q\) on Monday morning. How many cars are there are locations \(P\) and \(Q\) at the end of the day on Monday?
- How many are at locations \(P\) and \(Q\) at end of the day on Tuesday?
-
If we let \(P_k\) and \(Q_k\) be the number of cars at locations \(P\) and \(Q\text{,}\) respectively, at the end of day \(k\text{,}\) we then have
\begin{equation*} \begin{aligned} P_{k+1}& {}={} 0.8P_k + 0.6Q_k \\ Q_{k+1}& {}={} 0.2P_k + 0.4Q_k\text{.} \\ \end{aligned} \end{equation*}
We can write the vector \(\mathbf x_k = \twovec{P_k}{Q_k}\) to reflect the number of cars at the two locations at the end of day \(k\text{,}\) which says that
\begin{equation*} \begin{aligned} \mathbf x_{k+1} & {}={} \left[\begin{array}{rr} 0.8 & 0.6 \\ 0.2 & 0.4 \\ \end{array}\right] \mathbf x_k \\ \\ \text{or}\qquad \mathbf x_{k+1} & {}={} A\mathbf x_k \end{aligned} \end{equation*}where \(A=\left[\begin{array}{rr}0.8 & 0.6 \\ 0.2 & 0.4 \end{array}\right]\text{.}\)
Suppose that
\begin{equation*} \mathbf v_1 = \twovec{3}{1}, \qquad \mathbf v_2 = \twovec{-1}{1}\text{.} \end{equation*}Compute \(A\mathbf v_1\) and \(A\mathbf v_2\) to demonstrate that \(\mathbf v_1\) and \(\mathbf v_2\) are eigenvectors of \(A\text{.}\) What are the associated eigenvalues \(\lambda_1\) and \(\lambda_2\text{?}\)
- We said that 1000 cars are initially at location \(P\) and none at location \(Q\text{.}\) This means that the initial vector describing the number of cars is \(\mathbf x_0 = \twovec{1000}{0}\text{.}\) Write \(\mathbf x_0\) as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
- Remember that \(\mathbf v_1\) and \(\mathbf v_2\) are eigenvectors of \(A\text{.}\) Use the linearity of matrix multiplicaiton to write the vector \(\mathbf x_1 = A\mathbf x_0\text{,}\) describing the number of cars at the two locations at the end of the first day, as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
-
Write the vector \(\mathbf x_2 = A\mathbf x_1\) as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\) Then write the next few vectors as linear combinations of \(\mathbf v_1\) and \(\mathbf v_2\text{:}\)
- \(\mathbf x_3 = A\mathbf x_2\text{.}\)
- \(\mathbf x_4 = A\mathbf x_3\text{.}\)
- \(\mathbf x_5 = A\mathbf x_4\text{.}\)
- \(\mathbf x_6 = A\mathbf x_5\text{.}\)
- What will happen to the number of cars at the two locations after a very long time? Explain how writing \(\mathbf x_0\) as a linear combination of eigenvectors helps you determine the long-term behavior.
This activity is important and motivates much of our work with eigenvalues and eigenvectors so we will review it now making sure we have a clear understanding of the concepts that arise.
First, we compute
This shows that \(\mathbf v_1\) is an eigenvector of \(A\) with eigenvalue \(\lambda_1 = 1\) and \(\mathbf v_2\) is an eigenvector of \(A\) with eigenvalue \(\lambda_2=0.2\text{.}\)
By the linearity of matrix matrix multiplication, we have
Therefore, we will write the vector describing the initial distribution of cars \(\mathbf x_0=\twovec{1000}{0}\) as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{;}\) that is, \(\mathbf x_0 = c_1\mathbf v_2 + c_2 \mathbf v_2\text{.}\) To do, we form the augmented matrix and row reduce:
Therefore, \(\mathbf x_0 = 250\mathbf v_1 -250\mathbf v_2\text{.}\)
To determine the distribution of cars on subsequent days, we will repeatedly multiply by \(A\text{.}\) We find that
In particular, this shows us that
Taking notice of the pattern, we may write
Multiplying a number by \(0.2\) is the same as taking 20% of that number. As each day goes by, the second term is multiplied by \(0.2\) so the coefficient of \(\mathbf v_2\) in the expression for \(\mathbf x_k\) will eventually become extremely small. We therefore see that the distribution of cars will stabilize at \(\mathbf x=250\mathbf v_1=\twovec{750}{250}\text{.}\)
Notice how our use of the eigenvalues and eigenvectors of \(A\) enable us to look far into the future without having to repeatedly multiply a vector by the matrix \(A\text{.}\) Knowing the eigenvectors allows us to replace matrix multiplication with the simpler operation of scalar multiplication. This is a powerful tool that we will develop more in the rest of this chapter.
Notice also how this example relies on the fact that we can express the initial vector \(\mathbf x_0\) as a linear combination of eigenvectors. For this reason, we would like, given an \(n\times n\) matrix, to be able to create a basis of \(\mathbb R^n\) of eigenvectors. We will frequently return to this question in later sections.
Question 4.1.7.
Given an \(n\times n\) matrix \(A\text{,}\) when can we form a basis for \(\mathbb R^n\) consisting of eigenvectors of \(A\text{?}\)
Summary
We defined an eigenvector of a square matrix \(A\) to be a nonzero vector \(\mathbf v\) such that \(A\mathbf v = \lambda\mathbf v\) for some scalar \(\lambda\text{,}\) which is called the eigenvalue associated to \(\mathbf v\text{.}\)
- If \(\mathbf v\) is an eigenvector, then matrix multiplication by \(A\) reduces to the simpler operation of scalar multiplication by \(\lambda\text{.}\)
- Scalar multiples of an eigenvector are also eigenvectors. In fact, we will see that the eigenvectors associated to an eigenvalue \(\lambda\) form a subspace.
- If we can form a basis for \(\mathbb R^n\) consisting of eigenvectors of \(A\text{,}\) then \(A\) is, in some sense, equivalent to a diagonal matrix.
- Rewriting a vector \(\mathbf x\) as a linear combination of eigenvectors of \(A\) makes it easy to repeatedly multiply \(\mathbf x\) by \(A\text{.}\)
Exercises 4.1.4Exercises
Consider the matrix and vectors
-
Show that the vectors \(\mathbf v_1\) and \(\mathbf v_2\) are eigenvectors of \(A\) and find their associated eigenvalues.
- Express the vector \(\mathbf x = \twovec{-4}{-1}\) as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
- Use this expression to compute \(A\mathbf x\text{,}\) \(A^2\mathbf x\text{,}\) and \(A^{-1}\mathbf x\) as a linear combination of eigenvectors.
Consider the matrix and vectors
-
Show that the vectors \(\mathbf v_1\text{,}\) \(\mathbf v_2\text{,}\) and \(\mathbf v_3\) are eigenvectors of \(A\) and find their associated eigenvalues.
- Express the vector \(\mathbf x = \threevec{0}{-3}{-4}\) as a linear combination of the eigenvectors.
- Use this expression to compute \(A\mathbf x\text{,}\) \(A^2\mathbf x\text{,}\) and \(A^{-1}\mathbf x\) as a linear combination of eigenvectors.
Suppose that \(A\) is an \(n\times n\) matrix.
- Explain why \(\lambda = 0\) is an eigenvalue if and only if there is a non-trivial solution to the homogeneous equation \(A\mathbf x = 0\text{.}\)
- Explain why such a matrix \(A\) is not invertible if and only if \(\lambda=0\) is an eigenvalue.
- If \(A\) is an invertible \(n\times n\) matrix having an eigenvector \(\mathbf v\) and associated eigenvalue \(\lambda\text{,}\) explain why \(\mathbf v\) is also an eigenvector of \(A^{-1}\) with associated eigenvalue \(\lambda^{-1}\text{.}\)
- If \(A\) is an \(n\times n\) matrix with eigenvector \(\mathbf v\) and associated eigenvalue \(\lambda\text{,}\) explain why \(\mathbf v\) is also an eigenvector of \(A^2\) with associated eigenvalue \(\lambda^2\text{.}\)
- The matrix \(A=\ \left[\begin{array}{rr} 1 & 2 \\ 2 & 1 \\ \end{array}\right]\) has eigenvectors \(\mathbf v_1=\twovec{1}{1}\) and \(\mathbf v_2=\twovec{-1}{1}\) and associated eigenvalues \(\lambda_1 = 3\) and \(\lambda=-1\text{.}\) What are some eigenvectors and associated eigenvalues for \(A^5\text{?}\)
Suppose that \(A\) is a matrix with eigenvectors \(\mathbf v_1\) and \(\mathbf v_2\) and eigenvalues \(\lambda_1 = -1\) and \(\lambda_2=2\) as shown.
Sketch the vectors \(A\mathbf x\text{,}\) \(A^2\mathbf x\text{,}\) and \(A^{-1}\mathbf x\text{.}\)
For the following matrices, find the eigenvectors and associated eigenvalues by thinking geometrically about the corresponding matrix transformation.
- \(\left[\begin{array}{rr} 3 & 0 \\ 0 & 3 \\ \end{array}\right] \text{.}\)
- \(\left[\begin{array}{rr} -2 & 0 \\ 0 & 4 \\ \end{array}\right] \text{.}\)
- What are the eigenvectors and associated eigenvalues of the identity matrix?
- What are the eigenvectors and associated eigenvalues of a diagonal matrix with distinct diagonal entries?
Suppose that \(A\) is a \(2\times2\) matrix having eigenvectors
and associated eigenvalues \(\lambda_1=2\) and \(\lambda_2=-3\text{.}\) If \(\mathbf x=\twovec{5}{0}\text{,}\) find the vector \(A^4\mathbf x\text{.}\)
Determine whether the following statements are true or false and provide a justification for your response.
- The eigenvalues of a diagonal matrix are equal to the entries on the diagonal.
- If \(A\mathbf v=\lambda\mathbf v\text{,}\) then \(A^2\mathbf v=\lambda\mathbf v\) as well.
- Every vector is an eigenvector of the identity matrix.
- If \(\lambda=0\) is an eigenvalue of \(A\text{,}\) then \(A\) is invertible.
- For every \(n\times n\) matrix \(A\text{,}\) it is possible to find a basis of \(\mathbb R^n\) consisting of eigenvectors of \(A\text{.}\)
Consider the two special matrices below and find their eigenvectors and associated eigenvalues.
- \(\left[\begin{array}{rrr} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \\ \end{array}\right] \text{.}\)
- \(\left[\begin{array}{rrr} 1 & 1 & 1 \\ 2 & 2 & 2 \\ 3 & 3 & 3 \\ \end{array}\right] \text{.}\)
For each of the following matrix transformations, describe the eigenvalues and eigenvectors of the corresponding matrix \(A\text{.}\)
- A reflection in \(\mathbb R^2\) in the line \(y=x\text{.}\)
- A \(180^\circ\) rotation in \(\mathbb R^2\text{.}\)
- A \(180^\circ\) rotation in \(\mathbb R^3\) about the \(y\)-axis.
- A \(90^\circ\) rotation in \(\mathbb R^3\) about the \(x\)-axis.
Suppose we have two species, \(P\) and \(Q\text{,}\) where species \(P\) preys on \(Q\text{.}\) Their populations, in millions, in year \(k\) are denoted by \(P_k\) and \(Q_k\) and satisfy
We will keep track of the populations in year \(k\) using the vector \(\mathbf x_k=\twovec{P_k}{Q_k}\) so that
- Show that \(\mathbf v_1=\twovec{1}{3}\) and \(\mathbf v_2=\twovec{2}{1}\) are eigenvectors of \(A\) and find their associated eigenvalues.
- Suppose that the initial populations are described by the vector \(\mathbf x_0 = \twovec{38}{44}\text{.}\) Express \(\mathbf x_0\) as a linear combination of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
- Find the populations after one year, two years, and three years by writing the vectors \(\mathbf x_1\text{,}\) \(\mathbf x_2\text{,}\) and \(\mathbf x_3\) as linear combinations of \(\mathbf v_1\) and \(\mathbf v_2\text{.}\)
- What is the general form for \(\mathbf x_k\text{?}\)
- After a very long time, what is the ratio of \(P_k\) to \(Q_k\text{?}\)