5.7: Multiple Eigenvalues

 

It may very well happen that a matrix has some “repeated” eigenvalues. That is, the characteristic equation \(\det(A-\lambda I)=0\) may have repeated roots. As we have said before, this is actually unlikely to happen for a random matrix. If we take a small perturbation of \(A\) (we change the entries of \(A\) slightly), then we will get a matrix with distinct eigenvalues. As any system we will want to solve in practice is an approximation to reality anyway, it is not indispensable to know how to solve these corner cases. On the other hand, these cases do come up in applications from time to time. Furthermore, if we have distinct but very close eigenvalues, the behavior is similar to that of repeated eigenvalues, and so understanding that case will give us insight into what is going on.

Geometric Multiplicity

Take the diagonal matrix

\[ A = \begin{bmatrix}3&0\\0&3 \end{bmatrix} \nonumber \]

\(A\) has an eigenvalue \(3\) of multiplicity \(2\). We call the multiplicity of the eigenvalue in the characteristic equation the algebraic multiplicity. In this case, there also exist \(2\) linearly independent eigenvectors, \(\begin{bmatrix}1\\0 \end{bmatrix}\) and \(\begin{bmatrix} 0\\1 \end{bmatrix}\) corresponding to the eigenvalue \(3\). This means that the so-called geometric multiplicity of this eigenvalue is also \(2\).

In all the theorems where we required a matrix to have \(n\) distinct eigenvalues, we only really needed to have \(n\) linearly independent eigenvectors. For example, \(\vec{x} = A \vec{x} \) has the general solution

\[\vec{x} = c_1 \begin{bmatrix} 1\\0 \end{bmatrix} e^{3t} + c_2 \begin{bmatrix} 0\\1 \end{bmatrix} e^{3t}. \nonumber \]

The geometric multiplicity of an eigenvalue of algebraic multiplicity n is equal to the number of corresponding linearly independent eigenvectors. The geometric multiplicity is always less than or equal to the algebraic multiplicity. We have handled the case when these two multiplicities are equal. If the geometric multiplicity is equal to the algebraic multiplicity, then we say the eigenvalue is complete.

In other words, the hypothesis of the theorem could be stated as saying that if all the eigenvalues of \(P\) are complete, then there are \(n\) linearly independent eigenvectors and thus we have the given general solution.

If the geometric multiplicity of an eigenvalue is \(2\) or greater, then the set of linearly independent eigenvectors is not unique up to multiples as it was before. For example, for the diagonal matrix \(A = \begin{bmatrix} 3&0 \\ 0&3 \end{bmatrix} \) we could also pick eigenvectors \(\begin{bmatrix} 1\\1 \end{bmatrix} \) and \( \begin{bmatrix} 1\\-1 \end{bmatrix} \), or in fact any pair of two linearly independent vectors. The number of linearly independent eigenvectors corresponding to \(\lambda\) is the number of free variables we obtain when solving \(A\vec{v} = \lambda \vec{v} \). We pick specific values for those free variables to obtain eigenvectors. If you pick different values, you may get different eigenvectors.

Defective Eigenvalues

If an \(n \times n\) matrix has less than n linearly independent eigenvectors, it is said to be deficient. Then there is at least one eigenvalue with an algebraic multiplicity that is higher than its geometric multiplicity. We call this eigenvalue defective and the difference between the two multiplicities we call the defect.

The next theorem shows what to do in this situation.

Theorem 5.7.1

Suppose the \(n\times n\) matrix \(A\) has an eigenvalue \(\lambda_1\) of multiplicity \(\ge2\) and the associated eigenspace has dimension \(1;\) that is\(,\) all \(\lambda_1\)-eigenvectors of \(A\) are scalar multiples of an eigenvector \({\bf x}.\) Then there are infinitely many vectors \({\bf u}\) such that

\[\label{eq:10.5.2} (A-\lambda_1I){\bf u}={\bf x}.\]

Moreover\(,\) if \({\bf u}\) is any such vector then

\[\label{eq:10.5.3} {\bf y}_1={\bf x}e^{\lambda_1t}\quad\mbox{and }\quad {\bf y}_2={\bf u}e^{\lambda_1t}+{\bf x}te^{\lambda_1t}\]

are linearly independent solutions of \({\bf y}'=A{\bf y}.\)

 

Example 5.7.2

Use Theorem 5.7.1 to find the general solution of the system

\[\label{eq:10.5.6} {\bf y}'=\left[\begin{array}{cc}{11}&{-25}\\{4}&{-9}\end{array} \right]{\bf y}\]

considered in Example 5.7.1 .

Solution

In Example 5.7.1 we saw that \(\lambda_1=1\) is an eigenvalue of multiplicity \(2\) of the coefficient matrix \(A\) in Equation \ref{eq:10.5.6}, and that all of the eigenvectors of \(A\) are multiples of

\[{\bf x}=\begin{bmatrix} 5 \\ 2  \end{bmatrix}.\nonumber\]

Therefore

\[{\bf y}_1=\begin{bmatrix} 5 \\ 2  \end{bmatrix}e^t\nonumber\]

is a solution of Equation \ref{eq:10.5.6}. From Theorem 5.7.1 , a second solution is given by \({\bf y}_2={\bf u}e^t+{\bf x}te^t\), where \((A-I){\bf u}={\bf x}\). The augmented matrix of this system is

\[\left[\begin{array}{rrcr}10&-25&\vdots&5\\4&-10&\vdots&2\end{array}\right],\nonumber\]

which is row equivalent to

\[\left[\begin{array}{rrcr}1&-\frac{5}{2}&\vdots&\frac{1}{2}\\0&0&\vdots&0\end{array}\right],\nonumber\]

Therefore the components of \({\bf u}\) must satisfy

\[u_1-{5\over2}u_2={1\over2},\nonumber\]

where \(u_2\) is arbitrary. We choose \(u_2=0\), so that \(u_1=1/2\) and

\[{\bf u}=\begin{bmatrix} 1\over2 \\ 0  \end{bmatrix}.\nonumber\]

Thus,

\[{\bf y}_2=\begin{bmatrix} 1 \\ 0  \end{bmatrix}{e^t\over2}+\begin{bmatrix} 5 \\ 2  \end{bmatrix}te^t.\nonumber\]

Since \({\bf y}_1\) and \({\bf y}_2\) are linearly independent by Theorem 5.7.1 , they form a fundamental set of solutions. Therefore the general solution is

\[{\bf y}=c_1\begin{bmatrix} 5 \\ 2  \end{bmatrix}e^t+c_2\left(\begin{bmatrix} 1 \\ 0  \end{bmatrix}{e^t\over2}+\begin{bmatrix} 5 \\ 2  \end{bmatrix}te^t\right).\nonumber\]

Note that choosing the arbitrary constant \(u_2\) to be nonzero is equivalent to adding a scalar multiple of \({\bf y}_1\) to the second solution \({\bf y}_2\) (Exercise 10.5.33).

Example \(\PageIndex{2}\)

Find the general solution of

\[\label{eq:10.5.7} {\bf y}'=\left[\begin{array}{ccc}{3}&{4}&{-10}\\{2}&{1}&{-2}\\{2}&{2}&{-5}\end{array} \right] {\bf y}.\]

Solution

The characteristic polynomial of the coefficient matrix \(A\) in Equation \ref{eq:10.5.7} is

\[\left|\begin{array}{ccc} 3-\lambda & 4 & -10\\ 2 & 1-\lambda & -2\\ 2 & 2 &-5-\lambda\end{array}\right| =- (\lambda-1)(\lambda+1)^2.\nonumber\]

Hence, the eigenvalues are \(\lambda_1=1\) with multiplicity \(1\) and \(\lambda_2=-1\) with multiplicity \(2\). Eigenvectors associated with \(\lambda_1=1\) must satisfy \((A-I){\bf x}={\bf 0}\). The augmented matrix of this system is

\[\left[\begin{array}{rrrcr} 2 & 4 & -10 &\vdots & 0\\ 2& 0 & -2 &\vdots & 0\\ 2 & 2 & -6 & \vdots & 0\end{array}\right],\nonumber\]

which is row equivalent to

\[\left[\begin{array}{rrrcr} 1 & 0 & -1 &\vdots& 0\\ 0 & 1 & -2 &\vdots& 0\\ 0 & 0 & 0 &\vdots&0\end{array}\right].\nonumber\]

Hence, \(x_1 =x_3\) and \(x_2 =2 x_3\), where \(x_3\) is arbitrary. Choosing \(x_3=1\) yields the eigenvector

\[{\bf x}_1=\begin{bmatrix} 1 \\ 2 \\ 1  \end{bmatrix}  .\nonumber\]

Therefore

\[{\bf y}_1 =\begin{bmatrix} 1 \\ 2 \\ 1  \end{bmatrix} e^t\nonumber\]

is a solution of Equation \ref{eq:10.5.7}. Eigenvectors associated with \(\lambda_2 =-1\) satisfy \((A+I){\bf x}={\bf 0}\). The augmented matrix of this system is

\[\left[\begin{array}{rrrcr} 4 & 4 & -10 &\vdots & 0\\ 2 & 2 & -2 & \vdots & 0\\2 & 2 & -4 &\vdots & 0\end{array}\right],\nonumber\]

which is row equivalent to

\[\left[\begin{array}{rrrcr} 1 & 1 & 0 &\vdots& 0\\ 0 & 0 & 1 &\vdots& 0 \\ 0 & 0 & 0 &\vdots&0\end{array}\right].\nonumber\]

Hence, \(x_3=0\) and \(x_1 =-x_2\), where \(x_2\) is arbitrary. Choosing \(x_2=1\) yields the eigenvector

\[{\bf x}_2=\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} ,\nonumber\]

so

\[{\bf y}_2 =\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} e^{-t}\nonumber\]

is a solution of Equation \ref{eq:10.5.7}. Since all the eigenvectors of \(A\) associated with \(\lambda_2=-1\) are multiples of \({\bf x}_2\), we must now use Theorem 5.7.1 to find a third solution of Equation \ref{eq:10.5.7} in the form

\[\label{eq:10.5.8} {\bf y}_3={\bf u}e^{-t}+\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} te^{-t},\]

where \({\bf u}\) is a solution of \((A+I){\bf u=x}_2\). The augmented matrix of this system is

\[\left[\begin{array}{rrrcr} 4 & 4 & -10 &\vdots & -1\\ 2 & 2 & -2 & \vdots & 1\\ 2 & 2 & -4 &\vdots & 0\end{array}\right],\nonumber\]

which is row equivalent to

\[\left[\begin{array}{rrrcr} 1 & 1 & 0 &\vdots& 1\\ 0 & 0 & 1 &\vdots& {1\over2} \\ 0 & 0 & 0 &\vdots&0\end{array}\right].\nonumber\]

Hence, \(u_3=1/2\) and \(u_1 =1-u_2\), where \(u_2\) is arbitrary. Choosing \(u_2=0\) yields

\[{\bf u} =\begin{bmatrix} 1 \\ 0 \\ 1\over2  \end{bmatrix} ,\nonumber\]

and substituting this into Equation \ref{eq:10.5.8} yields the solution

\[{\bf y}_3=\begin{bmatrix} 2 \\ 0 \\ 1  \end{bmatrix} {e^{-t}\over2}+\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} te^{-t}\nonumber\]

of Equation \ref{eq:10.5.7}. Since the Wronskian of \(\{{\bf y}_1,{\bf y}_2,{\bf y}_3\}\) at \(t=0\) is

\[\left|\begin{array}{rrr} 1&-1&1\\2&1&0\\1&0&1\over2\end{array}\right|={1\over2},\nonumber\]

\(\{{\bf y}_1,{\bf y}_2,{\bf y}_3\}\) is a fundamental set of solutions of Equation \ref{eq:10.5.7}. Therefore the general solution of Equation \ref{eq:10.5.7} is

\[{\bf y}=c_1\begin{bmatrix} 1 \\ 2 \\ 1  \end{bmatrix} e^t+c_2\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} e^{-t}+c_3\left (\begin{bmatrix} 2 \\ 0 \\ 1  \end{bmatrix} {e^{-t}\over2}+\begin{bmatrix} -1 \\ 1 \\ 0  \end{bmatrix} e^{-t}\right).\nonumber\]

Below is a video on solving a defective system of differential equations.

Note that the system \( \vec{x}' = A \vec{x} \) has a simpler solution since \(A\) is a so-called upper triangular matrix, that is every entry below the diagonal is zero. In particular, the equation for \( x_2 \) does not depend on \(x_1\). Mind you, not every defective matrix is triangular.

Exercise 3.7.1

Solve \( \vec{x}' = \begin{bmatrix} 3&1\\ 0&3 \end{bmatrix} \vec{x} \) by first solving for \(x_2\) and then for \( x_1 \) independently. Check that you got the same solution as we did above.

Let us describe the general algorithm. Suppose that \(\lambda \) is an eigenvalue of multiplicity \(2\), defect \(1\). First find an eigenvector \(\vec{v_1} \) of \( \lambda\). Then, find a vector \( \vec{v_2} \) such that

\[ (A - \lambda I) \vec{v_2} = \vec{v_1} \nonumber \]

This gives us two linearly independent solutions

\[\begin{align}\begin{aligned} \vec{x_1} &= \vec{v_1} e^{\lambda t} \\ \vec{x_2} &= (\vec{v_2} + \vec{v_1} t )e^{\lambda t }\end{aligned}\end{align} \nonumber \]

Example \(\PageIndex{2}\)

Consider the system \[\vec{x}' = \begin{bmatrix} 2 & -5 & 0 \\ 0 & 2 & 0 \\ -1 & 4 & 1 \end{bmatrix} \vec{x} . \nonumber \] Compute the eigenvalues,

Solution

\[0 = \det(A-\lambda I) = \det\left( \begin{bmatrix} 2-\lambda & -5 & 0 \\ 0 & 2-\lambda & 0 \\ -1 & 4 & 1-\lambda \end{bmatrix} \right) = (2-\lambda)^2(1-\lambda) . \nonumber \] The eigenvalues are 1 and 2, where 2 has multiplicity 2. We leave it to the reader to find that \(\left[ \begin{smallmatrix} 0 \\ 0 \\ 1 \end{smallmatrix} \right]\) is an eigenvector for the eigenvalue \(\lambda = 1\).

Let’s focus on \(\lambda = 2\). We compute eigenvectors: \[\vec{0} = (A - 2 I) \vec{v} = \begin{bmatrix} 0 & -5 & 0 \\ 0 & 0 & 0 \\ -1 & 4 & -1 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix} . \nonumber \] The first equation says that \(v_2 = 0\), so the last equation is \(-v_1 -v_3 = 0\). Let \(v_3\) be the free variable to find that \(v_1 = -v_3\). Perhaps let \(v_3 = -1\) to find an eigenvector \(\left[ \begin{smallmatrix} 1 \\ 0 \\ -1 \end{smallmatrix} \right]\). Problem is that setting \(v_3\) to anything else just gets multiples of this vector and so we have a defect of 1. Let \(\vec{v}_1\) be the eigenvector and let’s look for a generalized eigenvector \(\vec{v}_2\): \[(A - 2 I) \vec{v}_2 = \vec{v}_1 , \nonumber \] or \[\begin{bmatrix} 0 & -5 & 0 \\ 0 & 0 & 0 \\ -1 & 4 & -1 \end{bmatrix} \begin{bmatrix} a \\ b \\ c \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} , \nonumber \] where we used \(a\), \(b\), \(c\) as components of \(\vec{v}_2\) for simplicity. The first equation says \(-5b = 1\) so \(b = \frac{-1}{5}\). The second equation says nothing. The last equation is \(-a + 4b - c = -1\), or \(a + \frac{4}{5} + c = 1\), or \(a + c = \frac{1}{5}\). We let \(c\) be the free variable and we choose \(c=0\). We find \(\vec{v}_2 = \left[ \begin{smallmatrix} \frac{1}{5} \\ \frac{-1}{5} \\ 0 \end{smallmatrix} \right]\).

The general solution is therefore, \[\vec{x} = c_1 \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} e^t + c_2 \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} e^{2t} + c_3 \left( \begin{bmatrix} \frac{1}{5} \\ \frac{-1}{5} \\ 0 \end{bmatrix} + \begin{bmatrix} 1 \\ 0 \\ -1 \end{bmatrix} t \right) e^{2t} . \nonumber \]

This machinery can also be generalized to higher multiplicities and higher defects. We will not go over this method in detail, but let us just sketch the ideas. Suppose that \(A\) has an eigenvalue \( \lambda \) of multiplicity \(m\). We find vectors such that

\[ (A - \lambda I ) ^k \vec(v) = \vec(0),\quad\text{but}\quad (A - \lambda I )^{k-1} \vec{v} \neq \vec{0} \nonumber \]

Such vectors are called generalized eigenvectors (then \(\vec{v_{1}}=(A-\lambda I)^{k-1}\vec{v}\) is an eigenvector). For every eigenvector \(\vec{v}_{1} \) we find a chain of generalized eigenvectors \( \vec{v}_{2} \) through \(vec{v}_{k} \) such that:

\[\begin{align}\begin{aligned} (A - \lambda I) \vec{v_1} &= \vec{0}, \\ (A - \lambda I)\vec{v_2} &= \vec{v_1}, \\ &\vdots \\ (A - \lambda I )\vec{v_k} &= \vec{v_{k-1}}. \end{aligned}\end{align} \nonumber \]

Really once you find the \(\vec{v}_k\) such that \({(A - \lambda I)}^k \vec{v}_k = \vec{0}\) but \({(A - \lambda I)}^{k-1} \vec{v}_k \not= \vec{0}\), you find the entire chain since you can compute the rest, \(\vec{v}_{k-1} = (A - \lambda I) \vec{v}_k\), \(\vec{v}_{k-2} = (A - \lambda I) \vec{v}_{k-1}\), etc. We form the linearly independent solutions

\[\begin{align}\begin{aligned} \vec{x_1} &= \vec{v_1} e^{\lambda t} \\ \vec{x_2} &= ( \vec{v_2} + \vec{v_1} t) e^{\lambda t } \\ & \vdots \\ \vec{x_k} &= \left( \vec{v_k}+ \vec{v}_{k-1} t + \vec{v}_{k-2} \frac{t^2}{2} + \cdots + \vec{v}_{2} \frac{t^{k-2}}{(k-2)!} + \vec{v}_{1}\frac{t^{k-1}}{(k-1)!} ) e^{\lambda t} \right) \end{aligned}\end{align} \nonumber \]

Recall that \(k! = 1 \cdot 2 \cdot 3 \cdots (k-1) \cdot k\) is the factorial. If you have an eigenvalue of geometric multiplicity \(\ell\), you will have to find \(\ell\) such chains (some of them might be short: just the single eigenvector equation). We go until we form \(m\) linearly independent solutions where \(m\) is the algebraic multiplicity. We don’t quite know which specific eigenvectors go with which chain, so start by finding \(\vec{v}_k\) first for the longest possible chain and go from there.

For example, if \(\lambda\) is an eigenvalue of \(A\) of algebraic multiplicity \(3\) and defect \(2\), then solve \[(A - \lambda I) \vec{v}_1 = \vec{0} , \qquad (A - \lambda I) \vec{v}_2 = \vec{v}_1 , \qquad (A - \lambda I) \vec{v}_3 = \vec{v}_2 . \nonumber \] That is, find \(\vec{v}_3\) such that \({(A - \lambda I)}^3 \vec{v}_3 = \vec{0}\), but \({(A - \lambda I)}^2 \vec{v}_3 \not= \vec{0}\). Then you are done as \(\vec{v}_2 = (A - \lambda I) \vec{v}_3\) and \(\vec{v}_1 = (A - \lambda I) \vec{v}_2\). The 3 linearly independent solutions are \[\vec{x}_1 = \vec{v}_1 e^{\lambda t} , \qquad \vec{x}_2 = ( \vec{v}_2 + \vec{v}_1 t ) \, e^{\lambda t} , \qquad \vec{x}_3 = \left( \vec{v}_3 + \vec{v}_2 t + \vec{v}_{1} \frac{t^2}{2} \right) \, e^{\lambda t} . \nonumber \]

If on the other hand \(A\) has an eigenvalue \(\lambda\) of algebraic multiplicity \(3\) and defect \(1\), then solve \[(A - \lambda I) \vec{v}_1 = \vec{0} , \qquad (A - \lambda I) \vec{v}_2 = \vec{0} , \qquad (A - \lambda I) \vec{v}_3 = \vec{v}_2 . \nonumber \] Here \(\vec{v}_1\) and \(\vec{v}_2\) are actual honest eigenvectors, and \(\vec{v}_3\) is a generalized eigenvector. So there are two chains. To solve, first find a \(\vec{v}_3\) such that \({(A - \lambda I)}^2 \vec{v}_3 = \vec{0}\), but \((A - \lambda I) \vec{v}_3 \not= \vec{0}\). Then \(\vec{v}_2 = (A - \lambda I) \vec{v}_3\) is going to be an eigenvector. Then solve for an eigenvector \(\vec{v}_1\) that is linearly independent from \(\vec{v}_2\). You get 3 linearly independent solutions \[\vec{x}_1 = \vec{v}_1 e^{\lambda t} , \qquad \vec{x}_2 = \vec{v}_2 e^{\lambda t} , \qquad \vec{x}_3 = ( \vec{v}_3 + \vec{v}_2 t ) \, e^{\lambda t} . \nonumber \]