5.1: The Eigenvalue Problem

View The Eigenvalue Problem on Youtube

View Finding Eigenvalue and Eigenvectors (Part 1) on Youtube

View Finding Eigenvalue and Eigenvectors (Part 2) on Youtube

Let A be an \(n\)-by- \(n\) matrix, \(x\) an \(n\)-by- 1 column vector, and \(\lambda\) a scalar. The eigenvalue problem for a given matrix A solves

\[A x=\lambda x \nonumber \]

for \(n\) eigenvalues \(\lambda_{i}\) with corresponding eigenvectors \(\mathrm{x}_{i}\). Since \(\mathrm{Ix}=\mathrm{x}\), where \(\mathrm{I}\) is the \(n\)-by- \(n\) identity matrix, we can rewrite the eigenvalue equation Equation \ref{5.1} as

\[(\mathrm{A}-\lambda \mathrm{I}) \mathrm{x}=0 . \nonumber \]

The trivial solution to this equation is \(x=0\), and for nontrivial solutions to exist, the \(n\)-by- \(n\) matrix A \(-\lambda I\), which is the matrix A with \(\lambda\) subtracted from its diagonal, must be singular. Hence, to determine the nontrivial solutions, we must require that

\[\operatorname{det}(\mathrm{A}-\lambda \mathrm{I})=0 . \nonumber \]

Using the big formula for the determinant, one can see that Equation \ref{5.3} is an \(n\)-th order polynomial equation in \(\lambda\), and is called the characteristic equation of A. The characteristic equation can be solved for the eigenvalues, and for each eigenvalue, a corresponding eigenvector can be determined directly from Equation \ref{5.2}.

We can demonstrate how to find the eigenvalues of a general 2-by-2 matrix given by

\[A=\left(\begin{array}{ll} a & b \\ c & d \end{array}\right) \text {. } \nonumber \]

We have

\[\begin{aligned} 0 &=\operatorname{det}(\mathrm{A}-\lambda \mathrm{I}) \\ &=\left|\begin{array}{cc} a-\lambda & b \\ c & d-\lambda \end{array}\right| \\ &=(a-\lambda)(d-\lambda)-b c \\ &=\lambda^{2}-(a+d) \lambda+(a d-b c), \end{aligned} \nonumber \]

which can be more generally written as

\[\lambda^{2}-\operatorname{Tr} \mathrm{A} \lambda+\operatorname{det} \mathrm{A}=0 \nonumber \]

where \(\operatorname{Tr} \mathrm{A}\) is the trace, or sum of the diagonal elements, of the matrix \(\mathrm{A}\).

Since the characteristic equation of a two-by-two matrix is a quadratic equation, it can have either (i) two distinct real roots; (ii) two distinct complex conjugate roots; or (iii) one degenerate real root. That is, eigenvalues and eigenvectors can be real or complex, and that for certain defective matrices, there may be less than \(n\) distinct eigenvalues and eigenvectors.

If \(\lambda_{1}\) is an eigenvalue of our 2-by-2 matrix \(A\), then the corresponding eigenvector \(\mathrm{x}_{1}\) may be found by solving

\[\left(\begin{array}{cc} a-\lambda_{1} & b \\ c & d-\lambda_{1} \end{array}\right)\left(\begin{array}{l} x_{11} \\ x_{21} \end{array}\right)=0 \nonumber \]

where the equation of the second row will always be a multiple of the equation of the first row because the determinant of the matrix on the left-hand-side is zero. The eigenvector \(\mathrm{x}_{1}\) can be multiplied by any nonzero constant and still be an eigenvector. We could normalize \(x_{1}\), for instance, by taking \(x_{11}=1\) or \(\left|x_{1}\right|=1\), or whatever, depending on our needs.

The equation from the first row of Equation \ref{5.5} is

\[\left(a-\lambda_{1}\right) x_{11}+b x_{21}=0, \nonumber \]

and we could take \(x_{11}=1\) to find \(x_{21}=\left(\lambda_{1}-a\right) / b\). These results are usually derived as needed when given specific matrices.

Example: Find the eigenvalues and eigenvectors of the following matrices:

\[\left(\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right) \text { and }\left(\begin{array}{ll} 3 & 1 \\ 1 & 3 \end{array}\right) \nonumber \]

For

\[\mathrm{A}=\left(\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right) \nonumber \]

the characteristic equation is

\[\lambda^{2}-1=0, \nonumber \]

with solutions \(\lambda_{1}=1\) and \(\lambda_{2}=-1\). The first eigenvector is found by solving \(\left(\mathrm{A}-\lambda_{1} \mathrm{I}\right) \mathrm{x}_{1}=0\), or

\[\left(\begin{array}{rr} -1 & 1 \\ 1 & -1 \end{array}\right)\left(\begin{array}{l} x_{11} \\ x_{21} \end{array}\right)=0, \nonumber \]

so that \(x_{21}=x_{11}\). The second eigenvector is found by solving \(\left(\mathrm{A}-\lambda_{2} \mathrm{I}\right) \mathrm{x}_{2}=0\), or

\[\left(\begin{array}{ll} 1 & 1 \\ 1 & 1 \end{array}\right)\left(\begin{array}{l} x_{12} \\ x_{22} \end{array}\right)=0, \nonumber \]

so that \(x_{22}=-x_{12}\). The eigenvalues and eigenvectors are therefore given by

\[\lambda_{1}=1, \mathrm{x}_{1}=\left(\begin{array}{l} 1 \\ 1 \end{array}\right) ; \quad \lambda_{2}=-1, \mathrm{x}_{2}=\left(\begin{array}{r} 1 \\ -1 \end{array}\right) \nonumber \]

To find the eigenvalues and eigenvectors of the second matrix we can follow this same procedure. Or better yet, we can take a shortcut. Let

\[A=\left(\begin{array}{ll} 0 & 1 \\ 1 & 0 \end{array}\right) \quad \text { and } \quad B=\left(\begin{array}{ll} 3 & 1 \\ 1 & 3 \end{array}\right) \nonumber \]

We know the eigenvalues and eigenvectors of \(\mathrm{A}\) and that \(\mathrm{B}=\mathrm{A}+3 \mathrm{I}\). Therefore, with \(\lambda_{\mathrm{B}}\) representing the eigenvalues of \(\mathrm{B}\), and \(\lambda_{\mathrm{A}}\) representing the eigenvalues of \(A\), we have

\[0=\operatorname{det}\left(\mathrm{B}-\lambda_{\mathrm{B}} \mathrm{I}\right)=\operatorname{det}\left(\mathrm{A}+3 \mathrm{I}-\lambda_{\mathrm{B}} \mathrm{I}\right)=\operatorname{det}\left(\mathrm{A}-\left(\lambda_{\mathrm{B}}-3\right) \mathrm{I}\right)=\operatorname{det}\left(\mathrm{A}-\lambda_{\mathrm{A}} \mathrm{I}\right) \nonumber \]

Therefore, \(\lambda_{\mathrm{B}}=\lambda_{\mathrm{A}}+3\) and the eigenvalues of \(\mathrm{B}\) are 4 and 2. The eigenvectors remain the same.

It is useful to notice that \(\lambda_{1}+\lambda_{2}=\operatorname{Tr} \mathrm{A}\) and that \(\lambda_{1} \lambda_{2}=\operatorname{det} \mathrm{A}\). The analogous result for \(n\)-by- \(n\) matrices is also true and worthwhile to remember. In particular, summing the eigenvalues and comparing to the trace of the matrix provides a rapid check on your algebra.

Example: Find the eigenvalues and eigenvectors of the following matrices:

\[\left(\begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array}\right) \text { and }\left(\begin{array}{ll} 0 & 1 \\ 0 & 0 \end{array}\right) \nonumber \]

For

\[A=\left(\begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array}\right) \nonumber \]

the characteristic equation is

\[\lambda^{2}+1=0 \nonumber \]

with solutions \(i\) and \(-i\). Notice that if the matrix \(A\) is real, then the complex conjugate of the eigenvalue equation \(A x=\lambda x\) is \(A \bar{x}=\bar{\lambda} \bar{x}\). So if \(\lambda\) and \(x\) is an eigenvalue and eigenvector of a real matrix \(A\) then so is the complex conjugates \(\bar{\lambda}\) and \(\bar{x}\). Eigenvalues and eigenvectors of a real matrix appear as complex conjugate pairs.

The eigenvector associated with \(\lambda=i\) is determined from \((\mathrm{A}-i \mathrm{I}) \mathrm{x}=0\), or

\[\left(\begin{array}{rr} -i & -1 \\ 1 & -i \end{array}\right)\left(\begin{array}{l} x_{1} \\ x_{2} \end{array}\right)=0 \nonumber \]

or \(x_{1}=i x_{2}\). The eigenvectors and eigenvectors of \(\mathrm{A}\) are therefore given by

\[\lambda=i, \quad \mathbf{X}=\left(\begin{array}{l} i \\ 1 \end{array}\right) ; \quad \bar{\lambda}=-i, \quad \overline{\mathrm{X}}=\left(\begin{array}{r} -i \\ 1 \end{array}\right) \nonumber \]

For

\[B=\left(\begin{array}{ll} 0 & 1 \\ 0 & 0 \end{array}\right) \nonumber \]

the characteristic equation is

\[\lambda^{2}=0, \nonumber \]

so that there is a degenerate eigenvalue of zero. The eigenvector associated with the zero eigenvalue if found from \(\mathrm{Bx}=0\) and has zero second component. Therefore, this matrix is defective and has only one eigenvalue and eigenvector given by

\[\lambda=0, \quad \mathrm{x}=\left(\begin{array}{l} 1 \\ 0 \end{array}\right) \nonumber \]

Example: Find the eigenvalues and eigenvectors of the rotation matrix:

\[R=\left(\begin{array}{rr} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{array}\right) \nonumber \]

The characteristic equation is given by

\[\lambda^{2}-2 \cos \theta \lambda+1=0, \nonumber \]

with solution

\[\lambda_{\pm}=\cos \theta \pm \sqrt{\cos ^{2} \theta-1}=\cos \theta \pm i \sin \theta=e^{\pm i \theta} \nonumber \]

The eigenvector corresponding to \(\lambda=e^{i \theta}\) is found from

\[-i \sin \theta x_{1}-\sin \theta x_{2}=0, \nonumber \]

or \(x_{2}=-i x_{1}\). Therefore, the eigenvalues and eigenvectors are

\[\lambda=e^{i \theta}, \quad x=\left(\begin{array}{r} 1 \\ -i \end{array}\right) \nonumber \]

and their complex conjugates.