7.1: Matrices, Determinants and the Eigenvalue Problem

View a lecture on matrix addition and multiplication on YouTube

View a lecture on determinants on YouTube

We begin by reviewing some basic matrix algebra. A matrix with \(n\) rows and \(m\) columns is called an \(n\)-by-\(m\) matrix. Here, we need only consider the simple case of two-by-two matrices.

A two-by-two matrix A, with two rows and two columns, can be written as \[A=\left(\begin{array}{cc}a&b \\ c&d\end{array}\right).\nonumber\]

The first row has elements \(a\) and \(b\), the second row has elements \(c\) and \(d\). The first column has elements \(a\) and \(c\); the second column has elements \(b\) and \(d\).

Matrices can be added and multiplied. Matrices can be added if they have the same dimension, and addition proceeds element by element, following \[\left(\begin{array}{cc}a&b \\ c&d\end{array}\right)+\left(\begin{array}{cc}e&f\\g&h\end{array}\right)=\left(\begin{array}{cc}a+e & b+f \\ c+g & d+h\end{array}\right).\nonumber\]

Matrices can be multiplied if the number of columns of the left matrix equals the number of rows of the right matrix. A particular element in the resulting product matrix, say in row \(k\) and column \(l\), is obtained by multiplying and summing the elements in row \(k\) of the left matrix with the elements in column \(l\) of the right matrix. For example, a two-by-two matrix can multiply a two-by-one column vector as follows \[\left(\begin{array}{rl}a&b \\ c&d\end{array}\right)\left(\begin{array}{c}x\\y\end{array}\right)=\left(\begin{array}{c}ax+by \\ cx+dy\end{array}\right).\nonumber\]

The first row of the left matrix is multiplied against and summed with the first (and only) column of the right matrix to obtain the element in the first row and first column of the product matrix, and so on for the element in the second row and first column. The product of two two-by-two matrices is given by \[\left(\begin{array}{cc}a&b \\ c&d\end{array}\right)\left(\begin{array}{cc}e&f \\ g&h\end{array}\right)=\left(\begin{array}{cc}ae+bg &af+bh \\ ce+dg&cf+dh\end{array}\right).\nonumber\]

A system of linear algebraic equations is easily represented in matrix form. For instance, with \(a_{ij}\) and \(b_i\) given numbers, and \(x_i\) unknowns, the (two-by-two) system of equations given by \[\begin{aligned}a_{11}x_1+a_{12}x_2&=b_1, \\ a_{21}x_1+a_{22}x_2&=b_2,\end{aligned}\] can be written in matrix form as \[\label{eq:1}\mathbf{Ax}=\mathbf{b},\] where \[\label{eq:2}\mathbf{A}=\left(\begin{array}{cc}a_{11}&a_{12} \\ a_{21}&a_{22}\end{array}\right),\quad\mathbf{x}=\left(\begin{array}{c}x_1 \\ x_2\end{array}\right),\quad\mathbf{b}=\left(\begin{array}{c}b_1\\b_2\end{array}\right).\]

When \(\mathbf{b} = 0\), we say that the system of equations given by \(\eqref{eq:1}\) is homogeneous. We now ask the following question: When does there exist a nontrivial (not identically zero) solution for \(\mathbf{x}\) when the linear system is homogeneous? For the simplest case of a two-by-two matrix, we can try and solve directly the homogeneous linear system of equations given by \[\label{eq:3}\begin{array}{l}ax_1+bx_2=0, \\ cx_1+dx_2=0.\end{array}\]

Multiplying the first equation by \(d\) and the second by \(b\), and subtracting the second equation from the first, results in \[(ad-bc)x_1=0.\nonumber\]

Similarly, multiplying the first equation by \(c\) and the second by \(a\), and subtracting the first equation from the second, results in \[(ad-bc)x_2=0.\nonumber\]

Therefore, a nontrivial solution of \(\eqref{eq:3}\) for a two-by-two matrix exists only if \(ad − bc = 0\). If we define the determinant of the \(2\times 2\) matrix \[\label{eq:4}\mathbf{A}=\left(\begin{array}{cc}a&b \\ c&d\end{array}\right)\] to be \(\det \mathbf{A} = ad − bc\), then we say that a nontrivial solution to \(\eqref{eq:3}\) exists provided \(\det \mathbf{A} = 0\).

The same calculation may be repeated for a \(3\times 3\) matrix. If \[\mathbf{A}=\left(\begin{array}{ccc}a&b&c \\ d&e&f \\ g&h&i\end{array}\right),\quad\mathbf{x}=\left(\begin{array}{c}x_1 \\ x_2 \\ x_3\end{array}\right),\nonumber\] then there exists a nontrivial solution to \(\mathbf{Ax} = 0\) provided \(\det \mathbf{A} = 0\), where \(\det \mathbf{A} = a(ei − f h) − b(di − f g) + c(dh − eg)\). The definition of the determinant can be further generalized to any \(n\times n\) matrix, and is typically taught in a first course on linear algebra.

We now consider the eigenvalue problem. For \(\mathbf{A}\) an \(n\times n\) matrix and \(\mathbf{v}\) an \(n\times 1\) column vector, the eigenvalue problem solves the equation \[\label{eq:5}\mathbf{Av}=\lambda\mathbf{v}\] for eigenvalues \(\lambda_i\) and corresponding eigenvectors \(\mathbf{v}_i\). We rewrite the eigenvalue equation \(\eqref{eq:5}\) as \[\label{eq:6}(\mathbf{A}-\lambda\mathbf{I})\mathbf{v}=0,\] where \(\mathbf{I}\) is the \(n\times n\) identity matrix, that is, the matrix with ones on the diagonal and zeros everywhere else. A nontrivial solution of \(\eqref{eq:6}\) exists provided \[\label{eq:7}\det (\mathbf{A}-\lambda\mathbf{I})=0.\]

Equation \(\eqref{eq:7}\) is an \(n\)-th order polynomial equation in \(\lambda\), and is called the characteristic equation of \(\mathbf{A}\). The characteristic equation can be solved for the eigenvalues, and for each eigenvalue, a corresponding eigenvector can be determined directly from \(\eqref{eq:5}\).

We can demonstrate how to find the eigenvalues and eigenvectors of the \(2\times 2\) matrix given by \(\eqref{eq:4}\). We have \[\begin{aligned}0&=\det (\mathbf{A}-\lambda\mathbf{I}) \\ &=\left|\begin{array}{cc}a-\lambda &b \\ c&d-\lambda\end{array}\right| \\ &=(a-\lambda)(d-\lambda)-bc \\ &=\lambda^2-(a+d)\lambda +(ad-bc).\end{aligned}\]

This characteristic equation can be more generally written as \[\label{eq:8}\lambda^2-\text{Tr }\mathbf{A}\lambda +\det\mathbf{A}=0,\] where \(\text{Tr }\mathbf{A}\) is the trace, or sum of the diagonal elements, of the matrix \(\mathbf{A}\). If \(\lambda\) is an eigenvalue of \(\mathbf{A}\), then the corresponding eigenvector \(\mathbf{v}\) may be found by solving \[\left(\begin{array}{cc}a-\lambda&b \\ c&d-\lambda\end{array}\right)\left(\begin{array}{c}v_1 \\ v_2\end{array}\right)=0,\nonumber\] where the equation of the second row will always be a multiple of the equation of the first row. The eigenvector \(\mathbf{v}\) has arbitrary normalization, and we may always choose for convenience \(v_1 = 1\). The equation from the first row is \[(a-\lambda)v_1+bv_2=0,\nonumber\] and with \(v_1=1\), we find \(v_2=(\lambda -a)/b\).

In the next section, we will see several examples of an eigenvector analysis.