19.1: Eigenvectors and Eigenvalues
- Page ID
- 67886
Understanding Eigenvector and Eigenvalues can be very challenging. These are complex topics with many facets. Different textbooks approach the problem from different directions. All have value. These facets include:
- Understanding the mathematical definition of Eigenvalues.
- Being able to calculate an Eigenvalue and Eigenvector.
- Understanding what Eigenvalues and Eigenvectors represent.
- Understanding how to use Eigenvalues and Eigenvectors to solve problems.
In this course we consider it more important to understand what eigenvectors and eigenvalues represent and how to use them. However, often this understanding comes from first learning how to calculate them.
Eigenvalues are a special set of scalars associated with a square matrix that are sometimes also known as characteristic roots, characteristic values (Hoffman and Kunze 1971), proper values, or latent roots (Marcus and Minc 1988, p. 144).
The determination of the eigenvalues and eigenvectors of a matrix is extremely important in physics and engineering, where it is equivalent to matrix diagonalization and arises in such common applications as stability analysis, the physics of rotating bodies, and small oscillations of vibrating systems, to name only a few.
The decomposition of a square matrix \(A\) into eigenvalues and eigenvectors is known in this work as eigen decomposition, and the fact that this decomposition is always possible as long as the matrix consisting of the eigenvectors of \(A\) is square. This is known as the eigen decomposition theorem.
From: http://mathworld.wolfram.com/Eigenvalue.html
The following video provides an intuition for eigenvalues and eigenvectors.
Definition
Let \(A\) be an \(n \times n\) matrix. Find a vector \(x\) in \(R^n\) such that:
\[Ax=\lambda x \nonumber \]
The above can be rewritten as the following homogeneous equation:
\[(A-\lambda I_n)x = 0 \nonumber \]
The trivial solution is \(x=0\).
However, if we define eigenvectors to be nonzero vectors then \(|A - \lambda I_n| = 0\). Nonzero (i.e. non-trivial) solutions to this system of equations can only exist if the matrix of coefficients is singular, i.e. the determinant of \(|A - \lambda I_n| = 0 \). Therefore, solving the equation \(|A - \lambda I_n| = 0 \) for \(\lambda\) leads to all the eigenvalues of \(A\).
The above logic is key. Make sure you understand. If not, ask questions.
Explain why nonzero solutions to a system of homogeneous systems require the matrix to be singular.
Examples:
Here are a few more examples of how eigenvalues and eigenvectors are used (You are not required to understand all):
Using singular value decomposition for image compression. This is a note explaining how you can compress an image by throwing away the small eigenvalues of \(A^{T}A\). It takes an 88 megapixel image of an Allosaurus and shows how the image looks after compressing by selecting the largest singular values.
Deriving Special Relativity is more natural in the language of linear algebra. In fact, Einstein’s second postulate really states that “Light is an eigenvector of the Lorentz transform.” This document goes over the full derivation in detail.
Spectral Clustering. Whether it’s in plants and biology, medical imaging, buisness and marketing, understanding the connections between fields on Facebook, or even criminology, clustering is an extremely important part of modern data analysis. It allows people to find important subsystems or patterns inside noisy data sets. One such method is spectral clustering, which uses the eigenvalues of the graph of a network. Even the eigenvector of the second smallest eigenvalue of the Laplacian matrix allows us to find the two largest clusters in a network.
Dimensionality Reduction/PCA. The principal components correspond to the largest eigenvalues of \(A^{T}A\), and this yields the least squared projection onto a smaller dimensional hyperplane, and the eigenvectors become the axes of the hyperplane. Dimensionality reduction is extremely useful in machine learning and data analysis as it allows one to understand where most of the variation in the data comes from.
Low rank factorization for collaborative prediction. This is what Netflix does (or once did) to predict what rating you’ll have for a movie you have not yet watched. It uses the singular value decomposition and throws away the smallest eigenvalues of \(A^{T}A\).
The Google Page Rank algorithm. The largest eigenvector of the graph of the internet is how the pages are ranked.
From: https://math.stackexchange.com/questions/1520832/real-life-examples-for-eigenvalues-eigenvectors