30.2: Diagonalization

%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import sympy as sym
from urllib.request import urlretrieve

sym.init_printing()
urlretrieve('https://raw.githubusercontent.com/colbrydi/jupytercheck/master/answercheck.py', 
            'answercheck.py');

Reminder

The eigenvalues of triangular (upper and lower) and diagonal matrices are easy:

• The eigenvalues for triangular matrices are the diagonal elements.
• The eigenvalues for the diagonal matrices are the diagonal elements.

Diagonalization

Definition

A square matrix \(A\) is said to be diagonalizable if there exist a matrix \(C\) such that \(D=C^{-1}AC\) is a diagonal matrix.

Definition

\(B\) is a similar matrix of \(A\) if we can find \(C\) such that \(B=C^{-1}AC\).

Given an \(n \times n\) matrix \(A\), can we find another \(n \times n\) invertable matrix \(C\) such that when \(D=C^{-1}AC\) is diagonal, i.e., \(A\) is diagonalizable?

• Because \(C\) is invertible, we have

\[C^{-1}AC=D \nonumber \]

\[CC^{-1}AC = CD \nonumber \]

\[ AC = CD \nonumber \]

• Generate \(C\) as the columns of \(n\) linearly independent vectors \((x_1 \ldots x_n)\) We can compute \(AC=CD\) as follows:

\[ A\begin{bmatrix} \vdots & \vdots & \vdots & \vdots \\ \vdots & \vdots & \vdots & \vdots \\ { x }_{ 1 } & { x }_{ 2 } & \dots & { x }_{ n } \\ \vdots & \vdots & \vdots & \vdots \end{bmatrix}=AC=CD=\begin{bmatrix} \vdots & \vdots & \vdots & \vdots \\ \vdots & \vdots & \vdots & \vdots \\ { x }_{ 1 } & { x }_{ 2 } & \dots & { x }_{ n } \\ \vdots & \vdots & \vdots & \vdots \end{bmatrix}\begin{bmatrix} { \lambda }_{ 1 } & 0 & 0 & 0 \\ 0 & { \lambda }_{ 2 } & 0 & 0 \\ \vdots & \vdots & { \dots } & \vdots \\ 0 & 0 & 0 & { \lambda }_{ n } \end{bmatrix} \nonumber \]

• Then we check the corresponding columns of the both sides. We have

\[Ax_1 = \lambda_1x_1 \\ \vdots \\ Ax_n=\lambda x_n \nonumber \]

• \(A\) has \(n\) linear independent eigenvectors.
• \(A\) is saied to be similar to the diagonal matrix \(D\), and the transformation of \(A\) into \(D\) is called a similarity transformation.

A simple example

Consider the following:

\[ A = \begin{bmatrix}7& -10\\3& -4\end{bmatrix},\quad C = \begin{bmatrix}2& 5\\1& 3\end{bmatrix} \nonumber \]

Do This

Find the similar matrix \(D=C^{-1}AC\) of \(A\).

#Put your answer to the above question here.

from answercheck import checkanswer

checkanswer.matrix(D, '8313fe0f529090d6a8cdb36248cfdd6c');

Do This

Find the eigenvalues and eigenvectors of \(A\). Set variables e1 and vec1 to be the smallest eigenvalue and it’s associated eigenvector and e2, vec2 to represent the largest.

#Put your answer to the above question here.

from answercheck import checkanswer
checkanswer.float(e1, "e4c2e8edac362acab7123654b9e73432");

from answercheck import checkanswer
checkanswer.float(e2, "d1bd83a33f1a841ab7fda32449746cc4");

from answercheck import checkanswer
checkanswer.eq_vector(vec1, "d28f0a721eedb3d5a4c714744883932e", decimal_accuracy = 4)

from answercheck import checkanswer
checkanswer.eq_vector(vec2, "09d9df5806bc8ef975074779da1f1023", decimal_accuracy = 4)

Theorem

Similar matrices have the same eigenvalues.

Proof

Assume \(B=C^{-1}AC\) is a similar matrix of \(A\), and \(\lambda\) is an eigenvalue of \(A\) with corresponding eigenvector \(x\). That is, \(Ax=\lambda x\). Then we have \(B(C^{-1}x) = C^{-1}AC(C^{-1}x) = C^{-1}Ax = C^{-1}(\lambda x) = \lambda (C^{-1}x)\). That is \(C^{-1}x\) is an eigenvector of \(B\) with eigenvalue \(\lambda\).

A second example

Do This

Consider \( A = \begin{bmatrix}-4& -6\\3& 5\end{bmatrix} \).

Find a matrix \(C\) such that \(C^{-1}AC\) is diagonal.

Hint: use the function diagonalize in sympy.

#Put your answer to the above question here. 

#Check the output type
assert(type(C)==sym.Matrix)

from answercheck import checkanswer
checkanswer.matrix(C,'ba963b7fef354b4a7ddd880ca4bac071')

The third example

Do This

Consider \( A = \begin{bmatrix}5& -3\\3& -1\end{bmatrix} \).

Can we find a matrix \(C\) such that \(C^{-1}AC\) is diagonal.

Hint: find eigenvalues and eigenvectors using sympy.

#Put your answer to the above question here. 

Dimensions of eigenspaces and diagonalization

Definition

The set of all eigenvectors of a \(n \times n\) matrix corresponding to a eigenvalue \(\lambda\), together with the zero vector, is a subspace of \(R^n\). This subspace spaces is called eigenspace.

• For the third example, we have that the characteristic equation \((\lambda - 2)^2 = 0\).
• Eigenvalue \(\lambda = 2\) has multiplicity 2, but the eigenspace has dimension 1, since we can not find two lineare independent eigenvector for \(\lambda = 2\).

The dimension of an eigenspace of a matrix is less than or equal to the multiplicity of the corresponding eigenvalue as a root of the characteristic equation.

A matrix is diagonalizable if and only if the dimension of every eigenspace is equal to the multiplicity of the corresponding eigenvalue as a root of the characteristic equation.

The fourth example

Do This

Consider \( A = \begin{bmatrix}2& -1\\1& 2\end{bmatrix} \).

Can we find a matrix \(C\) such that \(C^{-1}AC\) is diagonal.

#Put your answer to the above question here.