11.7: Singular-value decomposition

The singular-value decomposition generalizes the notion of diagonalization. To unitarily diagonalize $T\in \mathcal{L}(V)$ means to find an orthonormal basis $e$ such that $T$ is diagonal with respect to this basis, i.e.,

$$M(T;e,e)= [T]_e = \begin{bmatrix} \lambda_1 && 0\\ &\ddots&\\ 0 && \lambda_n \end{bmatrix},$$

where the notation $M(T;e,e)$ indicates that the basis $e$ is used both for the domain and codomain of $T$. The Spectral Theorem tells us that unitary diagonalization can only be done for normal operators. In general, we can find two orthonormal bases $e$ and $f$ such that

$$M(T;e,f) = \begin{bmatrix} s_1 && 0\\ &\ddots&\\ 0 && s_n \end{bmatrix},$$

which means that $Te_i = s_i f_i$ even if $T$ is not normal. The scalars $s_i$ are called singular values of $T$. If $T$ is diagonalizable, then these are the absolute values of the eigenvalues.

Theorem 11.7.1

All $T\in \mathcal{L}(V)$ have a singular-value decomposition. That is, there exist orthonormal bases $e=(e_1,\ldots,e_n)$ and $f=(f_1,\ldots,f_n)$ such that

$$Tv = s_1 \inner{v}{e_1} f_1 + \cdots + s_n \inner{v}{e_n} f_n,$$

where $s_i$ are the singular values of $T$.

Proof

Since $|T|\ge 0$, it is also also self-adjoint. Thus, by the Spectral Theorem, there is an orthonormal basis $e=(e_1,\ldots,e_n)$ for $V$ such that $|T|e_i = s_i e_i$. Let $U$ be the unitary matrix in the polar decomposition of $T$. Since $e$ is orthonormal, we can write any vector $v\in V$ as

$$\begin{equation*} v = \inner{v}{e_1} e_1 + \cdots + \inner{v}{e_n} e_n, \end{equation*}$$

and hence

$$\begin{equation*} Tv = U|T| v = s_1 \inner{v}{e_1} Ue_1 + \cdots + s_n \inner{v}{e_n} Ue_n. \end{equation*}$$

Now set $f_i = U e_i$ for all $1\le i\le n$. Since $U$ is unitary, $(f_1,\ldots,f_n)$ is also an orthonormal basis, proving the theorem.

$\square$

Contributors

• Isaiah Lankham, Mathematics Department at UC Davis
• Bruno Nachtergaele, Mathematics Department at UC Davis
• Anne Schilling, Mathematics Department at UC Davis

Both hardbound and softbound versions of this textbook are available online at WorldScientific.com.