$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}{\| #1 \|}$$ $$\newcommand{\inner}{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$

# 11.7: Singular-value decomposition

$$\newcommand{\vecs}{\overset { \rightharpoonup} {\mathbf{#1}} }$$ $$\newcommand{\vecd}{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}}$$$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}{\| #1 \|}$$ $$\newcommand{\inner}{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}{\| #1 \|}$$ $$\newcommand{\inner}{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$

The singular-value decomposition generalizes the notion of diagonalization. To unitarily diagonalize $$T\in \mathcal{L}(V)$$ means to find an orthonormal basis $$e$$ such that $$T$$ is diagonal with respect to this basis, i.e.,

$M(T;e,e)= [T]_e = \begin{bmatrix} \lambda_1 && 0\\ &\ddots&\\ 0 && \lambda_n \end{bmatrix},$

where the notation $$M(T;e,e)$$ indicates that the basis $$e$$ is used both for the domain and codomain of $$T$$. The Spectral Theorem tells us that unitary diagonalization can only be done for normal operators. In general, we can find two orthonormal bases $$e$$ and $$f$$ such that

$M(T;e,f) = \begin{bmatrix} s_1 && 0\\ &\ddots&\\ 0 && s_n \end{bmatrix},$

which means that $$Te_i = s_i f_i$$ even if $$T$$ is not normal. The scalars $$s_i$$ are called singular values of $$T$$. If $$T$$ is diagonalizable, then these are the absolute values of the eigenvalues.

Theorem 11.7.1

All $$T\in \mathcal{L}(V)$$ have a singular-value decomposition. That is, there exist orthonormal bases $$e=(e_1,\ldots,e_n)$$ and $$f=(f_1,\ldots,f_n)$$ such that

$Tv = s_1 \inner{v}{e_1} f_1 + \cdots + s_n \inner{v}{e_n} f_n,$

where $$s_i$$ are the singular values of $$T$$.

Proof

Since $$|T|\ge 0$$, it is also also self-adjoint. Thus, by the Spectral Theorem, there is an orthonormal basis $$e=(e_1,\ldots,e_n)$$ for $$V$$ such that $$|T|e_i = s_i e_i$$. Let $$U$$ be the unitary matrix in the polar decomposition of $$T$$. Since $$e$$ is orthonormal, we can write any vector $$v\in V$$ as

\begin{equation*}
v = \inner{v}{e_1} e_1 + \cdots + \inner{v}{e_n} e_n,
\end{equation*}

and hence

\begin{equation*}
Tv = U|T| v = s_1 \inner{v}{e_1} Ue_1 + \cdots + s_n \inner{v}{e_n} Ue_n.
\end{equation*}

Now set $$f_i = U e_i$$ for all $$1\le i\le n$$. Since $$U$$ is unitary, $$(f_1,\ldots,f_n)$$ is also an orthonormal basis, proving the theorem.

$$\square$$

## Contributors

Both hardbound and softbound versions of this textbook are available online at WorldScientific.com.