Skip to main content
\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)
Mathematics LibreTexts

11.7: Singular-value decomposition

  1. Last updated
  2. Save as PDF
  • Page ID
    311

    • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)

    The singular-value decomposition generalizes the notion of diagonalization. To unitarily diagonalize \(T\in \mathcal{L}(V)\) means to find an orthonormal basis \(e\) such that \(T\) is diagonal with respect to this basis, i.e.,

    \[ M(T;e,e)= [T]_e = \begin{bmatrix} \lambda_1 && 0\\ &\ddots&\\ 0 && \lambda_n \end{bmatrix}, \]

    where the notation \(M(T;e,e)\) indicates that the basis \(e\) is used both for the domain and codomain of \(T\). The Spectral Theorem tells us that unitary diagonalization can only be done for normal operators. In general, we can find two orthonormal bases \(e\) and \(f\) such that

    \[ M(T;e,f) = \begin{bmatrix} s_1 && 0\\ &\ddots&\\ 0 && s_n \end{bmatrix}, \]

    which means that \(Te_i = s_i f_i\) even if \(T\) is not normal. The scalars \(s_i\) are called singular values of \(T\). If \(T\) is diagonalizable, then these are the absolute values of the eigenvalues.

    Theorem 11.7.1

    All \(T\in \mathcal{L}(V)\) have a singular-value decomposition. That is, there exist orthonormal bases \(e=(e_1,\ldots,e_n)\) and \(f=(f_1,\ldots,f_n)\) such that

    \[ Tv = s_1 \inner{v}{e_1} f_1 + \cdots + s_n \inner{v}{e_n} f_n, \]

    where \(s_i\) are the singular values of \(T\).

    Proof

    Since \(|T|\ge 0\), it is also also self-adjoint. Thus, by the Spectral Theorem, there is an orthonormal basis \(e=(e_1,\ldots,e_n)\) for \(V\) such that \(|T|e_i = s_i e_i\). Let \(U\) be the unitary matrix in the polar decomposition of \(T\). Since \(e\) is orthonormal, we can write any vector \(v\in V\) as

    \begin{equation*}
    v = \inner{v}{e_1} e_1 + \cdots + \inner{v}{e_n} e_n,
    \end{equation*}

    and hence

    \begin{equation*}
    Tv = U|T| v = s_1 \inner{v}{e_1} Ue_1 + \cdots + s_n \inner{v}{e_n} Ue_n.
    \end{equation*}

    Now set \(f_i = U e_i\) for all \(1\le i\le n\). Since \(U\) is unitary, \((f_1,\ldots,f_n)\) is also an orthonormal basis, proving the theorem.

    \(\square\)