Skip to main content
\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)
Mathematics LibreTexts

14.3: Relating Orthonormal Bases

  • Page ID
    2087
  • [ "article:topic", "authortag:waldron", "orthogonal", "authorname:waldron", "Change of Basis" ]

    \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \)

    \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)

    Suppose \(T=\{u_{1}, \ldots, u_{n} \}\) and \(R=\{w_{1}, \ldots, w_{n} \}\) are two orthonormal bases for \(\Re^{n}\). Then:

    \begin{eqnarray*}
    w_{1} &=& (w_{1}\cdot u_{1}) u_{1} + \cdots + (w_{1}\cdot u_{n})u_{n}\\
     & \vdots & \\
    w_{n} &=& (w_{n}\cdot u_{1}) u_{1} + \cdots + (w_{n}\cdot u_{n})u_{n}\\
    \Rightarrow w_{i} &=& \sum_{j} u_{j}(u_{j}\cdot w_{i}) \\
    \end{eqnarray*}

    Thus the matrix for the change of basis from \(T\) to \(R\) is given by 

    \[P = (P^{j}_{i}) = (u_{j}\cdot w_{i}).\]

    We would like to calculate the product \(PP^{T}\). For that, we first develop a dirty trick for products of dot products:

    $$(u.v)(w.z)=(u^{T} v) (w^{T} z) = u^{T} (v w^{T}) z\, . $$

    The object \(v w^{T}\) is the square matrix made from the outer product of \(v\) and \(w\)! Now we are ready to compute the components of the matrix product \(PP^{T}\):

    \begin{eqnarray*}
    \sum_{i}(u_{j}\cdot w_{i})(w_{i}\cdot u_{k})&=&
     \sum_{i}(u_{j}^{T} w_{i}) (w_{i}^{T} u_{k})\\
    &=& u_{j}^{T} \left[\sum_{i} (w_{i} w_{i}^{T}) \right] u_{k} \\
    &\stackrel{(*)}=& u_{j}^{T} I_{n} u_{k} \\\
    &=& u_{j}^{T} u_{k} = \delta_{jk}.
    \end{eqnarray*}

    The equality \((*)\) is explained below. Assuming \((*)\) holds, we have shown that \(PP^{T}=I_{n}\), which implies that 

    \[P^{T}=P^{-1}.\]

    The equality in the line \((*)\) says that \(\sum_{i} w_{i} w_{i}^{T}=I_{n}\). To see this, we examine \(\left(\sum_{i} w_{i} w_{i}^{T}\right)v\) for an arbitrary vector \(v\). We can find constants \(c^{j}\) such that \(v=\sum_{j} c^{j}w_{j}\), so that:

    \begin{eqnarray*}
    \left(\sum_{i} w_{i} w_{i}^{T}\right)v &=& \left(\sum_{i} w_{i} w_{i}^{T}\right)\left(\sum_{j} c^{j}w_{j}\right) \\
    &=& \sum_{j} c^{j} \sum_{i} w_{i} w_{i}^{T} w_{j} \\
    &=& \sum_{j} c^{j} \sum_{i} w_{i} \delta_{ij} \\
    &=& \sum_{j} c^{j} w_{j} \textit{ since all terms with \(i\neq j\) vanish}\\
    &=&v.
    \end{eqnarray*}

    Thus, as a linear transformation, \(\sum_{i} w_{i} w_{i}^{T}=I_{n}\) fixes every vector, and thus must be the identity \(I_{n}\).

    Definition: Orthonality

    A matrix \(P\) is orthogonal if \(P^{-1}=P^{T}\).

    Then to summarize,

    Theorem: Orthonormality

    A change of basis matrix \(P\) relating two orthonormal bases is an orthogonal matrix. \(\textit{i.e.}\), \(P^{-1}=P^T.\)

    Example 123

    Consider \(\Re^{3}\) with the orthonormal basis 

    \[
    S=\left\{ 
    u_{1}=\begin{pmatrix}\frac{2}{\sqrt{6}}\\ \frac{1}{\sqrt{6}}\\ \frac{-1}{\sqrt{6}}\end{pmatrix},
    u_{2}=\begin{pmatrix}0\\ \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}\end{pmatrix},
    u_{3}=\begin{pmatrix}\frac{1}{\sqrt{3}}\\ \frac{-1}{\sqrt{3}}\\ \frac{1}{\sqrt{3}}\end{pmatrix}
    \right\}.
    \]

    Let \(E\) be the standard basis \(\{e_{1},e_{2},e_{3} \}\). Since we are changing from the standard basis to a new basis, then the columns of the change of basis matrix are exactly the standard basis vectors. Then the change of basis matrix from \(E\) to \(S\) is given by:

    \begin{eqnarray*}
    P=(P^{j}_{i})=(e_{j}u_{i})&=&
    \begin{pmatrix}
    e_{1}\cdot u_{1} & e_{1}\cdot u_{2} & e_{1}\cdot u_{3} \\
    e_{2}\cdot u_{1} & e_{2}\cdot u_{2} & e_{2}\cdot u_{3} \\
    e_{3}\cdot u_{1} & e_{3}\cdot u_{2} & e_{3}\cdot u_{3} \\
    \end{pmatrix} \\
    = \begin{pmatrix}
    u_{1} & u_{2} & u_{3}
    \end{pmatrix} 
    &=&
    \begin{pmatrix}
    \frac{2}{\sqrt{6}} & 0 & \frac{1}{\sqrt{3}} \\
    \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{2}}&\frac{-1}{\sqrt{3}}\\
    \frac{-1}{\sqrt{6}}& \frac{1}{\sqrt{2}}&\frac{1}{\sqrt{3}}\\
    \end{pmatrix}. \\
    \end{eqnarray*}

    From our theorem, we observe that:

    \begin{eqnarray*}
    P^{-1}=P^{T} &=& \begin{pmatrix}u_{1}^{T}\\u_{2}^{T}\\u_{3}^{T}\end{pmatrix} \\
    &=& \begin{pmatrix}
    \frac{2}{\sqrt{6}}& \frac{1}{\sqrt{6}}& \frac{-1}{\sqrt{6}}\\
    0 & \frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}} \\
    \frac{1}{\sqrt{3}}& \frac{-1}{\sqrt{3}}&\frac{1}{\sqrt{3}}\\
    \end{pmatrix}. \\
    \end{eqnarray*}

    We can check that \(P^{T}P=I\) by a lengthy computation, or more simply, notice that

    \begin{eqnarray*}
    (P^{T}P)_{ij}
    &=& \begin{pmatrix}u_{1}^{T}\\u_{2}^{T}\\u_{3}^{T}\end{pmatrix} \begin{pmatrix}u_{1} & u_{2}& u_{3}\end{pmatrix} \\
    &=& \begin{pmatrix}1&0&0\\0&1&0\\0&0&1\end{pmatrix}.
    \end{eqnarray*}

    Above we are using orthonormality of the \(u_{i}\) and the fact that matrix multiplication amounts to taking dot products between rows and columns. It is also very important to realize that the columns of an \(\textit{orthogonal}\) matrix are made from an \(\textit{orthonormal}\) set of vectors.

    Remark: (Orthonormal Change of Basis and Diagonal Matrices)

    Suppose \(D\) is a diagonal matrix and we are able to use an orthogonal matrix \(P\) to change to a new basis. Then the matrix \(M\) of \(D\) in the new basis is:

    \[
    M = PDP^{-1} = PDP^{T}.
    \]

    Now we calculate the transpose of \(M\).

    \begin{eqnarray*}
    M^{T} &=& (PDP^{T})^{T}\\
    &=& (P^{T})^{T}D^{T}P^{T} \\
    &=& PDP^{T}\\
    &=& M
    \end{eqnarray*}

    The matrix \(M=PDP^{T}\) is symmetric!

    Contributor