3.4: Inner Product Spaces

We have discussed the inner product (or dot product) between two column matrices. Recall that the inner product between, say, two three-by-one column matrices

\[\text{u}=\left(\begin{array}{c}u_1\\u_2\\u_3\end{array}\right),\quad\text{v}=\left(\begin{array}{c}v_1\\v_2\\v_3\end{array}\right)\nonumber \]

is given by

\[\text{u}^{\text{T}}\text{v}=u_1v_1+u_2v_2+u_3v_3.\nonumber \]

We now generalize the inner product so that it is applicable to any vector space, including those containing functions.

We will denote the inner product between any two vectors \(\text{u}\) and \(\text{v}\) as \((\text{u}, \text{v})\), and require the inner product to satisfy the same arithmetic rules that are satisfied by the dot product. With \(\text{u}\), \(\text{v}\), \(\text{w}\) vectors and \(c\) a scalar, these rules can be written as

\[(\text{u},\text{v})=(\text{v},\text{u}),\quad (\text{u}+\text{v},\text{w})=(\text{u},\text{w})+(\text{v},\text{w}),\quad (\text{cu},\text{v})=\text{c}(\text{u},\text{v})=(\text{u},\text{cv});\nonumber \]

and \((\text{u},\text{u})\geq 0\), where the equality holds if and only if \(\text{u}=0\).

Generalizing our definitions for column matrices, the norm of a vector \(\text{u}\) is defined as

\[||\text{u}||=(\text{u},\text{u})^{1/2}.\nonumber \]

A unit vector is a vector whose norm is one. Unit vectors are said to be normalized to unity, though sometimes we just say that they are normalized. We say two vectors are orthogonal if their inner product is zero. We also say that a basis is orthonormal (as in an orthonormal basis) if all the vectors are mutually orthogonal and are normalized to unity. For an orthonormal basis consisting of the vectors \(v_1,\: v_2,\cdots , v_n\), we write

\[(v_i,v_j)=\delta_{ij},\nonumber \]

where \(\delta_{ij}\) is called the Kronecker delta, defined as

\[\delta_{ij}=\left\{\begin{array}{ll}1,&\text{if }i=j; \\ 0,&\text{if }i\neq j.\end{array}\right.\nonumber \]

Oftentimes, basis vectors are used that are orthogonal but are normalized to other values besides unity.

Example \(\PageIndex{1}\)

Define an inner product for \(\mathbb{P}_n\).

Solution

Let \(p(x)\) and \(q(x)\) be two polynomials in \(\mathbb{P}_n\). One possible definition of an inner product is given by

\[(p,q)=\int_{-1}^1 p(x)q(x)dx.\nonumber \]

You can check that all the conditions of an inner product are satisfied.

Example \(\PageIndex{2}\)

Show that the first four Legendre polynomials form an orthogonal basis for \(\mathbb{P}_3\) using the inner product defined above.

Solution

The first four Legendre polynomials are given by

\[P_0(x)=1,\quad P_1(x)=x,\quad P_2(x)=\frac{1}{2}(3x^2-1),\quad P_3(x)=\frac{1}{2}(5x^3-3x),\nonumber \]

and these four polynomials form a basis for \(\mathbb{P}_3\). With an inner product defined on \(\mathbb{P}_n\) as

\[(p,q)=\int_{-1}^1p(x)q(x)dx,\nonumber \]

it can be shown by explicit integration that

\[(P_m,P_n)=\frac{2}{2n+1}\delta_{m,n},\nonumber \]

so that the first four Legendre polynomials are mutually orthogonal. They are normalized so that \(P_n(1) = 1\).

Example \(\PageIndex{3}\)

Define an inner product on \(\mathbb{P}_n\) such that the Hermite polynomials are orthogonal.

Solution

For instance, the first four Hermite polynomials are given by

\[H_0(x)=1,\quad H_1(x)=2x,\quad H_2(x)=4x^2-2,\quad H_3(x)=8x^3-12x,\nonumber \]

which also form a basis for \(\mathbb{P}_3\). Here, define an inner product on \(\mathbb{P}_n\) as

\[(p,q)=\int_{-\infty}^\infty p(x)q(x)e^{-x^2}dx.\nonumber \]

It can be shown that

\[(H_m,H_n)=2^n\pi^{1/2}n!\delta_{m,n},\nonumber \]

so that the Hermite polynomials are orthogonal with this definition of the inner product. These Hermite polynomials are normalized so that the leading coefficient of \(H_n\) is given by \(2^n\).