Inner Product Spaces

Inner Product Spaces

Definitions and Examples

We have seen how to take the dot product of any two vectors in Rⁿ.  In this discussion, we will generalize this idea to general vector spaces.  The dot product had certain properties that we will use to define this generalized product.

Definition

Let V be a vector space and u, v, and w be vectors in V and c be a constant.  Then an inner product ( , ) on V is a function with domain consisting of pairs of vectors and range real numbers satisfying the following properties.  

1.  (u, u)  >  0 with equality if and only if u  =  0.

2.  (u, v)  =  (v, u)

3.  (u + v, w)  =  (u, w) + (v, w)

4.  (cu, v)  =  (u, cv)  =  c(u, v)

A vector space with its inner product is called an inner product space.

Notice that the regular dot product satisfies these four properties.

Example

Let V be the vector space consisting of all continuous functions with the standard + and *.  Then define an inner product by

        \( (f, g) = \int_0^1 f(t)g(t) dt \)

For example

        \( (x, x^2) = \int_0^1 (x)(x^2)dx = \frac{1}{4} \)

The four properties follow immediately from the analogous property of the definite integral.  For example

       \( (f + g, h) = \int_0^1 (f+g)(t)h(t) dt \)

        \( = \int_0^1 (f(t)h(t) + g(t)h(t)) dt  = \int_0^1 f(t)h(t) dt + \int_0^1 g(t)h(t) dt \)

       \( = (f, h) = (g, h) \)

Example

Let V  =  R² and consider the function

        ((a,b), (c,d))  =  ac + 2bd

Then this function defines an inner product.  We will prove Property 1.  We have

        ((a,b), (a,b))  =  a² + 2b²  >  0

And equality occurs if and only if both a and b are 0.

Fourier Series

We will spend the rest of the discussion with a special case of an inner product space.  For any inner product space V we call vectors v and w orthogonal if 

        (v, w)  =  0

And we define the length of v by 

     \( length = \sqrt{(v, v)}\)

We will call a basis S for a vector space V orthonormal if every element of S is a unit vector (length one) and any two distinct elements are orthogonal.

The inner product space that we are interested in is the space of continuous functions with 

        \( (f, g) = \int_{-\pi}^\pi f(t) g(t)dt \)

Let S  =  {v₁, v₂, v₃, ...} be the infinite set of vectors (functions) defined by 

        \( v_1 = \frac{1}{\sqrt{2\pi}},  v_2 = \frac{1}{\sqrt{\pi}}cos(t),  v_3 = \frac{1}{\sqrt{\pi}}sin(t),    v_4 = \frac{1}{\sqrt{\pi}}cos(2t),  v_5 = \frac{1}{\sqrt{\pi}}sin(2t) \)

        \( v_6 = \frac{1}{\sqrt{\pi}}cos(3t),  v_7 = \frac{1}{\sqrt{\pi}}sin(3t),  v_8 = \frac{1}{\sqrt{\pi}}cos(4t),    v_9 = \frac{1}{\sqrt{\pi}}sin(4t) ... \)

We leave it as an exercise to show that 

        \( \int_{-\pi}^\pi cos(mt)sin(nt)dt = 0\)

and for m and n distinct positive integers,

        \(\int_{-\pi}^\pi cos(mt)cos(nt)dt =\int_{-\pi}^\pi sin(mt)sin(nt)dt = 0\)

and 

        \( \sqrt{ \frac{1}{\pi} \int_{-\pi}^\pi cos(mt)cos(mt)dt} = \sqrt{ \frac{1}{\pi} \int_{-\pi}^\pi sin(mt)sin(mt)dt} =  1 \)

The above identities imply that S is an orthonormal set of vectors.  To show that it is a basis is beyond the level of this discussion.  In fact, we have yet to understand about infinite dimensional bases. 

Recall that 

        proj_Wv  =  (v,w₁) w₁  + (v,w₂) w₂  + (v,w₃) w₃  + ...

Notice that we have adjusted the definition to use the inner product instead of the dot product and for an infinite orthonormal set instead of a finite set.  This definition provides us with a way of approximating a continuous function with elements of the orthonormal set.  Ths idea is analogous to idea of the Taylor Series.  When the basis is as above, the inner products are called the Fourier coefficients of the function.

Example

Calculate the first four nonzero Fourier coefficients for the function

        f(t)  =  |t|

Solution

We have

       \(\frac{1}{\sqrt{2\pi}} \int_{-\pi}^\pi (1)(|t|) dt \approx 3.94,  \frac{1}{\sqrt{\pi}} \int_{-\pi}^\pi (cos(t))(|t|) dt \approx -2.26, \frac{1}{\sqrt{\pi}} \int_{-\pi}^\pi (sin(t))(|t|) dt = 0\)

It turns out that the next two nonzero Fourier coefficients are

        \(  \frac{1}{\sqrt{\pi}} \int_{-\pi}^\pi (cos(3t))(|t|) dt \approx -0.251, \frac{1}{\sqrt{\pi}} \int_{-\pi}^\pi (cos(5t))(|t|) dt \approx -0.090\)

We can write

        \( |t| \approx 3.94(\frac{1}{\sqrt{2 \pi}})- 2.26(\frac{1}{\sqrt{ \pi}}cos(t)) - 0.251(\frac{1}{\sqrt{ \pi}}cos(3t)) - 0.090(\frac{1}{\sqrt{ \pi}}cos(5t))   \)

        \( = 1.57 - 1.28cos(t) - 0.142cos(3t) - 0.051cos(5t) \)

The graphs of y  =  |t| and the above equation are shown below.

Back to the Vectors Page