9.5: Formalizing our approach to recurrence equations
So far, our approach to solving recurrence equations has been based on intuition, and we've not given a lot of explanation for why the solutions we've given have been the general solution. In this section, we endeavor to remedy this. Some familiarity with the language of linear algebra will be useful for the remainder of this section, but it is not essential.
Our techniques for solving recurrence equations have their roots in a fundamentally important concept in mathematics, the notion of a vector space. Recall that a vector space consists of a set \(V\) of elements called vectors ; in addition, there is a binary operation called addition with the sum of vectors \(x\) and \(y\) denoted by \(x+y\); furthermore, there is an operation called scalar multiplication which combines a scalar (real number) \(\alpha\) and a vector \(x\) to form a product denoted .\(\alpha x\). These operations satisfy the following properties:
- \(x+y = y +x\) for every \(x,y, \in V\).
- \(x+(y+z) = (x+y) + z\), for every \(x,y,z \in V\).
- There is a vector called zero and denoted 0 so that \(x+0=x\) for every \(x \in V\). Note: We are again overloading an operator and using the symbol 0 for something other than a number.
- For every element \(x \in V\), there is an element \(y \in V\), called the additive inverse of \(x\) and denoted \(−x\) so that \(x+(−x)=0\). This property enables us to define subtraction , i.e., \(x-y = x+(-y)\).
- \(1x = x\) for every \(x \in X\).
- \(\alpha(\beta x) = (\alpha \beta)x\), for every \(\alpha, \beta \in \mathbb{R}\) and every \(x \in V\).
- \(\alpha(x+y) = \alpha x + \alpha y\) for every \(\alpha \in \mathbb{R}\) and every \(x,y \in V\).
- \((\alpha + \beta)x = \alpha x + \beta x\), for every \(\alpha, \beta \in \mathbb{R}\) and every \(x \in V\).
When \(V\) is a vector space, a function \(\phi:V \rightarrow V\) is called an linear operator , or just operator for short, when \(\phi(x+y)=\phi(x)+ \phi(y)\) and \(\phi(\alpha x)=\alpha \phi(x)\). When \(\phi:V \rightarrow V\) is an operator, it is customary to write \(\phi x\) rather than \(\phi(x)\), saving a set of parentheses. The set of all operators over a vector space \(V\) is itself a vector space with addition defined by \((\phi+\rho)x=\phi x+ \rho x\) and scalar multiplication by \((\alpha \phi)x= \alpha(\phi x)\).
In this chapter, we focus on the real vector space V consisting of all functions of the form \(f: \mathbb{Z} \rightarrow \mathbb{R}\). Addition is defined by \((f+g)(n)=f(n)+g(n)\) and scalar multiplication is defined by \((\alpha f)(n)=\alpha (f(n))\).
9.5.1 The Principal Theorem
Here is the basic theorem about solving recurrence equations (stated in terms of advancement operator equations)—and while we won't prove the full result, we will provide enough of an outline where it shouldn't be too difficult to fill in the missing details.
Let \(k\) be a positive integer \(k\) , and let \(c_0,c_1,…,c_k\) be constants with \(c_0,c_k \neq 0\). Then the set \(W\) of all solutions to the homogeneous linear equation
\[(c_0A^k + c_1A^{k-1} + c_2A^{k-2} + \cdot \cdot \cdot + c_k)f = 0 \label{9.5.1} \]
is a \(k\) -dimensional subspace of \(V\) .
The conclusion that the set \(W\) of all solutions is a subspace of \(V\) is immediate, since
\(p(A)(f+g) = p(A)f + p(A)g\) and \(p(a)(\alpha f) = \alpha p (A)(f)\).
What takes a bit of work is to show that \(W\) is a \(k\)-dimensional subspace. But once this is done, then to solve the advancement operator equation given in the form of Theorem 9.18 , it suffices to find a basis for the vector space \(W\). Every solution is just a linear combination of basis vectors. In the next several subsections, we outline how this goal can be achieved.
9.5.2 The Starting Case
The development proceeds by induction (surprise!) with the case \(k=1\) being the base case. In this case, we study a simple equation of the form \((c_0A+c_1)f=0\). Dividing by \(c_0\) and rewriting using subtraction rather than addition, it is clear that we are just talking about an equation of the form \((A−r)f=0\) where \(r \neq 0\).
Let \(r \neq 0\) , and let \(f\) be a solution to the operator equation \((A−r)f=0\) . If \(c=f(0)\), then \(f(n)=cr^n\) for every \(n \in \mathbb{Z}\) .
- Proof
-
We first show that \(f(n)=cr^n\) for every \(n \geq 0\), by induction on \(n\). The base case is trivial since \(c=f(0)=cr^0\). Now suppose that \(f(k)=cr^k\) for some non-negative integer \(k\). Then \((A−r)f=0\) implies that \(f(k+1)−rf(k)=0\), i.e.,
\(f(k+1)=rf(k)=rcr^k=cr^{k+1}\).
A very similar argument shows that \(f(−n)=cr^{−n}\) for every \(n \leq 0\).
Consider a nonhomogeneous operator equation of the form
\[p(A)f=(c_0A^k+c_1A^{k−1}+c_2A^{k−2}+ \cdot \cdot \cdot +c_k)f=g \label{9.5.2} \]
with \(c_0,c_k \neq 0\) , and let \(W\) be the subspace of \(V\) consisting of all solutions to the corresponding homogeneous equation
\[p(A)f=(c_0A^k+c_1A^{k−1}+c_2A^{k−2}+ \cdot \cdot \cdot +c_k)f=0 \label{9.5.3} \]
If \(f_0\) is a solution to \(\ref{9.5.2}\) , then every solution f to \(\ref{9.5.2}\) has the form \(f=f_0+f_1\) where \(f_1 \in W\) .
- Proof
-
Let \(f\) be a solution of \(\ref{9.5.2}\), and let \(f_1=f−f_0\). Then
\(p(A)f_1=p(A)(f−f_0)=p(A)f−p(A)f_0=g−g=0\).
This implies that \(f_1 \in W\) and that \(f=f_0+f_1\) so that all solutions to \(\ref{9.5.2}\) do in fact have the desired form.
Using the preceding two results, we can now provide an outline of the inductive step in the proof of Theorem 9.18 , at least in the case where the polynomial in the advancement operator has distinct roots.
Consider the following advancement operator equation
\[p(A)f=(A−r_1)(A−r_2)…(A−r_k)f=0 \label{9.5.4} \]
with \(r_1,r_2,…,r_k\) distinct non-zero constants. Then every solution to \(\ref{9.5.4}\) has the form
\(f(n)=c_1r_1^n+c_2r_2^n+c_3r_3^n+ \cdot \cdot \cdot +c_kr_k^n\).
- Proof
-
The case \(k=1\) is Lemma 9.19 . Now suppose we have established the theorem for some positive integer m and consider the case \(k=m+1\). Rewrite (9.5.4) as
\((A−r_1)(A−r_2)…(A−r_m)[(A−r_{m+1})f]=0\).
By the inductive hypothesis, it follows that if \(f\) is a solution to (9.5.4) , then f is also a solution to the nonhomogeneous equation
\[(A−r_{m+1})f=d_1r_1^n+d_2r_2^n+ \cdot \cdot \cdot +d_mr_m^n \label{9.5.5} \]
To find a particular solution \(f_0\) to \(\ref{9.5.5}\), we look for a solution having the form
\[f_0(n)=c_1r_1^n+c_2r_2^n+ \cdot \cdot \cdot +c_mr_m^n \label{9.5.6}\]
On the other hand, a simple calculation shows that for each \(i=1,2,…,m\), we have
\((A−r_{m+1})c_ir_i^n=c_ir_i^{n+1}−r_{m+1}c_ir_i^n=c_i(r_i−r_{m+1})r_i^n\),
so it suffices to choose \(c_i\) so that \(c_i(r_i−r_{m+1})=d_i\), for each \(i=1,2,…,m\). This can be done since \(r_{m+1}\) is distinct from \(r_i\) for \(i=1,2,…m\).
Now we have a particular solution \(f_0(n)= \sum_{i=1}^m c_ir_i^n\). Next we consider the corresponding homogeneous equation \((A−r_{m+1})f=0\). The general solution to this equation has the form \(f_1(n)=c_{m+1}r_{m+1}^n. It follows that every solution to the original equation has the form
\(f(n)=f_0(n)+f_1(n)=c_1r_1^n+c_2r_2^n+ \cdot \cdot \cdot +c_mr_m^n+cr_{m+1}^n\),
which is exactly what we want!
9.5.3 Repeated Roots
It is straightforward to modify the proof given in the preceding section to obtain the following result. We leave the details as an exercise.
Let \(k \geq 1\) and consider the equation
\[(A−r)^kf=0. \label{9.5.7}\]
Then the general solution to \(\lref{9.5.7}\) has the following form
\[f(n)=c_1r^n+c_2nr^n+c_3n^2r^n+c_4n^3r^n+ \cdot \cdot \cdot +c_kn^{k−1}r^n. \label{9.5.8}\]
9.5.4 The General Case
Combining the results in the preceding sections, we can quickly write the general solution of any homogeneous equation of the form \(p(A)f=0\) provided we can factor the polynomial \(p(A)\). Note that in general, this solution takes us into the field of complex numbers , since the roots of a polynomial with real coefficients are sometimes complex numbers—with non-zero imaginary parts.
We close this section with one more example to illustrate how quickly we can read off the general solution of a homogeneous advancement operator equation \(p(A)f=0\), provided that \(p(A)\) is factored.
Consider the advancement operator equation
\((A−1)^5(A+1)^3(A−3)^2(A+8)(A−9)^4f=0\).
Then every solution has the following form
\(f(n) = c_1 + c_2n + c_3n^2 + c_4n^3 + c_5n^4\)
\(+ c_6(-1)^n + c_7n(-1)^n + c_8n^2(-1)^n\)
\(+ c_93^n + c_{10}n3^n\)
\(+c_{11}(-8)^n\)
\(+ c_{12}9^n + c_{13}n9^n + c_{14}n^29^n + c_{15}n^39^n\).