A Theorem about Differential Equations
Differential equations are instrumental in solving a variety of problems throughout science, social science, and engineering. In this brief section, we will see that the set of solutions of a linear differential equation (with constant coefficients) is a vector space and we will calculate its dimension. The proof is pure linear algebra, although the applications are primarily in analysis. However, a key result (Lemma [lem:022938] below) can be applied much more widely.
We denote the derivative of a function \(f : \mathbb{R} \to \mathbb{R}\) by \(f^\prime\), and \(f\) will be called differentiable if it can be differentiated any number of times. If \(f\) is a differentiable function, the \(n\)th derivative \(f^{(n)}\) of \(f\) is the result of differentiating \(n\) times. Thus \(f^{(0)} = f, f^{(1)} = f^\prime, f^{(2)} = f^{(1)\prime}, \dots\), and in general \(f^{(n+1)} = f^{(n)\prime}\) for each \(n \geq 0\). For small values of \(n\) these are often written as \(f, f^\prime, f^\prime \prime, f^{\prime \prime\prime}, \dots\).
If \(a\), \(b\), and \(c\) are numbers, the differential equations
\[f^\prime \prime - af^\prime - bf = 0 \quad \mbox{or} \quad f^{\prime \prime\prime} - af^\prime \prime - bf^\prime - cf = 0 \nonumber \]
are said to be of second order and third-order, respectively. In general, an equation
\[ f^{(n)} - a_{n-1}f^{(n-1)} - a_{n-2}f^{(n-2)} - \cdots - a_{2}f^{(2)} - a_{1}f^{(1)} - a_{0}f^{(0)} = 0, a_i \mbox{ in } \mathbb{R} \]
is called a differential equation of order \(n\). We want to describe all solutions of this equation. Of course a knowledge of calculus is required.
The set \(\mathbf{F}\) of all functions \(\mathbb{R} \to \mathbb{R}\) is a vector space with operations as described in Example [exa:017760]. If \(f\) and \(g\) are differentiable, we have \((f + g)^\prime = f^\prime + g^\prime\) and \((af)^\prime = af^\prime\) for all \(a\) in \(\mathbb{R}\). With this it is a routine matter to verify that the following set is a subspace of \(\mathbf{F}\):
\[\mathbf{D}_n = f : \mathbb{R} \to \; \mathbb{R} | f \text{ is differentiable and is a solution to equation } \PageIndex{1} \nonumber \]
Our sole objective in this section is to prove
Theorem \(\PageIndex{1}\)
The space \(\mathbf{D}_{n}\) has dimension \(n\).
We have already used this theorem in Section 3.5
As will be clear later, the proof of Theorem \(\PageIndex{1}\) requires that we enlarge \(\mathbf{D}_{n}\) somewhat and allow our differentiable functions to take values in the set \(\mathbb{C}\) of complex numbers. To do this, we must clarify what it means for a function \(f : \mathbb{R} \to \mathbb{C}\) to be differentiable. For each real number \(x\) write \(f(x)\) in terms of its real and imaginary parts \(f_{r}(x)\) and \(f_{i}(x)\):
\[f(x) = f_r(x) + if_i(x) \nonumber \]
This produces new functions \(f_{r} : \mathbb{R} \to \mathbb{R}\) and \(f_{i} : \mathbb{R} \to \mathbb{R}\), called the real and imaginary parts of \(f\), respectively. We say that \(f\) is differentiable if both \(f_{r}\) and \(f_{i}\) are differentiable (as real functions), and we define the derivative \(f^\prime\) of \(f\) by
\[ f^\prime = f_{r}^\prime + if_{i}^\prime \nonumber \]
We refer to this frequently in what follows.
Note
Write \(|w|\) for the absolute value of any complex number \(w\). As for functions \(\mathbb{R} \rightarrow \mathbb{R}\), we say that \(\lim _{t \rightarrow 0} f(t)=w\) if, for all \(\varepsilon>0\) there exists \(\delta>0\) such that \(|f(t)-w|<\epsilon\) whenever \(|t|<\delta\). (Note that \(t\) represents a real number here.) In particular, given a real number \(x\), we define the derivative \(f^{\prime}\) of a function \(f: \mathbb{R} \rightarrow \mathbb{C}\) by \(f^{\prime}(x)=\lim _{t \rightarrow 0}\left\{\frac{1}{t}[f(x+t)-f(x)]\right\}\) and we say that \(f\) is differentiable if \(f^{\prime}(x)\) exists for all \(x\) in \(\mathbb{R}\). Then we can prove that \(f\) is differentiable if and only if both \(f_{\mathrm{r}}\) and \(f_{\mathrm{i}}\) are differentiable, and that \(f^{\prime}=f_{\mathrm{r}}{ }^{\prime}+i f_{\mathrm{i}}{ }^{\prime}\) in this case.
With this, write \(\mathbf{D}_{\infty}\) for the set of all differentiable complex valued functions \(f : \mathbb{R} \to \mathbb{C}\). This is a complex vector space using pointwise addition (see Example [exa:017760]), and the following scalar multiplication: For any \(w\) in \(\mathbb{C}\) and \(f\) in \(\mathbf{D}_{\infty}\), we define \(wf : \mathbb{R} \to \mathbb{C}\) by \((wf)(x) = wf(x)\) for all \(x\) in \(\mathbb{R}\). We will be working in \(\mathbf{D}_{\infty}\) for the rest of this section. In particular, consider the following complex subspace of \(\mathbf{D}_{\infty}\):
\[\mathbf{D}_{n}^{*} = \{f : \mathbb{R} \to \mathbb{C} \mid f \mbox{ is a solution to (\ref{eq:diffOrderN})}\} \nonumber \]
Clearly, \(\mathbf{D}_n \subseteq \mathbf{D}_{n}^{*}\), and our interest in \(\mathbf{D}_{n}^{*}\) comes from
Lemma \(\PageIndex{1}\)
If \(dim _\mathbb{C}(\mathbf{D}_{n}^{*}) = n\), then \(dim _\mathbb{R}(\mathbf{D}_{n}) = n\).
Proof. Observe first that if \(dim \;_\mathbb{C}(\mathbf{D}_{n}^{*}) = n\), then \(dim \;_\mathbb{R}(\mathbf{D}_{n}^{*}) = 2n\). [In fact, if \(\{g_{1}, \dots, g_{n}\}\) is a \(\mathbb{C}\)-basis of \(\mathbf{D}_{n}^{*}\) then \(\{g_1, \dots, g_n, ig_1, \dots, ig_n\}\) is a \(\mathbb{R}\)-basis of \(\mathbf{D}_{n}^{*}\)]. Now observe that the set \(\mathbf{D}_{n} \times \mathbf{D}_{n}\) of all ordered pairs \((f, g)\) with \(f\) and \(g\) in \(\mathbf{D}_{n}\) is a real vector space with componentwise operations. Define
\[\theta : \mathbf{D}_{n}^{*} \to \mathbf{D}_{n} \times \mathbf{D}_{n} \quad \mbox{given by} \quad \theta(f) = (f_r, f_i) \mbox{ for } f \mbox{ in } \mathbf{D}_{n}^{*} \nonumber \]
One verifies that \(\theta\) is onto and one-to-one, and it is \(\mathbb{R}\)-linear because \(f \to f_{r}\) and \(f \to f_{i}\) are both \(\mathbb{R}\)-linear. Hence \(\mathbf{D}_{n}^{*} \cong \mathbf{D}_n \times \mathbf{D}_n\) as \(\mathbb{R}\)-spaces. Since \(dim \;_\mathbb{R}(\mathbf{D}_{n}^{*})\) is finite, it follows that \(dim \;_{\mathbb{R}}(\mathbf{D}_{n})\) is finite, and we have
\[2 dim _\mathbb{R}(\mathbf{D}_n) = dim _\mathbb{R}(\mathbf{D}_n \times \mathbf{D}_n) = dim _\mathbb{R}(\mathbf{D}_{n}^{*}) = 2n \nonumber \]
Hence \(dim _{\mathbb{R}}(\mathbf{D}_{n}) = n\), as required.
It follows that to prove Theorem \(\PageIndex{1}\) it suffices to show that \(dim _{\mathbb{C}}(\mathbf{D}_{n}^{*}) = n\).
There is one function that arises frequently in any discussion of differential equations. Given a complex number \(w = a + ib\) (where \(a\) and \(b\) are real), we have \(e^{w} = e^{a}(\cos b + i\sin b)\). The law of exponents, \(e^{w}e^{v} = e^{w+v}\) for all \(w\), \(v\) in \(\mathbb{C}\) is easily verified using the formulas for \(\sin(b + b_{1})\) and \(\cos(b + b_{1})\). If \(x\) is a variable and \(w = a + ib\) is a complex number, define the exponential function \(e^{wx}\) by
\[e^{wx} = e^{ax}(\cos bx + i\sin bx) \nonumber \]
Hence \(e^{wx}\) is differentiable because its real and imaginary parts are differentiable for all \(x\). Moreover, the following can be proved using equation \(\PageIndex{1}\):
\[(e^{wx})^\prime = we^{wx} \nonumber \]
In addition, Equation \(\PageIndex{1}\) gives the product rule for differentiation:
\[\mbox{If } f \mbox{ and } g \mbox{ are in } \mathbf{D}_\infty, \mbox{ then } (fg)^\prime = f^\prime g + fg^\prime \nonumber \]
We omit the verifications.
To prove that \(dim \;_{\mathbb{C}}(\mathbf{D}_{n}^{*}) = n\), two preliminary results are required. Here is the first.
Lemma \(\PageIndex{2}\)
Given \(f\) in \(\mathbf{D}_{\infty}\) and \(w\) in \(\mathbb{C}\), there exists \(g\) in \(\mathbf{D}_{\infty}\) such that \(g^\prime - wg = f\).
Proof. Define \(p(x) = f(x)e^{-wx}\). Then \(p\) is differentiable, whence \(p_{r}\) and \(p_{i}\) are both differentiable, hence continuous, and so both have antiderivatives, say \(p_{r} = q_{r}^\prime\) and \(p_{i} = q_{i}^\prime\). Then the function \(q = q_{r} + iq_{i}\) is in \(\mathbf{D}_{\infty}\), and \(q^\prime = p\) by (equation \(\PageIndex{1}\)). Finally define \(g(x) = q(x)e^{wx}\). Then
\[g^\prime = q^{\prime}e^{wx} + qwe^{wx} = pe^{wx} + w(qe^{wx}) = f + wg \nonumber \]
by the product rule, as required.
The second preliminary result is important in its own right.
Lemma \(\PageIndex{3}\) Kernel Lemma
Let \(V\) be a vector space, and let \(S\) and \(T\) be linear operators \(V \to V\). If \(S\) is onto and both \(\text{ker}(S)\) and \(\text{ker}(T)\) are finite dimensional, then \(\text{ker}(TS)\) is also finite dimensional and \(dim \;[\text{ker}(TS)] = dim \;[\text{ker}(T)] + dim \;[\text{ker}(S)]\).
Proof. Let \(\{\mathbf{u}_{1}, \mathbf{u}_{2}, \dots, \mathbf{u}_{m}\}\) be a basis of \(\text{ker}(T)\) and let \(\{\mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{n}\}\) be a basis of \(\text{ker}(S)\). Since \(S\) is onto, let \(\mathbf{u}_{i} = S(\mathbf{w}_{i})\) for some \(\mathbf{w}_{i}\) in \(V\). It suffices to show that
\[B = \{\mathbf{w}_1, \mathbf{w}_2, \dots, \mathbf{w}_m, \mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_n\} \nonumber \]
is a basis of \(\text{ker}(TS)\). Note \(B \subseteq \text{ker}(TS)\) because \(TS(\mathbf{w}_{i}) = T(\mathbf{u}_{i}) = \mathbf{0}\) for each \(i\) and \(TS(\mathbf{v}_{j}) = T(\mathbf{0}) = \mathbf{0}\) for each \(j\).
Spanning. If \(\mathbf{v}\) is in \(\text{ker}(TS)\), then \(S(\mathbf{v})\) is in \(\text{ker}(T)\), say \(S(\mathbf{v}) = \sum r_i\mathbf{u}_i = \sum r_iS\left(\mathbf{w}_i\right) = S\left(\sum r_i \mathbf{w}_i\right)\). It follows that \(\mathbf{v} - \sum r_i\mathbf{w}_i\) is in \(\text{ker}(S) = span \;\{\mathbf{v}_{1}, \mathbf{v}_{2}, \dots, \mathbf{v}_{n}\}\), proving that \(\mathbf{v}\) is in \(span \;(B)\).
Independence. Let \(\sum r_i\mathbf{w}_i + \sum t_j\mathbf{v}_j = \mathbf{0}\). Applying \(S\), and noting that \(S(\mathbf{v}_{j}) = \mathbf{0}\) for each \(j\), yields
\(\mathbf{0} = \sum r_iS(\mathbf{w}_i) = \sum r_i\mathbf{u}_i\). Hence \(r_{i} = 0\) for each \(i\), and so \(\sum t_j\mathbf{v}_j = \mathbf{0}\). This implies that each \(t_{j} = 0\), and so proves the independence of \(B\).
Proof of Theorem \(\PageIndex{1}\). By Lemma \(\PageIndex{1}\), it suffices to prove that \(dim \;_{\mathbb{C}}(\mathbf{D}_n^*) = n\). This holds for \(n = 1\) because the proof of Theorem [thm:010427] goes through to show that \(\mathbf{D}_1^* = \mathbb{C}e^{a_0x}\). Hence we proceed by induction on \(n\). With an eye on equation ([eq:diffOrderN]), consider the polynomial
\[p(t) = t^n - a_{n-1}t^{n-1} - a_{n-2}t^{n-2} - \cdots - a_2t^2 - a_1t - a_0 \nonumber \]
(called the characteristic polynomial of equation ([eq:diffOrderN])). Now define a map \(D : \mathbf{D}_{\infty} \to \mathbf{D}_{\infty}\) by \(D(f) = f^\prime\) for all \(f\) in \(\mathbf{D}_{\infty}\). Then \(D\) is a linear operator, whence \(p(D) : \mathbf{D}_{\infty} \to \mathbf{D}_{\infty}\) is also a linear operator. Moreover, since \(D^{k}(f) = f^{(k)}\) for each \(k \geq 0\), equation ([eq:diffOrderN]) takes the form \(p(D)(f) = 0\). In other words,
\[\mathbf{D}_n^* = \text{ker}[p(D)] \nonumber \]
By the fundamental theorem of algebra, which is what allows the solutions to be complex valued, let \(w\) be a complex root of \(p(t)\), so that \(p(t) = q(t)(t - w)\) for some complex polynomial \(q(t)\) of degree \(n - 1\). It follows that \(p(D) = q(D)(D - w1_{\mathbf{D}_\infty})\). Moreover \(D - w1_{\mathbf{D}_\infty}\) is onto by Lemma [lem:022913], \(dim \;_{\mathbb{C}}[\text{ker}(D - w1_{\mathbf{D}_\infty})] = 1\) by the case \(n = 1\) above, and \(dim \;_{\mathbb{C}}(\text{ker}[q(D)]) = n - 1\) by induction. Hence Lemma [lem:022938] shows that \(\text{ker}[P(D)]\) is also finite dimensional and
\[dim \;_\mathbb{C}(\text{ker}[p(D)]) = dim \;_\mathbb{C}(\text{ker}[q(D)]) + dim \;_\mathbb{C}(\text{ker}[D - w1_{\mathbf{D}_\infty}]) = (n -1) + 1 = n. \nonumber \]
Since \(\mathbf{D}_n^* = \text{ker}[p(D)]\), this completes the induction, and so proves Theorem [thm:022833].
