$$\newcommand{\id}{\mathrm{id}}$$ $$\newcommand{\Span}{\mathrm{span}}$$ $$\newcommand{\kernel}{\mathrm{null}\,}$$ $$\newcommand{\range}{\mathrm{range}\,}$$ $$\newcommand{\RealPart}{\mathrm{Re}}$$ $$\newcommand{\ImaginaryPart}{\mathrm{Im}}$$ $$\newcommand{\Argument}{\mathrm{Arg}}$$ $$\newcommand{\norm}[1]{\| #1 \|}$$ $$\newcommand{\inner}[2]{\langle #1, #2 \rangle}$$ $$\newcommand{\Span}{\mathrm{span}}$$

# 8.5: Chaos

You have surely heard the story about the ﬂap of a butterﬂy wing in the Amazon causing hurricanes in the North Atlantic. In a prior section, we mentioned that a small change in initial conditions of the planets can lead to very diﬀerent conﬁguration of the planets in the long term. These are examples of chaotic systems. Mathematical chaos is not really chaos, there is precise order behind the scenes. Everything is still deterministic. However a chaotic system is extremely sensitive to initial conditions. This also means even small errors induced via numerical approximation create large errors very quickly, so it is almost impossible to numerically approximate for long times. This is large part of the trouble as chaotic systems cannot be in general solved analytically.

Take the weather for example. As a small change in the initial conditions (the temperature at every point of the atmosphere for example) produces drastically diﬀerent predictions in relatively short time, we cannot accurately predict weather. This is because we do not actually know the exact initial conditions, we measure temperatures at a few points with some error and then we somehow estimate what is in between. There is no way we can accurately measure the eﬀects of every butterﬂy wing. Then we will solve numerically introducing new errors. That is why you should not trust weather prediction more than a few days out.

The idea of chaotic behavior was ﬁrst noticed by Edward Lorenz8 in the 1960s when trying to model thermally induced air convection (movement). The equations Lorentz was looking at form the relatively simple looking system:

A small change in the initial conditions yield a very diﬀerent solution after a reasonably short time.

A very simple example the reader can experiment with, which displays chaotic behavior, is a double pendulum. The equations that govern this system are somewhat complicated and their derivation is quite tedious, so we will not bother to write them down. The idea is to put a pendulum on the end of another pendulum. If you look at the movement of the bottom mass, the movement will appear chaotic. This type of system is a basis for a whole number of oﬃce novelty desk toys. It is very simple to build a version. Take a piece of a string, and tie two heavy nuts at diﬀerent points of the string; one at the end, and one a bit above. Now give the bottom nut a little push, as long as the swings are not too big and the string stays tight, you have a double pendulum system.

#### 8.5.1 Duﬃng equation and strange attractors

Let us study the so-called Duﬃng equation:

Here , and  are constants. You will recognize that except for the  term, this equation looks like a forced mass-spring system. The  term comes up when the spring does not exactly obey Hooke’s law (which no real-world spring actually does obey exactly). When  is not zero, the equation does not have a nice closed form solution, so we have to resort to numerical solutions as is usual for nonlinear systems. Not all choices of constants and initial conditions will exhibit chaotic behavior. Let us study

The equation is not autonomous, so we will not be able to draw the vector ﬁeld in the phase plane. We can still draw the trajectories however.

In Figure 8.11 we plot trajectories for  going from 0 to 15, for two very close initial conditions  and , and also the solutions in the  space. The two trajectories are close at ﬁrst, but after a while diverge signiﬁcantly. This sensitivity to initial conditions is precisely what we mean by the system behaving chaotically.

Figure 8.11: On left, two trajectories in phase space for , for the Duﬃng equation one with initial conditions  and the other with . On right the two solutions in -space.

Figure 8.12: The solution to the given Duﬃng equation for  from 0 to 100.

Let us see the long term behavior. In Figure 8.12, we plot the behavior of the system for initial conditions , but for much longer period of time. Note that for this period of time it was necessary to use a ridiculously large number of steps in the numerical algorithm used to produce the graph, as even small errors quickly propagate9. From the graph it is hard to see any particular pattern in the shape of the solution except that it seems to oscillate, but each oscillation appears quite unique. The oscillation is expected due to the forcing term.

In general it is very diﬃcult to analyze chaotic systems, or to ﬁnd the order behind the madness, but let us try to do something that we did for the standard mass-spring system. One way we analyzed what happens is that we ﬁgured out what was the long term behavior (not dependent on initial conditions). From the ﬁgure above it is clear that we will not get a nice description of the long term behavior, but perhaps we can ﬁgure out some order to what happens on each “oscillation” and what do these oscillations have in common.

The concept we will explore is that of a Poincarè section10. Instead of looking at  in a certain interval, we will look at where the system is at a certain sequence of points in time. Imagine ﬂashing a strobe at a certain ﬁxed frequency and drawing the points where the solution is during the ﬂashes. The right strobing frequency depends on the system in question. The correct frequency to use for the forced Duﬃng equation (and other similar systems) is the frequency of the forcing term. For the Duﬃng equation above, ﬁnd a solution , and look at the points

As we are really not interested in the transient part of the solution, that is, the part of the solution that depends on the initial condition we skip some number of steps in the beginning. For example, we might skip the ﬁrst 100 such steps and start plotting points at , that is

The plot of these points is the Poincarè section. After plotting enough points, a curious pattern emerges in Figure 8.13 (the left hand picture), a so-called strange attractor.

Figure 8.13: Strange attractor. The left plot is with no phase shift, the right plot has phase shift .

If we have a sequence of points, then an attractor is a set towards which the points in the sequence eventually get closer and closer to, that is, they are attracted. The Poincarè section above is not really the attractor itself, but as the points are very close to it, we can see its shape. The strange attractor in the ﬁgure is a very complicated set, and it in fact has fractal structure, that is, if you would zoom in as far as you want, you would keep seeing the same complicated structure.

The initial condition does not really make any diﬀerence. If we started with diﬀerent initial condition, the points would eventually gravitate towards the attractor, and so as long as we throw away the ﬁrst few points, we always get the same picture.

An amazing thing is that a chaotic system such as the Duﬃng equation is not random at all. There is a very complicated order to it, and the strange attractor says something about this order. We cannot quite say what state the system will be in eventually, but given a ﬁxed strobing frequency we can narrow it down to the points on the attractor.

If you would use a phase shift, for example , and look at the times

you would obtain a slightly diﬀerent looking attractor. The picture is the right hand side of Figure 8.13. It is as if we had rotated, distorted slightly, and then moved the original. Therefore for each phase shift you can ﬁnd the set of points towards which the system periodically keeps coming back to.

You should study the pictures and notice especially the scales—where are these attractors located in the phase plane. Notice the regions where the strange attractor lives and compare it to the plot of the trajectories in Figure 8.11.

Let us compare the discussion in this section to the discussion in § 2.6 about forced oscillations. Take the equation

This is like the Duﬃng equation, but with no  term. The steady periodic solution is of the form

Strobing using the frequency  we would obtain a single point in the phase space. So the attractor in this setting is a single point—an expected result as the system is not chaotic. In fact it was the opposite of chaotic. Any diﬀerence induced by the initial conditions dies away very quickly, and we settle into always the same steady periodic motion.

#### 8.5.2 The Lorenz system

In two dimensions to have the kind of chaotic behavior we are looking for, we have to study forced, or non-autonomous, systems such as the Duﬃng equation. Due to the Poincarè-Bendoxson Theorem, if an autonomous two-dimensional system has a solution that exists for all time in the future and does not go towards inﬁnity, then we obtain a limit cycle or a closed trajectory. Hardly the chaotic behavior we are looking for.

The Lorenz system is an autonomous system in three dimensions exhibiting chaotic behavior. See the Figure 8.14 for a sample trajectory.

Figure 8.14: A trajectory in the Lorenz system.

The solutions will tend to an attractor in space, the so-called Lorenz attractor. In this case no strobing is necessary. Again we cannot quite see the attractor itself, but if we try to follow a solution for long enough, as in the ﬁgure, we will get a pretty good picture of what the attractor looks like.

The path is not just a repeating ﬁgure-eight. The trajectory will spin some seemingly random number of times on the left, then spin a number of times on the right, and so on. As this system arose in weather prediction, one can perhaps imagine a few days of warm weather and then a few days of cold weather, where it is not easy to predict when the weather will change, just as it is not really easy to predict far in advance when the solution will jump onto the other side. See Figure 8.15 for a plot of the  component of the solution drawn above.

Figure 8.15: Graph of the  component of the solution.

#### 8.5.3 Exercises

Exercise 8.5.1: For the non-chaotic equation , suppose we strobe with frequency  as we mentioned above. Use the known steady periodic solution to ﬁnd precisely the point which is the attractor for the Poincarè section.

Exercise 8.5.2 (project): A simple fractal attractor can be drawn via the following chaos game. Draw three points of a triangle (just the vertices) and number them, say  and . Start with some random point  (does not have to be one of the three points above) and draw it. Roll a die, and use it to pick of the , or  randomly (for example 1 and 4 mean , 2 and 5 mean , and 3 and 6 mean ). Suppose we picked , then let  be the point exactlyhalfway between  and . Draw this point and let  now refer to this new point . Rinse, repeat. Try to be precise and draw as many iterations as possible. Your points should be attracted to the so-called Sierpinski triangle. A computer was used to run the game for 10,000 iterations to obtain the picture in Figure 8.16.

Figure 8.16: 10,000 iterations of the chaos game producing the Sierpinski triangle.

Exercise 8.5.3 (project): Construct the double pendulum described in the text with a string and two nuts (or heavy beads). Play around with the position of the middle nut, and perhaps use diﬀerent weight nuts. Describe what you ﬁnd.

Exercise 8.5.4 (computer project): Use a computer software (such as Matlab, Octave, or perhaps even a spreadsheet), plot the solution of the given forced Duﬃng equation with Euler’s method. Plotting the solution for from 0 to 100 with several diﬀerent (small) step sizes. Discuss.

Exercise 8.5.101: Find critical points of the Lorenz system and the associated linearizations.

8Edward Norton Lorenz (1917–2008) was an American mathematician and meteorologist.

9In fact for reference, 30,000 steps were used with the Runge-Kutta algorithm, see exercises in § 1.7.

10Named for the French polymath Jules Henri Poincarè (1854–1912).

Contributors