16. Approximating differential equations

16.1. Motivation and plan

The laws of nature are expressed as differential equations.

This is because in describing nature we discuss how something changes, either from location to another, or as time goes by. These changes are expressed as derivatives of a function, and the laws of nature relate those derivativse to other functions.

16.2. A review of derivatives

First refer the class to a quick look at the animation of derivatives in the mini courses book, the chapter “pushing toward calculus”.

Then we will simply do a few examples of this calculation. Our purpose is not to do an in-depth review, but just to feel some agility, to remember a few simple formulae, and to show some of the simplest examples to people who have not yet taken a calculus class.

\begin{eqnarray} \frac{dx^2}{dx} & = \; & \lim_{h \to 0} \frac{(x+h)^2 - x^2}{h} \\ & = & \lim_{h \to 0} \frac{x^2 + 2 x h + h^2 - x^2}{h} \\ & = & \lim_{h \to 0} \frac{2 x h + h^2}{h} \\ & = & 2 x \end{eqnarray}

if we remember the Pascal triangle we note that \((x+h)^3 = x^3 + 3x^2 h + 3 x h^2 + h^3\). If we also remember that \(h^2\) and \(h^3\) get small very quickly and disappear in the limit, we get:

\begin{eqnarray} \frac{dx^3}{dx} & = \; & \lim_{h \to 0} \frac{(x+h)^3 - x^3}{h} \\ & = & \lim_{h \to 0} \frac{x^3 + 3 x^2 h + 3 x h^2 + h^3 - x^3}{h} \\ & = & \lim_{h \to 0} \frac{3 x^2 h + 3 x h^2 + h^3}{h} \\ & = & 3 x^2 \end{eqnarray}

and looking at higher order binomial expansions we see that:

\[\frac{dx^n}{dx} = n \times x^{n-1}\]

People sometimes keep a mnemonic for their math formulae. I tend to think that the n “comes down in front, and gets reduced in the exponent”.

Note that some basic properties hold:

\[\begin{split}\frac{d A f(x)}{dx} \;\; & = A \frac{df(x)}{dx} \\ \frac{d (f(x) + g(x))}{dx} & = \frac{df(x)}{dx} + \frac{dg(x)}{dx}\end{split}\]

Putting those together we get that:

\[\frac{d (A f(x) + B g(x))}{dx} = A \frac{df(x)}{dx} + B \frac{dg(x)}{dx}\]

This last property leads to the jargon that “the derivative is a linear operator.”

Now for some slightly more complex ones:

\[\begin{split}\frac{d \sin(x)}{dx} & = \cos(x) \\ \frac{d \cos(x)}{dx} & = -\sin(x) \\ \frac{d e^x}{dx} & = e^x \\ \frac{d \log(x)}{dx} & = \frac{1}{x}\end{split}\]

16.3. What is a differential equation?

A differential equation is one where you have functions of your variable and their derivatives, and you are trying to find the function, not the variable.

For example:

\[\frac{df(x)}{dx} = 7 x + 2\]

We solve this by educated guessing: the derivative of \(\frac{7}{2} x^2 \;\; \textrm{is} \;\; 7x\), and the derivative of \(2x\) is 2. This leads us to conclude that:

\[f(x) = \frac{7}{2} x^2 + 2 x + C\]

where C is the famous arbitrary constant.

16.4. A simple example: exponential growth

An example from ecology: start with two rabbits, and these rabbits have no constraints: they can reproduce, and never run out of food, and there no predators, and lots of other simplifying assumptions. Then the population is governed by this equation:

\[\frac{dP(t)}{dt} = r * P(t)\]

Let us pick this apart.

How did you arrive to it?

There are various ways in which scientists have written down equations that govern phenomena. In this case you can reason it through by saying that: “The more rabbits you have, the higher the growth rate”. This is the same as saying that the growth rate \((dP(t)/dt)\) is proportional to how many rabbits you already have \((P(t))\).

Is there some jargon to go with this?

Of course! I usually refer to the differential equation for a function of time (in our case \(P(t)\)) as the injection of a dynamic principle into the system. Other equations might give you information about setting up an initial system, but the differential equation with respect to time introduces dynamics into the picture.

What about \(r\)?

\(r\) can be thought of as the rate at which the population grows. A bigger \(r\) makes for a much faster population growth.

OK, so dynamical principle, initial, … Any other jargon? Ah yes: this was a first order linear differential equation.

First order differential equation

The highest order of the derivative in the equation. In this case we had the first derivative of \(P(t)\) so it was a first order differential equation.

Linear differential equation

The function \(P(t)\) and its derivatives always appeared linearly, i.e. they were all to the first power and multiplied by constants. (Think of the equation of a straight line \(y = m x + b\). This is linear in \(x\) because \(x\) does not appear to higher powers, and its only operation is to be multiplied by a constant.

Can we solve this equation analytically? (And here you pause and ask the “class what does analytically mean?”)

For this simple example yes, we can solve it analytically: we have encountered exactly one math function whose derivative is proportional to itself:

\[\frac{d e^t}{dt} = e^t\]

and therefore:

\[\frac{d e^{\alpha t}}{dt} = \alpha e^{\alpha t}\]

So the solution to our equation is:

\[P(t) = \textrm{constant} \times e^{rt}\]

Here you have an “arbitrary constant”, as they are sometimes called. Note that the original differential equation is such that multiplying by that constant does not change the result, so the constant is a key part of the general solution.

So what is that constant physically?

We can read directly from the equation that a time \(t = 0\) we will have \(e^{r \times 0} = 1\), so the constant is just \(P(0)\).

Using the shorthand \(P_0 = P(0)\), we end up with:

\[P(t) = P_0 \times e^{rt}\]

So in this case our “arbitrary constant” was not all that arbitrary: it represented an initial condition of the system (another piece of jargon in differential equations).

This often happens: a differential equation representing a dynamic situation will have an initial condition, and that initial condition can be found in those mathematical arbitrary constants. They are arbitrary to the mathematician, but to the practicing scientist they reflect the state of a natural system at the starting moment.

And you might have noticed that our first order differential equation for exponential growth had one arbitrary constant. Second order equations will have two arbitrary constants, and so forth.

16.5. Solving differential equations numerically: Euler’s method

At this time we will click here and follow that chapter to learn about solving differential equations numerically.

16.6. Some types of equations that we would like to solve

16.6.1. Ecology: the Lotka-Volterra equations.

16.6.2. Physics: free fall with air drag.

16.6.3. Ecology and economics: the logistic equation.

16.6.4. Physics: the non-linear pendulum

16.7. Scipy and the Runge-Kutta method