Zach Jarchow Differential Equations Project
Part 1
While it is possible to get arbitrarily close to the exact solution of a differential equation on a short time
interval using a numerical solver, it is possible to diverge from the exact solution quickly depending on
the solver and step sized utilized. With too large a step size, the approximation may miss significant
values simply because the solver “jumped over” them. While the solver might return to the solution
curve, it is not guaranteed. It is even possible for an approximation to be far more accurate in a
neighborhood around a certain value, but inaccurate at that value exactly. It is possible to both be both
above or below the exact solution. The order of magnitude of explicit error between an approximation
and an exact solution can differ greatly depending on the approximation method used.
It is possible to predict long-term solutions to initial value problems. However, the equation of the D.E.
must be considered, and the method used to approximate it must also be evaluated. When there is no
exact solution to compare an approximation to, one should evaluate the IVP using progressively smaller
step sizes of a fixed-step solver or compare the results of multiple solvers. It is possible with a fixed-step
solver that the numerical method will diverge from the actual behavior of the solution and not return to
the “actual” path. When this happens within the interval where a solution curve is being inspected, the
only recourse is to recalculate the approximation using a smaller step size.
Part 2
We consider the initial value problem (x)y’ + 2y=3x with y(1)=5. An explicit solution is y(x) = x + 4/x 2.
Using a Mathematica worksheet to evaluate Euler’s method, I found several values of y(2) for different
step sizes.
The actual value for y(2) is 3. Using a step size of .1, Euler’s method yielded a value of y(2)= 2.84211.
Then using a step size of .2, this same method yielded y(2) = 2.66667. Finally, Euler’s method was
evaluated using a step size of .05, which yielded y(2)= 2.92308.
It is possible to calculate the error for Euler’s method exactly in this case because the problem included
an explicit solution for y(x). The absolute value of error was calculated by subtracting yEuler(2)-yExplicit(2).
Using Mathematica, I calculated the error for each step size.
The approximations for y(2)=3 are listed in the following table along with the explicit error.
Step Size y(2) Explicit Error
.2 2.66667 0.333333
.1 2.84211 0.157895
.05 2.92308 0.0769231
Looking at this data, there appears to be a direct relationship between step size and the error. This
relationship is because Euler’s method works by defining a tangent line at a point, and then moving
along the tangent line a distance equal to the step size. The tangent line provides a rough projection of
Zach Jarchow Differential Equations Project
where the solution curve is heading from a certain point. A large step size means that the next step is
evaluated at a point on the tangent line relatively far from the previous point. This produces an
immediate error, as the next point will not be exactly on the explicit solution curve. This error is
compounded because further iterated evaluations of Euler’s Method will diverge further from points
that are already off the solution curve.
A smaller step size is preferable, because moving in small step sizes along the tangent line keeps the
points Euler’s method produces relatively close to the explicit solution. Unfortunately, there is a trade
off for increased accuracy, because a smaller step size means that more calculations must be preformed
to reach a certain point. When y(2)was reached with a step size of .2, difference between the
approximated value and the actual value was 1/3. Taking a step size of .1 to reach y(2) yielded a smaller
error, and taking an even smaller size of .05 both yielded respectively smaller errors. Therefore, the
bigger the step size, the bigger the error term will be. While computers can streamline the process of
evaluating an approximation, the inaccuracy of this method is still apparent and must be considered.
This is a very simple function and it is possible that for functions that are more complicated this method
will not give accurate results for any reasonable step size.
Zach Jarchow Differential Equations Project
Part 3
The project outlined three different methods for approximating numerical solutions to differential
equations. Using Mathematica, I was able to create a notebook to evaluate the methods for a
differential equation, and comparing their outputs at a single point. Since the initial value problem (x)y’
+ 2y=3x with y(1)=5 has an explicit solution of y(x) = x + 4/x2, I was able to compare the three
approximations to an actual value at a certain point.
The value of y(3) was evaluated using Euler’s Method, Heun’s Method (the modified Euler’s Method)
and the 4th Order Runge-Kutta method using a step size of .1 for all approximations. The explicit
solution to dy/dx =f(x,y) provides that y(3) = 3.44444. This explicit solution was used to calculate an
absolute value of error in each case of yApproximate(3)-yExplicit(3).
These calculations are summarized in the following table.
Method y(3) Explicit Error
Euler’s Method 3.35468 0.0897646
Heun’s Method 3.44665 0.00220513
4th Order Runge-Kutta 3.44445 3.07785*10-6
Exact Solution 3.44444 0
It is evident that each method produces a relatively accurate result for y(3). However, the degree of
accuracy varies with the method. Euler’s had the least accurate result, followed by Heun’s, and the 4th
Order Runge-Kutta method gave the best approximation. It is necessary to discuss each method in turn
to realize why the explicit error has several order of magnitudes difference between the best and worst
approximations.
As previously discussed in section 2, Euler’s method works by defining the numerical solution to a
differential equation as the slope of a tangent line at a point, and then moving along that tangent line
the distance of the step size, generating a new tangent line, and recursively iterating along these
tangent lines until a certain value is reached. As was previously noted, Euler’s method produces a
relatively large error as each step compounds upon the previous. Euler’s method is very simple to
calculate, but the next step is approximated using only one piece of data.
Heun’s method was the next most accurate method. It uses two averaged pieces of data in order to find
the next point in a solution curve. First it finds the slope of the tangent line at an initial point. Then it
finds the tangent line at the point Euler’s method would have called the next point. Finally, it computes
a new slope for a line to the next point by averaging these two points together. It then travels down this
slope the amount of the step size in order to find the next value. Heun’s method uses the second
intermediate step to build in some error calculation. For each step, Heun’s method does twice as many
calculations as Euler’s method, and produces a more accurate result.
Zach Jarchow Differential Equations Project
The 4th Order Runge-Kutta method was the most accurate approximation. It works similarly to Heun’s
method, but produces 4 intermediate slopes and then uses a weighted average of them to find the slope
of the line to the next point. These intermediate slopes provide additional accuracy because two of the
slopes are at the midpoint of interval between the initial point and the next point. These points are
weighted more heavily than the other two slopes (at the initial point and at an intermediate
approximation of the final point) . Averaging these four slopes seems to produce a fair more accurate
solution than either Euler’s Method or Heun’s Method.
In summary, we see that while Euler’s method produced a rough approximation, it was not even
accurate to one decimal point. At this point on the solution curve, Heun’s method was accurate to two
decimal points and the Runge-Kutta method was accurate to four. The order of magnitude for the
explicit error of the Runge-Kutta method was many times smaller than that of Euler’s method. It is
important to recall that all three methods were evaluated using the same step size. Therefore, the most
accurate solution seems to be given by the Runge-Kutta method, owing to the fact that for each step, it
makes several calculations to determine the next point, which decreases the value of the error at each
intermediate step. This is incredibly helpful when attempting to find a numerical solution to a D.E. at a
certain point because Runge-Kutta keeps the points this method generates from diverging significantly
from the solution curve before the solver even reaches the certain point. Therefore the preferred
method of those discussed in this project would be the Runge-Kutta method.
Zach Jarchow Differential Equations Project
Part 4
For part four, we consider the initial value problem:
, y(0) = 1 , over the range [0,30].
Euler’s method was used to obtain approximate solutions for this problem. This method was chosen
because its simplicity makes it easy to explain how its results are obtained. Approximate solution curves
were obtain for this IVP at various step sizes by choosing enough steps such that the final value would
be y(30). These curves were calculated using a Mathematica notebook, and the results of each step
were stored in a list so that a plot could be generated. The points were linked with smooth lines
between each point. The project asked for solutions on the range [0,30] using step sizes of .2, .15 and
.1.
Using Mathematica (notebook attached) and the process laid out previously, the following plots were
obtained.
Euler's M ethod Plot Step Size: , 0.15
Euler's M ethod Plot Step Size: , 0.2
3
20
2
15
1
10
5 10 15 20 25 30
5 1
2
5 10 15 20 25 30
3
Euler's M ethod Plot Step Size: , 0.1
1.5
1.0
0.5
5 10 15 20 25 30
The D.E. seems to behave differently than the D.E. discussed in parts 2 and 3. Euler’s method seems to
generate a drastically different solution curve for the IVP based on the step sized used. The graphs of
Zach Jarchow Differential Equations Project
the larger two step sizes both oscillate across the x-axis significantly earlier than the graph with the
smallest step size of .1. To explain this correlation between step size and the graph, we must remember
that Euler’s method works by computing a tangent line and then moving along it. All three graphics look
relatively similar on from x=0 to ~x=21. In all three graphs the solution curve was approaching the x-axis
until that point. However, we see that with the largest step size of .2, the graph because to oscillate
across the x-axis around x=20. Compare this with the behavior of the solution with a step size .1 where
the curve simply approaches the x-axis. At this point, just before the first curve starts jumping across
the x-axis, the values of y are very small, ~.05. Recalling the D.E. , we see that it is
sinusoidal dependent on the dependent variable y. For the larger step size, Euler’s method actually
“overshoots” what seems to be the solution curve and crosses the x-axis. This behavior has been
described as truncation error which occurs at every iteration through Euler’s method. At this point the
graph doesn’t rejoin the either of the other curves, and instead oscillates across the x-axis a few times
before growing and making some oscillations at y values of much higher values than either other curve
reaches.
The second graph also diverges from the general path of the smallest step-size graph. At approximately
x=25, it begins oscillating across the x-axis with an amplitude significantly large than the y-values in the
neighborhood approaching this region. Again, this happens because at a certain x-value, Euler’s method
will generate a corresponding y-value that crosses the x-axis and throws off the general path of the
solution. We can connect this with the D.E. because the sine of a small negative number is a small
negative number. When the y-value of the previous step is very small and negative, we see that the
slope can immediately reverse its direction from negative to positive because the term is
subtracted in the D.E. This explains the oscillating behavior that seems to affect all three graphs
eventually.
The relatively massive amplitudes these oscillations can display is also because of the sinusoidal
dependence on y. Remembering that the sin function is periodic, we know that for certain values of y,
sin(y) ~= 1. This is less of a problem (for most of the graph) when the step size is .1, but for the other
two step sizes, this periodic tendency to return 1 becomes a problem much sooner. When the sin
function returns a significantly large enough fraction, the subtraction term will overpower the constant
1 term in the D.E. (in either direction =/-) yielding a steep slope in the appropriate direction. With a
large step size, Euler’s method will take then return a point very far from the previous point. The
approximate solution curve is then completely removed from the long-term path of the solution. This
behavior afflicts Euler’s method for this D.E. even with a step size as small as .1. The solution seems to
be tending towards the x-axis indefinitely, and generates y values of approximately .03. However, the
Euler’s method will still eventually result in an iteration with a steep enough slope that .1 is a large
enough step size to begin causing oscillations and cross the x-axis.
Tying the behavior of these steps sizes and their graphs together, it is apparent that the sinusoidal
nature of the D.E. and the decreasing behavior the tangent line slope must
eventually display both contribute to a solution curve that will oscillate depending on the step size
chosen. The sin function seems to be especially dangerous in this D.E. It is very possible due to the
Zach Jarchow Differential Equations Project
periodic nature of this function that errors in the slope Euler’s method calculates can be introduced, as a
large step size causing the solution to “jump” past slopes that might change the amplitude or direction
of the sin function’s output. Since Euler’s method uses only a single calculation for the slope, there is
nothing in place to correct for this apparent deviation.
An additional correlation is made between step size and interval. Considering the following graph, it is
apparent that the three solutions all agree with each other for a significant interval. For a smaller
interval, two of the step sizes continue to agree with each other as well. Looking at the graphs of the
different approximations and noting when they begin to diverge, it becomes apparent that the larger
the step size is, the shorter the interval the approximation returns useful output is.
Euler's M ethod Plot Step Size: , 0.1
20
15
10
5
5 10 15 20 25 30
Zach Jarchow Differential Equations Project
Therefore, when using a numerical solver to approximate the solution to an IVP, one must consider the
region in which a solution is desired. Not only do smaller step sizes yield more accurate results, but a
sufficiently small step size may even be required in order to give usable output in a certain region.
Additionally, when no explicit solution is possible, the only method of evaluating the output of an
approximation of one size or method would be to perform another evaluation with a smaller step size or
different method. Euler’s method is therefore evaluated one final time, with a step size significantly
smaller than .1 in order to confirm the apparent behavior of the solution on this interval.
Euler's M ethod Plot Step Size: , 0.01
1.5
1.0
0.5
5 10 15 20 25 30