Loading [MathJax]/jax/element/mml/optable/BasicLatin.js
Skip to main content
Library homepage
 

Text Color

Text Size

 

Margin Size

 

Font Type

Enable Dyslexic Font
Mathematics LibreTexts

1.03: Sources of Error

( \newcommand{\kernel}{\mathrm{null}\,}\)

Lesson 1: Round-Off Errors

Learning Objectives

After successful completion of this lesson, you should be able to:

1) recognize that there are two inherent sources of error in numerical methods round-off and truncation error

2) define and exemplify round-off errors

Introduction

Error in solving an engineering or science problem can arise due to several factors. First, the error may be in the modeling technique. A mathematical model may be based on using assumptions that are not acceptable. For example, one may assume that the drag force on a car is proportional to the velocity of the car, but, actually it is proportional to the square of the velocity of the car. This itself can create huge errors in determining the performance of the car, no matter how accurate the numerical methods you may use. Second, errors may arise from mistakes in programs themselves or in the measurement of physical quantities. But, in applications of numerical methods themselves, the two errors we focus on are inherent to the subject, namely

1) round-off error

2) truncation error.

What is a round-off error?

A computer can only represent a number approximately. For example, a number like 13 may be represented as 0.333333 on a PC. Then the round-off error, in this case, is 130.333333=0.0000003ˉ3. Then there are other numbers that cannot be represented exactly. For example, π and 2 are numbers that need to be approximated in computer calculations.

Audiovisual Lecture 

Title: Sources of Error - Round-off Error

Summary: There are two sources of error - one comes from approximating numbers and another from approximating mathematical procedures. In this video, the error, round-off error, that is a result of approximating numbers, is defined and shown through an example.

What problems can be created by round-off errors?

Round-off errors while storing numbers or while conducting arithmetic operations can propagate errors. Here is a real-life example of what happened when a small round-off error was responsible for a tragedy during the 1991 US-Iraq war.

Twenty-eight Americans were killed on February 25, 1991. An Iraqi Scud hit the Army barracks in Dhahran, Saudi Arabia. The Patriot defense system had failed to track and intercept the Scud. What was the cause of this failure?

The Patriot defense system consists of an electronic detection device called the range gate. It calculates the area in the air space where it should look for a Scud. To find out where it should aim next, it calculates the velocity of the Scud and the last time the radar detected the Scud. Time is saved in a register that has 24 bits in length. Since the internal clock of the system is measured for every one-tenth of a second, 1/10 is expressed in a 24 bit-register as 0.00011001100110011001100. However, this is not an exact representation. In fact, it would need infinite numbers of bits to represent 1/10 exactly. So, the error in the representation in decimal format is

Patriot missile in the air.
Figure 1.3.1.1. Patriot missile (Courtesy of the US Armed Forces, http://www.redstone.army.mil/history/archives/patriot/patriot.html)

Error=110(0×21+0×22+0×23+1×24+...+1×222+0×223+0×224)=1102097152097152=9.537×108

The battery was on for 100 consecutive hours, hence causing an inaccuracy of

Cumulative Error=9.537×108s0.1 s×100 hr×3600 s1 hr=0.3433 s

The shift calculated in the range gate due to the error of 0.342 s was calculated as 687 m. For the Patriot missile defense system, the target is considered out of range if the shift is more than than 137 m. The shift of larger than 137 m resulted in the Scud not being targeted and hence killing 28 Americans in the barracks of Saudi Arabia.

Audiovisual Lecture

Title: Effect of Carrying Significant Digits

Summary: Not carrying enough significant digits can give us inaccurate answers.  In this lecture, a real-life illustration of the effect of using fixed decimal instead of significant digits is shown.

Show an example of how not carrying enough significant digits can affect the solution to a problem.

In Table 1.3.1.1, the instantaneous coefficient of thermal expansion of steel is given as a discrete function of temperature.

Table 1.3.1.1: Instantaneous Coefficient of Linear Thermal Expansion vs. Temperature for Steel
Temperature, T Instantaneous Coefficient of Linear Thermal Expansion, α
F μin/in/F
80 6.47
60 6.36
40 6.24
20 6.12
0 6.00
20 5.86
40 5.72
60 5.58
80 5.43
100 5.28
120 5.09
140 4.91
160 4.72
180 4.52
200 4.30
220 4.08
240 3.83
260 3.58
280 3.33
300 3.07
320 2.76
340 2.45

Regression is used to develop a simple formula in the form of a second-order polynomial to relate the coefficient of thermal expansion coefficient of steel as a function of temperature.

Using Microsoft Excel and using its trend line function, it gives the formula as shown in Figure 1.3.1.2.

Graph of the data points from Table 1.3.1, with a second-order polynomial trendline drawn through all points.
Figure 1.3.1.2. Regression model with default format for instantaneous coefficient of linear thermal expansion vs. temperature for steel.

The trend line given by Excel in its default format is

α=1×105T2+0.0062T+6.015(1.3.1.1)

At T=300F, the predicted value of the instantaneous coefficient of the thermal expansion from Equation (1.3.1.1) is

α=1×105(300)2+0.0062(300)+6.015=3.26 μin/in/F

From Table 1.3.1, the observed value at T=300F is

α=3.07 μin/in/F

which gives a residual at T=300F as

Ei=3.073.26=0.19 μin/in/F(1.3.1.2)

When a graduate student showed me the above numbers for the observed and predicted values, they did not match what was getting depicted in Figure 1.3.1.2. In the figure, the predicted value seemed much closer to the observed value than what is calculated above.

So what was the cause of this discrepancy? Notice that the default format setting for the trend line (Equation (1.3.1.1)) in Excel is the general format. But, now change the format setting to scientific format with say five significant digits, and we get the equation for the same trend line as

α=1.2278×105T2+6.1946×103T+6.0150(1.3.1.3)

Regression model with equation in scientific format for instantaneous coefficient of linear thermal expansion vs. temperature for steel.
Figure 1.3.1.3. Regression model with scientific format for instantaneous coefficient of linear thermal expansion vs. temperature for steel.

At T=300F, the predicted value of the instantaneous coefficient of the thermal expansion from Equation (3)

α=1.2278×105(300)2+6.1946×103(300)+6.0150=3.05 μin/in/F

From Table 1.3.1.1, the observed value at T=300F is

α=3.07 μin/in/F

which gives a residual of

Ei=3.073.05=0.02 μin/in/F(1.3.1.4)

The residual now looks reasonable.

The difference between the predicted value (Equation (1.3.1.2)) when using the default format of Excel

α=3.255 μin/in/F

and the predicted value when using scientific format with 5 significant digits (Equation (1.3.1.4))

α=3.051 μin/in/F

is attributed fully to the round-off error.

References

  1. This reference is the full GAO report of the investigation that resulted after the accident. “Patriot Missile Defense – Software Problem Led to System Failure at Dhahran, Saudi Arabia”, GAO Report, General Accounting Office, Washington DC, February 4, 1992.
  2. It should be pointed out that the Patriot missile was originally designed to be a mobile system and not used as an anti-ballistic system. In mobile systems, the clocks are reset more often. As per the article Operations: I Did Not Say You Could Do That! by Bill Barnes and Duke McMillin, here are some important observations: “It turns out that the original use case for this system was to be mobile and to defend against aircraft that move much more slowly than ballistic missiles. Because the system was intended to be mobile, it was expected that the computer would be periodically rebooted. In this way, any clock-drift error would not be propagated over extended periods and would not cause significant errors in range calculation. Because the Patriot system was not intended to run for extended times, it was probably never tested under those conditions—explaining why the problem was not discovered until the war was in progress. The fact that the system was also designed as an antiaircraft system probably also enabled the inclusion of such a design flaw because slower-moving airplanes would be easier to track and, therefore, less dependent upon a highly accurate clock value.”

Lesson 2: Truncation Error

Learning Objectives

After successful completion of this lesson, you should be able to:

1) define and exemplify truncation errors

2) recognize the difference between round-off and truncation error.

Introduction

Truncation error is defined as the error caused by approximating a mathematical procedure. Examples of mathematical procedure getting approximated include an infinite series where only the first few terms are used, integrals where the number of rectangles drawn to show the area under the curve is finite in number as opposed to infinite, and the derivative of a function is approximated by using the slope of a secant line as opposed to a tangent line.

Audiovisual Lecture 

Title: Sources of Error - Truncation Error

Summary: There are two sources of error - one comes from approximating numbers and another from approximating mathematical procedures. In this video, the truncation error, which results from approximating mathematical procedures, is defined.

Examples of Truncation Error

For example, the Maclaurin series for exis given as

ex=1+x+x22!+x33!+...(1.3.2.1)

This series has an infinite number of terms, but when using this series to calculate ex, only a finite number of terms can be used. For example, if one uses three terms to calculate ex, then

ex1+x+x22!

The truncation error for such an approximation for any value of x is

Truncation error= ex(1+x+x22!)=(1+x+x22!+x33!+....)(1+x+x22!)=x33!+x44!+.......................

Example 1.3.2.1

Given the following infinite series, find the truncation error for x=0.75 if only the first three terms of the series are used.

S=1+x+x2+x3+, |x|<1

Solution

Using only the first three terms of the series for x=0.75 gives

S3=(1+x+x2)x=0.75=1+0.75+(0.75)2=2.3125

The sum of an infinite geometrical series

S=a+ar+ar2+ar3+, |r|<1

is given by

S=a1r

For our series, a=1 and r=0.75, to give

S=110.75=4

The truncation error (TE) hence is

TE=42.3125=1.6875

Example 1.3.2.2

Use the concept of the relative approximate error to illustrate through an example how truncation error can be controlled. Assume that you are not privy to the exact value.

Solution

Let us show through an example how many terms need to be considered in Equation (1.3.2.1) to calculate e1.2 within 1% pre-specified tolerance. Of course, you are not privy to the exact answer.

The summation series for e1.2 is given by

e1.2=1+1.2+1.222!+1.233!+...................

In Table 1.3.2, we tabulate the value of e1.2, approximate error and absolute relative approximate error as a function of the number of terms of the series used, n.

Table 1.3.2: Values and approximate errors for e1.2 as a function of the number of terms of the Maclaurin series
n e1.2 Ea |ϵa|%
1 1
2 2.2 1.2 54.546
3 2.92 0.72 24.658
4 3.208 0.288 8.9776
5 3.2944 0.0864 2.6226
6 3.315136 0.020736 0.6255

Using a minimum of 6 terms of the series yields an absolute relative approximate error |ϵa|<1%. The reported value of e1.2 is 3.3151 and is shown up to 5 significant digits. As a side note, the exact value, up to 5 significant digits, of e1.2 is 3.3201.

Audiovisual Lecture 

Title: Example of Truncation Errors: Series

Summary: There are two sources of error - one comes from approximating numbers and another from approximating mathematical procedures. In this video, an example of truncation error is given using a Maclaurin series.

 

Lesson 3: Truncation Error Extended Examples

Learning Objectives

After successful completion of this lesson, you should be able to:

1) exemplify truncation errors through numerical differentiation and integration

Can you give me other examples of truncation errors?

In many textbooks on numerical methods, the Maclaurin series is used as the only example to illustrate truncation error. This example may lead you to believe that truncation errors are just a result of using a finite number of terms of the series. However, truncation errors take place in other mathematical procedures as well. In this lesson, we take two such examples.

Example 1.3.3.1

To find the exact derivative of a function, we define

f(x)=lim

But since we cannot use h \rightarrow 0, in numerical methods, we use a finite value of h, to give

f^{\prime}(x) \approx \frac{f(x + h) - f(x)}{h} \nonumber

So the truncation error is caused by choosing a finite value of {h } as opposed to a h \rightarrow 0.

Find the truncation error for finding f^{\prime}(3) for f(x) = x^{2} using a step size of h = 0.2

Solution

In finding f^{\prime}(3) for f(x) = x^{2}, we have the exact value of the derivative calculated as follows. Given

f(x) = x^{2} \nonumber

from the definition of the derivative of a function,

\begin{split} f^{\prime}(x) &= \lim_{h \rightarrow 0}\frac{f(x + h) - f(x)}{h}\\ &= \lim_{h \rightarrow 0}\frac{(x + h)^{2} - (x)^{2}}{\Delta x}\\ &= \lim_{h \rightarrow 0}\frac{x^{2} + 2xh + h^{2} - x^{2}}{\Delta x}\\ &= \lim_{h \rightarrow 0}(2x + h)\\ &= 2x\end{split} \nonumber

The above derivative is the same expression you would have obtained by directly using the formula from your differential calculus class

\frac{d}{dx}(x^{n}) = nx^{n - 1} \nonumber

By this formula for

f(x) = x^{2} \nonumber

f^{\prime}(x) = 2x \nonumber

The exact value of f^{\prime}(3) is

\begin{split} f^{\prime}(3) &= 2 \times 3\\ &= 6\end{split} \nonumber

Given the step size of h = 0.2, we get

\begin{split} f^{\prime}(3) &= \frac{f(3 + 0.2) - f(3)}{0.2}\\ &= \frac{f(3.2) - f(3)}{0.2}\\ &= \frac{3.2^{2} - 3^{2}}{0.2}\\ &= \frac{10.24 - 9}{0.2}\\ &= \frac{1.24}{0.2}\\ &= 6.2\end{split} \nonumber

\begin{split} \text{Truncation Error} &= \text{Exact Value} - \text{Approximate Value}\\ &= 6 - 6.2\\ &= -0.2\end{split} \nonumber

Note: The example purposefully chose a simple function f(x) = x^{2} with the value of x = 2 and h = 0.2 because we wanted to have no round-off error in our calculations so that the truncation error is the only error in the example.

Question: Can you reduce the truncation error by choosing a smaller h?

Example \PageIndex{3.2}

Another example of truncation error is the numerical integration of a function,

I = \int_{a}^{b}{f(x)dx} \nonumber

Exact calculations require us to calculate the area under the curve by adding the area of the rectangles, as shown in Figure \PageIndex{3.1}. However, exact calculations would require an infinite number of such rectangles. Since we cannot choose an infinite number of rectangles, we will have a truncation error.

Plot of y = x^2 showing the approximate area under the curve from x=3 to x=9 using two rectangles.
Figure \PageIndex{3.1}. Plot of y = x^{2} showing the approximate area under the curve from x = 3 to x = 9 using two rectangles.

For the integral

\int_{3}^{9}x^{2}{dx} \nonumber

find the truncation error if a two-segment left-hand Reimann sum with equal width of segments is used.

Solution

We have the exact value as

\begin{split} \int_{3}^{9}{x^{2}{dx}} &= \left\lbrack \frac{x^{3}}{3} \right\rbrack_{3}^{9}\\ &= \left\lbrack \frac{9^{3} - 3^{3}}{3} \right\rbrack\\ &= 234 \end{split} \nonumber

Using two rectangles of equal width to approximate the area (see Figure \PageIndex{3.1}) under the curve, the approximate value of the integral

\begin{split} \int_{3}^{9}{x^{2}{dx}} &\approx \left. \ (x^{2}) \right|_{x = 3}(6 - 3) + \left. \ (x^{2}) \right|_{x = 6}(9 - 6)\\ &= (3^{2})3 + (6^{2})3\\ &= 27 + 108\\ &= 135\end{split} \nonumber

\begin{split} \text{Truncation Error} &= \text{Exact Value} - \text{Approximate Value}\\ &= 234 - 135\\ &=99.\end{split} \nonumber

Note: Again, we purposefully chose a simple example with such numbers to produce numbers without round-off error in our calculations. This deliberate choice makes the obtained error purely due to truncation.

Question: Can you reduce the truncation error by choosing more rectangles as given in Figure \PageIndex{3.2}? What is the truncation error when four rectangles are chosen?

Plot of y=x^2 showing the approximate area under the curve from x = 3 to x = 9 using four rectangles.
Figure \PageIndex{3.2}. Plot of y = x^{2} showing the approximate area under the curve from x = 3 to x = 9 using four rectangles.

Audiovisual Lecture 

Title: Truncation Error: Example of Integration

Summary: There are two sources of error - one comes from approximating numbers and another from approximating mathematical procedures. In this video, an example of truncation error is given using a Maclaurin series.

Title: Truncation Error: Example of Differentiation

Summary: There are two sources of error - one comes from approximating numbers and another from approximating mathematical procedures. In this lecture, an example of truncation error is given using numerical differentiation.

 

Multiple Choice Test

(1). Truncation error is caused by approximating

(A) irrational numbers

(B) fractions

(C) rational numbers

(D) exact mathematical procedures.

(2). A computer that represents only 4 significant digits with chopping would calculate 66.666\times33.333 as

(A) 2220

(B) 2221

(C) 2221.17778

(D) 2222

(3). A computer that represents only 4 significant digits with rounding would calculate 66.666\times33.333 as

(A) 2220

(B) 2221

(C) 2221.17778

(D) 2222

(4). The truncation error in calculating f^\prime\left( 2 \right) for f\left( x \right) = x^{2} by \displaystyle f^{\prime}\left( x \right) \approx \frac{f\left( x + h \right) - f\left( x \right)}{h} with h = 0.2 is

(A) -0.2

(B) 0.2

(C) 4.0

(D) 4.2

(5). The truncation error in finding \displaystyle \int_{- 3}^{9}{x^{3}{dx}} using LRAM (left end point Riemann approximation method) with equally portioned points - 3 < 0 < 3 < 6 < 9 is

(A) 648

(B) 756

(C) 972

(D) 1620

(6). The number \displaystyle \frac{1}{10} is registered in a fixed 6 bit-register with all bits used for the fractional part. The difference is accumulated every \displaystyle \frac{1}{10}th of a second for one day. The magnitude of the accumulated difference is

(A) 0.082

(B) 135

(C) 270

(D) 5400

For complete solution, go to

http://nm.mathforcollege.com/mcquizzes/01aae/quiz_01aae_sourceserror_answers.pdf

Problem Set

(1). What is the round-off error in representing 200/3 in a 6-significant digit computer that chops the last significant digit?

Answer

0.00006666666....

(2). What is the round-off error in representing \dfrac{200}{3} in a 6-significant digit computer that rounds off the last significant digit?

Answer

-0.00003333333...

(3). Find the truncation error in the calculation of the f^{\prime}(x) that uses the approximation

f^{\prime}(x) \approx \frac{f(x + \Delta x) - f(x)}{\Delta x} \nonumber

for

f(x) = x^{3},\ \Delta x = 0.4,\ \text{and}\ x = 5 \nonumber

Answer

-6.16

(4) What is the truncation error in the calculation of \displaystyle \sin (\frac{\pi}{2}) if only first five terms of the Maclaurin series are used for the calculation? Ignore the round-off error in your calculations.

\sin(x) = x - \frac{x^{3}}{3!} + \frac{x^{5}}{5!} - \frac{x^{7}}{7!} + ...... \nonumber

Answer

-0.35427560 \times 10^{-5}

(5). The integral \displaystyle\int_{3}^{9} x^{2} \ dx can be calculated approximately by finding the area of the four rectangles as shown in Figure \PageIndex{1}. What is the truncation error due to this approximation?

Plot of y = x^2 showing the approximate area under the curve from x = 3 to x = 9 using four rectangles arranged as a left-hand Riemann sum.
Figure \PageIndex{1}. Plot of y = x^{2} showing the approximate area under the curve from x = 3 to x = 9 using four rectangles.
Answer

51.75

(6). Below is the data given for linear coefficient of thermal expansion for steel as a function of temperature.

Temperature Instantaneous Thermal Expansion Coefficient
^{\circ} \text{F} \mu \text{in/in/}^{\circ}\text{F}
80 6.47
60 6.36
40 6.24
20 6.12
0 6.00
-20 5.86
-40 5.72
-60 5.58
-80 5.43
-100 5.28
-120 5.09
-140 4.91
-160 4.72
-180 4.52
-200 4.30
-220 4.08
-240 3.83
-260 3.58
-280 3.33
-300 3.07
-320 2.76
-340 2.45

Regression is used to come up with a simple formula – a second order polynomial to relate the linear coefficient of thermal expansion of steel as a function of temperature.

a) The formula given by MS Excel using default settings is shown in Figure \PageIndex{2}. Using the formula, what is the value of the linear coefficient of thermal expansion at T = - 300, -200, - 100, 0 and 100^\circ \text{F}? Compare these values with those given in the table above.

Graph of the data points given above, with trendline equation in the Excel default form -10^5 x^2 + 0.0062x + 6.015.
Figure \PageIndex{2}. Default trend line given by Excel.

(b) Now the default settings for the trend line are changed to scientific format with five significant digits (Figure \PageIndex{3}). Using the formula, what is the value of linear coefficient of thermal expansion at T = - 300, - 200,- 100, 0 and 100^\circ \text{F}? Compare these values with those given in the table above.

Graph of the data points given above, with trendline equation in the five-significant-digit form -1.2278*10^5 x^2 + 6.1946*10^-3 x + 6.0150.
Figure \PageIndex{3}. Trend line given by Excel using scientific format.

c) Do you get different answers in part (a) and part (b) for the predicted value of the linear coefficient of thermal expansion at T = - 300^{\circ}\text{F}? Yes or No.

d) In part (c), what do you attribute the difference or lack of difference in answers to?

Answer

a) At T=-300^{\circ}\text{F}, \alpha=3.255 \text{ in/in/}^{\circ}\text{F}. From table, \alpha=3.07 \text{ in/in/}^{\circ}\text{F}

b) At T=-300^{\circ}\text{F}, \alpha=3.0516 \text{ in/in/}^{\circ}\text{F}. From table, \alpha=3.07 \text{ in/in/}^{\circ}\text{F}

c) No

d) Answer not given intentionally.


This page titled 1.03: Sources of Error is shared under a CC BY-NC-SA 4.0 license and was authored, remixed, and/or curated by Autar Kaw via source content that was edited to the style and standards of the LibreTexts platform.

Support Center

How can we help?

1.02: Quantifying Errors
1.04: Fixed-Point Binary Representation of Numbers