Kinds of Errors in Numerical Procedures

Computers use only a fixed number of digits
to represent a number.

• As a result, the numerical values stored in a computer are said to have finite precision.
• Limiting precision has the desirable effects of increasing the speed of numerical calculations and reducing memory required to store numbers.
• But, what are the undesirable effects?

Kinds of Errors:

i	Round-off Error
ii	Truncation Error
iii	Propagated Error

i	Round-off Error:

    1 #!/usr/bin/python3
    2 x=(4/3)*3; print(x)
    3 # 4.0
    4 a=4/3; print(a) # store double precision approx of 4/3
    5 # 1.3333333333333333
    6 b=a-1; print(b) # remove most significant digit
    7 # 0.33333333333333326
    8 c=1-3*b; print(c) # 3*b=1 in exact math
    9 # 2.220446049250313e-16 # should be 0!!

    1 from math import *
    2 # import numpy as np
    3 # kfirst=1.0; klast=360.0; kincrement=0.1
    4 # for j in np.arange(kfirst, klast + kincrement, kincrement):
    5 for j in range(1,360): # In degrees (1-360) as int. increment
    6     jj=j*(2*pi/360) # Conversion to radian
    7     a=cos(jj) # Return the cosine of jj (measured in radians)
    8     b=sin(jj) # Return the cosine of jj (measured in radians)
    9     z=a-(a/b)*b # Expected as being 0 !!
   10     print(j,jj,z)
   11 352 6.14355896702004 0.0
   12 353 6.161012259539984 1.1102230246251565e-16
   13 354 6.178465552059927 1.1102230246251565e-16
   14 355 6.19591884457987 1.1102230246251565e-16
   15 356 6.213372137099813 0.0
   16 357 6.230825429619756 0.0
   17 358 6.2482787221397 0.0
   18 359 6.265732014659643 0.0

    1 summation=1.0
    2 for i in range(10000):  # Adding 0.00001 to 1.0 as 10000 times
    3     summation=summation+0.00001
    4 print('summation = ', summation)
    5 # summation =  1.1000000000006551 # Expected result is just 1.1 !!
    6 print("summation = %f" % summation)
    7 # summation = 1.100000 # Now expected result??

To see the effects of roundoff in a simple calculation, one need only to force the computer to store the intermediate results.

• All computing devices represents numbers, except for integers and some fractions, with some imprecision.
• Floating-point numbers of fixed word length; the true values are usually not expressed exactly by such representations.

    1 import numpy as np
    2 x=np.tan(np.pi/6);  print(x)
    3 # 0.5773502691896257
    4 y=np.sin(np.pi/6)/np.cos(np.pi/6); print(y)
    5 # 0.5773502691896256
    6 if x==y:
    7     print("x and y are equal ")
    8 else:
    9     print("x and y are not equal : x-y=%e " % (x-y))
   10 # x and y are not equal : x-y=1.110223e-16

• The test is true only if $x$ and $y$ are exactly equal in bit pattern.
• Although $x$ and $y$ are equal in exact arithmetic, their values differ by a small, but nonzero, amount.
• When working with floating-point values the question “are $x$ and $y$ equal?” is replaced by “are $x$ and $y$ close?” or, equivalently, “is $x-y$ small enough?”

ii	Truncation Error: i.e., approximate $e^x$ by the cubic power $$P_3(x)=1+\frac{x}{1!}+\frac{x^2}{2!}+\frac{x^3}{3!};~~~~~~e^x=P_3(x)+\sum_{n=4}^\infty \frac{x^n}{n!}$$

• Approximating $e^x$ with the cubic gives an inexact answer. The error is due to truncating the series,
• When to cut series expansion $\Longrightarrow$ be satisfied with an approximation to the exact analytical answer.
• Unlike roundoff, which is controlled by the hardware and the computer language being used, truncation error is under control of the programmer or user.
• Truncation error can be reduced by selecting more accurate discrete approximations. But, it can not be eliminated entirely.

Evaluating the Series for sin(x) (Example py-file: sinser.py)

$$sin(x)=x-\frac{x^3}{3!}+\frac{x^5}{5!}-\frac{x^7}{7!}+\ldots$$

• An efficient implementation of the series uses recursion to avoid overflow in the evaluation of individual terms. If $T_k$ is the $k^{th}$ term ( $k={1,3,5,\ldots}$) then
  $$T_k=\frac{x^2}{k(k-1)}T_{k-2}$$

• Study the effect of the parameters tol and nmax by changing their values (Default values are 5e-9 and 15, respectively).

    1 import numpy as np
    2 def sinser(x,tol,n):
    3     term = x
    4     ssum = term #  Initialize series
    5     print("Series approximation to sin(%f) \n  k      term         ssum" % (x*360/(2*np.pi)))
    6     print("  1  %11.3e  %20.16f " % (term,ssum))
    7     for k in range(3, 2*n-1, 2):
    8         term = -term * x*x/(k*(k-1)) #  Next term in the series
    9         ssum = ssum + term
   10         print("%3d  %11.3e  %30.26f " % (k,term,ssum))
   11         if abs(term/ssum)<tol:
   12             break  # True at convergence
   13     print("Truncation error after %d terms is %g " %  ((k+1)/2,abs(ssum-np.sin(x))))
   14 sinser(np.pi/6,5e-9,10)
   15 print("sin(%f)=%f with numpy library " % (np.pi/6*360/(2*np.pi),np.sin(np.pi/6)))
   16 import math
   17 print("sin(%f)=%f with math libarary" % (math.pi/6*360/(2*math.pi),math.sin(math.pi/6)))

Figure 2.7: Output of sinser.py

Derivative of Sine function. Truncation & Round-off Errors (Example py-file: trunroun.py)

Figure 2.8: Output and Plot of trunroun.py

iii	Propagated Error:

• more subtle (difficult to analyse)
• by propagated we mean an error in the succeeding steps of a process due to an occurrence of an earlier error
• of critical importance
• stable numerical methods; errors made at early points die out as the method continues
• unstable numerical method; does not die out