Differentiation with a Computer

• Forward-difference approximation. A computer can calculate an approximation to the derivative, if a very small value is used for $\Delta x$.
  $$\boxed{\frac{df}{dx} \bigg\vert_{x=a}=\frac{f(a+\Delta x)-f(a)}{\Delta x}}$$
  • Recalculating with smaller and smaller values of $x$ starting from an initial value.
  • What happens if the value is not small enough?
  • We should expect to find an optimal value for $x$.
  • Because round-off errors in the numerator will become great as $x$ approaches zero.
• When we try this for
  $$f(x)=e^xsin(x)$$
  at $x = 1.9$. The analytical answer is 4.1653826.

Apply Forward-difference approximation to $f(x)=e^x sin(x)$. (Example py-file: myforwardderivative.py)

Table 4.1: Forward-difference approximation for $f(x)=e^x sin(x)$.

Figure 4.1: Forward-difference approximation for $f(x)=e^x sin(x)$.

• Starting with $\Delta x=0.05$ and halving $\Delta x$ each time. Table gives the results.
• We find that the errors of the approximation decrease as $\Delta x$ is reduced until about $\Delta x = 0.05/2097152$.
• Notice that each successive error is about 1/2 of the previous error as $\Delta x$ is halved until $\Delta x$ gets quite small, at which time round off affects the ratio.
• At values for $\Delta x$ smaller than $0.05/2097152$, the error of the approximation increases due to round off.
• In effect, the best value for $\Delta x$ is when the effects of round-off and truncation errors are balanced.
• If a backward-difference approximation is used; similar results are obtained.
• Backward-difference approximation.
  $$\boxed{\frac{df}{dx}\bigg\vert_{x=a}=\frac{f(a)-f(a-\Delta x)}{\Delta x}}$$

Apply Backward-difference approximation to $f(x)=e^x sin(x)$. (Example py-file: mybackwardderivative.py)

Table 4.2: Backward-difference approximation for $f(x)=e^x sin(x)$.

Figure 4.2: Backward-difference approximation for $f(x)=e^x sin(x)$.

• It is not by chance that the errors are about halved each time for both forward- and backward-difference approximations.
• Look at this Taylor series where we have used $h$ for $\Delta x$:
  $$f(x+h)=f(x)+f'(x)*h +f''(\xi)*h^2/2$$

• Where the last term is the error term. The value of $\xi$ is at some point between $x$ and $x + h$.
• If we solve this equation for $f'(x)$, we get

  $$f'(x)=\frac{f(x+h)-f(x)}{h}-f''(\xi)*\frac{h}{2}$$ (4.2)

• If we repeat this but begin with the Taylor series for $f(x - h)$, it turns out that

  $$f'(x)=\frac{f(x)-f(x-h)}{h}+f''(\zeta)*\frac{h}{2}$$ (4.3)

• Where $\zeta$ is between $x$ and $x - h$.
• The two error terms of Eqs. 4.2 and 4.3 are not identical though both are $O(h)$.

• If we add Eqs. 4.2 and 4.3, then divide by 2, we get the central-difference approximation to the derivative:

  $$f'(x)=\frac{f(x+h)-f(x-h)}{2h}-f'''(\xi)*\frac{h^2}{6}$$

• We had to extend the two Taylor series by an additional term to get the error because the $f''(x)$ terms cancel.
• This shows that using a central-difference approximation is a much preferred way to estimate the derivative.
• Even though we use the same number of computations of the function at each step,
• We approach the answer much more rapidly.
  $$\boxed{\frac{df}{dx}\bigg\vert_{x=a}=\frac{f(x+h)-f(x-h)}{2h}}$$

Apply Central-difference approximation to $f(x)=e^x sin(x)$. (Example py-file: mycentralderivative.py)

Table 4.3: Central-difference approximation for $f(x)=e^x sin(x)$.

Figure 4.3: Central-difference approximation for $f(x)=e^x sin(x)$.

Apply Forward-difference approximation to Simple Harmonic Motion. (Example py-file: shm.py)

Figure 4.4: Time change of position and velocity in motion under the force F=-kx.