round-off and truncation error（舍入和截断误差）

# round-off and truncation error
# 舍入和截断误差

1. Round-off errors are due to approximate representation of floating-point number

from math import pi
print(pi)

3.141592653589793

$\pi$=3.141592653589793238462643…

2. subtractive cancellation

$$x=1$$
$$y=1+10^{-15}\sqrt{2}$$

from math import sqrt
x = 1.0
y = 1.0 + 1e-15 * sqrt(2)
dt = 1e-15 * sqrt(2)
dn = y - x
print("dt = ", dt)
print("dn = ", dn)
print("Relative error: ", (dt - dn) / dt)

dt =  1.4142135623730953e-15
dn =  1.3322676295501878e-15
Relative error:  0.05794452478973511

similar accumulation of round-off in addition(large number + small number).

3. Truncation Errors

Truncation errors are created by truncating the math.

Example: MacLaurin Series for exponential function
$$e^x=\sum _{n=0}^{\infty }\frac{x^n}{n!}$$

Now, we only use three terms
$$e^x=\sum _{n=0}^{\infty }\frac{x^n}{n!}\approx 1+x+\frac{x^2}{2!}$$

4. Numerical Differentiation and numerical errors

definition of true derivative

$$f^{{}^{\prime }}(x)=\underset{dx\to 0}{\lim }\frac{f(x+dx)-f(x)}{dx}$$

approximation of derivative(numerical derivative)

$$f^{{}^{\prime }}(x_i)=\frac{f(x_{i+1})-f(x_i)}{x_{i+1}-x_i}$$

5. Truncation error estimate using Taylor series

$$f(x_{i+1})=f(x_i)+f^{{}^{\prime }}(x_i)(x_{i+1}-x_i)+R_1$$

$$f^{{}^{\prime }}(x_i)=\frac{f(x_{i+1})-f(x_i)}{x_{i+1}-x_i}-\frac{R_1}{x_{i+1}-x_i}$$

where $\frac{f(x_{i+1})-f(x_i)}{x_{i+1}-x_i}$ is the approximation term, and $\frac{R_1}{x_{i+1}-x_i}$ is the truncation error.

$$R_n=\frac{f^{n+1}(\xi )}{(n+1)!}{h}^{n+1}=O(h^{n+1})$$

three expressions of numerical derivative:

(1) Forward formula
$$f^{{}^{\prime }}(x_i)=\frac{f(x_{i+1})-f(x_i)}{x_{i+1}-x_i}+O(h)$$
(2) Backward formula
$$f^{{}^{\prime }}(x_i)=\frac{f(x_i)-f(x_{i-1})}{x_i-x_{i-1}}+O(h)$$
(3) Centered formula
$$f^{{}^{\prime }}(x_i)=\frac{f(x_{i+1})-f(x_{i-1})}{x_{i+1}-x_{i-1}}+O(h^2)$$

so, centered formula is more accurate than forward and backward formula

6. Error propagation and condition number

$$f(x)=f(x^{\ast })+f^{{}^{\prime }}(x^{\ast })(x-x^{\ast })$$

$$\frac{f(x)-f(x^{\ast })}{f(x^{\ast })}=\frac{f^{{}^{\prime }}(x^{\ast })(x-x^{\ast })}{f(x^{\ast })}$$

$$\frac{f(x)-f(x^{\ast })}{f(x^{\ast })}=\frac{x^{\ast }{f}^{{}^{\prime }}(x^{\ast })}{f(x^{\ast })}\frac{x-x^{\ast }}{x^{\ast }}$$

where $\frac{x^{\ast }{f}^{{}^{\prime }}(x^{\ast })}{f(x^{\ast })}$ is called condition number.

$$\frac{f(x)-f(x^{\ast })}{f(x^{\ast })}=Condition\_number\times \frac{x-x^{\ast }}{x^{\ast }}$$

Small condition number = error decreases.

Large condition number = ill-conditioned.