Newton Method in One Dimension

I : Question one : What is nonlinearity?

非线性是指两个变量间的数学关系，不是直线，而是曲线、曲面、或不确定的属性，是不成简单比例（即线性）关系的。

• 1: Prototype scalar equation: Find x∗ ∈ R such that
  F(x∗) = 0
  for a given function F that is differentiable with
  derivative F′(x) ̸= constant (non-constant derivative)

• 2: Prototype system of equations: Find ⃗x∗ ∈ Rn such that
  F⃗ ( ⃗x ∗ ) = ⃗0
  with Jacobian matrix F⃗′(⃗x) ̸= constant (non-constant Jacobian)

II: Question two : Why do we need numerical methods to solve nonlinear equations?

非线性方程，就是因变量与自变量之间的关系不是线性的关系，这类方程很多，例如平方关系、对数关系、指数关系、三角函数关系等等。

现实生活中大部分事情都是nonlinearity, nonlinearity可以很好的描述的生活中大部分事物。

III: Example of Nonlinear Equation:

• Polynomial(多项式) equations :

  
  
  截屏2021-12-29 10.52.34.png

(common approximations(近似值) for other nonlinear equations)

• Trigonometric equations
  sin(x)+2cos(x)=0etc.

• Many physics equations:
  n1 sin θ1 = n2 sin θ2, E = mc2 etc.

• Neural Network

  
  
  Neural Network.png

IV: Newton Method

It is rare for nonlinear equations to have a closed form solution that can be found through algebraic manipulation so people must require (approximate) solution.

• People have the differentiable function F(x)
  and its derivative F′(x).

• People have one initial point x0 that is close to the
  unknown zero x∗ such that F(x∗) = 0.

• People look for a sequence of iterates x0,x1,...such that

  
  
  iterates method.png

• 牛顿法具体思路：

  
  
  牛顿法具体思路

• Derivation(推导) of Newton’ Method

  
  
  牛顿法的推演

• Pros and cons（优点和缺点）
  Performance
  Fast (quadratic(二次方) convergence rate) (二次收敛率)
  Not robust（不稳健性）

  Other issues
  Requires the derivative function
  Requires a “good” initial guess