深度学习与PyTorch笔记14

链式法则

Derivative Rules

Rules	Function	Derivative
sum	$f+g$	$f^{{}^{\prime }}+g^{{}^{\prime }}$
difference	$f-g$	$f^{{}^{\prime }}-g^{{}^{\prime }}$
product	$fg$	$f{g}^{{}^{\prime }}+f^{{}^{\prime }}g$
quotient	$f/g$	$(f{g}^{{}^{\prime }}+f^{{}^{\prime }}g)/g^2$
chain(using$\frac{d}{dx}$)		$\frac{dy}{dx}=\frac{dy}{du}\frac{du}{dx}$

Chain rule

$$\frac{\partial y}{\partial x}=\frac{\partial y}{\partial u}\frac{\partial u}{\partial x}$$
$$y_1=x{w}_1+b_1$$
$$y_2=y_1{w}_2+b_2$$
$y_1$为中间变量
$$\frac{\partial y_2}{\partial w_1}=\frac{\partial f(y_1)}{\partial w_1}=\frac{\partial f(y_1)}{\partial y_1}\frac{\partial y_1}{\partial w_1}=w_2\ast x$$

从复杂到简单。