BP推导，普通版加矩阵版

前向传播用来计算整个卷积过程的输出值以及相应的误差值。反向传播则是想把误差值平摊至每个参数上，使得最终的输出值越来越逼近标签值。

普通版

以一个两层网络为例，绘图如下

前向传播过程

如下：

激活函数为sigmoid$$\delta (x)=\frac{1}{1+e^{-x}}$$
第一层
$$\begin{gathered} z_1=w_{11}{x}_1+w_{13}{x}_2+b_1 \\ {a}_1=\delta (z_1) \end{gathered}$$
$$\begin{gathered} z_2=w_{12}{x}_1+w_{14}{x}_2+b_2 \\ {a}_2=\delta (z_2) \end{gathered}$$
第二层
$$\begin{gathered} z_3=w_{21}{a}_1+w_{22}{a}_2+b_3 \\ {a}_3=\delta (z_3) \end{gathered}$$

反向传播过程

已知激活函数为sigmoid$$\delta (x)=\frac{1}{1+e^{-x}}$$
该激活函数的导函数为$${\delta }^{{}^{\prime }}(x)=\delta (x)(1-\delta (x))$$

损失函数定义为MSE均方值损失$$E=\frac{1}{2}(y-a_3{)}^2$$
根据梯度下降法，令损失函数对参数求导，并更新$$w=w-\eta \cdot \Delta w$$

具体如下：

第二层参数
$$\begin{gathered} \Delta w_{21}=\frac{\partial E}{\partial w_{21}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial w_{21}} \\ =-(y-a_3)\cdot a_3(1-a_3)\cdot a_1 \\ =g_{21}\cdot a_1 \end{gathered}$$
$$\begin{gathered} \Delta w_{22}=\frac{\partial E}{\partial w_{22}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial w_{22}} \\ =g_{21}\cdot a_2 \end{gathered}$$
$$\begin{gathered} \Delta b_3=\frac{\partial E}{\partial b_3}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial b_3} \\ =g_{21} \end{gathered}$$
记 $g_{21}=-(y-a_3)\cdot a_3(1-a_3)$

第一层参数

$$\begin{gathered} \Delta w_{11}=\frac{\partial E}{\partial w_{11}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_1}\cdot \frac{\partial a_1}{\partial z_1}\cdot \frac{\partial z_1}{\partial w_{11}} \\ =g_{21}\cdot w_{21}{a}_1(1-a_1)\cdot x_1 \\ =g_{21}\cdot g_{11}\cdot x_1 \end{gathered}$$
记 $g_{11}=w_{21}{a}_1(1-a_1)$
$$\begin{gathered} \Delta w_{13}=\frac{\partial E}{\partial w_{13}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_1}\cdot \frac{\partial a_1}{\partial z_1}\cdot \frac{\partial z_1}{\partial w_{13}} \\ =g_{21}\cdot w_{21}{a}_1(1-a_1)\cdot x_2 \\ =g_{21}\cdot g_{11}\cdot x_2 \end{gathered}$$
$$\begin{gathered} \Delta b_1=\frac{\partial E}{\partial b_1}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_1}\cdot \frac{\partial a_1}{\partial z_1}\cdot \frac{\partial z_1}{\partial b_1} \\ =g_{21}\cdot g_{11} \end{gathered}$$
$$\begin{gathered} \Delta w_{12}=\frac{\partial E}{\partial w_{12}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_2}\cdot \frac{\partial a_2}{\partial z_2}\cdot \frac{\partial z_2}{\partial w_{12}} \\ =g_{21}\cdot w_{22}{a}_2(1-a_2)\cdot x_1 \\ =g_{21}\cdot g_{12}\cdot x_1 \end{gathered}$$
记 $g_{12}=w_{22}{a}_2(1-a_2)$
$$\begin{gathered} \Delta w_{14}=\frac{\partial E}{\partial w_{14}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_2}\cdot \frac{\partial a_2}{\partial z_2}\cdot \frac{\partial z_2}{\partial w_{14}} \\ =g_{21}\cdot w_{22}{a}_2(1-a_2)\cdot x_2 \\ =g_{21}\cdot g_{12}\cdot x_2 \end{gathered}$$
$$\begin{gathered} \Delta b_2=\frac{\partial E}{\partial w_{12}}=\frac{\partial E}{\partial a_3}\cdot \frac{\partial a_3}{\partial z_3}\cdot \frac{\partial z_3}{\partial a_2}\cdot \frac{\partial a_2}{\partial z_2}\cdot \frac{\partial z_2}{\partial b_2} \\ =g_{21}\cdot g_{12} \end{gathered}$$

累死了。。。。。够详细了吧，绝对能看懂。。。敲得咱脑瓜子疼，看懂了三连一下呗。。。。

矩阵版

有时间再写，晚安，希望可以梦到静静，疫情封城可真难熬啊，早点解封吧