《深度学习入门》误差反向传播中批版本Affine层关于偏置db的反向传播的个人理解

误差反向传播中批版本Affine层关于偏置db的反向传播的证明过程如下

假定

 b = [b_{1},b_{2}]   偏置

y = x*w+b   Affine层关系式

L = \sum_{N}*\frac{1}{2}\sum_{k}(y_{k}-t_{k})^{2}    平方和损失函数(也可以换成交叉熵损失函数)

 

1.只有一个数据 N=0

所以

\frac{\partial L}{\partial b}=\frac{\partial L}{\partial y}*\frac{\partial y}{\partial b}

=\sum_{ka}(y_{k}-t_{k})*\frac{\partial (x*w+b)}{\partial b} =\sum_{k}(y_{k}-t_{k})*1 =[\frac{\partial L}{\partial b_{1}},\frac{\partial L}{\partial b_{2}}]

2.有多个数据 N!=0

所以

\frac{\partial L}{\partial b}=\frac{\partial L}{\partial y}*\frac{\partial y}{\partial b} 

=\sum_{N}\sum_{k}(y_{k}-t_{k})*\frac{\partial (x*w+b)}{\partial b} =\sum_{N}\sum_{k}(y_{k}-t_{k})*1 =\sum_{N}\begin{bmatrix} \frac{\partial L}{\partial b_{11}} & \frac{\partial L}{\partial b_{12}}\\ .& . \\ \frac{\partial L}{\partial b_{N1}}& \frac{\partial L}{\partial b_{N2}} \end{bmatrix}

由于要对N求和,即按列求和

 所以证明了如果时多个数据,则 

\frac{\partial L}{\partial b_{1}}=\sum_{n=1}^{N}\frac{\partial L}{\partial b_{n1}}, \frac{\partial L}{\partial b_{2}}=\sum_{n=1}^{N}\frac{\partial L}{\partial b_{n2}}

所以用Python表示为

import numpy as np
db = np.sum(dy,axis=0)

 

你可能感兴趣的:(深度学习,numpy)