二元交叉熵损失梯度推导

二元交叉熵损失（logistic 损失）定义如下：
$$L_{\text{logistic}}(\hat{y},y)=-ylog\hat{y}-(1-y)log(1-\hat{y})$$
其中
$y\in \{0,1\}$, $\hat{y}=\sigma (\bar{y})$, $\sigma (\bar{y})=\frac{1}{1+e^{-\bar{y}}}$, $\frac{\partial \sigma }{\partial \bar{y}}=\sigma (1-\sigma )$，
$\bar{y}=\mathbf{w}\cdot \mathbf{x}+b=\sum _j{w}_j{x}_j+b$, $\frac{\partial \bar{y}}{\partial w_j}=x_j$
样本$\mathbf{x}=(x_1,\cdots ,x_j,\cdots ,x_n)$共包含n个特征，权重向量$\mathbf{w}=(w_1,\cdots ,w_j,\cdots ,w_n)$共包含n个权重，与特征一一对应，则
$$\begin{array}{rl}\frac{\partial L}{\partial w_j}& =-y\frac{1}{\sigma }\frac{\partial \sigma }{\partial \bar{y}}\frac{\partial \bar{y}}{\partial w_j}-(1-y)\frac{-1}{1-\sigma }\frac{\partial \sigma }{\partial \bar{y}}\frac{\partial \bar{y}}{\partial w_j}\\ & =\frac{\sigma -y}{\sigma (1-\sigma )}\frac{\partial \sigma }{\partial \bar{y}}\frac{\partial \bar{y}}{\partial w_j}\\ & =\frac{\sigma -y}{\sigma (1-\sigma )}\sigma (1-\sigma )x_j\\ & =(\sigma -y)x_j\\ & =[\sigma (\mathbf{w}\cdot \mathbf{x}+b)-y]x_j\end{array}$$