Softmax函数和交叉熵Cross-entropy以及KL散度求导

参考链接：https://blog.csdn.net/qian99/article/details/78046329

交叉熵cross-entropy

对一个分类神经网络$f$，输出为$z=f(x;\theta ),z=[z_0,z_1,\cdots ,z_{C-1}]$,$z$为logits，其中类别数量为$C$,$y$为$x$的one-hot标签。通过softmax归一化来得到概率：
$$p_i=\frac{\exp z_i}{{\sum }_j\exp z_j}$$
负交叉熵误差为：
$$\mathcal{L}=-\sum _i{y}_i\log p_i$$
误差对于概率的梯度为：
$$\frac{\partial \mathcal{L}}{\partial p_i}=-y_i\frac{1}{p_i}$$
紧接着计算$\frac{\partial p_i}{\partial z_k},k=0,1,...,C-1$:
（1）当$k=i$时，
$$\begin{gathered} \frac{\partial p_i}{\partial z_i}=\frac{\partial (\frac{\exp z_i}{{\sum }_j\exp z_j})}{\partial z_i}=\frac{\exp z_i{\sum }_j\exp z_j-(\exp z_i{)}^2}{({\sum }_j\exp z_j{)}^2} \\ =(\frac{\exp z_i}{{\sum }_j\exp z_j})(1-\frac{\exp z_i}{{\sum }_j\exp z_j})=p_i(1-p_i) \end{gathered}$$

（2）当$k\ne i$时，
$$\frac{\partial p_i}{\partial z_k}=\frac{\partial (\frac{\exp z_i}{{\sum }_j\exp z_j})}{\partial z_k}=\frac{-\exp z_i\exp z_k}{({\sum }_j\exp z_j{)}^2}=-p_i{p}_k$$
根据求导的链式法则：
$$\begin{gathered} \frac{\partial \mathcal{L}}{\partial z_k}=\sum _j(\frac{\partial \mathcal{L}}{\partial p_j}\frac{\partial p_j}{\partial z_k}) \\ =\sum _{j=/k}(\frac{\partial \mathcal{L}}{\partial p_j}\frac{\partial p_j}{\partial z_k})+(\frac{\partial \mathcal{L}}{\partial p_k}\frac{\partial p_k}{\partial z_k}) \\ =\sum _{j=/k}(-y_j\frac{1}{p_j}\ast -p_j{p}_k)+(-y_k\frac{1}{p_k}\ast p_k(1-p_k)) \\ =\sum _{j=/k}(y_j{p}_k)-y_k+y_k{p}_k \\ =p_k\sum _j{y}_j-y_k \end{gathered}$$
因为$y$为one-hot编码，所以${\sum }_j{y}_j=1$,i.e.,
$$\frac{\partial \mathcal{L}}{\partial z_k}=p_k-y_k$$

相对熵KL散度

预测的概率分布$p$,真实概率分布为$q$，KL的散度为：
$$\mathcal{L}=KL(q||p)=\sum _k{q}_c\log \frac{q_k}{p_k}$$
求解对概率$p_k$的梯度
$$\frac{\partial \mathcal{L}}{\partial p_k}=-\frac{q_k}{p_k}$$
求解对logits $z_k$的梯度:
$$\begin{gathered} \frac{\partial \mathcal{L}}{\partial z_c}=\sum _j(\frac{\partial \mathcal{L}}{\partial p_j}\frac{\partial p_j}{\partial z_k}) \\ =\sum _{j=/k}(\frac{\partial \mathcal{L}}{\partial p_j}\frac{\partial p_j}{\partial z_k})+(\frac{\partial \mathcal{L}}{\partial p_k}\frac{\partial p_k}{\partial z_k}) \\ =\sum _{j=/k}(-\frac{q_j}{p_j}\ast -p_j{p}_k)+(-\frac{q_k}{p_k}\ast p_k(1-p_k)) \\ =\sum _{j=/k}(q_j{p}_k)+q_k{p}_k-q_k \\ =\sum _j{q}_j{p}_k-q_k \end{gathered}$$