矩阵求导推导

矩阵求导推导

1、定义两个矩阵相乘： $$A\cdot B=C$$
2、考虑loss函数：$$Loss=\sum _i^m\sum _j^n(C_{ij}-p{)}^2$$
3、考虑C的每一项导数： $$▽C_{ij}=\frac{\partial L}{\partial C_{ij}}$$
4、定义ABC都为2*2的矩阵，G为Loss对C的导数：$$A=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]B=\left[\begin{array}{cc}e& f\\ g& h\end{array}\right]C=\left[\begin{array}{cc}i& j\\ k& k\end{array}\right]G=\left[\begin{array}{cc}\frac{\partial L}{\partial i}& \frac{\partial L}{\partial j}\\ \frac{\partial L}{\partial k}& \frac{\partial L}{\partial l}\end{array}\right]=\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]$$
5、将$A\cdot B=C$展开得到：
$$C=\left[\begin{array}{ccc}i=ae+bg& & j=af+bh\\ k=ce+dg& & l=cf+dh\end{array}\right]$$
6、L对A中的每一项导数为：
$$▽A_{ij}=\frac{\partial L}{\partial A_{ij}}$$
展开式子如下
$$\frac{\partial L}{\partial a}=\frac{\partial L}{\partial i}\ast \frac{\partial i}{\partial a}+\frac{\partial L}{\partial j}\ast \frac{\partial j}{\partial a}$$
$$\frac{\partial L}{\partial b}=\frac{\partial L}{\partial i}\ast \frac{\partial i}{\partial b}+\frac{\partial L}{\partial j}\ast \frac{\partial j}{\partial b}$$
$$\frac{\partial L}{\partial c}=\frac{\partial L}{\partial k}\ast \frac{\partial k}{\partial c}+\frac{\partial L}{\partial l}\ast \frac{\partial l}{\partial c}$$
$$\frac{\partial L}{\partial d}=\frac{\partial L}{\partial k}\ast \frac{\partial k}{\partial d}+\frac{\partial L}{\partial l}\ast \frac{\partial l}{\partial d}$$
即：
$$\frac{\partial L}{\partial a}=we+xf$$
$$\frac{\partial L}{\partial b}=wg+xh$$
$$\frac{\partial L}{\partial c}=ye+zf$$
$$\frac{\partial L}{\partial d}=yg+zh$$
7、所以Loss对A的导数为：$$▽A=\left[\begin{array}{cc}we+xf& wg+xh\\ ye+zf& yg+zh\end{array}\right]▽A=\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]\left[\begin{array}{cc}e& g\\ f& h\end{array}\right]$$
$$▽A=G\cdot B^T$$
8、同理Loss对B的导数为：
$$\begin{gathered} \frac{\partial L}{\partial e}=wa+yc \\ \frac{\partial L}{\partial f}=xa+zc \\ \frac{\partial L}{\partial g}=wb+yb \\ \frac{\partial L}{\partial h}=xb+zd \end{gathered}$$
$$▽B=\left[\begin{array}{cc}wa+yc& xa+zc\\ wb+yd& xb+zd\end{array}\right]=\left[\begin{array}{cc}a& c\\ b& d\end{array}\right]\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]$$
即：$$▽B=A^T\cdot G$$
注：矩阵求导可以应用到BP神经网络的反向求导过程中