AutoCV第九课：ML基础

矩阵运算

前言

手写AI推出的全新保姆级从零手写自动驾驶CV课程，链接。记录下个人学习笔记，仅供自己参考。

本次课程主要学习矩阵运算的基础，考虑使用矩阵来表达多个线性回归模型。

课程大纲可看下面的思维导图。

1. 矩阵乘法和求导

先回忆下矩阵相关知识

定义矩阵乘法：
$$\left\{\begin{array}{cc}a& b\\ d& e\end{array}\right\}\times \left\{\begin{array}{cc}1& 3\\ 2& 4\end{array}\right\}=\left\{\begin{array}{cc}a1+b2& a3+b4\\ d1+e2& d3+e4\end{array}\right\}$$
记法：C[r][c] = 乘加(A中取 r 行，B中取 c 列)

图1 矩阵乘法示例

参考：https://www.cnblogs.com/ljy-endl/p/11411665.html

矩阵求导：

对于 $A\cdot B=C$ 定义 $L$ 是关于 $C$ 的损失函数

设 $G=\frac{\partial L}{\partial C}$ 若直接 $C$ 对 $A$ 求导，则 $G$ 定义为 $C$ 大小的全 1 矩阵，则有：
$$\frac{\partial L}{\partial A}=G\cdot B^T\frac{\partial L}{\partial B}=A^T\cdot G$$
矩阵求导推导：

1. 考虑矩阵乘法 $$A\cdot B=C$$

2. 考虑 Loss 函数 $$L=\sum _i^m\sum _j^n(C_{ij}-p{)}^2$$

3. 考虑 $C$ 的每一项导数 $$▽C_{ij}=\frac{\partial L}{\partial C_{ij}}$$

4. 考虑 $ABC$ 都为 2x2 矩阵时，定义 $G$ 为 $L$ 对 $C$ 的导数
   $$A=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]B=\left[\begin{array}{cc}e& f\\ g& h\end{array}\right]C=\left[\begin{array}{cc}i& j\\ k& l\end{array}\right]G=\frac{\partial L}{\partial C}=\left[\begin{array}{cc}\frac{\partial L}{\partial i}& \frac{\partial L}{\partial j}\\ \frac{\partial L}{\partial k}& \frac{\partial L}{\partial l}\end{array}\right]=\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]$$

5. 展开左边 $A\cdot B$

$$C=\left[\begin{array}{cc}i=ae+bg& j=af+bh\\ k=ce+dg& l=cf+dh\end{array}\right]$$

6. $L$ 对于每一个 $A$ 的导数
   $$▽A_{ij}=\frac{\partial L}{\partial A_{ij}}$$

   $$\begin{array}{rl}\frac{\partial L}{\partial a}& =\frac{\partial L}{\partial i}\ast \frac{\partial i}{\partial a}+\frac{\partial L}{\partial j}\ast \frac{\partial j}{\partial a}\\ \frac{\partial L}{\partial b}& =\frac{\partial L}{\partial i}\ast \frac{\partial i}{\partial b}+\frac{\partial L}{\partial j}\ast \frac{\partial j}{\partial b}\\ \frac{\partial L}{\partial c}& =\frac{\partial L}{\partial k}\ast \frac{\partial k}{\partial c}+\frac{\partial L}{\partial l}\ast \frac{\partial l}{\partial c}\\ \frac{\partial L}{\partial d}& =\frac{\partial L}{\partial k}\ast \frac{\partial k}{\partial d}+\frac{\partial L}{\partial l}\ast \frac{\partial l}{\partial d}\end{array}$$

   $$\begin{array}{rl}\frac{\partial L}{\partial a}& =we+xf\\ \frac{\partial L}{\partial b}& =wg+xh\\ \frac{\partial L}{\partial c}& =ye+zf\\ \frac{\partial L}{\partial d}& =yg+zh\end{array}$$

7. 因此 $A$ 的导数为

   $$\frac{\partial L}{\partial A}=\left[\begin{array}{cc}we+xf& wg+xh\\ ye+zf& yg+zh\end{array}\right]\frac{\partial L}{\partial A}=\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]\left[\begin{array}{cc}e& g\\ f& h\end{array}\right]$$

   $$\frac{\partial L}{\partial A}=G\cdot B^T$$

8. 同理 $B$ 的导数为
   $$\begin{array}{rl}\frac{\partial L}{\partial e}& =wa+yc\\ \frac{\partial L}{\partial f}& =xa+zc\\ \frac{\partial L}{\partial g}& =wb+yd\\ \frac{\partial L}{\partial h}& =xb+zd\end{array}$$

   $$\frac{\partial L}{\partial B}=\left[\begin{array}{cc}wa+yc& xa+zc\\ wb+yd& xb+zd\end{array}\right]\frac{\partial L}{\partial B}=\left[\begin{array}{cc}a& c\\ b& d\end{array}\right]\left[\begin{array}{cc}w& x\\ y& z\end{array}\right]$$

   $$\frac{\partial L}{\partial B}=A^T\cdot G$$

总结

本次课程学习了矩阵求导相关知识，后续实现多个线性回归或者逻辑逻辑回归模型可以考虑使用矩阵方式来表达