最小二乘法矩阵推导

已知方程组
$$Ax=b\text{\hspace{1em}(1)}$$
定义能量函数
$$E=\sum {\left(Ax-b\right)}^2=\Vert AX-b{\Vert }_2^2\text{\hspace{1em}(2)}$$
要使能量函数E最小，即
$$\underset{x\in R^n}{\min }E\text{\hspace{1em}(3)}$$
对能量函数E求偏导
$$\begin{gathered} \frac{\partial E}{\partial x}=\frac{\partial \Vert Ax-b{\Vert }_2^2}{\partial x} \\ =\frac{\partial {\left(Ax-b\right)}^T\left(Ax-b\right)}{\partial x} \\ =\frac{\partial \left(x^T{A}^{T}Ax-x^T{A}^{T}b-b^{T}Ax+b^{T}b\right)}{\partial x}\text{\hspace{1em}(4)} \end{gathered}$$
因为
$$\frac{\partial x^{T}A}{\partial x}=\frac{\partial A^{T}x}{\partial x}=A\text{\hspace{1em}(5)}$$

$$\frac{\partial x^{T}Ax}{\partial x}=Ax+A^{T}x\text{\hspace{1em}(6)}$$

所以
$$\frac{\partial E}{\partial x}=2A^{T}Ax-2A^{T}b\text{\hspace{1em}(7)}$$
令
$$\frac{\partial E}{\partial x}=0\text{\hspace{1em}(8)}$$
即
$$2A^{T}Ax-2A^{T}b=0\text{\hspace{1em}(9)}$$
因此
$$x={\left(A^{T}A\right)}^{-1}{A}^{T}b$$