最小二乘估计，矩阵方法求解

$$\begin{array}{rl}D& =\{(x_1,y_1),(x_2,y_2),...(x_N,y_N)\}\\ x_i& \in {\mathbb{R}}^p(x_i是一个列向量),y_i\in \mathbb{R},i=1,2,...,N\\ X& ={\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_N\end{array}\right]}^T\\ & =\left[\begin{array}{c}{x}_1^T\\ x_2^T\\ ⋮\\ x_N^T\end{array}\right]\\ & ={\left[\begin{array}{cccc}{x}_{11}& x_{12}& \cdots & x_{1P}\\ x_{21}& x_{22}& \cdots & x_{2P}\\ ⋮& ⋮& \ddots & ⋮\\ x_{N1}& x_{N2}& \cdots & x_{NP}\end{array}\right]}_{N\times P}\\ Y& ={\left[\begin{array}{c}{y}_1^T\\ y_2^T\\ ⋮\\ y_N^T\end{array}\right]}_{N\times 1}\\ 我们& 拟合的直线的方程：\\ f(W)& =W^{T}x+b\\ 为了& 方便计算，我们将b转变成w_0{x}_0,x_0等于1，w_0等于b\\ f(W)& =W^{T}x\\ W& ={\left[\begin{array}{c}{w}_1^T\\ w_2^T\\ ⋮\\ w_P^T\end{array}\right]}_{P\times 1}\\ & 最小二乘法估计：\\ L(W)& =\sum _{i=1}^N||W^T{x}_i-y_i|{|}^2\\ & =\sum _{i=1}^N(W^T{x}_i-y_i{)}^2\\ & =\left[\begin{array}{cccc}{W}^T{x}_1-y_1& W^T{x}_2-y_2& \cdots & W^T{x}_N-y_N\end{array}\right]\left[\begin{array}{c}{W}^T{x}_1-y_1\\ W^T{x}_2-y_2\\ ⋮\\ W^T{x}_N-y_N\end{array}\right]\\ & =(\left[\begin{array}{cccc}{W}^T{x}_1& W^T{x}_2& \cdots & W^T{x}_N\end{array}\right]-\left[\begin{array}{cccc}{y}_1& y_2& \cdots & y_N\end{array}\right])\left[\begin{array}{c}{W}^T{x}_1-y_1\\ W^T{x}_2-y_2\\ ⋮\\ W^T{x}_N-y_N\end{array}\right]\\ & =(W^T\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_N\end{array}\right]-\left[\begin{array}{cccc}{y}_1& y_2& \cdots & y_N\end{array}\right])\left[\begin{array}{c}{W}^T{x}_1-y_1\\ W^T{x}_2-y_2\\ ⋮\\ W^T{x}_N-y_N\end{array}\right]\\ & =(W^T{X}^T-Y^T)\left[\begin{array}{c}{W}^T{x}_1-y_1\\ W^T{x}_2-y_2\\ ⋮\\ W^T{x}_N-y_N\end{array}\right]\\ & =(W^T{X}^T-Y^T)(XW-Y)\\ & =W^T{X}^{T}XW-W^T{X}^{T}Y-Y^{T}XW+Y^{T}Y\\ & =W^T{X}^{T}XW-2W^T{X}^{T}Y+Y^{T}Y\\ & \\ \hat{W}& =argminL(W)\\ \frac{\partial L(W)}{\partial W}& =2X^{T}XW-2X^{T}Y=0\\ \therefore X^{T}XW& =X^{T}Y\\ W& =(X^{T}X{)}^{-1}{X}^{T}Y\end{array}$$