机器学习（西瓜书）之一元线性回归公式推导

一元线性回归公式推导

1.求解偏置b的公式推导思路：由最小二乘法导出损失函数$E(\omega ,b)$—>证明损失函数$E(\omega ,b)$ 是关于$w$和$b$的凸函数—>对损失函数$E(\omega ,b)$关于$b$求一阶偏导数—>令一阶偏导数等于$0$解出$b$

由最小二乘法导出损失函数$E(\omega ,b)$:

$$E_{(w,b)}=\sum _{i=1}^m(y_i-f(x_i){)}^2=\sum _{i=1}^m(y_i-(\omega x_i+b){)}^2=\sum _{i=1}^m(y_i-\omega x_i-b{)}^2\text{\hspace{1em}(p54.3.4)}$$

证明损失函数$E(\omega ,b)$是关于$\omega$和$b$的凸函数—— 求:$A=f_{xx}^{〞}(x,y)$

$$\frac{\partial E_{(w,b)}}{\partial \omega }=\frac{\partial }{\partial \omega }[\sum _{i=1}^m(y_i-\omega x_i-b{)}^2]=\sum _{i=1}^m\frac{\partial }{\partial \omega }(y_i-\omega x_i-b{)}^2=\sum _{i=1}^{m}2\cdot (y_i-\omega x_i-b)\cdot （-x_i）=2(\omega \sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i)$$上式即为即为（3.5)

$$A=f_{xx}^{〞}(x,y)=\frac{{\partial }^2{E}_{(\omega ,b)}}{\partial {\omega }^2}=\frac{\partial }{\partial \omega }(\sum _{i=1}^m\frac{\partial E_{(w,b)}}{\partial \omega })=\frac{\partial }{\partial \omega }[2(\omega \sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i)]=\frac{\partial }{\partial \omega }[2\omega \sum _{i=1}^m{x}_i^2]=2\sum _{i=1}^m{x}_i^2$$

求:$B=f_{xy}^{〞}(x,y)$

$$B=f_{xy}^{〞}(x,y)=\frac{{\partial }^2{E}_{(\omega ,b)}}{\partial \omega \partial b}=\frac{\partial }{\partial b}(\frac{\partial E_{(w,b)}}{\partial \omega })=\frac{\partial }{\partial b}[2(\omega \sum _{i=1}^m{x}_i^2-\sum _{i=1}^m(y_i-b)x_i)]=\frac{\partial }{\partial b}[-2\sum _{i=1}^m(y_i-b)x_i)]=2\sum _{i=1}^m{x}_i$$

求:$C=f_{yy}^{〞}(x,y)$

$$\frac{\partial E_{(w,b)}}{\partial b}=\frac{\partial }{\partial b}[\sum _{i=1}^m(y_i-\omega x_i-b{)}^2]=\sum _{i=1}^m\frac{\partial }{\partial b}(y_i-\omega x_i-b{)}^2=\sum _{i=1}^{m}2\cdot (y_i-\omega x_i-b)\cdot （-1）=2(mb-\sum _{i=1}^m(y_i-\omega x_i))$$上式即为即为（3.6)

所以

也即损失函数$E(\omega ,b)$ 是关于$\omega$和$b$的凸函数得证

对损失函数$E(\omega ,b)$ 关于b求一阶偏导数：

令一阶偏导数等于 0 解出b:

2.求解权重$\omega ,$的公式推导思路:对损失函数$E(\omega ,b)$关于$\omega$求一阶偏导数—>令一阶偏导数等于$0$解出$\omega$

令一阶偏导数等于 0 解出$\omega$：

3.将$\omega$向量化:

数学知识—二元函数判断凹凸性: