机器学习 【逻辑回归LR算法】 公式推导计算+详细过程 （入门必备）

• Logistic回归是”广义线性模型“，用于解决分类问题。是在线性回归的基础上加入了非线性映射sigmoid函数

线性回归公式
$$h_{\theta }(x)={\theta }_0{x}_0+{\theta }_1{x}_1+{\theta }_2{x}_2+...+{\theta }_n{x}_n$$

线性回归向量形式
$$h_{\theta }(x)={\theta }^{T}x$$

sigmoid函数

$$g(z)=\frac{1}{1+e^{-z}}$$

• 其中， $ z = h_\theta(x) = \theta^Tx $

$$g(z)=\frac{1}{1+e^{-{\theta }^{T}x}}$$

逻辑回归损失函数

$$J(\theta )=-\frac{1}{m}\sum _{i=0}^m[y^{(i)}log{h}_{\theta }(x^{(i)})+(1-y^{(i)})log(1-h_{\theta }(x^{(i)}))]$$

梯度下降更新$\theta$
$${\theta }_j:={\theta }_j-\alpha \frac{\partial }{\partial {\theta }_j}J(\theta )$$

sigmoid函数求导
$$g(x)=\frac{1}{1+e^{-x}}$$

$$g^{\prime }(x)=g(x)(1-g(x))(5.8)$$

求偏导数推导
$$\frac{\partial J(\theta )}{\partial {\theta }_j}=-\frac{1}{m}\sum _{i=0}^m[y^{(i)}\frac{1}{h_{\theta }(x^{(i)})}\ast \frac{\partial h_{\theta }(x^{(i)})}{\partial {\theta }_j}-(1-y^{(i)})\ast \frac{1}{1-h_{\theta }(x^{(i)})}\ast \frac{\partial h_{\theta }(x^{(i)})}{\partial {\theta }_j}]$$

$$=-\frac{1}{m}\sum _{i=0}^m[y^{(i)}\frac{1}{g({\theta }^T{x}^{(i)})}-(1-y^{(i)})\frac{1}{1-g({\theta }^T{x}^{(i)})}]\ast \frac{\partial g({\theta }^T{x}^{(i)})}{\partial {\theta }_j}$$

$$=-\frac{1}{m}\sum _{i=0}^m[y^{(i)}\frac{1}{g({\theta }^T{x}^{(i)})}-(1-y^{(i)})\frac{1}{1-g({\theta }^T{x}^{(i)})}]\ast g({\theta }^T{x}^{(i)})(1-g({\theta }^T{x}^{(i)})x_j^{(i)}$$

$$=-\frac{1}{m}\sum _{i=0}^m[y^{(i)}(1-g({\theta }^T{x}^{(i)})-(1-y^{(i)})g({\theta }^T{x}^{(i)})]x_j^{(i)}$$

$$=-\frac{1}{m}\sum _{i=0}^m(y^{(i)}-g({\theta }^T{x}^{(i)}))x_j^{(i)}$$

$$=\frac{1}{m}\sum _{i=0}^m（h_{\theta }(x^{(i)})-y^{(i)}）x_j^{(i)}$$