Logistic Regression推导过程

Logistic Regression推导过程

逻辑回归选用Sigmoid函数作为预测函数


概率函数形式如下

似然函数形式如下

便于计算,对数似然函数:

损失函数

对J取偏导,步骤如下
\begin{align} \frac{\partial}{\partial\theta_j}J(\theta) &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)}\frac{1}{h_\theta(x^{(i)})} \frac{\partial}{\partial\theta_j}h_\theta(x^{(i)})-(1-y^{(i)})\frac{1}{1-h_\theta(x^{(i)})} \frac{\partial}{\partial\theta_j} h_\theta(x^{(i)}) \right) \\ &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)}\frac{1}{g(\theta^Tx^{(i)})} - (1-y^{(i)})\frac{1}{1-g(\theta^Tx^{(i)})} \right) \frac{\partial}{\partial\theta_j}g(\theta^Tx^{(i)}) \\ &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)}\frac{1}{g(\theta^Tx^{(i)})} - (1-y^{(i)})\frac{1}{1-g(\theta^Tx^{(i)})} \right) g(\theta^Tx^{(i)}) \left(1-g(\theta^Tx^{(i)})\right) \frac{\partial}{\partial\theta_j} \theta^Tx^{(i)} \\ &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)} \left(1-g(\theta^Tx^{(i)})\right) - (1-y^{(i)})g(\theta^Tx^{(i)}) \right)x_{j}^{(i)} \\ &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)} - g(\theta^Tx^{(i)}) \right)x_{j}^{(i)} \\ &= -\frac{1}{m}\sum_{i=1}^m\left( y^{(i)} - h_\theta(x^{(i)}) \right)x_{j}^{(i)} \\ &= \frac{1}{m}\sum_{i=1}^m\left( h_\theta(x^{(i)}) - y^{(i)} \right)x_{j}^{(i)} \end{align}
梯度下降更新过程如下
\begin{align} \theta_j \mathrel{\mathop:} &= \theta_j - \alpha\frac{\partial}{\partial\theta_j}J(\theta), (j=0 \dots n) \\ \theta_j \mathrel{\mathop:} &= \theta_j - \alpha\frac{1}{m}\sum_{i=1}^m\left( h_\theta(x^{(i)}) - y^{(i)} \right) x_{j}^{(i)}, (j=0 \dots n) \\ \theta_j \mathrel{\mathop:} &= \theta_j - \alpha\sum_{i=1}^m\left( h_\theta(x^{(i)}) - y^{(i)} \right) x_{j}^{(i)}, (j=0 \dots n) \end{align}

你可能感兴趣的:(Logistic Regression推导过程)