机器学习-白板推导 P4_4 (逻辑回归)

逻辑回归

公式

$Sigmoidfunction$

$\sigma =\frac{1}{1+e^{-z}}$

$z\to +\infty ，\lim \sigma (z)=1$
$z\to 0，\lim \sigma (z)=0.5$
$z\to -\infty ，\lim \sigma (z)=0$

$p_1=p(y=1|x)=\sigma (w^{T}x)=\frac{1}{1+e^{-w^{T}x}},y=1$

$p_0=p(y=0|x)=1-p(y=1|x)=\frac{e^{-w^{T}x}}{1+e^{-w^{T}x}},y=0$

$p(y|x)=p_1^y{p}_0^{1-y}$

方法

MLE:
$$\begin{array}{rl}\hat{w}& =arg\underset{w}{\max }\log p(Y|X)\\ & =arg\underset{w}{\max }\log \prod _{i=1}^{N}p(y_i,x_i)\\ & =arg\underset{w}{\max }\sum _{i=1}^N\log p(y_i,x_i)\\ & =arg\underset{w}{\max }\sum _{i=1}^N(y_i\log \frac{1}{1+e^{-w^{T}x}}+(1-y_i)log(1-\frac{1}{1+e^{-w^{T}x}}))\end{array}$$
$MLE(max)\to lossfunction(minCrossEntroy)$