EM算法理解的第一层境界：期望E和最大化M（二）

前言：学生时代入门机器学习的时候接触的EM算法，当时感觉这后面应该有一套数学逻辑来约束EM算法的可行性。最近偶然在知乎上拜读了史博大佬的《EM算法理解的九层境界》^[1]，顿时感觉自己还是局限了。重新学习思考了一段时间，对EM算法有了更深的理解。

四、数学推导理解EM算法的形式

接下来我们通过严格的数学推导来理解EM算法形式式子的含义。接着上面给出的EM算法的式子：
$${\theta }^{(t+1)}=\underset{\theta }{\mathrm{argmax}}Q(\theta |{\theta }^{(t)})=\underset{\theta }{\mathrm{argmax}}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]$$
下面我们一步一步进行推导。
首先隐变量$Z=[z_1,z_2,...,z_n],z_i\in \{0,1\}$，分量$z_i=1$表示$i$轮次使用的是硬币$A$，分量$z_i=0$表示$i$轮次使用的是硬币$B$，一共进行了$n$轮次的投币行为。那么隐变量$Z$的变量空间为$\{0,1{\}}^n$，一共$2^n$个离散点组成的离散空间。于是我们有：
$$E_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]=\sum _{Z\in \{0,1{\}}^n}[P(Z|X,{\theta }^{(t)})\ast \log L(\theta ;X,Z)]$$
其中，隐变量$z_i$之间是独立同分布的，所以我们有：
$$P(Z|X,{\theta }^{(t)})=\prod _{i=1}^{n}P(z_i|x_i,{\theta }^{(t)})$$
对于每一个轮次$i$而言，在给定观测变量$x_i\in [0,\delta ]$和硬币的概率参数${\theta }^{(t)}$（即${\theta }_A^{(t)}$、${\theta }_B^{(t)}$和${\theta }_C^{(t)}$）之后，我们有：
$$\begin{array}{rl}P(z_i|x_i,{\theta }^{(t)})& =\frac{P(z_i,x_i|{\theta }^{(t)})}{P(x_i|{\theta }^{(t)})}\\ & =\frac{P(z_i,x_i|{\theta }^{(t)})}{{\sum }_{z_i\in 0,1}P(z_i,x_i|{\theta }^{(t)})}\end{array}$$
其中，给定${\theta }^{(t)}$之后，$(z_i,x_i)$的联合概率分布可以直接给出来，$P(z_i=1,x_i|{\theta }^{(t)})$为第$i$轮次是硬币$A$且抛出了$x_i$个正面的概率：
$$\begin{array}{rl}P(z_i=1,x_i|{\theta }^{(t)})& =P(z_i=1|{\theta }^{(t))}\ast P(x_i|z_i=1,{\theta }^{(t)})\\ & =P(z_i=1|{\theta }^{(t)}）\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\\ & ={\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\end{array}$$

$P(z_i=0,x_i,{\theta }^{(t)})$为第$i$轮次是硬币$B$且抛出了$x_i$个正面的概率：

$$\begin{array}{rl}P(z_i=0,x_i|{\theta }^{(t)})& =P(z_i=0|{\theta }^{(t)})\ast P(x_i|z_i=0,{\theta }^{(t)})\\ & =P(z_i=0|{\theta }^{(t)}）\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\\ & =(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\end{array}$$

接下来，给定$x_i,{\theta }^{(t)}$情况下轮次$i$是硬币A的概率$P(z_i=1|x_i,{\theta }^{(t)})$和是硬币B的概率$P(z_i=0|x_i,{\theta }^{(t)})$如下：
$$\begin{array}{rl}P(z_i=1|x_i,{\theta }^{(t)})& =\frac{P(z_i=1,x_i|{\theta }^{(t)})}{P(z_i=1,x_i|{\theta }^{(t)})+P(z_i=0,x_i|{\theta }^{(t)})}\\ & =\frac{{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}}{{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}+(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}}\end{array}$$

$$\begin{array}{rl}P(z_i=0|x_i,{\theta }^{(t)})& =\frac{P(z_i=0,x_i|{\theta }^{(t)})}{P(z_i=1,x_i|{\theta }^{(t)})+P(z_i=0,x_i|{\theta }^{(t)})}\\ & =\frac{(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}}{{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}+(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}}\end{array}$$

两个式子的分母是一样的，我们记
$${\eta }_i={\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}+(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}$$
则有：
$$P(z_i=0|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}$$
$$P(z_i=1|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}$$

变量$(Z|X,{\theta }^{(t)})$的分布我们表示出来了，下一步来求联合分布$(\theta ;X,Z)$的log-likelyhood: $\log L(\theta ;X,Z)$。
对于每一轮次的likelyhood，我们有：
$$\begin{array}{rl}L(\theta ;x_i,z_i)& =P(x_i|z_i,\theta )\ast P(z_i|\theta )\ast P(\theta )\\ & =(z_i\ast {\theta }_A+(1-z_i){\theta }_B{)}^{x_i}\ast (1-z_i\ast {\theta }_A-(1-z_i){\theta }_B{)}^{(\delta -x_i)}\ast {\theta }_C^{z_i}\ast (1-{\theta }_C{)}^{1-z_i}\ast 1\end{array}$$
同样，由于每个轮次都是独立同分布的，我们有：
$$\begin{array}{rl}L(\theta ;X,Z)& =\prod _{i=1}^{n}L(\theta ;x_i,z_i)\\ & =\prod _{i=1}^n(z_i\ast {\theta }_A+(1-z_i){\theta }_B{)}^{x_i}\ast (1-z_i\ast {\theta }_A-(1-z_i){\theta }_B{)}^{(\delta -x_i)}\ast {\theta }_C^{z_i}\ast (1-{\theta }_C{)}^{1-z_i}\end{array}$$
$\log$一下我们有：
$$\begin{array}{rl}\log L(\theta ;X,Z)& =\log \prod _{i=1}^n(z_i\ast {\theta }_A+(1-z_i){\theta }_B{)}^{x_i}\ast (1-z_i\ast {\theta }_A-(1-z_i){\theta }_B{)}^{(\delta -x_i)}\ast {\theta }_C^{z_i}\ast (1-{\theta }_C{)}^{1-z_i}\\ & =\sum _{i=1}^n[x_i\ast \log (z_i\ast {\theta }_A+(1-z_i){\theta }_B)+(\delta -x_i)\ast \log (1-z_i\ast {\theta }_A-(1-z_i){\theta }_B)+z_i\ast \log {\theta }_C+(1-z_i)\ast \log (1-{\theta }_C)]\end{array}$$

OK，到这一步$\log L(\theta ;X,Z)$和$P(Z|X,{\theta }^{(t)})$我们都表示出来了，接下来就是对他们求个期望：
$$\begin{array}{rl}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]& =\sum _{Z\in \{0,1{\}}^n}[P(Z|X,{\theta }^{(t)})\ast \log L(\theta ;X,Z)]\\ & =\sum _{Z\in \{0,1{\}}^n}\prod _{i=1}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=1}^{n}l(x_j,z_j,\theta )\end{array}$$
注意，这里我们为了防止变量重复，把$\log L(\theta ;X,Z)$中的遍历变量$i$替换为了$j$。其中
$$l(x_j,z_j,Q)=x_j\ast \log (z_j\ast {\theta }_A+(1-z_j){\theta }_B)+(\delta -x_j)\ast \log (1-z_j\ast {\theta }_A-(1-z_j){\theta }_B)+z_j\ast \log {\theta }_C+(1-z_j)\ast \log (1-{\theta }_C)$$
$$P(z_i=0|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}$$
$$P(z_i=1|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}$$

这里得到的$E_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]$式子开起来很吓人，直接使用期望的性质可以将其化简为简单的形式，这里我们不使用独立分布期望的性质，直接使用一些数学技巧来进行化简。

对于所有的变量空间$Z\in \{0,1{\}}^n$，我们将其拆分一下，$z_1$和$z2,z3,...,z_n$分别表示为$z_1$和$Z_2^n$，然后将其分为$z_1=1$和$z_1=0$两个部分。那么上面式子可以分解为：
$$\begin{array}{rl}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]& =\sum _{z1=1,Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=1}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=1}^{n}l(x_j,z_j,\theta )\\ & +\sum _{z1=0,Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=1}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=1}^{n}l(x_j,z_j,\theta )\\ & =\sum _{z1=1,Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\ast P(z_1=1|x_1,{\theta }^{(t)})\ast [l(x_1,z_1=1,\theta )+\sum _{j=2}^{n}l(x_j,z_j,\theta )]\\ & +\sum _{z1=0,Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\ast P(z_0=1|x_1,{\theta }^{(t)})\ast [l(x_1,z_1=0,\theta )+\sum _{j=2}^{n}l(x_j,z_j,\theta )]\\ & =P(z_1=1|x_1,{\theta }^{(t)})\ast l(x_1,z_1=1,\theta )\ast \sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\\ & +P(z_1=1|x_1,{\theta }^{(t)})\ast \sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=2}^{n}l(x_j,z_j,\theta )\\ & +P(z_1=0|x_1,{\theta }^{(t)})\ast l(x_1,z_1=0,\theta )\ast \sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\\ & +P(z_1=0|x_1,{\theta }^{(t)})\ast \sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=2}^{n}l(x_j,z_j,\theta )\end{array}$$

由概率分布之和为1的性质，我们有：
$$P(z_1=1|x_1,{\theta }^{(t)})+P(z_1=0|x_1,{\theta }^{(t)})=1$$
$$\sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})=\sum _{Z_2^n\in \{0,1{\}}^{n-1}}P(Z_2^n|X_2^n,{\theta }^{(t)})=1$$
所以我们有：
$$\begin{array}{rl}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]& =P(z_1=1|x_1,{\theta }^{(t)})\ast l(x_1,z_1=1,\theta )\\ & +P(z_1=0|x_1,{\theta }^{(t)})\ast l(x_1,z_1=0,\theta )\\ & +\sum _{Z_2^n\in \{0,1{\}}^{n-1}}\prod _{i=2}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=2}^{n}l(x_j,z_j,\theta )\\ & =P(z_1=1|x_1,{\theta }^{(t)})\ast l(x_1,z_1=1,\theta )\\ & +P(z_1=0|x_1,{\theta }^{(t)})\ast l(x_1,z_1=0,\theta )\\ & +P(z_2=1|x_1,{\theta }^{(t)})\ast l(x_2,z_2=1,\theta )\\ & +P(z_2=0|x_1,{\theta }^{(t)})\ast l(x_2,z_2=0,\theta )\\ & +\sum _{Z_3^n\in \{0,1{\}}^{n-2}}\prod _{i=3}^{n}P(z_i|x_i,{\theta }^{(t)})\ast \sum _{j=3}^{n}l(x_j,z_j,\theta )\\ & =......\\ & =\sum _{i=1}^{n}P(z_i=1|x_i,{\theta }^{(t)})\ast l(x_i,z_i=1,\theta )+\sum _{i=1}^{n}P(z_i=0|x_i,{\theta }^{(t)})\ast l(x_i,z_i=0,\theta )\end{array}$$
最终我们求得了需要maximum的式子：
$$E_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]=\sum _{i=1}^{n}P(z_i=1|x_i,{\theta }^{(t)})\ast l(x_i,z_i=1,Q)+\sum _{i=1}^{n}P(z_i=0|x_i,{\theta }^{(t)})\ast l(x_i,z_i=0,\theta )$$
其中
$$l(x_j,z_j,\theta )=x_j\ast \log (z_j\ast {\theta }_A+(1-z_j){\theta }_B)+(\delta -x_j)\ast \log (1-z_j\ast {\theta }_A-(1-z_j){\theta }_B)+z_j\ast \log {\theta }_C+(1-z_j)\ast \log (1-{\theta }_C)$$
$$P(z_i=0|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}$$
$$P(z_i=1|x_i,{\theta }^{(t)})=\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}$$
把上述式子带入，有：
$$\begin{array}{rl}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]& =\sum _{i=1}^{n}P(z_i=1|x_i,{\theta }^{(t)})\ast l(x_i,z_i=1,\theta )+\sum _{i=1}^{n}P(z_i=0|x_i,{\theta }^{(t)})\ast l(x_i,z_i=0,\theta )\\ & =\sum _{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \log {\theta }_A+(\delta -x_i)\ast \log (1-{\theta }_A)+\log {\theta }_C]\\ & +\sum _{i=1}^n\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \log {\theta }_B+(\delta -x_i)\ast \log (1-{\theta }_B)+\log (1-{\theta }_C)]\end{array}$$
其中,
$${\eta }_i={\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}+(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}$$

可以看到这个式子中，我们已经把所有隐变量$Z$积分消除掉了，到这里我们就可以进行求解了。我们需要求解

$$\begin{array}{rl}{\theta }^{(t+1)}& =\underset{\theta }{\mathrm{argmax}}Q(\theta |{\theta }^{(t)})=\underset{\theta }{\mathrm{argmax}}{E}_{Z|X,{\theta }^{(t)}}[\log L(\theta ;X,Z)]\\ & =\underset{\theta }{\mathrm{argmax}}\sum _{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \log {\theta }_A+(\delta -x_i)\ast \log (1-{\theta }_A)+\log {\theta }_C]\\ & +\sum _{i=1}^n\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \log {\theta }_B+(\delta -x_i)\ast \log (1-{\theta }_B)+\log (1-{\theta }_C)]\end{array}$$

其中，$({\theta }_A,{\theta }_B)$是需要求解的变量，其他$({\theta }_A^{(t)},{\theta }_C^{(t)},{\theta }_C^{(t)}),{\eta }_i$都是常量。这里我们联立偏导方程进行求解：
$$\{\begin{array}{r}\frac{\partial Q}{\partial {\theta }_A}=0\\ \frac{\partial Q}{\partial {\theta }_B}=0\\ \frac{\partial Q}{\partial {\theta }_C}=0\end{array}$$
一个一个来：
$$\begin{array}{rl}\frac{\partial Q}{\partial {\theta }_A}& =\sum _{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \frac{1}{{\theta }_A}-(\delta -x_i)\ast \frac{1}{(1-{\theta }_A)}]\\ & =\sum _{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast [x_i\ast \frac{1}{{\theta }_A}-(\delta -x_i)\ast \frac{1}{(1-{\theta }_A)}]\\ & =\sum _{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast [\frac{x_i-\delta \ast {\theta }_A}{{\theta }_A\ast (1-{\theta }_A)}]\end{array}$$
令$\frac{\partial Q}{\partial {\theta }_A}=0$，我们有：
$${\theta }_A^{(t+1)}=\frac{{\sum }_{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast x_i}{{\sum }_{i=1}^n\frac{1}{{\eta }_i}{\theta }_C^{(t)}\ast ({\theta }_A^{(t)}{)}^{x_i}\ast (1-{\theta }_A^{(t)}{)}^{\delta -x_i}\ast \delta }$$
同理我们有：
$${\theta }_B^{(t+1)}=\frac{{\sum }_{i=1}^n\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\ast x_i}{{\sum }_{i=1}^n\frac{1}{{\eta }_i}(1-{\theta }_C^{(t)})\ast ({\theta }_B^{(t)}{)}^{x_i}\ast (1-{\theta }_B^{(t)}{)}^{\delta -x_i}\ast \delta }$$
θ C ( t + 1 ) = ∑ i = 1 n 1 η i θ C ( t ) ∗ ( θ A ( t ) ) x i ∗ ( 1 − θ A ( t ) ) δ − x i ∑ i = 1 n 1 η i [ θ C ( t ) ∗ ( θ A ( t ) ) x i ∗ ( 1 − θ A ( t ) ) δ − x i + ( 1 − θ C ( t ) ) ∗ ( θ B ( t ) ) x i ∗ ( 1 − θ B ( t ) ) δ − x i ] = 1 n ∑ i = 1 n 1 η i θ C ( t ) ∗ ( θ A ( t ) ) x i ∗ ( 1 − θ A ( t ) ) δ − x i \begin{aligned} \theta_C^{(t+1)} &= \frac{\sum_{i=1}^{n}\frac{1}{\eta_i} \theta_C^{(t)}* (\theta_A^{(t)})^{x_i}*(1-\theta_A^{(t)})^{\delta - x_i}}{\sum_{i=1}^{n}\frac{1}{\eta_i} [\theta_C^{(t)}* (\theta_A^{(t)})^{x_i}*(1-\theta_A^{(t)})^{\delta - x_i}+(1-\theta_C^{(t)})* (\theta_B^{(t)})^{x_i}*(1-\theta_B^{(t)})^{\delta - x_i}]} \\ &=\frac{1}{n}\sum_{i=1}^{n}\frac{1}{\eta_i} \theta_C^{(t)}* (\theta_A^{(t)})^{x_i}*(1-\theta_A^{(t)})^{\delta - x_i} \end{aligned} θC(t+1)​​=∑i=1n​ηi​1​[θC(t)​∗(θA(t)​)xi​∗(1−θ