5.【笔记】统计学习方法—EM算法

1.EM是什么

EM是含有隐变量的概率模型的极大似然估计回极大后验概率估计的迭代算法。

假设：
$$P(Y|\theta )=\prod [\pi p^{y_i}(1-p{)}^{1-y_i}+(1-\pi )q^{y_i}(1-q{)}^{1-y_i}]$$

1.1 E：求期望

$${\mu }^{i+1}=\frac{\pi (p^i{)}^{y_i}(1-(p^i){)}^{1-y_i}}{\pi (p^i{)}^{y_i}(1-(p^i){)}^{1-y_i}+(1-\pi )(q^i{)}^{y_i}(1-(q^i){)}^{1-y_i}}$$

1.2 M：极大

$${\pi }^{i+1}=\frac{1}{n}\sum _{j=1}^n{\mu }_j^{i+1}$$

$$p^{i+1}=\frac{{\sum }_{j=1}^n{\mu }_j^{i+1}{y}_i}{{\sum }_{j=1}^n{\mu }_j^{i+1}}$$

$$q^{i+1}=\frac{{\sum }_{j=1}^n(1-{\mu }_j^{i+1}{y}_i)}{{\sum }_{j=1}^n(1-{\mu }_j^{i+1})}$$

2. 代码

#py3.7
class EM:
    def __init__(self, prob):
        self.pro_A, self.pro_B, self.pro_C = prob
        
    # e_step
    def pmf(self, i):
        pro_1 = self.pro_A * math.pow(self.pro_B, data[i]) * math.pow((1-self.pro_B), 1-data[i])
        pro_2 = (1 - self.pro_A) * math.pow(self.pro_C, data[i]) * math.pow((1-self.pro_C), 1-data[i])
        return pro_1 / (pro_1 + pro_2)
    
    # m_step
    def fit(self, data):
        count = len(data)
        print('init prob:{}, {}, {}'.format(self.pro_A, self.pro_B, self.pro_C))
        for d in range(count):
            _ = yield
            _pmf = [self.pmf(k) for k in range(count)]
            pro_A = 1/ count * sum(_pmf)
            pro_B = sum([_pmf[k]*data[k] for k in range(count)]) / sum([_pmf[k] for k in range(count)])
            pro_C = sum([(1-_pmf[k])*data[k] for k in range(count)]) / sum([(1-_pmf[k]) for k in range(count)])
            print('{}/{}  pro_a:{:.3f}, pro_b:{:.3f}, pro_c:{:.3f}'.format(d+1, count, pro_A, pro_B, pro_C))
            self.pro_A = pro_A
            self.pro_B = pro_B
            self.pro_C = pro_C