【ML从入门到入土系列08】EM

文章目录

    • 1 理论
    • 2 代码
    • 3 参考

1 理论

EM算法通过迭代求解观测数据的对数似然函数 L ( θ ) = log ⁡ P ( Y ∣ θ ) {L}(\theta)=\log {P}(\mathrm{Y} | \theta) L(θ)=logP(Yθ)的极大化,实现极大似然估计。每次迭代包括两步:

  • E E E步:求期望
    Q ( θ , θ ( i ) ) = ∑ z log ⁡ P ( Y , Z ∣ θ ) P ( Z ∣ Y , θ ( i ) ) Q\left(\theta, \theta^{(i)}\right)=\sum_{z} \log P(Y, Z \mid \theta) P\left(Z \mid Y, \theta^{(i)}\right) Q(θ,θ(i))=zlogP(Y,Zθ)P(ZY,θ(i))
  • M M M步:求极大
    θ ( i + 1 ) = arg ⁡ max ⁡ θ Q ( θ , θ ( i ) ) \theta^{(i+1)}=\arg \max _{\theta} Q\left(\theta, \theta^{(i)}\right) θ(i+1)=argθmaxQ(θ,θ(i))

2 代码

class EM:
    def __init__(self, prob):
        self.pro_A, self.pro_B, self.pro_C = prob

    # E步
    def pmf(self, i):
        pro_1 = self.pro_A * math.pow(self.pro_B, data[i]) * math.pow(
            (1 - self.pro_B), 1 - data[i])
        pro_2 = (1 - self.pro_A) * math.pow(self.pro_C, data[i]) * math.pow(
            (1 - self.pro_C), 1 - data[i])
        return pro_1 / (pro_1 + pro_2)

    # M步
    def fit(self, data):
        count = len(data)
        for d in range(count):
            _ = yield
            _pmf = [self.pmf(k) for k in range(count)]
            pro_A = 1 / count * sum(_pmf)
            pro_B = sum([_pmf[k] * data[k] for k in range(count)]) / sum(
                [_pmf[k] for k in range(count)])
            pro_C = sum([(1 - _pmf[k]) * data[k]
                         for k in range(count)]) / sum([(1 - _pmf[k])
                                                        for k in range(count)])
            self.pro_A = pro_A
            self.pro_B = pro_B
            self.pro_C = pro_C

3 参考

理论:周志华《机器学习》,李航《统计学习方法》
代码:https://github.com/fengdu78/lihang-code

你可能感兴趣的:(ML,基础,机器学习,EM)