李航-统计学习方法-习题-第九章
9.2 证明引理 9.2.
引理 9.2 若 P ~ θ ( Z ) = P ( Z ∣ Y , θ ) \widetilde P_\theta(Z)=P(Z|Y,\theta) P θ(Z)=P(Z∣Y,θ)，则
F ( P ~ , θ ) = l o g P ( Y ∣ θ ) F(\widetilde P, \theta)=logP(Y|\theta) F(P ,θ)=logP(Y∣θ).
证明：
F ( P ~ , θ ) = E P ~ [ l o g P ( Y , Z ∣ θ ) ] + H ( P ~ ) = E P ~ [ l o g P ( Y , Z ∣ θ ) ] − E P ~ l o g P ~ ( Z ) = ∑ Z l o g P ( Y , Z ∣ θ ) P ~ θ ( Z ) − ∑ Z l o g P ~ ( Z ) P ~ ( Z ) = ∑ Z l o g P ( Y , Z ∣ θ ) P ( Z ∣ Y , θ ) − ∑ Z l o g P ( Z ∣ Y , θ ) P ( Z ∣ Y , θ ) = ∑ Z P ( Z ∣ Y , θ ) ( l o g P ( Y , Z ∣ θ ) − l o g P ( Z ∣ Y , θ ) ) = ∑ Z P ( Z ∣ Y , θ ) l o g P ( Y , Z ∣ θ ) P ( Z ∣ Y , θ ) = ∑ Z P ( Z ∣ Y , θ ) l o g P ( Y ∣ θ ) = l o g P ( Y ∣ θ ) ∑ Z P ( Z ∣ Y , θ ) = l o g P ( Y ∣ θ ) ⋅ 1 = l o g P ( Y ∣ θ ) \begin{aligned} F(\widetilde P, \theta) &= E_{\widetilde P}[logP(Y,Z|\theta)]+H(\widetilde P) \\ &= E_{\widetilde P}[logP(Y,Z|\theta)]-E_{\widetilde P}log\widetilde P(Z) \\ &= \sum_ZlogP(Y,Z|\theta)\widetilde P_\theta(Z) - \sum_Zlog\widetilde P(Z)\widetilde P(Z) \\ &= \sum_ZlogP(Y,Z|\theta)P(Z|Y,\theta)- \sum_ZlogP(Z|Y,\theta)P(Z|Y,\theta) \\ &= \sum_ZP(Z|Y,\theta)(logP(Y,Z|\theta)-logP(Z|Y,\theta)) \\ &= \sum_ZP(Z|Y,\theta)log\dfrac{P(Y,Z|\theta)}{P(Z|Y,\theta)} \\ &= \sum_ZP(Z|Y,\theta)logP(Y|\theta) \\ &= logP(Y|\theta)\sum_ZP(Z|Y,\theta) \\ &= logP(Y|\theta)\cdot1 \\ &= logP(Y|\theta) \end{aligned} F(P ,θ)=EP [logP(Y,Z∣θ)]+H(P )=EP [logP(Y,Z∣θ)]−EP logP (Z)=Z∑logP(Y,Z∣θ)P θ(Z)−Z∑logP (Z)P (Z)=Z∑logP(Y,Z∣θ)P(Z∣Y,θ)−Z∑logP(Z∣Y,θ)P(Z∣Y,θ)=Z∑P(Z∣Y,θ)(logP(Y,Z∣θ)−logP(Z∣Y,θ))=Z∑P(Z∣Y,θ)logP(Z∣Y,θ)P(Y,Z∣θ)=Z∑P(Z∣Y,θ)logP(Y∣θ)=logP(Y∣θ)Z∑P(Z∣Y,θ)=logP(Y∣θ)⋅1=logP(Y∣θ)
证毕.