《统计学习方法》读书笔记
文章目录
- 10.2.3 后向算法
- 10.2.4 一些概率与期望值的计算
10.2.3 后向算法
β t ( i ) = ∑ j = 1 N a i j b j ( o t + 1 ) β t + 1 ( j ) \beta_t(i)=\displaystyle \sum_{j=1}^N a_{ij} b_j(o_{t+1}) \beta_{t+1}(j) βt(i)=j=1∑Naijbj(ot+1)βt+1(j)（10.20）
证明：
∑ j = 1 N a i j b j ( o t + 1 ) β t + 1 ( j ) = ∑ j = 1 N a i j P ( o t + 1 ∣ i t + 1 = q j ) P ( o t + 2 , o t + 3 , ⋯ , o T ∣ i t + 1 = q j , λ ) = ∑ j = 1 N a i j P ( P ( o t + 1 , o t + 2 , ⋯ , o T ∣ i t + 1 = q j , λ ) = ∑ j = 1 N P ( i t + 1 = q j ∣ i t = q i ) P ( o t + 1 , o t + 2 , ⋯ , o T ∣ i t + 1 = q j , λ ) = ∑ j = 1 N P ( i t = q i ∣ i t + 1 = q j ) P ( i t + 1 = q j ) P ( i t = q i ) P ( o t + 1 , o t + 2 , ⋯ , o T ∣ i t + 1 = q j , λ ) = 1 P ( i t = q i ) ∑ j = 1 N P ( o t + 1 , o t + 2 , ⋯ , o T , i t = q i ∣ i t + 1 = q j , λ ) P ( i t + 1 = q j ) = ∑ j = 1 N P ( o t + 1 , o t + 2 , ⋯ , o T , i t = q i , i t + 1 = q j ∣ λ ) P ( i t = q i ) = P ( o t + 1 , o t + 2 , ⋯ , o T , i t = q i ∣ λ ) P ( i t = q i ) = P ( o t + 1 , o t + 2 , ⋯ , o T , ∣ i t = q i , λ ) = β t ( i ) \begin{aligned} &\displaystyle \sum_{j=1}^N a_{ij} b_j(o_{t+1}) \beta_{t+1}(j) \\ =&\displaystyle \sum_{j=1}^N a_{ij} P(o_{t+1}|i_{t+1}=q_j) P(o_{t+2},o_{t+3},\dotsb,o_{T}| i_{t+1}=q_j,\lambda) \\ =&\displaystyle \sum_{j=1}^N a_{ij} P(P(o_{t+1},o_{t+2},\dotsb,o_{T} | i_{t+1}=q_j,\lambda) \\ =&\displaystyle \sum_{j=1}^N P(i_{t+1}=q_j | i_t=q_i) P(o_{t+1},o_{t+2},\dotsb,o_{T} | i_{t+1}=q_j,\lambda) \\ =&\displaystyle \sum_{j=1}^N \dfrac {P(i_t=q_i | i_{t+1}=q_j)P(i_{t+1}=q_j)}{P(i_t=q_i)} P(o_{t+1},o_{t+2},\dotsb,o_{T} | i_{t+1}=q_j,\lambda)\\ =&\dfrac{1}{P(i_t=q_i)}\displaystyle \sum_{j=1}^N P(o_{t+1},o_{t+2},\dotsb,o_{T},i_t=q_i | i_{t+1}=q_j,\lambda) P(i_{t+1}=q_j)\\ =&\dfrac{\displaystyle \sum_{j=1}^NP(o_{t+1},o_{t+2},\dotsb,o_{T},i_t=q_i,i_{t+1}=q_j|\lambda)}{P(i_t=q_i)} \\ =&\dfrac{P(o_{t+1},o_{t+2},\dotsb,o_{T},i_t=q_i|\lambda)}{P(i_t=q_i)} \\ =&P(o_{t+1},o_{t+2},\dotsb,o_{T},|i_t=q_i, \lambda)\\ =&\beta_t(i) \end{aligned} =========j=1∑Naijbj(ot+1)βt+1(j)j=1∑NaijP(ot+1∣it+1=qj)P(ot+2,ot+3,⋯,oT∣it+1=qj,λ)j=1∑NaijP(P(ot+1,ot+2,⋯,oT∣it+1=qj,λ)j=1∑NP(it+1=qj∣it=qi)P(ot+1,ot+2,⋯,oT∣it+1=qj,λ)j=1∑NP(it=qi)P(it=qi∣it+1=qj)P(it+1=qj)P(ot+1,ot+2,⋯,oT∣it+1=qj,λ)P(it=qi)1j=1∑NP(ot+1,ot+2,⋯,oT,it=qi∣it+1=qj,λ)P(it+1=qj)P(it=qi)j=1∑NP(ot+1,ot+2,⋯,oT,it=qi,it+1=qj∣λ)P(it=qi)P(ot+1,ot+2,⋯,oT,it=qi∣λ)P(ot+1,ot+2,⋯,oT,∣it=qi,λ)βt(i)
10.2.4 一些概率与期望值的计算
P ( i t = q i , i t + 1 = q j , O ∣ λ ) = α t ( i ) a i j b j ( o t + 1 ) β t + 1 ( j ) P(i_t=q_i,i_{t+1}=q_j,O|\lambda)=\alpha_t(i)a_{ij}b_j(o_{t+1})\beta_{t+1}(j) P(it=qi,it+1=qj,O∣λ)=αt(i)aijbj(ot+1)βt+1(j)
证明：
α t ( i ) a i j b j ( o t + 1 ) β t + 1 ( j ) = P ( o 1 , o 2 , ⋯ , o t , i t = q i ∣ λ ) P ( i t + 1 = q j ∣ i t = q i ) b j ( o t + 1 ) β t + 1 ( j ) = P ( o 1 , o 2 , ⋯ , o t , , i t = q i , i t + 1 = q j ∣ λ ) b j ( o t + 1 ) β t + 1 ( j ) = P ( o 1 , o 2 , ⋯ , o t , , i t = q i ∣ i t + 1 = q j , λ ) P ( i t + 1 = q j ∣ λ ) b j ( o t + 1 ) β t + 1 ( j ) = P ( o 1 , o 2 , ⋯ , o t , , i t = q i ∣ i t + 1 = q j , λ ) P ( i t + 1 = q j ∣ λ ) P ( o t + 1 ∣ i t + 1 = q j ) P ( o t + 2 , o t + 3 , ⋯ , o T ∣ i t + 1 = q j , λ ) = P ( o 1 , o 2 , ⋯ , o t , , i t = q i ∣ i t + 1 = q j , λ ) P ( i t + 1 = q j ∣ λ ) P ( o t + 1 , o t + 2 , ⋯ , o T ∣ i t + 1 = q j , λ ) = P ( o 1 , o 2 , ⋯ , o T , i t = q i ∣ i t + 1 = q j , λ ) P ( i t + 1 = q j ∣ λ ) = P ( o 1 , o 2 , ⋯ , o T , i t = q i , i t + 1 = q j ∣ λ ) = P ( i t = q i , i t + 1 = q j , O ∣ λ ) \begin{aligned} &\alpha_t(i)a_{ij}b_j(o_{t+1})\beta_{t+1}(j)\\ =&P(o_1,o_2,\dotsb,o_t,i_t=q_i|\lambda)P(i_{t+1}=q_j|i_t=q_i)b_j(o_{t+1})\beta_{t+1}(j) \\ =&P(o_1,o_2,\dotsb,o_t,,i_t=q_i,i_{t+1}=q_j|\lambda)b_j(o_{t+1})\beta_{t+1}(j) \\ =&P(o_1,o_2,\dotsb,o_t,,i_t=q_i|i_{t+1}=q_j,\lambda) P(i_{t+1}=q_j| \lambda)b_j(o_{t+1})\beta_{t+1}(j) \\ =&P(o_1,o_2,\dotsb,o_t,,i_t=q_i|i_{t+1}=q_j,\lambda) P(i_{t+1}=q_j| \lambda)P(o_{t+1}|i_{t+1}=q_j)P(o_{t+2},o_{t+3},\dotsb,o_T|i_{t+1}=q_j,\lambda) \\ =&P(o_1,o_2,\dotsb,o_t,,i_t=q_i|i_{t+1}=q_j,\lambda) P(i_{t+1}=q_j| \lambda)P(o_{t+1},o_{t+2},\dotsb,o_T|i_{t+1}=q_j,\lambda) \\ =&P(o_1,o_2,\dotsb,o_T,i_t=q_i|i_{t+1}=q_j,\lambda)P(i_{t+1}=q_j| \lambda)\\ =&P(o_1,o_2,\dotsb,o_T,i_t=q_i,i_{t+1}=q_j|\lambda)\\ =&P(i_t=q_i,i_{t+1}=q_j,O|\lambda)\\ \end{aligned} ========αt(i)aijbj(ot+1)βt+1(j)P(o1,o2,⋯,ot,it=qi∣λ)P(it+1=qj∣it=qi)bj(ot+1)βt+1(j)P(o1,o2,⋯,ot,,it=qi,it+1=qj∣λ)bj(ot+1)βt+1(j)P(o1,o2,⋯,ot,,it=qi∣it+1=qj,λ)P(it+1=qj∣λ)bj(ot+1)βt+1(j)P(o1,o2,⋯,ot,,it=qi∣it+1=qj,λ)P(it+1=qj∣λ)P(ot+1∣it+1=qj)P(ot+2,ot+3,⋯,oT∣it+1=qj,λ)P(o1,o2,⋯,ot,,it=qi∣it+1=qj,λ)P(it+1=qj∣λ)P(ot+1,ot+2,⋯,oT∣it+1=qj,λ)P(o1,o2,⋯,oT,it=qi∣it+1=qj,λ)P(it+1=qj∣λ)P(o1,o2,⋯,oT,it=qi,it+1=qj∣λ)P(it=qi,it+1=qj,O∣λ)