李航《统计学习方法》第2版读书笔记第10章（附详细推导过程）

隐马尔可夫模型

–李航《统计学习方法》第2版第10章（附详细推导过程）

一、基本概念

隐马尔可夫模型（hidden Markov model，HMM）是时序概率模型，描述由一个隐藏的马尔可夫链随机生成的不可观测的状态随机序列（state sequence），再由每个状态生产一个观测从而产生观测序列（observation sequence）的过程。

令$Q=\{q_1,q_2,\cdots ,q_N\}$是所有可能的状态的集合，$V=\{v_1,v_2,\cdots ,v_M\}$是所有可能的观测集合；

$I=(i_1,i_2,\cdots ,i_T)$是长度为$T$的状态序列，$O=(o_1,o_2,\cdots ,o_T)$是对应的观测序列 ；

$A=[a_{ij}{]}_{N\times N}$是状态转移概率矩阵，其中：$a_{ij}=P(i_{t+1}=q_j|i_t=q_i),i,j=1,2,\cdots ,N$；

$B=[b_j(k){]}_{N\times M}$是观测概率矩阵，其中：$b_j(k)=P(o_t=v_k|i_t=q_j),j=1,2,\cdots ,N,k=1,2,\cdots ,M$；

$\pi =({\pi }_i)$是初始概率向量，其中：${\pi }_i=P(i_1=q_i),i=1,2,\cdots ,N$。

$I$由$A$和$\pi$确定，$O$由$B$确定，由定义可知，HMM假设如下：

• 齐次马尔可夫链假设（homogeneous Markov chain），即状态转移概率矩阵$A$，观测概率矩阵$B$，均与时刻$t$无关，$\lambda =(A,B,\pi )$唯一确定了一个HMM;

• 一阶马尔可夫链假设（first-order Markov chain），即任意时刻的状态只依赖于其前一时刻的状态：

  $$P(i_t|i_{t-1},o_{t-1},\cdots ,i_1,o_1)=P(i_t|i_{t-1}),t=1,2,\cdots ,T$$

• 观测独立性假设，即任意时刻的观测只依赖于该时刻的状态：

  $$P(o_t|o_T,i_T,o_{T-1},i_{T-1},\cdots ,o_{t+1},i_{t+1},o_t,i_t,o_{t-1},i_{t-1},\cdots ,o_1,i_1)=P(o_t|i_t)$$

二、概率计算

直接计算法

略

前向计算法

给定一个HMM $\lambda$，前向概率${\alpha }_t(i)=P（o_1,o_2,\cdots ,o_t,i_t=q_i)$是到时刻$t$观测序列为$o_1,o_2,\cdots ,o_t$切状态为$q_i$的概率。

$$\begin{array}{rl}{\alpha }_1(i)& =P(o_1,i_1=q_i)=P(o_1|i_1=q_i)P(i_1=q_i)=b_i(o_1){\pi }_i\\ {\alpha }_{t+1}(i)& =P(o_1,o_2,\cdots ,o_t,o_{t+1},i_{t+1}=q_i)\\ & =\sum _{j=1}^{N}P(o_1,o_2,\cdots ,o_t,o_{t+1},i_t=q_j,i_{t+1}=q_i)\\ & =\sum _{j=1}^{N}P(o_{t+1}|o_1,o_2,\cdots ,o_t,i_t=q_j,i_{t+1}=q_i)P(i_{t+1}=q_i|o_1,o_2,\cdots ,o_t,i_t=q_j)P(o_1,o_2,\cdots ,o_t,i_t=q_j)\\ & =\sum _{j=1}^{N}P(o_{t+1}|i_{t+1}=q_i)P(i_{t+1}=q_i|i_t=q_j)P(o_1,o_2,\cdots ,o_t,i_t=q_j)\\ & =\sum _{j=1}^N{b}_i(o_{t+1})a_{ji}{\alpha }_t(j)\\ P(O)& =P(o_1,o_2,\cdots ,o_T)=\sum _{i=1}^{N}P(o_1,o_2,\cdots ,o_T,i_T=q_i)=\sum _{i=1}^N{\alpha }_T(i)\end{array}$$

后向计算法

给定HMM $\lambda$，定义$t$时刻状态为$q_i$的条件下，$t+1$时刻到$T$时刻观测序列为$o_{t+1},ot+2,\cdots ,o_T$的概率为后向概率，${\beta }_t(i)=P(o_{t+1},ot+2,\cdots ,o_T|i_t=q_i)$

$$\begin{array}{rl}{\beta }_T(i)=& 1,i=1,2,\cdots ,N\\ {\beta }_t(i)=& P(o_{t+1},o_{t+2},\cdots ,o_T|i_t=q_i)\\ =& \sum _{j=1}^{N}P(o_{t+1},o_{t+2},\cdots ,o_T,i_{t+1}=q_j|i_t=q_i)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1},o_{t+2},\cdots ,o_T,i_{t+1}=q_j,i_t=q_i)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1},o_{t+2},\cdots ,o_T,i_t=q_i|i_{t+1}=q_j)P(i_{t+1}=q_j)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)P(i_t=q_i|i_{t+1}=q_j)P(i_{t+1}=q_j)(d-separation)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1},o_{t+2},\cdots ,o_T,i_{t+1}=q_j)P(i_t=q_i|i_{t+1}=q_j)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1}|o_{t+2},\cdots ,o_T,i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T,i_{t+1}=q_j)P(i_t=q_i|i_{t+1}=q_j)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1}|i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T|i_{t+1}=q_j)P(i_{t+1}=q_j)P(i_t=q_i|i_{t+1}=q_j)\\ =& \frac{1}{P(i_t=q_i)}\sum _{j=1}^{N}P(o_{t+1}|i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T|i_{t+1}=q_j)P(i_{t+1}=q_j|i_t=q_i)P(i_t=q_i)\\ =& \sum _{j=1}^{N}P(o_{t+1}|i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T|i_{t+1}=q_j)P(i_{t+1}=q_j|i_t=q_i)\\ =& \sum _{j=1}^N{b}_j(o_{t+1}){\beta }_{t+1}(j)a_{ij},t=T-1,T-2,\cdots ,1,i=1,2,\cdots ,N\\ P(O)=& \sum _{i=1}^N{\beta }_1(i)b_1(o_1){\pi }_i\end{array}$$

前后向概率的关系

$$\begin{array}{rl}P(O)& =P(o_1,o_2,\cdots ,o_N)\\ & =\sum _{i=1}^N\sum _{j=1}^{N}P(o_1,o_2,\cdots ,o_N,i_t=q_i,i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^{N}P(o_1,o_2,\cdots ,o_t,i_t=q_i|i_{t+1}=q_j)P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)P(i_{t+1}=q_j)(d-separation)\\ & =\sum _{i=1}^N\sum _{j=1}^{N}P(o_1,o_2,\cdots ,o_t,i_t=q_i,i_{t+1}=q_j)P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^{N}P(i_{t+1}=q_j|o_1,o_2,\cdots ,o_t,i_t=q_i)P(o_1,o_2,\cdots ,o_t,i_t=q_i)P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^{N}P(i_{t+1}=q_j|i_t=q_i)P(o_1,o_2,\cdots ,o_t,i_t=q_i)P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^N{a}_{ij}{\alpha }_t(i)P(o_{t+1},o_{t+2},\cdots ,o_T|i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^N{a}_{ij}{\alpha }_t(i)P(o_{t+1},o_{t+2},\cdots ,o_T,i_{t+1}=q_j)\frac{1}{P(i_{t+1}=q_j)}\\ & =\sum _{i=1}^N\sum _{j=1}^N{a}_{ij}{\alpha }_t(i)P(o_{t+1}|o_{t+2},\cdots ,o_T,i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T,i_{t+1}=q_j)\frac{1}{P(i_{t+1}=q_j)}\\ & =\sum _{i=1}^N\sum _{j=1}^N{a}_{ij}{\alpha }_t(i)P(o_{t+1}|i_{t+1}=q_j)P(o_{t+2},\cdots ,o_T|i_{t+1}=q_j)\\ & =\sum _{i=1}^N\sum _{j=1}^N{a}_{ij}{\alpha }_t(i)b_j(o_{t+1}){\beta }_{t+1}(j)\end{array}$$

其他一些概率和期望的计算

1. 给定一个HMM $\lambda$和观测$O$，则$t$时刻状态为$i_t=q_i$的概率：

   $$\begin{array}{rl}{\gamma }_t(i)& =P(i_t=q_i|O)=\frac{1}{P(O)}P(O|i_t=q_i)P(i_t=q_i)\\ & =\frac{1}{P(O)}P(o_1,\cdots ,o_t|i_t=q_i)P(o_{t+1},\cdots ,o_T|i_t=q_i)P(i_t=q_i)(d-seperation)\\ & =\frac{1}{P(O)}P(o_1,\cdots ,o_t,i_t=q_i)P(o_{t+1},\cdots ,o_T|i_t=q_i)\\ & =\frac{{\alpha }_t(i){\beta }_t(i)}{P(O)}=\frac{{\alpha }_t(i){\beta }_t(i)}{{\sum }_{i=1}^{N}P(i_t=q_i,O)}=\frac{{\alpha }_t(i){\beta }_t(i)}{{\sum }_{i=1}^N{\alpha }_t(i){\beta }_t(i)}\end{array}$$
   给定一个HMM $\lambda$和观测$O$，则$t$时刻状态为$i_t=q_i$且$i_{t+1}=q_j$的概率：

   $$\begin{array}{rl}{\xi }_t(i,j)& =P(i_t=q_i,i_{t+1}=q_j|O)\\ & =\frac{1}{P(O)}P(i_t=q_i,i_{t+1}=q_j,O)\\ & =\frac{a_{ij}{\alpha }_t(i)b_j(o_{t+1}){\beta }_{t+1}(j)}{P(O)}\\ & =\frac{a_{ij}{\alpha }_t(i)b_j(o_{t+1}){\beta }_{t+1}(j)}{{\sum }_{i=1}^N{\sum }_{j=1}^N{a}_{ij}{\alpha }_t(i)b_j(o_{t+1}){\beta }_{t+1}(j)}\end{array}$$

2. 一些期望：观测$O$下状态为$q_i$的期望为${\sum }_{t=1}^T{\gamma }_t(i)$；

   观测$O$下状态由$q_i$转移的期望为${\sum }_{t=1}^{T-1}{\gamma }_t(i)$；

   观测$O$下状态由$q_i$转移为$q_j$的期望为${\sum }_{t=1}^T{\xi }_t(i,j)$

三、学习算法

监督学习算法

略

Baum-Welch算法

给定训练数据为观测序列$O_1,O_2,\cdots ,O_S$，学习$\lambda =(\pi ,A,B)$，将观测序列作为观测数据$O$，将状态序列作为补课观测数据$I$，可以将$HMM$看做一个含有隐变量的概率模型$P(O|\lambda )={\sum }_{I}P(O|I,\lambda )P(I|\lambda )$，可以利用EM算法进行学习。

1. 确定完全数据的对数似然函数。完全数据$（O,I）=(o_1,o_2,\cdots ,o_T,i_1,i_2,\cdots ,i_T)$，其对数似然函数为$logP(O,I|\lambda )$。

2. EM算法的E步，求Q函数$Q(\lambda ,\overline{\lambda })$：

   $Q(\lambda ,\overline{\lambda })={\sum }_{I}logP(O,I|\lambda )P(O,I|\overline{\lambda })$，其中$\overline{\lambda }$是模型参数当前的估计值，$\lambda$是要极大化的模型参数。

   P ( O , I ∣ λ ) = P ( o 1 ∣ o 2 , … , o T , I , λ ) P ( o 2 , … , o T , I , λ ) = P ( o 1 ∣ i 1 , λ ) P ( o 2 , … , o T , I , λ ) = P ( o 1 ∣ i 1 , λ ) P ( o 2 ∣ i 2 , λ ) P ( o 3 , … , o T , I , λ ) = ⋯ = P ( o 1 ∣ i 1 , λ ) P ( o 2 ∣ i 2 , λ ) ⋯ P ( o T ∣ i T , λ ) P ( I ∣ λ ) = ∏ t = 1 T P ( o t ∣ i t , λ ) P ( I ∣ λ ) = ∏ t = 1 T P ( o t ∣ i t , λ ) P ( i T ∣ i 1 , ⋯   , i T − 1 , λ ) P ( i 1 , ⋯   , i T − 1 ∣ λ ) = ∏ t = 1 T P ( o t ∣ i t , λ ) P ( i T ∣ i T − 1 , λ ) P ( i 1 , ⋯   , i T − 1 ∣ λ ) = ⋯ = ∏ t = 1 T P ( o t ∣ i t , λ ) P ( i T ∣ i T − 1 , λ ) P ( i T − 1 ∣ i T − 2 , λ ) ⋯ P ( i 2 ∣ i 1 , λ ) P ( i 1 ∣ λ ) = π i 1 b i 1 ( o 1 ) a i 1 , i 2 b i 2 ( o 2 ) a i 2 , i 3 ⋯ b i T − 1 ( o T − 1 ) a i T − 1 , i T b i T ( O T ) \begin{aligned} P(O,I|\lambda)&=P(o_1|o_2,\dots,o_T,I,\lambda)P(o_2,\dots,o_T,I,\lambda)\\ &=P(o_1|i_1,\lambda)P(o_2,\dots,o_T,I,\lambda)\\ &=P(o_1|i_1,\lambda)P(o_2|i_2,\lambda)P(o_3,\dots,o_T,I,\lambda) =\cdots\\ &=P(o_1|i_1,\lambda)P(o_2|i_2,\lambda)\cdots P(o_T|i_T,\lambda)P(I|\lambda)\\ &=\prod_{t=1}^TP(o_t|i_t,\lambda)P(I|\lambda)\\ &=\prod_{t=1}^TP(o_t|i_t,\lambda)P(i_T|i_1,\cdots,i_{T-1},\lambda)P(i_1,\cdots,i_{T-1}|\lambda)\\ &=\prod_{t=1}^TP(o_t|i_t,\lambda)P(i_T|i_{T-1},\lambda)P(i_1,\cdots,i_{T-1}|\lambda)=\cdots\\ &=\prod_{t=1}^TP(o_t|i_t,\lambda)P(i_T|i_{T-1},\lambda)P(i_{T-1}|i_{T-2},\lambda)\cdots P(i_2|i_1,\lambda)P(i_1|\lambda)\\ &=\pi_{i_1}b_{i_1}(o_1)a_{i_1,i_2}b_{i_2}(o_2)a_{i_2,i_3}\cdots b_{i_{T-1}}(o_{T-1})a_{i_{T-1},i_T}b_{i_T}(O_T) \end{aligned} P(O,I∣λ)​=P(o1​∣o2​,…,oT​,I,λ)P(o2​,…,oT​,I,λ)=P(o1​∣i1​,λ)P(o2​,…,oT​,I,λ)=P(o1​∣i1​,λ)P(o2​∣i2​,λ)P(o3​,…,oT​,I,λ)=⋯=P(o1​∣i1​,λ)P(o2​∣i2​,λ)⋯P(oT​∣iT​,λ)P(I∣λ)=t=1∏T​P(ot​∣it​,λ)P(I∣λ)=t=1∏T​P(ot​∣it​,λ)P(iT​∣i1​,⋯,iT−1​,λ)P(i1​,⋯,iT−1​∣λ)=t=1∏T​P(ot​∣it​,λ)P(iT​∣iT−1​,λ)P(i1​,⋯,iT−1​∣λ)=⋯=t=1∏T​P(ot​∣it​,λ)P(iT​∣iT−1​,λ)P(iT−1​∣iT−2​,λ)⋯P(i2​∣i1​,λ)P(i1​∣λ)=πi1​​bi1​​(o1​)ai1​,i2​​bi2​​(o2​)ai2​,i3​​⋯biT−1​​(oT−1​)aiT−1​,iT​​biT​​(OT​)​

3. EM算法的M步，极大化Q函数求模型参数

   $$Q(\lambda ,\overline{\lambda })=\sum _{I}log{\pi }_{i_1}P(O,I|\overline{\lambda })+\sum _I(\sum _{t=1}^{T-1}log{a}_{i_t,i_{t+1}}）P(O,I|\overline{\lambda })+\sum _I(\sum _{t=1}^{T}log{b}_{i_t}(o_t)）P(O,I|\overline{\lambda })$$

   三部分分别求最大值

   • 第一部分：

   $$\begin{array}{rl}\sum _I\log {\pi }_{i_1}P(O,I|\overline{\lambda })& =\sum _{i_1}\sum _{i_2}\cdots \sum _{i_T}\log {\pi }_{i_1}P(O,I|\overline{\lambda })\\ & =\sum _{i_1}\log {\pi }_{i_1}\sum _{i_2}\cdots \sum _{i_T}P(O,i_1,i_2,\cdots ,i_T|\overline{\lambda })\\ & =\sum _{i_1}\log {\pi }_{i_1}P(O,i_1|\overline{\lambda })\\ & =\sum _{i=1}^N\log {\pi }_{i}P(O,i_1=q_i|\overline{\lambda })\end{array}$$
   由于${\pi }_i$满足${\sum }_{i=1}^N{\pi }_i=1$，故可应用拉格朗日乘子法，相应的拉格朗日函数为：

   $$\sum _{i=1}^N\log {\pi }_{i}P(O,i_1=q_i|\overline{\lambda })+\gamma (\sum _{i=1}^N{\pi }_i-1)$$
   对${\pi }_i$求偏导

   $$\begin{array}{rl}& \frac{\partial }{\partial {\pi }_i}[\sum _{i=1}^N\log {\pi }_{i}P(O,i_1=q_i|\overline{\lambda })+\gamma (\sum _{i=1}^N{\pi }_i-1)]=0\\ & \frac{1}{{\pi }_i}P(O,i_1=q_i|\overline{\lambda })+\gamma =0\\ & P(O,i_1=q_i|\overline{\lambda })+\gamma {\pi }_i=0\\ & \gamma =\gamma \sum _{i=1}^N{\pi }_i=-\sum _{i=1}^{N}P(O,i_1=q_i|\overline{\lambda })=-P(O|\overline{\lambda })\end{array}$$
   带入可得${\pi }_i=\frac{P(O,i_1=q_i|\overline{\lambda })}{P(O|\overline{\lambda })}={\gamma }_1(i)$

   • 第二部分：

   $$\sum _I(\sum _{t=1}^{T-1}\log a_{i_t,i_{t+1}}）P(O,I|\overline{\lambda })=\sum _{t=1}^{T-1}\sum _{i=1}^N\sum _{j=1}^N\log a_{i_t,i_{t+1}}）P(O,i_t=q_i,i_{t+1}=q_j|\overline{\lambda })$$
   根据${\sum }_{j=1}^N{a}_{ij}=1$，应用拉格朗日乘子法可得：

   $$a_{ij}=\frac{{\sum }_{t=1}^{T-1}P(O,i_t=q_i,i_{t+1}=q_j|\overline{\lambda })}{{\sum }_{t=1}^{T-1}P(O,i_t=q_i|\overline{\lambda })}=\frac{{\sum }_{t=1}^{T-1}{\xi }_t(i,j)}{{\sum }_{t=1}^{T-1}{\gamma }_t(i)}$$

   • 第三部分：

   $$\sum _I(\sum _{t=1}^T\log b_{i_t}(o_t)）P(O,I|\overline{\lambda })=\sum _{t=1}^T\sum _{i=1}^N\log b_i(o_t)P(O,i_t=q_i|\overline{\lambda })$$
   根据${\sum }_{k=1}^K{b}_i(k)=1$，应用拉格朗日乘子法：

   $$\begin{array}{rl}& \frac{\partial }{\partial b_i(k)}[\sum _{t=1}^T\sum _{i=1}^N\log b_i(o_t)P(O,i_t=q_i|\overline{\lambda })+\gamma (\sum _{k=1}^K{b}_i(k)-1)]=0\\ & \sum _{t=1}^T\frac{1}{b_i(o_t)}\frac{\partial b_i(o_t)}{\partial b_i(k)}P(O,i_t=q_i|\overline{\lambda })+\gamma =0\\ & \sum _{t=1}^T\frac{1}{b_i(o_t)}I(o_t=v_k)P(O,i_t=q_i|\overline{\lambda })+\gamma =0\\ & \sum _{t=1}^T\frac{1}{b_i(k)}I(o_t=v_k)P(O,i_t=q_i|\overline{\lambda })+\gamma =0\\ & \sum _{t=1}^{T}I(o_t=v_k)P(O,i_t=q^{}\\ & \\ & \\ & \\ & \end{array}$$