变分推断variational Inference

十一、变分推断

1.背景

$$\{\begin{array}{l}频率角度，优化问题\{\begin{array}{l}回归\{\begin{array}{l}Model\\ 策略\\ 算法\{\begin{array}{l}解析解\\ 数值解\end{array}\end{array}\\ SVM\\ EM\\ 等等\end{array}\\ 贝叶斯角度，积分问题\{\begin{array}{l}贝叶斯Infernece(求贝叶斯后验)\\ P(\theta \mid x)=\frac{P(x\mid \theta )P(\theta )}{P(x)}\\ 贝叶斯决策(预测，最后还是求贝叶斯后验)\\ P(\tilde{x}\mid x)={\int }_{\theta }P(\tilde{x},\theta \mid x)d\theta ={\int }_{\theta }P(\tilde{x}\mid \theta )P(\theta \mid x)d\theta =E_{\theta \mid x}[P(\tilde{x}\mid \theta )]\end{array}\end{array}$$

$$Inference\{\begin{array}{l}精确推断（后验简单）\\ 近似推断/近似推断的期望(参数空间、隐变量非常复杂)\\ \{\begin{array}{l}确定性近似\to VI\\ 随机近似\to MCMC,MH,Gibbs\end{array}\end{array}$$

2.公式推导

$x$:observed data
$z$:later variable + parameter
$(x,z)$:complete data

ELBO + KL
$$\log P(x)=L(q)+KL(q||p)$$
$\hat{q}(z)=\arg {\max }_{q(z)}L(q)\to \hat{q}(z)\approx p(z\mid x)$

基于物理的平均场理论
$q(z)={\prod }_{i=1}^M{q}_i(z_i)$，计算时固定一维$q_j(z_j)$
$$L(q)={\int }_{z}q(z)\log P(x,z)dz-{\int }_{z}q(z)\log q(z)dz$$

$$\begin{array}{rl}{\int }_{z}q(z)\log P(x,z)dz& ={\int }_z\prod _{i=1}^M{q}_i(z_i)\log P(x,z)dz\\ & ={\int }_{z_j}{q}_j(z_j)d{z}_j\left({\int }_{z_i}\prod _i^M{q}_i(z_i)\log P(x,z)d{z}_i\right)(i\ne j)\\ & ={\int }_{z_j}{q}_j(z_j)d{z}_j{E}_{{\prod }_i^M{q}_i(z_i)}[\log P(x,z)]\\ {\int }_{z}q(z)\log q(z)dz& ={\int }_z\prod _{i=1}^M{q}_i(z_i)\log \prod _{i=1}^M{q}_i(z_i)dz\\ & ={\int }_z\prod _{i=1}^M{q}_i(z_i)\sum _{i=1}^M\log q_i(z_i)dz\\ & ={\int }_z\prod _{i=1}^M{q}_i(z_i)[\log q_1(z_1)+\log q_2(z_2)+\cdots +\log q_M(z_M)]dz\\ & =\sum _{i=1}^M{\int }_{z_i}{q}_i(z_i)\log q_i(z_i)d{z}_i\\ & ={\int }_{z_j}{q}_j(z_j)\log q_j(z_j)d{z}_j+C\\ L(q)& ={\int }_{z_j}{q}_j(z_j)\log \frac{\hat{p}(x,z_j)}{q_j(z_j)}d{z}_j\\ & =-KL(q_j||\hat{p}(x,z_j))\le 0\end{array}$$

用EM算法求解含隐变量的极大似然估计，极大似然估计是关于后验概率的函数，将不等式划等号q=p，一般p很复杂不易求得，最小化KL可以得到最优解$\hat{q}$，基于平均场理论，使用相互独立的${\prod }_{i=1}^M{q}_i(z_i)$近似推断后验p

3.再回首

VI (mean field)$\to$Classical VI

Assumption:$q(z)={\prod }_{i=1}^M{q}_i(z_i)$
$$\begin{array}{rl}\log q_j(z_j)& =E_{{\prod }_i{q}_i(z_i)}[\log P_{\theta }(x^{(i)},z)]+C\\ & ={\int }_{q_1}\cdots {\int }_{q_{j-1}}{\int }_{q_{j+1}}\cdots {\int }_{q_M}{q}_1\cdots q_{j-1}{q}_{j+1}\cdots q_M[\log P_{\theta }(x^{(i)}),z]d{q}_1\cdots d{q}_{j-1}d{q}_{j+1}\cdots d{q}_M\end{array}$$

目标函数：
$$\begin{array}{rl}& \hat{q}=\arg \underset{q}{\min }KL(q||p)=\arg \underset{q}{\max }L(q)\\ & {\hat{q}}_1(z_1)={\int }_{q_2}\cdots {\int }_{q_M}{q}_2\cdots q_M[\log P_{\theta }(x^{(i)}),z]d{q}_2\cdots d{q}_M\\ & {\hat{q}}_2(z_2)={\int }_{{\hat{q}}_1}\cdots {\int }_{q_M}{\hat{q}}_1\cdots q_M[\log P_{\theta }(x^{(i)}),z]d{\hat{q}}_1\cdots d{q}_M\\ & {\hat{q}}_M(z_M)={\int }_{{\hat{q}}_1}{\int }_{{\hat{q}}_2}\cdots {\int }_{{\hat{q}}_{M-1}}{\hat{q}}_1{\hat{q}}_2\cdots {\hat{q}}_{M-1}[\log P_{\theta }(x^{(i)}),z]d{\hat{q}}_1{\hat{q}}_2\cdots d{\hat{q}}_{M-1}\end{array}$$

类似于坐标上升梯度上升，收敛终止

Classical VI存在的问题：

• 假设太强
• intractable(依然要求积分)

4.SGVI

随机梯度变分推断

不再求$q(z)$的具体值，假设$q(z)$服从某种分布，求这个分布的参数$\varphi$

BELO
$$\begin{array}{rl}L(\varphi )& =E_{q_{\varphi }(z)}\left[\log \frac{P_{\theta }(x^{(i)},z)}{q_{\varphi }(z)}\right]\\ \hat{\varphi }& =\arg \max L(\varphi )\\ {\nabla }_{\varphi }L(\varphi )& ={\nabla }_{\varphi }{E}_{q_{\varphi }(z)}\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]\\ & ={\nabla }_{\varphi }\int q_{\varphi }(z)\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]dz\\ & =\int {\nabla }_{\varphi }{q}_{\varphi }(z)\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]dz\\ & +\int q_{\varphi }(z){\nabla }_{\varphi }\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]dz\\ & =\int q_{\varphi }(z){\nabla }_{\varphi }\log q_{\varphi }(z)\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]dz-\int {\nabla }_{\varphi }{q}_{\varphi }(z)dz\\ & =E_{q_{\varphi }(z)}\left[{\nabla }_{\varphi }\log q_{\varphi }(z)\left(\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right)\right]\end{array}$$
因此可以用MC，从$q_{\varphi }(z)$中采样，根据大数定理，用均值近似期望$E$

$z^{(l)}\sim q_{\varphi }(z),l=1,2,\cdots ,L$
$$\approx \frac{1}{L}\sum _{i=1}^L{\nabla }_{\varphi }\log q_{\varphi }(z^{(l)})\log P_{\theta }(x^{(i)},z^{(l)}-\log q_{\varphi }(z^{(l)}))$$

存在的问题：
  在于这部分${\nabla }_{\varphi }\log q_{\varphi }(z)$，当采样到的值接近于0时，在对数log中变化很快(很敏感，方差很大)，需要更多的样本，才能比较好的近似；
yon用期望近似$q_{\varphi }(z)$的梯度，而我们的目标函数是$\hat{\varphi }$，因此误差是非常大的。

Reparameterization Trick 重参化技巧

$${\nabla }_{\varphi }L(\varphi )={\nabla }_{\varphi }{E}_{q_{\varphi }(z)}\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]$$
  期望是关于$q_{\varphi }(z)$，$q_{\varphi }(z)$和$\varphi$有关系，函数也和$\varphi$有关系，导致复杂度很高。为简化问题，假设$q_{\varphi }(z)$和$\varphi$没有关系，用一个确定的分布$p(\epsilon )$替代$q_{\varphi }(z)$，就可以对直接对函数求导，不用对期望求导。$z\sim p_{\varphi }(z\mid x)$，引入重参化技巧把$z$和$\varphi$的关系解耦。

假设$z=g_{\varphi }(\epsilon ,x^{(i)}),\epsilon \sim p(\epsilon )$，$z$和$\epsilon$为映射关系，各自的积分为1，有如下关系：
$$\left|p_{\varphi }(z\mid x^{(i)})dz\right|=\left|p(\epsilon )d\epsilon \right|$$
$$\begin{array}{rl}{\nabla }_{\varphi }L(\varphi )& ={\nabla }_{\varphi }\int \left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]q_{\varphi }(z)dz\\ & ={\nabla }_{\varphi }\int \left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]q(\epsilon )d\epsilon \\ & ={\nabla }_{\varphi }{E}_{p(\epsilon )}\left[\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right]\\ & =E_{p(\epsilon )}\left[{\nabla }_{\varphi }\left(\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z)\right)\right]\\ & =E_{p(\epsilon )}\left[{\nabla }_z\left(\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z\mid x^{(i)})\right)\cdot {\nabla }_{\varphi }z\right]\\ & =E_{p(\epsilon )}\left[{\nabla }_z\left(\log P_{\theta }(x^{(i)},z)-\log q_{\varphi }(z\mid x^{(i)})\right)\cdot {\nabla }_{\varphi }{g}_{\varphi }(\epsilon ,x^{(i)})\right]\end{array}$$