公式编辑

一、 标题CSDN 中编写公式：

1.我们定义(行内公式：$...$)$f(x)={\sum }_{i=0}^N{\int }_a^{b}g(t,i)\text{ d}t$.

2.定义$f(x)$如下（行间公式）:$$ ... $$
$$f(x)=\sum _{i=0}^N{\int }_a^{b}g(t,i)\text{ d}t\text{\hspace{1em}(1)}$$

二、KL divergence

$$\begin{array}{r}KL(p||q)=\sum p(x)\log \frac{p(x)}{q(x)}\\ KL(p||q)=\int p(x)\log \frac{p(x)}{q(x)}dx\end{array}$$

三、Variational Graph Auto-Encoders

$$\begin{array}{rl}KL(q(z)||p(z|X))& =\int q(z)\log \frac{q(z)}{p(z|X)}dz\\ & =\int q(z)[\log q(z)-\log p(z|X)]dz\\ & =\int q(z)[\log q(z)-\log p(X|z)-\log (z)+\log p(X)]dz\\ & =\int q(z)[\log q(z)-\log p(X|z)-\log p(z)]dz+\log p(X)\end{array}$$
$$\begin{array}{r}\log p(X)-KL(q(z)||p(z|X))=\int q(z)\log p(X|z)dz-KL(q(z)||p(z))\end{array}$$
我们虽然不大容易求出p(X)，但我们知道当X给定的情况下，p(X)是个固定值。那么如果我们希望KL(q(z)||p(z|X))尽可能地小，也就相当于让等号右边的那部分尽可能地大。

Given generative model
$$p(A|Z)=\prod _{i=1}^N\prod _{j=1}^{N}p(A_{ij}|z_i,z_j)\text{\hspace{1em}(2)}$$,with$$p(A_{ij}=1|z_i,z_j)=\sigma (z_i^{\mathrm{T}}{z}_j)\text{\hspace{1em}(3)}$$
Learning We optimize the variational lower bound $\mathcal{L}$ w.r.t. the variatiomnal parameters $W_i$：
$$\mathcal{L}={\mathbb{E}}_{q(Z|X,A)}[\log p(A|Z)]-KL[q(Z|X,A)||p(Z)]\text{\hspace{1em}(4)}$$
where $KL[q(\cdot )||p(\cdot )]$ is the Kullback-Leibler divergence between $q(\cdot )$ and $p(\cdot )$.We further take a Gaussian prior $p(Z)={\prod }_{i}p(z_i)={\prod }_i\mathcal{N}(z_i|0,1)$. For very sparse $A$, it can be beneficial to re-weight terns with $A_{ij}=1$ in $\mathcal{L}$ or alternatively sub-sample terms with $A_{ij}=0$.We choose the former for the following experiments. We perform full-batch gradient descent and make use of the reparameterization trick for training. For a featureless approach, we simply drop the dependence on $X$ and replace $X$ with the identity matrix in the GCN.
The Eq.4 second term on the right:
$$\begin{array}{rl}\int q_{\theta }(z)\log p(z)dz& =\int \mathcal{N}(z;\mu ,{\sigma }^2)\log \mathcal{N}(z;0,1)dz\\ & =-\frac{J}{2}\log (2\pi )-\frac{1}{2}\sum _{j=1}^J({\mu }_j^2+{\sigma }_j^2)\end{array}$$
And
$$\begin{array}{rl}\int q_{\theta }(z)\log q_{\theta }(z)dz& =\int \mathcal{N}(z;\mu ,{\sigma }^2)\log \mathcal{N}(z;\mu ,{\sigma }^2)dz\\ & =-\frac{J}{2}\log (2\pi )-\frac{1}{2}\sum _{j=1}^J(1+\log {\sigma }_j^2)\end{array}$$
Therefore:
$$\begin{array}{rl}-D_{KL}((q_{\varphi }(z)||p_{\varphi }(z))& =\int q_{\theta }(z)(\log p_{\theta }(z)-\log q_{\theta }(z))dz\\ & =\frac{1}{2}\sum _{j=1}^J(1+\log (({\sigma }_j{)}^2)-({\mu }_j{)}^2-({\sigma }_j{)}^2)\end{array}$$
Here,the variational lower bound (the objective to be maximized) contains a KL term that can often be integrated analytically. Here we give the solution when both the prior $p_{\theta }(z)=\mathcal{N}(0;I)$ and the posterior approximation $q_{\varphi }(z|x^i)$ are Gaussian. Let $J$ be the dimensionality of z. Let $\mu$and $\sigma$ denote the variational mean and s.d. evaluated at datapoint $i$, and let ${\mu }_j$ and ${\sigma }_j$ simply denote the j-th element of these vectors.
The Eq.4 first term on the right:
$$\begin{array}{rl}{\mathbb{E}}_{q(Z|X,A)}[\log p(A|Z)]& =\int q(Z|X,A)\log p(A|Z)dZ\\ & =\int \prod _{i=1}^{N}q(z_i|X,A)\log p(A|Z)dZ\\ & =\int \prod _{i=1}^{N}q(z_i|X,A)\prod _i^N\prod _j^N\log p(A_{ij}|z_i,z_j)dZ\\ & =\int \prod _{i=1}^N\mathcal{N}(z_i|{\mu }_i,diag({\sigma }_i^2))\prod _{i=1}^N\prod _{j=1}^N\log \sigma (z_i^{\mathrm{T}}{z}_j)dZ\\ & =\int \prod _{i=1}^N\mathcal{N}({\eta }_i|0,1)f({\eta }_i;W)d{\eta }_i\\ & =\frac{1}{S}\sum _{s=1}^{S}f({\eta }_i;W)\end{array}$$

where
$$\begin{array}{r}{\eta }_i=(z_i-{\mu }_i(W))/{\sigma }_i(W)\sim N(\eta |0,1)\end{array}$$
$$\begin{array}{rl}{\mathbb{E}}_{q(\mathbf{Z}\mid \mathbf{X},\mathbf{A})}\left[\log p(\mathbf{y},\mathbf{A}\mid \mathbf{Z})\right]& ={\mathbb{E}}_{q(\mathbf{Z}\mid \mathbf{X},\mathbf{A})}\left[\log [p(\mathbf{y}\mid \mathbf{Z})p(\mathbf{A}\mid \mathbf{Z})]\right]\\ & ={\mathbb{E}}_{q(\mathbf{Z}|\mathbf{X},\mathbf{A})}[\log p(\mathbf{y}|\mathbf{Z})]+{\mathbb{E}}_{q(\mathbf{Z}|\mathbf{X},\mathbf{A})}[\log p(\mathbf{A}\mid \mathbf{Z})\\ {\mathbb{E}}_{q(Z|X,A)}[\log p(A|Z)]& \approx \frac{1}{L}\sum _{l=1}^L(\log p_{\theta }{x}^{(i)}|z^{(i,l)})\end{array}$$
proof of The reparameterization trick
我们调用了另一种从$q_{\varphi }(z|x)$生成样本的方法，基本参数化非常简单。让$z$ 作为一个连续的随机变量，并且服从$z\sim q_{\varphi }(z|x)$ 条件分布。通常可以将随机变量表示为确定性变量$z=g_{\varphi }(ϵ,x)$,这里的 $ϵ$ 是一个具有独立边际的辅助变量$p(ϵ)$.