吉布斯采样

吉布斯采样

https://en.wikipedia.org/wiki/Gibbs_sampling
As with other MCMC algorithms, Gibbs sampling generates a Markov chain of samples, each of which is correlated with nearby samples. As a result, care must be taken if independent samples are desired. Generally, samples from the beginning of the chain (the burn-in period) may not accurately represent the desired distribution and are usually discarded. It has been shown, however, that using a longer chain instead (e.g. a chain that is n times as long as the initially considered chain using a thinning factor of n) leads to better estimates of the true posterior. Thus, thinning should only be applied when time or computer memory are restricted.
Gibbs sampling is applicable when the joint distribution is not known explicitly or is difficult to sample from directly, but the conditional distribution of each variable is known and is easy (or at least, easier) to sample from. The Gibbs sampling algorithm generates an instance from the distribution of each variable in turn, conditional on the current values of the other variables. It can be shown that the sequence of samples constitutes a Markov chain, and the stationary distribution of that Markov chain is just the sought-after joint distribution.

受限玻尔兹曼机吉布斯采样

https://blog.csdn.net/xingzhedai/article/details/53068019

个人理解

什么是Gibbs Sampling
Gibbs Sampling是MCMC算法中的一种，用来构造多变量概率分布的随机样本，比如构造两个或多个变量的联合分布，求积分，期望。
为什么需要Gibbs Sampling
这不是废话，肯定是积分，期望或者联合分布很难计算出来，通常情况下当前面三个问题是NP问题时才需要Gibbs Sampling。不然的话，直接计算就可以了嘛，既准确又快速，干嘛还要Gibbs Sampling呢。补充一句Gibbs Sampling只是（也只能）到近似解。
应用场景
a、积分，期望，样本概率很难计算出来；b、条件概率很容易计算。具体一点的例子：受限玻尔兹曼机（RBM）的训练，贝叶斯网络，LDA都用到Gibbs Sampling。
为什么Gibbs Sampling有效
当Gibbs Sapling算法执行多次之后，产生的样本服从真实样本的分布，即相当于直接从联合分布中采样。