DDIM的一点笔记

1 Title

        DENOISING DIFFUSION IMPLICIT MODELS（Jiaming Song, Chenlin Meng & Stefano Ermon）

2 Conclusion

        Denoising diffusion probabilistic models (DDPMs) have achieved high quality image generation without adversarial training, yet they require simulating a Markov chain for many steps in order to produce a sample. To accelerate sampling, we present denoising diffusion implicit models (DDIMs), a more efficient class of iterative implicit probabilistic models with the same training procedure as DDPMs. In DDPMs, the generative process is defined as the reverse of a particular Markovian diffusion process. This paper generalize DDPMs via a class of non-Markovian diffusion processes that lead to the same training objective. These non-Markovian processes can correspond to generative processes that are deterministic, giving rise to implicit models that produce high quality samples much faster

3 Good Sentences

        1. A critical drawback of these models is that they require many iterations to produce a high quality sample. For DDPMs, this is because that the generative process (from noise to data) approximates the reverse of the forward diffusion process (from data to noise), which could have thousands of steps; iterating over all the steps is required to produce a single sample, which is much slower compared to GANs, which only needs one pass through a network.(The shortcoming of DDPM and GANs)
        2. Since there are many inference distributions (joints) with the same marginals, we explore alternative inference processes that are non-Markovian, which leads to new generative processes (Figure 1, right). These non-Markovian inference process lead to the same surrogate objective function as DDPM, as we will show below(The improvement of this study had done)
        3.DDIM, on the other hand, is an implicit generative model where samples are uniquely determined from the latent variables. Hence, DDIM has certain properties that resemble GANs  and invertible flows, such as the ability to produce semantically meaningful interpolations.(why superior sample quality compared to DDPM under fewer iterations. )

---

        DDIM是DDPM的改进，主要是速度方面的改进，训练方案与DDPM基本一致。DDIM把DDPM中的马尔可夫前向过程非马尔可夫化了,还给出了逆向过程中的后验概率分布表达式，但是目标函数还是DDPM那个，并没有改变。

值得一提的是DDIM的采样，有一种比较特殊的采样

不同的σ导致不同的生成过程，但是模型其实是同一个，σ只影响采样的结果时，目标函数其实就和DDPM一样。

DDIM加速的原因除了训练过程的非马尔科夫链化以外，还有L1带来的特殊性质导致的采样过程的加速，respacing。关键在于T到0的迭代过程中，可以在序列上找到一个子集，整个生成过程就可以在这个子集上生成样本，而不用在原来的T上训练，质量没有下降太多，而数量减少了，于是就加速了