A Survey on Diffusion Model Through SDE

• 该Survey为笔者在Research & Development Center, AI Lab, Sony(China)实习期间所写。

• 该Survey从Diffusion Model最本质的物理模型出发，以生成式模型(Generative Models)要解决的核心问题为起点，沿着SDE参数确定、前向SDE、反向SDE、求解SDE，优化求解过程的思路进行总结，概述了Diffusion Model的Baseline模型以及Milestone式的优化方案。由于大多数文章都是从创新点讲起，而并不直接说出其思路背后对应的物理模型，因此刚接触Diffusion Model不久的读者可能会感到困惑，笔者希望本文能够对Diffusion Model的理解与学习带来一些灵感或启迪。

• 本文默认读者已经了解Diffusion Model最基本的DDPM[1]、NCSN[2]、Score-based Model[3]等文章，并对扩散模型有基本的认识。

目录如下：

1. Diffusion Model and SDE

2. Discretizing Forward SDE

3. Discretizing Reverse SDE

4. Probability Flow ODE

5. Accelerating Sampling Process

[1] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 6840–6851.

[2] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution.In Advances in Neural Information Processing Systems, pp. 11895–11907, 2019

[3] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in International Conference on Learning Representations, 2021.