应该知道的训练GAN的技巧

有经验的朋友知道，训练和微调GANs网络是非常困难的。这里有一些技巧我们应该时刻放在心上。和深度学习很多问题一样，与其说科学不如说是炼金术：这些技巧只是经验，不是理论指导。他们基于当前现象的直观理解，并且证明是可行的。

1. generator的最后一层替代sigmoid而用tanh

2. We sample points from the latent space using a normal distribution (Gaussian distribution), not a uniform distribution.

3. Stochasticity is good to induce robustness. Since GAN training results in a dynamic equilibrium, GANs are likely to get "stuck" in all sorts of ways. Introducing randomness during training helps prevent this. We introduce randomness in two ways: 1) we use dropout in the discriminator, 2) we add some random noise to the labels for the discriminator.

4. Sparse gradients can hinder GAN training. In deep learning, sparsity is often a desirable property, but not in GANs. There are two things that can induce gradient sparsity: 1) max pooling operations, 2) ReLU activations. Instead of max pooling, we recommend using strided convolutions for downsampling, and we recommend using a LeakyReLU layer instead of a ReLU activation. It is similar to ReLU but it relaxes sparsity constraints by allowing small negative activation values.

5. In generated images, it is common to see "checkerboard artifacts" caused by unequal coverage of the pixel space in the generator. To fix this, we use a kernel size that is divisible by the stride size, whenever we use a strided Conv2DTranpose or Conv2D in both the generator and discriminator.