GAN

Tue Aug 6 14:07:09 CST 2019

What I want to know

• What is GAN?
• Why does it work?
• How does learn the data distribution?
• How can it help my project?

Materials

[Thesis] Learning to Synthesize and Manipulate Natural Images (SIG18 best PhD thesis)

[Video] Introduction to GANs, NIPS 2016, Ian Goodfellow, OpenAI (30mins)

[Video] Ian Goodfellow: Generative Adversarial Networks (NIPS 2016 tutorial) (2hrs)

[WebPage] MSRA: 到底什么是生成式对抗网络GAN？

[WebPage] A Beginner's Guide to Generative Adversarial Networks (GANs)

Notes

Basics

GAN网络有一个生成器和一个判别器，生成器的目标是为了学习数据分布，判别器的目标是判定输入的数据是否符合真实分布。

优化方程

网络的两个部分

• input noise -> -> x sampled from , -> -> tries to make , tries to make

• x sampled from data -> ->

Training

最直观的处理办法就是分别对D和g进行交互迭代，固定g，优化D，一段时间后，固定D再优化g，直到过程收敛。

High level understanding

Basically, GAN consists of a generator and a discriminator. The generator takes a feature vector, which in this case is random noise, and outputs a sample that mimics the data distribution. The discriminator takes a sample either from the original data or generated by generator, its job is to successfully distinguish real ones from fake ones.

As shown in the following figure, discriminator is supervised by ground truth label, while the generator is supervised by discriminator.
[图片上传失败...(image-68adc6-1647766988833)]

Looking at it separately, the discriminator does a standard binary classification task, real or fake. The network returns a probability, a number between 0 and 1, with 1 representing a prediction of authenticity and 0 representing fake.

The generator takes a feature and expands it to a data sample. It is the opposite process of discriminator. The goal of a generator is to learn the data distribution, how to tell if you have successfully learned the distribution? The generator generates data from random noise, if the discriminator thinks it is real, the generator can be said to have learned the real distribution of the dataset.

The question a generative algorithm tries to answer is: Assuming this email is spam, how likely are these features (words it contains)? While discriminative models care about the relation between (label) and (data), generative models care about “how you get .” They allow you to capture , the probability of given , or the probability of features given a label or category.

The discriminator, on the other hand, captures , the probability of given . Which is given an email (words it contains), how likely the email is spam.

GANs, Encoder-decoder, Autoencoders and VAEs

The generator in GAN serves the similar function as a decoder in Encoder-decoder network??? What are the differences.

[quote]
You can bucket generative algorithms into one of three types:

• Given a label, they predict the associated features (Naive Bayes)
• Given a hidden representation, they predict the associated features (VAE, GAN)
• Given some of the features, they predict the rest (inpainting, imputation)

Applications

• Same domain
  • super resolution
  • image filling / repairing
• Cross domain transfer
  • 2d to 3d
  • text to image
  • picture style transfer
• Learn joint distribution
  • learn attributes from images

Notes on NIPS tutorial

RoadMap

• Why study generative modeling?
• How do generative models work? How do GANs compare to others?
• How do GANs work?
• Tips and tricks
• Research frontiers
• Combining GANs with other methods