【NIPS2018】Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Language

---

Paper: https://papers.nips.cc/paper/7290-text-adaptive-generative-adversarial-networks-manipulating-images-with-natural-language.pdf

Github: https://github.com/woozzu/tagan

---

 

---

Task:  manipulating images using natural language description

semantically modify visual attributes of an object in an image according to the text describing the new visual appearance

 a sample:

existing methods:

 they do not fully preserve text-irrelevant contents of the original image

a sample: background changed

Our method:

The key to our method is the text-adaptive discriminator that creates word-level local discriminators according to
input text to classify fine-grained attributes independently.

With this discriminator, the generator learns to generate images where only regions that correspond to the
given text are modified.

---

 

 

---

Related task: image-to-image translation, text-to-image synthesis

Network:

每个词都有个local discriminator, 来判断是否与原图relevant. 

---

 

---

卖点：之前没人考虑做保持背景的只改变visual attribute的任务，算是第一人，效果对比【惊艳】。

缺点：没有量化指标，只能靠User Study 和主观判断

---

 

 

---

Loss:

 

 

 

 

 

主要靠reconstruction loss来控制背景的不变

引用自《Arbitrary facial attribute editing: Only change what you want》  submmited to TIP

---