DA（语义分割2）ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation

ADVENT: Adversarial Entropy Minimization for Domain Adaptation  in Semantic Segmentation

来源：CVPR 2019

作者：Tuan-Hung Vu，Himalaya Jain，Maxime Bucher，Matthieu Cord， Patrick P´erez

机构：索邦大学（位于法国巴黎），valeo.ai（位于法国巴黎）

代码:作者在github上放出了代码，https://github.com/valeoai/ADVENT

实现环境：pytorch，All experiments  are done on a single NVIDIA 1080TI GPU with 11  GB memory.

数据集：SYNTHIA，GTA5作为原域，Cityscapes dataset作为目标域

参考博客：https://blog.csdn.net/fuyouzhiyi/article/details/93648323

与众不同之处：

To our knowledge, we are first to  successfully apply entropy based UDA training to obtain competitive performance on semantic segmentation task。

大致框架实现：

two proposed approaches  for entropy minimization using

(i) an unsupervised entropy loss

(ii) adversarial training.

1 动机

基于熵的UDA方法动机

根据图1中的熵图可以看出，原域图片的熵值很小，目标域图片的熵值很大，熵值小意味着模型的预测结果是over-confident，熵值大意味着模型的预测结果是under-confident。作者猜测，减少目标域图片预测结果的熵值可能会缩小原域和目标域之间的差别。

问题：最小化谁的熵？怎么最小化？

最小化目标：目标域图片预测结果的熵值

直接方法：  最小化一个熵损失

间接方法：最小化一个对抗损失

2 主体

整体方法

定义原域的标签为1，目标域的标签为0。

带权重的自信息图

带权重的自信息图的计算方法

由Ix的计算方法可知，它比较像香农熵的计算方法，故可以把它看成香农熵的泛化形式。

自信息的解释

3.损失函数

3.1采用直接的方法，即熵最小化时，最终的优化目标

最终的优化目标

熵损失计算公式

3.2采用间接方法，对抗损失最小化时，最终的优化目标

we introduce a unified adversarial training  framework which indirectly minimizes the entropy by  having target’s entropy distribution similar to the source.

To this end, we formulate the UDA taskas minimizing distribution distance between source and targeton the weighted self-information space.

Our adversarialapproach is motivated by the fact that the trained model naturallyproduces low-entropy predictions on source-like images.By aligning weighted self-information distributionsof target and source domains, we indirectly minimize theentropy of target predictions.

对抗方法总结下来就是在weighted self-information space上，让目标域样本的分布尽可能的接近原域样本的分布，这样做的原因是，模型对于像原域样本的输入，会产生熵比较低的预测。这样算是间接减小了熵。

（9）是优化目标

4.实验结果：

4.1网络结构

本文用Deeplab-V2 [2] as the  base semantic segmentation architecture F.

To better capture the scene context, Atrous Spatial Pyramid Pooling (ASPP) is applied on the last layer’s feature output.

We experiment on the two different base deep CNN architectures: VGG-16  and ResNet-101.

The adversarial network D introduced in Section 3.2 has the same architecture as the one used in DCGAN

实验结果1

实验结果2

5.概念介绍：

1.用于语义分割的非监督域适配的对抗训练

Adversarial training for UDA is the most explored approach for semantic segmentation. It involves two networks.One network predicts the segmentation maps for the input image, which could be from source or target domain,    while  another network acts as a discriminator which takes  the feature maps from the segmentation network and triesto predict domain of the input. The segmentation network  tries to fool the discriminator, thus making the features fromthe two domains have a similar distribution

6.总结别人的工作：

1.

Some methods build on generative networks to generate  target images conditioned on the source. Hoffman et al. [14] propose Cycle-Consistent Adversarial Domain Adaptation  (CyCADA), in which they adapt at both pixel-level  and feature-level representation. For pixel-level adaptation  they use Cycle-GAN [48] to generate target images conditioned on the source images.

2.CL approach

The authors in [50] use generative adversarial networks (GAN) [11] to align the source and target embeddings.Also, they replace the cross-entropy loss by a conservative loss (CL) that penalizes the easy and hard cases of source examples.

The CL approach is orthogonal to most  of the UDA methods, including ours: it could benefit any method that uses cross-entropy for source.

[50] Penalizing  top performers: Conservative loss for semantic segmentation  adaptation. In ECCV, September 2018. 3

3.自训练

Another approach for UDA is self-training. The idea is to use the prediction from an ensembled model or a previous  state of model as pseudo-labels for the unlabeled data to train the current model. Many semi-supervised methods [20, 39] use self-training. In [51], self-training is employed for UDA on semantic segmentation which is further extended with class balancing and spatial prior。

Pseudo-labeling is a simple yet efficient approach for semi-supervised learning [21]. Recently, the approach has been applied to UDA in semantic segmentation task with an iterative self-training (ST) procedure [51].

7.单词与句子

preliminary：adj.     预备性的，初步的;   n.初步行动