ADVENT: Adversarial Entropy Minimization for Domain Adaptation? in Semantic Segmentation
來(lái)源：CVPR 2019
作者：Tuan-Hung Vu，Himalaya Jain，Maxime Bucher，Matthieu Cord， Patrick P′erez
機(jī)構(gòu)：索邦大學(xué)（位于法國(guó)巴黎），valeo.ai（位于法國(guó)巴黎）
代碼:作者在github上放出了代碼，https://github.com/valeoai/ADVENT
實(shí)現(xiàn)環(huán)境：pytorch，All experiments? are done on a single NVIDIA 1080TI GPU with 11? GB memory.
數(shù)據(jù)集：SYNTHIA，GTA5作為原域，Cityscapes dataset作為目標(biāo)域
參考博客：https://blog.csdn.net/fuyouzhiyi/article/details/93648323
與眾不同之處：
To our knowledge, we are first to? successfully apply entropy based UDA training to obtain competitive performance on semantic segmentation task。
大致框架實(shí)現(xiàn)：
two proposed approaches? for entropy minimization using
(i) an unsupervised entropy loss
(ii) adversarial training.
1 動(dòng)機(jī)
根據(jù)圖1中的熵圖可以看出，原域圖片的熵值很小，目標(biāo)域圖片的熵值很大，熵值小意味著模型的預(yù)測(cè)結(jié)果是over-confident，熵值大意味著模型的預(yù)測(cè)結(jié)果是under-confident。作者猜測(cè)，減少目標(biāo)域圖片預(yù)測(cè)結(jié)果的熵值可能會(huì)縮小原域和目標(biāo)域之間的差別。
問(wèn)題：最小化誰(shuí)的熵？怎么最小化？
最小化目標(biāo)：目標(biāo)域圖片預(yù)測(cè)結(jié)果的熵值
直接方法：? 最小化一個(gè)熵?fù)p失
間接方法：最小化一個(gè)對(duì)抗損失
2 主體
定義原域的標(biāo)簽為1，目標(biāo)域的標(biāo)簽為0。
由Ix的計(jì)算方法可知，它比較像香農(nóng)熵的計(jì)算方法，故可以把它看成香農(nóng)熵的泛化形式。
3.損失函數(shù)
3.1采用直接的方法，即熵最小化時(shí)，最終的優(yōu)化目標(biāo)
3.2采用間接方法，對(duì)抗損失最小化時(shí)，最終的優(yōu)化目標(biāo)
we introduce a unified adversarial training? framework which indirectly minimizes the entropy by? having target’s entropy distribution similar to the source.
To this end, we formulate the UDA taskas minimizing distribution distance between source and targeton the weighted self-information space.
Our adversarialapproach is motivated by the fact that the trained model naturallyproduces low-entropy predictions on source-like images.By aligning weighted self-information distributionsof target and source domains, we indirectly minimize theentropy of target predictions.
對(duì)抗方法總結(jié)下來(lái)就是在weighted self-information space上，讓目標(biāo)域樣本的分布盡可能的接近原域樣本的分布，這樣做的原因是，模型對(duì)于像原域樣本的輸入，會(huì)產(chǎn)生熵比較低的預(yù)測(cè)。這樣算是間接減小了熵。
4.實(shí)驗(yàn)結(jié)果：
4.1網(wǎng)絡(luò)結(jié)構(gòu)
本文用Deeplab-V2 [2] as the? base semantic segmentation architecture F.
To better capture the scene context, Atrous Spatial Pyramid Pooling (ASPP) is applied on the last layer’s feature output.
We experiment on the two different base deep CNN architectures: VGG-16? and ResNet-101.
The adversarial network D introduced in Section 3.2 has the same architecture as the one used in DCGAN
5.概念介紹：
1.用于語(yǔ)義分割的非監(jiān)督域適配的對(duì)抗訓(xùn)練
Adversarial training for UDA is the most explored approach for semantic segmentation. It involves two networks.One network predicts the segmentation maps for the input image, which could be from source or target domain,??? while? another network acts as a discriminator which takes? the feature maps from the segmentation network and triesto predict domain of the input. The segmentation network? tries to fool the discriminator, thus making the features fromthe two domains have a similar distribution
6.總結(jié)別人的工作：
1.
Some methods build on generative networks to generate? target images conditioned on the source. Hoffman et al. [14] propose Cycle-Consistent Adversarial Domain Adaptation? (CyCADA), in which they adapt at both pixel-level? and feature-level representation. For pixel-level adaptation? they use Cycle-GAN [48] to generate target images conditioned on the source images.
2.CL approach
The authors in [50] use generative adversarial networks (GAN) [11] to align the source and target embeddings.Also, they replace the cross-entropy loss by a conservative loss (CL) that penalizes the easy and hard cases of source examples.
The CL approach is orthogonal to most? of the UDA methods, including ours: it could benefit any method that uses cross-entropy for source.
[50] Penalizing? top performers: Conservative loss for semantic segmentation? adaptation. In ECCV, September 2018. 3
3.自訓(xùn)練
Another approach for UDA is self-training. The idea is to use the prediction from an ensembled model or a previous? state of model as pseudo-labels for the unlabeled data to train the current model. Many semi-supervised methods [20, 39] use self-training. In [51], self-training is employed for UDA on semantic segmentation which is further extended with class balancing and spatial prior。
Pseudo-labeling is a simple yet efficient approach for semi-supervised learning [21]. Recently, the approach has been applied to UDA in semantic segmentation task with an iterative self-training (ST) procedure [51].
7.單詞與句子
preliminary：adj. ? ? 預(yù)備性的，初步的;?? n.初步行動(dòng)
