【论文笔记】Neighboring Region Dropout for HIC

M. E. Paoletti, J. M. Haut, J. Plaza and A. Plaza, “Neighboring Region Dropout for Hyperspectral Image Classification,” in IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 6, pp. 1032-1036, June 2020, doi: 10.1109/LGRS.2019.2940467.
Paoletti M E , Haut J M , Plaza J , et al. Neighboring Region Dropout for Hyperspectral Image Classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, PP(99):1-5.
github：DeepNRD

1.贡献点

  针对网络容易过拟合，泛化能力不强的问题，提出了neighbor region DO（dropout）策略。

2.论文细节

2.1 传统的dropout

  DO的原理：

Although this hierarchical mechanism is appealing during the training stage, introduce weak links between the neurons of adjacent layers, and hampering the inference step [9].
（CNN的层次结构使用局部连接，这有助于训练，但却妨碍了测试）
In this context, traditional DO [15] is applied between the activation and pooling layers as a regularization method to avoid overfitting and provide some independence between adjacent layers’ neurons, by setting to zero some randomly selected neural activations.
（传统的DO被用于激活层和池化层之间，其能防止过拟合，使得相邻层的神经元保持一定的独立性。
DO改进了反向传播过程，神经元应该以单独的方式进行调整，而不是与其他神经元建立无关的依赖关系。这种方法背后的主要动机是迫使层的神经元自己提取出更强的鲁棒性和显著性的特征。）

  DO的可视化结果：

the DO injects random noise to the feature maps in order to disentangle the behavior of adjacent layers’ neurons. However, this noise is not structured, which makes it not completely effective in the task of removing semantic information of the feature map, where nearby features still contain related spatial information.
（由下图的可视化结果可知，DO向特征图注入随机噪声，以分离相邻层神经元的行为。但是，这种噪声不是结构化的，这使得它在去除特征图的语义信息时不能完全有效，因为附近的特征仍然包含着相关的空间信息。）
（关于这点，个人理解是卷积时采用滑动窗口的形式的，假设步长s为1，则卷积时会有重叠区域A、B等等，若对A dropout，而对B没有dropout，则一部分输入特征还是传到了下一层）

2.2 提出的neighboring region dropout

  NRDO的原理：

the application of a rigorous NRDO with a fixed value of p(l) can negatively affect the performance of the network, while the implementation of a soft NRDO may not provide the desired robustness. In order to overcome the limitations, our model is trained with a p(l) whose value increases linearly and progressively through the epochs [16], from zero probability to the maximum indicated value of p(l), with the goal of progressively adapting the performance, extracting more robust and independent features at each epoch.
（采用固定p(l)值的严格NRDO可能会对网络性能产生负面影响，而采用软NRDO可能无法提供所需的鲁棒性。为了克服这一局限性，我们的模型采用的p(l)的值随着epoch[16]线性递增，从0到p(l)的最大值，以逐步适应性能，在每个epoch提取更鲁棒和独立的特征。）

  NRDO的可视化结果：

3.实验

  文章设计了一个2DCNN网络，检验DO、NRDO、RO（一种数据增强策略）的有效性。

  实验一：比较DO和NRDO（p(l)越大难道代表丢弃的越多？？？伯努利分布不是等于1的概率为p吗）。

It must be noted that, since NRDO occludes entire windows, it prevents the model from seeing all the complete features of the input data, forcing the network to look for more robust parameters.

  实验二：不同算法的对比。

  四种2DCNN的Loss的acc曲线。

  从上图可以发现“raw”随着epoch增加，loss反而增加，即产生了过拟合，而“RO”在最后的epoch中loss也增加了，也是产生了过拟合，而“NRDO”收敛快，表现稳定。

4.单词（我认识你，永远记得你）

今日推荐：楚门的世界 The Truman Show (1998)
2020-6-8 17:20:01
私人生活被暴露在聚光灯下，任由别人的评价操纵着自己的喜怒哀乐，这是不是很多明星患上抑郁症的原因呢。