[Auto-Aug] Scale-aware Automatic Automentation for Object Detection(CVPR. 2021)

1. Motivation

• 这篇文章主要关注于目标检测中的数据增强。

  This paper focuses on data augmentation for object detection.

• 之前的工作，对于如何将尺度适应性融入网络的方法主要来源与网络的结构（FPN）以及数据增强。

• Previous work handles this challenge which brings the scale adaptation to the network efficiently mainly from two aspects, namely network architecture and data augmentation.

• Though inspiring performance gain has achieved, these data augmentation strategies usually rely on heavy expert experience.

• Thus, scale-aware property and efficiency issue are essential to address for searching augmentation in box-level tasks.

2. Contribution

• In this paper, we propose a new way to automatically learn scale-aware data augmentation strategies for object detection and relevant box-level tasks.

• We first introduce scale-awareness to the search space from two image-levels and box-levels.

  For image-level augmentations, zoom-in and -out operations are included with their probabilities and zooming ratios for search.

  For box-level augmentations, the augmenting areas are generalized with a new searchable parameter, i.e., area ratio.

3. Method

The designed scale-aware search space contains both image-level and box-level augmentations.

3.1 Scale-aware Search Space

3.1.1 Image-Level augmentations

3.1.2 Box-Level augmentations

引入可学习的参数 area-ratio

• Different from [51], the proposed approach further smooths the augmentations and relaxes it to contain learnable factors, i.e., area ratio.

传统的方法会带来2个问题，一个是原始图像和增强图像的边界的gap，另一个是训练和测试时候的gap。

• previous box-level augmentation [51] works exactly bring two issues, which generate an obvi- ous boundary gap between the augmented and original region, and which brings the gap between training and inference.

为了解决这个问题，作者将原始的矩形增强替换为了高斯方法的增强。

• To solve this issue, we extend the original rectangle augmentation to a gaussian-based manner.

增强区域A的公式如下，其中α表示Gaussian map，I和T分别表示Input以及Transform。作者认为这种disigned gaussian-based方法可以soften边界的gap。

• The second issue in previous operations is the lack of considering receptive fields and object scales.

如表1所示，如果将COCO 验证集中的context信息（也就是background）删除，那么APs会降低，但是APl会增大。这就给了作者一个启发，仅仅对boxes内或外增强并不能解决所有尺度的物体。因此作者引出了area ratio这个参数，使得增强区域自适应物体的大小。

• This motivates us that augmentations merely inside/outside object boxes may not deal with objects in all scales appropriately.
• To this end, we introduce a searchable parameter, area ratio, which makes the aug area adaptive to object sizes.

因此，高斯图可以被box annotations(xc, yc, h, w)进一步表示为公式3：

augmentation area V用公式4表示：

3.1.3 Search space summary

总体的搜索空间是$(6^2{)}^2\times (((6\times 6^2)\times (8\times 6^2){)}^5\times 10^3)=1.2^{30}$。

3.2 Scale-aware Estimation Metric

• The evaluation metric is established based on an observation that balanced optimization over different scales would be beneficial to training.

3.3 Search Framework

4. Experiment