Gradient-weighted Class Activation Mapping - Grad-CAM

CAM

“Learning Deep Features for Discriminative Localization. In CVPR, 2016” propesed a technique called Class Activation Mapping(CAM) for identifying discriminative regoins used by a restricted class of classification CNNs which do not contain any fully-connected layers.
But this work needs to altering nerwork architecture which requires all net layers to be trained again.

Figure 1: The structure of CAM

CAM has to modify the base network to remove all fully conneted layers at the end, and includiung a tensor product(followed by softmax), which takes as input the Global-Average-Pooling convolutional feature maps, and output the probability for each class. Since I'm not talking about the CAM, so I will neglect the details of it.

Grad-CAM

• Grad-CAM可以有哪些应用？
  (1)CNNs with fully-connected layers(VGG)
  (2)CNNs used for structured outputs(caption)
  (3)CNNs used in tasks with multi-modal inputs(VQA) or reinforcement learning.

• What is guided back-propagation, and why need it?
  intuition: Neglect all the negative influnences(gradients) and focus only on the positive influences(gradients)
  This gives a better picture of the true influence of the input.
  简单来说，就是将forward propagation 和 backward propagation为负数的梯度值设置为0，避免负数的影响。
  https://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf

• What is grad-cam?
  gradient-weighted class activation mapping (Grad-CAm), uses the gradient of any target concept(say logits for 'dog’or even a caption), flowing into the final convolutional layer to produce a coarse localization map highlighting the importance regoins in the image for predicting the concept.

Figure 2: Grad-CAM overview

(1)给定一张图片和一个label（例如：‘tiger cat’ 或者是其他任一种不同的输出）作为输入，将imgage前向传播通过CNN部分后，通过"task-specific computations’(即选择一种任务), 获得输出结果。
(2)将target class的梯度设为1，其他类别的梯度都设为0.
(3)设置完梯度以后进行反向传播，分别获得一个Grad-CAM localization(Figure2中蓝色heatmap)和一个Guided backprop图像
(4)通过点乘的方式，将蓝色heatmap和Guided backprop合成Guided Gram-CAM

• 为什么grad-CAM需要Guided back-propagation?
  Because Grad-CAM lack the ability to show fine-grained importance like pixel-space gradient visualization methods.

• Grad-CAM 详细推倒过程

[外链图片转存失败(img-yuxGs5kF-1568473585401)(https://ftp.bmp.ovh/imgs/2019/09/edba49625ea6a290.png)]

[外链图片转存失败(img-xaF8MZep-1568473585402)(https://ftp.bmp.ovh/imgs/2019/09/59ac34faf1e7a8ec.png)]

• 结果

Figure 3