[深度学习论文笔记][Visualizing] Deep Inside Convolutional Networks Visualising Image Classification

Simonyan, Karen, Andrea Vedaldi, and Andrew Zisserman. “Deep inside convolutional networks: Visualising image classification models and saliency maps.” arXiv preprint arXiv:1312.6034 (2013). (Citations: 227).

1 Deconv Approach
Compute gradient of class k wrt the image using backpropagation.

1.1 Class Saliency Map
The class saliency map M ∈ R^D0 ×H0 ×W0 is

The white areas indicate that if you jitter the pixels of the object, they will influence the class score in some way.

1.2 Weakly Supervised Object Localization by Class Saliency Map
Use GraphCut colour segmentation on top of class saliency map. The saliency map might capture only the most discriminative part of an object, so saliency tresholding might not be able to highlight the whole object. Therefore, we use GraphCut to capture the color continuity cues.

1.3 Result
See Fig. 7.2.

2 Optimization Approach
2.1 Goal

Generate an image that maximizes some class score by solving 

The optimization is performed wrt the image instead of the weights. The weights are fixed during optimization.

We used the unnormalized class scores s k , rather than class posteriors 

The reason is that the maximization of the class posterior can be achieved by minimizing the scores of other classes. Therefore, we optimise sk to ensure that the optimisation
concentrates only on the class k.

2.2 Method
1 Initialize the optimization with X = 0 .
2 Set the gradient of scores vector ∂s_k/∂⃗s = [0 . . . 1 . . . 0] , then backprop to image to get ∂s_k/∂X .
3 Do a small image update.
4 Forward the image through the network.
5 Go back to 2.
6 After the optimization, add the training set mean image to the result.

2.3 Result

The result can be seen in Fig. 7.3. One goose is good, ten goose is much likely good.

3 References
[1]. F.-F. Li, A. Karpathy, and J. Johnson. http://cs231n.stanford.edu/slides/winter1516_lecture9.pdf.