Striving for Simplicity: The All Convolutional Net

http://arxiv.org/abs/1412.6806

CNN网络貌似由简单到复杂，再由复杂到简单的趋势

当前主流的CNN网络一般都包括以下几个模块：convolution and max-pooling layers和交替，最后是少数全链接层。本文对此深入分析，提出了只有卷积层的网络，发现效果不错。

首先来看看 pooling， why pooling can help in CNNs，可能的解释有三个：
1） the p-norm makes the representation in a CNN more invariant
2） the spatial dimensionality reduction performed by pooling makes
covering larger parts of the input in higher layers possible
3） the feature-wise nature of the pooling operation (as opposed to a convolutional layer where features get mixed) could make optimization easier.

假定第二条是最重要的，那么我们可以有2中方法得到去除pooling但保留spatial dimensionality reduction
1）We can remove each pooling layer and increase the stride of the convolutional layer that preceded it accordingly.
2）We can replace the pooling layer by a normal convolution with stride larger than one。（对于一个k=3,r=2的池化层，我们可以用一个对应的stride和kernel的卷积层取代，输入个数输出个数一样。）

第一个方法有一个缺陷： we significantly reduce the overlap of the convolutional layer that preceded the pooling layer. It is equivalent to a pooling operation in which only the top-left feature response is considered and can result in less accurate recognition.

第二个方法则没有这个问题。

对于全链接层， if the image area covered by units in the topmost convolutional layer covers a portion of the image large enough to recognize its content (i.e. the object we want to recognize) then fully connected layers can also be replaced by simple 1-by-1 convolutions.