DenseNet网络框架

一、DenseNet

Densely Connected Convolutional Networks (密集连接卷积网络)

• 论文链接：https://arxiv.org/pdf/1608.06993.pdf
• 论文代码：
  1、（tf）https://github.com/taki0112/Densenet-Tensorflow
  2、https://github.com/liuzhuang13/DenseNet

二、DenseNet框架

1、DenseNet优势：

• 减轻了vanishing-gradient（梯度消失）
• 加强了feature的传递
• 更有效地利用了feature
• 一定程度上较少了参数数量

2、DenseNet主要框架


DenseNet会先对输入的tensor做一个卷积核大小为[7×7]，步长为2卷积，然后再进行核大小为[3×3]，步长为2的最大池化。 之后，便是DenseNet和transition的交替连接，最后跟一个含有[7×7]全局平局池化、1000的全连接和softmax的分类层。

TensoeFlow代码：

    def Dense_net(self, input_x):
        x = conv_layer(input_x, filter=2 * self.filters, kernel=[7,7], stride=2, layer_name='conv0')
        # x = Max_Pooling(x, pool_size=[3,3], stride=2)


        """
        for i in range(self.nb_blocks) :
            # 6 -> 12 -> 48
            x = self.dense_block(input_x=x, nb_layers=4, layer_name='dense_'+str(i))
            x = self.transition_layer(x, scope='trans_'+str(i))
        """




        x = self.dense_block(input_x=x, nb_layers=6, layer_name='dense_1')
        x = self.transition_layer(x, scope='trans_1')

        x = self.dense_block(input_x=x, nb_layers=12, layer_name='dense_2')
        x = self.transition_layer(x, scope='trans_2')

        x = self.dense_block(input_x=x, nb_layers=48, layer_name='dense_3')
        x = self.transition_layer(x, scope='trans_3')

        x = self.dense_block(input_x=x, nb_layers=32, layer_name='dense_final')



        # 100 Layer
        x = Batch_Normalization(x, training=self.training, scope='linear_batch')
        x = Relu(x)
        x = Global_Average_Pooling(x)
        x = flatten(x)
        x = Linear(x)


        # x = tf.reshape(x, [-1, 10])
        return x

3、Dense Block

在传统的卷积神经网络中，如果你有L层，那么就会有L个连接，但是在DenseNet中，会有L(L+1)/2个连接。简单讲，就是每一层的输入来自前面所有层的输出。如上图：x0是input，H1的输入是x0（input），H2的输入是x0和x1（x1是H1的输出）……

ResNet和DenseNet公式：

• ResNet：ℓ表示层，xℓ表示ℓ层的输出，Hℓ表示一个非线性变换。所以对于ResNet而言，ℓ层的输出是ℓ-1层的输出加上对ℓ-1层输出的非线性变换。
  
• DenseNet：[x0,x1,…,xℓ-1]表示将0到ℓ-1层的输出Feature Map做concatenation；concatenation是做通道的合并，就像Inception那样。而前面ResNet是做值的相加，通道数是不变的。Hℓ包括BN，ReLU和3*3的卷积。
  
  Dense Block的TensorFlow代码：

def dense_block(self, input_x, nb_layers, layer_name):
        with tf.name_scope(layer_name):
            layers_concat = list()
            layers_concat.append(input_x)

            x = self.bottleneck_layer(input_x, scope=layer_name + '_bottleN_' + str(0))

            layers_concat.append(x)

            for i in range(nb_layers - 1):
                x = Concatenation(layers_concat)
                x = self.bottleneck_layer(x, scope=layer_name + '_bottleN_' + str(i + 1))
                layers_concat.append(x)

            x = Concatenation(layers_concat)

            return x

4、Bottleneck_layer

Bottleneck由两个部分组成：[1×1]的卷积组和[3×3]的卷积组，其意义在于[1×1]的卷积层能减少输入的特征图，之后再用[3×3]的卷积核进行处理。

TensorFlow代码：

def bottleneck_layer(self, x, scope):
        # print(x)
        with tf.name_scope(scope):
            x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')
            x = Relu(x)
            x = conv_layer(x, filter=4 * self.filters, kernel=[1,1], layer_name=scope+'_conv1')
            x = Drop_out(x, rate=dropout_rate, training=self.training)

            x = Batch_Normalization(x, training=self.training, scope=scope+'_batch2')
            x = Relu(x)
            x = conv_layer(x, filter=self.filters, kernel=[3,3], layer_name=scope+'_conv2')
            x = Drop_out(x, rate=dropout_rate, training=self.training)

            # print(x)

            return x

5、Transition_layer

Transition_layer是介于两个Dense block之间的转换模块，每一个Dense block输出的feature maps都比较多，如果统统都输入到下一层，将会极大的增加神经网络的参数，所以transition_layer的主要工作就是降维。

TensorFlow代码：

    def transition_layer(self, x, scope):
        with tf.name_scope(scope):
            x = Batch_Normalization(x, training=self.training, scope=scope+'_batch1')
            x = Relu(x)
            # x = conv_layer(x, filter=self.filters, kernel=[1,1], layer_name=scope+'_conv1')
            
            # https://github.com/taki0112/Densenet-Tensorflow/issues/10
            
            in_channel = x.shape[-1]
            x = conv_layer(x, filter=in_channel*0.5, kernel=[1,1], layer_name=scope+'_conv1')
            x = Drop_out(x, rate=dropout_rate, training=self.training)
            x = Average_pooling(x, pool_size=[2,2], stride=2)

            return x