深度学习第J3周：DenseNet算法实战与解析
目录
一、前言
二、设计理念
三、网络结构
四、与其他算法进行对比
五、使用Pytorch实现DenseNet121
六、使用Tensorflow实现DenseNet网络
1.DenseLayer
2.Block
3.Transition
4.DenseNet
七、实战演练
本文为[365天深度学习训练营]内部限免文章（版权归 *K同学啊* 所有）
作者：[K同学啊]
数据集下载：百度网盘 请输入提取码（提取码：0mhm）
本周任务：
●1.请根据本文 Pytorch 代码，编写出相应的 TensorFlow 代码（建议使用上周的数据测试一下模型是否构建正确）
●2.了解并研究 DenseNet与ResNetV 的区别
●3.改进思路是否可以迁移到其他地方呢（自由探索，虽然不强求但是请认真对待这个哦）
一、前言
在计算机视觉领域，卷积神经网络（CNN）已经成为最主流的方法，比如GoogLenet，VGG-16，Incepetion等模型。CNN史上的一个里程碑事件是ResNet模型的出现，ResNet可以训练出更深的CNN模型，从而实现更高的准确度。ResNet模型的核心是通过建立前面层与后面层之间的“短路连接”（shortcuts，skip connection），进而训练出更深的CNN网络。
今天我们要介绍的是DenseNet模型，它的基本思路与ResNet一致，但是它建立的是前面所有层与后面层的密集连接（dense connection），它的名称也是由此而来。DenseNet的另一大特色是通过特征在channel上的连接来实现特征重用（feature reuse）。这些特点让DenseNet在参数和计算成本更少的情形下实现比ResNet更优的性能，DenseNet也因此斩获CVPR 2017的最佳论文奖。
DenseNet论文
二、设计理念
相比ResNet，DenseNet提出了一个更激进的密集连接机制：即互相连接所有的层，具体来说就是每个层都会接受其前面所有层作为其额外的输入。
图1为ResNet网络的残差连接机制，作为对比，图2为DenseNet的密集连接机制。可以看到，ResNet是每个层与前面的某层（一般是2~4层）短路连接在一起，连接方式是通过元素相加。而在DenseNet中，每个层都会与前面所有层在channel维度上连接（concat）在一起（即元素叠加），并作为下一层的输入。
对于一个 L 层的网络，DenseNet共包含 L(L+1)/2 个连接，相比ResNet，这是一种密集连接。而且DenseNet是直接concat来自不同层的特征图，这可以实现特征重用，提升效率，这一特点是DenseNet与ResNet最主要的区别。
图1：ResNet网络的短路连接机制（其中+代表的是元素级相加操作）
图2：DenseNet网络的密集连接机制（其中c代表的是channel级连接操作）
而对于DesNet，则是采用跨通道concat的形式来连接，会连接前面所有层作为输入，输入和输出的公式是Xl=Hl(X0,X1,...,Xl−1)。这里要注意所有的层的输入都来源于前面所有层在channel维度的concat，用一张动图体会一下：
三、网络结构
CNN网络一般要经过Pooling或者stride>1的Conv来降低特征图的大小，而DenseNet的密集连接方式需要特征图大小保持一致。为了解决这个问题，DenseNet网络中使用DenseBlock+Transition的结构，其中DenseBlock是包含很多层的模块，每个层的特征图大小相同，层与层之间采用密集连接方式。而Transition层是连接两个相邻的DenseBlock，并且通过Pooling使特征图大小降低。图5给出了DenseNet的网路结构，它共包含4个DenseBlock，各个DenseBlock之间通过Transition层连接在一起。
使用DenseBlock+Transition的DenseNet网络
在DenseBlock中，各个层的特征图大小一致，可以在channel维度上连接。DenseBlock中的非线性组合函数 H ( ⋅ )的是 BN + ReLU + 3x3 Conv 的结构，如图所示。另外值得注意的一点是，与ResNet不同，所有DenseBlock中各个层卷积之后均输入 k 个特征图，即得到的特征图的channel数为 k，或者说采用 k 个卷积核。k 在DenseNet称为growth rate，这是一个超参数。一般情况下使用较小的 k （比如12），就可以得到较佳的性能。假定输入层的特征图的channel数为 k 0，那么 l 层输入的channel数为 k 0 + k ( 1 , 2 , … , l − 1 ) ，因此随着层数增加，尽管 k 设定得较小，DenseBlock的输入会非常多，不过这是由于特征重用所造成的，每个层仅有 k 个特征是自己独有的。
由于后面层的输入会非常大，DenseBlock内部可以采用bottleneck层来减少计算量，主要是原有的结构中增加1x1 Conv，如图7所示，即 BN + ReLU + 1x1 Conv + BN + ReLU + 3x3 Conv，称为DenseNet-B结构。其中1x1 Conv得到 4 k 个特征图，它起到的作用是降低特征数量，从而提升计算的效率。
对于Transition层，，它主要是连接两个相邻的DenseBlock，并且降低特征图大小。Transition层包括一个1x1的卷积和2x2的AvgPooling，结构为BN+ReLU+1x1Conv+2x2AvgPooling。另外，Transition层可以起到压缩模型的作用。假定层的上接DenseBlock得到的特征图channels数为m，Transition层可以产生【θm】个特征（通过卷积层），其中 是压缩系数θ∈（0，1]（compression rate）。当 θ=1时，特征个数经过Transition层没有变化，即无压缩，而当压缩系数小于1时，这种结构称为DenseNet-C，文中使用θ=0.5。对于使用bottleneck层的DenseBlock结构和压缩系数小于1的Transition组合结构称为DenseNet-BC。
对于ImageNet数据集，图片输入大小为224×224，网络结构采用包含4个DenseBlock的DenseNet-BC，其首先是一个stride=2的7x7卷积层，然后是一个stride=2的3x3 MaxPooling层，后面才进入DenseBlock。ImageNet数据集所采用的网络配置如表1所示：
四、与其他算法进行对比
ResNet与DenseNet的对比
五、使用Pytorch实现DenseNet121
这里我们采用了Pytorch的框架来实现DenseNet，首先实现DenseBlock中的内部结构，这里是BN+ReLU+1×1Conv+BN+ReLU+3×3Conv结构，最后也加入dropout层用于训练过程。
class _DenseLayer(nn.Module):
def __init__(self, num_input_features, growth_rate, bn_size, drop_rate, efficient=False):
super(_DenseLayer, self).__init__()
self.add_module('norm1', nn.BatchNorm2d(num_input_features)),
self.add_module('relu1', nn.ReLU(inplace=True)),
self.add_module('conv1', nn.Conv2d(num_input_features, bn_size * growth_rate,
kernel_size=1, stride=1, bias=False)),
self.add_module('norm2', nn.BatchNorm2d(bn_size * growth_rate)),
self.add_module('relu2', nn.ReLU(inplace=True)),
self.add_module('conv2', nn.Conv2d(bn_size * growth_rate, growth_rate,
kernel_size=3, stride=1, padding=1, bias=False)),
self.drop_rate = drop_rate
self.efficient = efficient
def forward(self, *prev_features):
bn_function = _bn_function_factory(self.norm1, self.relu1, self.conv1)
if self.efficient and any(prev_feature.requires_grad for prev_feature in prev_features):
bottleneck_output = cp.checkpoint(bn_function, *prev_features)
else:
bottleneck_output = bn_function(*prev_features)
new_features = self.conv2(self.relu2(self.norm2(bottleneck_output)))
if self.drop_rate > 0:
new_features = F.dropout(new_features, p=self.drop_rate, training=self.training)
return new_features实现DenseBlock模块，内部是密集连接方式（输入特征数线性增长）：
class _DenseBlock(nn.Module):
def __init__(self, num_layers, num_input_features, bn_size, growth_rate, drop_rate, efficient=False):
super(_DenseBlock, self).__init__()
for i in range(num_layers):
layer = _DenseLayer(
num_input_features + i * growth_rate,
growth_rate=growth_rate,
bn_size=bn_size,
drop_rate=drop_rate,
efficient=efficient,
)
self.add_module('denselayer%d' % (i + 1), layer)
def forward(self, init_features):
features = [init_features]
for name, layer in self.named_children():
new_features = layer(*features)
features.append(new_features)
return torch.cat(features, 1)实现Transition层，它主要是一个卷积层和一个池化层：
class _Transition(nn.Sequential):
def __init__(self, num_input_features, num_output_features):
super(_Transition, self).__init__()
self.add_module('norm', nn.BatchNorm2d(num_input_features))
self.add_module('relu', nn.ReLU(inplace=True))
self.add_module('conv', nn.Conv2d(num_input_features, num_output_features,
kernel_size=1, stride=1, bias=False))
self.add_module('pool', nn.AvgPool2d(kernel_size=2, stride=2))
实现DenseNet网络：
class DenseNet(nn.Module):
r"""Densenet-BC model class, based on
`"Densely Connected Convolutional Networks" `
Args:
growth_rate (int) - how many filters to add each layer (`k` in paper)
block_config (list of 3 or 4 ints) - how many layers in each pooling block
num_init_features (int) - the number of filters to learn in the first convolution layer
bn_size (int) - multiplicative factor for number of bottle neck layers
(i.e. bn_size * k features in the bottleneck layer)
drop_rate (float) - dropout rate after each dense layer
num_classes (int) - number of classification classes
small_inputs (bool) - set to True if images are 32x32. Otherwise assumes images are larger.
efficient (bool) - set to True to use checkpointing. Much more memory efficient, but slower.
"""
def __init__(self, growth_rate=12, block_config=(16, 16, 16), compression=0.5,
num_init_features=24, bn_size=4, drop_rate=0,
num_classes=10, small_inputs=True, efficient=False):
super(DenseNet, self).__init__()
assert 0 < compression <= 1, 'compression of densenet should be between 0 and 1'
# First convolution
if small_inputs:
self.features = nn.Sequential(OrderedDict([
('conv0', nn.Conv2d(3, num_init_features, kernel_size=3, stride=1, padding=1, bias=False)),
]))
else:
self.features = nn.Sequential(OrderedDict([
('conv0', nn.Conv2d(3, num_init_features, kernel_size=7, stride=2, padding=3, bias=False)),
]))
self.features.add_module('norm0', nn.BatchNorm2d(num_init_features))
self.features.add_module('relu0', nn.ReLU(inplace=True))
self.features.add_module('pool0', nn.MaxPool2d(kernel_size=3, stride=2, padding=1,
ceil_mode=False))
# Each denseblock
num_features = num_init_features
for i, num_layers in enumerate(block_config):
block = _DenseBlock(
num_layers=num_layers,
num_input_features=num_features,
bn_size=bn_size,
growth_rate=growth_rate,
drop_rate=drop_rate,
efficient=efficient,
)
self.features.add_module('denseblock%d' % (i + 1), block)
num_features = num_features + num_layers * growth_rate
if i != len(block_config) - 1:
trans = _Transition(num_input_features=num_features,
num_output_features=int(num_features * compression))
self.features.add_module('transition%d' % (i + 1), trans)
num_features = int(num_features * compression)
# Final batch norm
self.features.add_module('norm_final', nn.BatchNorm2d(num_features))
# Linear layer
self.classifier = nn.Linear(num_features, num_classes)
# Initialization
for name, param in self.named_parameters():
if 'conv' in name and 'weight' in name:
n = param.size(0) * param.size(2) * param.size(3)
param.data.normal_().mul_(math.sqrt(2. / n))
elif 'norm' in name and 'weight' in name:
param.data.fill_(1)
elif 'norm' in name and 'bias' in name:
param.data.fill_(0)
elif 'classifier' in name and 'bias' in name:
param.data.fill_(0)
def forward(self, x):
features = self.features(x)
out = F.relu(features, inplace=True)
out = F.adaptive_avg_pool2d(out, (1, 1))
out = torch.flatten(out, 1)
out = self.classifier(out)
return out 六、使用Tensorflow实现DenseNet网络
1.DenseLayer
class DenseLayer(Model):
def __init__(self,bottleneck_size,growth_rate):
super().__init__()
self.filters=growth_rate
self.bottleneck_size=bottleneck_size
self.b1=BatchNormalization()
self.a1=Activation('relu')
self.c1=Conv2D(filters=self.bottleneck_size,kernel_size=(1,1),strides=1)
self.b2=BatchNormalization()
self.a2=Activation('relu')
self.c2=Conv2D(filters=32,kernel_size=(3,3),strides=1,padding='same')
def call(self,*x):
x=tf.concat(x,2)
x=self.b1(x)
x=self.a1(x)
x=self.c1(x)
x=self.b2(x)
x=self.a2(x)
y=self.c2(x)
return y2.Block
class DenseBlock(Model):
def __init__(self,Dense_layers_num,growth_rate):#Dense_layers_num每个denseblock中的denselayer数，growth
super().__init__()
self.Dense_layers_num=Dense_layers_num
self.Dense_layers=[]
bottleneck_size=4*growth_rate
for i in range(Dense_layers_num):
layer=DenseLayer(bottleneck_size,growth_rate)
self.Dense_layers.append(layer)
def call(self,input):
x=[input]
for layer in self.Dense_layers:
output=layer(*x)
x.append(output)
y=tf.concat(x,2)
return y3.Transition
class Transition(Model):
def __init__(self,filters):
super().__init__()
self.b=BatchNormalization()
self.a=Activation('relu')
self.c=Conv2D(filters=filters,kernel_size=(1,1),strides=1)
self.p=AveragePooling2D(pool_size=(2,2),strides=2)
def call(self,x):
x=self.b(x)
x=self.a(x)
x=self.c(x)
y=self.p(x)
return y 4.DenseNet
class DenseNet(Model):
def __init__(self,block_list=[6,12,24,16],compression_rate=0.5,filters=64):
super().__init__()
growth_rate=32
self.padding=ZeroPadding2D(((1,2),(1,2)))
self.c1=Conv2D(filters=filters,kernel_size=(7,7),strides=2,padding='valid')
self.b1=BatchNormalization()
self.a1=Activation('relu')
self.p1=MaxPooling2D(pool_size=(3,3),strides=2,padding='same')
self.blocks=tf.keras.models.Sequential()
input_channel=filters
for i,layers_in_block in enumerate(block_list):
if i<3 :
self.blocks.add(DenseBlock(layers_in_block,growth_rate))
block_out_channels=input_channel+layers_in_block*growth_rate
self.blocks.add(Transition(filters=block_out_channels*0.5))
if i==3:
self.blocks.add(DenseBlock(Dense_layers_num=layers_in_block,growth_rate=growth_rate))
self.p2=GlobalAveragePooling2D()
self.d2=Dense(1000,activation='softmax')
def call(self,x):
x=self.padding(x)
x=self.c1(x)
x=self.b1(x)
x=self.a1(x)
x=self.p1(x)
x=self.blocks(x)
x=self.p2(x)
y=self.d2(x)
return y
model=DenseNet()
print(model)七、实战演练
在Google colab中实验
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow import keras
print(tf.__version__)导入gpu，我没配成
import matplotlib.pyplot as plt
import numpy as np
import tensorflow as tf
gpus = tf.config.list_physical_devices("GPU")
if gpus:
gpu0 = gpus[0] #如果有多个GPU，仅使用第0个GPU
tf.config.experimental.set_memory_growth(gpu0, True) #设置GPU显存用量按需使用
tf.config.set_visible_devices([gpu0],"GPU")
链接云盘，我存到如下图位置
from google.colab import drive
drive.mount('/content/drive')data_dir = r'./drive/My Drive/365/data/bird_photos'
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
data_dir,
validation_split=0.2, #分割数据集
subset="training", #数据集类型
seed=123,
image_size=(224, 224),
batch_size=32)
val_ds = tf.keras.preprocessing.image_dataset_from_directory(
data_dir,
validation_split=0.2, #分割数据集
subset="validation", #数据集类型
seed=123,
image_size=(224, 224),
batch_size=32)
运行： 
class BottleNeck(keras.Model):
def __init__(self, growth_rate, bn_size = 4, dropout = 0.3):
super().__init__()
self.bn1 = layers.BatchNormalization()
self.relu = layers.Activation("relu"),
self.conv1 = layers.Conv2D(filters=bn_size * growth_rate, kernel_size=(1, 1),
strides=1, padding='same')
self.bn2 = layers.BatchNormalization()
self.conv2 = layers.Conv2D(filters=growth_rate, kernel_size=(3, 3),
strides=1, padding='same')
self.dropout = layers.Dropout(rate=dropout)
self.listLayers = [
self.bn1,
self.relu,
self.conv1,
self.bn2,
self.relu,
self.conv2,
self.dropout
]
def call(self, x):
tem = x
for layer in self.listLayers.layers:
x = layer(x)
return layers.concatenate([tem, x], axis=-1)
class Transition(tf.keras.Model):
def __init__(self, growth_rate):
super().__init__()
self.bn1 = layers.BatchNormalization()
self.relu = layers.Activation('relu')
self.conv1 = layers.Conv2D(filters = growth_rate, kernel_size=(1, 1),
strides = 1, activation = 'relu', padding='same')
self.pooling = layers.AveragePooling2D(pool_size=(2,2), strides = 2,padding='same')
self.listLayers = [
self.bn1,
self.relu,
self.conv1,
self.pooling
]
def call(self,x):
for layer in self.listLayers.layers:
x = layer(x)
return x
class DenseBlock(tf.keras.Model):
def __init__(self, num_layer, growth_rate, bn_size = 4, dropout = 0.3, efficient=False):
super().__init__()
self.efficient = efficient
self.listLayers = []
if self.efficient:
_x = tf.recompute_grad(BottleNeck(growth_rate, bn_size = bn_size, dropout = dropout))
else:_x =BottleNeck(growth_rate, bn_size = bn_size, dropout = dropout)
for _ in range(num_layer):
self.listLayers.append(BottleNeck(growth_rate, bn_size = bn_size, dropout = dropout))
def call(self, x):
for layer in self.listLayers.layers:
x = layer(x)
return x
class DenseNet(tf.keras.Model):
def __init__(self, num_init_feature, growth_rate, block_config, num_classes,
bn_size=4, dropout=0.3, compression_rate=0.5, efficient=False):
super().__init__()
self.num_channels = num_init_feature
self.conv = layers.Conv2D(filters = num_init_feature, kernel_size=7,
strides = 2, padding='same')
self.bn = layers.BatchNormalization()
self.relu = layers.Activation('relu')
self.max_pool = layers.MaxPool2D(pool_size=3, strides=2, padding='same')
self.dense_block_layers = []
for i in block_config[:-1]:
self.dense_block_layers.append( DenseBlock(num_layer =i, growth_rate = growth_rate,
bn_size = bn_size, dropout = dropout, efficient=efficient))
self.num_channels = compression_rate * (self.num_channels + growth_rate * i)
self.dense_block_layers.append( Transition(self.num_channels))
self.dense_block_layers.append( DenseBlock(num_layer =block_config[-1], growth_rate = growth_rate,
bn_size = bn_size, dropout = dropout, efficient=efficient))
self.avgpool = layers.GlobalAveragePooling2D()
self.fc = tf.keras.layers.Dense(units=num_classes, activation=tf.keras.activations.softmax)
def call(self,x):
x = self.conv(x)
x = self.bn(x)
x = self.relu(x)
x = self.max_pool(x)
for layer in self.dense_block_layers.layers:
x = layer(x)
x = self.avgpool(x)
return self.fc(x)打印模型
model = DenseNet(num_init_feature=64,
growth_rate=32,
block_config=[6,12,24,16],
compression_rate=0.5,
num_classes = 4,
dropout=0.0,
efficient=True)
x = tf.random.normal((1,224,224,3))
for layer in model.layers:
x = layer(x)
print(layer.name, 'output shape:\t', x.shape)
训练模型，用adam模型
AUTOTUNE = tf.data.AUTOTUNE
train_ds = train_ds.cache().shuffle(1000).prefetch(buffer_size=AUTOTUNE)
val_ds = val_ds.cache().prefetch(buffer_size=AUTOTUNE)
opt = tf.keras.optimizers.legacy.Adam(learning_rate=0.002,decay=0.01)
model.compile(optimizer=opt,
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
epochs = 10
history = model.fit(train_ds,validation_data=val_ds,epochs=epochs)
最后可以指定图片进行预测
import numpy as np
import matplotlib.pyplot as plt
plt.figure(figsize=(18,3))
plt.suptitle('test')
for images,label in val_ds.take(1):
for i in range(8):
ax=plt.subplot(1, 8, i+1)
plt.imshow(images[i].numpy().astype("uint8"))
img_array=tf.expand_dims(images[i],0)
predictions=model.predict(img_array)
plt.title([np.argmax(predictions)])
plt.axis('off')
