随着网络深度的增加,精确度变得饱和,然后迅速退化。出乎意料是,这种退化并不是由于模型的过拟合造成的,而且在适当深度的模型中增加更多的层会导致更高的训练误差,为了训练更深层次的神经网络,提出了一种全新的网络,叫做深度残差网络。深度残差网络获得了ILSVRC & COCO 2015 竞赛第一名,而且在ImageNet检测、ImageNet定位、COCO检测以及COCO分割上均获得了第一名的成绩。
一个残差块包含两部分:identity mapping 和 residual mapping,其中identity mapping指的是曲线部分,resdual mapping表示的是非曲线部分,
形式上,将期望的底层映射为H(x),将堆叠的非线性层拟合另一个映射F(x)=H(x) - x。原始的映射就变成F(x) + x,这里假设残差映射比原始的,为参考的映射更容易优化。
残差模块具有两种形式如上图所示:左边适用于小型网络结构,右图适用于较深的网络结构主要用来降低网络的计算量和参数。
右图主要使用两个1*1的卷积和一个3*3的卷积,1*1的卷积先对特征图进行降维,再增加特征图的维度。
使用瓶颈结构的残差块参数为:1 * 1 * 256 * 64 + 3 * 3 * 64 * 64 + 1 * 1 * 64 * 256 = 69632
不使用瓶颈结构的残差块参数为:3 * 3 * 256 * 256 * 2 = 1179648
参数大概相差了17倍。
这里平原网络的基准主要受到VGG网络的启发,遵循两个简单的设计原则:
(1)相同的输出特征图尺寸,层的卷积核数量也是相同的。
(2)特征图的尺寸减半,卷积核的数量就加倍,保证每层的时间复杂度相同。
通过strides为2的卷积层来降低特征图的尺寸,如下图中间的网络所示。
残差网络是在平原网络的基础上增加短路连接。当输入和输出维度相同时,可以shortcut可以直接使用;不相同时,需要进行维度匹配。其中主要的做法如下:
(1)采用零填充来增加维度;
(2)通过1 * 1 的卷积层来增加维度;
主要的残差网络有resnet50/101/152:
resnet50主要由五个部分组成:
使用keras对ResNet50网络进行复现:网络主要包括五个阶段.
from keras.layers import Input
from keras.layers import MaxPooling2D, GlobalAveragePooling2D, ZeroPadding2D
from keras.layers import Conv2D, BatchNormalization, Activation, add, Dense
from keras.models import Model
from keras.utils.vis_utils import plot_model
def conv_block(input_tensor, kernel_size, filters, stage, block, strides):
assert len(filters) == 3
filter1, filter2, filter3 = filters
conv_name_base = 'conv'+ str(stage) + block + '_branch'
bn_name_base = 'bn' + str(stage) + block + '_branch'
# 1 * 1
x = Conv2D(filter1, kernel_size=(1, 1),
strides=strides, # stage2, strides=1; stage345, strides=2
kernel_initializer='he_normal',
name=conv_name_base + '2a')(input_tensor)
x = BatchNormalization(axis=3, name=bn_name_base + '2a')(x)
x = Activation('relu')(x)
# 3 * 3
x = Conv2D(filter2, kernel_size, padding='same',
kernel_initializer='he_normal', name=conv_name_base + '2b')(x)
x = BatchNormalization(axis=3, name=bn_name_base + '2b')(x)
x = Activation('relu')(x)
# 1 * 1
x = Conv2D(filter3, kernel_size=(1, 1),
kernel_initializer='he_normal',
name=conv_name_base + '2c')(x)
x = BatchNormalization(axis=3, name=bn_name_base + '2c')(x)
shortcut = Conv2D(filter3, kernel_size=(1, 1),
strides=strides,
kernel_initializer='he_normal',
name=conv_name_base + '1')(input_tensor)
shortcut = BatchNormalization(axis=3, name=bn_name_base + '1')(shortcut)
x = add([x, shortcut])
x = Activation('relu')(x)
return x
def identity_block(input_tensor, kernel_size, filters, stage, block):
assert len(filters) == 3
filter1, filter2, filter3 = filters
conv_name = 'res' + str(stage) + block + '_barnch'
bn_name = 'bn' + str(stage) + block + '_branch'
# 1 * 1
x = Conv2D(filter1, kernel_size=(1, 1),
kernel_initializer='he_normal',
name=conv_name + '2a')(input_tensor)
x = BatchNormalization(axis=3, name=bn_name + '2a')(x)
x = Activation('relu')(x)
# 3 * 3
x = Conv2D(filter2, kernel_size,
padding='same',
kernel_initializer='he_normal',
name=conv_name + '2b')(x)
x = BatchNormalization(axis=3, name=bn_name + '2b')(x)
x = Activation('relu')(x)
# 1 * 1
x = Conv2D(filter3, kernel_size=(1, 1),
kernel_initializer='he_normal',
name=conv_name + '2c')(x)
x = BatchNormalization(axis=3, name=bn_name + '2c')(x)
print(x.shape, input_tensor.shape)
x = add([x, input_tensor])
x = Activation('relu')(x)
return x
def resnet50(input_tensor, include_top=True, classes=1000):
# stage 1
x = ZeroPadding2D((3, 3), name='padding')(input_tensor)
x = Conv2D(64, kernel_size=(7, 7), strides=(2, 2),
kernel_initializer='he_normal',
name='conv1')(x)
x = BatchNormalization(axis=3, name='conv1_bn')(x)
x = Activation('relu', name='conv1_relu')(x)
x = MaxPooling2D((3, 3), strides=(2, 2), name='pool')(x)
# stage 2 repeat 3
x = conv_block(x, 3, [64, 64, 256], stage=2, block='a', strides=(1, 1))
x = identity_block(x, 3, [64, 64, 256], stage=2, block='b')
x = identity_block(x, 3, [64, 64, 256], stage=2, block='c')
# stage 3 repeat 4
x = conv_block(x, 3, [128, 128, 512], stage=3, block='a', strides=(2, 2))
x = identity_block(x, 3, [128, 128, 512], stage=3, block='b')
x = identity_block(x, 3, [128, 128, 512], stage=3, block='c')
x = identity_block(x, 3, [128, 128, 512], stage=3, block='d')
# stage 4 repeat 6
x = conv_block(x, 3, [256, 256, 1024], stage=4, block='a', strides=(2, 2))
x = identity_block(x, 3, [256, 256, 1024], stage=4, block='b')
x = identity_block(x, 3, [256, 256, 1024], stage=4, block='c')
x = identity_block(x, 3, [256, 256, 1024], stage=4, block='d')
x = identity_block(x, 3, [256, 256, 1024], stage=4, block='e')
x = identity_block(x, 3, [256, 256, 1024], stage=4, block='f')
# stage 5 repeat 3
x = conv_block(x, 3, [512, 512, 2048], stage=5, block='a', strides=(2, 2))
x = identity_block(x, 3, [512, 512, 2048], stage=5, block='b')
x = identity_block(x, 3, [512, 512, 2048], stage=5, block='c')
if include_top:
x = GlobalAveragePooling2D(name='avg_pool')(x)
x = Dense(classes, activation='softmax', name='fc1000')(x)
model = Model(input=inputs, output=x, name='ResNet50')
return model
inputs = Input(shape=(224, 224, 3))
model = resnet50(inputs)
plot_model(model, to_file='./resnet50.jpg', show_shapes=True)