AlexNet模型简介，Tensorflow2与Pytorch的实现
AlexNet
2012年，AlexNet横空出世，该模型的名字源于论文第一作者的姓名Alex Krizhevsky 。AlexNet使用了8层卷积神经网络，以很大的优势赢得了ImageNet 2012图像识别挑战赛。它首次证明了学习到的特征可以超越手工设计的特征，从而一举打破计算机视觉研究的方向。
特点
- 8层变换：5层卷积，2层全连接隐藏层，1层全连接输出层
- 第一层卷积形状 11x11，第二层卷积形状 5x5，其他卷积形状都是3x3
- 所有池化都是3x3,步幅为2
- 将sigmoid激活函数改成了ReLu激活函数，计算更简单，更容易训练
- 通过DropOut控制全连接层的模型复杂度
- 引入了大量的图像增强，如翻转、裁剪和颜色变化，进一步扩大数据集缓解过拟合
1.网络结构
2.TF2实现
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Flatten,Dense,Convolution2D,MaxPooling2D,Dropout,AvgPool2D
AlexNet = Sequential([
Convolution2D(filters=64, kernel_size=11, strides=4, activation='relu',input_shape=(227,227,3)),
MaxPooling2D(pool_size=3, strides=2),
Convolution2D(filters=192, kernel_size=5, strides=1, padding='same', activation='relu'),
MaxPooling2D(pool_size=3, strides=2),
Convolution2D(filters=384, kernel_size=3, strides=1, padding='same', activation='relu'),
Convolution2D(filters=256, kernel_size=3, strides=1, padding='same', activation='relu'),
Convolution2D(filters=256, kernel_size=3, strides=1, padding='same', activation='relu'),
MaxPooling2D(pool_size=3, strides=2),
Flatten(),
Dropout(0.5),
Dense(units=4096, activation='relu'),
Dropout(0.5),
Dense(units=4096, activation='relu'),
Dropout(0.5),
Dense(units=1000, activation='softmax')
])
AlexNet.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 55, 55, 64) 23296
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 27, 27, 64) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 27, 27, 192) 307392
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 13, 13, 192) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 13, 13, 384) 663936
_________________________________________________________________
conv2d_3 (Conv2D) (None, 13, 13, 256) 884992
_________________________________________________________________
conv2d_4 (Conv2D) (None, 13, 13, 256) 590080
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 6, 6, 256) 0
_________________________________________________________________
flatten (Flatten) (None, 9216) 0
_________________________________________________________________
dropout (Dropout) (None, 9216) 0
_________________________________________________________________
dense (Dense) (None, 4096) 37752832
_________________________________________________________________
dropout_1 (Dropout) (None, 4096) 0
_________________________________________________________________
dense_1 (Dense) (None, 4096) 16781312
_________________________________________________________________
dropout_2 (Dropout) (None, 4096) 0
_________________________________________________________________
dense_2 (Dense) (None, 1000) 4097000
=================================================================
Total params: 61,100,840
Trainable params: 61,100,840
Non-trainable params: 0
_________________________________________________________________
3.PyTorch实现
import torchvision.models as models
from torchsummary import summary
AlexNet = models.alexnet(pretrained=False,progress=True) # 默认为未训练好的模型
print(AlexNet)
summary(AlexNet.cuda(),(3,227,227))
AlexNet(
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
(1): ReLU(inplace=True)
(2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(4): ReLU(inplace=True)
(5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(7): ReLU(inplace=True)
(8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(11): ReLU(inplace=True)
(12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(avgpool): AdaptiveAvgPool2d(output_size=(6, 6))
(classifier): Sequential(
(0): Dropout(p=0.5, inplace=False)
(1): Linear(in_features=9216, out_features=4096, bias=True)
(2): ReLU(inplace=True)
(3): Dropout(p=0.5, inplace=False)
(4): Linear(in_features=4096, out_features=4096, bias=True)
(5): ReLU(inplace=True)
(6): Linear(in_features=4096, out_features=1000, bias=True)
)
)
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv2d-1 [-1, 64, 56, 56] 23,296
ReLU-2 [-1, 64, 56, 56] 0
MaxPool2d-3 [-1, 64, 27, 27] 0
Conv2d-4 [-1, 192, 27, 27] 307,392
ReLU-5 [-1, 192, 27, 27] 0
MaxPool2d-6 [-1, 192, 13, 13] 0
Conv2d-7 [-1, 384, 13, 13] 663,936
ReLU-8 [-1, 384, 13, 13] 0
Conv2d-9 [-1, 256, 13, 13] 884,992
ReLU-10 [-1, 256, 13, 13] 0
Conv2d-11 [-1, 256, 13, 13] 590,080
ReLU-12 [-1, 256, 13, 13] 0
MaxPool2d-13 [-1, 256, 6, 6] 0
AdaptiveAvgPool2d-14 [-1, 256, 6, 6] 0
Dropout-15 [-1, 9216] 0
Linear-16 [-1, 4096] 37,752,832
ReLU-17 [-1, 4096] 0
Dropout-18 [-1, 4096] 0
Linear-19 [-1, 4096] 16,781,312
ReLU-20 [-1, 4096] 0
Linear-21 [-1, 1000] 4,097,000
================================================================
Total params: 61,100,840
Trainable params: 61,100,840
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.59
Forward/backward pass size (MB): 8.49
Params size (MB): 233.08
Estimated Total Size (MB): 242.16
----------------------------------------------------------------