7.1 动手实现AlexNet

AlexNet引入了dropput层
7.1 动手实现AlexNet_第1张图片

代码


import torch
from torch import nn
from d2l import torch as d2l

net = nn.Sequential(
    # 样本数为1,通道数为96,11x11的卷积核,步幅为4,减少输出的高度和深度。 LeNet的通道数才6,此处96,为什么要增加这么多通道呢?
    nn.Conv2d(1,96,kernel_size=11,stride=4,padding=1),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    # 减小卷积窗口,使用填充2使输出的高与宽一致,且增大输出通道数
    nn.Conv2d(96,256,kernel_size=5,padding=2),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    # 连续使用3个卷积层,通道数继续增加
    nn.Conv2d(256,384,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(384,384,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(384,256,kernel_size=3,padding=1),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    nn.Flatten(),
    # 相对于LeNet,全连接增加了几倍,用dropout来减少过拟合
    nn.Linear(6400,4096),nn.ReLU(),
    nn.Dropout(p=0.5),
    nn.Linear(4096,4096),nn.ReLU(),
    nn.Dropout(p=0.5),
    nn.Linear(4096,10)
)

# 样本数为1,通道数为1,224x224
X = torch.randn(1,1,224,224)
for layer in net:
    X = layer(X)
    print(layer.__class__.__name__,'output shape:\t',X.shape)
Conv2d output shape:	 torch.Size([1, 96, 54, 54])
ReLU output shape:	 torch.Size([1, 96, 54, 54])
MaxPool2d output shape:	 torch.Size([1, 96, 26, 26])
Conv2d output shape:	 torch.Size([1, 256, 26, 26])
ReLU output shape:	 torch.Size([1, 256, 26, 26])
MaxPool2d output shape:	 torch.Size([1, 256, 12, 12])
Conv2d output shape:	 torch.Size([1, 384, 12, 12])
ReLU output shape:	 torch.Size([1, 384, 12, 12])
Conv2d output shape:	 torch.Size([1, 384, 12, 12])
ReLU output shape:	 torch.Size([1, 384, 12, 12])
Conv2d output shape:	 torch.Size([1, 256, 12, 12])
ReLU output shape:	 torch.Size([1, 256, 12, 12])
MaxPool2d output shape:	 torch.Size([1, 256, 5, 5])
Flatten output shape:	 torch.Size([1, 6400])
Linear output shape:	 torch.Size([1, 4096])
ReLU output shape:	 torch.Size([1, 4096])
Dropout output shape:	 torch.Size([1, 4096])
Linear output shape:	 torch.Size([1, 4096])
ReLU output shape:	 torch.Size([1, 4096])
Dropout output shape:	 torch.Size([1, 4096])
Linear output shape:	 torch.Size([1, 10])
# 读取数据集, fashion_mnist的图片是28x28,为了满足AlexNet的输出,resize为224x224,通常来说这样并不好
batch_size = 128
train_iter,test_iter = d2l.load_data_fashion_mnist(batch_size,resize=224)

# 训练
lr,num_epochs = 0.01,10
d2l.train_ch6(net,train_iter,test_iter,num_epochs,lr,device=d2l.try_gpu())

运行结果:

7.1 动手实现AlexNet_第2张图片

当训练轮数增加到20的结果

# 训练
lr,num_epochs = 0.01,20
d2l.train_ch6(net,train_iter,test_iter,num_epochs,lr,device=d2l.try_gpu())

7.1 动手实现AlexNet_第3张图片
结果提升了2点多。

改变模型的前两层,使模型可以直接输入28x28的图片

# 将输入由224变成28
net = nn.Sequential(
    nn.Conv2d(1,96,kernel_size=5,padding=2),nn.ReLU(),
    # 样本数为1,通道数为96,11x11的卷积核,步幅为4,减少输出的高度和深度。 LeNet的通道数才6,此处96,为什么要增加这么多通道呢?
    nn.MaxPool2d(kernel_size=3,stride=1),
    # 减小卷积窗口,使用填充2使输出的高与宽一致,且增大输出通道数
    nn.Conv2d(96,256,kernel_size=5,padding=2),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    # 连续使用3个卷积层,通道数继续增加
    nn.Conv2d(256,384,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(384,384,kernel_size=3,padding=1),nn.ReLU(),
    nn.Conv2d(384,256,kernel_size=3,padding=1),nn.ReLU(),
    nn.MaxPool2d(kernel_size=3,stride=2),
    nn.Flatten(),
    # 相对于LeNet,全连接增加了几倍,用dropout来减少过拟合
    nn.Linear(6400,4096),nn.ReLU(),
    nn.Dropout(p=0.5),
    nn.Linear(4096,4096),nn.ReLU(),
    nn.Dropout(p=0.5),
    nn.Linear(4096,10)
)
X = torch.randn(1,1,28,28)
for layer in net:
    X = layer(X)
    print(layer.__class__.__name__,'output shape:\t',X.shape)
Conv2d output shape:	 torch.Size([1, 96, 28, 28])
ReLU output shape:	 torch.Size([1, 96, 28, 28])
MaxPool2d output shape:	 torch.Size([1, 96, 26, 26])
Conv2d output shape:	 torch.Size([1, 256, 26, 26])
ReLU output shape:	 torch.Size([1, 256, 26, 26])
MaxPool2d output shape:	 torch.Size([1, 256, 12, 12])
Conv2d output shape:	 torch.Size([1, 384, 12, 12])
ReLU output shape:	 torch.Size([1, 384, 12, 12])
Conv2d output shape:	 torch.Size([1, 384, 12, 12])
ReLU output shape:	 torch.Size([1, 384, 12, 12])
Conv2d output shape:	 torch.Size([1, 256, 12, 12])
ReLU output shape:	 torch.Size([1, 256, 12, 12])
MaxPool2d output shape:	 torch.Size([1, 256, 5, 5])
Flatten output shape:	 torch.Size([1, 6400])
Linear output shape:	 torch.Size([1, 4096])
ReLU output shape:	 torch.Size([1, 4096])
Dropout output shape:	 torch.Size([1, 4096])
Linear output shape:	 torch.Size([1, 4096])
ReLU output shape:	 torch.Size([1, 4096])
Dropout output shape:	 torch.Size([1, 4096])
Linear output shape:	 torch.Size([1, 10])
# 读取数据集, fashion_mnist的图片是28x28,为了满足AlexNet的输出,resize为224x224,通常来说这样并不好
batch_size = 128
# train_iter,test_iter = d2l.load_data_fashion_mnist(batch_size,resize=224)
train_iter,test_iter = d2l.load_data_fashion_mnist(batch_size)

# 训练
lr,num_epochs = 0.01,20
d2l.train_ch6(net,train_iter,test_iter,num_epochs,lr,device=d2l.try_gpu())

7.1 动手实现AlexNet_第4张图片
结果下降了1点左右

AlexNet哪一部分主要占用显存?哪一层占用最多的显存?

AlexNet里面不同层需要的参数大小决定了占用显存的大小

  第一层卷积层卷积核参数个数:11x11x3x96=34,848
  第二层卷积层卷积核参数个数:5x5x96x256=614,400
  第三层卷积层卷积核参数个数:3x3x256x384=884,736
  第四层卷积层卷积核参数个数:3x3x384x384=1,327,104
  第五层卷积层卷积核参数个数:3x3x384x256=884,736
  第一层全连层参数(权重+偏移):6400x4096+4096=26,218,496
  第二层全连层参数(权重+偏移):4096x4096+4096=16,781,312
  第三层全连层参数(权重+偏移):4096x1000+1000=4,100,096

所以是第一层全连层占用了最多的显存

在AlexNet中主要是哪部分需要更多的计算?

把运算分为乘法运算、加法运算和特殊运算(ReLu、池化)

卷积层的计算次数:Ci x Co x Kw x Kh x Nw x Nh + Ci x Co x (Kw x Kh -1)x Nw x Nh + Co x (Ci-1) x Nw x Nh
池化层的计算次数:Nh x Nw x Ci
全连接层的计算次数:权重与变量相乘、结果相加、偏置项

第一层卷积层计算次数:3x96x(2x11x11-1)x54x54+96x2x54x54=202,953,600
第一层卷积层的池化层计算次数:26x26x96=64896
第二层卷积层卷计算次数:96*256*(2*5*5-1)*26*26+256*95*26*26=830,495,744
第二层卷积层的池化层计算次数:12x12x256=36864
第三层卷积层计算次数:256*384*(2*3*3-1)*12*12+384*255*12*12=254748672
第四层卷积层计算次数:384*384*(2*3*3-1)*12*12+384*383*12*12=382150656
第五层卷积层计算次数:384*256*(2*3*3-1)*12*12+256*383*12*12=254767104
第五层卷积层池化层计算次数:5x5x256=6400
第一层全连层计算次数:6400*4096+4096*(6400-1)+4096=52,428,800
第二层全连层计算次数:4096*4096+4096*(4096-1)+4096=33,554,432
第三层全连层计算次数:1000*4096+1000*4095+1000=8,192,000

第二层卷积层需要最多次数的计算,卷积层的运算次数与输入通道数、输出通道数、图片大小、卷积核大小都有关。

参考链接

【动手学深度学习】之 现代卷积神经网络 AlexNet VGGNet NIN 习题解答
https://blog.csdn.net/weixin_51580530/article/details/128619145

你可能感兴趣的:(动手学深度学习(计算机视觉篇),深度学习,pytorch,神经网络)