LeNet 网络复现

LeNet-5 网络复现

• LeNet-5论文传送门
  https://ieeexplore.ieee.org/document/726791

参数总量：61706

一些固定的超参数

batch_size = 64
lr = 0.001
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(...)
Epochs = 10

AvgPool 与 MaxPool性能比较

• 网络结构

class LeNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.c1 = nn.Conv2d(1,6,5,1,0)
        self.c3 = nn.Conv2d(6,16,5,1,0)
        self.c5 = nn.Conv2d(16,120,5,1,0)
        
        self.s2 = nn.AvgPool2d(2,2)
        self.s4 = nn.AvgPool2d(2,2)
        
        self.f6 = nn.Linear(120,84)
        self.fout = nn.Linear(84,10)
    
    def forward(self, x):
        x = self.c1(x)
        x = self.s2(x)
        x = self.c3(x)
        x = self.s4(x)
        x = self.c5(x)
        x = x.view(x.size()[0],-1)
        x = self.f6(x)
        x = torch.tanh(x)
        out = self.fout(x)
        return out

数据集：MNIST

性能比较使用数据集：MNIST - testset

模型现在training set上进行训练，然后在testing set上面进行Accuracy测试

• Accuracy (定义为1-（错误分类数据总数）/测试集总量)
  • AvgPool：0.9670 (loss ~ 0.016)
  • MaxPool：0.9824 (loss ~ 0.002)

**结论：**MaxPool的效果要好于AvgPool

激活函数对模型性能的影响

在使用MaxPool的基础上，每一个卷积层后面添加relu激活函数

    def forward(self, x):
        x = F.relu(self.c1(x))
        x = self.s2(x)
        x = F.relu(self.c3(x))
        x = self.s4(x)
        x = F.relu(self.c5(x))
        x = x.view(x.size()[0],-1)
        x = self.f6(x)
        x = torch.tanh(x)
        out = self.fout(x)
        return out

• Accuracy:
  • with ReLU: 0.9897 (loss ~ 0.0001)

进一步将f6的tanh改为ReLU

• Accuracy:

• with Tanh replaced to be ReLU: 0.9897 (loss ~ 0.0001)