[Pytorch专栏]基本概念之nn.Module

pytorch在深度学习领域的重要性我就不再多罗嗦了，相信你看到这里的时候也一定对torch、pytorch有了些许了解，并且希望深入学习，在这个专栏带领大家一起玩转pytorch。

参考资料：https://pytorch.org/tutorials/

基本概念这一部分讲解使用pytorch模型的基本组成元素，其中包括nn.Module、DataLoader、Tranformer、visualization、DistributedTraining等部分，在后续章节中会分别进行介绍

nn.Module

训练模型，必须首先创建一个模型，而创建模型的所需要的父类就是nn.Module。

class SimpleNet(nn.Module):
    def __init__(self, num_classes=1000):
        super(SimpleNet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(192, 384, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
        )
        self.avgpool = nn.AvgPool2d((1, 1))
        self.classifier = nn.Sequential(
            nn.Dropout(),
            nn.Linear(256 * 6 * 6, 4096),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, num_classes),
        )

    def forward(self, x):
        x = self.features(x)
        x = self.avgpool(x)
        x = x.view(x.size(0), 256 * 6 * 6)
        x = self.classifier(x)
        return x

上面代码创建了一个简单的分类模型，每一个模型都是按照如上格式进行模型的。创建好网络后可以使用print打印一下

SimpleNet(
  (features): Sequential(
    (0): Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=(2, 2))
    (1): ReLU(inplace)
    (2): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (3): Conv2d(64, 192, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
    (4): ReLU(inplace)
    (5): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
    (6): Conv2d(192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (7): ReLU(inplace)
    (8): Conv2d(384, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (9): ReLU(inplace)
    (10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (11): ReLU(inplace)
    (12): MaxPool2d(kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (avgpool): AvgPool2d(kernel_size=(1, 1), stride=(1, 1), padding=0)
  (classifier): Sequential(
    (0): Dropout(p=0.5)
    (1): Linear(in_features=9216, out_features=4096, bias=True)
    (2): ReLU(inplace)
    (3): Dropout(p=0.5)
    (4): Linear(in_features=4096, out_features=4096, bias=True)
    (5): ReLU(inplace)
    (6): Linear(in_features=4096, out_features=1000, bias=True)
  )
)

如果前后层尺寸不对应的话，打印是看不出来错误的，只能在训练或者前向时才可以看出来。还可以打印网络层名称和权重

for name, parameters in net.named_parameters():
    print('name: {}, param: {}'.format(name, parameters))

网络层名称和权重输出结果如下，从下面输出结果可以看出，name即为网络层名称，也就是在定义网络的时候写的层名，当加载权重的时候也是通过匹配name进行权重赋值的，param就是层参数，参数的基本单元是tensor，tensor是高维数组，可以与numpy进行互相转换，对于卷积层和全连接层，有权重和偏置两个参数。激活层ReLU和MaxPool层没有参数

name: features.3.bias, param: Parameter containing:
tensor([ 1.4957e-02,  9.2688e-04, -1.5689e-02, -1.7813e-02,  2.0555e-02,
         2.4371e-02, -1.5469e-02,  1.3342e-02,  3.3973e-03,  2.4930e-02,
         2.0547e-03,  1.8555e-02, -3.7325e-03,  2.0697e-02,  1.6567e-02,
        -2.0579e-02,  1.5670e-02, -5.5653e-03,  9.8694e-03,  1.6047e-02,
        -5.5923e-03, -8.6149e-03, -1.9673e-03, -1.7893e-02,  2.4934e-02,
        -1.4138e-02,  8.2390e-03, -1.4437e-02,  2.0114e-02, -1.4466e-02,
        -1.3437e-02, -8.9125e-03, -2.1675e-02,  8.2867e-04, -1.4810e-02,
        -1.3505e-02, -4.0385e-04, -4.7128e-03, -1.5937e-02,  1.0475e-02,
         1.0923e-02,  8.2931e-03,  2.0249e-02, -2.2489e-02, -2.3102e-02,
        -2.3941e-02,  1.3586e-02,  9.4410e-03, -1.3969e-02,  1.8650e-02,
         6.5202e-03, -2.0268e-02,  2.0344e-02, -2.4513e-02, -2.1452e-02,
        -2.2310e-02, -1.4287e-02,  2.3486e-02,  3.5223e-03,  1.7498e-02,
        -6.4333e-03, -1.8280e-02,  1.6511e-02,  3.1823e-03, -1.0159e-02,
        -1.8306e-02,  8.8460e-04,  4.2371e-03, -2.4310e-02,  6.1531e-03,
         1.7120e-02,  9.4392e-03,  3.9597e-03, -1.5277e-02,  1.1027e-02,
        -1.8415e-02,  1.9837e-02,  1.2948e-02, -1.6227e-02, -1.8143e-02,
        -1.1087e-02,  1.5955e-02, -1.3796e-02, -7.9715e-03, -8.2901e-03,
        -1.1397e-02,  2.1885e-02, -1.3258e-02, -9.6440e-03,  2.0368e-02,
        -5.2773e-04, -3.4875e-03, -1.5337e-02,  2.3905e-02,  1.8523e-02,
         3.4253e-03, -2.4514e-02, -1.9495e-02, -1.5818e-02,  7.3031e-03,
        -1.9106e-02, -1.1330e-03, -2.3167e-02,  2.0370e-02, -2.5699e-05,
        -2.0461e-02, -2.1582e-02,  1.5271e-02,  1.7045e-02, -2.1056e-02,
         1.7383e-03, -3.1803e-03,  1.5177e-02, -6.6120e-03,  2.3724e-02,
         9.1531e-03, -7.9457e-03, -1.3581e-02, -2.4429e-02,  2.2770e-02,
        -4.8633e-03,  1.3242e-02,  2.3412e-02,  1.6137e-02, -1.0654e-02,
        -2.3665e-02, -6.4708e-03,  5.3640e-03, -5.2605e-03, -1.2003e-02,
        -1.5830e-02,  1.8685e-02,  1.7946e-02, -2.2646e-02, -2.3571e-02,
         3.1388e-03, -1.5248e-02,  2.2037e-02,  1.4381e-02,  8.1348e-03,
        -2.3800e-02,  1.9604e-02,  2.5116e-03, -1.1058e-03, -1.9687e-02,
         1.2248e-02,  2.3413e-02,  2.1731e-02,  5.0117e-03,  8.7472e-04,
        -7.5963e-03, -1.1021e-02,  1.4873e-02, -2.2508e-02,  2.1154e-03,
        -9.1743e-04, -1.7245e-02,  1.7589e-02,  1.1800e-02,  8.5025e-03,
        -2.3779e-02, -2.0176e-02, -2.0294e-02, -1.1143e-02,  4.0771e-03,
        -1.5206e-02,  2.2434e-02,  1.0912e-02, -1.6116e-02,  9.9414e-03,
         2.7820e-03, -5.8191e-03, -3.6565e-03,  4.7741e-03, -5.2534e-03,
         2.2425e-02, -3.1049e-03, -9.2849e-05, -5.2352e-03, -1.5395e-02,
        -2.3302e-02,  1.3749e-02,  7.7268e-03, -1.9604e-03, -1.3374e-02,
         1.9139e-02, -1.8436e-02, -3.1385e-03,  1.9631e-02,  2.1478e-02,
         1.8698e-02, -1.4259e-02], requires_grad=True)
name: features.6.weight, param: Parameter containing:
tensor([[[[ 1.4673e-02, -1.2573e-03, -4.9999e-03],
          [-1.1767e-04, -2.3442e-03,  9.0190e-03],
          [-2.2670e-02,  9.2900e-04,  2.3845e-02]],

         [[ 7.1682e-03,  1.0164e-02,  7.7916e-03],
          [-1.2084e-02,  8.5199e-03,  1.7568e-02],
          [-1.6802e-02, -7.6553e-03, -1.4794e-02]],

         [[ 2.3868e-02,  5.9265e-03,  1.8999e-02],
          [ 1.0245e-02, -5.0897e-03, -9.1414e-03],
          [-1.2725e-02,  1.5891e-02, -2.2689e-05]],

         ...,

从上面的结果我们看不出来网络层feature map的具体情况，还可以将网络进行可视化，下面使用hiddenlayer对网络进行可视化，hiddenlayer借助onnx生成静态图然后使用graphviz完成网络结构图绘制

hl_graph = hl.build_graph(net, torch.zeros([1, 3, 224, 224]))
hl_graph.theme = hl.graph.THEMES["blue"].copy()  # Two options: basic and blue
hl_graph.save("simplenet", format="png")

当使用pytorch1.3版本的时候可能会出现一下问题，当将版本回退到1.0就没有问题了 ，参考这里，具体原因还没搞清楚

AttributeError: ‘torch._C.Value’ object has no attribute ‘uniqueName’

也可能会遇到一下问题，应该是onnx不支持导致，可以参考这里，将nn.AdaptiveAvgPool2d((6, 6))改为nn.AvgPool2d((1, 1))

AssertionError: Only output_size=[1, 1] is supported

基本上创建了一个网络，最后还可以尝试输入一个数据看看输出结果

input_image = torch.FloatTensor(10, 3, 224, 224)
input_image = Variable(input_image)

output = net(input_image)
print('output: {}'.format(output))
print(output.shape)
print(output.size())

output: tensor([[-0.0007,  0.0012,  0.0134,  ...,  0.0062, -0.0009,  0.0047],
        [ 0.0005,  0.0043,  0.0097,  ...,  0.0084, -0.0027,  0.0035],
        [-0.0009,  0.0066,  0.0108,  ...,  0.0078, -0.0074,  0.0050],
        ...,
        [-0.0013,  0.0022,  0.0134,  ...,  0.0118, -0.0027,  0.0049],
        [-0.0013,  0.0021,  0.0120,  ...,  0.0082, -0.0027,  0.0055],
        [ 0.0021,  0.0031,  0.0131,  ...,  0.0122, -0.0021,  0.0033]],
       grad_fn=)
torch.Size([10, 1000])
torch.Size([10, 1000])

写道这里基本完成了pytorch创建一个分类模型，提一个问题，大家知道上面是哪个经典模型吗？