Pytorch-模型参数:named_parameters()、parameters()、state_dict()区别

torch中存在3个功能极其类似的方法,它们分别是model.parameters()、model.named_parameters()、model.state_dict(),下面就具体来说说这三个函数的差异:

一、model.parameters()和model.named_parameters()差别

  • named_parameters()返回的list中,每个元组(与list相似,只是数据不可修改)打包了2个内容,分别是layer-namelayer-param(网络层的名字和参数的迭代器);
  • parameters()只有后者layer-param(参数的迭代器)

1、model.named_parameters()里的网络层名字

import torchvision.models as models

model = models.resnet18()
for param_tuple in model.named_parameters():
    name, param = param_tuple
    print("name = ", name)
    print("-" * 100)

打印结果:

name =  conv1.weight
----------------------------------------------------------------------------------------------------
name =  bn1.weight
----------------------------------------------------------------------------------------------------
name =  bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer1.0.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer1.0.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer1.0.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer1.0.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer1.0.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer1.1.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer1.1.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer1.1.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer1.1.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer1.1.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer1.1.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer2.0.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer2.0.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer2.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name =  layer2.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name =  layer2.1.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer2.1.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer2.1.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer2.1.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer2.1.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer2.1.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer3.0.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer3.0.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer3.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name =  layer3.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name =  layer3.1.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer3.1.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer3.1.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer3.1.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer3.1.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer3.1.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer4.0.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer4.0.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.bn2.bias
----------------------------------------------------------------------------------------------------
name =  layer4.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name =  layer4.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name =  layer4.1.conv1.weight
----------------------------------------------------------------------------------------------------
name =  layer4.1.bn1.weight
----------------------------------------------------------------------------------------------------
name =  layer4.1.bn1.bias
----------------------------------------------------------------------------------------------------
name =  layer4.1.conv2.weight
----------------------------------------------------------------------------------------------------
name =  layer4.1.bn2.weight
----------------------------------------------------------------------------------------------------
name =  layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
name =  fc.weight
----------------------------------------------------------------------------------------------------
name =  fc.bias
----------------------------------------------------------------------------------------------------

Process finished with exit code 0

2、model.named_parameters()里的网络层名字、参数

import torchvision.models as models

model = models.resnet18()
for param_tuple in model.named_parameters():
    name, param = param_tuple
    print("name = ", name)
    print("-" * 100)
    print("param_tuple = ", param_tuple)
    print("*" * 200)

打印结果:

C:\Program_Files_AI\Anaconda3531\python.exe C:/Users/Admin/OneDrive/WorkSpace_AI/0-基于知识库的智能问答系统-华控智加/01-意图识别/test.py
name =  conv1.weight
----------------------------------------------------------------------------------------------------
param_tuple =  ('conv1.weight', Parameter containing:
tensor([[[[-1.4115e-05,  2.9187e-02,  2.9325e-03,  ..., -4.2247e-02,
            1.7490e-02, -4.5253e-02],
          [-2.4594e-02, -3.0836e-02,  3.8604e-02,  ...,  3.5473e-02,
           -4.7046e-03, -2.9440e-02],
          [ 2.4811e-02,  1.2679e-02,  1.0070e-02,  ..., -8.3476e-03,
            1.7960e-02, -1.7406e-02],
          ...,
          [-1.3021e-02,  2.9023e-02, -6.1800e-02,  ..., -5.2802e-02,
           -4.7817e-02, -2.2377e-02],
          [-3.8513e-03, -1.0603e-02, -3.9712e-02,  ...,  5.1941e-03,
            8.2868e-03, -8.3469e-03],
          [ 3.8993e-03,  3.2017e-02, -3.6292e-02,  ..., -2.0210e-02,
           -4.0358e-02,  1.7709e-02]],

         [[-1.0894e-03,  1.5720e-02,  7.0129e-03,  ..., -1.2024e-02,
            1.8644e-02,  1.7892e-02],
          [-2.3866e-02,  9.1136e-03,  3.5243e-02,  ..., -1.6756e-02,
            1.4441e-03,  4.7943e-02],
          [-2.0514e-03,  4.3022e-02,  2.6358e-02,  ..., -2.3662e-02,
           -7.8241e-04,  1.0167e-02],
        ...

         [[-4.6689e-02, -1.1407e-03,  1.8674e-02,  ...,  1.2649e-03,
           -2.9532e-02,  6.4535e-04],
          [ 1.4171e-03, -1.9274e-02, -8.6811e-03,  ...,  2.4428e-02,
            6.9516e-03,  4.3715e-02],
          [ 1.9982e-02,  1.3124e-02,  9.1508e-03,  ...,  2.5405e-02,
           -1.3132e-02,  4.0835e-02],
          ...,
          [-3.4174e-03,  1.8623e-02, -1.4386e-02,  ...,  1.0627e-03,
           -5.1297e-04,  2.2055e-02],
          [ 2.7333e-02,  2.4858e-02, -5.4305e-02,  ..., -1.2139e-02,
            1.7735e-03, -3.4184e-03],
          [ 1.1412e-03,  1.5794e-02, -2.0699e-02,  ..., -1.7846e-02,
            3.7425e-02, -1.6059e-02]]],


        ...,


        [[[-2.7389e-02, -3.8327e-02, -2.9043e-02,  ..., -7.6396e-03,
           -1.6519e-02,  3.9659e-02],
          [ 2.8740e-03, -1.0621e-02, -9.2430e-03,  ...,  2.2581e-02,
            5.1526e-03, -2.0006e-02],
          [ 1.3575e-02,  1.5290e-02, -1.7260e-02,  ...,  6.3830e-03,
           -1.9759e-02,  1.5501e-02],
          ...,
          [ 1.6091e-02,  2.4038e-02,  2.4507e-02,  ..., -4.5613e-02,
           -3.6233e-02,  2.1632e-02],
          [-1.1573e-02, -3.6514e-02,  4.1576e-02,  ...,  1.8090e-02,
           -2.3350e-02, -8.7074e-03],
          [-1.5837e-02, -3.1353e-02,  1.8726e-02,  ...,  9.3698e-03,
            3.0781e-02,  1.0976e-02]],

         [[-2.7063e-02,  8.7158e-03,  2.7193e-03,  ..., -1.6670e-03,
           -4.3033e-03,  7.2011e-04],
          [ 2.7870e-03,  1.4264e-02, -5.0581e-02,  ...,  2.5463e-02,
            7.6864e-03, -4.9655e-02],
          [ 2.6030e-03,  2.5918e-02,  2.9615e-02,  ...,  3.0676e-02,
           -2.7723e-02, -7.3628e-03],
          ...,
          [ 2.5969e-02, -1.4247e-02,  1.2516e-02,  ...,  5.9602e-03,
           -3.2843e-02,  3.5822e-02],
          [ 1.2845e-02, -2.0035e-02,  9.9398e-04,  ..., -3.1800e-02,
            5.7984e-03,  2.8756e-02],
          [ 2.3458e-02,  3.8193e-02, -2.3754e-03,  ..., -1.3867e-02,
            8.0831e-03, -3.2438e-02]],
...

         [[-9.9291e-03, -5.6023e-03, -1.7064e-02,  ...,  8.8544e-03,
           -5.8145e-03,  2.3248e-02],
          [ 1.2148e-02, -1.0730e-02, -1.2682e-02,  ...,  9.4389e-03,
            1.2149e-02,  3.8613e-03],
          [ 3.5913e-02, -5.2048e-04, -8.7133e-02,  ..., -2.0969e-03,
           -5.4117e-03,  5.4637e-05],
          ...,
          [ 4.0351e-03, -1.3189e-02,  3.1229e-02,  ...,  3.2340e-02,
           -2.8351e-02,  1.0634e-02],
          [ 2.6041e-02, -3.0633e-04, -1.2732e-02,  ...,  2.9417e-02,
           -7.3859e-03,  1.7207e-02],
          [ 6.9960e-04,  3.8486e-03,  1.0397e-02,  ...,  1.4535e-03,
           -3.6449e-02,  3.4848e-02]]]], requires_grad=True))
********************************************************************************************************************************************************************************************************
name =  bn1.weight
----------------------------------------------------------------------------------------------------
param_tuple =  ('bn1.weight', Parameter containing:
tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
        1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
        1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
        1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name =  bn1.bias
----------------------------------------------------------------------------------------------------
param_tuple =  ('bn1.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
       requires_grad=True))
********************************************************************************************************************************************************************************************************

********************************************************************************************************************************************************************************************************
name =  layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
param_tuple =  ('layer1.0.conv2.weight', Parameter containing:
tensor([[[[-8.6159e-02,  1.8507e-04,  5.4006e-03],
          [-6.3063e-03,  3.9225e-03, -6.3141e-02],
          [-7.0145e-02, -3.9266e-02,  1.9724e-03]],

         [[ 4.6454e-02,  2.1519e-02,  5.3696e-02],
          [ 1.1086e-02,  1.6269e-01, -7.0579e-02],
          [-1.1220e-01, -4.9811e-02, -7.5515e-02]],

         [[ 5.5275e-02, -8.2407e-02, -8.9807e-02],
          [ 5.8418e-02,  4.4029e-02,  3.0584e-03],
          [ 5.2371e-02, -1.5983e-02,  5.1494e-02]],

         ...,

         [[ 7.3441e-02,  4.5401e-02, -1.9175e-02],
          [-6.2500e-02, -8.5905e-03, -7.4856e-02],
          [-1.6170e-02,  3.7529e-02, -5.1231e-02]],

         [[ 7.7501e-04, -5.7506e-02,  1.8422e-01],
          [ 2.4594e-02,  1.7378e-02,  4.0000e-02],
          [-8.6796e-02, -6.0548e-02,  2.6795e-02]],

         [[ 5.3264e-02, -8.6190e-02,  4.2443e-02],
          [-6.8029e-03, -1.6581e-02,  7.8568e-02],
          [ 3.2037e-02, -7.3002e-02,  4.9353e-02]]]], requires_grad=True))
...********************************************************************************************************************************************************************************************************
name =  layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
param_tuple =  ('layer4.1.bn2.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
        0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name =  fc.weight
----------------------------------------------------------------------------------------------------
param_tuple =  ('fc.weight', Parameter containing:
tensor([[-0.0125,  0.0437, -0.0014,  ..., -0.0230,  0.0280,  0.0249],
        [-0.0105,  0.0242,  0.0291,  ...,  0.0153,  0.0366, -0.0236],
        [-0.0315,  0.0306, -0.0216,  ...,  0.0387,  0.0403,  0.0056],
        ...,
        [-0.0068, -0.0222, -0.0027,  ..., -0.0243,  0.0260,  0.0065],
        [ 0.0213,  0.0167, -0.0379,  ..., -0.0140,  0.0037, -0.0372],
        [ 0.0180,  0.0101, -0.0341,  ..., -0.0295, -0.0146,  0.0416]],
       requires_grad=True))
********************************************************************************************************************************************************************************************************
name =  fc.bias
----------------------------------------------------------------------------------------------------
param_tuple =  ('fc.bias', Parameter containing:
tensor([ 3.5711e-02,  3.2682e-02,  7.5932e-03, -3.1623e-02, -9.6316e-03,
        -2.4051e-02, -1.0393e-02,  2.3210e-02, -3.6044e-02,  2.3099e-02,
        -3.5723e-02, -3.9482e-02,  4.8526e-03, -3.2688e-02,  3.7720e-03,
        -2.2014e-02, -4.0935e-02,  4.0533e-02, -4.1172e-02,  3.9513e-02,
        -3.0332e-02,  3.2777e-02,  1.3342e-02,  2.3394e-02,  8.2328e-03,
         1.3757e-02, -1.7578e-02, -2.7165e-02,  3.8495e-03, -3.2116e-02,
         7.9903e-03,  9.9640e-04, -8.3106e-03,  2.5033e-02, -3.0446e-02,
        -1.8282e-02, -3.8420e-03, -8.6129e-03, -4.2712e-03,  1.7169e-02,
       ...,
        -5.3570e-05, -3.7353e-02, -9.8633e-03, -9.1069e-03,  3.2688e-02,
         2.2457e-02,  7.6379e-03, -3.6287e-02, -1.0444e-02,  2.1669e-02,
         2.5270e-02, -4.3881e-02,  2.1960e-02,  2.6293e-02, -3.5049e-02,
        -2.0074e-02, -9.7686e-03, -2.3766e-02, -5.0265e-03, -2.1095e-02,
         2.0981e-02, -3.5132e-02,  8.6407e-03,  1.8453e-02,  2.4282e-02,
         3.8392e-02, -1.7470e-02,  3.6958e-02, -3.7590e-02, -4.1951e-02,
        -1.8246e-02,  9.0818e-03,  3.8774e-02,  7.3408e-03,  1.7728e-02,
         3.5547e-02, -7.2857e-03, -2.7015e-02, -8.6983e-03, -2.3785e-02],
       requires_grad=True))
********************************************************************************************************************************************************************************************************

Process finished with exit code 0

二、model.named_parameters()和model.state_dict()差别

它们的差异主要体现在3方面:

  • 返回值类型不同
  • 存储的模型参数的种类不同
  • 返回的值的require_grad属性不同
named_parameters() state_dict()
将layer_name : layer_param的键值信息打包成一个元祖然后再存到list当中 将layer_name : layer_param的键值信息存储为dict形式
只保存可学习、可被更新的参数,model.buffer()中的参数不包含在model.named_parameters()中 存储的是该model中包含的所有layer中的所有参数
require_grad属性都是True 存储的模型参数tensor的require_grad属性都是False

为何model.parameters()迭代出来的所有参数的require_grad属性都是True,因为它们在被创建时,默认的require_grad就是True。这也符合逻辑,即,使用nn.Parameter()创建的变量是模型参数,本就是要参与学习和更新的;而不参与模型学习和更新的呢?我们再看一个例子:




参考资料:
pytorch中state_dict()和named_parameters()的差别,以及model.buffer/model.parameter
【pytorch】named_parameters()、parameters()、state_dict()==>给出网络的名字和参数的迭代器
【pytorch】named_parameters()和parameters()

你可能感兴趣的:(AI/模型调优,parameters)