Pythorch的nn.functional和Parameter

Convolution函数

2D卷积

torch.nn.functional.conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1)

参数：

• input – 输入张量 (minibatch x in_channels x iH x iW)
• weight – 过滤器张量 (out_channels, in_channels/groups, kH, kW)
• bias – 可选偏置张量 (out_channels)
• stride – 卷积核的步长，可以是单个数字或一个元组 (sh x sw)。默认为1
• padding – 输入上隐含零填充。可以是单个数字或元组。 默认值：0 - groups – 将输入分成组，in_channels应该被组数除尽

exp:

>>> # With square kernels and equal stride
>>> filters = autograd.Variable(torch.randn(8,4,3,3))
>>> inputs = autograd.Variable(torch.randn(1,4,5,5))
>>> F.conv2d(inputs, filters, padding=1)

最后的output：1x8x5x5

Parameter

Tensors, as you might know, are multi dimensional matrices. Parameter, in its raw form, is a tensor i.e. a multi dimensional matrix. It sub-classes the Variable class.

The difference between a Variable and a Parameter comes in when associated with a module. When a Parameter is associated with a module as a model attribute, it gets added to the parameter list automatically and can be accessed using the ‘parameters’ iterator.

Initially in Torch, a Variable (which could for example be an intermediate state) would also get added as a parameter of the model upon assignment. Later on there were use cases identified where a need to cache the variables instead of having them added to the parameter list was identified.

One such case, as mentioned in the documentation is that of RNN, where in you need to save the last hidden state so you don’t have to pass it again and again. The need to cache a Variable instead of having it automatically register as a parameter to the model is why we have an explicit way of registering parameters to our model i.e. nn.Parameter class.

For instance, run the following code -

import torch
import torch.nn as nn
from torch.optim import Adam

class NN_Network(nn.Module):
    def __init__(self,in_dim,hid,out_dim):
        super(NN_Network, self).__init__()
        self.linear1 = nn.Linear(in_dim,hid)
        self.linear2 = nn.Linear(hid,out_dim)
        self.linear1.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
        self.linear1.bias = torch.nn.Parameter(torch.ones(hid))
        self.linear2.weight = torch.nn.Parameter(torch.zeros(in_dim,hid))
        self.linear2.bias = torch.nn.Parameter(torch.ones(hid))

    def forward(self, input_array):
        h = self.linear1(input_array)
        y_pred = self.linear2(h)
        return y_pred

in_d = 5
hidn = 2
out_d = 3
net = NN_Network(in_d, hidn, out_d)

for param in net.parameters():
    print(type(param.data), param.size())

""" Output
 torch.Size([5, 2])
 torch.Size([2])
 torch.Size([5, 2])
 torch.Size([2])
"""

链接：
https://stackoverflow.com/questions/50935345/understanding-torch-nn-parameter