Pytorch 中 torch.nn.Conv2d和torch.nn.functional.conv2d的区别

1 函数引用与参数解释

import torch

class torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros')

• in_channels (int) – Number of channels in the input image

• out_channels (int) – Number of channels produced by the convolution

• kernel_size (int or tuple) – Size of the convolving kernel

• stride (int or tuple, optional) – Stride of the convolution. Default: 1

• padding (int, tuple or str, optional) – Padding added to all four sides of the input. Default: 0

• padding_mode (str, optional) – 'zeros', 'reflect', 'replicate' or 'circular'. Default: 'zeros'

• dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1

• groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1

• bias (bool, optional) – If True, adds a learnable bias to the output. Default: True

import torch

torch.nn.functional.conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1)

• input – input tensor of shape (minibatch,in_channels,��,��)(minibatch,in_channels,iH,iW)

• weight – filters of shape (out_channels,in_channelsgroups,��,��)(out_channels,groupsin_channels,kH,kW)

• bias – optional bias tensor of shape (out_channels)(out_channels). Default: None

• stride – the stride of the convolving kernel. Can be a single number or a tuple (sH, sW). Default: 1

• padding –

implicit paddings on both sides of the input. Can be a string {‘valid’, ‘same’}, single number or a tuple (padH, padW). Default: 0 padding='valid' is the same as no padding. padding='same' pads the input so the output has the same shape as the input. However, this mode doesn’t support any stride values other than 1

2 两者的不同

1. 首先是属性不同, torch.nn.Conv2d() 是一个 class, 而 torch.nn.functional.conv2d() 是一个函数

2. 调用方式不同:

nn.Conv2d() 需要在定义时设置超参数, 然后再将输入以调用的方式传入 torch.nn.Conv2d()

# torch.nn
inputs =  torch.randn(64, 3, 244, 244)
self.conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)
outputs = self.conv(inputs)

# torch.nn.functional    需要同时传入数据和 weight，bias等参数
inputs =  torch.randn(64, 3, 244, 244)
weight = torch.randn(64, 3, 3, 3)
bias = torch.randn(64)
outputs = nn.functinoal.conv2d(inputs, weight, bias, padding=1)

3. nn.xxx 能够放在 nn.Sequential里，而 nn.functional.xxx 不行

4. nn.functional.xxx 需要自己定义 weight，每次调用时都需要手动传入 weight，而 nn.xxx 则不用

import torch
import torch.nn as nn

# torch.nn 定义的CNN
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()

        self.conv_1 = nn.Conv2d(1, 16, krenel_size=5, padding=0)
        self.relu_1 = nn.ReLU(inplace=True)
        self.maxpool_1 = nn.MaxPool2d(kernel_size=2)

        self.conv_2 = nn.Conv2d(16, 32, krenel_size=5, padding=0)
        self.relu_2 = nn.ReLU(inplace=True)
        self.maxpool_2 = nn.MaxPool2d(kernel_size=2)   

        self.linear = nn.Linear(4*4*32, 10)

    def forward(self, x):
        x = x.view(x.size(0), -1)
        out = self.maxpool_1(self.relu_1(self.conv_1(x)))
        out = self.maxpool_2(self.relu_2(self.conv_2(out)))
        out = self.linear(out.view(x.size(0), -1))
        return out

import torch
import torch.nn as nn
import torch.nn.functional as F

# torch.nn.functional 定义一个相同的CNN
class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()

        self.conv_1_weight = nn.Parameter(torch.randn(16, 1, 5, 5))
        self.bias_1_weight = nn.Parameter(torch.randn(16))

        self.conv_2_weight = nn.Parameter(torch.randn(32, 16, 5, 5))
        self.bias_2_weight = nn.Parameter(torch.randn(32))

        self.linear_weight = nn.Parameter(torch.randn(4 * 4 * 32, 10))
        self.bias_weight = nn.Parameter(torch.randn(10))

    def forward(self, x):
        x = x.view(x.size(0), -1)
        out = F.conv2d(x, self.conv_1_weight, self.bias_1_weight)
        out = F.conv2d(out, self.conv_2_weight, self.bias_2_weight)
        out = F.linear(out.view(x.size(0), -1), self.linear_weight, self.bias_weight)

5. 在使用Dropout时，推荐使用 nn.xxx。因为一般只有训练时才使用 Dropout，在验证或测试时不需要使用 Dropout。使用 nn.Dropout时，如果调用 model.eval() ，模型的 Dropout 层都会关闭；但如果使用 nn.functional.dropout，在调用 model.eval() 时，不会关闭 Dropout

6. 当我们想要自定义卷积核时，是不能使用 torch.nn.ConvNd 的，因为它里面的权重都是需要学习的参数，没有办法自行定义。但是，我们可以使用 torch.nn.functional.conv2d()