1.两者的调用方式不同
调用nn.xxx时要先在里面传入超参数，然后再将数据以函数调用的方式输进nn.xxx里，例如：
inputs = torch.rand(64, 3, 244, 244)
conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)
out = conv(inputs)
而nn.functional.xxx则要同时输入数据和weight,bias等参数，例如：
weight = torch.rand(64,3,3,3)
bias = torch.rand(64)
out = nn.functional.conv2d(inputs, weight, bias, padding=1)
2.nn.xxx能够放在nn.Sequential里，而nn.functional.xxx就不行
3.nn.functional.xxx需要自己定义weight,每次调用时都需要手动传入weight,而nn.xxx则不用，例如：
使用nn.xxx定义一个cnn:
class CNN(nn.Moudle)
def __init__(self):
super(CNN, self).__init__()
self.cnn1 = nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5,padding=0)
self.relu1 = nn.ReLU()
self.maxpool1 = nn.MaxPool2d(kernel_size=2)
self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, padding=0)
self.relu2 = nn.ReLU()
self.maxpool2 = nn.MaxPool2d(kernel_size=2)
self.linear1 = nn.Linear(4 * 4 * 32, 10)
def forward(self, x):
x = x.view(x.size(0), -1)
out = self.maxpool1(self.relu1(self.cnn1(x)))
out = self.maxpool2(self.relu2(self.cnn2(out)))
out = self.linear1(out.view(x.size(0), -1))
return out
使用nn.functional.xxx定义一个与上面相同的cnn:
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.cnn1_weight = nn.Parameter(torch.rand(16, 1, 5, 5))
self.bias1_weight = nn.Parameter(torch.rand(16))
self.cnn2_weight = nn.Parameter(torch.rand(32, 16, 5, 5))
self.bias2_weight = nn.Parameter(torch.rand(32))
self.linear1_weight = nn.Parameter(torch.rand(4 * 4 * 32, 10))
self.bias3_weight = nn.Parameter(torch.rand(10))
def forward(self, x):
x = x.view(x.size(0), -1)
out = F.conv2d(x, self.cnn1_weight, self.bias1_weight)
out = F.conv2d(x, self.cnn2_weight, self.bias2_weight)
out = F.linear(x, self.linear1_weight, self.bias3_weight)
return out
4.关于dropout,推荐使用nn.xxx。因为一般情况下只有训练时才用dropout,在eval不需要dropout。使用nn.Dropout,在调用model.eval()后，模型的dropout层都关闭，但用nn.functional.dropout，在调用model.eval()后不会关闭dropout.
5.有一种情况用nn.functional.xxx会更好，即如果行为相同，但参数矩阵相同的两个layer共享参数，可直接多次调用nn的Module。但若行为不同，比如想让两个dilation不同但kernel相同的卷积层共享参数，就可用到nn.functional,例如：
import torch
import torch.nn as nn
import torch.nn.functional as F
class Model(nn.Module):
def __init__(self):
super(Model, self).__init__()
self.weight = nn.Parameter(torch.Tensor(10,10,3,3))
def forward(self, x):
x_1 = F.conv2d(x, self.weight,dilation=1, padding=1)
x_2 = F.conv2d(x, self.weight,dilation=2, padding=2)
return x_1 + x_2
建议：在构建模型框架时（nn.Module）可用nn.xxx，在训练模型时可用nn.functional.xxx。另外，如果涉及到参数计算的，那用nn.；若不需要涉及更新参数，只是一次性计算，那用nn.functional。