pytorch学习笔记--nn.Module、nn.Sequential的搭建模型、损失函数、反向传播和训练器
nn.Module、nn.Sequential的搭建模型、损失函数和训练器
- 一、nn.Module
- 二、nn.Sequential
- 三、损失函数、反向传播和优化器
nn.Module和nn.equential都是pytorch中container的方法，用于保存深度学习层信息，即容器、模型
一、nn.Module
#使用nn.Module模块构建一个LeNet
import torch
import torch.nn as nn
import torch.nn.functional as fun
class LeNet(nn.Module):
#初始化即创建模型包含的层
def __init__(self):
super(LeNet, self).__init__()
self.Conv2D1 = nn.Conv2d(in_channels=3,out_channels=6,kernel_size=5,stride=1,padding=0)
self.activation1 = fun.sigmoid
self.Maxpool2d1 = nn.MaxPool2d(kernel_size=2,stride=2,padding=0)
self.Conv2D2 = nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5,stride=1,padding=0)
self.activation2 = fun.sigmoid
self.Maxpool2d2 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
self.flatten = nn.Flatten()
self.linear1 = nn.Linear(in_features=400,out_features=120)
self.activation3 = fun.sigmoid
self.linear2 = nn.Linear(in_features=120,out_features=84)
self.activation4 = fun.sigmoid
self.linear3 = nn.Linear(in_features=84,out_features=10)
#前向计算
def forward(self,x):
x = self.Conv2D1(x)
x = self.activation1(x)
x = self.Maxpool2d1(x)
x = self.Conv2D2(x)
x = self.activation2(x)
x = self.Maxpool2d2(x)
x = self.flatten(x)
x = self.linear1(x)
x = self.activation3(x)
x = self.linear2(x)
x = self.activation4(x)
x = self.linear3(x)
return x
if __name__ == '__main__':
input = torch.ones((64,3,32,32))
test = LeNet()
output = test(input)
print(output.shape)
输入一个batch_size 64，3通道，32*32像素的图片后
输出：
torch.Size([64, 10])
二、nn.Sequential
#使用nn.equential模块构建一个LeNet
import torch
import torch.nn as nn
LeNet = nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=6,kernel_size=5,stride=1,padding=0),
nn.Sigmoid(),
nn.MaxPool2d(kernel_size=2,stride=2,padding=0),
nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5,stride=1,padding=0),
nn.Sigmoid(),
nn.MaxPool2d(kernel_size=2, stride=2, padding=0),
nn.Flatten(),
nn.Linear(in_features=400,out_features=120),
nn.Sigmoid(),
nn.Linear(in_features=120,out_features=84),
nn.Sigmoid(),
nn.Linear(in_features=84,out_features=10),
)
if __name__ == '__main__':
input = torch.ones((64,3,32,32),dtype=torch.float32)
for layer in LeNet:
input = layer(input)
print(layer.__class__.__name__, 'output shape: \t', input.shape)
输出：
Conv2d output shape: torch.Size([64, 6, 28, 28])
Sigmoid output shape: torch.Size([64, 6, 28, 28])
MaxPool2d output shape: torch.Size([64, 6, 14, 14])
Conv2d output shape: torch.Size([64, 16, 10, 10])
Sigmoid output shape: torch.Size([64, 16, 10, 10])
MaxPool2d output shape: torch.Size([64, 16, 5, 5])
Flatten output shape: torch.Size([64, 400])
Linear output shape: torch.Size([64, 120])
Sigmoid output shape: torch.Size([64, 120])
Linear output shape: torch.Size([64, 84])
Sigmoid output shape: torch.Size([64, 84])
Linear output shape: torch.Size([64, 10])
模型.add_module(layer)可以往模型里添加层
三、损失函数、反向传播和优化器
损失函数都包含在torch.nn库中，Loss Functions官方文档
| 损失函数 | 作用 |
|---|---|
| nn.L1Loss | 平均绝对误差 |
| nn.MSELoss | 均方误差 |
| nn.CrossEntropyLoss | softmax和交叉熵误差的组合 |
常用的就是nn.CrossEntropyLoss
得到Loss之后，计算细节都保存在了loss赋值的变量中，对该变量进行loss_value.backward()即可完成反向传播，反向梯度信息都保存在了变量中。
使用优化器即可根据梯度信息而更新权重和偏置，优化器一般用SGD。torch.optim官方文档
from torch import nn,optim
from torchvision import datasets,transforms as tf
from torch.utils.data import DataLoader
#读取数据
train_datas = datasets.CIFAR10(root='./dataset',transform=tf.ToTensor(),train=True,download=True)
test_datas = datasets.CIFAR10(root='./dataset',transform=tf.ToTensor(),train=False,download=True)
train_iter = DataLoader(dataset=train_datas,batch_size=64,shuffle=True,num_workers=0,drop_last=True)
#创建模型
LeNet = nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=6,kernel_size=5,stride=1,padding=0),
nn.Sigmoid(),
nn.MaxPool2d(kernel_size=2,stride=2,padding=0),
nn.Conv2d(in_channels=6,out_channels=16,kernel_size=5,stride=1,padding=0),
nn.Sigmoid(),
nn.MaxPool2d(kernel_size=2, stride=2, padding=0),
nn.Flatten(),
nn.Linear(in_features=400,out_features=120),
nn.Sigmoid(),
nn.Linear(in_features=120,out_features=84),
nn.Sigmoid(),
nn.Linear(in_features=84,out_features=10),
)
loss = nn.CrossEntropyLoss()
train_model = LeNet
optimer = optim.SGD(train_model.parameters(),lr=0.9)
for epoch in range(3):
loss_all = 0.0
for data in train_iter:
img,label = data
y_hat = train_model(img)
l = loss(y_hat,label)
optimer.zero_grad()
l.backward()
optimer.step()
loss_all += l
print(loss_all)
损失函数、反向传播和优化器的使用按以下步骤：
〇设置损失函数和优化器
①得出loss
②训练器保存的梯度清零
③反向传播得到梯度
④训练器step()命令进行训练
结果：损失确实在减小
tensor(1805.3182, grad_fn=<AddBackward0>)
tensor(1801.2908, grad_fn=<AddBackward0>)
tensor(1800.9563, grad_fn=<AddBackward0>)