PyTorch入门笔记（自用）
pytorch入门笔记
- 1.环境安装及配置
-
- 1.1 安装anaconda
- 1.2 验证安装
- 1.3 检查显卡驱动是否正常
- 1.4有序的安装环境---conda
- 1.5安装pytorch
- 1.6 检查是或否安装成功
- 2.第一个pytorch工程
-
- 2.1使用pycharm创建pytorch工程
-
- pycharm好处
- pycharm查看项目解释器
- 2.2使用jupyter创建pytorch工程
-
- 2.2.1 在pytorch环境中安装jupyter
- 2.2.2 jupyter 工作空间修改
- 2.3pycharm、pycharm console、jupyter三种不同环境的优劣比较
-
- 其中jupyter中help格式更加清楚
- 3.python中的两大函数---dir()、help()
-
- 3.1dir()、help()简单使用
- 4.pytorch如何加载数据
-
- 4.1 Dataset使用
-
- 4.1.1 Dataset官方解释
- 4.1.2 Dataset使用
- 4.1.3 tensorBoard使用
-
- add_scalar（）的使用
- add_image（）的使用
- 4.1.4 torchVision的transform使用
- 4.1.5常见的transforms的使用----Normalize(归一化)
- 4.1.5常见的transforms的使用----Resize(改变图片尺寸)
- 4.1.6常见的transforms的使用----RandomCrop(随机裁剪)
- 4.1.7torchversions数据集的使用
- 4.2dataloader使用
- 5.神经网络的搭建---nn.model
-
- 5.1 Containers神经网络骨架搭建
- 5.2卷积操作
-
- 5.2.1torch.nn.functional的使用
- 5.3 神经网络--卷积层（以二维卷积为例）
- 5.4神经网络--最大池化的使用
- 5.5神经网络---非线性激活
- 5.6神经网络--正则化层
- 5.7神经网络--recurrent层
- 5.8 神经网络--Transformer Layers
- 5.9神经网络--Linear Layers
- 5.10神经网络--Dropout Layers
- 5.11实战：搭建神经为网络和Sequential的使用
- 6. 衡量误差--Loss Functions--反向传播
-
- 6.1 L1LOSS
- 6.2 MSELOSS
- 6.3交叉熵--CROSSENTROPYLOSS
- 6.4 loss function应用
- 6.5 如何根据据loss function 的输出调优
- 7. 优化器---torch.optim
- 8. 网络模型的加载和修改
- 9.模型的保存和加载
- 10. 完整的模型训练
-
- 10.1正确率
- 11.使用gpu训练
-
- 1. 方式一
- 2. 方式二
- 使用 google colab 免费每周30小时
1.环境安装及配置
1.1 安装anaconda
官方下载地址：https://www.anaconda.com/products/individual#Downloads
1.2 验证安装
安装成功
1.3 检查显卡驱动是否正常
任务管理器性能处显示nvidia则正常
1.4有序的安装环境—conda
Conda 是一个开源的软件包管理系统和环境管理系统，用于安装多个版本的软件包及其依赖关系，并在它们之间轻松切换。
新创建一个环境，下面在这个环境中安装pytorch
指令如下：
conda create -n pytorch python=3.6
- n：name环境的名字
- pythoon=3.6 指定python版本为3.6版本
安装成功：
相当于当前anaconda环境中安装了两个不同python版本的环境：
切换到当前环境：
查看安装的环境及位置：
查看当前存在的包：
命令：
pip list
1.5安装pytorch
上一步查看新创建的环境中并没有pytorch，所以接下来进行pytorch的安装
查看GPU型号：
pytorch官方地址：https://pytorch.org/
官网下拉，根据自己机器情况选择：
Compute Platform选择：
conda 命令支持英伟达显卡驱动396.26版本以上。
查看自己显卡驱动版本：
复制生成命令运行：
下载过程中输入y确认下载。
1.6 检查是或否安装成功
1.输入python
2.输入 import torch
3. 查看pytorch是否可用电脑GPU
torch.cuda.is_available()
2.第一个pytorch工程
2.1使用pycharm创建pytorch工程
（1）
（2）
（3）上一步确定后
（4）验证是否成功导入anaconda环境
pycharm好处
控制台右侧可显示变量赋值情况，比较直观。
pycharm查看项目解释器
2.2使用jupyter创建pytorch工程
在安装anaconda时，jupyter已经安装完成。(默认安装在base环境中,之前新建的pytorch环境没有)
2.2.1 在pytorch环境中安装jupyter
(1)查看jupyter使用的package
anaconda prompt 命令（base环境中安装了jupyter）
conda list
(2）进入pytorch环境
conda activate pytorch
查看所含包并不含有上面那个
conda list
（3）安装jupyter所需包
中途y确定
（4）启动运行
jupyter notebook
回到自动打开：
命令：
（5）使用codee环境的pytorch创建新项目
创建完成：
shift+enter跳转到下一个代码块，并且运行上一个代码块
验证是否成功
2.2.2 jupyter 工作空间修改
jupyter工作空间修改
2.3pycharm、pycharm console、jupyter三种不同环境的优劣比较
ps: shift+enter #跳转到下一行开始输出
其中jupyter中help格式更加清楚
方式一：
方式二：
3.python中的两大函数—dir()、help()
3.1dir()、help()简单使用
#查看torch目录
dir(torch)
#查看torch下的cuda目录
dir(torch.cuda)
#查看torch下的cuda下的is_available目录
dir(torch.cuda.is_available())
#查看帮助
**#注意：方法不加括号**
help(torch.cuda.is_available)
4.pytorch如何加载数据
# 数据读取
Dataset 类
Dataloader 类
4.1 Dataset使用
4.1.1 Dataset官方解释
1.使用Dataset必须继承Dataset
2.子类必须重写getitem方法，支持获取给定键下标的数据示例
3.选择重写len方法获取长度
4.1.2 Dataset使用
from torch.utils.data import Dataset
from PIL import Image
import os
class MyData(Dataset):
def __init__(self,root_dir,label_dir):
#构造函数定义全局变量
self.root_dir=root_dir
self.label_dir=label_dir
self.path=os.path.join(self.root_dir,self.label_dir)
self.img_path=os.listdir(self.path)
def __getitem__(self,idx):
img_name=self.img_path[idx]
img_item_path=os.path.join(self.root_dir,self.label_dir,img_name)
img=Image.open(img_item_path)
label=self.label_dir
return img,label
def __len__(self):
return len(self.img_path)
root_dir="dataset/train"
ants_label_dir="ants"
bees_label_dir="bees"
ants_dataset=MyData(root_dir,ants_label_dir)
bees_dataset=MyData(root_dir,bees_label_dir)
train_dataset=ants_dataset+bees_dataset
4.1.3 tensorBoard使用
add_scalar（）的使用
安装tensorboard包：在anconda的pytorch环境中：
pip install tensorboard
如果报错： ModuleNotFoundError: No module named ‘past’
pip install future
绘制一个简单函数：
from torch.utils.tensorboard import SummaryWriter
#1.初始化SummaryWriter并设置存储文件夹
writer=SummaryWriter("logs")
# writer.add_image()
for i in range(100):
writer.add_scalar("y=x",2*i,i)
writer.close()
打开绘制函数：
tensorboard --logdir=logs --port=6007
--port 指定端口，默认端口：6006
出现：no dashboards are active for the current data set.可能是路径问题
注意：在pycharm的终端打开可以是相对路径
在anconda的终端中打开为绝对路径
add_image（）的使用
使用python图像库PIL读取的图片格式不是add_image()所需要的数据类型。使用opencv读取的图像格式为numpy型符合，安装opencv。
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=768x512 at 0x1EBDD91BCF8>
使用淘宝镜像安装opencv
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple opencv-python
tensorboard的SummaryWriter类中的add_image()函数中img_tensor（图片参数）默认格式为（3,H,W）3通道，高，宽，也可以使用（H,W,3）格式，但需要设置dataformats参数，numpy型格式图片的数学格式为（H,W,3）所以需要设置dataformats参数
源码实例：
# If you have non-default dimension setting, set the dataformats argument.
writer.add_image('my_image_HWC', img_HWC, 0, dataformats='HWC')
writer.close()
展示图片代码：
from torch.utils.tensorboard import SummaryWriter
from PIL import Image
import numpy as np
image_path="dataset/train/ants_image/0013035.jpg"
#使用PIL中的Image读取图片
img_PIL=Image.open(image_path)
#使用numpy转换图片格式
img_array=np.array(img_PIL)
print(type(img_array))
print(img_array.shape)
writer=SummaryWriter("logs")
#1为设置为第一步
writer.add_image("我是tag",img_array,1,dataformats='HWC')
writer.close()
当设置为第二步后：
from torch.utils.tensorboard import SummaryWriter
from PIL import Image
import numpy as np
image_path="dataset/train/bees_image/39672681_1302d204d1.jpg"
#使用PIL中的Image读取图片
img_PIL=Image.open(image_path)
#使用numpy转换图片格式
img_array=np.array(img_PIL)
print(type(img_array))
print(img_array.shape)
writer.add_image("我是tag",img_array,2,dataformats='HWC')
writer.close()
修改属性global_step=2添加第二步：
修改了tag属性不会添加图片显示步骤：
4.1.4 torchVision的transform使用
transform相当于一个工具，里面包含各种工具类，对图片数据进行转换。（主要是将图片转换成tensor格式，便于神经网络训练数据）
transform使用:
from PIL import Image
from torchvision import transforms
img_path="dataset/train/ants_image/0013035.jpg"
img_PIL=Image.open(img_path)
#transform该如何使用
#实例化Totensor类
tensor_transform=transforms.ToTensor()
img_tensor=tensor_transform(img_PIL)
print(img_tensor)
输出转化为tensor类型后的图片：
tensor([[[0.3137, 0.3137, 0.3137, ..., 0.3176, 0.3098, 0.2980],
[0.3176, 0.3176, 0.3176, ..., 0.3176, 0.3098, 0.2980],
[0.3216, 0.3216, 0.3216, ..., 0.3137, 0.3098, 0.3020],
...,
[0.3412, 0.3412, 0.3373, ..., 0.1725, 0.3725, 0.3529],
[0.3412, 0.3412, 0.3373, ..., 0.3294, 0.3529, 0.3294],
[0.3412, 0.3412, 0.3373, ..., 0.3098, 0.3059, 0.3294]],
[[0.5922, 0.5922, 0.5922, ..., 0.5961, 0.5882, 0.5765],
[0.5961, 0.5961, 0.5961, ..., 0.5961, 0.5882, 0.5765],
[0.6000, 0.6000, 0.6000, ..., 0.5922, 0.5882, 0.5804],
...,
[0.6275, 0.6275, 0.6235, ..., 0.3608, 0.6196, 0.6157],
[0.6275, 0.6275, 0.6235, ..., 0.5765, 0.6275, 0.5961],
[0.6275, 0.6275, 0.6235, ..., 0.6275, 0.6235, 0.6314]],
[[0.9137, 0.9137, 0.9137, ..., 0.9176, 0.9098, 0.8980],
[0.9176, 0.9176, 0.9176, ..., 0.9176, 0.9098, 0.8980],
[0.9216, 0.9216, 0.9216, ..., 0.9137, 0.9098, 0.9020],
...,
[0.9294, 0.9294, 0.9255, ..., 0.5529, 0.9216, 0.8941],
[0.9294, 0.9294, 0.9255, ..., 0.8863, 1.0000, 0.9137],
[0.9294, 0.9294, 0.9255, ..., 0.9490, 0.9804, 0.9137]]])
tensor数据类型包含了神经王网络需要的一些参数
4.1.5常见的transforms的使用----Normalize(归一化)
将图片所传三个信道的值，归一化。
from PIL import Image
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter
img=Image.open("练手数据集/train/ants_image/0013035.jpg")
print(img)
#转换图片格式为tensor格式
trans_totensor=transforms.ToTensor()
img_tensor=trans_totensor(img)
#绘制步骤
writer=SummaryWriter("logs")
writer.add_image("Totensor",img_tensor)
#Normalize-----归一化
print(img_tensor[0][0][0])
#归一化计算公式 ：output[channel] = (input[channel] - mean[channel]) / std[channel]
#输入三个均值三个标准差
trans_norm=transforms.Normalize([0.5,0.5,0.5],[0.5,0.5,0.5])
img_norm=trans_norm(img_tensor)
print(img_norm[0][0][0])
writer.add_image("Normal",img_norm)
writer.close()
<PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=768x512 at 0x19EC556ADA0>
tensor(0.3137)
tensor(-0.3725)
4.1.5常见的transforms的使用----Resize(改变图片尺寸)
from PIL import Image
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter
img=Image.open("练手数据集/train/ants_image/0013035.jpg")
print(img)
#转换图片格式为tensor格式
trans_totensor=transforms.ToTensor()
img_tensor=trans_totensor(img)
#绘制步骤
writer=SummaryWriter("logs")
writer.add_image("Totensor",img_tensor)
#Resize ---改变尺寸
print(img.size)
trans_resize=transforms.Resize((512,512))
#传入PIL格式就返回PIL格式，传入tensor格式就返回tensor格式
img_resize=trans_resize(img)
img_resize=trans_resize(img_tensor)
writer.add_image("Resize",img_resize,0)
print(img_resize)
writer.close()
这个compose相当于是一个方法的顺序组合，列表里面放的都是方法，前一个的输出会当成后一个的输入进行，可以说是一种减少代码的方式
#Compose()----Resize 2. 输入一边 -----等比缩放
trans_resize_2=transforms.Resize(333)
trans_compose=transforms.Compose([trans_resize_2,trans_totensor])
# img_resize_2=trans_compose(img)
img_resize_2=trans_compose(img)
writer.add_image("Resize",img_resize_2,1)
writer.close()
from PIL import Image
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter
img=Image.open("练手数据集/train/ants_image/0013035.jpg")
print(img)
#转换图片格式为tensor格式
trans_totensor=transforms.ToTensor()
img_tensor=trans_totensor(img)
#绘制步骤
writer=SummaryWriter("logs")
writer.add_image("Totensor",img_tensor)
4.1.6常见的transforms的使用----RandomCrop(随机裁剪)
from PIL import Image
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter
img=Image.open("练手数据集/train/ants_image/0013035.jpg")
print(img)
#转换图片格式为tensor格式
trans_totensor=transforms.ToTensor()
img_tensor=trans_totensor(img)
#绘制步骤
writer=SummaryWriter("logs")
trans_random=transforms.RandomCrop(512)
trans_compose_2=transforms.Compose([trans_random,trans_totensor])
for i in range(10):
img_crop=trans_compose_2(img)
writer.add_image("RanDomCrop",img_crop,i)
writer.close()
4.1.7torchversions数据集的使用
数据集源码中有数据的下载地址
4.2dataloader使用
dataloader官网介绍：dataloader官方文档
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p11_dataloader.py
# @Time :2021/10/16 16:11
# @Author :Administrator
import torch
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
#准备测试数据集
test_data=torchvision.datasets.CIFAR10("./dataset",
train=False,
transform=torchvision.transforms.ToTensor())
#drop_last=False全部加载 =true丢掉零散的留下
test_dateloader=DataLoader(dataset=test_data,batch_size=64,shuffle=True,num_workers=0,drop_last=True)
#测试数据集中第一张图片
img,target=test_data[0]
print(img.shape)
print(target)
writer=SummaryWriter("dataloader")
#取出4个为一组的数据
# step=0
# for data in test_dateloader:
# imgs,targets=data
# print(imgs.shape)
# print(targets)
# writer.add_images("test_dataloader_drop_last=False",imgs,step)
# step=step+1
#测试每次加载的数据顺序
for epoch in range(2):
step=0
for data in test_dateloader:
imgs,targets=data
print(imgs.shape)
print(targets)
writer.add_images("Epoch{}".format(epoch),imgs,step)
step=step+1
writer.close()
5.神经网络的搭建—nn.model
torch.nn : https://pytorch.org/docs/stable/nn.html
5.1 Containers神经网络骨架搭建
Containers: https://pytorch.org/docs/stable/nn.html#containers
Module :提供神经网络基本骨架的类 https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module
前向传播：
forwards函数：
一个简单的神经网络：
import torch
from torch import nn
class ModelTest(nn.Module):
def __init__(self):
super().__init__()
def forward(self,input):
output=input+1
return output
modeltest=ModelTest()
#tensor类型的数据1
x=torch.tensor(1.0)
output=modeltest(x)
print(output)
5.2卷积操作
torch.nn对torch.nn.functional进行了封装，更方便使用。
5.2.1torch.nn.functional的使用
torch.nn.functional.conv2d(input, weight, bias=None, stride=1, padding=0, dilation=1, groups=1) → Tensor
对应位置相乘再相加。
padding=1时的计算方式：
使用上面的图像和卷积核进行卷积操作：
import torch
import torch.nn.functional as F
#输入图像矩阵
input=torch.tensor([[1,2,0,3,1],
[0,1,2,3,1],
[1,2,1,0,0],
[5,2,3,1,1],
[2,1,0,1,1]])
#卷积核矩阵
kernel=torch.tensor([[1,2,1],
[0,1,0],
[2,1,0]])
#input – input tensor of shape (minibatch,in_channels,iH,iW)
# 平面图minibatch=1，in_channels=1,5*5
input=torch.reshape(input,(1,1,5,5))
kernel=torch.reshape(kernel,(1,1,3,3))
print(input.shape)
print(kernel.shape)
output=F.conv2d(input,kernel,stride=1)
print(output)
output2=F.conv2d(input,kernel,stride=2)
print(output2)
output3=F.conv2d(input,kernel,stride=1,padding=1)
print(output3)
输出：
torch.Size([1, 1, 5, 5])
torch.Size([1, 1, 3, 3])
tensor([[[[10, 12, 12],
[18, 16, 16],
[13, 9, 3]]]])
tensor([[[[10, 12],
[13, 3]]]])
tensor([[[[ 1, 3, 4, 10, 8],
[ 5, 10, 12, 12, 6],
[ 7, 18, 16, 16, 8],
[11, 13, 9, 3, 4],
[14, 13, 9, 7, 4]]]])
5.3 神经网络–卷积层（以二维卷积为例）
https://pytorch.org/docs/stable/nn.html#convolution-layers
strides&padding图示：
蓝色部分—输入图像
绿色部分—输出图像
灰色部分—卷积核
kernel_size :定义了卷积核大小（尺寸）例如：kernal_size=3 --> 3*3的卷积核，参数从采样中方获取。
in_channels =1,out_channels=1 ,只会使用一个卷积核进行卷积，得到一个输出结果，
，out_channels=2,卷积层会生成2个卷积核进行卷积，得到两个输出结果，再将两个结果叠加作为输出。
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p14_nn_conv2d.py
# @Time :2021/10/17 21:29
# @Author :Administrator
import torch
import torchvision
from torch import nn
from torch.nn import Conv2d
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
dataset=torchvision.datasets.CIFAR10("./dataset",
train=False,
transform=torchvision.transforms.ToTensor(),
download=True)
dataloader=DataLoader(dataset,b )
class ConvolutionTest(nn.Module):
def __init__(self):
super(ConvolutionTest, self).__init__()
self.conv1=Conv2d(in_channels=3,out_channels=6,
kernel_size=3,stride=1,padding=0)
def forward(self,x):
x=self.conv1(x);
return x
convtest=ConvolutionTest()
print(convtest)
writer=SummaryWriter("./p14logs")
step=0
for data in dataloader:
imgs,targets=data
output=convtest(imgs)
print(imgs.shape)
print(output.shape)
#torch.Size([64, 3, 32, 32])
writer.add_images("p14_input",imgs,step)
#torch.Size([64, 6, 30, 30])---卷积核要求输入如三个通道所以要对数据进行转变
#--->[xxx,3,30,30]
output=torch.reshape(output,(-1,3,30,30))
writer.add_images("p14_output",output,step)
step=step+1;
writer.close()
5.4神经网络–最大池化的使用
https://pytorch.org/docs/stable/generated/torch.nn.MaxPool2d.html#torch.nn.MaxPool2d
作用：保持输入的特征，并把数据量减小。
**dilation:**设置卷积核的取样间隔。
ceil_model: true向上取整，false向下取整。
import torch
from torch import nn
from torch.nn import MaxPool2d
input=torch.tensor([[1,2,0,3,1],
[0,1,2,3,1],
[1,2,1,0,0],
[5,2,3,1,1],
[2,1,0,1,1]],dtype=torch.float32)
#batch_size=-1 //自己计算
#channel=1
#5*5矩阵
input=torch.reshape(input,(-1,1,5,5))
print(input.shape)
class MaxpoolTest(nn.Module):
def __init__(self):
super(MaxpoolTest, self).__init__()
self.maxpool1=MaxPool2d(kernel_size=3,ceil_mode=True)
def forward(self,input):
output=self.maxpool1(input)
return output;
maxpoolTest=MaxpoolTest()
output=maxpoolTest(input)
print(output)
输出：
torch.Size([1, 1, 5, 5])
tensor([[[[2., 3.],
[5., 1.]]]])
示例：
import torch
from torch import nn
from torch.nn import MaxPool2d
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
dataset=torchvision.datasets.CIFAR10("./dataset",
train=False,
transform=torchvision.transforms.ToTensor())
dataloader=DataLoader(dataset,batch_size=64)
class MaxpoolTest(nn.Module):
def __init__(self):
super(MaxpoolTest, self).__init__()
self.maxpool1=MaxPool2d(kernel_size=3,ceil_mode=True)
def forward(self,input):
output=self.maxpool1(input)
return output;
maxpoolTest=MaxpoolTest()
writer=SummaryWriter("./logs_maxpool")
step=0
for data in dataloader:
imgs,targets=data
writer.add_images("maxpool_input", img_tensor=imgs,global_step=step)
ouput=maxpoolTest(imgs)
writer.add_images("maxpool_output",ouput,step)
step=step+1
writer.close()
tensorboard展示：
5.5神经网络—非线性激活
给神经网络中引入非线性特质。
https://pytorch.org/docs/stable/generated/torch.nn.ReLU.html#torch.nn.ReLU 为例
代码：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p16_nn_relu.py
# @Time :2021/10/19 22:39
# @Author :Administrator
import torch
from torch import nn
from torch.nn import ReLU
input=torch.tensor([[1,-0.5],
[-1,3]])
#-1表示batch_size（不设置参数）自己算 1表示一维(是数据只有一层)
#这里主要为了得出batch_size
input=torch.reshape(input,(-1,1,2,2))
print(input.shape)
class NonLinerActionTest(nn.Module):
def __init__(self):
super(NonLinerActionTest, self).__init__()
self.relu1=ReLU()
def forward(self,input):
output=self.relu1(input)
return output
test=NonLinerActionTest()
output=test(input)
print(output)
输出：输出结果根据图像得出-----输入中小于0的数被截断
torch.Size([1, 1, 2, 2])
tensor([[[[1., 0.],
[0., 3.]]]])
https://pytorch.org/docs/stable/generated/torch.nn.Sigmoid.html#torch.nn.Sigmoid
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p16_nn_relu.py
# @Time :2021/10/19 22:39
# @Author :Administrator
import torch
import torchvision
from torch import nn
from torch.nn import ReLU, Sigmoid
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
input=torch.tensor([[1,-0.5],
[-1,3]])
#-1表示batch_size（不设置参数）自己算 1表示一维(是数据只有一层)
#这里主要为了得出batch_size
input=torch.reshape(input,(-1,1,2,2))
print(input.shape)
dataset=torchvision.datasets.CIFAR10("./dataset",train=False,download=True,
transform=torchvision.transforms.ToTensor())
dataloader=DataLoader(dataset,batch_size=64)
class NonLinerActionTest(nn.Module):
def __init__(self):
super(NonLinerActionTest, self).__init__()
self.relu1=ReLU()
self.sigmoid1=Sigmoid()
def forward(self,input):
output=self.sigmoid1(input)
return output
test=NonLinerActionTest()
output=test(input)
print(output)
print("-------------------")
writer=SummaryWriter("./logs_sigMoid")
step=0
for data in dataloader:
imgs,targets=data
writer.add_images("input",imgs,step)
output=test(imgs)
writer.add_images("output",output,step)
step=step+1
writer.close()
5.6神经网络–正则化层
https://pytorch.org/docs/stable/nn.html#normalization-layers
5.7神经网络–recurrent层
5.8 神经网络–Transformer Layers
5.9神经网络–Linear Layers
import torch
import torchvision
from torch import nn
from torch.nn import Linear
from torch.utils.data import DataLoader
dataset=torchvision.datasets.CIFAR10("./dataset",train=False,
transform=torchvision.transforms.ToTensor(),download=True)
dataloader=DataLoader(dataset,batch_size=64,drop_last=True)
class LinearTest(nn.Module):
def __init__(self):
super(LinearTest, self).__init__()
self.linear1=Linear(196608,10)
def forward(self,input):
output=self.linear1(input)
return output
testLinear=LinearTest()
for data in dataloader:
imgs,target=data
print(imgs.shape)
#将输入摊平
# output=torch.reshape(imgs,(1,1,1,-1))
output=torch.flatten(imgs)
print(output.shape)
output=testLinear(output)
print(output.shape)
5.10神经网络–Dropout Layers
防止过拟合
比较常用于自然语言处理中
5.11实战：搭建神经为网络和Sequential的使用
CIFAR10模型结构：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p18_nn_seq.py
# @Time :2021/10/20 22:16
# @Author :Administrator
import torch
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
class SeqTest(nn.Module):
def __init__(self):
# super(SeqTest, self).__init__()
# self.conv1=Conv2d(in_channels=3,
# out_channels=32,
# kernel_size=5,
# padding=2)
# self.maxpool1=MaxPool2d(kernel_size=2)
# self.conv2=Conv2d(in_channels=32,
# out_channels=32,
# kernel_size=5,
# padding=2)
# self.maxpool2=MaxPool2d(kernel_size=2)
# self.conv3=Conv2d(in_channels=32,
# out_channels=64,
# kernel_size=5,
# padding=2)
# self.maxpool3=MaxPool2d(kernel_size=2)
# self.flatten=Flatten()
# self.linear1=Linear(in_features=1024,
# out_features=64)
# self.linear2=Linear(in_features=64,
# out_features=10)
self.model1=Sequential(
Conv2d(in_channels=3,
out_channels=32,
kernel_size=5,
padding=2),
MaxPool2d(kernel_size=2),
Conv2d(in_channels=32,
out_channels=32,
kernel_size=5,
padding=2),
MaxPool2d(kernel_size=2),
Conv2d(in_channels=32,
out_channels=32,
kernel_size=5,
padding=2),
MaxPool2d(kernel_size=2),
Flatten(),
Linear(in_features=1024,
out_features=64),
Linear(in_features=64,
out_features=10)
)
def forward(self,x):
x=self.conv1(x)
x=self.maxpool1(x)
x = self.conv2(x)
x = self.maxpool2(x)
x = self.conv3(x)
x = self.maxpool3(x)
x=self.flatten(x)
x=self.linear1(x)
x=self.linear2(x)
return x
test=SeqTest()
print(test)
#pytorch提供了许多创建数据的函数
input=torch.ones(64,3,32,32)
output=test(input)
print(output.shape)
```python
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p18_nn_seq.py
# @Time :2021/10/20 22:16
# @Author :Administrator
import torch
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.tensorboard import SummaryWriter
class SeqTest(nn.Module):
def __init__(self):
super(SeqTest, self).__init__()
# self.conv1=Conv2d(in_channels=3,
# out_channels=32,
# kernel_size=5,
# padding=2)
# self.maxpool1=MaxPool2d(kernel_size=2)
# self.conv2=Conv2d(in_channels=32,
# out_channels=32,
# kernel_size=5,
# padding=2)
# self.maxpool2=MaxPool2d(kernel_size=2)
# self.conv3=Conv2d(in_channels=32,
# out_channels=64,
# kernel_size=5,
# padding=2)
# self.maxpool3=MaxPool2d(kernel_size=2)
# self.flatten=Flatten()
# self.linear1=Linear(in_features=1024,
# out_features=64)
# self.linear2=Linear(in_features=64,
# out_features=10)
self.model1=Sequential(
Conv2d(in_channels=3,out_channels=32,kernel_size=5,padding=2),
MaxPool2d(kernel_size=2),
Conv2d(in_channels=32,out_channels=32,kernel_size=5,padding=2),
MaxPool2d(kernel_size=2),
Conv2d(in_channels=32,out_channels=64,kernel_size=5,padding=2),
MaxPool2d(kernel_size=2),
Flatten(),
Linear(in_features=1024,out_features=64),
Linear(in_features=64,out_features=10)
)
def forward(self,x):
# x=self.conv1(x)
# x=self.maxpool1(x)
# x = self.conv2(x)
# x = self.maxpool2(x)
# x = self.conv3(x)
# x = self.maxpool3(x)
# x=self.flatten(x)
# x=self.linear1(x)
# x=self.linear2(x)
self.model1(x)
return x
test=SeqTest()
print(test)
#pytorch提供了许多创建数据的函数
input=torch.ones((64,3,32,32))
output=test(input)
print(output.shape)
writer=SummaryWriter("./logs_seq")
writer.add_graph(test,input)
writer.close()
输出：
SeqTest(
(conv1): Conv2d(3, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(maxpool1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2): Conv2d(32, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(maxpool2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))
(maxpool3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(flatten): Flatten(start_dim=1, end_dim=-1)
(linear1): Linear(in_features=1024, out_features=64, bias=True)
(linear2): Linear(in_features=64, out_features=10, bias=True)
)
E:\Program Files\anaconda\Anaconda3\envs\pytorch\lib\site-packages\torch\nn\functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ..\c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
torch.Size([64, 10])
Process finished with exit code 0
6. 衡量误差–Loss Functions–反向传播
https://pytorch.org/docs/stable/nn.html#loss-functions
类比，一张试卷100分（target） ，实际考的分数（ouput）和满分100分的差距。
衡量误差存在的意义就是根据这个差距来衡量模型和的误差，再尽可能的减少误差，优化模型。
6.1 L1LOSS
https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html#torch.nn.L1Loss
原理：根据输入和输出判断误差。
import torch
from torch.nn import L1Loss
inputs=torch.tensor([1,2,3],dtype=torch.float32)
targets=torch.tensor([1,2,5],dtype=torch.float32)
# #1 batch_size,1 channel, 1行3列
# inputs=torch.reshape(inputs,(1,1,1,3))
# inputs=torch.reshape(targets,(1,1,1,3))
# loss=L1Loss(reduction="sum")
loss=L1Loss()
results=loss(inputs,targets)
print(results)
输出：
tensor(0.6667)
6.2 MSELOSS
import torch
from torch.nn import L1Loss
from torch import nn
inputs=torch.tensor([1,2,3],dtype=torch.float32)
targets=torch.tensor([1,2,5],dtype=torch.float32)
loss_mse=nn.MSELoss()
results=loss_mse(inputs,targets)
print(results)
tensor(1.3333)
6.3交叉熵–CROSSENTROPYLOSS
公式原理：
符号exp是高等数学中表示以自然常数e为底的指数函数。
通俗来讲，exp()表示的含义就是e的几次方，比如exp(3)就表示e³。
import torch
from torch import nn
x=torch.tensor([0.1,0.2,0.3])
y=torch.tensor([1])
#1batch_size 有3类
x=torch.reshape(x,(1,3))
loss_cross=nn.CrossEntropyLoss()
result_cross=loss_cross(x,y)
print(result_cross)
输出：
tensor(1.1019)
6.4 loss function应用
import torch
import torchvision
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
dataset=torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor,download=True)
dataLoader=DataLoader(dataset=dataset,batch_size=64,)
class SeqTest(nn.Module):
def __init__(self):
super(SeqTest, self).__init__()
self.conv1=Conv2d(in_channels=3,
out_channels=32,
kernel_size=5,
padding=2)
self.maxpool1=MaxPool2d(kernel_size=2)
self.conv2=Conv2d(in_channels=32,
out_channels=32,
kernel_size=5,
padding=2)
self.maxpool2=MaxPool2d(kernel_size=2)
self.conv3=Conv2d(in_channels=32,
out_channels=64,
kernel_size=5,
padding=2)
self.maxpool3=MaxPool2d(kernel_size=2)
self.flatten=Flatten()
self.linear1=Linear(in_features=1024,
out_features=64)
self.linear2=Linear(in_features=64,
out_features=10)
# self.model1=Sequential(
# Conv2d(in_channels=3,out_channels=32,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Conv2d(in_channels=32,out_channels=32,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Conv2d(in_channels=32,out_channels=64,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Flatten(),
# Linear(in_features=1024,out_features=64),
# Linear(in_features=64,out_features=10)
# )
def forward(self,x):
x=self.conv1(x)
x=self.maxpool1(x)
x = self.conv2(x)
x = self.maxpool2(x)
x = self.conv3(x)
x = self.maxpool3(x)
x=self.flatten(x)
x=self.linear1(x)
x=self.linear2(x)
# x=self.model1(x)
return x
loss_cross=nn.CrossEntropyLoss()
test=SeqTest()
for data in dataLoader:
imgs,targets=data
outputs=test(imgs)
print("-------------------------outputs-----------------------")
print(outputs)
print("-------------------------targets-----------------------")
print(targets)
result_loss=loss_cross(outputs,targets)
print("-------------------------result_loss--------------------")
print(result_loss)
#反向传播计算梯度
result_loss.backward()
一组输出：
outputs是预测值，target是真实值，result_loss是神经网络预测值和真是值得误差。
-------------------------outputs-----------------------
tensor([[ 3.7552e-02, 6.4410e-02, 8.3789e-02, 9.6082e-03, 3.4134e-02,
-3.6156e-02, 6.5377e-02, 1.8092e-01, 7.6961e-02, 4.0065e-02],
[ 3.7418e-02, 6.1485e-02, 9.7359e-02, 3.7872e-03, 3.7664e-02,
-3.4812e-02, 5.4483e-02, 1.9028e-01, 8.4479e-02, 4.2509e-02],
[ 3.9113e-02, 6.6564e-02, 9.9516e-02, 3.4380e-03, 1.8161e-02,
-4.6894e-02, 4.6006e-02, 1.8605e-01, 9.9289e-02, 3.3509e-02],
[ 4.3493e-02, 7.4313e-02, 9.0312e-02, -3.3124e-03, 2.3912e-02,
-3.3897e-02, 4.4228e-02, 1.8335e-01, 8.8963e-02, 3.1292e-02],
[ 3.6908e-02, 5.8817e-02, 8.8594e-02, 1.0138e-02, 1.8756e-02,
-5.0329e-02, 6.7451e-02, 1.7342e-01, 8.7347e-02, 4.8388e-02],
[ 4.7056e-02, 8.7556e-02, 8.8112e-02, -1.0755e-02, 2.4851e-02,
-3.0836e-02, 7.1425e-02, 1.7452e-01, 8.6627e-02, 1.9660e-02],
[ 5.1619e-02, 7.5532e-02, 8.9899e-02, -1.3963e-02, 1.1116e-02,
-3.6120e-02, 5.3261e-02, 1.9038e-01, 9.4373e-02, 4.3228e-02],
[ 4.5692e-02, 6.8340e-02, 1.1157e-01, -1.9689e-02, 9.8723e-03,
-3.6184e-02, 1.4657e-02, 2.1161e-01, 1.2959e-01, 1.0795e-02],
[ 3.5025e-02, 6.6935e-02, 7.7888e-02, 1.6519e-03, 3.3473e-02,
-2.6925e-02, 8.3356e-02, 1.6662e-01, 7.2409e-02, 4.6379e-02],
[ 4.1965e-02, 6.6662e-02, 1.0446e-01, 1.4804e-02, 3.4399e-02,
-4.5847e-02, 6.2242e-02, 1.7467e-01, 9.5238e-02, 4.5444e-02],
[ 5.3438e-02, 6.0671e-02, 8.5751e-02, -3.4994e-03, 1.7321e-02,
-3.9534e-02, 7.4328e-02, 1.7189e-01, 9.5918e-02, 3.2937e-02],
[ 3.5964e-02, 6.6520e-02, 6.3098e-02, -1.4790e-02, 1.8050e-02,
-3.8724e-02, 8.4594e-02, 1.6214e-01, 6.2974e-02, 5.1876e-02],
[ 5.1205e-02, 7.3963e-02, 1.0138e-01, -3.0158e-02, 1.2683e-02,
-5.4160e-02, 1.9545e-02, 2.0819e-01, 1.2105e-01, 5.7859e-03],
[ 4.6458e-02, 6.8363e-02, 1.0108e-01, -2.8238e-03, 2.4385e-02,
-3.2310e-02, 3.5553e-02, 1.9156e-01, 1.1049e-01, 3.7765e-02],
[ 3.7014e-02, 6.7696e-02, 1.0432e-01, -1.2529e-02, 2.5199e-02,
-4.5576e-02, 3.3879e-02, 2.0607e-01, 1.1805e-01, 1.6299e-02],
[ 3.4600e-02, 6.5787e-02, 9.8796e-02, 3.3232e-03, 2.2426e-02,
-3.2910e-02, 5.3063e-02, 1.9640e-01, 1.0627e-01, 1.3252e-02],
[ 6.5476e-02, 6.7269e-02, 1.0025e-01, -8.5620e-03, 2.4712e-02,
-3.8320e-02, 4.9723e-02, 1.8369e-01, 1.0927e-01, 4.0947e-02],
[ 4.4738e-02, 6.2579e-02, 9.3848e-02, -3.6691e-03, 2.8587e-02,
-3.3038e-02, 5.7647e-02, 1.8222e-01, 8.7216e-02, 4.3150e-02],
[ 4.4928e-02, 6.0862e-02, 9.7200e-02, -8.2783e-03, 1.7079e-02,
-5.4661e-02, 5.6732e-02, 1.8344e-01, 1.0233e-01, 3.3588e-02],
[ 4.3563e-02, 7.4965e-02, 9.8287e-02, -5.7807e-03, 1.7122e-02,
-3.2483e-02, 3.4094e-02, 1.8516e-01, 1.0466e-01, 3.4094e-02],
[ 5.2795e-02, 6.9953e-02, 9.0784e-02, -2.3940e-02, 1.8081e-02,
-4.9283e-02, 5.2217e-02, 1.8904e-01, 8.3833e-02, 3.6422e-02],
[ 4.9884e-02, 7.5366e-02, 1.0621e-01, -2.6932e-03, 2.6925e-02,
-3.7889e-02, 1.9987e-02, 1.9975e-01, 1.1211e-01, 2.9898e-02],
[ 4.5040e-02, 8.0663e-02, 1.0628e-01, -6.3092e-03, 1.1769e-02,
-3.2534e-02, 2.8865e-02, 1.9922e-01, 1.0275e-01, 1.8025e-02],
[ 3.5443e-02, 6.8844e-02, 1.1379e-01, -3.0028e-02, 1.3558e-02,
-3.2873e-02, 5.6545e-03, 2.1678e-01, 1.3584e-01, 1.4099e-02],
[ 5.2294e-02, 7.9077e-02, 1.2023e-01, -8.2734e-03, 2.7933e-02,
-3.8756e-02, 1.0541e-02, 2.1420e-01, 1.2128e-01, 2.0939e-02],
[ 4.1155e-02, 7.4399e-02, 9.7676e-02, -3.9840e-03, 2.5349e-02,
-3.2788e-02, 3.0812e-02, 2.0104e-01, 1.0533e-01, 2.5656e-02],
[ 4.2729e-02, 7.8216e-02, 9.9952e-02, -6.2153e-03, 9.1319e-03,
-4.0137e-02, 4.1663e-02, 1.8600e-01, 1.0623e-01, 2.9714e-02],
[ 5.3146e-02, 7.7104e-02, 8.2418e-02, -2.8274e-02, 4.1431e-03,
-2.9398e-02, 3.6638e-02, 2.0036e-01, 1.2231e-01, -3.4463e-03],
[ 4.1126e-02, 7.1232e-02, 1.0066e-01, 1.6677e-03, 3.5951e-02,
-3.5747e-02, 3.9963e-02, 2.0618e-01, 9.3462e-02, 2.9012e-02],
[ 2.9576e-02, 7.4254e-02, 1.3402e-01, 5.9384e-03, 4.5582e-02,
-3.3619e-02, -2.1406e-04, 2.3509e-01, 1.2578e-01, 1.3809e-02],
[ 4.4625e-02, 6.1910e-02, 8.6346e-02, -1.1014e-02, 4.6476e-03,
-4.3238e-02, 6.6938e-02, 1.7904e-01, 1.0984e-01, 2.6792e-02],
[ 3.9996e-02, 8.6279e-02, 1.0779e-01, -2.3226e-03, 1.1552e-02,
-3.6112e-02, 1.4071e-02, 2.1219e-01, 1.0677e-01, 1.3734e-02],
[ 5.5728e-02, 6.6403e-02, 1.0220e-01, -1.5560e-02, 3.1718e-02,
-4.3329e-02, 4.7500e-02, 2.0520e-01, 9.7674e-02, 3.6237e-02],
[ 4.5364e-02, 6.8187e-02, 9.3100e-02, 2.2515e-03, 3.5801e-02,
-3.4786e-02, 4.6033e-02, 1.9583e-01, 9.7681e-02, 3.1045e-02],
[ 5.3247e-02, 6.5139e-02, 9.1759e-02, -6.5726e-03, 3.2744e-02,
-5.1284e-02, 4.6367e-02, 1.9006e-01, 9.4760e-02, 3.5563e-02],
[ 5.2639e-02, 8.2943e-02, 1.0273e-01, -1.5164e-02, 1.2060e-02,
-3.5409e-02, 3.9387e-02, 1.9337e-01, 1.0079e-01, 3.5458e-02],
[ 4.6999e-02, 7.1007e-02, 1.1809e-01, -6.7775e-03, 2.2589e-02,
-2.9001e-02, 3.4007e-02, 2.0353e-01, 1.1682e-01, 3.7655e-02],
[ 3.3926e-02, 6.5998e-02, 8.1836e-02, 1.9663e-02, 1.3361e-02,
-4.5032e-02, 7.1060e-02, 1.6552e-01, 7.9978e-02, 4.7255e-02],
[ 4.7067e-02, 6.9157e-02, 8.2214e-02, 7.5241e-03, 2.5655e-02,
-4.1797e-02, 5.5234e-02, 1.7456e-01, 8.6544e-02, 3.9139e-02],
[ 4.8377e-02, 7.8865e-02, 1.0172e-01, -1.4404e-02, 1.9714e-02,
-3.8841e-02, 3.3828e-02, 2.0017e-01, 1.1691e-01, 1.9860e-02],
[ 4.0690e-02, 6.4806e-02, 1.2260e-01, -6.1800e-03, 3.1907e-02,
-4.2603e-02, 1.8128e-03, 2.2517e-01, 1.3336e-01, 2.0534e-02],
[ 5.3634e-02, 7.7372e-02, 1.0836e-01, -2.2430e-02, 1.8259e-02,
-3.4539e-02, 1.0394e-02, 2.2268e-01, 1.2000e-01, 8.4756e-03],
[ 4.6425e-02, 7.1814e-02, 7.2300e-02, 3.2654e-03, 1.3225e-02,
-2.5276e-02, 6.4737e-02, 1.6135e-01, 9.2523e-02, 3.7056e-02],
[ 4.4556e-02, 5.6595e-02, 7.4036e-02, -1.4091e-02, 3.9290e-02,
-2.8681e-02, 7.7379e-02, 1.8513e-01, 8.7132e-02, 2.9954e-02],
[ 6.1430e-02, 5.7709e-02, 8.6445e-02, -2.8731e-03, 2.7795e-02,
-5.3314e-02, 7.0701e-02, 1.6406e-01, 7.5343e-02, 5.6913e-02],
[ 4.3709e-02, 7.7684e-02, 7.9247e-02, 4.8706e-03, 1.8451e-02,
-3.6902e-02, 6.4597e-02, 1.6295e-01, 8.1663e-02, 4.3154e-02],
[ 4.2408e-02, 7.1148e-02, 9.5582e-02, -3.1638e-03, 1.5995e-02,
-3.0401e-02, 3.7001e-02, 1.8053e-01, 1.0812e-01, 3.4665e-02],
[ 5.6712e-02, 5.5502e-02, 9.1106e-02, -4.5377e-03, 4.4237e-02,
-3.5257e-02, 6.8138e-02, 1.8443e-01, 9.3305e-02, 3.4813e-02],
[ 4.2568e-02, 5.9715e-02, 1.0365e-01, 2.4595e-03, 3.4655e-02,
-3.3955e-02, 4.3675e-02, 1.9512e-01, 9.6210e-02, 3.5900e-02],
[ 4.1211e-02, 7.8950e-02, 1.1866e-01, -1.8702e-02, 1.6075e-02,
-1.9562e-02, 3.8823e-02, 1.8941e-01, 1.2959e-01, 2.2440e-02],
[ 4.6865e-02, 7.4048e-02, 8.7111e-02, -1.6634e-02, 2.2270e-02,
-4.3426e-02, 3.8470e-02, 1.9670e-01, 8.2416e-02, 2.7993e-02],
[ 4.6475e-02, 7.1963e-02, 1.1071e-01, -2.2251e-02, 2.1632e-02,
-3.3473e-02, 4.0497e-02, 2.0818e-01, 1.0301e-01, 1.5888e-02],
[ 4.4548e-02, 6.5409e-02, 9.2321e-02, 7.5803e-04, 1.6367e-02,
-3.4900e-02, 6.4615e-02, 1.6492e-01, 8.4607e-02, 5.8241e-02],
[ 4.1620e-02, 6.7470e-02, 9.2814e-02, 4.6260e-03, 2.6289e-02,
-3.9896e-02, 5.1027e-02, 1.8289e-01, 9.5061e-02, 3.5992e-02],
[ 4.0511e-02, 6.1022e-02, 1.0108e-01, -2.1134e-02, 1.7980e-02,
-4.1835e-02, 3.8339e-02, 1.9648e-01, 1.2243e-01, 1.8094e-02],
[ 3.9376e-02, 6.3891e-02, 9.6884e-02, -3.5065e-03, 1.6051e-02,
-4.1612e-02, 4.3669e-02, 1.8123e-01, 1.0936e-01, 3.2718e-02],
[ 4.5057e-02, 8.6487e-02, 1.0585e-01, -1.4185e-02, 9.3067e-03,
-3.8538e-02, 3.5926e-02, 2.0631e-01, 1.1622e-01, 1.3628e-02],
[ 4.8965e-02, 7.0799e-02, 7.0412e-02, -3.1760e-03, 1.8455e-02,
-3.3409e-02, 7.3456e-02, 1.5382e-01, 7.6266e-02, 4.7298e-02],
[ 3.9909e-02, 6.3246e-02, 9.8658e-02, -1.4415e-03, 2.7626e-02,
-3.9524e-02, 3.3380e-02, 1.8458e-01, 1.0181e-01, 4.1649e-02],
[ 3.7444e-02, 6.8192e-02, 8.0125e-02, 2.7156e-03, 1.6934e-02,
-4.6260e-02, 6.2959e-02, 1.6454e-01, 8.4767e-02, 4.5780e-02],
[ 6.6929e-02, 5.0171e-02, 9.7169e-02, 5.9488e-04, 7.2087e-03,
-1.4845e-02, 8.4930e-02, 1.5208e-01, 1.2398e-01, 3.1786e-02],
[ 4.6997e-02, 6.0560e-02, 1.1440e-01, -1.4719e-02, 4.3504e-02,
-2.9432e-02, 2.7738e-02, 2.1371e-01, 9.9563e-02, 2.1910e-02],
[ 3.1766e-02, 6.9583e-02, 9.1724e-02, -1.3318e-02, 3.9961e-02,
-3.7815e-02, 4.8845e-02, 1.9668e-01, 1.2078e-01, 7.4767e-03],
[ 5.0185e-02, 6.9386e-02, 9.6159e-02, -2.8642e-03, 1.9889e-02,
-3.8453e-02, 4.7161e-02, 1.7794e-01, 9.3146e-02, 4.5568e-02]],
grad_fn=)
-------------------------targets-----------------------
tensor([4, 7, 7, 2, 6, 5, 6, 9, 7, 7, 5, 7, 0, 8, 6, 7, 8, 3, 8, 9, 3, 2, 9, 2,
7, 4, 1, 9, 7, 0, 8, 1, 2, 0, 6, 5, 1, 4, 2, 8, 2, 8, 3, 3, 1, 4, 9, 8,
1, 1, 7, 9, 9, 4, 6, 8, 5, 3, 2, 2, 3, 8, 5, 1])
-------------------------result_loss--------------------
tensor(2.2917, grad_fn=)
6.5 如何根据据loss function 的输出调优
loss function 为卷积核的参数设置了grand(梯度)，根据梯度对参数进行优化。
例如梯度下降。
7. 优化器—torch.optim
https://pytorch.org/docs/stable/optim.html
import torch
import torchvision
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
dataset=torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
dataLoader=DataLoader(dataset=dataset,batch_size=64,)
class SeqTest(nn.Module):
def __init__(self):
super(SeqTest, self).__init__()
self.conv1=Conv2d(in_channels=3,
out_channels=32,
kernel_size=5,
padding=2)
self.maxpool1=MaxPool2d(kernel_size=2)
self.conv2=Conv2d(in_channels=32,
out_channels=32,
kernel_size=5,
padding=2)
self.maxpool2=MaxPool2d(kernel_size=2)
self.conv3=Conv2d(in_channels=32,
out_channels=64,
kernel_size=5,
padding=2)
self.maxpool3=MaxPool2d(kernel_size=2)
self.flatten=Flatten()
self.linear1=Linear(in_features=1024,
out_features=64)
self.linear2=Linear(in_features=64,
out_features=10)
# self.model1=Sequential(
# Conv2d(in_channels=3,out_channels=32,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Conv2d(in_channels=32,out_channels=32,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Conv2d(in_channels=32,out_channels=64,kernel_size=5,padding=2),
# MaxPool2d(kernel_size=2),
# Flatten(),
# Linear(in_features=1024,out_features=64),
# Linear(in_features=64,out_features=10)
# )
def forward(self,x):
x=self.conv1(x)
x=self.maxpool1(x)
x = self.conv2(x)
x = self.maxpool2(x)
x = self.conv3(x)
x = self.maxpool3(x)
x=self.flatten(x)
x=self.linear1(x)
x=self.linear2(x)
# x=self.model1(x)
return x
loss_cross=nn.CrossEntropyLoss()
test=SeqTest()
# 设置优化器
optim=torch.optim.SGD(test.parameters(),
lr=0.01)#lr=learning rate 学习速率
for epoch in range(20):
running_loss=0.0
#对数据进行一轮学习不不能有效减少result_loss ---误差，因此进行多轮学习
for data in dataLoader:
imgs,targets=data
#模型预测
outputs=test(imgs)
# print("-------------------------outputs-----------------------")
# print(outputs)
# print("-------------------------targets-----------------------")
# print(targets)
result_loss=loss_cross(outputs,targets)
# print("-------------------------result_loss--------------------")
# print(result_loss)
#初始化梯度
optim.zero_grad()
# 反向传播计算梯度
result_loss.backward()
optim.step()
# #输出一轮学习每次的误差--loss
# print(result_loss)
#一轮误差之和
running_loss=running_loss+result_loss
print(running_loss)
输出每轮的误差：
前5次
tensor(360.6162, grad_fn=)
tensor(356.1118, grad_fn=)
tensor(336.8101, grad_fn=)
tensor(319.0069, grad_fn=)
tensor(310.9478, grad_fn=)
8. 网络模型的加载和修改
代码：
import torchvision
import torch.nn
# ImageNet_ddataset=torchvision.datasets.ImageNet("./ImageNet_dataset",
# split="train",
# transform=torchvision.transforms.ToTensor(),
# )
#pretrained=false 加载模型，不下载参数
from torch import nn
vgg16_false=torchvision.models.vgg16(pretrained=False,progress=True)
#pretrained=true 加载模型，并且下载训练好的参数
vgg16_true=torchvision.models.vgg16(pretrained=True)
# print(vgg16_true)
# print(vgg16_false)
train_data=torchvision.datasets.CIFAR10("./dataset",train=True,transform=torchvision.transforms.ToTensor(),download=True)
#vgg模型默认最后的线性层输出的是1000，加一层修改输出是10（10分类）
vgg16_true.classifier.add_module("add_liner",nn.Linear(1000,10))
print(vgg16_true)
#修改vgg16 模型
print(vgg16_false)
vgg16_false.classifier[6]=nn.Linear(4096,10)
print(vgg16_false)
数据集：ImageNet
查看数据集所需要的包：conda list scipy
# packages in environment at E:\Program Files\anaconda\Anaconda3\envs\pytorch:
#
# Name Version Build Channel
scipy 1.5.4 pypi_0 pypi
vgg模型地址
C:\Users\Administrator.cache\torch\hub\checkpoints
9.模型的保存和加载
import torch
import torchvision
#加载没有进行预训练的模型
vgg16=torchvision.models.vgg16(pretrained=False)
#保存方式 1---（保存了网络模型的结构和参数）
torch.save(vgg16,"vgg16_method1.pth")
#读取方式 1保存的模型
model1=torch.load("./vgg16_method1.pth")
print(model1)
#保存方式 2(官方推荐)---保存网络模型参数（字典格式）
torch.save(vgg16.state_dict(),"vgg16_method2.pth")
#加载方式二保存的模型
#1)加载参数字典
model2_dict=torch.load("vgg16_method2.pth")
#2)新建网络模型型结构
vgg16=torchvision.models.vgg16(pretrained=False)
#3)模型读取字典存储的模型参数
model2=vgg16.load_state_dict(model2_dict)
print("-------------------------------------")
print(model2)
print(model2)
10. 完整的模型训练
模型代码：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p25_complete_train_module.py
# @Time :2021/10/28 23:59
# @Author :Administrator
# 3 搭建神经网络 CIFAR10有十个类别 -搭建10分类神经网络
import torch
from torch import nn
class CompleteTrain(nn.Module):
def __init__(self):
super(CompleteTrain, self).__init__()
self.model=nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=32,kernel_size=5,stride=1,padding=2),
nn.MaxPool2d(2),
nn.Conv2d(32,32,5,1,2),
nn.MaxPool2d(2),
nn.Conv2d(32,64,5,1,2),
nn.MaxPool2d(2),
nn.Flatten(),
nn.Linear(64*4*4,64),
nn.Linear(64,10),
)
def forward(self,x):
x=self.model(x)
return x
#测试网络
if __name__ == '__main__':
test=CompleteTrain()
input=torch.ones((64,3,32,32))
output=test(input)
print(output.shape)
print(output)
步骤代码：
import torch
import torchvision
# 1 准备数据集
from torch import nn
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from p25_complete_train_module import *
train_data=torchvision.datasets.CIFAR10(root="./dataset",train=True,transform=torchvision.transforms.ToTensor(),download=True)
test_data=torchvision.datasets.CIFAR10(root="./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
#数据集大小
train_data_size=len(train_data)
test_data_size=len(test_data)
print("训练数据集的长度{}".format(train_data_size))
print("测试数据集的长度{}".format(test_data_size))
# 2 加载数据集 Dataloader
train_dataloader=DataLoader(train_data,batch_size=64)
test_dataloader=DataLoader(test_data,batch_size=64)
# 4.创建网络模型
test=CompleteTrain()
# 5.创建损失函数
loss_func=nn.CrossEntropyLoss()
# 6.优化器
# learn_rate=0.01
learn_rate=1e-2;
optimizer=torch.optim.SGD(test.parameters(),lr=learn_rate)
# 7. 设置训练网络的参数
#训练的次数
total_train_step=0
#测试的次数
total_test_step=0
#训练的轮数
epoch=3
# 9.添加tensorboard
writer=SummaryWriter("./completeTrain_logs")
#训练开始
test.train()#只对特定的层起作用 pytorch文档有说明
for i in range(epoch):
print("------------------------第{}轮开始---------------------".format(i+1))
for data in train_dataloader:
imgs,target=data
outputs=test(imgs)
#计算损失
loss=loss_func(outputs,target)
#梯度清空
optimizer.zero_grad()
#反向传播,计算梯度
loss.backward()
#优化
optimizer.step()
#训练次数+1
total_train_step=total_train_step+1
if total_train_step %100==0:
print("训练次数：{}，loss:{}".format(total_train_step,loss.item()))
#9-1
writer.add_scalar("train_loss",loss.item(),total_train_step)
# 8. 在测试数据集上跑一边，进行检验
total_test_loss=0
# 10. 整体正确率
total_accuracy=0
test.eval()#只对特定的层起作用 pytorch文档有说明
with torch.no_grad():
for data in test_dataloader:
imgs,target=data
# print("====")
# print(target)
outputs=test(imgs)
loss=loss_func(outputs,target)
total_test_loss=total_test_loss+loss.item()
#10-1 求每一组最大一项的预测分数
accuracy=(outputs.argmax(1)==target).sum()
total_accuracy=total_accuracy+accuracy
print("第{}次整体测试集测试上的loss:{}".format(i+1,total_test_loss))
# 9-2
writer.add_scalar("test_loss", total_test_loss, total_test_step)
total_test_step = total_test_step + 1
#10-3
print("第{}次整体测试集上的正确率：{}".format(i+1,total_accuracy/test_data_size))
writer.add_scalar("test_accuracy",total_accuracy/test_data_size,total_test_step)
torch.save(test,"./save_module/test_module_{}.pth".format(i+1))
#官方推荐 torch.save(test.state_dict(),"./save_module/test_module_{}.pth".format(i+1))
print("模型已保存！")
writer.close()
10.1正确率
11.使用gpu训练
1. 方式一
模型代码：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p25_complete_train_module.py
# @Time :2021/10/28 23:59
# @Author :Administrator
# 3 搭建神经网络 CIFAR10有十个类别 -搭建10分类神经网络
import torch
from torch import nn
class CompleteTrain(nn.Module):
def __init__(self):
super(CompleteTrain, self).__init__()
self.model=nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=32,kernel_size=5,stride=1,padding=2),
nn.MaxPool2d(2),
nn.Conv2d(32,32,5,1,2),
nn.MaxPool2d(2),
nn.Conv2d(32,64,5,1,2),
nn.MaxPool2d(2),
nn.Flatten(),
nn.Linear(64*4*4,64),
nn.Linear(64,10),
)
def forward(self,x):
x=self.model(x)
return x
#测试网络
if __name__ == '__main__':
test=CompleteTrain()
input=torch.ones((64,3,32,32))
output=test(input)
print(output.shape)
print(output)
流程代码：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p25_complete_train_gpu.py
# @Time :2021/10/30 21:58
# @Author :Administrator
import torch
import torchvision
# 1 准备数据集
from torch import nn
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from p25_complete_train_module import *
train_data=torchvision.datasets.CIFAR10(root="./dataset",train=True,transform=torchvision.transforms.ToTensor(),download=True)
test_data=torchvision.datasets.CIFAR10(root="./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
#数据集大小
train_data_size=len(train_data)
test_data_size=len(test_data)
print("训练数据集的长度{}".format(train_data_size))
print("测试数据集的长度{}".format(test_data_size))
# 2 加载数据集 Dataloader
train_dataloader=DataLoader(train_data,batch_size=64)
test_dataloader=DataLoader(test_data,batch_size=64)
# 4.创建网络模型
test=CompleteTrain()
# 11. 使用GPU训练 共4步
# 11-1网络模型转移到cuda
if torch.cuda.is_available():
test=test.cuda()
# 5.创建损失函数
loss_func=nn.CrossEntropyLoss()
#11-2
if torch.cuda.is_available():
loss_func=loss_func.cuda()
# 6.优化器
# learn_rate=0.01
learn_rate=1e-2;
optimizer=torch.optim.SGD(test.parameters(),lr=learn_rate)
# 7. 设置训练网络的参数
#训练的次数
total_train_step=0
#测试的次数
total_test_step=0
#训练的轮数
epoch=3
# 9.添加tensorboard
writer=SummaryWriter("./completeTrain_logs")
#训练开始
test.train()#只对特定的层起作用 pytorch文档有说明
for i in range(epoch):
print("------------------------第{}轮开始---------------------".format(i+1))
for data in train_dataloader:
imgs,target=data
# 11-3
if torch.cuda.is_available():
imgs=imgs.cuda()
target=target.cuda()
outputs=test(imgs)
#计算损失
loss=loss_func(outputs,target)
#梯度清空
optimizer.zero_grad()
#反向传播,计算梯度
loss.backward()
#优化
optimizer.step()
#训练次数+1
total_train_step=total_train_step+1
if total_train_step %100==0:
print("训练次数：{}，loss:{}".format(total_train_step,loss.item()))
#9-1
writer.add_scalar("train_loss",loss.item(),total_train_step)
# 8. 在测试数据集上跑一边，进行检验
total_test_loss=0
# 10. 整体正确率
total_accuracy=0
test.eval()#只对特定的层起作用 pytorch文档有说明
with torch.no_grad():
for data in test_dataloader:
imgs,target=data
#11-4
if torch.cuda.is_available():
imgs = imgs.cuda()
target = target.cuda()
# print("====")
# print(target)
outputs=test(imgs)
loss=loss_func(outputs,target)
total_test_loss=total_test_loss+loss.item()
#10-1 求每一组最大一项的预测分数
accuracy=(outputs.argmax(1)==target).sum()
total_accuracy=total_accuracy+accuracy
print("第{}次整体测试集测试上的loss:{}".format(i+1,total_test_loss))
# 9-2
writer.add_scalar("test_loss", total_test_loss, total_test_step)
total_test_step = total_test_step + 1
#10-3
print("第{}次整体测试集上的正确率：{}".format(i+1,total_accuracy/test_data_size))
writer.add_scalar("test_accuracy",total_accuracy/test_data_size,total_test_step)
torch.save(test,"./save_module/test_module_{}.pth".format(i+1))
#官方推荐 torch.save(test.state_dict(),"./save_module/test_module_{}.pth".format(i+1))
print("模型已保存！")
writer.close()
2. 方式二
#!/usr/bin/env python
# -*- coding:utf-8 -*-
# @FileName :p25_complete_train_gpu.py
# @Time :2021/10/30 21:58
# @Author :Administrator
import torch
import torchvision
# 1 准备数据集
from torch import nn
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
#
# from p25_complete_train_module import *
# 12.定义训练的设备
# device=torch.device("cpu")
#device=torch.device("cuda")
device=torch.device("cuda" if torch.cuda.is_available() else "cpu")
train_data=torchvision.datasets.CIFAR10(root="./dataset",train=True,transform=torchvision.transforms.ToTensor(),download=True)
test_data=torchvision.datasets.CIFAR10(root="./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)
#数据集大小
train_data_size=len(train_data)
test_data_size=len(test_data)
print("训练数据集的长度{}".format(train_data_size))
print("测试数据集的长度{}".format(test_data_size))
# 2 加载数据集 Dataloader
train_dataloader=DataLoader(train_data,batch_size=64)
test_dataloader=DataLoader(test_data,batch_size=64)
# 4.创建网络模型
class CompleteTrain(nn.Module):
def __init__(self):
super(CompleteTrain, self).__init__()
self.model=nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=32,kernel_size=5,stride=1,padding=2),
nn.MaxPool2d(2),
nn.Conv2d(32,32,5,1,2),
nn.MaxPool2d(2),
nn.Conv2d(32,64,5,1,2),
nn.MaxPool2d(2),
nn.Flatten(),
nn.Linear(64*4*4,64),
nn.Linear(64,10),
)
def forward(self,x):
x=self.model(x)
return x
test=CompleteTrain()
# 11. 使用GPU训练 共4步
# 11-1网络模型转移到cuda
# if torch.cuda.is_available():
# test=test.cuda()
# 12-1 gpu使用方式二
test=test.to(device)
# #同12-1 不用赋值
# test.to(device)
# 5.创建损失函数
loss_func=nn.CrossEntropyLoss()
#11-2
# if torch.cuda.is_available():
# loss_func=loss_func.cuda()
#12-2 gpu使用方式二
loss_func=loss_func.to(device)
# #同12-2 不用赋值
# loss_func.to(device)
# 6.优化器
# learn_rate=0.01
learn_rate=1e-2;
optimizer=torch.optim.SGD(test.parameters(),lr=learn_rate)
# 7. 设置训练网络的参数
#训练的次数
total_train_step=0
#测试的次数
total_test_step=0
#训练的轮数
epoch=3
# 9.添加tensorboard
writer=SummaryWriter("./completeTrain_logs")
#训练开始
test.train()#只对特定的层起作用 pytorch文档有说明
for i in range(epoch):
print("------------------------第{}轮开始---------------------".format(i+1))
for data in train_dataloader:
imgs,target=data
# 11-3
# if torch.cuda.is_available():
# imgs=imgs.cuda()
# target=target.cuda()
#12-3
imgs=imgs.to(device)
target=target.to(device)
outputs=test(imgs)
#计算损失
loss=loss_func(outputs,target)
#梯度清空
optimizer.zero_grad()
#反向传播,计算梯度
loss.backward()
#优化
optimizer.step()
#训练次数+1
total_train_step=total_train_step+1
if total_train_step %100==0:
print("训练次数：{}，loss:{}".format(total_train_step,loss.item()))
#9-1
writer.add_scalar("train_loss",loss.item(),total_train_step)
# 8. 在测试数据集上跑一边，进行检验
total_test_loss=0
# 10. 整体正确率
total_accuracy=0
test.eval()#只对特定的层起作用 pytorch文档有说明
with torch.no_grad():
for data in test_dataloader:
imgs,target=data
#11-4
# if torch.cuda.is_available():
# imgs = imgs.cuda()
# target = target.cuda()
#12-4
imgs = imgs.to(device)
target = target.to(device)
# print("====")
# print(target)
outputs=test(imgs)
loss=loss_func(outputs,target)
total_test_loss=total_test_loss+loss.item()
#10-1 求每一组最大一项的预测分数
accuracy=(outputs.argmax(1)==target).sum()
total_accuracy=total_accuracy+accuracy
print("第{}次整体测试集测试上的loss:{}".format(i+1,total_test_loss))
# 9-2
writer.add_scalar("test_loss", total_test_loss, total_test_step)
total_test_step = total_test_step + 1
#10-3
print("第{}次整体测试集上的正确率：{}".format(i+1,total_accuracy/test_data_size))
writer.add_scalar("test_accuracy",total_accuracy/test_data_size,total_test_step)
torch.save(test,"./save_module/test_module_{}.pth".format(i+1))
#官方推荐 torch.save(test.state_dict(),"./save_module/test_module_{}.pth".format(i+1))
print("模型已保存！")
writer.close()
使用 google colab 免费每周30小时
!nvidia-smi
