pytorch官方教程学习笔记06：WHAT IS TORCH.NN REALLY?dataset，Add validation等新的东西，

1.下载数据集：

from pathlib import Path
import requests
# ======================================= 数据集包的下载： ======================================= #
DATA_PATH = Path("data")
PATH = DATA_PATH / "mnist"
PATH.mkdir(parents=True, exist_ok=True)
URL = "http://deeplearning.net/data/mnist/"
FILENAME = "mnist.pkl.gz"
if not (PATH / FILENAME).exists():
        content = requests.get(URL + FILENAME).content
        (PATH / FILENAME).open("wb").write(content)

①：Path 包的使用：

直接就可以用 / 将路径进行拼接：

from pathlib import Path
import requests

DATA_PATH = Path("data")
PATH = DATA_PATH / "mnist"
print(PATH)

结果：

②：使用requests下载指定网页的东西：

content = requests.get(URL + FILENAME).content
(PATH / FILENAME).open("wb").write(content)

2.提取训练集和测试集：

import pickle
import gzip
with gzip.open((PATH / FILENAME).as_posix(), "rb") as f:
        ((x_train, y_train), (x_valid, y_valid), _) = pickle.load(f, encoding="latin-1")

3.数据集的大小：

训练集的大小(50000, 784)，即50000张，像素为28*28=784的图片。
imshow用来显示图片。

from matplotlib import pyplot
import numpy as np

pyplot.imshow(x_train[0].reshape((28, 28)), cmap="gray")
print(x_train.shape)

4.convert our data to torch.tensor：

import torch
x_train, y_train, x_valid, y_valid = map(
    torch.tensor, (x_train, y_train, x_valid, y_valid)
)
n, c = x_train.shape
x_train, x_train.shape, y_train.min(), y_train.max()
print(x_train, y_train)
print(x_train.shape)
print(y_train.min(), y_train.max())

①.用几个单个的变量提取出张量中的内容：

a1, a2, a3 = torch.tensor([4, 5, 6])
print(a1, a2, a3)

结果：

因此原代码中，n和c分别把张量的两个维度提取出来了。

②：用map tuple批量将变量的类型转变为pytorch.tensor：

a1 = [4, 5, 6]
a2 = [1, 2, 3]
a3, a4 = map(
    torch.tensor, (a1, a2)
)
print(a3, a4)

5.依然是一个详细的一步步引入pytorch求梯度，反向传播的教程，这一部分先跳过了。

6.Dataset系列：

①：Dataset

原来求每一次喂入模型的数据：

xb = x_train[start_i:end_i]
yb = y_train[start_i:end_i]

使用dataset将数据封装在了一起：

train_ds = TensorDataset(x_train, y_train)
xb,yb = train_ds[i*bs : i*bs+bs]

总体代码：

model, opt = get_model()

for epoch in range(epochs):
    for i in range((n - 1) // bs + 1):
        xb, yb = train_ds[i * bs: i * bs + bs]
        pred = model(xb)
        loss = loss_func(pred, yb)

        loss.backward()
        opt.step()
        opt.zero_grad()

print(loss_func(model(xb), yb))

②：DataLoader 在Dataset的基础上封装，实现了指定每次的batch：

from torch.utils.data import DataLoader

train_ds = TensorDataset(x_train, y_train)
train_dl = DataLoader(train_ds, batch_size=bs)

训练代码中直接用for in 结构自动提取指定批次大小的数据了：

model, opt = get_model()

for epoch in range(epochs):
    for xb, yb in train_dl:
        pred = model(xb)
        loss = loss_func(pred, yb)

        loss.backward()
        opt.step()
        opt.zero_grad()

print(loss_func(model(xb), yb))

7.Add validation:

使用DataLoader封裝训练集和测试集：

train_ds = TensorDataset(x_train, y_train)
train_dl = DataLoader(train_ds, batch_size=bs, shuffle=True)

valid_ds = TensorDataset(x_valid, y_valid)**加粗样式**
valid_dl = DataLoader(valid_ds, batch_size=bs * 2)

---

①：model.train() 和 model.eval()：

Note that we always call model.train() before training, and model.eval() before inference, because these are used by layers such as nn.BatchNorm2d and nn.Dropout to ensure appropriate behaviour for these different phases.

②：使用验证集，也就是在模型训练完成后，测试一下模型现在怎么样了？

model, opt = get_model()

for epoch in range(epochs):
    model.train()
    for xb, yb in train_dl:
        pred = model(xb)
        loss = loss_func(pred, yb)

        loss.backward()
        opt.step()
        opt.zero_grad()

    model.eval()
    with torch.no_grad():
        valid_loss = sum(loss_func(model(xb), yb) for xb, yb in valid_dl)

    print(epoch, valid_loss / len(valid_dl))

③：我们发现上述使用验证集，只是进行了loss的计算，而没有进行反向传播，更新参数等操作，这是正确的，否则肯定是一个对验证集过拟合的模型。

④：封装一个函数，

We pass an optimizer in for the training set, and use it to perform backprop. For the validation set, we don’t pass an optimizer, so the method doesn’t perform backprop.

def loss_batch(model, loss_func, xb, yb, opt=None):
    loss = loss_func(model(xb), yb)

    if opt is not None:
        loss.backward()
        opt.step()
        opt.zero_grad()

    return loss.item(), len(xb)

⑤：使用了上述函数，因为验证集只需要进行计算的是loss，不需要反向传播等计算，就给了他一个 with torch.no_grad(): 来告诉不需要多余的构建计算图了。

import numpy as np

def fit(epochs, model, loss_func, opt, train_dl, valid_dl):
    for epoch in range(epochs):
        model.train()
        for xb, yb in train_dl:
            loss_batch(model, loss_func, xb, yb, opt)

        model.eval()
        with torch.no_grad():
            losses, nums = zip(
                *[loss_batch(model, loss_func, xb, yb) for xb, yb in valid_dl]
            )
        val_loss = np.sum(np.multiply(losses, nums)) / np.sum(nums)

        print(epoch, val_loss)

⑥：继续将整个训练+使用验证集验证的过程封装到一个fit中，

import numpy as np

def fit(epochs, model, loss_func, opt, train_dl, valid_dl):
    for epoch in range(epochs):
        model.train()
        for xb, yb in train_dl:
            loss_batch(model, loss_func, xb, yb, opt)

        model.eval()
        with torch.no_grad():
            losses, nums = zip(
                *[loss_batch(model, loss_func, xb, yb) for xb, yb in valid_dl]
            )
        val_loss = np.sum(np.multiply(losses, nums)) / np.sum(nums)

        print(epoch, val_loss)

8.直接可以使用Sequential来定义网络，这样就不需要定义一个class了：

①：使用class定义：

class Mnist_CNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1)
        self.conv2 = nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1)
        self.conv3 = nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1)

    def forward(self, xb):
        xb = xb.view(-1, 1, 28, 28)
        xb = F.relu(self.conv1(xb))
        xb = F.relu(self.conv2(xb))
        xb = F.relu(self.conv3(xb))
        xb = F.avg_pool2d(xb, 4)
        return xb.view(-1, xb.size(1))

lr = 0.1

②：使用Sequential来定义：

model = nn.Sequential(
    Lambda(preprocess),
    nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.AvgPool2d(4),
    Lambda(lambda x: x.view(x.size(0), -1)),
)

opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9)

fit(epochs, model, loss_func, opt, train_dl, valid_dl)