【PyTorch】前馈神经网络实现/feedforward_neural_network

1. PyTorch的特点是动态计算图，构建好网络模型，可以实现自动微分，反向传播和参数更新代码如下：

 # Backward and optimize
 optimizer.zero_grad() # clear the grad
 loss.backward() # loss backward and update grad
 optimizer.step() # updata all parameters

2. 构建前馈神经网络实现手写数字MNIST分类，具体代码如下：

import torch  
import torch.nn as nn
import torchvision
import torchvision.transforms as transforms

# Device configuration 
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu') # select cuda or cpu

# Hyper-parameters 
input_size = 784
hidden_size = 500
num_classes = 10
num_epochs = 5
batch_size = 100
learning_rate = 0.001

# MNIST dataset
train_dataset = torchvision.datasets.MNIST(
    root='data', # url of the data
    train=True,
    transform = transforms.ToTensor(), 
    download=True
)
test_dataset = torchvision.datasets.MNIST(
    root='data',
    train=False, # is not train data
    transform=transforms.ToTensor() # work on imput and change the dimension
)

# Data loader
train_loader = torch.utils.data.DataLoader(
    dataset=train_dataset,
    batch_size=batch_size,
    shuffle=True # shuffle the data to be random
)

test_loader = torch.utils.data.DataLoader(
    dataset=test_dataset,
    batch_size=batch_size,
    shuffle=False
)

# Fully connected neural network with one hidden layer
class NeuralNet(nn.Module):
    def __init__(self, input_size, hidden_size, num_classes):
        super(NeuralNet, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size) # from input layer to hidden layer
        self.relu = nn.ReLU() # relu function
        self.fc2 = nn.Linear(hidden_size, num_classes) # from hidden layer to output layer

    def forward(self, x):# connect the layers to net
        out = self.fc1(x) 
        out = self.relu(out)
        out = self.fc2(out)
        return out

model = NeuralNet(input_size, hidden_size, num_classes).to(device) # three layers of net

# Loss and optimizer
criterion = nn.CrossEntropyLoss() # loss criterion
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) # default optimizer

# Train the model
total_step = len(train_loader) # num of train data
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        # Move tensors to the configured device
        images = images.reshape(-1, 28*28).to(device) # reshaoe the train data to one line
        labels = labels.to(device)

        # Forward pass
        outputs = model(images) # put into model
        loss = criterion(outputs, labels) # calculate loss of each step

        # Backward and optimize
        optimizer.zero_grad() # clear the grad
        loss.backward() # loss backward and update grad
        optimizer.step() # updata all parameters

        if (i+1) % 100 == 0:
            print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, i+1, total_step, loss.item())) # print for each step

# Test the model
# In test phase, we don't need to compute gradients(for memory efficiency)
with torch.no_grad():
    correct = 0
    total = 0
    for images, labels in test_loader:
        images = images.reshape(-1, 28*28).to(device)
        labels = labels.to(device)
        outputs = model(images)
        _,predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()
    
    print('Accuracy of the network on the 10000 test images: {} %'.format(100 * correct / total))

# Save the model checkpoint for later load
torch.save(model.state_dict(), 'model.ckpt')

3. 程序输出如下，测试时间较快，精度也比较高。

Epoch [1/5], Step [100/600], Loss: 0.3190
Epoch [1/5], Step [200/600], Loss: 0.2061
Epoch [1/5], Step [300/600], Loss: 0.2171
Epoch [1/5], Step [400/600], Loss: 0.1938
Epoch [1/5], Step [500/600], Loss: 0.1509
Epoch [1/5], Step [600/600], Loss: 0.1270
Epoch [2/5], Step [100/600], Loss: 0.0943
Epoch [2/5], Step [200/600], Loss: 0.1652
Epoch [2/5], Step [300/600], Loss: 0.1776
Epoch [2/5], Step [400/600], Loss: 0.0790
Epoch [2/5], Step [500/600], Loss: 0.0335
Epoch [2/5], Step [600/600], Loss: 0.1219
Epoch [3/5], Step [100/600], Loss: 0.0359
Epoch [3/5], Step [200/600], Loss: 0.0632
Epoch [3/5], Step [300/600], Loss: 0.0213
Epoch [3/5], Step [400/600], Loss: 0.0240
Epoch [3/5], Step [500/600], Loss: 0.0642
Epoch [3/5], Step [600/600], Loss: 0.0638
Epoch [4/5], Step [100/600], Loss: 0.0419
Epoch [4/5], Step [200/600], Loss: 0.0875
Epoch [4/5], Step [300/600], Loss: 0.0562
Epoch [4/5], Step [400/600], Loss: 0.0346
Epoch [4/5], Step [500/600], Loss: 0.0592
Epoch [4/5], Step [600/600], Loss: 0.0527
Epoch [5/5], Step [100/600], Loss: 0.0203
Epoch [5/5], Step [200/600], Loss: 0.0482
Epoch [5/5], Step [300/600], Loss: 0.0374
Epoch [5/5], Step [400/600], Loss: 0.0511
Epoch [5/5], Step [500/600], Loss: 0.0061
Epoch [5/5], Step [600/600], Loss: 0.0057
Accuracy of the network on the 10000 test images: 98.01 %