Pytorch搭建DenseNet
文章目录
- DenseNet与ResNeXt
- DenseNet网络的搭建
- Growth_rate
- Transition Layer
- Bottleneck
- 定义网络
- 小测试
- 问题1
- 训练与测试
- 可视化网络结构
DenseNet与ResNeXt
首先回顾一下DenseNet的结构，DenseNet的每一层都都与前面层相连，实现了特征重用。
下图表示一个DenseBlock
如图所示,在一个DenseBlock中,第i层的输入不仅与i-1层的输出相关,还有所有之前层的输出有关.记作:
DenseNet网络的搭建
Growth_rate
在一个DenseBlock里面，每个非线性变换H输出的channels数为恒定的Growth_rate，那么第i层的输入的channels数便是k+i* Growth_rate, k为Input
的channels数，比如，假设我们把Growth_rate设为4，上图中H1的输入的size为8 * 32 * 32，输出为4 * 32 * 32， 则H2的输入的size为12 * 32 * 32，
输出还是4 * 32 * 32，H3、H4以此类推，在实验中，用较小的Growth_rate就能实现较好的效果。
Transition Layer
请注意， 在一个DenseBlock里面，feature size并没有发生改变，因为需要对不同层的feature map进行concatenate操作，这需要保持相同的feature size。
因此在相邻的DenseBlock中间使用Down Sampling来增大感受野（卷积神经网络每一层输出的特征图（feature map）上的像素点在原始图像上映射的区域大小），即使用Transition Layer来实现，一般的Transition Layer包含BN、Conv和Avg_pool，
同时减少维度，压缩率(compress rate)通常为0.5， 即减少一半的维度。
例如，假设block1的输出c * w * h是24 * 32 * 32，那么经过transition之后，block2的输入就是12 * 16 * 16。
Bottleneck
为了减少参数和计算量，DenseNet的非线性变换H采用了Bottleneck结构BN-ReLU-Conv(1×1)-BN-ReLU-Conv(3×3)，1×1的卷积用于降低维度，将channels数降
低至4 * Growth_rate。
Bottleneck是这样一种网络，其输入输出channel差距较大，就像一个瓶颈一样，上窄下宽亦或上宽下窄，特征图的大小会因为最后一步的cat从N×in_planes×H×W变成N×(in_planes+growth_rate)×H×W。而transition则用来把N×in_planes×H×W变成N×growth_rate×0.5H×0.5W
定义网络
C=torch.cat((A,B),1)就表示按维数1（两位情况是列，四维是channel）拼接A和B，也就是横着拼接，A左B右。此时需要注意：行数必须一致，即维数0数值要相同，方能行对齐。拼接后的C的第1维是两个维数1数值和。
kernelsize为1，padding为0和kernelsize为3，padding为1都会让卷积后的特征图的大小与原来的图片一致：一个是(N-1)/1+1,一个是(N+2*1-3)/1+1，由此我们可以得出结论，在stride为1时，只要kernelsize=2*padding+1，就不会改变特征图的大小。
注意：stride会默认为核的大小，因此在下面的实验里,stride的值为4。一般情况下，池化层的输出大小为(N-kernelsize)/stride+1，当stride等于kernelsize的时候，也就是不填stride时，输出的大小为N/kernel。
# Load necessary modules here
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import torch.backends.cudnn as cudnn
import os
from tensorboardX import SummaryWriter
writer = SummaryWriter()
class Bottleneck(nn.Module):
'''
the above mentioned bottleneck, including two conv layer, one's kernel size is 1×1, another's is 3×3
in_planes可以理解成channel
after non-linear operation, concatenate the input to the output
'''
def __init__(self, in_planes, growth_rate):
super(Bottleneck, self).__init__()
self.bn1 = nn.BatchNorm2d(in_planes)
self.conv1 = nn.Conv2d(in_planes, 4*growth_rate, kernel_size=1, bias=False)
self.bn2 = nn.BatchNorm2d(4*growth_rate)
self.conv2 = nn.Conv2d(4*growth_rate, growth_rate, kernel_size=3, padding=1, bias=False)
def forward(self, x):
out = self.conv1(F.relu(self.bn1(x)))
out = self.conv2(F.relu(self.bn2(out)))
# input and output are concatenated here
out = torch.cat([out,x], 1)
return out
class Transition(nn.Module):
'''
transition layer is used for down sampling the feature
when compress rate is 0.5, out_planes is a half of in_planes
'''
def __init__(self, in_planes, out_planes):
super(Transition, self).__init__()
self.bn = nn.BatchNorm2d(in_planes)
self.conv = nn.Conv2d(in_planes, out_planes, kernel_size=1, bias=False)
def forward(self, x):
out = self.conv(F.relu(self.bn(x)))
# use average pooling change the size of feature map here
out = F.avg_pool2d(out, 2)
return out
class DenseNet(nn.Module):
def __init__(self, block, nblocks, growth_rate=12, reduction=0.5, num_classes=10):
super(DenseNet, self).__init__()
'''
Args:
block: bottleneck
nblock: a list, the elements is number of bottleneck in each denseblock
growth_rate: channel size of bottleneck's output
reduction:
'''
self.growth_rate = growth_rate
num_planes = 2*growth_rate
self.conv1 = nn.Conv2d(3, num_planes, kernel_size=3, padding=1, bias=False)
# a DenseBlock and a transition layer
self.dense1 = self._make_dense_layers(block, num_planes, nblocks[0])
num_planes += nblocks[0]*growth_rate
# the channel size is superposed, mutiply by reduction to cut it down here, the reduction is also known as compress rate
out_planes = int(math.floor(num_planes*reduction))
self.trans1 = Transition(num_planes, out_planes)
num_planes = out_planes
# a DenseBlock and a transition layer
self.dense2 = self._make_dense_layers(block, num_planes, nblocks[1])
num_planes += nblocks[1]*growth_rate
# the channel size is superposed, mutiply by reduction to cut it down here, the reduction is also known as compress rate
out_planes = int(math.floor(num_planes*reduction))
self.trans2 = Transition(num_planes, out_planes)
num_planes = out_planes
# a DenseBlock and a transition layer
self.dense3 = self._make_dense_layers(block, num_planes, nblocks[2])
num_planes += nblocks[2]*growth_rate
# the channel size is superposed, mutiply by reduction to cut it down here, the reduction is also known as compress rate
out_planes = int(math.floor(num_planes*reduction))
self.trans3 = Transition(num_planes, out_planes)
num_planes = out_planes
# only one DenseBlock
self.dense4 = self._make_dense_layers(block, num_planes, nblocks[3])
num_planes += nblocks[3]*growth_rate
# the last part is a linear layer as a classifier
self.bn = nn.BatchNorm2d(num_planes)
self.linear = nn.Linear(num_planes, num_classes)
def _make_dense_layers(self, block, in_planes, nblock):
layers = []
# number of non-linear transformations in one DenseBlock depends on the parameter you set
for i in range(nblock):
layers.append(block(in_planes, self.growth_rate))
in_planes += self.growth_rate
return nn.Sequential(*layers)
def forward(self, x):
out = self.conv1(x)
out = self.trans1(self.dense1(out))
out = self.trans2(self.dense2(out))
out = self.trans3(self.dense3(out))
out = self.dense4(out)
out = F.avg_pool2d(F.relu(self.bn(out)), 4)
out = out.view(out.size(0), -1)
out = self.linear(out)
return out
def densenet():
return DenseNet(Bottleneck, [2, 5, 4, 6])
print(densenet())
DenseNet(
(conv1): Conv2d(3, 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(dense1): Sequential(
(0): Bottleneck(
(bn1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(24, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(1): Bottleneck(
(bn1): BatchNorm2d(36, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(36, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(trans1): Transition(
(bn): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv): Conv2d(48, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(dense2): Sequential(
(0): Bottleneck(
(bn1): BatchNorm2d(24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(24, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(1): Bottleneck(
(bn1): BatchNorm2d(36, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(36, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(2): Bottleneck(
(bn1): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(48, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(3): Bottleneck(
(bn1): BatchNorm2d(60, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(60, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(4): Bottleneck(
(bn1): BatchNorm2d(72, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(72, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(trans2): Transition(
(bn): BatchNorm2d(84, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv): Conv2d(84, 42, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(dense3): Sequential(
(0): Bottleneck(
(bn1): BatchNorm2d(42, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(42, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(1): Bottleneck(
(bn1): BatchNorm2d(54, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(54, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(2): Bottleneck(
(bn1): BatchNorm2d(66, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(66, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(3): Bottleneck(
(bn1): BatchNorm2d(78, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(78, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(trans3): Transition(
(bn): BatchNorm2d(90, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv): Conv2d(90, 45, kernel_size=(1, 1), stride=(1, 1), bias=False)
)
(dense4): Sequential(
(0): Bottleneck(
(bn1): BatchNorm2d(45, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(45, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(1): Bottleneck(
(bn1): BatchNorm2d(57, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(57, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(2): Bottleneck(
(bn1): BatchNorm2d(69, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(69, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(3): Bottleneck(
(bn1): BatchNorm2d(81, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(81, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(4): Bottleneck(
(bn1): BatchNorm2d(93, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(93, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(5): Bottleneck(
(bn1): BatchNorm2d(105, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv1): Conv2d(105, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(bn2): BatchNorm2d(48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(48, 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(bn): BatchNorm2d(117, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(linear): Linear(in_features=117, out_features=10, bias=True)
)
小测试
x = torch.randn((3,3,4,8))
y = F.max_pool2d(x, 4)
print(x.shape)
print(y.shape)
b = Bottleneck(3, 5)
print(b(x).shape)
t = Transition(3, 5)
print(t(x).shape)
torch.Size([3, 3, 4, 8])
torch.Size([3, 3, 1, 2])
torch.Size([3, 8, 4, 8])
torch.Size([3, 5, 2, 4])
问题1
上面的定义的DenseNet为多少层DenseNet（只计算卷积层与全连接层）？请定义一个卷积层总数为52层的DenseNet。
答：每个Bottleneck包含了两个卷积层，每个Transition包含一个卷积层，总共有2+5+4+6=17个Bottleneck和3个transition，也就是37个卷积层，再加上最前面的卷积层和最后的全连接层，最后总共有39层。
def densenet52():
return DenseNet(Bottleneck, [6, 4, 8, 6])
训练与测试
import torchvision
import torchvision.transforms as transforms
from torch.autograd import Variable
def train(epoch, model, lossFunction, optimizer, device, trainloader):
"""train model using loss_fn and optimizer. When this function is called, model trains for one epoch.
Args:
train_loader: train data
model: prediction model
loss_fn: loss function to judge the distance between target and outputs
optimizer: optimize the loss function
get_grad: True, False
output:
total_loss: loss
average_grad2: average grad for hidden 2 in this epoch
average_grad3: average grad for hidden 3 in this epoch
"""
print('\nEpoch: %d' % epoch)
model.train() # enter train mode
train_loss = 0 # accumulate every batch loss in a epoch
correct = 0 # count when model' prediction is correct i train set
total = 0 # total number of prediction in train set
for batch_idx, (inputs, targets) in enumerate(trainloader):
inputs, targets = inputs.to(device), targets.to(device) # load data to gpu device
inputs, targets = Variable(inputs), Variable(targets)
optimizer.zero_grad() # clear gradients of all optimized torch.Tensors'
outputs = model(inputs) # forward propagation return the value of softmax function
loss = lossFunction(outputs, targets) #compute loss
loss.backward() # compute gradient of loss over parameters
optimizer.step() # update parameters with gradient descent
train_loss += loss.item() # accumulate every batch loss in a epoch
_, predicted = outputs.max(1) # make prediction according to the outputs
total += targets.size(0)
correct += predicted.eq(targets).sum().item() # count how many predictions is correct
if (batch_idx+1) % 100 == 0:
# print loss and acc
print( 'Train loss: %.3f | Train Acc: %.3f%% (%d/%d)'
% (train_loss/(batch_idx+1), 100.*correct/total, correct, total))
print( 'Train loss: %.3f | Train Acc: %.3f%% (%d/%d)'
% (train_loss/(batch_idx+1), 100.*correct/total, correct, total))
def test(model, lossFunction, optimizer, device, testloader):
"""
test model's prediction performance on loader.
When thid function is called, model is evaluated.
Args:
loader: data for evaluation
model: prediction model
loss_fn: loss function to judge the distance between target and outputs
output:
total_loss
accuracy
"""
global best_acc
model.eval() #enter test mode
test_loss = 0 # accumulate every batch loss in a epoch
correct = 0
total = 0
with torch.no_grad():
for batch_idx, (inputs, targets) in enumerate(testloader):
inputs, targets = inputs.to(device), targets.to(device)
outputs = model(inputs)
loss = lossFunction(outputs, targets) #compute loss
test_loss += loss.item() # accumulate every batch loss in a epoch
_, predicted = outputs.max(1) # make prediction according to the outputs
total += targets.size(0)
correct += predicted.eq(targets).sum().item() # count how many predictions is correct
# print loss and acc
print('Test Loss: %.3f | Test Acc: %.3f%% (%d/%d)'
% (test_loss/(batch_idx+1), 100.*correct/total, correct, total))
def data_loader():
# define method of preprocessing data for evaluating
transform_train = transforms.Compose([
transforms.Resize(32),
transforms.RandomCrop(32, padding=4),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
# Normalize a tensor image with mean and standard variance
transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
])
transform_test = transforms.Compose([
transforms.Resize(32),
transforms.ToTensor(),
# Normalize a tensor image with mean and standard variance
transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
])
# prepare dataset by ImageFolder, data should be classified by directory
trainset = torchvision.datasets.ImageFolder(root='./mnist/train', transform=transform_train)
testset = torchvision.datasets.ImageFolder(root='./mnist/test', transform=transform_test)
# Data loader.
# Combines a dataset and a sampler,
trainloader = torch.utils.data.DataLoader(trainset, batch_size=64, shuffle=True)
testloader = torch.utils.data.DataLoader(testset, batch_size=100, shuffle=False)
return trainloader, testloader
def run(model, num_epochs):
# load model into GPU device
# device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
device = 'cpu'
model.to(device)
# if device == 'cuda:0':
# model = torch.nn.DataParallel(model)
# cudnn.benchmark = True
# define the loss function and optimizer
lossFunction = nn.CrossEntropyLoss()
lr = 0.01
optimizer = optim.SGD(model.parameters(), lr=lr, momentum=0.9, weight_decay=5e-4)
trainloader, testloader = data_loader()
for epoch in range(num_epochs):
train(epoch, model, lossFunction, optimizer, device, trainloader)
test(model, lossFunction, optimizer, device, testloader)
if (epoch + 1) % 50 == 0 :
lr = lr / 10
for param_group in optimizer.param_groups:
param_group['lr'] = lr
用你自己定义的DenseNet进行测试
# start training and testing
model = densenet52()
# num_epochs is adjustable
run(model, num_epochs=20)
Epoch: 0
Train loss: 2.247 | Train Acc: 19.900% (398/2000)
Test Loss: 2.314 | Test Acc: 10.000% (100/1000)
Epoch: 1
Train loss: 1.916 | Train Acc: 38.700% (774/2000)
Test Loss: 2.055 | Test Acc: 25.300% (253/1000)
Epoch: 2
Train loss: 1.398 | Train Acc: 56.450% (1129/2000)
Test Loss: 1.486 | Test Acc: 39.300% (393/1000)
Epoch: 3
Train loss: 0.988 | Train Acc: 72.300% (1446/2000)
Test Loss: 1.224 | Test Acc: 56.600% (566/1000)
Epoch: 4
Train loss: 0.716 | Train Acc: 78.250% (1565/2000)
Test Loss: 0.755 | Test Acc: 75.400% (754/1000)
Epoch: 5
Train loss: 0.561 | Train Acc: 81.000% (1620/2000)
Test Loss: 0.795 | Test Acc: 70.300% (703/1000)
Epoch: 6
Train loss: 0.487 | Train Acc: 82.150% (1643/2000)
Test Loss: 0.634 | Test Acc: 75.500% (755/1000)
Epoch: 7
Train loss: 0.475 | Train Acc: 82.300% (1646/2000)
Test Loss: 0.644 | Test Acc: 77.800% (778/1000)
Epoch: 8
Train loss: 0.458 | Train Acc: 83.550% (1671/2000)
Test Loss: 0.395 | Test Acc: 84.300% (843/1000)
Epoch: 9
Train loss: 0.403 | Train Acc: 86.000% (1720/2000)
Test Loss: 0.551 | Test Acc: 82.500% (825/1000)
Epoch: 10
Train loss: 0.351 | Train Acc: 87.050% (1741/2000)
Test Loss: 0.674 | Test Acc: 77.100% (771/1000)
Epoch: 11
Train loss: 0.321 | Train Acc: 89.100% (1782/2000)
Test Loss: 0.338 | Test Acc: 88.900% (889/1000)
Epoch: 12
Train loss: 0.256 | Train Acc: 92.400% (1848/2000)
Test Loss: 0.394 | Test Acc: 87.800% (878/1000)
Epoch: 13
Train loss: 0.251 | Train Acc: 92.550% (1851/2000)
Test Loss: 0.448 | Test Acc: 84.300% (843/1000)
Epoch: 14
Train loss: 0.259 | Train Acc: 92.150% (1843/2000)
Test Loss: 0.414 | Test Acc: 86.000% (860/1000)
Epoch: 15
Train loss: 0.229 | Train Acc: 93.850% (1877/2000)
Test Loss: 0.321 | Test Acc: 88.300% (883/1000)
Epoch: 16
Train loss: 0.196 | Train Acc: 94.400% (1888/2000)
Test Loss: 0.393 | Test Acc: 87.100% (871/1000)
Epoch: 17
Train loss: 0.178 | Train Acc: 95.350% (1907/2000)
Test Loss: 0.295 | Test Acc: 90.600% (906/1000)
Epoch: 18
Train loss: 0.184 | Train Acc: 94.650% (1893/2000)
Test Loss: 0.351 | Test Acc: 87.700% (877/1000)
Epoch: 19
Train loss: 0.168 | Train Acc: 95.200% (1904/2000)
Test Loss: 0.164 | Test Acc: 95.500% (955/1000)
可视化网络结构
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'
model.to(device)
train_loader, _ = data_loader()
dataiter = iter(train_loader)
images, _ = dataiter.next() # get a batch of images
images = images.to(device)
with SummaryWriter(comment="densenet52") as s:
s.add_graph(model, (Variable(images),))
s.close()