经典CNN算法解析实战-第J3周：DenseNet算法实战与解析
- 本文为365天深度学习训练营中的学习记录博客
- 原作者：K同学啊|接辅导、项目定制
目录
- 一、课题背景和开发环境
- 二、设计理念
- 三、网络结构
- 四、与其他算法效果对比
- 五、使用Pytorch/TensorFlow实现DenseNet121
- 六、DenseNet121网络结构图
一、课题背景和开发环境
第J3周：DenseNet算法实战与解析
- 语言：Python3、Pytorch
- 本周任务：
– 1.请根据本文Pytorch代码，编写出相应的TensorFlow代码（建议使用上周的数据测试一下模型是否构建正确）
– 2.了解并研究DenseNet与ResNetV的区别
– 3.改进思路是否可以迁移到其他地方呢（自由探索，虽然不强求但是请认真对待这个哦）
**注：**从前几周开始训练营的难度逐渐提升，具体体现在不再直接提供源代码。任务中会给大家提供一些算法改进的思路/方向，希望大家这一块可以积极探索。（这个探索的过程很重要，也将学到更多）
在计算机视觉领域，卷积神经网络（CNN）已经成为最主流的方法，比如GoogLenet，VGG-16，Inception等模型。CNN史上的一个里程碑事件是ResNet模型的出现，ResNet可以训练出更深的CNN模型，从而实现更高的准确度。ResNet模型的核心是通过建立前面层与后面层之间的“短路连接”（shortcuts，skip connection），进而训练出更深的CNN网络。
今天我们要介绍的是DenseNet模型，它的基本思路与ResNet一致，但是它建立的是前面所有层与后面层的密集连接（dense connection），它的名称也是由此而来。DenseNet的另一大特色是通过特征在channel上的连接来实现特征重用（feature reuse）。这些特点让DenseNet在参数和计算成本更少的情形下实现比ResNet更优的性能，DenseNet也因此斩获CVPR 2017的最佳论文奖。
DenseNet论文原文：
Densely Connected Convolutional Networks
二、设计理念
相比ResNet，DenseNet提出了一个更激进的密集连接机制：即互相连接所有的层，具体来说就是每个层都会接受其前面所有层作为其额外的输入。
图1为ResNet网络的残差连接机制，作为对比，图2为DenseNet的密集连接机制。可以看到，ResNet是每个层与前面的某层（一般是2~4层）短路连接在一起，连接方式是通过 元素相加。而在DenseNet中，每个层都会与前面所有层在channel维度上连接（concat）在一起（即 元素叠加），并作为下一层的输入。
对于一个 L L L 层的网络，DenseNet共包含 L ( L + 1 ) 2 \frac{L(L+1)}{2} 2L(L+1) 个连接，相比ResNet，这是一种密集连接。而且DenseNet是直接concat来自不同层的特征图，这可以实现特征重用，提升效率，这一特点是DenseNet与ResNet最主要的区别。
- 输入和输出公式是 X l = H l ( X l − 1 ) X_l=H_l(X_{l-1}) Xl=Hl(Xl−1) ，其中 X l X_l Xl 是一个组合函数，通常包括BN、ReLU、Pooling、Conv操作， X l − 1 X_{l-1} Xl−1 是第 l l l 层输入的特征图， X l X_l Xl 是第 l l l 层输出的特征图。
- ResNet是跨层相加，输入和输出的公式是 X l = H l ( X l − 1 ) + X l − 1 X_l=H_l(X_{l-1})+X_{l-1} Xl=Hl(Xl−1)+Xl−1 。
- 而对于DenseNet，则是采用跨通道concat的形式来连接，会连接前面所有层作为输入，输入和输出的公式是 $$。这里要注意所有的层的输入都来源于前面所有层在channel维度的concat，我们用一张动图体会一下：
三、网络结构
具体介绍网络的具体实现细节如图4所示。
CNN网络一般要经过Pooling或者stride>1的Conv来降低特征图的大小，而DenseNet的密集连接方式需要特征图大小保持一致。为了解决这个问题，DenseNet网络中使用DenseBlock+Transition的结构，其中DenseBlock是包含很多层的模块，每个层的特征图大小相同，层与层之间采用密集连接方式。而Transition层是连接两个相邻的DenseBlock，并且通过Pooling使特征图大小降低。图5给出了DenseNet的网络结构，它共包含4个DenseBlock，各个DenseBlock之间通过Transition层连接在一起。
在DenseBlock中，各个层的特征图大小一致，可以在channel维度上连接。DenseBlock中的非线性组合函数 H ( ⋅ ) H(\cdot) H(⋅) 的是 BN + ReLU + 3x3 Conv 的结构，如图6所示。另外值得注意的一点是，与ResNet不同，所有DenseBlock中各个层卷积之后均输入 k k k 个特征图，即得到的特征图的channel数为 k k k，或者说采用 k k k 个卷积核。 k k k 在DenseNet称为growth rate，这是一个超参数。一般情况下使用较小的 k k k（比如12），就可以得到较佳的性能。假定输入层的特征图的channel数为 k 0 k_0 k0，那么 l l l 层输入的channel数为 k 0 + k ( 1 , 2 , … , l − 1 ) k_0+k_{(1,2,\ldots,l-1)} k0+k(1,2,…,l−1)，因此随着层数增加，尽管 k k k 设定得较小，DenseBlock的输入会非常多，不过这是由于特征重用所造成的，每个层仅有 k k k 个特征是自己独有的。
由于后面层的输入会非常大，DenseBlock内部可以采用bottleneck层来减少计算量，主要是原有的结构中增加1x1 Conv，如图7所示，即 BN + ReLU + 1x1 Conv + BN + ReLU + 3x3 Conv，称为DenseNet-B结构。其中1x1 Conv得到 4 k 4k 4k 个特征图，它起到的作用是降低特征数量，从而提升计算的效率。
对于Transition层，它主要是连接两个相邻的DenseBlock，并且降低特征图大小。Transition层包括一个1x1的卷积和2x2的AvgPooling，结构为 BN + ReLU + 1x1Conv + 2x2AvgPooling。另外，Transition层可以起到压缩模型的作用。假定Transition层的上接DenseBlock得到的特征图channels数为 m m m，Transition层可以产生 ⌊ θ m ⌋ \lfloor \theta m\rfloor ⌊θm⌋ 个特征（通过卷积层），其中 θ ∈ ( 0 , 1 ] \theta\in(0,1] θ∈(0,1] 是压缩系数（compression rate）。当 θ = 1 \theta=1 θ=1 时，特征个数经过Transition层没有变化，即无压缩，而当压缩系数小于1时，这种结构称为DenseNet-C，文章使用 θ = 0.5 \theta=0.5 θ=0.5。对于使用Bottleneck层的DenseBlock结构和压缩系数小于1的Transition组合结构称为DenseNet-BC。
对于ImageNet数据集，图片输入大小为 224 × 224 224\times224 224×224，网络结构采用包含4个DenseBlock的DenseNet-BC，其首先时一个stride=2的7x7卷积层，然后是一个stride=2的3x3 MaxPooling层，后面才进入DenseBlock。
ImageNet数据集所采用的网络配置如表1所示：
四、与其他算法效果对比
五、使用Pytorch/TensorFlow实现DenseNet121
这里我们采用Pytorch框架来实现DenseNet，首先实现DenseBlock中的内部结构，这里是 BN + ReLU + 1x1Conv + BN + ReLU + 3x3Conv 结构，最后也加入Dropout层用于训练过程。
Pytorch
''' Basic unit of DenseBlock (using bottleneck layer) '''
class DenseLayer(nn.Sequential):
def __init__(self, in_channel, growth_rate, bn_size, drop_rate):
super(DenseLayer, self).__init__()
self.add_module('norm1', nn.BatchNorm2d(in_channel))
self.add_module('relu1', nn.ReLU(inplace=True))
self.add_module('conv1', nn.Conv2d(in_channel, bn_size*growth_rate,
kernel_size=1, stride=1, bias=False))
self.add_module('norm2', nn.BatchNorm2d(bn_size*growth_rate))
self.add_module('relu2', nn.ReLU(inplace=True))
self.add_module('conv2', nn.Conv2d(bn_size*growth_rate, growth_rate,
kernel_size=3, stride=1, padding=1, bias=False))
self.drop_rate = drop_rate
def forward(self, x):
new_feature = super(DenseLayer, self).forward(x)
if self.drop_rate>0:
new_feature = F.dropout(new_feature, p=self.drop_rate, training=self.training)
return torch.cat([x, new_feature], 1)
tensorFlow
''' Basic unit of DenseBlock (using bottleneck layer) '''
def DenseLayer(x, bn_size, growth_rate, drop_rate, name=None):
f = BatchNormalization(name=name+'_1_bn')(x)
f = Activation('relu', name=name+'_1_relu')(f)
f = Conv2D(bn_size*growth_rate, 1, strides=1, use_bias=False, name=name+'_1_conv')(f)
f = BatchNormalization(name=name+'_2_bn')(f)
f = Activation('relu', name=name+'_2_relu')(f)
f = Conv2D(growth_rate, 3, strides=1, padding=1, use_bias=False, name=name+'_2_conv')(f)
if drop_rate>0:
f = Dropout(drop_rate)(f)
x = layers.Concatenate(axis=-1)([x, f])
return x
据此，实现DenseBlock模块，内部是密集连接方式（输入特征数线性增长）：
Pytorch
''' DenseBlock '''
class DenseBlock(nn.Sequential):
def __init__(self, num_layers, in_channel, bn_size, growth_rate, drop_rate):
super(DenseBlock, self).__init__()
for i in range(num_layers):
layer = DenseLayer(in_channel+i*growth_rate, growth_rate, bn_size, drop_rate)
self.add_module('denselayer%d'%(i+1,), layer)
tensorFlow
''' DenseBlock '''
def DenseBlock(x, num_layers, bn_size, growth_rate, drop_rate, name=None):
for i in range(num_layers):
x = DenseLayer(x, bn_size, growth_rate, drop_rate, name=name+'_denselayer'+str(i+1))
return x
此外，我们实现Transition层，它主要是一个卷积层和一个池化层：
Pytorch
''' Transition layer between two adjacent DenseBlock '''
class Transition(nn.Sequential):
def __init__(self, in_channel, out_channel):
super(Transition, self).__init__()
self.add_module('norm', nn.BatchNorm2d(in_channel))
self.add_module('relu', nn.ReLU(inplace=True))
self.add_module('conv', nn.Conv2d(in_channel, out_channel,
kernel_size=1, stride=1, bias=False))
self.add_module('pool', nn.AvgPool2d(2, stride=2))
tensorFlow
''' Transition layer between two adjacent DenseBlock '''
def Transition(x, out_channel):
x = BatchNormalization(name=name+'_bn')(x)
x = Activation('relu', name=name+'_relu')(x)
x = Conv2D(out_channel, 1, strides=1, use_bias=False, name=name+'_conv')(x)
x = AveragePooling2D((2, 2), name='pool')(x)
return x
最后我们实现DenseNet网络：
Pytorch
''' DenseNet-BC model '''
class DenseNet(nn.Module):
def __init__(self, growth_rate=32, block_config=(6,12,24,16), init_channel=64,
bn_size=4, compression_rate=0.5, drop_rate=0, num_classes=1000):
'''
:param growth_rate: (int) number of filters used in DenseLayer, `k` in the paper
:param block_config: (list of 4 ints) number of layers in eatch DenseBlock
:param init_channel: (int) number of filters in the first Conv2d
:param bn_size: (int) the factor using in the bottleneck layer
:param compression_rate: (float) the compression rate used in Transition Layer
:param drop_rate: (float) the drop rate after each DenseLayer
:param num_classes: (int) number of classes for classification
'''
super(DenseNet, self).__init__()
# first Conv2d
self.features = nn.Sequential(OrderedDict([
('conv0', nn.Conv2d(3, init_channel, kernel_size=7, stride=2, padding=3, bias=False)),
('norm0', nn.BatchNorm2d(init_channel)),
('relu0', nn.ReLU(inplace=True)),
('pool0', nn.MaxPool2d(3, stride=2, padding=1))
]))
# DenseBlock
num_features = init_channel
for i, num_layers in enumerate(block_config):
block = DenseBlock(num_layers, num_features, bn_size, growth_rate, drop_rate)
self.features.add_module('denseblock%d'%(i+1), block)
num_features += num_layers*growth_rate
if i != len(block_config)-1:
transition = Transition(num_features, int(num_features*compression_rate))
self.features.add_module('transition%d'%(i+1), transition)
num_features = int(num_features*compression_rate)
# final BN+ReLU
self.features.add_module('norm5', nn.BatchNorm2d(num_features))
self.features.add_module('relu5', nn.ReLU(inplace=True))
# classification layer
self.classifier = nn.Linear(num_features, num_classes)
# params initialization
for m in self.modules():
if isinstance(m, nn.Conv2d):
nn.init.kaiming_normal_(m.weight)
elif isinstance(m, nn.BatchNorm2d):
nn.init.constant_(m.bias, 0)
nn.init.constant_(m.weight, 1)
elif isinstance(m, nn.Linear):
nn.init.constant_(m.bias, 0)
def forward(self, x):
x = self.features(x)
x = F.avg_pool2d(x, 7, stride=1).view(x.size(0), -1)
x = self.classifier(x)
return x
tensorFlow
''' DenseNet-BC model '''
def DenseNet(input_tensor=None, # 可选的keras张量，用作模型的图像输入
input_shape=None,
init_channel=64,
growth_rate=32,
block_config=(6,12,24,16),
bn_size=4,
compression_rate=0.5,
drop_rate=0,
classes=1000): # 用于分类图像的可选类数
img_input = Input(shape=input_shape)
# first Conv2d
x = ZeroPadding2D(padding=((3, 3), (3, 3)), name='conv1_pad')(img_input)
x = Conv2D(64, 7, strides=2, use_bias=False, name='conv1_conv')(x)
x = BatchNormalization(name='conv1_bn')(x)
x = Activation('relu', name='conv1_relu')(x)
x = MaxPooling2D(3, strides=2, padding=1, name='conv1_pool')(x)
# DenseBlock
num_features = init_channel
for i, num_layers in enumerate(block_config):
x = DenseBlock(x, num_layers, bn_size, growth_rate, drop_rate, name='denseblock'+str(i+1))
num_features += num_layers*growth_rate
if i!=len(block_config)-1:
x = Transition(x, int(num_features*compression_rate))
num_features = int(num_features*compression_rate)
# final bn+ReLU
x = BatchNormalization(name='final_bn')(x)
x = Activation('relu', name='final_relu')(x)
x = GlobalAveragePooling2D(name='final_pool')(x)
x = Dense(classes, activation='softmax', name='predictions')(x)
model = Model(img_input, x, name='DenseNet')
return model
选择不同的网络参数，就可以实现不同深度的DenseNet，这里实现DenseNet-121网络，而且Pytorch提供了预训练好的网络参数：
Pytorch
''' DenseNet121 '''
def densenet121(n_classes=1000, pretrained=False, **kwargs):
model = DenseNet(init_channel=64, growth_rate=32, block_config=(6,12,24,16),
num_classes=n_classes, **kwargs)
if pretrained:
# '.'s are no longer allowed in module names, but previous DenseLayer
# has keys 'norm.1', 'relu.1', 'conv.1', 'norm.2', 'relu.2', 'conv.2'.
# They are also in the checkpoints in model_urls. This pattern is used
# to find such keys.
pattern = re.compile(
r'^(.*denselayer\d+\.(?:norm|relu|conv))\.((?:[12])\.(?:weight|bias|running_mean|running_var))$')
state_dict = model_zoo.load_url(model_urls['densenet121'])
for key in list(state_dict.keys()):
res = pattern.match(key)
if res:
new_key = res.group(1) + res.group(2)
state_dict[new_key] = state_dict[key]
del state_dict[key]
model.load_state_dict(state_dict)
return model
tensorFlow
''' DenseNet121 '''
def densenet121(n_classes=1000, pretrained=False, **kwargs):
model = DenseNet(init_channel=64, growth_rate=32, block_config=(6,12,24,16),
classes=n_classes, **kwargs)
if pretrained:
# '.'s are no longer allowed in module names, but previous DenseLayer
# has keys 'norm.1', 'relu.1', 'conv.1', 'norm.2', 'relu.2', 'conv.2'.
# They are also in the checkpoints in model_urls. This pattern is used
# to find such keys.
pass
return model
输出日志：
----------------------------------------------------------------
Layer (type) Output Shape Param #
================================================================
Conv2d-1 [-1, 64, 112, 112] 9,408
BatchNorm2d-2 [-1, 64, 112, 112] 128
ReLU-3 [-1, 64, 112, 112] 0
MaxPool2d-4 [-1, 64, 56, 56] 0
BatchNorm2d-5 [-1, 64, 56, 56] 128
ReLU-6 [-1, 64, 56, 56] 0
Conv2d-7 [-1, 128, 56, 56] 8,192
BatchNorm2d-8 [-1, 128, 56, 56] 256
ReLU-9 [-1, 128, 56, 56] 0
Conv2d-10 [-1, 32, 56, 56] 36,864
BatchNorm2d-11 [-1, 96, 56, 56] 192
ReLU-12 [-1, 96, 56, 56] 0
Conv2d-13 [-1, 128, 56, 56] 12,288
BatchNorm2d-14 [-1, 128, 56, 56] 256
ReLU-15 [-1, 128, 56, 56] 0
Conv2d-16 [-1, 32, 56, 56] 36,864
BatchNorm2d-17 [-1, 128, 56, 56] 256
ReLU-18 [-1, 128, 56, 56] 0
Conv2d-19 [-1, 128, 56, 56] 16,384
BatchNorm2d-20 [-1, 128, 56, 56] 256
ReLU-21 [-1, 128, 56, 56] 0
Conv2d-22 [-1, 32, 56, 56] 36,864
BatchNorm2d-23 [-1, 160, 56, 56] 320
ReLU-24 [-1, 160, 56, 56] 0
Conv2d-25 [-1, 128, 56, 56] 20,480
BatchNorm2d-26 [-1, 128, 56, 56] 256
ReLU-27 [-1, 128, 56, 56] 0
Conv2d-28 [-1, 32, 56, 56] 36,864
BatchNorm2d-29 [-1, 192, 56, 56] 384
ReLU-30 [-1, 192, 56, 56] 0
Conv2d-31 [-1, 128, 56, 56] 24,576
BatchNorm2d-32 [-1, 128, 56, 56] 256
ReLU-33 [-1, 128, 56, 56] 0
Conv2d-34 [-1, 32, 56, 56] 36,864
BatchNorm2d-35 [-1, 224, 56, 56] 448
ReLU-36 [-1, 224, 56, 56] 0
Conv2d-37 [-1, 128, 56, 56] 28,672
BatchNorm2d-38 [-1, 128, 56, 56] 256
ReLU-39 [-1, 128, 56, 56] 0
Conv2d-40 [-1, 32, 56, 56] 36,864
BatchNorm2d-41 [-1, 256, 56, 56] 512
ReLU-42 [-1, 256, 56, 56] 0
Conv2d-43 [-1, 128, 56, 56] 32,768
AvgPool2d-44 [-1, 128, 28, 28] 0
BatchNorm2d-45 [-1, 128, 28, 28] 256
ReLU-46 [-1, 128, 28, 28] 0
Conv2d-47 [-1, 128, 28, 28] 16,384
BatchNorm2d-48 [-1, 128, 28, 28] 256
ReLU-49 [-1, 128, 28, 28] 0
Conv2d-50 [-1, 32, 28, 28] 36,864
BatchNorm2d-51 [-1, 160, 28, 28] 320
ReLU-52 [-1, 160, 28, 28] 0
Conv2d-53 [-1, 128, 28, 28] 20,480
BatchNorm2d-54 [-1, 128, 28, 28] 256
ReLU-55 [-1, 128, 28, 28] 0
Conv2d-56 [-1, 32, 28, 28] 36,864
BatchNorm2d-57 [-1, 192, 28, 28] 384
ReLU-58 [-1, 192, 28, 28] 0
Conv2d-59 [-1, 128, 28, 28] 24,576
BatchNorm2d-60 [-1, 128, 28, 28] 256
ReLU-61 [-1, 128, 28, 28] 0
Conv2d-62 [-1, 32, 28, 28] 36,864
BatchNorm2d-63 [-1, 224, 28, 28] 448
ReLU-64 [-1, 224, 28, 28] 0
Conv2d-65 [-1, 128, 28, 28] 28,672
BatchNorm2d-66 [-1, 128, 28, 28] 256
ReLU-67 [-1, 128, 28, 28] 0
Conv2d-68 [-1, 32, 28, 28] 36,864
BatchNorm2d-69 [-1, 256, 28, 28] 512
ReLU-70 [-1, 256, 28, 28] 0
Conv2d-71 [-1, 128, 28, 28] 32,768
BatchNorm2d-72 [-1, 128, 28, 28] 256
ReLU-73 [-1, 128, 28, 28] 0
Conv2d-74 [-1, 32, 28, 28] 36,864
BatchNorm2d-75 [-1, 288, 28, 28] 576
ReLU-76 [-1, 288, 28, 28] 0
Conv2d-77 [-1, 128, 28, 28] 36,864
BatchNorm2d-78 [-1, 128, 28, 28] 256
ReLU-79 [-1, 128, 28, 28] 0
Conv2d-80 [-1, 32, 28, 28] 36,864
BatchNorm2d-81 [-1, 320, 28, 28] 640
ReLU-82 [-1, 320, 28, 28] 0
Conv2d-83 [-1, 128, 28, 28] 40,960
BatchNorm2d-84 [-1, 128, 28, 28] 256
ReLU-85 [-1, 128, 28, 28] 0
Conv2d-86 [-1, 32, 28, 28] 36,864
BatchNorm2d-87 [-1, 352, 28, 28] 704
ReLU-88 [-1, 352, 28, 28] 0
Conv2d-89 [-1, 128, 28, 28] 45,056
BatchNorm2d-90 [-1, 128, 28, 28] 256
ReLU-91 [-1, 128, 28, 28] 0
Conv2d-92 [-1, 32, 28, 28] 36,864
BatchNorm2d-93 [-1, 384, 28, 28] 768
ReLU-94 [-1, 384, 28, 28] 0
Conv2d-95 [-1, 128, 28, 28] 49,152
BatchNorm2d-96 [-1, 128, 28, 28] 256
ReLU-97 [-1, 128, 28, 28] 0
Conv2d-98 [-1, 32, 28, 28] 36,864
BatchNorm2d-99 [-1, 416, 28, 28] 832
ReLU-100 [-1, 416, 28, 28] 0
Conv2d-101 [-1, 128, 28, 28] 53,248
BatchNorm2d-102 [-1, 128, 28, 28] 256
ReLU-103 [-1, 128, 28, 28] 0
Conv2d-104 [-1, 32, 28, 28] 36,864
BatchNorm2d-105 [-1, 448, 28, 28] 896
ReLU-106 [-1, 448, 28, 28] 0
Conv2d-107 [-1, 128, 28, 28] 57,344
BatchNorm2d-108 [-1, 128, 28, 28] 256
ReLU-109 [-1, 128, 28, 28] 0
Conv2d-110 [-1, 32, 28, 28] 36,864
BatchNorm2d-111 [-1, 480, 28, 28] 960
ReLU-112 [-1, 480, 28, 28] 0
Conv2d-113 [-1, 128, 28, 28] 61,440
BatchNorm2d-114 [-1, 128, 28, 28] 256
ReLU-115 [-1, 128, 28, 28] 0
Conv2d-116 [-1, 32, 28, 28] 36,864
BatchNorm2d-117 [-1, 512, 28, 28] 1,024
ReLU-118 [-1, 512, 28, 28] 0
Conv2d-119 [-1, 256, 28, 28] 131,072
AvgPool2d-120 [-1, 256, 14, 14] 0
BatchNorm2d-121 [-1, 256, 14, 14] 512
ReLU-122 [-1, 256, 14, 14] 0
Conv2d-123 [-1, 128, 14, 14] 32,768
BatchNorm2d-124 [-1, 128, 14, 14] 256
ReLU-125 [-1, 128, 14, 14] 0
Conv2d-126 [-1, 32, 14, 14] 36,864
BatchNorm2d-127 [-1, 288, 14, 14] 576
ReLU-128 [-1, 288, 14, 14] 0
Conv2d-129 [-1, 128, 14, 14] 36,864
BatchNorm2d-130 [-1, 128, 14, 14] 256
ReLU-131 [-1, 128, 14, 14] 0
Conv2d-132 [-1, 32, 14, 14] 36,864
BatchNorm2d-133 [-1, 320, 14, 14] 640
ReLU-134 [-1, 320, 14, 14] 0
Conv2d-135 [-1, 128, 14, 14] 40,960
BatchNorm2d-136 [-1, 128, 14, 14] 256
ReLU-137 [-1, 128, 14, 14] 0
Conv2d-138 [-1, 32, 14, 14] 36,864
BatchNorm2d-139 [-1, 352, 14, 14] 704
ReLU-140 [-1, 352, 14, 14] 0
Conv2d-141 [-1, 128, 14, 14] 45,056
BatchNorm2d-142 [-1, 128, 14, 14] 256
ReLU-143 [-1, 128, 14, 14] 0
Conv2d-144 [-1, 32, 14, 14] 36,864
BatchNorm2d-145 [-1, 384, 14, 14] 768
ReLU-146 [-1, 384, 14, 14] 0
Conv2d-147 [-1, 128, 14, 14] 49,152
BatchNorm2d-148 [-1, 128, 14, 14] 256
ReLU-149 [-1, 128, 14, 14] 0
Conv2d-150 [-1, 32, 14, 14] 36,864
BatchNorm2d-151 [-1, 416, 14, 14] 832
ReLU-152 [-1, 416, 14, 14] 0
Conv2d-153 [-1, 128, 14, 14] 53,248
BatchNorm2d-154 [-1, 128, 14, 14] 256
ReLU-155 [-1, 128, 14, 14] 0
Conv2d-156 [-1, 32, 14, 14] 36,864
BatchNorm2d-157 [-1, 448, 14, 14] 896
ReLU-158 [-1, 448, 14, 14] 0
Conv2d-159 [-1, 128, 14, 14] 57,344
BatchNorm2d-160 [-1, 128, 14, 14] 256
ReLU-161 [-1, 128, 14, 14] 0
Conv2d-162 [-1, 32, 14, 14] 36,864
BatchNorm2d-163 [-1, 480, 14, 14] 960
ReLU-164 [-1, 480, 14, 14] 0
Conv2d-165 [-1, 128, 14, 14] 61,440
BatchNorm2d-166 [-1, 128, 14, 14] 256
ReLU-167 [-1, 128, 14, 14] 0
Conv2d-168 [-1, 32, 14, 14] 36,864
BatchNorm2d-169 [-1, 512, 14, 14] 1,024
ReLU-170 [-1, 512, 14, 14] 0
Conv2d-171 [-1, 128, 14, 14] 65,536
BatchNorm2d-172 [-1, 128, 14, 14] 256
ReLU-173 [-1, 128, 14, 14] 0
Conv2d-174 [-1, 32, 14, 14] 36,864
BatchNorm2d-175 [-1, 544, 14, 14] 1,088
ReLU-176 [-1, 544, 14, 14] 0
Conv2d-177 [-1, 128, 14, 14] 69,632
BatchNorm2d-178 [-1, 128, 14, 14] 256
ReLU-179 [-1, 128, 14, 14] 0
Conv2d-180 [-1, 32, 14, 14] 36,864
BatchNorm2d-181 [-1, 576, 14, 14] 1,152
ReLU-182 [-1, 576, 14, 14] 0
Conv2d-183 [-1, 128, 14, 14] 73,728
BatchNorm2d-184 [-1, 128, 14, 14] 256
ReLU-185 [-1, 128, 14, 14] 0
Conv2d-186 [-1, 32, 14, 14] 36,864
BatchNorm2d-187 [-1, 608, 14, 14] 1,216
ReLU-188 [-1, 608, 14, 14] 0
Conv2d-189 [-1, 128, 14, 14] 77,824
BatchNorm2d-190 [-1, 128, 14, 14] 256
ReLU-191 [-1, 128, 14, 14] 0
Conv2d-192 [-1, 32, 14, 14] 36,864
BatchNorm2d-193 [-1, 640, 14, 14] 1,280
ReLU-194 [-1, 640, 14, 14] 0
Conv2d-195 [-1, 128, 14, 14] 81,920
BatchNorm2d-196 [-1, 128, 14, 14] 256
ReLU-197 [-1, 128, 14, 14] 0
Conv2d-198 [-1, 32, 14, 14] 36,864
BatchNorm2d-199 [-1, 672, 14, 14] 1,344
ReLU-200 [-1, 672, 14, 14] 0
Conv2d-201 [-1, 128, 14, 14] 86,016
BatchNorm2d-202 [-1, 128, 14, 14] 256
ReLU-203 [-1, 128, 14, 14] 0
Conv2d-204 [-1, 32, 14, 14] 36,864
BatchNorm2d-205 [-1, 704, 14, 14] 1,408
ReLU-206 [-1, 704, 14, 14] 0
Conv2d-207 [-1, 128, 14, 14] 90,112
BatchNorm2d-208 [-1, 128, 14, 14] 256
ReLU-209 [-1, 128, 14, 14] 0
Conv2d-210 [-1, 32, 14, 14] 36,864
BatchNorm2d-211 [-1, 736, 14, 14] 1,472
ReLU-212 [-1, 736, 14, 14] 0
Conv2d-213 [-1, 128, 14, 14] 94,208
BatchNorm2d-214 [-1, 128, 14, 14] 256
ReLU-215 [-1, 128, 14, 14] 0
Conv2d-216 [-1, 32, 14, 14] 36,864
BatchNorm2d-217 [-1, 768, 14, 14] 1,536
ReLU-218 [-1, 768, 14, 14] 0
Conv2d-219 [-1, 128, 14, 14] 98,304
BatchNorm2d-220 [-1, 128, 14, 14] 256
ReLU-221 [-1, 128, 14, 14] 0
Conv2d-222 [-1, 32, 14, 14] 36,864
BatchNorm2d-223 [-1, 800, 14, 14] 1,600
ReLU-224 [-1, 800, 14, 14] 0
Conv2d-225 [-1, 128, 14, 14] 102,400
BatchNorm2d-226 [-1, 128, 14, 14] 256
ReLU-227 [-1, 128, 14, 14] 0
Conv2d-228 [-1, 32, 14, 14] 36,864
BatchNorm2d-229 [-1, 832, 14, 14] 1,664
ReLU-230 [-1, 832, 14, 14] 0
Conv2d-231 [-1, 128, 14, 14] 106,496
BatchNorm2d-232 [-1, 128, 14, 14] 256
ReLU-233 [-1, 128, 14, 14] 0
Conv2d-234 [-1, 32, 14, 14] 36,864
BatchNorm2d-235 [-1, 864, 14, 14] 1,728
ReLU-236 [-1, 864, 14, 14] 0
Conv2d-237 [-1, 128, 14, 14] 110,592
BatchNorm2d-238 [-1, 128, 14, 14] 256
ReLU-239 [-1, 128, 14, 14] 0
Conv2d-240 [-1, 32, 14, 14] 36,864
BatchNorm2d-241 [-1, 896, 14, 14] 1,792
ReLU-242 [-1, 896, 14, 14] 0
Conv2d-243 [-1, 128, 14, 14] 114,688
BatchNorm2d-244 [-1, 128, 14, 14] 256
ReLU-245 [-1, 128, 14, 14] 0
Conv2d-246 [-1, 32, 14, 14] 36,864
BatchNorm2d-247 [-1, 928, 14, 14] 1,856
ReLU-248 [-1, 928, 14, 14] 0
Conv2d-249 [-1, 128, 14, 14] 118,784
BatchNorm2d-250 [-1, 128, 14, 14] 256
ReLU-251 [-1, 128, 14, 14] 0
Conv2d-252 [-1, 32, 14, 14] 36,864
BatchNorm2d-253 [-1, 960, 14, 14] 1,920
ReLU-254 [-1, 960, 14, 14] 0
Conv2d-255 [-1, 128, 14, 14] 122,880
BatchNorm2d-256 [-1, 128, 14, 14] 256
ReLU-257 [-1, 128, 14, 14] 0
Conv2d-258 [-1, 32, 14, 14] 36,864
BatchNorm2d-259 [-1, 992, 14, 14] 1,984
ReLU-260 [-1, 992, 14, 14] 0
Conv2d-261 [-1, 128, 14, 14] 126,976
BatchNorm2d-262 [-1, 128, 14, 14] 256
ReLU-263 [-1, 128, 14, 14] 0
Conv2d-264 [-1, 32, 14, 14] 36,864
BatchNorm2d-265 [-1, 1024, 14, 14] 2,048
ReLU-266 [-1, 1024, 14, 14] 0
Conv2d-267 [-1, 512, 14, 14] 524,288
AvgPool2d-268 [-1, 512, 7, 7] 0
BatchNorm2d-269 [-1, 512, 7, 7] 1,024
ReLU-270 [-1, 512, 7, 7] 0
Conv2d-271 [-1, 128, 7, 7] 65,536
BatchNorm2d-272 [-1, 128, 7, 7] 256
ReLU-273 [-1, 128, 7, 7] 0
Conv2d-274 [-1, 32, 7, 7] 36,864
BatchNorm2d-275 [-1, 544, 7, 7] 1,088
ReLU-276 [-1, 544, 7, 7] 0
Conv2d-277 [-1, 128, 7, 7] 69,632
BatchNorm2d-278 [-1, 128, 7, 7] 256
ReLU-279 [-1, 128, 7, 7] 0
Conv2d-280 [-1, 32, 7, 7] 36,864
BatchNorm2d-281 [-1, 576, 7, 7] 1,152
ReLU-282 [-1, 576, 7, 7] 0
Conv2d-283 [-1, 128, 7, 7] 73,728
BatchNorm2d-284 [-1, 128, 7, 7] 256
ReLU-285 [-1, 128, 7, 7] 0
Conv2d-286 [-1, 32, 7, 7] 36,864
BatchNorm2d-287 [-1, 608, 7, 7] 1,216
ReLU-288 [-1, 608, 7, 7] 0
Conv2d-289 [-1, 128, 7, 7] 77,824
BatchNorm2d-290 [-1, 128, 7, 7] 256
ReLU-291 [-1, 128, 7, 7] 0
Conv2d-292 [-1, 32, 7, 7] 36,864
BatchNorm2d-293 [-1, 640, 7, 7] 1,280
ReLU-294 [-1, 640, 7, 7] 0
Conv2d-295 [-1, 128, 7, 7] 81,920
BatchNorm2d-296 [-1, 128, 7, 7] 256
ReLU-297 [-1, 128, 7, 7] 0
Conv2d-298 [-1, 32, 7, 7] 36,864
BatchNorm2d-299 [-1, 672, 7, 7] 1,344
ReLU-300 [-1, 672, 7, 7] 0
Conv2d-301 [-1, 128, 7, 7] 86,016
BatchNorm2d-302 [-1, 128, 7, 7] 256
ReLU-303 [-1, 128, 7, 7] 0
Conv2d-304 [-1, 32, 7, 7] 36,864
BatchNorm2d-305 [-1, 704, 7, 7] 1,408
ReLU-306 [-1, 704, 7, 7] 0
Conv2d-307 [-1, 128, 7, 7] 90,112
BatchNorm2d-308 [-1, 128, 7, 7] 256
ReLU-309 [-1, 128, 7, 7] 0
Conv2d-310 [-1, 32, 7, 7] 36,864
BatchNorm2d-311 [-1, 736, 7, 7] 1,472
ReLU-312 [-1, 736, 7, 7] 0
Conv2d-313 [-1, 128, 7, 7] 94,208
BatchNorm2d-314 [-1, 128, 7, 7] 256
ReLU-315 [-1, 128, 7, 7] 0
Conv2d-316 [-1, 32, 7, 7] 36,864
BatchNorm2d-317 [-1, 768, 7, 7] 1,536
ReLU-318 [-1, 768, 7, 7] 0
Conv2d-319 [-1, 128, 7, 7] 98,304
BatchNorm2d-320 [-1, 128, 7, 7] 256
ReLU-321 [-1, 128, 7, 7] 0
Conv2d-322 [-1, 32, 7, 7] 36,864
BatchNorm2d-323 [-1, 800, 7, 7] 1,600
ReLU-324 [-1, 800, 7, 7] 0
Conv2d-325 [-1, 128, 7, 7] 102,400
BatchNorm2d-326 [-1, 128, 7, 7] 256
ReLU-327 [-1, 128, 7, 7] 0
Conv2d-328 [-1, 32, 7, 7] 36,864
BatchNorm2d-329 [-1, 832, 7, 7] 1,664
ReLU-330 [-1, 832, 7, 7] 0
Conv2d-331 [-1, 128, 7, 7] 106,496
BatchNorm2d-332 [-1, 128, 7, 7] 256
ReLU-333 [-1, 128, 7, 7] 0
Conv2d-334 [-1, 32, 7, 7] 36,864
BatchNorm2d-335 [-1, 864, 7, 7] 1,728
ReLU-336 [-1, 864, 7, 7] 0
Conv2d-337 [-1, 128, 7, 7] 110,592
BatchNorm2d-338 [-1, 128, 7, 7] 256
ReLU-339 [-1, 128, 7, 7] 0
Conv2d-340 [-1, 32, 7, 7] 36,864
BatchNorm2d-341 [-1, 896, 7, 7] 1,792
ReLU-342 [-1, 896, 7, 7] 0
Conv2d-343 [-1, 128, 7, 7] 114,688
BatchNorm2d-344 [-1, 128, 7, 7] 256
ReLU-345 [-1, 128, 7, 7] 0
Conv2d-346 [-1, 32, 7, 7] 36,864
BatchNorm2d-347 [-1, 928, 7, 7] 1,856
ReLU-348 [-1, 928, 7, 7] 0
Conv2d-349 [-1, 128, 7, 7] 118,784
BatchNorm2d-350 [-1, 128, 7, 7] 256
ReLU-351 [-1, 128, 7, 7] 0
Conv2d-352 [-1, 32, 7, 7] 36,864
BatchNorm2d-353 [-1, 960, 7, 7] 1,920
ReLU-354 [-1, 960, 7, 7] 0
Conv2d-355 [-1, 128, 7, 7] 122,880
BatchNorm2d-356 [-1, 128, 7, 7] 256
ReLU-357 [-1, 128, 7, 7] 0
Conv2d-358 [-1, 32, 7, 7] 36,864
BatchNorm2d-359 [-1, 992, 7, 7] 1,984
ReLU-360 [-1, 992, 7, 7] 0
Conv2d-361 [-1, 128, 7, 7] 126,976
BatchNorm2d-362 [-1, 128, 7, 7] 256
ReLU-363 [-1, 128, 7, 7] 0
Conv2d-364 [-1, 32, 7, 7] 36,864
BatchNorm2d-365 [-1, 1024, 7, 7] 2,048
ReLU-366 [-1, 1024, 7, 7] 0
Linear-367 [-1, 4] 4,100
================================================================
Total params: 6,957,956
Trainable params: 6,957,956
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 294.57
Params size (MB): 26.54
Estimated Total Size (MB): 321.69
----------------------------------------------------------------
DenseNet(
(features): Sequential(
(conv0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(norm0): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(inplace=True)
(pool0): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(denseblock1): DenseBlock(
(denselayer1): DenseLayer(
(norm1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer2): DenseLayer(
(norm1): BatchNorm2d(96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(96, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer3): DenseLayer(
(norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer4): DenseLayer(
(norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer5): DenseLayer(
(norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer6): DenseLayer(
(norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(transition1): Transition(
(norm): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
(denseblock2): DenseBlock(
(denselayer1): DenseLayer(
(norm1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer2): DenseLayer(
(norm1): BatchNorm2d(160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(160, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer3): DenseLayer(
(norm1): BatchNorm2d(192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(192, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer4): DenseLayer(
(norm1): BatchNorm2d(224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(224, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer5): DenseLayer(
(norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer6): DenseLayer(
(norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer7): DenseLayer(
(norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer8): DenseLayer(
(norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer9): DenseLayer(
(norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer10): DenseLayer(
(norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer11): DenseLayer(
(norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer12): DenseLayer(
(norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(transition2): Transition(
(norm): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
(denseblock3): DenseBlock(
(denselayer1): DenseLayer(
(norm1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer2): DenseLayer(
(norm1): BatchNorm2d(288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(288, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer3): DenseLayer(
(norm1): BatchNorm2d(320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(320, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer4): DenseLayer(
(norm1): BatchNorm2d(352, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(352, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer5): DenseLayer(
(norm1): BatchNorm2d(384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(384, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer6): DenseLayer(
(norm1): BatchNorm2d(416, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(416, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer7): DenseLayer(
(norm1): BatchNorm2d(448, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(448, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer8): DenseLayer(
(norm1): BatchNorm2d(480, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(480, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer9): DenseLayer(
(norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer10): DenseLayer(
(norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer11): DenseLayer(
(norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer12): DenseLayer(
(norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer13): DenseLayer(
(norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer14): DenseLayer(
(norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer15): DenseLayer(
(norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer16): DenseLayer(
(norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer17): DenseLayer(
(norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer18): DenseLayer(
(norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer19): DenseLayer(
(norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer20): DenseLayer(
(norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer21): DenseLayer(
(norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer22): DenseLayer(
(norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer23): DenseLayer(
(norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer24): DenseLayer(
(norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(transition3): Transition(
(norm): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(pool): AvgPool2d(kernel_size=2, stride=2, padding=0)
)
(denseblock4): DenseBlock(
(denselayer1): DenseLayer(
(norm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer2): DenseLayer(
(norm1): BatchNorm2d(544, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(544, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer3): DenseLayer(
(norm1): BatchNorm2d(576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(576, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer4): DenseLayer(
(norm1): BatchNorm2d(608, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(608, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer5): DenseLayer(
(norm1): BatchNorm2d(640, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(640, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer6): DenseLayer(
(norm1): BatchNorm2d(672, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(672, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer7): DenseLayer(
(norm1): BatchNorm2d(704, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(704, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer8): DenseLayer(
(norm1): BatchNorm2d(736, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(736, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer9): DenseLayer(
(norm1): BatchNorm2d(768, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(768, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer10): DenseLayer(
(norm1): BatchNorm2d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(800, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer11): DenseLayer(
(norm1): BatchNorm2d(832, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer12): DenseLayer(
(norm1): BatchNorm2d(864, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(864, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer13): DenseLayer(
(norm1): BatchNorm2d(896, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(896, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer14): DenseLayer(
(norm1): BatchNorm2d(928, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(928, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer15): DenseLayer(
(norm1): BatchNorm2d(960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(960, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
(denselayer16): DenseLayer(
(norm1): BatchNorm2d(992, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(inplace=True)
(conv1): Conv2d(992, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(norm2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(inplace=True)
(conv2): Conv2d(128, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
)
)
(norm5): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu5): ReLU(inplace=True)
)
(classifier): Linear(in_features=1024, out_features=4, bias=True)
)
Start training...
[2023-02-23 09:44:09] Epoch: 1, Train_acc:30.1%, Train_loss:1.392, Test_acc:33.6%, Test_loss:1.414, Lr:1.00E-07
acc = 33.6%, saving model to best.pkl
[2023-02-23 09:44:21] Epoch: 2, Train_acc:32.1%, Train_loss:1.389, Test_acc:32.7%, Test_loss:1.395, Lr:1.00E-07
[2023-02-23 09:44:33] Epoch: 3, Train_acc:31.0%, Train_loss:1.389, Test_acc:31.0%, Test_loss:1.405, Lr:1.00E-07
[2023-02-23 09:44:44] Epoch: 4, Train_acc:30.3%, Train_loss:1.390, Test_acc:35.4%, Test_loss:1.393, Lr:1.00E-07
acc = 35.4%, saving model to best.pkl
[2023-02-23 09:44:55] Epoch: 5, Train_acc:30.8%, Train_loss:1.387, Test_acc:34.5%, Test_loss:1.377, Lr:1.00E-07
[2023-02-23 09:45:07] Epoch: 6, Train_acc:31.4%, Train_loss:1.384, Test_acc:36.3%, Test_loss:1.389, Lr:1.00E-07
acc = 36.3%, saving model to best.pkl
[2023-02-23 09:45:18] Epoch: 7, Train_acc:32.1%, Train_loss:1.385, Test_acc:36.3%, Test_loss:1.411, Lr:1.00E-07
[2023-02-23 09:45:30] Epoch: 8, Train_acc:31.6%, Train_loss:1.378, Test_acc:33.6%, Test_loss:1.385, Lr:1.00E-07
[2023-02-23 09:45:41] Epoch: 9, Train_acc:32.5%, Train_loss:1.374, Test_acc:37.2%, Test_loss:1.397, Lr:1.00E-07
acc = 37.2%, saving model to best.pkl
[2023-02-23 09:45:53] Epoch:10, Train_acc:32.7%, Train_loss:1.376, Test_acc:33.6%, Test_loss:1.403, Lr:1.00E-07
Done
EVAL 0.37168, 1.37485
六、DenseNet121网络结构图
