MXNet-图像分类(gluon版本)【附源码】

文章目录

  • 前言
  • 图像的发展史及意义
  • 一、图像分类模型构建
    • 1.LeNet的实现(1994年)
    • 2.AlexNet的实现(2012年)
    • 3.VGG的实现(2014年)
    • 4.GoogleNet的实现(2014年)
    • 5.ResNet的实现(2015年)
    • 6.MobileNet_V1的实现(2017年)
    • 7.MobileNet_V2的实现(2019年)
    • 8.MobileNet_V3的实现(2019年)
    • 9.DenseNet的实现(2017年)
    • 10.ResNest的实现(2020年)
    • 11.ResNext的实现(2016年)
    • 12.SeNet的实现(2017年)
    • 13.Xception的实现
    • 14.InceptionV3的实现
    • 15.SqueezeNet的实现
    • 16.NinNet的实现
    • 17.NasNet的实现
    • 18.HrNet的实现
    • 19.DLA的实现
  • 二、数据集的准备
    • 1.数据集描述
    • 2.数据集准备
  • 三、模型训练/预测
    • 1.基础信息配置
    • 2.数据加载器
    • 3.模型构建
    • 4.模型训练
    • 5.模型预测
  • 四、模型主入口


前言

  本文基于mxnet的gluon实现图像分类,本文基于多个网络结构实现图像分类


图像的发展史及意义

  在神经网络和深度学习领域,Yann LeCun可以说是元老级人物。他于1998年在 IEEE 上发表了一篇42页的长文,文中首次提出卷积-池化-全连接的神经网络结构,由LeCun提出的七层网络命名LeNet5,因而也为他赢得了卷积神经网络之父的美誉。CNN在近几年的发展历程中,从经典的LeNet5网络到最近号称最好的图像分类网络EfficientNet,大量学者不断的做出了努力和创新。
  在后面的一些图像算法过程中,图像分类的网络结构起到了举足轻重的作用,他们都依赖于分类网络做主干特征提取网络,有一定的程度体现出特征提取能力。


一、图像分类模型构建

  本文基于mxnet实现图像分类的网络结构,下边对各个网络结构进行介绍,后续只需要稍作修改即可完成网络结构的替换


1.LeNet的实现(1994年)

论文地址:https://ieeexplore.ieee.org/document/726791
网络结构:
在这里插入图片描述
  LeNet由Yann Lecun 提出,是一种经典的卷积神经网络,是现代卷积神经网络的起源之一。Yann将该网络用于邮局的邮政的邮政编码识别,有着良好的学习和识别能力。LeNet又称LeNet-5,具有一个输入层,两个卷积层,两个池化层,3个全连接层(其中最后一个全连接层为输出层)。
  LeNet-5是一种经典的卷积神经网络结构,于1998年投入实际使用中。该网络最早应用于手写体字符识别应用中。普遍认为,卷积神经网络的出现开始于LeCun 等提出的LeNet 网络,可以说LeCun 等是CNN 的缔造者,而LeNet-5 则是LeCun 等创造的CNN 经典之作。
  LeNet5 一共由7 层组成,分别是C1、C3、C5 卷积层,S2、S4 降采样层(降采样层又称池化层),F6 为一个全连接层,输出是一个高斯连接层,该层使用softmax 函数对输出图像进行分类。为了对应模型输入结构,将MNIST 中的28* 28 的图像扩展为32* 32 像素大小。下面对每一层进行详细介绍。C1 卷积层由6 个大小为5* 5 的不同类型的卷积核组成,卷积核的步长为1,没有零填充,卷积后得到6 个28* 28 像素大小的特征图;S2 为最大池化层,池化区域大小为2* 2,步长为2,经过S2 池化后得到6 个14* 14 像素大小的特征图;C3 卷积层由16 个大小为5* 5 的不同卷积核组成,卷积核的步长为1,没有零填充,卷积后得到16 个10* 10 像素大小的特征图;S4 最大池化层,池化区域大小为2* 2,步长为2,经过S2 池化后得到16 个5* 5 像素大小的特征图;C5 卷积层由120 个大小为5* 5 的不同卷积核组成,卷积核的步长为1,没有零填充,卷积后得到120 个1* 1 像素大小的特征图;将120 个1* 1 像素大小的特征图拼接起来作为F6 的输入,F6 为一个由84 个神经元组成的全连接隐藏层,激活函数使用sigmoid 函数;最后一层输出层是一个由10 个神经元组成的softmax 高斯连接层,可以用来做分类任务。

代码实现(gluon):

class _LeNet(nn.HybridBlock):
    def __init__(self, classes=1000, **kwargs):
        super(_LeNet, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            with self.features.name_scope():
                self.features.add(nn.Conv2D(6, kernel_size=5))
                self.features.add(nn.BatchNorm())
                self.features.add(nn.Activation('sigmoid'))
                self.features.add(nn.MaxPool2D(pool_size=2, strides=2))
                self.features.add(nn.Conv2D(16, kernel_size=5))
                self.features.add(nn.BatchNorm())
                self.features.add(nn.Activation('sigmoid'))
                self.features.add(nn.MaxPool2D(pool_size=2, strides=2))
                self.features.add(nn.Dense(120))
                self.features.add(nn.BatchNorm())
                self.features.add(nn.Activation('sigmoid'))
                self.features.add(nn.Dense(84))
                self.features.add(nn.BatchNorm())
                self.features.add(nn.Activation('sigmoid'))
                self.features.add()
                self.features.add()
                self.features.add()
                self.features.add()
            self.output = nn.Dense(classes)

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x

def get_lenet(class_num):
    return _LeNet(classes=class_num)

2.AlexNet的实现(2012年)

论文地址:https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
网络结构:
在这里插入图片描述
  AlexNet是2012年ImageNet竞赛冠军获得者Hinton和他的学生Alex Krizhevsky设计的。也是在那年之后,更多的更深的神经网络被提出,比如优秀的vgg,GoogLeNet。 这对于传统的机器学习分类算法而言,已经相当的出色。
AlexNet中包含了几个比较新的技术点,也首次在CNN中成功应用了ReLU、Dropout和LRN等Trick。同时AlexNet也使用了GPU进行运算加速。
  AlexNet将LeNet的思想发扬光大,把CNN的基本原理应用到了很深很宽的网络中。AlexNet主要使用到的新技术点如下:

  • 成功使用ReLU作为CNN的激活函数,并验证其效果在较深的网络超过了Sigmoid,成功解决了Sigmoid在网络较深时的梯度弥散问题。虽然ReLU激活函数在很久之前就被提出了,但是直到AlexNet的出现才将其发扬光大。
  • 训练时使用Dropout随机忽略一部分神经元,以避免模型过拟合。Dropout虽有单独的论文论述,但是AlexNet将其实用化,通过实践证实了它的效果。在AlexNet中主要是最后几个全连接层使用了Dropout。
  • 在CNN中使用重叠的最大池化。此前CNN中普遍使用平均池化,AlexNet全部使用最大池化,避免平均池化的模糊化效果。并且AlexNet中提出让步长比池化核的尺寸小,这样池化层的输出之间会有重叠和覆盖,提升了特征的丰富性。
  • 提出了LRN层,对局部神经元的活动创建竞争机制,使得其中响应比较大的值变得相对更大,并抑制其他反馈较小的神经元,增强了模型的泛化能力。
  • 使用CUDA加速深度卷积网络的训练,利用GPU强大的并行计算能力,处理神经网络训练时大量的矩阵运算。AlexNet使用了两块GTX 580 GPU进行训练,单个GTX 580只有3GB显存,这限制了可训练的网络的最大规模。因此作者将AlexNet分布在两个GPU上,在每个GPU的显存中储存一半的神经元的参数。因为GPU之间通信方便,可以互相访问显存,而不需要通过主机内存,所以同时使用多块GPU也是非常高效的。同时,AlexNet的设计让GPU之间的通信只在网络的某些层进行,控制了通信的性能损耗。
  • 数据增强,随机地从256256的原始图像中截取224224大小的区域(以及水平翻转的镜像),相当于增加了2*(256-224)^2=2048倍的数据量。如果没有数据增强,仅靠原始的数据量,参数众多的CNN会陷入过拟合中,使用了数据增强后可以大大减轻过拟合,提升泛化能力。进行预测时,则是取图片的四个角加中间共5个位置,并进行左右翻转,一共获得10张图片,对他们进行预测并对10次结果求均值。同时,AlexNet论文中提到了会对图像的RGB数据进行PCA处理,并对主成分做一个标准差为0.1的高斯扰动,增加一些噪声,这个Trick可以让错误率再下降1%。

代码实现(gluon):

class _AlexNet(nn.HybridBlock):
    def __init__(self, classes=1000,alpha=4, **kwargs):
        super(AlexNet, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            with self.features.name_scope():
                self.features.add(nn.Conv2D(96//alpha, kernel_size=11, strides=4, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
                self.features.add(nn.Conv2D(256//alpha, kernel_size=5, padding=2, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
                self.features.add(nn.Conv2D(384//alpha, kernel_size=3, padding=1, activation='relu'))
                self.features.add(nn.Conv2D(384//alpha, kernel_size=3, padding=1, activation='relu'))
                self.features.add(nn.Conv2D(256//alpha, kernel_size=3, padding=1, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
                self.features.add(nn.Flatten())
                self.features.add(nn.Dense(4096//alpha, activation='relu'))
                self.features.add(nn.Dropout(0.5))
                self.features.add(nn.Dense(4096//alpha, activation='relu'))
                self.features.add(nn.Dropout(0.5))
            self.output = nn.Dense(classes)

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_alexnet(class_num):
    net = _AlexNet(classes=class_num)
    return net

3.VGG的实现(2014年)

论文地址:https://arxiv.org/pdf/1409.1556.pdf
网络结构:
在这里插入图片描述
在这里插入图片描述
  VGG模型是2014年ILSVRC竞赛的第二名,第一名是GoogLeNet。但是VGG模型在多个迁移学习任务中的表现要优于googLeNet。而且,从图像中提取CNN特征,VGG模型是首选算法。它的缺点在于,参数量有140M之多,需要更大的存储空间。但是这个模型很有研究价值。
  模型的名称——“VGG”代表了牛津大学的Oxford Visual Geometry Group,该小组隶属于1985年成立的Robotics Research Group,该Group研究范围包括了机器学习到移动机器人。下面是一段来自网络对同年GoogLeNet和VGG的描述:
VGG的特点:

  • 小卷积核。作者将卷积核全部替换为3x3(极少用了1x1);
  • 小池化核。相比AlexNet的3x3的池化核,VGG全部为2x2的池化核;
  • 层数更深特征图更宽。基于前两点外,由于卷积核专注于扩大通道数、池化专注于缩小宽和高,使得模型架构上更深更宽的同时,计算量的增加放缓;
  • 全连接转卷积。网络测试阶段将训练阶段的三个全连接替换为三个卷积,测试重用训练时的参数,使得测试得到的全卷积网络因为没有全连接的限制,因而可以接收任意宽或高为的输入。

代码实现(gluon):

class _VGG(nn.HybridBlock):
    def __init__(self, layers, filters, classes=1000, batch_norm=False, **kwargs):
        super(_VGG, self).__init__(**kwargs)
        assert len(layers) == len(filters)
        with self.name_scope():
            self.features = self._make_features(layers, filters, batch_norm)
            self.features.add(nn.Dense(4096, activation='relu', weight_initializer='normal', bias_initializer='zeros'))
            self.features.add(nn.Dropout(rate=0.5))
            self.features.add(nn.Dense(4096, activation='relu', weight_initializer='normal', bias_initializer='zeros'))
            self.features.add(nn.Dropout(rate=0.5))
            self.output = nn.Dense(classes, weight_initializer='normal', bias_initializer='zeros')

    def _make_features(self, layers, filters, batch_norm):
        featurizer = nn.HybridSequential(prefix='')
        for i, num in enumerate(layers):
            for _ in range(num):
                featurizer.add(nn.Conv2D(filters[i], kernel_size=3, padding=1, weight_initializer=Xavier(rnd_type='gaussian', factor_type='out', magnitude=2), bias_initializer='zeros'))
                if batch_norm:
                    featurizer.add(nn.BatchNorm())
                featurizer.add(nn.Activation('relu'))
            featurizer.add(nn.MaxPool2D(strides=2))
        return featurizer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
_vgg_spec = {11: ([1, 1, 2, 2, 2], [64, 128, 256, 512, 512]),
            13: ([2, 2, 2, 2, 2], [64, 128, 256, 512, 512]),
            16: ([2, 2, 3, 3, 3], [64, 128, 256, 512, 512]),
            19: ([2, 2, 4, 4, 4], [64, 128, 256, 512, 512])}
def get_vgg(num_layers,class_num,bn = False, **kwargs):
    if bn:
        kwargs['batch_norm'] = True
    layers, filters = _vgg_spec[num_layers]
    net = _VGG(layers, filters,classes = class_num, **kwargs)
    return net

4.GoogleNet的实现(2014年)

论文地址:https://arxiv.org/pdf/1409.4842.pdf
网络结构:
在这里插入图片描述
  GoogLeNet是2014年Christian Szegedy提出的一种全新的深度学习结构,在这之前的AlexNet、VGG等结构都是通过增大网络的深度(层数)来获得更好的训练效果,但层数的增加会带来很多负作用,比如overfit、梯度消失、梯度爆炸等。inception的提出则从另一种角度来提升训练结果:能更高效的利用计算资源,在相同的计算量下能提取到更多的特征,从而提升训练结果。
  inception模块的基本机构如图1,整个inception结构就是由多个这样的inception模块串联起来的。inception结构的主要贡献有两个:一是使用1x1的卷积来进行升降维;二是在多个尺寸上同时进行卷积再聚合。在这里插入图片描述
1x1卷积
作用1:在相同尺寸的感受野中叠加更多的卷积,能提取到更丰富的特征。这个观点来自于Network in Network,图1里三个1x1卷积都起到了该作用。
在这里插入图片描述
  图2左侧是是传统的卷积层结构(线性卷积),在一个尺度上只有一次卷积;图2右图是Network in Network结构(NIN结构),先进行一次普通的卷积(比如3x3),紧跟再进行一次1x1的卷积,对于某个像素点来说1x1卷积等效于该像素点在所有特征上进行一次全连接的计算,所以图2右侧图的1x1卷积画成了全连接层的形式,需要注意的是NIN结构中无论是第一个3x3卷积还是新增的1x1卷积,后面都紧跟着激活函数(比如relu)。将两个卷积串联,就能组合出更多的非线性特征。举个例子,假设第1个3x3卷积+激活函数近似于f1(x)=ax2+bx+c,第二个1x1卷积+激活函数近似于f2(x)=mx2+nx+q,那f1(x)和f2(f1(x))比哪个非线性更强,更能模拟非线性的特征?答案是显而易见的。NIN的结构和传统的神经网络中多层的结构有些类似,后者的多层是跨越了不同尺寸的感受野(通过层与层中间加pool层),从而在更高尺度上提取出特征;NIN结构是在同一个尺度上的多层(中间没有pool层),从而在相同的感受野范围能提取更强的非线性。
  作用2:使用1x1卷积进行降维,降低了计算复杂度。图2中间3x3卷积和5x5卷积前的1x1卷积都起到了这个作用。当某个卷积层输入的特征数较多,对这个输入进行卷积运算将产生巨大的计算量;如果对输入先进行降维,减少特征数后再做卷积计算量就会显著减少。图3是优化前后两种方案的乘法次数比较,同样是输入一组有192个特征、32x32大小,输出256组特征的数据,图3第一张图直接用3x3卷积实现,需要192x256x3x3x32x32=452984832次乘法;图3第二张图先用1x1的卷积降到96个特征,再用3x3卷积恢复出256组特征,需要192x96x1x1x32x32+96x256x3x3x32x32=245366784次乘法,使用1x1卷积降维的方法节省了一半的计算量。有人会问,用1x1卷积降到96个特征后特征数不就减少了么,会影响最后训练的效果么?答案是否定的,只要最后输出的特征数不变(256组),中间的降维类似于压缩的效果,并不影响最终训练的结果。
在这里插入图片描述
  多个尺寸上进行卷积再聚合
  图2可以看到对输入做了4个分支,分别用不同尺寸的filter进行卷积或池化,最后再在特征维度上拼接到一起。这种全新的结构有什么好处呢?Szegedy从多个角度进行了解释:
  解释1:在直观感觉上在多个尺度上同时进行卷积,能提取到不同尺度的特征。特征更为丰富也意味着最后分类判断时更加准确。
  解释2:利用稀疏矩阵分解成密集矩阵计算的原理来加快收敛速度。举个例子图4左侧是个稀疏矩阵(很多元素都为0,不均匀分布在矩阵中),和一个2x2的矩阵进行卷积,需要对稀疏矩阵中的每一个元素进行计算;如果像图4右图那样把稀疏矩阵分解成2个子密集矩阵,再和2x2矩阵进行卷积,稀疏矩阵中0较多的区域就可以不用计算,计算量就大大降低。这个原理应用到inception上就是要在特征维度上进行分解!传统的卷积层的输入数据只和一种尺度(比如3x3)的卷积核进行卷积,输出固定维度(比如256个特征)的数据,所有256个输出特征基本上是均匀分布在3x3尺度范围上,这可以理解成输出了一个稀疏分布的特征集;而inception模块在多个尺度上提取特征(比如1x1,3x3,5x5),输出的256个特征就不再是均匀分布,而是相关性强的特征聚集在一起(比如1x1的的96个特征聚集在一起,3x3的96个特征聚集在一起,5x5的64个特征聚集在一起),这可以理解成多个密集分布的子特征集。这样的特征集中因为相关性较强的特征聚集在了一起,不相关的非关键特征就被弱化,同样是输出256个特征,inception方法输出的特征“冗余”的信息较少。用这样的“纯”的特征集层层传递最后作为反向计算的输入,自然收敛的速度更快。
在这里插入图片描述
  解释3:Hebbin赫布原理。Hebbin原理是神经科学上的一个理论,解释了在学习的过程中脑中的神经元所发生的变化,用一句话概括就是fire togethter, wire together。赫布认为“两个神经元或者神经元系统,如果总是同时兴奋,就会形成一种‘组合’,其中一个神经元的兴奋会促进另一个的兴奋”。比如狗看到肉会流口水,反复刺激后,脑中识别肉的神经元会和掌管唾液分泌的神经元会相互促进,“缠绕”在一起,以后再看到肉就会更快流出口水。用在inception结构中就是要把相关性强的特征汇聚到一起。这有点类似上面的解释2,把1x1,3x3,5x5的特征分开。因为训练收敛的最终目的就是要提取出独立的特征,所以预先把相关性强的特征汇聚,就能起到加速收敛的作用。
  在inception模块中有一个分支使用了max pooling,作者认为pooling也能起到提取特征的作用,所以也加入模块中。注意这个pooling的stride=1,pooling后没有减少数据的尺寸。

代码实现(gluon):

def _make_basic_conv(in_channels, channels, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    out = nn.HybridSequential(prefix='')
    out.add(nn.Conv2D(in_channels=in_channels, channels=channels, use_bias=False, **kwargs))
    out.add(norm_layer(in_channels=channels, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
    out.add(nn.Activation('relu'))
    return out
def _make_branch(use_pool, norm_layer, norm_kwargs, *conv_settings):
    out = nn.HybridSequential(prefix='')
    if use_pool == 'avg':
        out.add(nn.AvgPool2D(pool_size=3, strides=1, padding=1))
    elif use_pool == 'max':
        out.add(nn.MaxPool2D(pool_size=3, strides=1, padding=1))
    setting_names = ['in_channels', 'channels', 'kernel_size', 'strides', 'padding']
    for setting in conv_settings:
        kwargs = {}
        for i, value in enumerate(setting):
            if value is not None:
                if setting_names[i] == 'in_channels':
                    in_channels = value
                elif setting_names[i] == 'channels':
                    channels = value
                else:
                    kwargs[setting_names[i]] = value
        out.add(_make_basic_conv(in_channels, channels, norm_layer, norm_kwargs, **kwargs))
    return out
def _make_Mixed_3a(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 64, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 96, 1, None, None), (96, 128, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 16, 1, None, None), (16, 32, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_3b(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 128, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 128, 1, None, None), (128, 192, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 32, 1, None, None), (32, 96, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_4a(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 192, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 96, 1, None, None), (96, 208, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 16, 1, None, None), (16, 48, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_4b(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 160, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 112, 1, None, None), (112, 224, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 24, 1, None, None), (24, 64, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_4c(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 128, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 128, 1, None, None), (128, 256, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 24, 1, None, None), (24, 64, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_4d(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 112, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 144, 1, None, None), (144, 288, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 32, 1, None, None), (32, 64, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_4e(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 256, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 160, 1, None, None), (160, 320, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 32, 1, None, None), (32, 128, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_5a(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 256, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 160, 1, None, None), (160, 320, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 32, 1, None, None), (32, 128, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_Mixed_5b(in_channels, pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 384, 1, None, None)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 192, 1, None, None), (192, 384, 3, None, 1)))
        out.add(_make_branch(None, norm_layer, norm_kwargs, (in_channels, 48, 1, None, None), (48, 128, 3, None, 1)))
        out.add(_make_branch('max', norm_layer, norm_kwargs, (in_channels, pool_features, 1, None, None)))
    return out
def _make_aux(in_channels, classes, norm_layer, norm_kwargs):
    out = nn.HybridSequential(prefix='')
    out.add(nn.AvgPool2D(pool_size=5, strides=3))
    out.add(_make_basic_conv(in_channels=in_channels, channels=128, kernel_size=1, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

    out.add(nn.Flatten())
    out.add(nn.Dense(units=1024, in_units=2048))
    out.add(nn.Activation('relu'))
    out.add(nn.Dropout(0.7))
    out.add(nn.Dense(units=classes, in_units=1024))
    return out
class _GoogLeNet(nn.HybridBlock):
    def __init__(self, classes=1000, norm_layer=nn.BatchNorm, dropout_ratio=0.4, aux_logits=False,norm_kwargs=None, partial_bn=False, **kwargs):
        super(_GoogLeNet, self).__init__(**kwargs)
        self.dropout_ratio = dropout_ratio
        self.aux_logits = aux_logits

        with self.name_scope():
            self.conv1 = _make_basic_conv(in_channels=3, channels=64, kernel_size=7,
                                          strides=2, padding=3,
                                          norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.maxpool1 = nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True)

            if partial_bn:
                if norm_kwargs is not None:
                    norm_kwargs['use_global_stats'] = True
                else:
                    norm_kwargs = {}
                    norm_kwargs['use_global_stats'] = True

            self.conv2 = _make_basic_conv(in_channels=64, channels=64, kernel_size=1,
                                          norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.conv3 = _make_basic_conv(in_channels=64, channels=192,
                                          kernel_size=3, padding=1,
                                          norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.maxpool2 = nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True)

            self.inception3a = _make_Mixed_3a(192, 32, 'Mixed_3a_', norm_layer, norm_kwargs)
            self.inception3b = _make_Mixed_3b(256, 64, 'Mixed_3b_', norm_layer, norm_kwargs)
            self.maxpool3 = nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True)

            self.inception4a = _make_Mixed_4a(480, 64, 'Mixed_4a_', norm_layer, norm_kwargs)
            self.inception4b = _make_Mixed_4b(512, 64, 'Mixed_4b_', norm_layer, norm_kwargs)
            self.inception4c = _make_Mixed_4c(512, 64, 'Mixed_4c_', norm_layer, norm_kwargs)
            self.inception4d = _make_Mixed_4d(512, 64, 'Mixed_4d_', norm_layer, norm_kwargs)
            self.inception4e = _make_Mixed_4e(528, 128, 'Mixed_4e_', norm_layer, norm_kwargs)
            self.maxpool4 = nn.MaxPool2D(pool_size=2, strides=2)

            self.inception5a = _make_Mixed_5a(832, 128, 'Mixed_5a_', norm_layer, norm_kwargs)
            self.inception5b = _make_Mixed_5b(832, 128, 'Mixed_5b_', norm_layer, norm_kwargs)

            if self.aux_logits:
                self.aux1 = _make_aux(512, classes, norm_layer, norm_kwargs)
                self.aux2 = _make_aux(528, classes, norm_layer, norm_kwargs)

            self.head = nn.HybridSequential(prefix='')
            self.avgpool = nn.AvgPool2D(pool_size=7)
            self.dropout = nn.Dropout(self.dropout_ratio)
            self.output = nn.Dense(units=classes, in_units=1024)
            self.head.add(self.avgpool)
            self.head.add(self.dropout)
            self.head.add(self.output)

    def hybrid_forward(self, F, x):
        x = self.conv1(x)
        x = self.maxpool1(x)

        x = self.conv2(x)
        x = self.conv3(x)
        x = self.maxpool2(x)

        x = self.inception3a(x)
        x = self.inception3b(x)
        x = self.maxpool3(x)

        x = self.inception4a(x)
        if self.aux_logits:
            aux1 = self.aux1(x)
        x = self.inception4b(x)
        x = self.inception4c(x)
        x = self.inception4d(x)
        if self.aux_logits:
            aux2 = self.aux2(x)
        x = self.inception4e(x)
        x = self.maxpool4(x)

        x = self.inception5a(x)
        x = self.inception5b(x)
        x = self.head(x)

        if self.aux_logits:
            return (x, aux2, aux1)
        return x
def get_googlenet(class_num=1000, dropout_ratio=0.4, aux_logits=False,partial_bn=False, **kwargs):
    net = _GoogLeNet(classes=class_num, partial_bn=partial_bn,  dropout_ratio=dropout_ratio, aux_logits=aux_logits, **kwargs)
    return net

5.ResNet的实现(2015年)

论文地址:https://arxiv.org/pdf/1512.03385.pdf
网络结构(VGG vs ResNet):
在这里插入图片描述
在这里插入图片描述
  ResNet的发明者是何恺明(Kaiming He)、张翔宇(Xiangyu Zhang)、任少卿(Shaoqing Ren)和孙剑(Jiangxi Sun)。
  在2015年的ImageNet大规模视觉识别竞赛(ImageNet Large Scale Visual Recognition Challenge, ILSVRC)中获得了图像分类和物体识别的优胜。 残差网络的特点是容易优化,并且能够通过增加相当的深度来提高准确率。其内部的残差块使用了跳跃连接,缓解了在深度神经网络中增加深度带来的梯度消失问题。

代码实现(gluon):

def _conv3x3(channels, stride, in_channels):
    return nn.Conv2D(channels, kernel_size=3, strides=stride, padding=1, use_bias=False, in_channels=in_channels)
class _BasicBlockV1(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_BasicBlockV1, self).__init__(**kwargs)
        self.body = nn.HybridSequential(prefix='')
        self.body.add(_conv3x3(channels, stride, in_channels))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(_conv3x3(channels, 1, channels))
        if not last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            self.downsample.add(nn.Conv2D(channels, kernel_size=1, strides=stride, use_bias=False, in_channels=in_channels))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x
        x = self.body(x)
        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(residual+x, act_type='relu')

        return x
class _BottleneckV1(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_BottleneckV1, self).__init__(**kwargs)
        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Conv2D(channels//4, kernel_size=1, strides=stride))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(_conv3x3(channels//4, 1, channels//4))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(channels, kernel_size=1, strides=1))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if not last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            self.downsample.add(nn.Conv2D(channels, kernel_size=1, strides=stride, use_bias=False, in_channels=in_channels))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x

        x = self.body(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(x + residual, act_type='relu')
        return x
class _BasicBlockV2(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_BasicBlockV2, self).__init__(**kwargs)
        self.bn1 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        self.conv1 = _conv3x3(channels, stride, in_channels)
        if not last_gamma:
            self.bn2 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        else:
            self.bn2 = norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs))
        self.conv2 = _conv3x3(channels, 1, channels)

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if downsample:
            self.downsample = nn.Conv2D(channels, 1, stride, use_bias=False, in_channels=in_channels)
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x
        x = self.bn1(x)
        x = F.Activation(x, act_type='relu')
        if self.downsample:
            residual = self.downsample(x)
        x = self.conv1(x)

        x = self.bn2(x)
        x = F.Activation(x, act_type='relu')
        x = self.conv2(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        return x + residual
class _BottleneckV2(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_BottleneckV2, self).__init__(**kwargs)
        self.bn1 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        self.conv1 = nn.Conv2D(channels//4, kernel_size=1, strides=1, use_bias=False)
        self.bn2 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        self.conv2 = _conv3x3(channels//4, stride, channels//4)
        if not last_gamma:
            self.bn3 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        else:
            self.bn3 = norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs))
        self.conv3 = nn.Conv2D(channels, kernel_size=1, strides=1, use_bias=False)

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if downsample:
            self.downsample = nn.Conv2D(channels, 1, stride, use_bias=False, in_channels=in_channels)
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x
        x = self.bn1(x)
        x = F.Activation(x, act_type='relu')
        if self.downsample:
            residual = self.downsample(x)
        x = self.conv1(x)

        x = self.bn2(x)
        x = F.Activation(x, act_type='relu')
        x = self.conv2(x)

        x = self.bn3(x)
        x = F.Activation(x, act_type='relu')
        x = self.conv3(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        return x + residual
class _ResNetV1(nn.HybridBlock):
    def __init__(self, block, layers, channels, classes=1000, thumbnail=False, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_ResNetV1, self).__init__(**kwargs)
        assert len(layers) == len(channels) - 1
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            if thumbnail:
                self.features.add(_conv3x3(channels[0], 1, 0))
            else:
                self.features.add(nn.Conv2D(channels[0], 7, 2, 3, use_bias=False))
                self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.MaxPool2D(3, 2, 1))

            for i, num_layer in enumerate(layers):
                stride = 1 if i == 0 else 2
                self.features.add(self._make_layer(block, num_layer, channels[i+1], stride, i+1, in_channels=channels[i], last_gamma=last_gamma, use_se=use_se, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.features.add(nn.GlobalAvgPool2D())

            self.output = nn.Dense(classes, in_units=channels[-1])

    def _make_layer(self, block, layers, channels, stride, stage_index, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
        layer = nn.HybridSequential(prefix='stage%d_'%stage_index)
        with layer.name_scope():
            layer.add(block(channels, stride, channels != in_channels, in_channels=in_channels, last_gamma=last_gamma, use_se=use_se, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            for _ in range(layers-1):
                layer.add(block(channels, 1, False, in_channels=channels, last_gamma=last_gamma, use_se=use_se, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        return layer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)

        return x
class _ResNetV2(nn.HybridBlock):
    def __init__(self, block, layers, channels, classes=1000, thumbnail=False, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_ResNetV2, self).__init__(**kwargs)
        assert len(layers) == len(channels) - 1
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            self.features.add(norm_layer(scale=False, center=False, **({} if norm_kwargs is None else norm_kwargs)))
            if thumbnail:
                self.features.add(_conv3x3(channels[0], 1, 0))
            else:
                self.features.add(nn.Conv2D(channels[0], 7, 2, 3, use_bias=False))
                self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.MaxPool2D(3, 2, 1))

            in_channels = channels[0]
            for i, num_layer in enumerate(layers):
                stride = 1 if i == 0 else 2
                self.features.add(self._make_layer(block, num_layer, channels[i+1], stride, i+1, in_channels=in_channels, last_gamma=last_gamma, use_se=use_se, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
                in_channels = channels[i+1]
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.GlobalAvgPool2D())
            self.features.add(nn.Flatten())

            self.output = nn.Dense(classes, in_units=in_channels)

    def _make_layer(self, block, layers, channels, stride, stage_index, in_channels=0,
                    last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
        layer = nn.HybridSequential(prefix='stage%d_'%stage_index)
        with layer.name_scope():
            layer.add(block(channels, stride, channels != in_channels, in_channels=in_channels, last_gamma=last_gamma, use_se=use_se, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            for _ in range(layers-1):
                layer.add(block(channels, 1, False, in_channels=channels, last_gamma=last_gamma, use_se=use_se, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        return layer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
_resnet_spec = {18: ('basic_block', [2, 2, 2, 2], [64, 64, 128, 256, 512]),
               34: ('basic_block', [3, 4, 6, 3], [64, 64, 128, 256, 512]),
               50: ('bottle_neck', [3, 4, 6, 3], [64, 256, 512, 1024, 2048]),
               101: ('bottle_neck', [3, 4, 23, 3], [64, 256, 512, 1024, 2048]),
               152: ('bottle_neck', [3, 8, 36, 3], [64, 256, 512, 1024, 2048])}
_resnet_net_versions = [_ResNetV1, _ResNetV2]
_resnet_block_versions = [{'basic_block': _BasicBlockV1, 'bottle_neck': _BottleneckV1}, {'basic_block': _BasicBlockV2, 'bottle_neck': _BottleneckV2}]

def get_resnet(version, num_layers,class_num,use_se=False):
    block_type, layers, channels = _resnet_spec[num_layers]
    resnet_class = _resnet_net_versions[version-1]
    block_class = _resnet_block_versions[version-1][block_type]
    net = resnet_class(block_class, layers, channels,use_se=use_se, classes=class_num)
    return net

6.MobileNet_V1的实现(2017年)

论文地址:https://arxiv.org/pdf/1704.04861.pdf
网络结构:
在这里插入图片描述
  MobileNet V1是由google2016年提出,2017年发布的文章。其主要创新点在于深度可分离卷积,而整个网络实际上也是深度可分离模块的堆叠。
  深度可分离卷积被证明是轻量化网络的有效设计,深度可分离卷积由逐深度卷积(Depthwise)和逐点卷积(Pointwise)构成。
  对比于标准卷积,逐深度卷积将卷积核拆分成为单通道形式,在不改变输入特征图像的深度的情况下,对每一通道进行卷积操作,这样就得到了和输入特征图通道数一致的输出特征图。
  逐点卷积就是1×1卷积。主要作用就是对特征图进行升维和降维。

代码实现(gluon):

class _ReLU6(nn.HybridBlock):
    def __init__(self, **kwargs):
        super(_ReLU6, self).__init__(**kwargs)

    def hybrid_forward(self, F, x):
        return F.clip(x, 0, 6, name="relu6")
def _add_conv(out, channels=1, kernel=1, stride=1, pad=0, num_group=1, active=True, relu6=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
    out.add(nn.Conv2D(channels, kernel, stride, pad, groups=num_group, use_bias=False))
    out.add(norm_layer(scale=True, **({} if norm_kwargs is None else norm_kwargs)))
    if active:
        out.add(_ReLU6() if relu6 else nn.Activation('relu'))
def _add_conv_dw(out, dw_channels, channels, stride, relu6=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
    _add_conv(out, channels=dw_channels, kernel=3, stride=stride,
              pad=1, num_group=dw_channels, relu6=relu6,
              norm_layer=norm_layer, norm_kwargs=norm_kwargs)
    _add_conv(out, channels=channels, relu6=relu6,
              norm_layer=norm_layer, norm_kwargs=norm_kwargs)
class _LinearBottleneck(nn.HybridBlock):
    def __init__(self, in_channels, channels, t, stride, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_LinearBottleneck, self).__init__(**kwargs)
        self.use_shortcut = stride == 1 and in_channels == channels
        with self.name_scope():
            self.out = nn.HybridSequential()

            if t != 1:
                _add_conv(self.out, in_channels * t, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            _add_conv(self.out, in_channels * t, kernel=3, stride=stride, pad=1, num_group=in_channels * t, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            _add_conv(self.out, channels, active=False, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

    def hybrid_forward(self, F, x):
        out = self.out(x)
        if self.use_shortcut:
            out = F.elemwise_add(out, x)
        return out
class _MobileNetV1(nn.HybridBlock):
    def __init__(self, multiplier=1.0, classes=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_MobileNetV1, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            with self.features.name_scope():
                _add_conv(self.features, channels=int(32 * multiplier), kernel=3, pad=1, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
                dw_channels = [int(x * multiplier) for x in [32, 64] + [128] * 2 + [256] * 2 + [512] * 6 + [1024]]
                channels = [int(x * multiplier) for x in [64] + [128] * 2 + [256] * 2 + [512] * 6 + [1024] * 2]
                strides = [1, 2] * 3 + [1] * 5 + [2, 1]
                for dwc, c, s in zip(dw_channels, channels, strides):
                    _add_conv_dw(self.features, dw_channels=dwc, channels=c, stride=s, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
                self.features.add(nn.GlobalAvgPool2D())
                self.features.add(nn.Flatten())

            self.output = nn.Dense(classes)

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_mobilenet_v1(multiplier,class_num, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    net = _MobileNetV1(multiplier,classes = class_num, norm_layer=norm_layer, norm_kwargs=norm_kwargs, **kwargs)
    return net

7.MobileNet_V2的实现(2019年)

论文地址:https://arxiv.org/pdf/1801.04381.pdf
网络结构:
在这里插入图片描述
在这里插入图片描述

mobilenetV1的缺点:

  • V1结构过于简单,没有复用图像特征,即没有concat/eltwise+ 等操作进行特征融合,而后续的一系列的ResNet, DenseNet等结构已经证明复用图像特征的有效性。
  • 在处理低维数据(比如逐深度的卷积)时,relu函数会造成信息的丢失。
  • DW 卷积由于本身的计算特性决定它自己没有改变通道数的能力,上一层给它多少通道,它就只能输出多少通道。所以如果上一层给的通道数本身很少的话,DW 也只能很委屈的在低维空间提特征,因此效果不够好。

V2使用了跟V1类似的深度可分离结构,不同之处也正对应着V1中逐深度卷积的缺点改进:

  • V2 去掉了第二个 PW 的激活函数改为线性激活。
  • V2 在 DW 卷积之前新加了一个 PW 卷积。

代码实现(gluon):

class _ReLU6(nn.HybridBlock):
    def __init__(self, **kwargs):
        super(_ReLU6, self).__init__(**kwargs)

    def hybrid_forward(self, F, x):
        return F.clip(x, 0, 6, name="relu6")
def _add_conv(out, channels=1, kernel=1, stride=1, pad=0, num_group=1, active=True, relu6=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
    out.add(nn.Conv2D(channels, kernel, stride, pad, groups=num_group, use_bias=False))
    out.add(norm_layer(scale=True, **({} if norm_kwargs is None else norm_kwargs)))
    if active:
        out.add(_ReLU6() if relu6 else nn.Activation('relu'))
def _add_conv_dw(out, dw_channels, channels, stride, relu6=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
    _add_conv(out, channels=dw_channels, kernel=3, stride=stride,
              pad=1, num_group=dw_channels, relu6=relu6,
              norm_layer=norm_layer, norm_kwargs=norm_kwargs)
    _add_conv(out, channels=channels, relu6=relu6,
              norm_layer=norm_layer, norm_kwargs=norm_kwargs)
class _LinearBottleneck(nn.HybridBlock):
    def __init__(self, in_channels, channels, t, stride, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_LinearBottleneck, self).__init__(**kwargs)
        self.use_shortcut = stride == 1 and in_channels == channels
        with self.name_scope():
            self.out = nn.HybridSequential()

            if t != 1:
                _add_conv(self.out, in_channels * t, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            _add_conv(self.out, in_channels * t, kernel=3, stride=stride, pad=1, num_group=in_channels * t, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            _add_conv(self.out, channels, active=False, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

    def hybrid_forward(self, F, x):
        out = self.out(x)
        if self.use_shortcut:
            out = F.elemwise_add(out, x)
        return out
class _MobileNetV2(nn.HybridBlock):
    def __init__(self, multiplier=1.0, classes=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_MobileNetV2, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='features_')
            with self.features.name_scope():
                _add_conv(self.features, int(32 * multiplier), kernel=3, stride=2, pad=1, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

                in_channels_group = [int(x * multiplier) for x in [32] + [16] + [24] * 2 + [32] * 3 + [64] * 4 + [96] * 3 + [160] * 3]
                channels_group = [int(x * multiplier) for x in [16] + [24] * 2 + [32] * 3 + [64] * 4 + [96] * 3 + [160] * 3 + [320]]
                ts = [1] + [6] * 16
                strides = [1, 2] * 2 + [1, 1, 2] + [1] * 6 + [2] + [1] * 3

                for in_c, c, t, s in zip(in_channels_group, channels_group, ts, strides):
                    self.features.add(_LinearBottleneck(in_channels=in_c, channels=c, t=t, stride=s, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

                last_channels = int(1280 * multiplier) if multiplier > 1.0 else 1280
                _add_conv(self.features, last_channels, relu6=True, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

                self.features.add(nn.GlobalAvgPool2D())

            self.output = nn.HybridSequential(prefix='output_')
            with self.output.name_scope():
                self.output.add(nn.Conv2D(classes, 1, use_bias=False, prefix='pred_'), nn.Flatten())

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_mobilenet_v2(multiplier,class_num, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    net = _MobileNetV2(multiplier,classes = class_num, norm_layer=norm_layer, norm_kwargs=norm_kwargs, **kwargs)
    return net

8.MobileNet_V3的实现(2019年)

论文地址:https://arxiv.org/pdf/1905.02244.pdf
网络结构:
MXNet-图像分类(gluon版本)【附源码】_第1张图片
MXNet-图像分类(gluon版本)【附源码】_第2张图片
根据MobileNetV3论文总结,网络存在以下3点需要大家注意的:

  • 更新了Block(bneck),在v3版本中原论文称之为bneck,在v2版倒残差结构上进行了简单的改动。
  • 使用了NAS(Neural Architecture Search)搜索参数
  • 重新设计了耗时结构:作者使用NAS搜索之后得到的网络,接下来对网络每一层的推理时间进行分析,针对某些耗时的层结构做了进一步的优化

  可以看出V3版本的Large 1.0(V3-Large 1.0)的Top-1是75.2,对于V2 1.0 它的Top-1是72,相当于提升了3.2%
  在推理速度方面也有一定的提升,(V3-Large 1.0)在P-1手机上推理时间为51ms,而V2是64ms,很明显V3比V2,不仅准确率更高了,而且速度更快了
  V3-Small版本的Top-1是 67.4,而V2 0.35 (0.35表示卷积核的倍率因子)的Top-1只有60.8,准确率提升了6.6%

代码实现(gluon):


class _ReLU6(nn.HybridBlock):
    def __init__(self, **kwargs):
        super(_ReLU6, self).__init__(**kwargs)

    def hybrid_forward(self, F, x):
        return F.clip(x, 0, 6, name="relu6")
class _HardSigmoid(nn.HybridBlock):
    def __init__(self, **kwargs):
        super(_HardSigmoid, self).__init__(**kwargs)
        self.act = _ReLU6()
    def hybrid_forward(self, F, x):
        return self.act(x + 3.) / 6.
class _HardSwish(nn.HybridBlock):
    def __init__(self, **kwargs):
        super(_HardSwish, self).__init__(**kwargs)
        self.act = _HardSigmoid()

    def hybrid_forward(self, F, x):
        return x * self.act(x)
def _make_divisible(x, divisible_by=8):
    return int(np.ceil(x * 1. / divisible_by) * divisible_by)
class _Activation_mobilenetv3(nn.HybridBlock):
    def __init__(self, act_func, **kwargs):
        super(_Activation_mobilenetv3, self).__init__(**kwargs)
        if act_func == "relu":
            self.act = nn.Activation('relu')
        elif act_func == "relu6":
            self.act = _ReLU6()
        elif act_func == "hard_sigmoid":
            self.act = _HardSigmoid()
        elif act_func == "swish":
            self.act = nn.Swish()
        elif act_func == "hard_swish":
            self.act = _HardSwish()
        elif act_func == "leaky":
            self.act = nn.LeakyReLU(alpha=0.375)
        else:
            raise NotImplementedError

    def hybrid_forward(self, F, x):
        return self.act(x)
class _SE(nn.HybridBlock):
    def __init__(self, num_out, ratio=4, act_func=("relu", "hard_sigmoid"), use_bn=False, prefix='', **kwargs):
        super(_SE, self).__init__(**kwargs)
        self.use_bn = use_bn
        num_mid = _make_divisible(num_out // ratio)
        self.pool = nn.GlobalAvgPool2D()
        self.conv1 = nn.Conv2D(channels=num_mid, kernel_size=1, use_bias=True, prefix=('%s_fc1_' % prefix))
        self.act1 = _Activation_mobilenetv3(act_func[0])
        self.conv2 = nn.Conv2D(channels=num_out, kernel_size=1, use_bias=True, prefix=('%s_fc2_' % prefix))
        self.act2 = _Activation_mobilenetv3(act_func[1])

    def hybrid_forward(self, F, x):
        out = self.pool(x)
        out = self.conv1(out)
        out = self.act1(out)
        out = self.conv2(out)
        out = self.act2(out)
        return F.broadcast_mul(x, out)
class _Unit(nn.HybridBlock):
    def __init__(self, num_out, kernel_size=1, strides=1, pad=0, num_groups=1, use_act=True, act_type="relu", prefix='', norm_layer=nn.BatchNorm, **kwargs):
        super(_Unit, self).__init__(**kwargs)
        self.use_act = use_act
        self.conv = nn.Conv2D(channels=num_out, kernel_size=kernel_size, strides=strides, padding=pad, groups=num_groups, use_bias=False, prefix='%s-conv2d_'%prefix)
        self.bn = norm_layer(prefix='%s-batchnorm_'%prefix)
        if use_act is True:
            self.act = _Activation_mobilenetv3(act_type)

    def hybrid_forward(self, F, x):
        out = self.conv(x)
        out = self.bn(out)
        if self.use_act:
            out = self.act(out)
        return out
class _ResUnit(nn.HybridBlock):
    def __init__(self, num_in, num_mid, num_out, kernel_size, act_type="relu", use_se=False, strides=1, prefix='', norm_layer=nn.BatchNorm, **kwargs):
        super(_ResUnit, self).__init__(**kwargs)
        with self.name_scope():
            self.use_se = use_se
            self.first_conv = (num_out != num_mid)
            self.use_short_cut_conv = True
            if self.first_conv:
                self.expand = _Unit(num_mid, kernel_size=1, strides=1, pad=0, act_type=act_type, prefix='%s-exp'%prefix, norm_layer=norm_layer)
            self.conv1 = _Unit(num_mid, kernel_size=kernel_size, strides=strides, pad=self._get_pad(kernel_size), act_type=act_type, num_groups=num_mid, prefix='%s-depthwise'%prefix, norm_layer=norm_layer)
            if use_se:
                self.se = _SE(num_mid, prefix='%s-se'%prefix)
            self.conv2 = _Unit(num_out, kernel_size=1, strides=1, pad=0, act_type=act_type, use_act=False, prefix='%s-linear'%prefix, norm_layer=norm_layer)
            if num_in != num_out or strides != 1:
                self.use_short_cut_conv = False

    def hybrid_forward(self, F, x):
        out = self.expand(x) if self.first_conv else x
        out = self.conv1(out)
        if self.use_se:
            out = self.se(out)
        out = self.conv2(out)
        if self.use_short_cut_conv:
            return x + out
        else:
            return out

    def _get_pad(self, kernel_size):
        if kernel_size == 1:
            return 0
        elif kernel_size == 3:
            return 1
        elif kernel_size == 5:
            return 2
        elif kernel_size == 7:
            return 3
        else:
            raise NotImplementedError
class _MobileNetV3(nn.HybridBlock):
    def __init__(self, cfg, cls_ch_squeeze, cls_ch_expand, multiplier=1., classes=1000, norm_kwargs=None, last_gamma=False, final_drop=0., use_global_stats=False, name_prefix='', norm_layer=nn.BatchNorm):
        super(_MobileNetV3, self).__init__(prefix=name_prefix)
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        if use_global_stats:
            norm_kwargs['use_global_stats'] = True
        # initialize residual networks
        k = multiplier
        self.last_gamma = last_gamma
        self.norm_kwargs = norm_kwargs
        self.inplanes = 16

        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            self.features.add(nn.Conv2D(channels=_make_divisible(k*self.inplanes), kernel_size=3, padding=1, strides=2, use_bias=False, prefix='first-3x3-conv-conv2d_'))
            self.features.add(norm_layer(prefix='first-3x3-conv-batchnorm_'))
            self.features.add(_HardSwish())
            i = 0
            for layer_cfg in cfg:
                layer = self._make_layer(kernel_size=layer_cfg[0], exp_ch=_make_divisible(k * layer_cfg[1]), out_channel=_make_divisible(k * layer_cfg[2]), use_se=layer_cfg[3], act_func=layer_cfg[4], stride=layer_cfg[5], prefix='seq-%d'%i,)
                self.features.add(layer)
                i += 1
            self.features.add(nn.Conv2D(channels= _make_divisible(k*cls_ch_squeeze), kernel_size=1, padding=0, strides=1, use_bias=False, prefix='last-1x1-conv1-conv2d_'))
            self.features.add(norm_layer(prefix='last-1x1-conv1-batchnorm_', **({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(_HardSwish())
            self.features.add(nn.GlobalAvgPool2D())
            self.features.add(nn.Conv2D(channels=cls_ch_expand, kernel_size=1, padding=0, strides=1, use_bias=False, prefix='last-1x1-conv2-conv2d_'))
            self.features.add(_HardSwish())

            if final_drop > 0:
                self.features.add(nn.Dropout(final_drop))
            self.output = nn.HybridSequential(prefix='output_')
            with self.output.name_scope():
                self.output.add(
                    nn.Conv2D(in_channels=cls_ch_expand, channels=classes, kernel_size=1, prefix='fc_'),
                    nn.Flatten())


    def _make_layer(self, kernel_size, exp_ch, out_channel, use_se, act_func, stride=1, prefix=''):

        mid_planes = exp_ch
        out_planes = out_channel
        layer = _ResUnit(self.inplanes, mid_planes, out_planes, kernel_size, act_func, strides=stride, use_se=use_se, prefix=prefix)
        self.inplanes = out_planes
        return layer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_mobilenet_v3(version, multiplier=1.,class_num=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    if version == "large":
        cfg = [
            # k, exp, c,  se,     nl,  s,
            [3, 16, 16, False, 'relu', 1],
            [3, 64, 24, False, 'relu', 2],
            [3, 72, 24, False, 'relu', 1],
            [5, 72, 40, True, 'relu', 2],
            [5, 120, 40, True, 'relu', 1],
            [5, 120, 40, True, 'relu', 1],
            [3, 240, 80, False, 'hard_swish', 2],
            [3, 200, 80, False, 'hard_swish', 1],
            [3, 184, 80, False, 'hard_swish', 1],
            [3, 184, 80, False, 'hard_swish', 1],
            [3, 480, 112, True, 'hard_swish', 1],
            [3, 672, 112, True, 'hard_swish', 1],
            [5, 672, 160, True, 'hard_swish', 2],
            [5, 960, 160, True, 'hard_swish', 1],
            [5, 960, 160, True, 'hard_swish', 1],
            ]
        cls_ch_squeeze = 960
        cls_ch_expand = 1280
    elif version == "small":
        cfg = [
            # k, exp, c,  se,     nl,  s,
            [3, 16, 16, True, 'relu', 2],
            [3, 72, 24, False, 'relu', 2],
            [3, 88, 24, False, 'relu', 1],
            [5, 96, 40, True, 'hard_swish', 2],
            [5, 240, 40, True, 'hard_swish', 1],
            [5, 240, 40, True, 'hard_swish', 1],
            [5, 120, 48, True, 'hard_swish', 1],
            [5, 144, 48, True, 'hard_swish', 1],
            [5, 288, 96, True, 'hard_swish', 2],
            [5, 576, 96, True, 'hard_swish', 1],
            [5, 576, 96, True, 'hard_swish', 1],
            ]
        cls_ch_squeeze = 576
        cls_ch_expand = 1280
    else:
        raise NotImplementedError
    net = _MobileNetV3(cfg, cls_ch_squeeze, cls_ch_expand,classes=class_num, multiplier=multiplier, final_drop=0.2, norm_layer=norm_layer, **kwargs)

    return net

9.DenseNet的实现(2017年)

论文地址:https://arxiv.org/pdf/1608.06993.pdf
网络结构:
MXNet-图像分类(gluon版本)【附源码】_第3张图片
  经典网络DenseNet(Dense Convolutional Network)由Gao Huang等人于2017年提出,论文名为:《Densely Connected Convolutional Networks》,论文见:https://arxiv.org/pdf/1608.06993.pdf
  DenseNet以前馈的方式(feed-forward fashion)将每个层与其它层连接起来。在传统卷积神经网络中,对于L层的网络具有L个连接,而在DenseNet中,会有L(L+1)/2个连接。每一层的输入来自前面所有层的输出。
DenseNet网络:

  • 减轻梯度消失(vanishing-gradient)。
  • 加强feature传递。
  • 鼓励特征重用(encourage feature reuse)。
  • 较少的参数数量。

代码实现(gluon):

def _make_dense_block(num_layers, bn_size, growth_rate, dropout, stage_index, norm_layer, norm_kwargs):
    out = nn.HybridSequential(prefix='stage%d_'%stage_index)
    with out.name_scope():
        for _ in range(num_layers):
            out.add(_make_dense_layer(growth_rate, bn_size, dropout, norm_layer, norm_kwargs))
    return out
def _make_dense_layer(growth_rate, bn_size, dropout, norm_layer, norm_kwargs):
    new_features = nn.HybridSequential(prefix='')
    new_features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
    new_features.add(nn.Activation('relu'))
    new_features.add(nn.Conv2D(bn_size * growth_rate, kernel_size=1, use_bias=False))
    new_features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
    new_features.add(nn.Activation('relu'))
    new_features.add(nn.Conv2D(growth_rate, kernel_size=3, padding=1, use_bias=False))
    if dropout:
        new_features.add(nn.Dropout(dropout))
    out = HybridConcurrent(axis=1, prefix='')
    out.add(Identity())
    out.add(new_features)

    return out
def _make_transition(num_output_features, norm_layer, norm_kwargs):
    out = nn.HybridSequential(prefix='')
    out.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
    out.add(nn.Activation('relu'))
    out.add(nn.Conv2D(num_output_features, kernel_size=1, use_bias=False))
    out.add(nn.AvgPool2D(pool_size=2, strides=2))
    return out
class _DenseNet(nn.HybridBlock):
    def __init__(self, num_init_features, growth_rate, block_config, bn_size=4, dropout=0, classes=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_DenseNet, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            self.features.add(nn.Conv2D(num_init_features, kernel_size=7, strides=2, padding=3, use_bias=False))
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.MaxPool2D(pool_size=3, strides=2, padding=1))
            # Add dense blocks
            num_features = num_init_features
            for i, num_layers in enumerate(block_config):
                self.features.add(_make_dense_block(num_layers, bn_size, growth_rate, dropout, i+1, norm_layer, norm_kwargs))
                num_features = num_features + num_layers * growth_rate
                if i != len(block_config) - 1:
                    self.features.add(_make_transition(num_features // 2, norm_layer, norm_kwargs))
                    num_features = num_features // 2
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.AvgPool2D(pool_size=7))
            self.features.add(nn.Flatten())

            self.output = nn.Dense(classes)

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
_densenet_spec = {121: (64, 32, [6, 12, 24, 16]),
                 161: (96, 48, [6, 12, 36, 24]),
                 169: (64, 32, [6, 12, 32, 32]),
                 201: (64, 32, [6, 12, 48, 32])}
def get_densenet(num_layers, class_num, **kwargs):
    num_init_features, growth_rate, block_config = _densenet_spec[num_layers]
    net = _DenseNet(num_init_features, growth_rate, block_config,classes=class_num, **kwargs)
    return net

10.ResNest的实现(2020年)

论文地址:https://arxiv.org/pdf/2004.08955.pdf
网络结构:
MXNet-图像分类(gluon版本)【附源码】_第4张图片
  ResNeSt在图像分类上中ImageNet数据集上超越了其前辈ResNet、ResNeXt、SENet以及EfficientNet。使用ResNeSt-50为基本骨架的Faster-RCNN比使用ResNet-50的mAP要高出3.08%。使用ResNeSt-50为基本骨架的DeeplabV3比使用ResNet-50的mIOU要高出3.02%。涨点效果非常明显。

  • 提出了split-attention blocks构造的ResNeSt,与现有的ResNet变体相比,不需要增加额外的计算量。而且ResNeSt可以作为其它任务的骨架。
  • 图像分类和迁移学习应用的大规模基准。 利用ResNeSt主干的模型能够在几个任务上达到最先进的性能,即:图像分类,对象检测,实例分割和语义分割。 与通过神经架构搜索生成的最新CNN模型[55]相比,所提出的ResNeSt性能优于所有现有ResNet变体,并且具有相同的计算效率,甚至可以实现更好的速度精度折衷。单个Cascade-RCNN [3]使用ResNeSt-101主干的模型在MS-COCO实例分割上实现了48.3%的box mAP和41.56%的mask mAP。 单个DeepLabV3 [7]模型同样使用ResNeSt-101主干,在ADE20K场景分析验证集上的mIoU达到46.9%,比以前的最佳结果高出1%mIoU以上。

代码实现(gluon):

class _ResNeSt(nn.HybridBlock):
    def __init__(self, block, layers, cardinality=1, bottleneck_width=64, classes=1000, dilated=False, dilation=1, norm_layer=nn.BatchNorm, norm_kwargs=None, last_gamma=False, deep_stem=False, stem_width=32, avg_down=False, final_drop=0.0, use_global_stats=False, name_prefix='', dropblock_prob=0.0, input_size=224, use_splat=False, radix=2, avd=False, avd_first=False, split_drop_ratio=0):
        self.cardinality = cardinality
        self.bottleneck_width = bottleneck_width
        self.inplanes = stem_width * 2 if deep_stem else 64
        self.radix = radix
        self.split_drop_ratio = split_drop_ratio
        self.avd_first = avd_first
        super(_ResNeSt, self).__init__(prefix=name_prefix)
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        if use_global_stats:
            norm_kwargs['use_global_stats'] = True
        self.norm_kwargs = norm_kwargs
        with self.name_scope():
            if not deep_stem:
                self.conv1 = nn.Conv2D(channels=64, kernel_size=7, strides=2, padding=3, use_bias=False, in_channels=3)
            else:
                self.conv1 = nn.HybridSequential(prefix='conv1')
                self.conv1.add(nn.Conv2D(channels=stem_width, kernel_size=3, strides=2, padding=1, use_bias=False, in_channels=3))
                self.conv1.add(norm_layer(in_channels=stem_width, **norm_kwargs))
                self.conv1.add(nn.Activation('relu'))
                self.conv1.add(nn.Conv2D(channels=stem_width, kernel_size=3, strides=1, padding=1, use_bias=False, in_channels=stem_width))
                self.conv1.add(norm_layer(in_channels=stem_width, **norm_kwargs))
                self.conv1.add(nn.Activation('relu'))
                self.conv1.add(nn.Conv2D(channels=stem_width * 2, kernel_size=3, strides=1, padding=1, use_bias=False, in_channels=stem_width))
            input_size = _update_input_size(input_size, 2)
            self.bn1 = norm_layer(in_channels=64 if not deep_stem else stem_width * 2, **norm_kwargs)
            self.relu = nn.Activation('relu')
            self.maxpool = nn.MaxPool2D(pool_size=3, strides=2, padding=1)
            input_size = _update_input_size(input_size, 2)
            self.layer1 = self._make_layer(1, block, 64, layers[0], avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, use_splat=use_splat, avd=avd)
            self.layer2 = self._make_layer(2, block, 128, layers[1], strides=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, use_splat=use_splat, avd=avd)
            input_size = _update_input_size(input_size, 2)
            if dilated or dilation == 4:
                self.layer3 = self._make_layer(3, block, 256, layers[2], strides=1, dilation=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
                self.layer4 = self._make_layer(4, block, 512, layers[3], strides=1, dilation=4, pre_dilation=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
            elif dilation == 3:
                self.layer3 = self._make_layer(3, block, 256, layers[2], strides=1, dilation=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
                self.layer4 = self._make_layer(4, block, 512, layers[3], strides=2, dilation=2, pre_dilation=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
            elif dilation == 2:
                self.layer3 = self._make_layer(3, block, 256, layers[2], strides=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
                self.layer4 = self._make_layer(4, block, 512, layers[3], strides=1, dilation=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
            else:
                self.layer3 = self._make_layer(3, block, 256, layers[2], strides=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
                input_size = _update_input_size(input_size, 2)
                self.layer4 = self._make_layer(4, block, 512, layers[3], strides=2, avg_down=avg_down, norm_layer=norm_layer, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd)
                input_size = _update_input_size(input_size, 2)
            self.avgpool = nn.GlobalAvgPool2D()
            self.flat = nn.Flatten()
            self.drop = None
            if final_drop > 0.0:
                self.drop = nn.Dropout(final_drop)
            self.fc = nn.Dense(in_units=512 * block.expansion, units=classes)

    def _make_layer(self, stage_index, block, planes, blocks, strides=1, dilation=1, pre_dilation=1, avg_down=False, norm_layer=None, last_gamma=False, dropblock_prob=0, input_size=224, use_splat=False, avd=False):
        downsample = None
        if strides != 1 or self.inplanes != planes * block.expansion:
            downsample = nn.HybridSequential(prefix='down%d_' % stage_index)
            with downsample.name_scope():
                if avg_down:
                    if pre_dilation == 1:
                        downsample.add(nn.AvgPool2D(pool_size=strides, strides=strides, ceil_mode=True, count_include_pad=False))
                    elif strides == 1:
                        downsample.add(nn.AvgPool2D(pool_size=1, strides=1, ceil_mode=True, count_include_pad=False))
                    else:
                        downsample.add(
                            nn.AvgPool2D(pool_size=pre_dilation * strides, strides=strides, padding=1, ceil_mode=True, count_include_pad=False))
                    downsample.add(nn.Conv2D(channels=planes * block.expansion, kernel_size=1, strides=1, use_bias=False, in_channels=self.inplanes))
                    downsample.add(norm_layer(in_channels=planes * block.expansion, **self.norm_kwargs))
                else:
                    downsample.add(nn.Conv2D(channels=planes * block.expansion, kernel_size=1, strides=strides, use_bias=False, in_channels=self.inplanes))
                    downsample.add(norm_layer(in_channels=planes * block.expansion, **self.norm_kwargs))

        layers = nn.HybridSequential(prefix='layers%d_' % stage_index)
        with layers.name_scope():
            if dilation in (1, 2):
                layers.add(block(planes, cardinality=self.cardinality, bottleneck_width=self.bottleneck_width, strides=strides, dilation=pre_dilation, downsample=downsample, previous_dilation=dilation, norm_layer=norm_layer, norm_kwargs=self.norm_kwargs, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd, avd_first=self.avd_first, radix=self.radix, in_channels=self.inplanes, split_drop_ratio=self.split_drop_ratio))
            elif dilation == 4:
                layers.add(block(planes, cardinality=self.cardinality, bottleneck_width=self.bottleneck_width, strides=strides, dilation=pre_dilation, downsample=downsample, previous_dilation=dilation, norm_layer=norm_layer, norm_kwargs=self.norm_kwargs, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd, avd_first=self.avd_first, radix=self.radix, in_channels=self.inplanes, split_drop_ratio=self.split_drop_ratio))
            else:
                raise RuntimeError("=> unknown dilation size: {}".format(dilation))

            input_size = _update_input_size(input_size, strides)
            self.inplanes = planes * block.expansion
            for i in range(1, blocks):
                layers.add(block(planes, cardinality=self.cardinality, bottleneck_width=self.bottleneck_width, dilation=dilation, previous_dilation=dilation, norm_layer=norm_layer, norm_kwargs=self.norm_kwargs, last_gamma=last_gamma, dropblock_prob=dropblock_prob, input_size=input_size, use_splat=use_splat, avd=avd, avd_first=self.avd_first, radix=self.radix, in_channels=self.inplanes, split_drop_ratio=self.split_drop_ratio))

        return layers

    def hybrid_forward(self, F, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.maxpool(x)

        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)

        x = self.avgpool(x)
        x = self.flat(x)
        if self.drop is not None:
            x = self.drop(x)
        x = self.fc(x)

        return x
class _DropBlock(nn.HybridBlock):
    def __init__(self, drop_prob, block_size, c, h, w):
        super(_DropBlock, self).__init__()
        self.drop_prob = drop_prob
        self.block_size = block_size
        self.c, self.h, self.w = c, h, w
        self.numel = c * h * w
        pad_h = max((block_size - 1), 0)
        pad_w = max((block_size - 1), 0)
        self.padding = (pad_h//2, pad_h-pad_h//2, pad_w//2, pad_w-pad_w//2)
        self.dtype = 'float32'

    def hybrid_forward(self, F, x):
        if not mx.autograd.is_training() or self.drop_prob <= 0:
            return x
        gamma = self.drop_prob * (self.h * self.w) / (self.block_size ** 2) / ((self.w - self.block_size + 1) * (self.h - self.block_size + 1))
        # generate mask
        mask = F.random.uniform(0, 1, shape=(1, self.c, self.h, self.w), dtype=self.dtype) < gamma
        mask = F.Pooling(mask, pool_type='max',
                         kernel=(self.block_size, self.block_size), pad=self.padding)
        mask = 1 - mask
        y = F.broadcast_mul(F.broadcast_mul(x, mask), (1.0 * self.numel / mask.sum(axis=0, exclude=True).expand_dims(1).expand_dims(1).expand_dims(1)))
        return y

    def cast(self, dtype):
        super(_DropBlock, self).cast(dtype)
        self.dtype = dtype

    def __repr__(self):
        reprstr = self.__class__.__name__ + '(' +'drop_prob: {}, block_size{}'.format(self.drop_prob, self.block_size) +')'
        return reprstr
def _update_input_size(input_size, stride):
    sh, sw = (stride, stride) if isinstance(stride, int) else stride
    ih, iw = (input_size, input_size) if isinstance(input_size, int) else input_size
    oh, ow = math.ceil(ih / sh), math.ceil(iw / sw)
    input_size = (oh, ow)
    return input_size
class _SplitAttentionConv(nn.HybridBlock):
    def __init__(self, channels, kernel_size, strides=(1, 1), padding=(0, 0), dilation=(1, 1), groups=1, radix=2, in_channels=None, r=2, norm_layer=nn.BatchNorm, norm_kwargs=None, drop_ratio=0, *args, **kwargs):
        super(_SplitAttentionConv, self).__init__()
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        inter_channels = max(in_channels*radix//2//r, 32)
        self.radix = radix
        self.cardinality = groups
        self.conv = Conv2D(channels*radix, kernel_size, strides, padding, dilation,
                           groups=groups*radix, *args, in_channels=in_channels, **kwargs)
        self.use_bn = norm_layer is not None
        if self.use_bn:
            self.bn = norm_layer(in_channels=channels*radix, **norm_kwargs)
        self.relu = nn.Activation('relu')
        self.fc1 = Conv2D(inter_channels, 1, in_channels=channels, groups=self.cardinality)
        if self.use_bn:
            self.bn1 = norm_layer(in_channels=inter_channels, **norm_kwargs)
        self.relu1 = nn.Activation('relu')
        if drop_ratio > 0:
            self.drop = nn.Dropout(drop_ratio)
        else:
            self.drop = None
        self.fc2 = Conv2D(channels*radix, 1, in_channels=inter_channels, groups=self.cardinality)
        self.channels = channels

    def hybrid_forward(self, F, x):
        x = self.conv(x)
        if self.use_bn:
            x = self.bn(x)
        x = self.relu(x)
        if self.radix > 1:
            splited = F.reshape(x.expand_dims(1), (0, self.radix, self.channels, 0, 0))
            gap = F.sum(splited, axis=1)
        else:
            gap = x
        gap = F.contrib.AdaptiveAvgPooling2D(gap, 1)
        gap = self.fc1(gap)
        if self.use_bn:
            gap = self.bn1(gap)
        atten = self.relu1(gap)
        if self.drop:
            atten = self.drop(atten)
        atten = self.fc2(atten).reshape((0, self.cardinality, self.radix, -1)).swapaxes(1, 2)
        if self.radix > 1:
            atten = F.softmax(atten, axis=1).reshape((0, self.radix, -1, 1, 1))
        else:
            atten = F.sigmoid(atten).reshape((0, -1, 1, 1))
        if self.radix > 1:
            outs = F.broadcast_mul(atten, splited)
            out = F.sum(outs, axis=1)
        else:
            out = F.broadcast_mul(atten, x)
        return out
class _Bottleneck(nn.HybridBlock):
    expansion = 4
    def __init__(self, channels, cardinality=1, bottleneck_width=64, strides=1, dilation=1, downsample=None, previous_dilation=1, norm_layer=None, norm_kwargs=None, last_gamma=False, dropblock_prob=0, input_size=None, use_splat=False, radix=2, avd=False, avd_first=False, in_channels=None, split_drop_ratio=0, **kwargs):
        super(_Bottleneck, self).__init__()
        group_width = int(channels * (bottleneck_width / 64.)) * cardinality
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        self.dropblock_prob = dropblock_prob
        self.use_splat = use_splat
        self.avd = avd and (strides > 1 or previous_dilation != dilation)
        self.avd_first = avd_first
        if self.dropblock_prob > 0:
            self.dropblock1 = _DropBlock(dropblock_prob, 3, group_width, *input_size)
            if self.avd:
                if avd_first:
                    input_size = _update_input_size(input_size, strides)
                self.dropblock2 = _DropBlock(dropblock_prob, 3, group_width, *input_size)
                if not avd_first:
                    input_size = _update_input_size(input_size, strides)
            else:
                input_size = _update_input_size(input_size, strides)
                self.dropblock2 = _DropBlock(dropblock_prob, 3, group_width, *input_size)
            self.dropblock3 = _DropBlock(dropblock_prob, 3, channels * 4, *input_size)
        self.conv1 = nn.Conv2D(channels=group_width, kernel_size=1, use_bias=False, in_channels=in_channels)
        self.bn1 = norm_layer(in_channels=group_width, **norm_kwargs)
        self.relu1 = nn.Activation('relu')
        if self.use_splat:
            self.conv2 = _SplitAttentionConv(channels=group_width, kernel_size=3,
                                            strides=1 if self.avd else strides,
                                            padding=dilation, dilation=dilation, groups=cardinality, use_bias=False, in_channels=group_width, norm_layer=norm_layer, norm_kwargs=norm_kwargs, radix=radix, drop_ratio=split_drop_ratio, **kwargs)
        else:
            self.conv2 = nn.Conv2D(channels=group_width, kernel_size=3,
                                   strides=1 if self.avd else strides,
                                   padding=dilation, dilation=dilation, groups=cardinality,
                                   use_bias=False, in_channels=group_width, **kwargs)
            self.bn2 = norm_layer(in_channels=group_width, **norm_kwargs)
            self.relu2 = nn.Activation('relu')
        self.conv3 = nn.Conv2D(channels=channels * 4, kernel_size=1, use_bias=False,
                               in_channels=group_width)
        if not last_gamma:
            self.bn3 = norm_layer(in_channels=channels * 4, **norm_kwargs)
        else:
            self.bn3 = norm_layer(in_channels=channels * 4, gamma_initializer='zeros',
                                  **norm_kwargs)
        if self.avd:
            self.avd_layer = nn.AvgPool2D(3, strides, padding=1)
        self.relu3 = nn.Activation('relu')
        self.downsample = downsample
        self.dilation = dilation
        self.strides = strides

    def hybrid_forward(self, F, x):
        residual = x

        out = self.conv1(x)
        out = self.bn1(out)
        if self.dropblock_prob > 0:
            out = self.dropblock1(out)
        out = self.relu1(out)

        if self.avd and self.avd_first:
            out = self.avd_layer(out)

        if self.use_splat:
            out = self.conv2(out)
            if self.dropblock_prob > 0:
                out = self.dropblock2(out)
        else:
            out = self.conv2(out)
            out = self.bn2(out)
            if self.dropblock_prob > 0:
                out = self.dropblock2(out)
            out = self.relu2(out)

        if self.avd and not self.avd_first:
            out = self.avd_layer(out)

        out = self.conv3(out)
        out = self.bn3(out)

        if self.downsample is not None:
            residual = self.downsample(x)

        if self.dropblock_prob > 0:
            out = self.dropblock3(out)

        out = out + residual
        out = self.relu3(out)

        return out
def get_resnest(num_layers,class_num,image_size):
    if num_layers==14:
        net = _ResNeSt(_Bottleneck, [1, 1, 1, 1],
                       radix=2, cardinality=1, bottleneck_width=64,
                       deep_stem=True, avg_down=True,
                       avd=True, avd_first=False,
                       use_splat=True, dropblock_prob=0.0,
                       name_prefix='resnest_14',classes=class_num,input_size=image_size)
    elif num_layers == 26:
        net = _ResNeSt(_Bottleneck, [2, 2, 2, 2],
                        radix=2, cardinality=1, bottleneck_width=64,
                        deep_stem=True, avg_down=True,
                        avd=True, avd_first=False,
                        use_splat=True, dropblock_prob=0.1,
                        name_prefix='resnest_26', classes=class_num,input_size=image_size)
    elif num_layers == 50:
        net = _ResNeSt(_Bottleneck, [3, 4, 6, 3],
                        radix=2, cardinality=1, bottleneck_width=64,
                        deep_stem=True, avg_down=True,
                        avd=True, avd_first=False,
                        use_splat=True, dropblock_prob=0.1,
                        name_prefix='resnest_50', classes=class_num,input_size=image_size)
    elif num_layers == 101:
        net = _ResNeSt(_Bottleneck, [3, 4, 23, 3],
                        radix=2, cardinality=1, bottleneck_width=64,
                        deep_stem=True, avg_down=True, stem_width=64,
                        avd=True, avd_first=False, use_splat=True, dropblock_prob=0.1,
                        name_prefix='resnest_101', classes=class_num,input_size=image_size)
    elif num_layers == 200:
        net = _ResNeSt(_Bottleneck, [3, 24, 36, 3], deep_stem=True, avg_down=True, stem_width=64,
                        avd=True, use_splat=True, dropblock_prob=0.1, final_drop=0.2,
                        name_prefix='resnest_200', classes=class_num,input_size=image_size)
    elif num_layers == 269:
        net = _ResNeSt(_Bottleneck, [3, 30, 48, 8], deep_stem=True, avg_down=True, stem_width=64,
                        avd=True, use_splat=True, dropblock_prob=0.1, final_drop=0.2,
                        name_prefix='resnest_', classes=class_num,input_size=image_size)
    else:
        net = None
    return net

11.ResNext的实现(2016年)

论文地址:https://arxiv.org/pdf/1611.05431.pdf
网络结构:
在这里插入图片描述
在这里插入图片描述
  ResNeXt对ResNet进行了改进,采用了多分支的策略,在论文中作者提出了三种等价的模型结构,最后的ResNeXt用了C的结构来构建我们的ResNeXt、这里面和我们的Inception是不同的,在Inception中,每一部分的拓扑结构是不同的,比如一部分是1x1卷积,3x3卷积还有5x5卷积,而我们ResNeXt是用相同的拓扑结构,并在保持参数量的情况下提高了准确率。

代码实现(gluon):

class _Block_resnext(nn.HybridBlock):
    def __init__(self, channels, cardinality, bottleneck_width, stride, downsample=False, last_gamma=False, use_se=False, avg_down=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_Block_resnext, self).__init__(**kwargs)
        D = int(math.floor(channels * (bottleneck_width / 64)))
        group_width = cardinality * D

        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Conv2D(group_width, kernel_size=1, use_bias=False))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(group_width, kernel_size=3, strides=stride, padding=1, groups=cardinality, use_bias=False))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(channels * 4, kernel_size=1, use_bias=False))
        if last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Conv2D(channels // 4, kernel_size=1, padding=0))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Conv2D(channels * 4, kernel_size=1, padding=0))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            if avg_down:
                self.downsample.add(nn.AvgPool2D(pool_size=stride, strides=stride, ceil_mode=True, count_include_pad=False))
                self.downsample.add(nn.Conv2D(channels=channels * 4, kernel_size=1, strides=1, use_bias=False))
            else:
                self.downsample.add(nn.Conv2D(channels * 4, kernel_size=1, strides=stride, use_bias=False))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x

        x = self.body(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w)

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(x + residual, act_type='relu')
        return x
class _ResNext(nn.HybridBlock):
    def __init__(self, layers, cardinality, bottleneck_width, classes=1000, last_gamma=False, use_se=False, deep_stem=False, avg_down=False, stem_width=64, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_ResNext, self).__init__(**kwargs)
        self.cardinality = cardinality
        self.bottleneck_width = bottleneck_width
        channels = 64

        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            if not deep_stem:
                self.features.add(nn.Conv2D(channels=64, kernel_size=7, strides=2, padding=3, use_bias=False))
            else:
                self.features.add(nn.Conv2D(channels=stem_width, kernel_size=3, strides=2, padding=1, use_bias=False))
                self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.Conv2D(channels=stem_width, kernel_size=3, strides=1, padding=1, use_bias=False))
                self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.Conv2D(channels=stem_width * 2, kernel_size=3, strides=1, padding=1, use_bias=False))

            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.MaxPool2D(3, 2, 1))
            for i, num_layer in enumerate(layers):
                stride = 1 if i == 0 else 2
                self.features.add(self._make_layer(channels, num_layer, stride, last_gamma, use_se, False if i == 0 else avg_down, i + 1, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
                channels *= 2
            self.features.add(nn.GlobalAvgPool2D())

            self.output = nn.Dense(classes)

    def _make_layer(self, channels, num_layers, stride, last_gamma, use_se, avg_down, stage_index, norm_layer=nn.BatchNorm, norm_kwargs=None):
        layer = nn.HybridSequential(prefix='stage%d_' % stage_index)
        with layer.name_scope():
            layer.add(_Block_resnext(channels, self.cardinality, self.bottleneck_width, stride, True, last_gamma=last_gamma, use_se=use_se, avg_down=avg_down, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            for _ in range(num_layers - 1):
                layer.add(_Block_resnext(channels, self.cardinality, self.bottleneck_width, 1, False, last_gamma=last_gamma, use_se=use_se, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        return layer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)

        return x
_resnext_spec = {50: [3, 4, 6, 3], 101: [3, 4, 23, 3]}
def get_resnextnet(version,num_layers,class_num,use_se=True):
    def get_resnext(num_layers,cardinality=32,class_num=1000, bottleneck_width=4, use_se=False, deep_stem=False, avg_down=False,  **kwargs):
        if use_se:
            kwargs['use_se'] = True
        layers = _resnext_spec[num_layers]
        net = _ResNext(layers, cardinality, bottleneck_width,classes=class_num, use_se=use_se, deep_stem=deep_stem, avg_down=avg_down, **kwargs)
        return net
    if version=='101e' or version=='50e':
        net_model = get_resnext(num_layers=(int)(version.replace('e','')),cardinality=num_layers,class_num=class_num, bottleneck_width = 4,deep_stem=True, avg_down=True)
    else:
        net_model = get_resnext(num_layers=(int)(version), cardinality=num_layers, class_num=class_num,bottleneck_width=4)
    return net_model

12.SeNet的实现(2017年)

论文地址:https://arxiv.org/pdf/1709.01507.pdf
网络结构:
MXNet-图像分类(gluon版本)【附源码】_第5张图片
SENET是在论文《Squeeze-and-Excitation Networks》中提出来的,应用在图像处理上的。主要思想:通过建模channel之间的关系来矫正channel的特征,以此提升神经网络的表征能力。特征矫正是使用全局的信息去加强有用的特征,淡化无用的特征。

代码实现(gluon):

class _SEBlock(nn.HybridBlock):
    def __init__(self, channels, cardinality, bottleneck_width, stride, downsample=False, downsample_kernel_size=3, avg_down=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_SEBlock, self).__init__(**kwargs)
        D = int(math.floor(channels * (bottleneck_width / 64)))
        group_width = cardinality * D

        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Conv2D(group_width // 2, kernel_size=1, use_bias=False))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(group_width, kernel_size=3, strides=stride, padding=1, groups=cardinality, use_bias=False))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(channels * 4, kernel_size=1, use_bias=False))
        self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        self.se = nn.HybridSequential(prefix='')
        self.se.add(nn.Conv2D(channels // 4, kernel_size=1, padding=0))
        self.se.add(nn.Activation('relu'))
        self.se.add(nn.Conv2D(channels * 4, kernel_size=1, padding=0))
        self.se.add(nn.Activation('sigmoid'))

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            if avg_down:
                self.downsample.add(nn.AvgPool2D(pool_size=stride, strides=stride, ceil_mode=True, count_include_pad=False))
                self.downsample.add(nn.Conv2D(channels=channels * 4, kernel_size=1, strides=1, use_bias=False))
            else:
                downsample_padding = 1 if downsample_kernel_size == 3 else 0
                self.downsample.add(nn.Conv2D(channels * 4, kernel_size=downsample_kernel_size, strides=stride, padding=downsample_padding, use_bias=False))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x

        x = self.body(x)

        w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
        w = self.se(w)
        x = F.broadcast_mul(x, w)

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(x + residual, act_type='relu')
        return x
class _SENet(nn.HybridBlock):
    def __init__(self, layers, cardinality, bottleneck_width, avg_down=False, classes=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(_SENet, self).__init__(**kwargs)
        self.cardinality = cardinality
        self.bottleneck_width = bottleneck_width
        channels = 64
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            self.features.add(nn.Conv2D(channels, 3, 2, 1, use_bias=False))
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.Conv2D(channels, 3, 1, 1, use_bias=False))
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.Conv2D(channels * 2, 3, 1, 1, use_bias=False))
            self.features.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
            self.features.add(nn.Activation('relu'))
            self.features.add(nn.MaxPool2D(3, 2, ceil_mode=True))

            for i, num_layer in enumerate(layers):
                stride = 1 if i == 0 else 2
                self.features.add(self._make_layer(channels, num_layer, stride, i + 1, avg_down=(False if i == 0 else avg_down), norm_layer=norm_layer, norm_kwargs=norm_kwargs))
                channels *= 2
            self.features.add(nn.GlobalAvgPool2D())
            self.features.add(nn.Dropout(0.2))

            self.output = nn.Dense(classes)

    def _make_layer(self, channels, num_layers, stride, stage_index, avg_down=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
        layer = nn.HybridSequential(prefix='stage%d_' % stage_index)
        downsample_kernel_size = 1 if stage_index == 1 else 3
        with layer.name_scope():
            layer.add(_SEBlock(channels, self.cardinality, self.bottleneck_width, stride, True, downsample_kernel_size, avg_down=avg_down, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            for _ in range(num_layers - 1):
                layer.add(_SEBlock(channels, self.cardinality, self.bottleneck_width, 1, False, prefix='', norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        return layer

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)

        return x
_senet_spec = {50: [3, 4, 6, 3],
                101: [3, 4, 23, 3],
                152: [3, 8, 36, 3]}
def get_senet(version,num_layers,class_num, cardinality=64, bottleneck_width=4, **kwargs):
    layers = _senet_spec[num_layers]
    if 'e' in version:
        avg_down = True
    else:
        avg_down = False
    net = _SENet(layers, cardinality, bottleneck_width, avg_down,classes=class_num, **kwargs)
    return net

13.Xception的实现

代码实现(gluon):

class _SeparableConv2d(nn.HybridBlock):
    def __init__(self, inplanes, planes, kernel_size=3, stride=1, dilation=1, bias=False, norm_layer=None, norm_kwargs=None):
        super(_SeparableConv2d, self).__init__()
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        self.kernel_size = kernel_size
        self.dilation = dilation
        self.conv1 = nn.Conv2D(in_channels=inplanes, channels=inplanes, kernel_size=kernel_size, strides=stride, padding=0, dilation=dilation, groups=inplanes, use_bias=bias)
        self.bn = norm_layer(in_channels=inplanes, **norm_kwargs)
        self.pointwise = nn.Conv2D(in_channels=inplanes, channels=planes, kernel_size=1, use_bias=bias)

    def hybrid_forward(self, F, x):
        x = self.fixed_padding(x, F, self.kernel_size, dilation=self.dilation)
        x = self.conv1(x)
        x = self.bn(x)
        x = self.pointwise(x)
        return x

    def fixed_padding(self, inputs, F, kernel_size, dilation):
        kernel_size_effective = kernel_size + (kernel_size - 1) * (dilation - 1)
        pad_total = kernel_size_effective - 1
        pad_beg = pad_total // 2
        pad_end = pad_total - pad_beg
        padded_inputs = F.pad(inputs, mode="constant", constant_value=0, pad_width=(0, 0, 0, 0, pad_beg, pad_end, pad_beg, pad_end))
        return padded_inputs
class _Block(nn.HybridBlock):
    def __init__(self, inplanes, planes, reps, stride=1, dilation=1, norm_layer=None, norm_kwargs=None, start_with_relu=True, grow_first=True, is_last=False):
        super(_Block, self).__init__()
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        if planes != inplanes or stride != 1:
            self.skip = nn.Conv2D(in_channels=inplanes, channels=planes, kernel_size=1, strides=stride, use_bias=False)
            self.skipbn = norm_layer(in_channels=planes, **norm_kwargs)
        else:
            self.skip = None
        self.relu = nn.Activation('relu')
        self.rep = nn.HybridSequential()
        filters = inplanes
        if grow_first:
            if start_with_relu:
                self.rep.add(self.relu)
            self.rep.add(_SeparableConv2d(inplanes, planes, 3, 1, dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.rep.add(norm_layer(in_channels=planes, **norm_kwargs))
            filters = planes
        for i in range(reps - 1):
            if grow_first or start_with_relu:
                self.rep.add(self.relu)
            self.rep.add(_SeparableConv2d(filters, filters, 3, 1, dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.rep.add(norm_layer(in_channels=filters, **norm_kwargs))
        if not grow_first:
            self.rep.add(self.relu)
            self.rep.add(_SeparableConv2d(inplanes, planes, 3, 1, dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.rep.add(norm_layer(in_channels=planes, **norm_kwargs))
        if stride != 1:
            self.rep.add(self.relu)
            self.rep.add(_SeparableConv2d(planes, planes, 3, stride, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.rep.add(norm_layer(in_channels=planes, **norm_kwargs))
        elif is_last:
            self.rep.add(self.relu)
            self.rep.add(_SeparableConv2d(planes, planes, 3, 1, dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.rep.add(norm_layer(in_channels=planes, **norm_kwargs))

    def hybrid_forward(self, F, inp):
        x = self.rep(inp)
        if self.skip is not None:
            skip = self.skip(inp)
            skip = self.skipbn(skip)
        else:
            skip = inp
        x = x + skip
        return x
class _Xception65(nn.HybridBlock):
    def __init__(self, classes=1000, output_stride=32, norm_layer=nn.BatchNorm, norm_kwargs=None):
        super(_Xception65, self).__init__()
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        if output_stride == 32:
            entry_block3_stride = 2
            exit_block20_stride = 2
            middle_block_dilation = 1
            exit_block_dilations = (1, 1)
        elif output_stride == 16:
            entry_block3_stride = 2
            exit_block20_stride = 1
            middle_block_dilation = 1
            exit_block_dilations = (1, 2)
        elif output_stride == 8:
            entry_block3_stride = 1
            exit_block20_stride = 1
            middle_block_dilation = 2
            exit_block_dilations = (2, 4)
        else:
            raise NotImplementedError

        with self.name_scope():
            self.conv1 = nn.Conv2D(in_channels=3, channels=32, kernel_size=3, strides=2, padding=1, use_bias=False)
            self.bn1 = norm_layer(in_channels=32, **norm_kwargs)
            self.relu = nn.Activation('relu')

            self.conv2 = nn.Conv2D(in_channels=32, channels=64, kernel_size=3, strides=1, padding=1, use_bias=False)
            self.bn2 = norm_layer(in_channels=64)

            self.block1 = _Block(64, 128, reps=2, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False)
            self.block2 = _Block(128, 256, reps=2, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False, grow_first=True)
            #print('self.block2', self.block2)
            self.block3 = _Block(256, 728, reps=2, stride=entry_block3_stride, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=True, is_last=True)
            # Middle flow
            self.midflow = nn.HybridSequential()
            for i in range(4, 20):
                self.midflow.add((_Block(728, 728, reps=3, stride=1, dilation=middle_block_dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=True)))

            # Exit flow
            self.block20 = _Block(728, 1024, reps=2, stride=exit_block20_stride, dilation=exit_block_dilations[0], norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=False, is_last=True)

            self.conv3 = _SeparableConv2d(1024, 1536, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn3 = norm_layer(in_channels=1536, **norm_kwargs)

            self.conv4 = _SeparableConv2d(1536, 1536, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn4 = norm_layer(in_channels=1536, **norm_kwargs)

            self.conv5 = _SeparableConv2d(1536, 2048, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn5 = norm_layer(in_channels=2048, **norm_kwargs)
            self.avgpool = nn.GlobalAvgPool2D()
            self.flat = nn.Flatten()
            self.fc = nn.Dense(in_units=2048, units=classes)

    def hybrid_forward(self, F, x):
        # Entry flow
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)

        x = self.conv2(x)
        x = self.bn2(x)
        x = self.relu(x)

        x = self.block1(x)
        # add relu here
        x = self.relu(x)
        #c1 = x
        x = self.block2(x)
        #c2 = x
        x = self.block3(x)

        # Middle flow
        x = self.midflow(x)
        #c3 = x

        # Exit flow
        x = self.block20(x)
        x = self.relu(x)
        x = self.conv3(x)
        x = self.bn3(x)
        x = self.relu(x)

        x = self.conv4(x)
        x = self.bn4(x)
        x = self.relu(x)

        x = self.conv5(x)
        x = self.bn5(x)
        x = self.relu(x)

        x = self.avgpool(x)
        x = self.flat(x)

        x = self.fc(x)
        return x
class _Xception71(nn.HybridBlock):
    def __init__(self, classes=1000, output_stride=32, norm_layer=nn.BatchNorm, norm_kwargs=None):
        super(_Xception71, self).__init__()
        norm_kwargs = norm_kwargs if norm_kwargs is not None else {}
        if output_stride == 32:
            entry_block3_stride = 2
            exit_block20_stride = 2
            middle_block_dilation = 1
            exit_block_dilations = (1, 1)
        elif output_stride == 16:
            entry_block3_stride = 2
            exit_block20_stride = 1
            middle_block_dilation = 1
            exit_block_dilations = (1, 2)
        elif output_stride == 8:
            entry_block3_stride = 1
            exit_block20_stride = 1
            middle_block_dilation = 2
            exit_block_dilations = (2, 4)
        else:
            raise NotImplementedError
        # Entry flow
        with self.name_scope():
            self.conv1 = nn.Conv2D(in_channels=3, channels=32, kernel_size=3, strides=2, padding=1, use_bias=False)
            self.bn1 = norm_layer(in_channels=32, **norm_kwargs)
            self.relu = nn.Activation('relu')

            self.conv2 = nn.Conv2D(in_channels=32, channels=64, kernel_size=3, strides=1, padding=1, use_bias=False)
            self.bn2 = norm_layer(in_channels=64)

            self.block1 = _Block(64, 128, reps=2, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False)
            self.block2 = nn.HybridSequential()

            self.block2.add(_Block(128, 256, reps=2, stride=1, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False, grow_first=True))
            self.block2.add(_Block(256, 256, reps=2, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False, grow_first=True))
            self.block2.add(_Block(256, 728, reps=2, stride=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=False, grow_first=True))
            self.block3 = _Block(728, 728, reps=2, stride=entry_block3_stride, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=True, is_last=True)
            # Middle flow
            self.midflow = nn.HybridSequential()
            for i in range(4, 20):
                self.midflow.add((_Block(728, 728, reps=3, stride=1, dilation=middle_block_dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=True)))

            # Exit flow
            self.block20 = _Block(728, 1024, reps=2, stride=exit_block20_stride, dilation=exit_block_dilations[0], norm_layer=norm_layer, norm_kwargs=norm_kwargs, start_with_relu=True, grow_first=False, is_last=True)

            self.conv3 = _SeparableConv2d(1024, 1536, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn3 = norm_layer(in_channels=1536, **norm_kwargs)

            self.conv4 = _SeparableConv2d(1536, 1536, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn4 = norm_layer(in_channels=1536, **norm_kwargs)

            self.conv5 = _SeparableConv2d(1536, 2048, 3, stride=1, dilation=exit_block_dilations[1], norm_layer=norm_layer, norm_kwargs=norm_kwargs)
            self.bn5 = norm_layer(in_channels=2048, **norm_kwargs)
            self.avgpool = nn.GlobalAvgPool2D()
            self.flat = nn.Flatten()
            self.fc = nn.Dense(in_units=2048, units=classes)

    def hybrid_forward(self, F, x):
        # Entry flow
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)

        x = self.conv2(x)
        x = self.bn2(x)
        x = self.relu(x)

        x = self.block1(x)
        # add relu here
        x = self.relu(x)
        # low_level_feat = x
        x = self.block2(x)
        #c2 = x
        x = self.block3(x)

        # Middle flow
        x = self.midflow(x)
        #c3 = x

        # Exit flow
        x = self.block20(x)
        x = self.relu(x)
        x = self.conv3(x)
        x = self.bn3(x)
        x = self.relu(x)

        x = self.conv4(x)
        x = self.bn4(x)
        x = self.relu(x)

        x = self.conv5(x)
        x = self.bn5(x)
        x = self.relu(x)

        x = self.avgpool(x)
        x = self.flat(x)

        x = self.fc(x)
        return x
def get_xception(version, class_num, **kwargs):
    if version == 65:
        net = _Xception65(classes=class_num,**kwargs)
    elif version == 71:
        net = _Xception71(classes=class_num, ** kwargs)
    else:
        net = None
    return net

14.InceptionV3的实现

代码实现(gluon):

def _make_basic_conv_inceptionv3(norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    out = nn.HybridSequential(prefix='')
    out.add(nn.Conv2D(use_bias=False, **kwargs))
    out.add(norm_layer(epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
    out.add(nn.Activation('relu'))
    return out
def _make_branch_inceptionv3(use_pool, norm_layer, norm_kwargs, *conv_settings):
    out = nn.HybridSequential(prefix='')
    if use_pool == 'avg':
        out.add(nn.AvgPool2D(pool_size=3, strides=1, padding=1))
    elif use_pool == 'max':
        out.add(nn.MaxPool2D(pool_size=3, strides=2))
    setting_names = ['channels', 'kernel_size', 'strides', 'padding']
    for setting in conv_settings:
        kwargs = {}
        for i, value in enumerate(setting):
            if value is not None:
                kwargs[setting_names[i]] = value
        out.add(_make_basic_conv_inceptionv3(norm_layer, norm_kwargs, **kwargs))
    return out
def _make_A(pool_features, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (64, 1, None, None)))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (48, 1, None, None), (64, 5, None, 2)))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (64, 1, None, None), (96, 3, None, 1), (96, 3, None, 1)))
        out.add(_make_branch_inceptionv3('avg', norm_layer, norm_kwargs, (pool_features, 1, None, None)))
    return out
def _make_B(prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, 3, 2, None)))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (64, 1, None, None), (96, 3, None, 1), (96, 3, 2, None)))
        out.add(_make_branch_inceptionv3('max', norm_layer, norm_kwargs))
    return out
def _make_C(channels_7x7, prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (192, 1, None, None)))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (channels_7x7, 1, None, None), (channels_7x7, (1, 7), None, (0, 3)), (192, (7, 1), None, (3, 0))))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (channels_7x7, 1, None, None), (channels_7x7, (7, 1), None, (3, 0)), (channels_7x7, (1, 7), None, (0, 3)), (channels_7x7, (7, 1), None, (3, 0)), (192, (1, 7), None, (0, 3))))
        out.add(_make_branch_inceptionv3('avg', norm_layer, norm_kwargs, (192, 1, None, None)))
    return out
def _make_D(prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (192, 1, None, None), (320, 3, 2, None)))
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (192, 1, None, None), (192, (1, 7), None, (0, 3)), (192, (7, 1), None, (3, 0)), (192, 3, 2, None)))
        out.add(_make_branch_inceptionv3('max', norm_layer, norm_kwargs))
    return out
def _make_E(prefix, norm_layer, norm_kwargs):
    out = HybridConcurrent(axis=1, prefix=prefix)
    with out.name_scope():
        out.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (320, 1, None, None)))

        branch_3x3 = nn.HybridSequential(prefix='')
        out.add(branch_3x3)
        branch_3x3.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, 1, None, None)))
        branch_3x3_split = HybridConcurrent(axis=1, prefix='')
        branch_3x3_split.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, (1, 3), None, (0, 1))))
        branch_3x3_split.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, (3, 1), None, (1, 0))))
        branch_3x3.add(branch_3x3_split)

        branch_3x3dbl = nn.HybridSequential(prefix='')
        out.add(branch_3x3dbl)
        branch_3x3dbl.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (448, 1, None, None), (384, 3, None, 1)))
        branch_3x3dbl_split = HybridConcurrent(axis=1, prefix='')
        branch_3x3dbl.add(branch_3x3dbl_split)
        branch_3x3dbl_split.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, (1, 3), None, (0, 1))))
        branch_3x3dbl_split.add(_make_branch_inceptionv3(None, norm_layer, norm_kwargs, (384, (3, 1), None, (1, 0))))

        out.add(_make_branch_inceptionv3('avg', norm_layer, norm_kwargs, (192, 1, None, None)))
    return out
def make_aux_inceptionv3(classes, norm_layer, norm_kwargs):
    out = nn.HybridSequential(prefix='')
    out.add(nn.AvgPool2D(pool_size=5, strides=3))
    out.add(_make_basic_conv_inceptionv3(channels=128, kernel_size=1, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
    out.add(_make_basic_conv_inceptionv3(channels=768, kernel_size=5, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
    out.add(nn.Flatten())
    out.add(nn.Dense(classes))
    return out
class _Inception3(nn.HybridBlock):
    def __init__(self, classes=1000, norm_layer=nn.BatchNorm, norm_kwargs=None, partial_bn=False, **kwargs):
        super(_Inception3, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            self.features.add(_make_basic_conv_inceptionv3(channels=32, kernel_size=3, strides=2, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            if partial_bn:
                if norm_kwargs is not None:
                    norm_kwargs['use_global_stats'] = True
                else:
                    norm_kwargs = {}
                    norm_kwargs['use_global_stats'] = True

            self.features.add(_make_basic_conv_inceptionv3(channels=32, kernel_size=3, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.features.add(_make_basic_conv_inceptionv3(channels=64, kernel_size=3, padding=1, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
            self.features.add(_make_basic_conv_inceptionv3(channels=80, kernel_size=1, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.features.add(_make_basic_conv_inceptionv3(channels=192, kernel_size=3, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
            self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
            self.features.add(_make_A(32, 'A1_', norm_layer, norm_kwargs))
            self.features.add(_make_A(64, 'A2_', norm_layer, norm_kwargs))
            self.features.add(_make_A(64, 'A3_', norm_layer, norm_kwargs))
            self.features.add(_make_B('B_', norm_layer, norm_kwargs))
            self.features.add(_make_C(128, 'C1_', norm_layer, norm_kwargs))
            self.features.add(_make_C(160, 'C2_', norm_layer, norm_kwargs))
            self.features.add(_make_C(160, 'C3_', norm_layer, norm_kwargs))
            self.features.add(_make_C(192, 'C4_', norm_layer, norm_kwargs))
            self.features.add(_make_D('D_', norm_layer, norm_kwargs))
            self.features.add(_make_E('E1_', norm_layer, norm_kwargs))
            self.features.add(_make_E('E2_', norm_layer, norm_kwargs))
            self.features.add(nn.AvgPool2D(pool_size=8))
            self.features.add(nn.Dropout(0.5))

            self.output = nn.Dense(classes)

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_inception_v3(class_num, partial_bn=False, **kwargs):
    net = _Inception3(classes=class_num,partial_bn = partial_bn,**kwargs)
    return net

15.SqueezeNet的实现

def _make_fire(squeeze_channels, expand1x1_channels, expand3x3_channels):
    out = nn.HybridSequential(prefix='')
    out.add(_make_fire_conv(squeeze_channels, 1))
    paths = HybridConcurrent(axis=1, prefix='')
    paths.add(_make_fire_conv(expand1x1_channels, 1))
    paths.add(_make_fire_conv(expand3x3_channels, 3, 1))
    out.add(paths)
    return out
def _make_fire_conv(channels, kernel_size, padding=0):
    out = nn.HybridSequential(prefix='')
    out.add(nn.Conv2D(channels, kernel_size, padding=padding))
    out.add(nn.Activation('relu'))
    return out
class _SqueezeNet(nn.HybridBlock):
    def __init__(self, version, classes=1000, **kwargs):
        super(_SqueezeNet, self).__init__(**kwargs)
        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            if version == '1.0':
                self.features.add(nn.Conv2D(96, kernel_size=7, strides=2))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.MaxPool2D(pool_size=2, strides=2, ceil_mode=True))
                self.features.add(_make_fire(16, 64, 64))
                self.features.add(_make_fire(16, 64, 64))
                self.features.add(_make_fire(32, 128, 128))
                self.features.add(nn.MaxPool2D(pool_size=2, strides=2, ceil_mode=True))
                self.features.add(_make_fire(32, 128, 128))
                self.features.add(_make_fire(48, 192, 192))
                self.features.add(_make_fire(48, 192, 192))
                self.features.add(_make_fire(64, 256, 256))
                self.features.add(nn.MaxPool2D(pool_size=2, strides=2, ceil_mode=True))
                self.features.add(_make_fire(64, 256, 256))
            else:
                self.features.add(nn.Conv2D(64, kernel_size=3, strides=2))
                self.features.add(nn.Activation('relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True))
                self.features.add(_make_fire(16, 64, 64))
                self.features.add(_make_fire(16, 64, 64))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True))
                self.features.add(_make_fire(32, 128, 128))
                self.features.add(_make_fire(32, 128, 128))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2, ceil_mode=True))
                self.features.add(_make_fire(48, 192, 192))
                self.features.add(_make_fire(48, 192, 192))
                self.features.add(_make_fire(64, 256, 256))
                self.features.add(_make_fire(64, 256, 256))
            self.features.add(nn.Dropout(0.5))

            self.output = nn.HybridSequential(prefix='')
            self.output.add(nn.Conv2D(classes, kernel_size=1))
            self.output.add(nn.Activation('relu'))
            self.output.add(nn.AvgPool2D((1, 1)))
            self.output.add(nn.Flatten())

    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x
def get_squeezenet(version,class_num,  **kwargs):
    net = _SqueezeNet(version,classes = class_num,  **kwargs)
    return net

16.NinNet的实现

代码实现(gluon):

class _NinNet(nn.HybridBlock):
    def __init__(self, classes=1000, **kwargs):
        super(_NinNet, self).__init__(**kwargs)

        with self.name_scope():
            self.features = nn.HybridSequential(prefix='')
            with self.features.name_scope():
                self.features.add(nn.Conv2D(96, 11, 4, 0, activation='relu'))
                self.features.add(nn.Conv2D(96, kernel_size=1, activation='relu'))
                self.features.add(nn.Conv2D(96, kernel_size=1, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))

                self.features.add(nn.Conv2D(256, 5, 1, 2, activation='relu'))
                self.features.add(nn.Conv2D(256, kernel_size=1, activation='relu'))
                self.features.add(nn.Conv2D(256, kernel_size=1, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))

                self.features.add(nn.Conv2D(384, 3, 1, 1, activation='relu'))
                self.features.add(nn.Conv2D(384, kernel_size=1, activation='relu'))
                self.features.add(nn.Conv2D(384, kernel_size=1, activation='relu'))
                self.features.add(nn.MaxPool2D(pool_size=3, strides=2))
                self.features.add(nn.Dropout(0.5))

                self.features.add(nn.Conv2D(classes, 3, 1, 1, activation='relu'))
                self.features.add(nn.Conv2D(classes, kernel_size=1, activation='relu'))
                self.features.add(nn.Conv2D(classes, kernel_size=1, activation='relu'))

                self.features.add(nn.GlobalAvgPool2D())
                self.features.add(nn.Flatten())

            self.output = nn.Dense(classes)
    def hybrid_forward(self, F, x):
        x = self.features(x)
        x = self.output(x)
        return x

def get_ninnet(class_num):
    return _NinNet(classes=class_num)

17.NasNet的实现

代码实现(gluon):

class MaxPoolPad(nn.HybridBlock):
    def __init__(self):
        super(MaxPoolPad, self).__init__()
        self.pool = nn.MaxPool2D(3, strides=2, padding=1)

    def hybrid_forward(self, F, x):
        x = F.pad(x, pad_width=(0, 0, 0, 0, 1, 0, 1, 0), mode='constant', constant_value=0)
        x = self.pool(x)
        x = F.slice(x, begin=(0, 0, 1, 1), end=(None, None, None, None))
        return x
class AvgPoolPad(nn.HybridBlock):
    def __init__(self, stride=2, padding=1):
        super(AvgPoolPad, self).__init__()
        self.pool = nn.AvgPool2D(3, strides=stride, padding=padding, count_include_pad=False)

    def hybrid_forward(self, F, x):
        x = F.pad(x, pad_width=(0, 0, 0, 0, 1, 0, 1, 0), mode='constant', constant_value=0)
        x = self.pool(x)
        x = F.slice(x, begin=(0, 0, 1, 1), end=(None, None, None, None))
        return x
class SeparableConv2d_nasnet(nn.HybridBlock):
    def __init__(self, in_channels, channels, dw_kernel, dw_stride, dw_padding, use_bias=False):
        super(SeparableConv2d_nasnet, self).__init__()
        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Conv2D(in_channels, kernel_size=dw_kernel, strides=dw_stride, padding=dw_padding, use_bias=use_bias, groups=in_channels))
        self.body.add(nn.Conv2D(channels, kernel_size=1, strides=1, use_bias=use_bias))

    def hybrid_forward(self, F, x):
        x = self.body(x)
        return x
class BranchSeparables(nn.HybridBlock):
    def __init__(self, in_channels, out_channels, kernel_size, stride, padding, norm_layer, norm_kwargs, use_bias=False):
        super(BranchSeparables, self).__init__()
        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Activation('relu'))
        self.body.add(SeparableConv2d_nasnet(in_channels, in_channels, kernel_size, stride, padding, use_bias=use_bias))
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(SeparableConv2d_nasnet(in_channels, out_channels, kernel_size, 1, padding, use_bias=use_bias))
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

    def hybrid_forward(self, F, x):
        x = self.body(x)
        return(x)
class BranchSeparablesStem(nn.HybridBlock):
    def __init__(self, in_channels, out_channels, kernel_size, stride, padding, norm_layer, norm_kwargs, use_bias=False):
        super(BranchSeparablesStem, self).__init__()
        self.body = nn.HybridSequential(prefix='')

        self.body.add(nn.Activation('relu'))
        self.body.add(SeparableConv2d_nasnet(in_channels, out_channels, kernel_size, stride, padding, use_bias=use_bias))
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(SeparableConv2d_nasnet(out_channels, out_channels, kernel_size, 1, padding, use_bias=use_bias))
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

    def hybrid_forward(self, F, x):
        x = self.body(x)
        return(x)
class BranchSeparablesReduction(nn.HybridBlock):
    def __init__(self, in_channels, out_channels, kernel_size, stride, padding,
                 z_padding=1, use_bias=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
        super(BranchSeparablesReduction, self).__init__()

        self.z_padding = z_padding
        self.separable = SeparableConv2d_nasnet(in_channels, in_channels, kernel_size, stride, padding, use_bias=use_bias)

        self.body = nn.HybridSequential(prefix='')
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(SeparableConv2d_nasnet(in_channels, out_channels, kernel_size, 1, padding, use_bias=use_bias))
        self.body.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

    def hybrid_forward(self, F, x):
        x = F.Activation(x, act_type='relu')
        x = F.pad(x, pad_width=(0, 0, 0, 0, self.z_padding, 0, self.z_padding, 0), mode='constant', constant_value=0)
        x = self.separable(x)
        x = F.slice(x, begin=(0, 0, 1, 1), end=(None, None, None, None))
        x = self.body(x)
        return(x)
class CellStem0(nn.HybridBlock):
    def __init__(self, stem_filters, norm_layer, norm_kwargs, num_filters=42):
        super(CellStem0, self).__init__()

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(num_filters, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.comb_iter_0_left = BranchSeparables(num_filters, num_filters, 5, 2, 2, norm_layer, norm_kwargs)
        self.comb_iter_0_right = BranchSeparablesStem(stem_filters, num_filters, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_1_left = nn.MaxPool2D(3, strides=2, padding=1)
        self.comb_iter_1_right = BranchSeparablesStem(stem_filters, num_filters, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_2_left = nn.AvgPool2D(3, strides=2, padding=1, count_include_pad=False)
        self.comb_iter_2_right = BranchSeparablesStem(stem_filters, num_filters, 5, 2, 2, norm_layer, norm_kwargs)

        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparables(num_filters, num_filters, 3, 1, 1, norm_layer, norm_kwargs)
        self.comb_iter_4_right = nn.MaxPool2D(3, strides=2, padding=1)

    def hybrid_forward(self, F, x):
        x1 = self.conv_1x1(x)

        x_comb_iter_0_left = self.comb_iter_0_left(x1)
        x_comb_iter_0_right = self.comb_iter_0_right(x)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x1)
        x_comb_iter_1_right = self.comb_iter_1_right(x)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x1)
        x_comb_iter_2_right = self.comb_iter_2_right(x)
        x_comb_iter_2 = x_comb_iter_2_left + x_comb_iter_2_right

        x_comb_iter_3_right = self.comb_iter_3_right(x_comb_iter_0)
        x_comb_iter_3 = x_comb_iter_3_right + x_comb_iter_1

        x_comb_iter_4_left = self.comb_iter_4_left(x_comb_iter_0)
        x_comb_iter_4_right = self.comb_iter_4_right(x1)
        x_comb_iter_4 = x_comb_iter_4_left + x_comb_iter_4_right

        x_out = F.concat(x_comb_iter_1, x_comb_iter_2, x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out
class CellStem1(nn.HybridBlock):
    def __init__(self, num_filters, norm_layer, norm_kwargs):
        super(CellStem1, self).__init__()

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(num_filters, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.path_1 = nn.HybridSequential(prefix='')
        self.path_1.add(nn.AvgPool2D(1, strides=2, count_include_pad=False))
        self.path_1.add(nn.Conv2D(num_filters//2, 1, strides=1, use_bias=False))

        self.path_2 = nn.HybridSequential(prefix='')
        # No nn.ZeroPad2D in gluon
        self.path_2.add(nn.AvgPool2D(1, strides=2, count_include_pad=False))
        self.path_2.add(nn.Conv2D(num_filters//2, 1, strides=1, use_bias=False))

        self.final_path_bn = norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs))

        self.comb_iter_0_left = BranchSeparables(num_filters, num_filters, 5, 2, 2, norm_layer, norm_kwargs)
        self.comb_iter_0_right = BranchSeparables(num_filters, num_filters, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_1_left = nn.MaxPool2D(3, strides=2, padding=1)
        self.comb_iter_1_right = BranchSeparables(num_filters, num_filters, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_2_left = nn.AvgPool2D(3, strides=2, padding=1, count_include_pad=False)
        self.comb_iter_2_right = BranchSeparables(num_filters, num_filters, 5, 2, 2, norm_layer, norm_kwargs)

        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparables(num_filters, num_filters, 3, 1, 1, norm_layer, norm_kwargs)
        self.comb_iter_4_right = nn.MaxPool2D(3, strides=2, padding=1)

    def hybrid_forward(self, F, x_conv0, x_stem_0):
        x_left = self.conv_1x1(x_stem_0)

        x_relu = F.Activation(x_conv0, act_type='relu')
        x_path1 = self.path_1(x_relu)
        x_path2 = F.pad(x_relu, pad_width=(0, 0, 0, 0, 0, 1, 0, 1), mode='constant', constant_value=0)
        x_path2 = F.slice(x_path2, begin=(0, 0, 1, 1), end=(None, None, None, None))
        x_path2 = self.path_2(x_path2)
        x_right = self.final_path_bn(F.concat(x_path1, x_path2, dim=1))

        x_comb_iter_0_left = self.comb_iter_0_left(x_left)
        x_comb_iter_0_right = self.comb_iter_0_right(x_right)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x_left)
        x_comb_iter_1_right = self.comb_iter_1_right(x_right)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x_left)
        x_comb_iter_2_right = self.comb_iter_2_right(x_right)
        x_comb_iter_2 = x_comb_iter_2_left + x_comb_iter_2_right

        x_comb_iter_3_right = self.comb_iter_3_right(x_comb_iter_0)
        x_comb_iter_3 = x_comb_iter_3_right + x_comb_iter_1

        x_comb_iter_4_left = self.comb_iter_4_left(x_comb_iter_0)
        x_comb_iter_4_right = self.comb_iter_4_right(x_left)
        x_comb_iter_4 = x_comb_iter_4_left + x_comb_iter_4_right

        x_out = F.concat(x_comb_iter_1, x_comb_iter_2, x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out
class FirstCell(nn.HybridBlock):
    def __init__(self, out_channels_left, out_channels_right, norm_layer, norm_kwargs):
        super(FirstCell, self).__init__()

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(out_channels_right, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.path_1 = nn.HybridSequential(prefix='')
        self.path_1.add(nn.AvgPool2D(1, strides=2, count_include_pad=False))
        self.path_1.add(nn.Conv2D(out_channels_left, 1, strides=1, use_bias=False))

        self.path_2 = nn.HybridSequential(prefix='')
        # No nn.ZeroPad2D in gluon
        self.path_2.add(nn.AvgPool2D(1, strides=2, count_include_pad=False))
        self.path_2.add(nn.Conv2D(out_channels_left, 1, strides=1, use_bias=False))

        self.final_path_bn = norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs))

        self.comb_iter_0_left = BranchSeparables(out_channels_right, out_channels_right, 5, 1, 2, norm_layer, norm_kwargs)
        self.comb_iter_0_right = BranchSeparables(out_channels_right, out_channels_right, 3, 1, 1, norm_layer, norm_kwargs)

        self.comb_iter_1_left = BranchSeparables(out_channels_right, out_channels_right, 5, 1, 2, norm_layer, norm_kwargs)
        self.comb_iter_1_right = BranchSeparables(out_channels_right, out_channels_right, 3, 1, 1, norm_layer, norm_kwargs)

        self.comb_iter_2_left = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_3_left = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)
        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparables(out_channels_right, out_channels_right, 3, 1, 1, norm_layer, norm_kwargs)

    def hybrid_forward(self, F, x, x_prev):
        x_relu = F.Activation(x_prev, act_type='relu')
        x_path1 = self.path_1(x_relu)
        x_path2 = F.pad(x_relu, pad_width=(0, 0, 0, 0, 0, 1, 0, 1), mode='constant', constant_value=0)
        x_path2 = F.slice(x_path2, begin=(0, 0, 1, 1), end=(None, None, None, None))
        x_path2 = self.path_2(x_path2)
        x_left = self.final_path_bn(F.concat(x_path1, x_path2, dim=1))

        x_right = self.conv_1x1(x)

        x_comb_iter_0_left = self.comb_iter_0_left(x_right)
        x_comb_iter_0_right = self.comb_iter_0_right(x_left)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x_left)
        x_comb_iter_1_right = self.comb_iter_1_right(x_left)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x_right)
        x_comb_iter_2 = x_comb_iter_2_left + x_left

        x_comb_iter_3_left = self.comb_iter_3_left(x_left)
        x_comb_iter_3_right = self.comb_iter_3_right(x_left)
        x_comb_iter_3 = x_comb_iter_3_left + x_comb_iter_3_right

        x_comb_iter_4_left = self.comb_iter_4_left(x_right)
        x_comb_iter_4 = x_comb_iter_4_left + x_right

        x_out = F.concat(x_left, x_comb_iter_0, x_comb_iter_1, x_comb_iter_2,
                         x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out, x
class NormalCell(nn.HybridBlock):
    def __init__(self, out_channels_left, out_channels_right, norm_layer, norm_kwargs):
        super(NormalCell, self).__init__()

        self.conv_prev_1x1 = nn.HybridSequential(prefix='')
        self.conv_prev_1x1.add(nn.Activation('relu'))
        self.conv_prev_1x1.add(nn.Conv2D(out_channels_left, 1, strides=1, use_bias=False))
        self.conv_prev_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(out_channels_right, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.comb_iter_0_left = BranchSeparables(out_channels_right, out_channels_right, 5, 1, 2, norm_layer, norm_kwargs)
        self.comb_iter_0_right = BranchSeparables(out_channels_left, out_channels_left, 3, 1, 1, norm_layer, norm_kwargs)

        self.comb_iter_1_left = BranchSeparables(out_channels_left, out_channels_left, 5, 1, 2, norm_layer, norm_kwargs)
        self.comb_iter_1_right = BranchSeparables(out_channels_left, out_channels_left, 3, 1, 1, norm_layer, norm_kwargs)

        self.comb_iter_2_left = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_3_left = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)
        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparables(out_channels_right, out_channels_right, 3, 1, 1, norm_layer, norm_kwargs)

    def hybrid_forward(self, F, x, x_prev):
        x_left = self.conv_prev_1x1(x_prev)
        x_right = self.conv_1x1(x)

        x_comb_iter_0_left = self.comb_iter_0_left(x_right)
        x_comb_iter_0_right = self.comb_iter_0_right(x_left)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x_left)
        x_comb_iter_1_right = self.comb_iter_1_right(x_left)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x_right)
        x_comb_iter_2 = x_comb_iter_2_left + x_left

        x_comb_iter_3_left = self.comb_iter_3_left(x_left)
        x_comb_iter_3_right = self.comb_iter_3_right(x_left)
        x_comb_iter_3 = x_comb_iter_3_left + x_comb_iter_3_right

        x_comb_iter_4_left = self.comb_iter_4_left(x_right)
        x_comb_iter_4 = x_comb_iter_4_left + x_right

        x_out = F.concat(x_left, x_comb_iter_0, x_comb_iter_1, x_comb_iter_2, x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out, x
class ReductionCell0(nn.HybridBlock):
    def __init__(self, out_channels_left, out_channels_right, norm_layer, norm_kwargs):
        super(ReductionCell0, self).__init__()

        self.conv_prev_1x1 = nn.HybridSequential(prefix='')
        self.conv_prev_1x1.add(nn.Activation('relu'))
        self.conv_prev_1x1.add(nn.Conv2D(out_channels_left, 1, strides=1, use_bias=False))
        self.conv_prev_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(out_channels_right, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.comb_iter_0_left = BranchSeparablesReduction(out_channels_right, out_channels_right, 5, 2, 2, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
        self.comb_iter_0_right = BranchSeparablesReduction(out_channels_right, out_channels_right, 7, 2, 3, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.comb_iter_1_left = MaxPoolPad()
        self.comb_iter_1_right = BranchSeparablesReduction(out_channels_right, out_channels_right, 7, 2, 3, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.comb_iter_2_left = AvgPoolPad()
        self.comb_iter_2_right = BranchSeparablesReduction(out_channels_right, out_channels_right, 5, 2, 2, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparablesReduction(out_channels_right, out_channels_right, 3, 1, 1, norm_layer=norm_layer, norm_kwargs=norm_kwargs)
        self.comb_iter_4_right = MaxPoolPad()

    def hybrid_forward(self, F, x, x_prev):
        x_left = self.conv_prev_1x1(x_prev)
        x_right = self.conv_1x1(x)

        x_comb_iter_0_left = self.comb_iter_0_left(x_right)
        x_comb_iter_0_right = self.comb_iter_0_right(x_left)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x_right)
        x_comb_iter_1_right = self.comb_iter_1_right(x_left)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x_right)
        x_comb_iter_2_right = self.comb_iter_2_right(x_left)
        x_comb_iter_2 = x_comb_iter_2_left + x_comb_iter_2_right

        x_comb_iter_3_right = self.comb_iter_3_right(x_comb_iter_0)
        x_comb_iter_3 = x_comb_iter_3_right + x_comb_iter_1

        x_comb_iter_4_left = self.comb_iter_4_left(x_comb_iter_0)
        x_comb_iter_4_right = self.comb_iter_4_right(x_right)
        x_comb_iter_4 = x_comb_iter_4_left + x_comb_iter_4_right

        x_out = F.concat(x_comb_iter_1, x_comb_iter_2, x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out, x
class ReductionCell1(nn.HybridBlock):
    def __init__(self, out_channels_left, out_channels_right, norm_layer, norm_kwargs):
        super(ReductionCell1, self).__init__()

        self.conv_prev_1x1 = nn.HybridSequential(prefix='')
        self.conv_prev_1x1.add(nn.Activation('relu'))
        self.conv_prev_1x1.add(nn.Conv2D(out_channels_left, 1, strides=1, use_bias=False))
        self.conv_prev_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.conv_1x1 = nn.HybridSequential(prefix='')
        self.conv_1x1.add(nn.Activation('relu'))
        self.conv_1x1.add(nn.Conv2D(out_channels_right, 1, strides=1, use_bias=False))
        self.conv_1x1.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.comb_iter_0_left = BranchSeparables(out_channels_right, out_channels_right, 5, 2, 2, norm_layer, norm_kwargs)
        self.comb_iter_0_right = BranchSeparables(out_channels_right, out_channels_right, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_1_left = nn.MaxPool2D(3, strides=2, padding=1)
        self.comb_iter_1_right = BranchSeparables(out_channels_right, out_channels_right, 7, 2, 3, norm_layer, norm_kwargs)

        self.comb_iter_2_left = nn.AvgPool2D(3, strides=2, padding=1, count_include_pad=False)
        self.comb_iter_2_right = BranchSeparables(out_channels_right, out_channels_right, 5, 2, 2, norm_layer, norm_kwargs)

        self.comb_iter_3_right = nn.AvgPool2D(3, strides=1, padding=1, count_include_pad=False)

        self.comb_iter_4_left = BranchSeparables(out_channels_right, out_channels_right, 3, 1, 1, norm_layer, norm_kwargs)
        self.comb_iter_4_right = nn.MaxPool2D(3, strides=2, padding=1)

    def hybrid_forward(self, F, x, x_prev):
        x_left = self.conv_prev_1x1(x_prev)
        x_right = self.conv_1x1(x)

        x_comb_iter_0_left = self.comb_iter_0_left(x_right)
        x_comb_iter_0_right = self.comb_iter_0_right(x_left)
        x_comb_iter_0 = x_comb_iter_0_left + x_comb_iter_0_right

        x_comb_iter_1_left = self.comb_iter_1_left(x_right)
        x_comb_iter_1_right = self.comb_iter_1_right(x_left)
        x_comb_iter_1 = x_comb_iter_1_left + x_comb_iter_1_right

        x_comb_iter_2_left = self.comb_iter_2_left(x_right)
        x_comb_iter_2_right = self.comb_iter_2_right(x_left)
        x_comb_iter_2 = x_comb_iter_2_left + x_comb_iter_2_right

        x_comb_iter_3_right = self.comb_iter_3_right(x_comb_iter_0)
        x_comb_iter_3 = x_comb_iter_3_right + x_comb_iter_1

        x_comb_iter_4_left = self.comb_iter_4_left(x_comb_iter_0)
        x_comb_iter_4_right = self.comb_iter_4_right(x_right)
        x_comb_iter_4 = x_comb_iter_4_left + x_comb_iter_4_right

        x_out = F.concat(x_comb_iter_1, x_comb_iter_2, x_comb_iter_3, x_comb_iter_4, dim=1)

        return x_out, x
class NASNetALarge(nn.HybridBlock):
    def __init__(self, repeat=6, penultimate_filters=4032, stem_filters=96, filters_multiplier=2, classes=1000, use_aux=False, norm_layer=nn.BatchNorm, norm_kwargs=None):
        super(NASNetALarge, self).__init__()

        filters = penultimate_filters // 24

        self.conv0 = nn.HybridSequential(prefix='')
        self.conv0.add(nn.Conv2D(stem_filters, 3, padding=0, strides=2, use_bias=False))
        self.conv0.add(norm_layer(momentum=0.1, epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))

        self.cell_stem_0 = CellStem0(stem_filters, norm_layer, norm_kwargs, num_filters=filters // (filters_multiplier ** 2))
        self.cell_stem_1 = CellStem1(filters // filters_multiplier, norm_layer, norm_kwargs)

        self.norm_1 = nn.HybridSequential(prefix='')
        self.norm_1.add(FirstCell(out_channels_left=filters//2, out_channels_right=filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        for _ in range(repeat - 1):
            self.norm_1.add(NormalCell(out_channels_left=filters, out_channels_right=filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

        self.reduction_cell_0 = ReductionCell0(out_channels_left=2*filters, out_channels_right=2*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.norm_2 = nn.HybridSequential(prefix='')
        self.norm_2.add(FirstCell(out_channels_left=filters, out_channels_right=2*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        for _ in range(repeat - 1):
            self.norm_2.add(NormalCell(out_channels_left=2*filters, out_channels_right=2*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

        if use_aux:
            self.out_aux = nn.HybridSequential(prefix='')
            self.out_aux.add(nn.Conv2D(filters // 3, kernel_size=1, use_bias=False))
            self.out_aux.add(norm_layer(epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
            self.out_aux.add(nn.Activation('relu'))
            self.out_aux.add(nn.Conv2D(2*filters, kernel_size=5, use_bias=False))
            self.out_aux.add(norm_layer(epsilon=0.001, **({} if norm_kwargs is None else norm_kwargs)))
            self.out_aux.add(nn.Activation('relu'))
            self.out_aux.add(nn.Dense(classes))
        else:
            self.out_aux = None

        self.reduction_cell_1 = ReductionCell1(out_channels_left=4*filters, out_channels_right=4*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.norm_3 = nn.HybridSequential(prefix='')
        self.norm_3.add(FirstCell(out_channels_left=2*filters, out_channels_right=4*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))
        for _ in range(repeat - 1):
            self.norm_3.add(NormalCell(out_channels_left=4*filters, out_channels_right=4*filters, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

        self.out = nn.HybridSequential(prefix='')
        self.out.add(nn.Activation('relu'))
        self.out.add(nn.GlobalAvgPool2D())
        self.out.add(nn.Dropout(0.5))
        self.out.add(nn.Dense(classes))

    def hybrid_forward(self, F, x):
        x_conv0 = self.conv0(x)
        x_stem_0 = self.cell_stem_0(x_conv0)
        x_stem_1 = self.cell_stem_1(x_conv0, x_stem_0)

        x = x_stem_1
        x_prev = x_stem_0
        for cell in self.norm_1._children.values():
            x, x_prev = cell(x, x_prev)
        x, x_prev = self.reduction_cell_0(x, x_prev)
        for cell in self.norm_2._children.values():
            x, x_prev = cell(x, x_prev)
        if self.out_aux:
            x_aux = F.contrib.AdaptiveAvgPooling2D(x, output_size=5)
            x_aux = self.out_aux(x_aux)
        x, x_prev = self.reduction_cell_1(x, x_prev)
        for cell in self.norm_3._children.values():
            x, x_prev = cell(x, x_prev)

        x = self.out(x)
        if self.out_aux:
            return x, x_aux
        else:
            return x
def get_nasnet(repeat=6, penultimate_filters=4032,class_num=1000,**kwargs):
    net = NASNetALarge(repeat=repeat, penultimate_filters=penultimate_filters,classes=class_num, **kwargs)
    return net

18.HrNet的实现

代码实现(gluon):

def _conv3x3(channels, stride, in_channels):
    return nn.Conv2D(channels, kernel_size=3, strides=stride, padding=1, use_bias=False, in_channels=in_channels)
class BasicBlockV1(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(BasicBlockV1, self).__init__(**kwargs)
        self.body = nn.HybridSequential(prefix='')
        self.body.add(_conv3x3(channels, stride, in_channels))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(_conv3x3(channels, 1, channels))
        if not last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            self.downsample.add(nn.Conv2D(channels, kernel_size=1, strides=stride, use_bias=False, in_channels=in_channels))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x
        x = self.body(x)
        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(residual+x, act_type='relu')

        return x
class BottleneckV1(nn.HybridBlock):
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(BottleneckV1, self).__init__(**kwargs)
        self.body = nn.HybridSequential(prefix='')
        self.body.add(nn.Conv2D(channels//4, kernel_size=1, strides=stride))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(_conv3x3(channels//4, 1, channels//4))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(nn.Conv2D(channels, kernel_size=1, strides=1))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if not last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            self.downsample.add(nn.Conv2D(channels, kernel_size=1, strides=stride, use_bias=False, in_channels=in_channels))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x

        x = self.body(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(x + residual, act_type='relu')
        return x
class HRBasicBlock(BasicBlockV1):
    expansion = 1
class HRBottleneck(BottleneckV1):
    expansion = 4
class OrigHRBottleneck(nn.HybridBlock):
    expansion = 4
    def __init__(self, channels, stride, downsample=False, in_channels=0, last_gamma=False, use_se=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(OrigHRBottleneck, self).__init__(**kwargs)
        self.body = nn.HybridSequential(prefix='')
        # add use_bias=False here to match with the original implementation
        self.body.add(nn.Conv2D(channels//4, kernel_size=1, strides=1, use_bias=False))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        self.body.add(_conv3x3(channels//4, stride, channels//4))
        self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        self.body.add(nn.Activation('relu'))
        # add use_bias=False here to match with the original implementation
        self.body.add(nn.Conv2D(channels, kernel_size=1, strides=1, use_bias=False))

        if use_se:
            self.se = nn.HybridSequential(prefix='')
            self.se.add(nn.Dense(channels // 16, use_bias=False))
            self.se.add(nn.Activation('relu'))
            self.se.add(nn.Dense(channels, use_bias=False))
            self.se.add(nn.Activation('sigmoid'))
        else:
            self.se = None

        if not last_gamma:
            self.body.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.body.add(norm_layer(gamma_initializer='zeros', **({} if norm_kwargs is None else norm_kwargs)))

        if downsample:
            self.downsample = nn.HybridSequential(prefix='')
            self.downsample.add(nn.Conv2D(channels, kernel_size=1, strides=stride, use_bias=False, in_channels=in_channels))
            self.downsample.add(norm_layer(**({} if norm_kwargs is None else norm_kwargs)))
        else:
            self.downsample = None

    def hybrid_forward(self, F, x):
        residual = x

        x = self.body(x)

        if self.se:
            w = F.contrib.AdaptiveAvgPooling2D(x, output_size=1)
            w = self.se(w)
            x = F.broadcast_mul(x, w.expand_dims(axis=2).expand_dims(axis=2))

        if self.downsample:
            residual = self.downsample(residual)

        x = F.Activation(x + residual, act_type='relu')
        return x
class HighResolutionModule(nn.HybridBlock):
    def __init__(self, num_branches, blocks, num_blocks, num_channels, fuse_method, num_inchannels=None, multi_scale_output=True, interp_type='nearest', norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(HighResolutionModule, self).__init__(**kwargs)

        if num_inchannels is not None:
            self.num_inchannels = num_inchannels
        else:
            self.num_inchannels = num_channels

        self.fuse_method = fuse_method
        self.num_branches = num_branches

        self.multi_scale_output = multi_scale_output
        self.interp_type = interp_type

        self.branches = self._make_branches(
            num_branches, blocks, num_blocks, num_channels)
        self.fuse_layers = self._make_fuse_layers(norm_layer=norm_layer, norm_kwargs=norm_kwargs)

    def _make_one_branch(self, branch_index, block, num_blocks, num_channels, stride=1):
        downsample = stride != 1 or self.num_inchannels[branch_index] != num_channels[branch_index] * block.expansion

        layers = nn.HybridSequential()
        layers.add(block(num_channels[branch_index]* block.expansion, stride, downsample, self.num_inchannels[branch_index]))
        self.num_inchannels[branch_index] = num_channels[branch_index] * block.expansion
        for i in range(1, num_blocks[branch_index]):
            layers.add(block(num_channels[branch_index]* block.expansion, 1, False, self.num_inchannels[branch_index]))

        return layers


    def _make_branches(self, num_branches, block, num_blocks, num_channels):
        branches = nn.HybridSequential()

        for i in range(num_branches):
            branches.add(
                self._make_one_branch(i, block, num_blocks, num_channels)
            )
        return branches

    def _make_fuse_layers(self, norm_layer=nn.BatchNorm, norm_kwargs=None):
        if self.num_branches == 1:
            return None

        num_branches = self.num_branches
        num_inchannels = self.num_inchannels
        fuse_layers = nn.HybridSequential()
        for i in range(num_branches if self.multi_scale_output else 1):
            fuse_layer = nn.HybridSequential()
            for j in range(num_branches):
                if j > i:
                    seq = nn.HybridSequential()

                    seq.add(
                        nn.Conv2D(num_inchannels[i], 1, 1, 0, use_bias=False),
                        norm_layer(**({} if norm_kwargs is None else norm_kwargs))
                    )
                    fuse_layer.add(seq)
                elif j == i:
                    fuse_layer.add(contrib.nn.Identity())
                else:
                    conv3x3s = nn.HybridSequential()
                    for k in range(i-j):
                        if k == i - j - 1:
                            num_outchannels_conv3x3 = num_inchannels[i]
                            conv3x3s.add(
                                nn.Conv2D(num_outchannels_conv3x3, 3, 2, 1, use_bias=False),
                                norm_layer(**({} if norm_kwargs is None else norm_kwargs))
                            )
                        else:
                            num_outchannels_conv3x3 = num_inchannels[j]
                            conv3x3s.add(
                                nn.Conv2D(num_outchannels_conv3x3, 3, 2, 1, use_bias=False),
                                norm_layer(**({} if norm_kwargs is None else norm_kwargs)),
                                nn.Activation('relu')
                            )
                    fuse_layer.add(conv3x3s)
            fuse_layers.add(fuse_layer)

        return fuse_layers

    def get_num_inchannels(self):
        return self.num_inchannels

    def hybrid_forward(self, F, x, *args, **kwargs):
        x = self.branches[0](x)
        if self.num_branches == 1:
            return [x]

        X = []
        X.append(x)

        for i in range(1, self.num_branches):
            X.append(self.branches[i](args[i-1]))

        x_fuse = []
        for i in range(len(self.fuse_layers)):
            y = X[0] if i == 0 else self.fuse_layers[i][0](X[0])
            for j in range(1, self.num_branches):
                if j > i:
                    if self.interp_type == 'nearest':
                        y = F.broadcast_add(y, F.UpSampling(
                            self.fuse_layers[i][j](X[j]),
                            scale=2**(j-i),
                            sample_type='nearest'))
                    elif self.interp_type == 'bilinear':
                        y = F.broadcast_add(y, F.contrib.BilinearResize2D(
                            self.fuse_layers[i][j](X[j]),
                            scale_height=2**(j-i),
                            scale_width=2**(j-i),
                            align_corners=False
                        ))
                    elif self.interp_type == 'bilinear_like':
                        y = F.broadcast_add(y, F.contrib.BilinearResize2D(
                            self.fuse_layers[i][j](X[j]),
                            like=X[i],
                            mode='like',
                            align_corners=False
                        ))
                    else:
                        raise NotImplementedError

                else:
                    y = y + self.fuse_layers[i][j](X[j])
            x_fuse.append(F.relu(y))

        return x_fuse
BLOCKS_DICT = {
    'BASIC': HRBasicBlock,
    'BOTTLENECK': OrigHRBottleneck
}
class HighResolutionBaseNet(nn.HybridBlock):
    def __init__(self, cfg, stage_interp_type='nearst', norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        self.stage_interp_type = stage_interp_type
        super(HighResolutionBaseNet, self).__init__()

        self.conv1 = nn.Conv2D(64, kernel_size=3, strides=2, padding=1, use_bias=False)
        self.bn1 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))
        self.conv2 = nn.Conv2D(64, kernel_size=3, strides=2, padding=1, use_bias=False)
        self.bn2 = norm_layer(**({} if norm_kwargs is None else norm_kwargs))

        self.stage1_cfg = cfg[0]
        num_channels = self.stage1_cfg[3][0]
        block = BLOCKS_DICT[self.stage1_cfg[1]]
        num_blocks = self.stage1_cfg[2][0]
        self.layer1 = self._make_layer(block, num_channels, num_blocks, inplanes=64)
        stage1_out_channel = block.expansion*num_channels

        self.stage2_cfg = cfg[1]
        num_channels = self.stage2_cfg[3]
        block = BLOCKS_DICT[self.stage2_cfg[1]]
        num_channels = [num_channels[i] * block.expansion for i in range(len(num_channels))]
        self.transition1 = self._make_transition_layer([stage1_out_channel], num_channels, norm_layer, norm_kwargs)
        self.stage2, pre_stage_channels = self._make_stage(self.stage2_cfg, num_channels)

        self.stage3_cfg = cfg[2]
        num_channels = self.stage3_cfg[3]
        block = BLOCKS_DICT[self.stage3_cfg[1]]
        num_channels = [
            num_channels[i] * block.expansion for i in range(len(num_channels))]
        self.transition2 = self._make_transition_layer(
            pre_stage_channels, num_channels, norm_layer, norm_kwargs)
        self.stage3, pre_stage_channels = self._make_stage(
            self.stage3_cfg, num_channels)

        self.stage4_cfg = cfg[3]
        num_channels = self.stage4_cfg[3]
        block = BLOCKS_DICT[self.stage4_cfg[1]]
        num_channels = [
            num_channels[i] * block.expansion for i in range(len(num_channels))]
        self.transition3 = self._make_transition_layer(
            pre_stage_channels, num_channels, norm_layer, norm_kwargs)
        self.stage4, pre_stage_channels = self._make_stage(
            self.stage4_cfg, num_channels, multi_scale_output=True)

        self.pre_stage_channels = pre_stage_channels

    def _make_transition_layer(self, num_channels_pre_layer, num_channels_cur_layer, norm_layer=nn.BatchNorm, norm_kwargs=None):
        num_branches_cur = len(num_channels_cur_layer)
        num_branches_pre = len(num_channels_pre_layer)

        transition_layers = nn.HybridSequential()

        for i in range(num_branches_cur):
            if i < num_branches_pre:
                if num_channels_cur_layer[i] != num_channels_pre_layer[i]:
                    transition_layer = nn.HybridSequential()
                    transition_layer.add(
                        nn.Conv2D(num_channels_cur_layer[i], 3, 1, 1, use_bias=False, in_channels=num_channels_pre_layer[i]),
                        norm_layer(**({} if norm_kwargs is None else norm_kwargs)),
                        nn.Activation('relu')
                    )
                    transition_layers.add(transition_layer)
                else:
                    transition_layers.add(contrib.nn.Identity())
            else:
                conv3x3s = nn.HybridSequential()
                for j in range(i+1-num_branches_pre):
                    inchannels = num_channels_pre_layer[-1]
                    outchannels = num_channels_cur_layer[i] if j == i-num_branches_pre else inchannels
                    cba = nn.HybridSequential()
                    cba.add(
                        nn.Conv2D(outchannels, 3, 2, 1, use_bias=False, in_channels=inchannels),
                        norm_layer(**({} if norm_kwargs is None else norm_kwargs)),
                        nn.Activation('relu')
                    )
                    conv3x3s.add(cba)
                transition_layers.add(conv3x3s)

        return transition_layers

    def _make_layer(self, block, planes, blocks, inplanes=0, stride=1):
        downsample = stride != 1 or inplanes != planes * block.expansion

        layers = nn.HybridSequential()
        layers.add(block(planes* block.expansion, stride, downsample, inplanes))
        for i in range(1, blocks):
            layers.add(block(planes* block.expansion, 1, False, inplanes))

        return layers

    def _make_stage(self, layer_config, num_inchannels, multi_scale_output=True):
        num_modules = layer_config[0]
        num_blocks = layer_config[2]
        num_branches = len(num_blocks)
        num_channels = layer_config[3]
        block = BLOCKS_DICT[layer_config[1]]
        fuse_method = layer_config[4]

        blocks = nn.HybridSequential()
        for i in range(num_modules):
            # multi_scale_output is only used last module
            if not multi_scale_output and i == num_modules - 1:
                reset_multi_scale_output = False
            else:
                reset_multi_scale_output = True
            hrm = HighResolutionModule(num_branches, block, num_blocks, num_channels, fuse_method, num_inchannels, reset_multi_scale_output, self.stage_interp_type)
            blocks.add(hrm)
            num_inchannels = hrm.get_num_inchannels()

        return blocks, num_inchannels

    def hybrid_forward(self, F, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = F.relu(x)
        x = self.conv2(x)
        x = self.bn2(x)
        x = F.relu(x)
        x = self.layer1(x)
        x_list = []

        for i in range(len(self.stage2_cfg[2])):
            x_list.append(self.transition1[i](x))

        y_list = x_list
        for s in self.stage2:
            y_list = s(*y_list)

        x_list = []
        for i in range(len(self.stage3_cfg[2])):
            if i < len(self.stage2_cfg[2]):
                x_list.append(self.transition2[i](y_list[i]))
            else:
                x_list.append(self.transition2[i](y_list[-1]))

        y_list = x_list
        for s in self.stage3:
            y_list = s(*y_list)

        x_list = []
        for i in range(len(self.stage4_cfg[2])):
            if i < len(self.stage3_cfg[2]):
                x_list.append(self.transition3[i](y_list[i]))
            else:
                x_list.append(self.transition3[i](y_list[-1]))

        y_list = x_list
        for s in self.stage4:
            y_list = s(*y_list)

        return y_list
class HighResolutionClsNet(HighResolutionBaseNet):
    def __init__(self, config, stage_interp_type='nearest', norm_layer=nn.BatchNorm, norm_kwargs=None,num_classes=1000, **kwargs):
        super(HighResolutionClsNet, self).__init__(config, stage_interp_type=stage_interp_type, norm_layer=norm_layer, norm_kwargs=norm_kwargs)

        self.incre_blocks, self.downsamp_blocks, self.final_layer = self._make_head(self.pre_stage_channels, norm_layer, norm_kwargs)
        self.avg = nn.GlobalAvgPool2D()
        self.classifier = nn.Dense(num_classes, in_units=2048)

    def hybrid_forward(self, F, x):
        y_list = super(HighResolutionClsNet, self).hybrid_forward(F, x)

        y = self.incre_blocks[0](y_list[0])
        for i in range(len(self.downsamp_blocks)):
            y = self.incre_blocks[i+1](y_list[i+1]) + self.downsamp_blocks[i](y)

        y = self.final_layer(y)
        y = self.avg(y)
        y = self.classifier(y)

        return y

    def _make_head(self, pre_stage_channels, norm_layer=nn.BatchNorm, norm_kwargs=None):
        head_block = BLOCKS_DICT['BOTTLENECK']
        head_channels = [32, 64, 128, 256]

        incre_blocks = nn.HybridSequential()
        for i, channels in enumerate(pre_stage_channels):
            incre_block = self._make_layer(head_block, head_channels[i], 1, channels, stride=1)
            incre_blocks.add(incre_block)

        downsamp_blocks = nn.HybridSequential()
        for i in range(len(pre_stage_channels)-1):
            in_channels = head_channels[i] * head_block.expansion
            out_channels = head_channels[i+1] * head_block.expansion

            downsamp_block = nn.HybridSequential()
            downsamp_block.add(
                nn.Conv2D(out_channels, 3, 2, 1, in_channels=in_channels),
                norm_layer(**({} if norm_kwargs is None else norm_kwargs)),
                nn.Activation('relu')
            )
            downsamp_blocks.add(downsamp_block)

        final_layer = nn.HybridSequential()
        final_layer.add(
            nn.Conv2D(2048, 1, 1, 0, in_channels=head_channels[3] * head_block.expansion),
            norm_layer(**({} if norm_kwargs is None else norm_kwargs)),
            nn.Activation('relu')
        )

        return incre_blocks, downsamp_blocks, final_layer
HRNET_SPEC = {}
HRNET_SPEC['w18'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],              'SUM'),
    (1,    'BASIC',      [4]*2,         [18, 36],          'SUM'),
    (4,    'BASIC',      [4]*3,         [18, 36, 72],      'SUM'),
    (3,    'BASIC',      [4]*4,         [18, 36, 72, 144], 'SUM')
]
HRNET_SPEC['w18_small_v1'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [1],           [32],              'SUM'),
    (1,    'BASIC',      [2]*2,         [16, 32],          'SUM'),
    (1,    'BASIC',      [2]*3,         [16, 32, 64],      'SUM'),
    (1,    'BASIC',      [2]*4,         [16, 32, 64, 128], 'SUM')
]
HRNET_SPEC['w18_small_v2'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [2],           [64],               'SUM'),
    (1,    'BASIC',      [2]*2,         [18, 36],           'SUM'),
    (3,    'BASIC',      [2]*3,         [18, 36, 72],       'SUM'),
    (2,    'BASIC',      [2]*4,         [18, 36, 72, 144],  'SUM')
]
HRNET_SPEC['w30'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],               'SUM'),
    (1,    'BASIC',      [4]*2,         [30, 60],           'SUM'),
    (4,    'BASIC',      [4]*3,         [30, 60, 120],      'SUM'),
    (3,    'BASIC',      [4]*4,         [30, 60, 120, 240], 'SUM')
]
HRNET_SPEC['w32'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],               'SUM'),
    (1,    'BASIC',      [4]*2,         [32, 64],           'SUM'),
    (4,    'BASIC',      [4]*3,         [32, 64, 128],      'SUM'),
    (3,    'BASIC',      [4]*4,         [32, 64, 128, 256], 'SUM')
]
HRNET_SPEC['w40'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],               'SUM'),
    (1,    'BASIC',      [4]*2,         [40, 80],           'SUM'),
    (4,    'BASIC',      [4]*3,         [40, 80, 160],      'SUM'),
    (3,    'BASIC',      [4]*4,         [40, 80, 160, 320], 'SUM')
]
HRNET_SPEC['w44'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],               'SUM'),
    (1,    'BASIC',      [4]*2,         [44, 88],           'SUM'),
    (4,    'BASIC',      [4]*3,         [44, 88, 176],      'SUM'),
    (3,    'BASIC',      [4]*4,         [44, 88, 176, 352], 'SUM')
]
HRNET_SPEC['w48'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],               'SUM'),
    (1,    'BASIC',      [4]*2,         [48, 96],           'SUM'),
    (4,    'BASIC',      [4]*3,         [48, 96, 192],      'SUM'),
    (3,    'BASIC',      [4]*4,         [48, 96, 192, 384], 'SUM')
]
HRNET_SPEC['w64'] = [
    #modules, block_type, blocks, channels, fuse_method
    (1,    'BOTTLENECK', [4],           [64],                'SUM'),
    (1,    'BASIC',      [4]*2,         [64, 128],           'SUM'),
    (4,    'BASIC',      [4]*3,         [64, 128, 256],      'SUM'),
    (3,    'BASIC',      [4]*4,         [64, 128, 256, 512], 'SUM')
]
def get_hrnet(model_name, num_classes=1000, stage_interp_type='nearest', norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
    spec = HRNET_SPEC[model_name]
    net = HighResolutionClsNet(spec, stage_interp_type, norm_layer, norm_kwargs, num_classes = num_classes, **kwargs)
    return net

19.DLA的实现

代码实现(gluon):

def conv3x3_dla(in_planes, out_planes, stride=1):
    return nn.Conv2D(channels=out_planes, kernel_size=3, strides=stride, padding=1, use_bias=False, in_channels=in_planes)
class BasicBlock(nn.HybridBlock):
    def __init__(self, inplanes, planes, stride=1, dilation=1, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(BasicBlock, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        with self.name_scope():
            self.conv1 = nn.Conv2D(in_channels=inplanes, channels=planes, kernel_size=3, strides=stride, padding=dilation, use_bias=False, dilation=dilation)
            self.bn1 = norm_layer(in_channels=planes, **norm_kwargs)
            self.relu = nn.Activation('relu')
            self.conv2 = nn.Conv2D(in_channels=planes, channels=planes, kernel_size=3, strides=1, padding=dilation, use_bias=False, dilation=dilation)
            self.bn2 = norm_layer(in_channels=planes, **norm_kwargs)
            self.stride = stride

    def hybrid_forward(self, F, x, residual=None):
        if residual is None:
            residual = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)

        out = out + residual
        out = self.relu(out)

        return out
class Bottleneck_dla(nn.HybridBlock):
    expansion = 2
    def __init__(self, inplanes, planes, stride=1, dilation=1, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(Bottleneck_dla, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        expansion = Bottleneck_dla.expansion
        bottle_planes = planes // expansion
        with self.name_scope():
            self.conv1 = nn.Conv2D(in_channels=inplanes, channels=bottle_planes, kernel_size=1, use_bias=False)
            self.bn1 = norm_layer(in_channels=bottle_planes, **norm_kwargs)
            self.conv2 = nn.Conv2D(in_channels=bottle_planes, channels=bottle_planes, kernel_size=3, strides=stride, padding=dilation, use_bias=False, dilation=dilation)
            self.bn2 = norm_layer(in_channels=bottle_planes, **norm_kwargs)
            self.conv3 = nn.Conv2D(in_channels=bottle_planes, channels=planes, kernel_size=1, use_bias=False)
            self.bn3 = norm_layer(**norm_kwargs)
            self.relu = nn.Activation('relu')
            self.stride = stride

    def hybrid_forward(self, F, x, residual=None):
        if residual is None:
            residual = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)
        out = self.relu(out)

        out = self.conv3(out)
        out = self.bn3(out)

        out = out + residual
        out = self.relu(out)

        return out
class BottleneckX(nn.HybridBlock):
    expansion = 2
    cardinality = 32

    def __init__(self, inplanes, planes, stride=1, dilation=1, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(BottleneckX, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        cardinality = BottleneckX.cardinality
        bottle_planes = planes * cardinality // 32
        with self.name_scope():
            self.conv1 = nn.Conv2D(in_channels=inplanes, channels=bottle_planes, kernel_size=1, use_bias=False)
            self.bn1 = norm_layer(in_channels=bottle_planes, **norm_kwargs)
            self.conv2 = nn.Conv2D(in_channels=bottle_planes, channels=bottle_planes, kernel_size=3, strides=stride, padding=dilation, use_bias=False, dilation=dilation, groups=cardinality)
            self.bn2 = norm_layer(in_channels=bottle_planes, **norm_kwargs)
            self.conv3 = nn.Conv2D(in_channels=bottle_planes, channels=planes, kernel_size=1, use_bias=False)
            self.bn3 = norm_layer(**norm_kwargs)
            self.relu = nn.Activation('relu')
            self.stride = stride

    def hybrid_forward(self, F, x, residual=None):
        if residual is None:
            residual = x

        out = self.conv1(x)
        out = self.bn1(out)
        out = self.relu(out)

        out = self.conv2(out)
        out = self.bn2(out)
        out = self.relu(out)

        out = self.conv3(out)
        out = self.bn3(out)

        out = out + residual
        out = self.relu(out)

        return out
class Root(nn.HybridBlock):
    def __init__(self, in_channels, out_channels, kernel_size, residual, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(Root, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        with self.name_scope():
            self.conv = nn.Conv2D(in_channels=in_channels, channels=out_channels, kernel_size=1, strides=1, use_bias=False, padding=(kernel_size - 1) // 2)
            self.bn = norm_layer(in_channels=out_channels, **norm_kwargs)
            self.relu = nn.Activation('relu')
            self.residual = residual

    def hybrid_forward(self, F, *x):
        children = x
        x = self.conv(F.concat(*x, dim=1))
        x = self.bn(x)
        if self.residual:
            x = x + children[0]
        x = self.relu(x)

        return x
class Tree(nn.HybridBlock):
    def __init__(self, levels, block, in_channels, out_channels, stride=1, level_root=False, root_dim=0, root_kernel_size=1, dilation=1, root_residual=False, norm_layer=nn.BatchNorm, norm_kwargs=None, **kwargs):
        super(Tree, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        if root_dim == 0:
            root_dim = 2 * out_channels
        if level_root:
            root_dim = root_dim + in_channels
        with self.name_scope():
            self.downsample = nn.HybridSequential()
            self.project = nn.HybridSequential()
            if levels == 1:
                self.tree1 = block(in_channels, out_channels, stride, dilation=dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='block_tree_1_')
                self.tree2 = block(out_channels, out_channels, 1, dilation=dilation, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='block_tree_2_')
                if in_channels != out_channels:
                    self.project.add(*[
                        nn.Conv2D(in_channels=in_channels, channels=out_channels, kernel_size=1, strides=1, use_bias=False, prefix='proj_conv0_'),
                        norm_layer(in_channels=out_channels, prefix='proj_bn0_', **norm_kwargs)])
            else:
                self.tree1 = Tree(levels - 1, block, in_channels, out_channels, stride, root_dim=0, root_kernel_size=root_kernel_size, dilation=dilation, root_residual=root_residual, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='tree_1_')
                self.tree2 = Tree(levels - 1, block, out_channels, out_channels, root_dim=root_dim + out_channels, root_kernel_size=root_kernel_size, dilation=dilation, root_residual=root_residual, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='tree_2_')
            if levels == 1:
                self.root = Root(root_dim, out_channels, root_kernel_size, root_residual, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='root_')
            self.level_root = level_root
            self.root_dim = root_dim
            self.levels = levels
            if stride > 1:
                self.downsample.add(nn.MaxPool2D(stride, strides=stride, prefix='maxpool'))

    def hybrid_forward(self, F, x, residual=None, children=None):
        children = [] if children is None else children
        bottom = self.downsample(x)
        residual = self.project(bottom)
        if self.level_root:
            children.append(bottom)
        x1 = self.tree1(x, residual)
        if self.levels == 1:
            x2 = self.tree2(x1)
            x = self.root(x2, x1, *children)
        else:
            children.append(x1)
            x = self.tree2(x1, None, children)
        return x
class DLA(nn.HybridBlock):
    def __init__(self, levels, channels, classes=1000, block=BasicBlock, momentum=0.9, norm_layer=nn.BatchNorm, norm_kwargs=None, residual_root=False, linear_root=False, use_feature=False, **kwargs):
        super(DLA, self).__init__(**kwargs)
        if norm_kwargs is None:
            norm_kwargs = {}
        norm_kwargs['momentum'] = momentum
        self._use_feature = use_feature
        self.channels = channels
        self.base_layer = nn.HybridSequential('base')
        self.base_layer.add(nn.Conv2D(in_channels=3, channels=channels[0], kernel_size=7, strides=1, padding=3, use_bias=False))
        self.base_layer.add(norm_layer(in_channels=channels[0], **norm_kwargs))
        self.base_layer.add(nn.Activation('relu'))

        self.level0 = self._make_conv_level(channels[0], channels[0], levels[0], norm_layer, norm_kwargs)
        self.level1 = self._make_conv_level(channels[0], channels[1], levels[1], norm_layer, norm_kwargs, stride=2)
        self.level2 = Tree(levels[2], block, channels[1], channels[2], 2, level_root=False, root_residual=residual_root, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='level2_')
        self.level3 = Tree(levels[3], block, channels[2], channels[3], 2, level_root=True, root_residual=residual_root, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='level3_')
        self.level4 = Tree(levels[4], block, channels[3], channels[4], 2, level_root=True, root_residual=residual_root, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='level4_')
        self.level5 = Tree(levels[5], block, channels[4], channels[5], 2, level_root=True, root_residual=residual_root, norm_layer=norm_layer, norm_kwargs=norm_kwargs, prefix='level5_')

        if not self._use_feature:
            self.global_avg_pool = nn.GlobalAvgPool2D()
            self.fc = nn.Dense(units=classes)

    def _make_level(self, block, inplanes, planes, blocks, norm_layer, norm_kwargs, stride=1):
        downsample = None
        if stride != 1 or inplanes != planes:
            downsample = nn.HybridSequential()
            downsample.add(*[
                nn.MaxPool2D(stride, strides=stride),
                nn.Conv2D(channels=planes, in_channels=inplanes, kernel_size=1, strides=1, use_bias=False),
                norm_layer(in_channels=planes, **norm_kwargs)])

        layers = []
        layers.append(block(inplanes, planes, stride, norm_layer=norm_layer, norm_kwargs=norm_kwargs, downsample=downsample))
        for _ in range(1, blocks):
            layers.append(block(inplanes, planes, norm_layer=norm_layer, norm_kwargs=norm_kwargs))

        curr_level = nn.HybridSequential()
        curr_level.add(*layers)
        return curr_level

    def _make_conv_level(self, inplanes, planes, convs, norm_layer, norm_kwargs, stride=1, dilation=1):
        modules = []
        for i in range(convs):
            modules.extend([
                nn.Conv2D(in_channels=inplanes, channels=planes, kernel_size=3, strides=stride if i == 0 else 1, padding=dilation, use_bias=False, dilation=dilation),
                norm_layer(**norm_kwargs),
                nn.Activation('relu')])
            inplanes = planes
        curr_level = nn.HybridSequential()
        curr_level.add(*modules)
        return curr_level

    def hybrid_forward(self, F, x):
        y = []
        x = self.base_layer(x)
        for i in range(6):
            x = getattr(self, 'level{}'.format(i))(x)
            if self._use_feature:
                y.append(x)
            else:
                y.append(F.flatten(self.global_avg_pool(x)))
        if self._use_feature:
            return y
        flat = F.concat(*y, dim=1)
        out = self.fc(flat)
        return out
def get_dla(class_num):
    net = DLA(levels=[1, 1, 1, 2, 2, 1], channels=[16, 32, 64, 128, 256, 512],classes=class_num, block=BasicBlock)
    return net

二、数据集的准备

1.数据集描述

  因ImageNet数据集过于庞大,在学习训练上比较费资源,因此本次数据集使用中国象棋数据集,中国象棋红黑棋子一共有14种类,经过预处理后会得到单独象棋图像,如图:
在这里插入图片描述

2.数据集准备

  数据集主要以文件夹形式进行区分,每一个类别代表一个文件夹,如图:
在这里插入图片描述
  在训练的时候直接把此文件夹目录放入代码中即可,对于训练集和验证集,代码会在这里的文件夹中做一个根据提供的划分比例随机拆分。


三、模型训练/预测

1.基础信息配置

self.ctx = [mx.gpu(USE_GPU[i]) for i in range(len(USE_GPU))] if len(USE_GPU) > 0 else [mx.cpu()]
self.image_size = image_size

2.数据加载器

def get_data_iter(TrainDir,ValDir,image_size,batch_size,num_workers):
    ValDir = TrainDir if ValDir is None else ValDir
    train_ds = gluon.data.vision.ImageFolderDataset(TrainDir, flag=1)
    test_ds = gluon.data.vision.ImageFolderDataset(ValDir, flag=1)
    class_names = train_ds.synsets

    transform_train = transforms.Compose([
        transforms.Resize(image_size),  #
        # transforms.RandomResizedCrop(self.image_size),
        transforms.RandomFlipLeftRight(),
        transforms.RandomFlipTopBottom(),  #
        transforms.RandomBrightness(0.5),  #
        transforms.RandomColorJitter(brightness=0.4, contrast=0.4, saturation=0.4),
        transforms.RandomLighting(alpha=0.1),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    transform_val = transforms.Compose([
        transforms.Resize(image_size),
        # transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    train_data = gluon.data.DataLoader(train_ds.transform_first(transform_train), batch_size=batch_size, shuffle=True, num_workers=num_workers)
    val_data = gluon.data.DataLoader(test_ds.transform_first(transform_val), batch_size=batch_size, shuffle=False, num_workers=num_workers)
    return train_data,val_data,class_names

3.模型构建

self.model,self.model_name = get_classification_model(self.ModelType,len(self.class_names),version,num_layers,self.image_size,self.ctx,Pre_Model)

4.模型训练

with tqdm(total=len(self.train_iter), desc=f'Epoch_Train {epoch + 1}/{trainNum}', postfix=dict,mininterval=0.3) as pbar:
   for i, batch in enumerate(self.train_iter):
        data = gluon.utils.split_and_load(batch[0], ctx_list=self.ctx, batch_axis=0, even_split=False)
        label = gluon.utils.split_and_load(batch[1], ctx_list=self.ctx, batch_axis=0, even_split=False)
        with autograd.record():
            outputs = [self.model(X) for X in data]
            loss = [L(yhat, y) for yhat, y in zip(outputs, label)]
        for l in loss:
            l.backward()

        trainer.step(batch[0].shape[0])
        train_loss += sum([l.mean().asscalar() for l in loss]) / len(loss)
        metric.update(label, outputs)

        pbar.set_postfix(**{'train_loss': train_loss / (i+1),'step/s': time.time() - start_time})
        pbar.update(1)
        start_time = time.time()
_, train_acc = metric.get()

5.模型预测

def predict(self,image):
    start_timer = time.time()
    img = nd.array(image)
    img = self.transform_test(img)
    img = img.expand_dims(axis=0)
    # img = nd.transpose(img, (2, 0, 1))
    img = img.as_in_context(self.ctx[0])
    out = self.model(img)
    preds = []
    preds.extend(out.argmax(axis=1).astype(int).asnumpy())
    out = nd.SoftmaxActivation(out)
    out = out.asnumpy()[0]
    return out[preds[0]],self.class_names[str(preds[0])],time.time()-start_timer

四、模型主入口

if __name__ == '__main__':
    RunModel = False

    if RunModel:
        ctu = Ctu_Classification(USE_GPU=[0], image_size=224)
        ctu.InitModel(r"E:\Ctu\Ctu_Project\WZ_DL\DataSet\DataSet_Classification_Chess\DataImage", batch_size=4, Pre_Model=None,ModeType=16, version=None,num_layers=0.25)
        ctu.train(trainNum=500, learning_rate=0.001, optim=0, ModelPath="./Classification_Model",StopNum=30)
        del ctu
    else:
        ctu = Ctu_Classification(USE_GPU=[0])
        ctu.LoadModel(ModelPath="./Classification_Model_mobilenetv1_0.25")
        cv2.namedWindow("origin", 0)
        cv2.resizeWindow("origin", 640, 480)
        for root, dirs, files in os.walk(r"E:\Ctu\Ctu_Project\WZ_DL\DataSet\DataSet_Classification_Chess\test"):
            for f in files:
                img_cv = cv2_imread(os.path.join(root, f), is_color=True, model=True)
                res = ctu.predict(img_cv)
                print(f, res)
                cv2.imshow("origin", img_cv)
                cv2.waitKey()
        del ctu

你可能感兴趣的:(深度学习-mxnet,mxnet,gluon,图像分类,深度学习)