【神经网络】(13) ShuffleNetV2 代码复现,网络解析,附Tensorflow完整代码

各位同学好,今天和大家分享一下如何使用 Tensorflow 复现轻量化神经网络 ShuffleNetV2

为了能将神经网络模型用于移动端(手机)和终端(安防监控、无人驾驶)的实时计算,通常这些设备计算能力有限,因此我们需要减少模型参数量、减小计算量、更少的内存访问量、更少的能耗MobileNet、ShuffleNet 等轻量化网络就非常适合于一些计算能力受限的设备,牺牲少量准确率换取更快的运算速度。

在之前的章节中,我详细介绍了 MobileNetV1 和 MobileNetV2 网络模型,今天介绍的 ShuffleNet 网络也会使用到 MobileNet 的深度可分离卷积。建议先看一下我的前两篇文章:

MobileNetV1:https://blog.csdn.net/dgvv4/article/details/123415708

MobileNetV2:https://blog.csdn.net/dgvv4/article/details/123417739

左图是各个神经网络模型的准确率--参数量散点图,越靠近左上角代表模型参数量少而且准确率高。右图单位参数量对准确率的贡献,ShuffleNet 的计算量和参数量很少,但是计算的效率很高,计算速度很快。

【神经网络】(13) ShuffleNetV2 代码复现,网络解析,附Tensorflow完整代码_第1张图片   


1. ShuffleNetV1 创新点

在讲 ShuffleNetV2 之前我们需要对 ShuffleNetV1 有一定的了解,这里主要介绍ShuffleNetV1的两个核心内容。(1)分组1x1卷积;(2)通道重排

1.1 分组卷积(Group Convolution)

(1)在标准卷积中输入有多少个通道,卷积核就有多少个通道

举个例子:若输入的图像shape为5x5x6,一个卷积核的shape为3x3x6,使用3个卷积核,得到的特征图shape为3x3x3。参数量 = 5x5x6x3 = 450

(2)在分组卷积中每个卷积核只处理部分通道。如下图,红色的卷积核只处理输入图像中红色的两个通道,绿色的卷积核只处理输入图像中间的两个绿色的通道,第三个卷积核只处理黄色的两个通道。此时,每个卷积核有两个通道,每个卷积核生成一个特征图

举个例子:若输入的图像shape为5x5x6,一个分组卷积核的shape为3x3x2,使用3个分组卷积核,得到的特征图shape为3x3x3。参数量 = 5x5x(6/3)x(3/3)x3 = 5x5x2x1x3 = 150 。可见,分成三组,参数量为原来的三分之一。

因此,分组卷积能够有效地降低参数量和计算量

(3)深度可分离卷积是分组卷积的一个特例。随着分组的组数不断增加,输入图像有多少个通道就分几组,每个卷积核处理一个通道的特征,每个卷积核生成一个特征图,输入图像有多少通道就生成多少张特征图,再堆叠起来。

深度可分离卷积详细内容,看我前一篇MobileNetV1的文章。


1.2 通道重排(Channel Shuffle)

分组卷积存在一个问题,各个分组之间相互独立,没有特征融合通道重排方法实现跨组的信息交融

如下图(a)所示,卷积核分三组,生成特征图也是三组,每组只在内部进行信息交互,组与组之间没有任何信息交融。

如图(b, c)所示,将每个组的第一份,收集起来作为下一组;每组的第二份收集起来作为下一组....这样就实现了跨组的信息交流。

举个例子来说,如下图。分组卷积生成的三组特征图,第一组1~4;第二组5~8;第三组9~12。先将特征图重塑,为三行N列的矩形。然后进行转置,变成N行三列。最后压平,从二维tensor变成一维tensor,每一组的特征图交叉组合在一起。实现各组之间的信息交融。


2. ShuffleNetV2 模型

2.1 四条轻量化网络的设计原则

(1)输入输出通道数相同时,内存访问量MAC最小

输出通道数对应的是卷积核的数量。比如输入图像的通道数是256个,那么卷积核个数最好也是256个。

(2)分组数过大的分组卷积会增加MAC

ShuffleNetV1使用的分组1x1卷积,减少了参数量、计算量,增大网络表达能力。但是增加了内存占用量。

(3)碎片化操作对并行加速(GPU)不友好

碎片化操作指多分枝、多通路,如Inception系列网络的过多的分支

(4)逐元素操作(残差网络求和、relu激活等)带来的内存和耗时不可忽略

如:避免使用残差求和,将原来的 layers.Add() 变成 layers.Concat()


2.2 网络主干结构

作者根据这四条原则设计出来 ShuffleNetV2 轻量化网络。如下图。

Channel Spilt 模块将输入图像的通道数平均分成两份,一份用于残差连接,一份用于特征提取。

Channel Shuffle 模块将堆叠的特征图的通道重新排序,实现各分组之间的特征融合。

在基本模块中,特征图size不变,通道数不变。在下采样模块中,特征图的长宽减半,通道数加倍。


3. ShuffleNetV2 代码复现

根据论文中给出的网络构架,搭建模型。其中 Stage模块一个ShuffleNetv2下采样模块若干个ShuffleNetv2基本模块 组成。输入图像的shape为 224x224x3。output channel列的几个宽度超参数,一般只用超参数=1那一列。FLOPs代表乘-加浮点运算次数

【神经网络】(13) ShuffleNetV2 代码复现,网络解析,附Tensorflow完整代码_第2张图片


3.1 构建各个模块

(1)标准卷积模块

标准卷积模块是由 普通卷积+批标准化BN层+ReLU激活函数 构成

#(1)标准卷积块
def conv_block(input_tensor, filters, kernel_size, stride=1):

    # 卷积+批标准化+relu激活
    x = layers.Conv2D(filters, kernel_size, 
                      strides = stride,  # 步长
                      padding = 'same',  # strides=1卷积过程中特征图size不变,strides=2卷积过程中size减半
                      use_bias = False)(input_tensor)  # 有BN层就不需要偏置

    x = layers.BatchNormalization()(x)  # 批标准化

    x = layers.ReLU()(x)  # relu激活

    return x  # 返回一次标准卷积后的tensor

(2)深度可分离卷积模块

由2.2中的网络结构图可知,深度可分离卷积块只需要包含 3x3的深度卷积 和 批标准化BN层

#(2)深度可分离卷积块
def depthwise_conv_block(input_tensor, stride=1):

    # 深度可分离卷积+批标准化
    # 不需要传入卷积核个数,输入有几个通道,就有几个卷积核,每个卷积核负责一个通道
    x = layers.DepthwiseConv2D(kernel_size = (3,3),  # 深度卷积核size默认3*3
                               strides = stride,  # 步长
                               padding = 'same',  # strides=1卷积过程中特征图size不变,strides=2卷积过程中size减半
                               use_bias = False)(input_tensor)  # 有BN层就不需要偏置

    x = layers.BatchNormalization()(x)  # 批标准化

    return x  # 返回深度可分离卷积后的tensor

(3)通道重排模块

Channel Shuffle 模块用于对两组堆叠在一起的特征图进行重新排列,实现各组之间的信息融合。

#(3)通道重排,跨组信息交融
def channel_shuffle(input_tensor, num=2): # 默认时2组特征:shortcut和卷积后的x

    # 先得到输入特征图的shape,b:batch size,h,w:一张图的size,c:通道数
    b, h, w, c = input_tensor.shape

    # 确定shape = [b, h, w, num, c//num]。通道维度原来是一个长为c的一维tensor,变成2行n列的矩阵
    # 在通道维度上将特征图reshape为2行n列的矩阵。
    x_reshaped = tf.reshape(input_tensor, [-1, h, w, num, c//num])

    # 确定转置的矩形的shape = [b, h, w, c//num, num]
    # 矩阵转置,最后两个维度从2行n列变成n行2列
    x_transposed = tf.transpose(x_reshaped, [0,1,2,4,3])

    # 重新排列,shotcut和x的通道像素交叉排列,通道维度重新变成一维tensor
    output = tf.reshape(x_transposed, [-1, h, w, c])

    return output  # 返回通道维度交叉排序后的tensor

(4)ShuffleNetV2 基本模块

步长为1时,先将输入特征图在通道维度上平均分成两份。如2.2中的结构图,左分支输入等于输出,右分支特征传递,将两个分支的结果堆叠在一起,通道数变成原始大小。

#(4)步长=1时的卷积块
def shufflent_unit_1(input_tensor, filters):

    # 首先将输入特征图在通道维度上平均分成两份:一部分用于残差连接,一部分卷积提取特征
    shortcut, x = tf.split(input_tensor, 2, axis=-1)  # axis指定轴
    # 现在shotcut和x的通道数都只有原来的二分之一

    # 1*1卷积+3*3深度卷积+1*1卷积
    x = conv_block(x, filters//2, kernel_size=(1,1), stride=1)  # 1*1卷积,通道数保持不变
    x = depthwise_conv_block(x, stride=1)  # 3*3深度卷积
    x = conv_block(x, filters//2, kernel_size=(1,1), stride=1)  # 1*1卷积跨通道信息融合

    # 堆叠shoutcut和x,要求两个tensor的size相同
    x = tf.concat([shortcut, x], axis=-1)  # 在通道维度上堆叠

    # 将堆叠后2组特征图,在通道维度上重新排列
    x = channel_shuffle(x)

    return x  # 返回步长为1时的卷积块输出的tensor

(5)ShuffleNetV2 下采样模块

步长为2时,输入图像不平分通道数。左分支先3*3深度卷积(步长=2),然后1*1卷积,输出特征图的长宽减半,通道数不变右分支先1*1卷积降维,然后3*3深度卷积(步长=2),再1*1卷积升维。

这里要注意左分支输出特征图数量+右分支输出特征图数量=下采样模块输出特征图数量

已知左分支输出特征图的通道数保持不变(in_channel),下采样模块输出特征图数量(out_channel)已给出,可计算得到右分支输出特征图的通道数,将它作为右分支最后一个1*1卷积的卷积核个数。

#(5)步长=2时(下采样)的卷积块
def shufflenet_unit_2(input_tensor, out_channel):

    # 输入特征图的通道数
    in_channel = input_tensor.shape[-1]

    # 首先将输入特征图复制一份,分别用于左右两个分支的卷积
    shortcut = input_tensor

    # ① 左分支的卷积部分==深度卷积+逐点卷积,输出特征图通道数等于原通道数
    shortcut = depthwise_conv_block(shortcut, stride=2)  # 特征图size减半
    shortcut = conv_block(shortcut, filters=in_channel, kernel_size=(1,1), stride=1)  # 输出特征图个数不变

    # ② 右分支==1*1卷积下降通道数+3*3深度卷积+1*1卷积上升通道数
    x = conv_block(input_tensor, in_channel//2, kernel_size=(1,1), stride=1)
    x = depthwise_conv_block(x, stride=2)
    # 右分支的通道数和左分支的通道数叠加==输出特征图的通道数out_channel
    x = conv_block(x, out_channel-in_channel, kernel_size=(1,1), stride=1)

    # ③ 左右分支的输出特征图在通道维度上堆叠,并且output.shape[-1]==out_channel
    output = tf.concat([shortcut, x], axis=-1)

    # ④ 堆叠后的2组特征在通道维度上重新排列
    output = channel_shuffle(output)

    return output  # 返回步长=2时的输出结果

(6)Stage 模块

一个Stage模块是由一个下采样模块,和多个基本模块组合而成的

#(6)构建shufflenet卷积块
# 一个shuffle卷积块是由一个shufflenet_unit_2下采样单元,和若干个shufflenet_unit_1特征传递单元构成
def stage(input_tensor, filters, n):   # filters代表输出通道数

    # 下采样单元
    x = shufflenet_unit_2(input_tensor, out_channel=filters)

    # 特征传递单元循环n次
    for i in range(n):
        x = shufflent_unit_1(x, filters=filters)
    
    return x  # 返回一个shufflenet卷积结果

3.2 完整代码展示

结合上文的分析,完整代码如下,代码中每行都有注释,有疑问的在评论区留言

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, Model

#(1)标准卷积块
def conv_block(input_tensor, filters, kernel_size, stride=1):

    # 卷积+批标准化+relu激活
    x = layers.Conv2D(filters, kernel_size, 
                      strides = stride,  # 步长
                      padding = 'same',  # strides=1卷积过程中特征图size不变,strides=2卷积过程中size减半
                      use_bias = False)(input_tensor)  # 有BN层就不需要偏置

    x = layers.BatchNormalization()(x)  # 批标准化

    x = layers.ReLU()(x)  # relu激活

    return x  # 返回一次标准卷积后的tensor


#(2)深度可分离卷积块
def depthwise_conv_block(input_tensor, stride=1):

    # 深度可分离卷积+批标准化
    # 不需要传入卷积核个数,输入有几个通道,就有几个卷积核,每个卷积核负责一个通道
    x = layers.DepthwiseConv2D(kernel_size = (3,3),  # 深度卷积核size默认3*3
                               strides = stride,  # 步长
                               padding = 'same',  # strides=1卷积过程中特征图size不变,strides=2卷积过程中size减半
                               use_bias = False)(input_tensor)  # 有BN层就不需要偏置

    x = layers.BatchNormalization()(x)  # 批标准化

    return x  # 返回深度可分离卷积后的tensor


#(3)通道重排,跨组信息交融
def channel_shuffle(input_tensor, num=2): # 默认时2组特征:shortcut和卷积后的x

    # 先得到输入特征图的shape,b:batch size,h,w:一张图的size,c:通道数
    b, h, w, c = input_tensor.shape

    # 确定shape = [b, h, w, num, c//num]。通道维度原来是一个长为c的一维tensor,变成2行n列的矩阵
    # 在通道维度上将特征图reshape为2行n列的矩阵。
    x_reshaped = tf.reshape(input_tensor, [-1, h, w, num, c//num])

    # 确定转置的矩形的shape = [b, h, w, c//num, num]
    # 矩阵转置,最后两个维度从2行n列变成n行2列
    x_transposed = tf.transpose(x_reshaped, [0,1,2,4,3])

    # 重新排列,shotcut和x的通道像素交叉排列,通道维度重新变成一维tensor
    output = tf.reshape(x_transposed, [-1, h, w, c])

    return output  # 返回通道维度交叉排序后的tensor


#(4)步长=1时的卷积块
def shufflent_unit_1(input_tensor, filters):

    # 首先将输入特征图在通道维度上平均分成两份:一部分用于残差连接,一部分卷积提取特征
    shortcut, x = tf.split(input_tensor, 2, axis=-1)  # axis指定轴
    # 现在shotcut和x的通道数都只有原来的二分之一

    # 1*1卷积+3*3深度卷积+1*1卷积
    x = conv_block(x, filters//2, kernel_size=(1,1), stride=1)  # 1*1卷积,通道数保持不变
    x = depthwise_conv_block(x, stride=1)  # 3*3深度卷积
    x = conv_block(x, filters//2, kernel_size=(1,1), stride=1)  # 1*1卷积跨通道信息融合

    # 堆叠shoutcut和x,要求两个tensor的size相同
    x = tf.concat([shortcut, x], axis=-1)  # 在通道维度上堆叠

    # 将堆叠后2组特征图,在通道维度上重新排列
    x = channel_shuffle(x)

    return x  # 返回步长为1时的卷积块输出的tensor


#(5)步长=2时(下采样)的卷积块
def shufflenet_unit_2(input_tensor, out_channel):

    # 输入特征图的通道数
    in_channel = input_tensor.shape[-1]

    # 首先将输入特征图复制一份,分别用于左右两个分支的卷积
    shortcut = input_tensor

    # ① 左分支的卷积部分==深度卷积+逐点卷积,输出特征图通道数等于原通道数
    shortcut = depthwise_conv_block(shortcut, stride=2)  # 特征图size减半
    shortcut = conv_block(shortcut, filters=in_channel, kernel_size=(1,1), stride=1)  # 输出特征图个数不变

    # ② 右分支==1*1卷积下降通道数+3*3深度卷积+1*1卷积上升通道数
    x = conv_block(input_tensor, in_channel//2, kernel_size=(1,1), stride=1)
    x = depthwise_conv_block(x, stride=2)
    # 右分支的通道数和左分支的通道数叠加==输出特征图的通道数out_channel
    x = conv_block(x, out_channel-in_channel, kernel_size=(1,1), stride=1)

    # ③ 左右分支的输出特征图在通道维度上堆叠,并且output.shape[-1]==out_channel
    output = tf.concat([shortcut, x], axis=-1)

    # ④ 堆叠后的2组特征在通道维度上重新排列
    output = channel_shuffle(output)

    return output  # 返回步长=2时的输出结果


#(6)构建shufflenet卷积块
# 一个shuffle卷积块是由一个shufflenet_unit_2下采样单元,和若干个shufflenet_unit_1特征传递单元构成
def stage(input_tensor, filters, n):   # filters代表输出通道数

    # 下采样单元
    x = shufflenet_unit_2(input_tensor, out_channel=filters)

    # 特征传递单元循环n次
    for i in range(n):
        x = shufflent_unit_1(x, filters=filters)
    
    return x  # 返回一个shufflenet卷积结果


#(7)构建网络模型
def ShuffleNet(input_shape, classes):

    # 构建网络输入的tensor
    inputs = keras.Input(shape=input_shape)

    # [224,224,3]==>[112,112,24]
    x = layers.Conv2D(filters=24, kernel_size=(3,3), strides=2, padding='same')(inputs)  # 普通卷积

    # [112,112,24]==>[56,56,24]
    x = layers.MaxPooling2D(pool_size=(3,3), strides=2, padding='same')(x)  # 最大池化

    # [56,56,24]==>[28,28,116]
    x = stage(x, filters=116, n=3)

    # [28,28,116]==>[14,14,232]
    x = stage(x, filters=232, n=7)

    # [14,14,232]==>[7,7,464]
    x = stage(x, filters=464, n=3)

    # [7,7,464]==>[7,7,1024]
    x = layers.Conv2D(filters=1024, kernel_size=(1,1), strides=1, padding='same')(x)  # 1*1普通卷积

    # [7,7,1024]==>[None,1024]
    x = layers.GlobalAveragePooling2D()(x)  # 在通道维度上全局平均池化

    # 按论文输出层使用全连接层,也可改为卷积层再Reshape
    logits = layers.Dense(classes)(x)  # 为了网络稳定,训练时再使用Softmax函数

    # 完成网络架构
    model = Model(inputs, logits)

    return model  # 返回网络模型


#(8)接收网络模型
if __name__ == '__main__':

    model = ShuffleNet(input_shape=[224,224,3],  # 输入图像的shape
                       classes=1000)  # 图像分类类别

    model.summary()  # 查看网络结构

3.3 查看网络构架

通过 model.summary() 查看网络结构,可见 ShuffleNetV2 的参数量只有两百多万,对比 MobileNetV2 的三百多万的参数量,已经非常轻量化了。

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
conv2d (Conv2D)                 (None, 112, 112, 24) 672         input_1[0][0]                    
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, 56, 56, 24)   0           conv2d[0][0]                     
__________________________________________________________________________________________________
conv2d_2 (Conv2D)               (None, 56, 56, 12)   288         max_pooling2d[0][0]              
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 56, 56, 12)   48          conv2d_2[0][0]                   
__________________________________________________________________________________________________
re_lu_1 (ReLU)                  (None, 56, 56, 12)   0           batch_normalization_2[0][0]      
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 28, 28, 24)   216         max_pooling2d[0][0]              
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 28, 28, 12)   108         re_lu_1[0][0]                    
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 28, 28, 24)   96          depthwise_conv2d[0][0]           
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 28, 28, 12)   48          depthwise_conv2d_1[0][0]         
__________________________________________________________________________________________________
conv2d_1 (Conv2D)               (None, 28, 28, 24)   576         batch_normalization[0][0]        
__________________________________________________________________________________________________
conv2d_3 (Conv2D)               (None, 28, 28, 92)   1104        batch_normalization_3[0][0]      
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 28, 28, 24)   96          conv2d_1[0][0]                   
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 28, 28, 92)   368         conv2d_3[0][0]                   
__________________________________________________________________________________________________
re_lu (ReLU)                    (None, 28, 28, 24)   0           batch_normalization_1[0][0]      
__________________________________________________________________________________________________
re_lu_2 (ReLU)                  (None, 28, 28, 92)   0           batch_normalization_4[0][0]      
__________________________________________________________________________________________________
tf.concat (TFOpLambda)          (None, 28, 28, 116)  0           re_lu[0][0]                      
                                                                 re_lu_2[0][0]                    
__________________________________________________________________________________________________
tf.reshape (TFOpLambda)         (None, 28, 28, 2, 58 0           tf.concat[0][0]                  
__________________________________________________________________________________________________
tf.compat.v1.transpose (TFOpLam (None, 28, 28, 58, 2 0           tf.reshape[0][0]                 
__________________________________________________________________________________________________
tf.reshape_1 (TFOpLambda)       (None, 28, 28, 116)  0           tf.compat.v1.transpose[0][0]     
__________________________________________________________________________________________________
tf.split (TFOpLambda)           [(None, 28, 28, 58), 0           tf.reshape_1[0][0]               
__________________________________________________________________________________________________
conv2d_4 (Conv2D)               (None, 28, 28, 58)   3364        tf.split[0][1]                   
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 28, 28, 58)   232         conv2d_4[0][0]                   
__________________________________________________________________________________________________
re_lu_3 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_5[0][0]      
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 58)   522         re_lu_3[0][0]                    
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 28, 28, 58)   232         depthwise_conv2d_2[0][0]         
__________________________________________________________________________________________________
conv2d_5 (Conv2D)               (None, 28, 28, 58)   3364        batch_normalization_6[0][0]      
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 28, 28, 58)   232         conv2d_5[0][0]                   
__________________________________________________________________________________________________
re_lu_4 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_7[0][0]      
__________________________________________________________________________________________________
tf.concat_1 (TFOpLambda)        (None, 28, 28, 116)  0           tf.split[0][0]                   
                                                                 re_lu_4[0][0]                    
__________________________________________________________________________________________________
tf.reshape_2 (TFOpLambda)       (None, 28, 28, 2, 58 0           tf.concat_1[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_1 (TFOpL (None, 28, 28, 58, 2 0           tf.reshape_2[0][0]               
__________________________________________________________________________________________________
tf.reshape_3 (TFOpLambda)       (None, 28, 28, 116)  0           tf.compat.v1.transpose_1[0][0]   
__________________________________________________________________________________________________
tf.split_1 (TFOpLambda)         [(None, 28, 28, 58), 0           tf.reshape_3[0][0]               
__________________________________________________________________________________________________
conv2d_6 (Conv2D)               (None, 28, 28, 58)   3364        tf.split_1[0][1]                 
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 28, 28, 58)   232         conv2d_6[0][0]                   
__________________________________________________________________________________________________
re_lu_5 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_8[0][0]      
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 28, 28, 58)   522         re_lu_5[0][0]                    
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 28, 28, 58)   232         depthwise_conv2d_3[0][0]         
__________________________________________________________________________________________________
conv2d_7 (Conv2D)               (None, 28, 28, 58)   3364        batch_normalization_9[0][0]      
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 28, 28, 58)   232         conv2d_7[0][0]                   
__________________________________________________________________________________________________
re_lu_6 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_10[0][0]     
__________________________________________________________________________________________________
tf.concat_2 (TFOpLambda)        (None, 28, 28, 116)  0           tf.split_1[0][0]                 
                                                                 re_lu_6[0][0]                    
__________________________________________________________________________________________________
tf.reshape_4 (TFOpLambda)       (None, 28, 28, 2, 58 0           tf.concat_2[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_2 (TFOpL (None, 28, 28, 58, 2 0           tf.reshape_4[0][0]               
__________________________________________________________________________________________________
tf.reshape_5 (TFOpLambda)       (None, 28, 28, 116)  0           tf.compat.v1.transpose_2[0][0]   
__________________________________________________________________________________________________
tf.split_2 (TFOpLambda)         [(None, 28, 28, 58), 0           tf.reshape_5[0][0]               
__________________________________________________________________________________________________
conv2d_8 (Conv2D)               (None, 28, 28, 58)   3364        tf.split_2[0][1]                 
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 28, 28, 58)   232         conv2d_8[0][0]                   
__________________________________________________________________________________________________
re_lu_7 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_11[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 28, 28, 58)   522         re_lu_7[0][0]                    
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 28, 28, 58)   232         depthwise_conv2d_4[0][0]         
__________________________________________________________________________________________________
conv2d_9 (Conv2D)               (None, 28, 28, 58)   3364        batch_normalization_12[0][0]     
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 28, 28, 58)   232         conv2d_9[0][0]                   
__________________________________________________________________________________________________
re_lu_8 (ReLU)                  (None, 28, 28, 58)   0           batch_normalization_13[0][0]     
__________________________________________________________________________________________________
tf.concat_3 (TFOpLambda)        (None, 28, 28, 116)  0           tf.split_2[0][0]                 
                                                                 re_lu_8[0][0]                    
__________________________________________________________________________________________________
tf.reshape_6 (TFOpLambda)       (None, 28, 28, 2, 58 0           tf.concat_3[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_3 (TFOpL (None, 28, 28, 58, 2 0           tf.reshape_6[0][0]               
__________________________________________________________________________________________________
tf.reshape_7 (TFOpLambda)       (None, 28, 28, 116)  0           tf.compat.v1.transpose_3[0][0]   
__________________________________________________________________________________________________
conv2d_11 (Conv2D)              (None, 28, 28, 58)   6728        tf.reshape_7[0][0]               
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 28, 28, 58)   232         conv2d_11[0][0]                  
__________________________________________________________________________________________________
re_lu_10 (ReLU)                 (None, 28, 28, 58)   0           batch_normalization_16[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 14, 14, 116)  1044        tf.reshape_7[0][0]               
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 14, 14, 58)   522         re_lu_10[0][0]                   
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_5[0][0]         
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 14, 14, 58)   232         depthwise_conv2d_6[0][0]         
__________________________________________________________________________________________________
conv2d_10 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_14[0][0]     
__________________________________________________________________________________________________
conv2d_12 (Conv2D)              (None, 14, 14, 116)  6728        batch_normalization_17[0][0]     
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 14, 14, 116)  464         conv2d_10[0][0]                  
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 14, 14, 116)  464         conv2d_12[0][0]                  
__________________________________________________________________________________________________
re_lu_9 (ReLU)                  (None, 14, 14, 116)  0           batch_normalization_15[0][0]     
__________________________________________________________________________________________________
re_lu_11 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_18[0][0]     
__________________________________________________________________________________________________
tf.concat_4 (TFOpLambda)        (None, 14, 14, 232)  0           re_lu_9[0][0]                    
                                                                 re_lu_11[0][0]                   
__________________________________________________________________________________________________
tf.reshape_8 (TFOpLambda)       (None, 14, 14, 2, 11 0           tf.concat_4[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_4 (TFOpL (None, 14, 14, 116,  0           tf.reshape_8[0][0]               
__________________________________________________________________________________________________
tf.reshape_9 (TFOpLambda)       (None, 14, 14, 232)  0           tf.compat.v1.transpose_4[0][0]   
__________________________________________________________________________________________________
tf.split_3 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_9[0][0]               
__________________________________________________________________________________________________
conv2d_13 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_3[0][1]                 
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 14, 14, 116)  464         conv2d_13[0][0]                  
__________________________________________________________________________________________________
re_lu_12 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_19[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 14, 14, 116)  1044        re_lu_12[0][0]                   
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_7[0][0]         
__________________________________________________________________________________________________
conv2d_14 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_20[0][0]     
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 14, 14, 116)  464         conv2d_14[0][0]                  
__________________________________________________________________________________________________
re_lu_13 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_21[0][0]     
__________________________________________________________________________________________________
tf.concat_5 (TFOpLambda)        (None, 14, 14, 232)  0           tf.split_3[0][0]                 
                                                                 re_lu_13[0][0]                   
__________________________________________________________________________________________________
tf.reshape_10 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_5[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_5 (TFOpL (None, 14, 14, 116,  0           tf.reshape_10[0][0]              
__________________________________________________________________________________________________
tf.reshape_11 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_5[0][0]   
__________________________________________________________________________________________________
tf.split_4 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_11[0][0]              
__________________________________________________________________________________________________
conv2d_15 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_4[0][1]                 
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 14, 14, 116)  464         conv2d_15[0][0]                  
__________________________________________________________________________________________________
re_lu_14 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_22[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 14, 14, 116)  1044        re_lu_14[0][0]                   
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_8[0][0]         
__________________________________________________________________________________________________
conv2d_16 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_23[0][0]     
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 14, 14, 116)  464         conv2d_16[0][0]                  
__________________________________________________________________________________________________
re_lu_15 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_24[0][0]     
__________________________________________________________________________________________________
tf.concat_6 (TFOpLambda)        (None, 14, 14, 232)  0           tf.split_4[0][0]                 
                                                                 re_lu_15[0][0]                   
__________________________________________________________________________________________________
tf.reshape_12 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_6[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_6 (TFOpL (None, 14, 14, 116,  0           tf.reshape_12[0][0]              
__________________________________________________________________________________________________
tf.reshape_13 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_6[0][0]   
__________________________________________________________________________________________________
tf.split_5 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_13[0][0]              
__________________________________________________________________________________________________
conv2d_17 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_5[0][1]                 
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 14, 14, 116)  464         conv2d_17[0][0]                  
__________________________________________________________________________________________________
re_lu_16 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_25[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 14, 14, 116)  1044        re_lu_16[0][0]                   
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_9[0][0]         
__________________________________________________________________________________________________
conv2d_18 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_26[0][0]     
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 14, 14, 116)  464         conv2d_18[0][0]                  
__________________________________________________________________________________________________
re_lu_17 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
tf.concat_7 (TFOpLambda)        (None, 14, 14, 232)  0           tf.split_5[0][0]                 
                                                                 re_lu_17[0][0]                   
__________________________________________________________________________________________________
tf.reshape_14 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_7[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_7 (TFOpL (None, 14, 14, 116,  0           tf.reshape_14[0][0]              
__________________________________________________________________________________________________
tf.reshape_15 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_7[0][0]   
__________________________________________________________________________________________________
tf.split_6 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_15[0][0]              
__________________________________________________________________________________________________
conv2d_19 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_6[0][1]                 
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 14, 14, 116)  464         conv2d_19[0][0]                  
__________________________________________________________________________________________________
re_lu_18 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_28[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 14, 14, 116)  1044        re_lu_18[0][0]                   
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_10[0][0]        
__________________________________________________________________________________________________
conv2d_20 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_29[0][0]     
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 14, 14, 116)  464         conv2d_20[0][0]                  
__________________________________________________________________________________________________
re_lu_19 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_30[0][0]     
__________________________________________________________________________________________________
tf.concat_8 (TFOpLambda)        (None, 14, 14, 232)  0           tf.split_6[0][0]                 
                                                                 re_lu_19[0][0]                   
__________________________________________________________________________________________________
tf.reshape_16 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_8[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_8 (TFOpL (None, 14, 14, 116,  0           tf.reshape_16[0][0]              
__________________________________________________________________________________________________
tf.reshape_17 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_8[0][0]   
__________________________________________________________________________________________________
tf.split_7 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_17[0][0]              
__________________________________________________________________________________________________
conv2d_21 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_7[0][1]                 
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 14, 14, 116)  464         conv2d_21[0][0]                  
__________________________________________________________________________________________________
re_lu_20 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 14, 14, 116)  1044        re_lu_20[0][0]                   
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_11[0][0]        
__________________________________________________________________________________________________
conv2d_22 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_32[0][0]     
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 14, 14, 116)  464         conv2d_22[0][0]                  
__________________________________________________________________________________________________
re_lu_21 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_33[0][0]     
__________________________________________________________________________________________________
tf.concat_9 (TFOpLambda)        (None, 14, 14, 232)  0           tf.split_7[0][0]                 
                                                                 re_lu_21[0][0]                   
__________________________________________________________________________________________________
tf.reshape_18 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_9[0][0]                
__________________________________________________________________________________________________
tf.compat.v1.transpose_9 (TFOpL (None, 14, 14, 116,  0           tf.reshape_18[0][0]              
__________________________________________________________________________________________________
tf.reshape_19 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_9[0][0]   
__________________________________________________________________________________________________
tf.split_8 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_19[0][0]              
__________________________________________________________________________________________________
conv2d_23 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_8[0][1]                 
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 14, 14, 116)  464         conv2d_23[0][0]                  
__________________________________________________________________________________________________
re_lu_22 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 14, 14, 116)  1044        re_lu_22[0][0]                   
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_12[0][0]        
__________________________________________________________________________________________________
conv2d_24 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_35[0][0]     
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 116)  464         conv2d_24[0][0]                  
__________________________________________________________________________________________________
re_lu_23 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
tf.concat_10 (TFOpLambda)       (None, 14, 14, 232)  0           tf.split_8[0][0]                 
                                                                 re_lu_23[0][0]                   
__________________________________________________________________________________________________
tf.reshape_20 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_10[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_10 (TFOp (None, 14, 14, 116,  0           tf.reshape_20[0][0]              
__________________________________________________________________________________________________
tf.reshape_21 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_10[0][0]  
__________________________________________________________________________________________________
tf.split_9 (TFOpLambda)         [(None, 14, 14, 116) 0           tf.reshape_21[0][0]              
__________________________________________________________________________________________________
conv2d_25 (Conv2D)              (None, 14, 14, 116)  13456       tf.split_9[0][1]                 
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 116)  464         conv2d_25[0][0]                  
__________________________________________________________________________________________________
re_lu_24 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_37[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 14, 14, 116)  1044        re_lu_24[0][0]                   
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 116)  464         depthwise_conv2d_13[0][0]        
__________________________________________________________________________________________________
conv2d_26 (Conv2D)              (None, 14, 14, 116)  13456       batch_normalization_38[0][0]     
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 116)  464         conv2d_26[0][0]                  
__________________________________________________________________________________________________
re_lu_25 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
tf.concat_11 (TFOpLambda)       (None, 14, 14, 232)  0           tf.split_9[0][0]                 
                                                                 re_lu_25[0][0]                   
__________________________________________________________________________________________________
tf.reshape_22 (TFOpLambda)      (None, 14, 14, 2, 11 0           tf.concat_11[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_11 (TFOp (None, 14, 14, 116,  0           tf.reshape_22[0][0]              
__________________________________________________________________________________________________
tf.reshape_23 (TFOpLambda)      (None, 14, 14, 232)  0           tf.compat.v1.transpose_11[0][0]  
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 14, 14, 116)  26912       tf.reshape_23[0][0]              
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 14, 14, 116)  464         conv2d_28[0][0]                  
__________________________________________________________________________________________________
re_lu_27 (ReLU)                 (None, 14, 14, 116)  0           batch_normalization_42[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 7, 7, 232)    2088        tf.reshape_23[0][0]              
__________________________________________________________________________________________________
depthwise_conv2d_15 (DepthwiseC (None, 7, 7, 116)    1044        re_lu_27[0][0]                   
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 7, 7, 232)    928         depthwise_conv2d_14[0][0]        
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 116)    464         depthwise_conv2d_15[0][0]        
__________________________________________________________________________________________________
conv2d_27 (Conv2D)              (None, 7, 7, 232)    53824       batch_normalization_40[0][0]     
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 7, 7, 232)    26912       batch_normalization_43[0][0]     
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 7, 7, 232)    928         conv2d_27[0][0]                  
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 232)    928         conv2d_29[0][0]                  
__________________________________________________________________________________________________
re_lu_26 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
re_lu_28 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
tf.concat_12 (TFOpLambda)       (None, 7, 7, 464)    0           re_lu_26[0][0]                   
                                                                 re_lu_28[0][0]                   
__________________________________________________________________________________________________
tf.reshape_24 (TFOpLambda)      (None, 7, 7, 2, 232) 0           tf.concat_12[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_12 (TFOp (None, 7, 7, 232, 2) 0           tf.reshape_24[0][0]              
__________________________________________________________________________________________________
tf.reshape_25 (TFOpLambda)      (None, 7, 7, 464)    0           tf.compat.v1.transpose_12[0][0]  
__________________________________________________________________________________________________
tf.split_10 (TFOpLambda)        [(None, 7, 7, 232),  0           tf.reshape_25[0][0]              
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 7, 7, 232)    53824       tf.split_10[0][1]                
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 232)    928         conv2d_30[0][0]                  
__________________________________________________________________________________________________
re_lu_29 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_45[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_16 (DepthwiseC (None, 7, 7, 232)    2088        re_lu_29[0][0]                   
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 7, 7, 232)    928         depthwise_conv2d_16[0][0]        
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 7, 7, 232)    53824       batch_normalization_46[0][0]     
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 7, 7, 232)    928         conv2d_31[0][0]                  
__________________________________________________________________________________________________
re_lu_30 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_47[0][0]     
__________________________________________________________________________________________________
tf.concat_13 (TFOpLambda)       (None, 7, 7, 464)    0           tf.split_10[0][0]                
                                                                 re_lu_30[0][0]                   
__________________________________________________________________________________________________
tf.reshape_26 (TFOpLambda)      (None, 7, 7, 2, 232) 0           tf.concat_13[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_13 (TFOp (None, 7, 7, 232, 2) 0           tf.reshape_26[0][0]              
__________________________________________________________________________________________________
tf.reshape_27 (TFOpLambda)      (None, 7, 7, 464)    0           tf.compat.v1.transpose_13[0][0]  
__________________________________________________________________________________________________
tf.split_11 (TFOpLambda)        [(None, 7, 7, 232),  0           tf.reshape_27[0][0]              
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 7, 7, 232)    53824       tf.split_11[0][1]                
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 7, 7, 232)    928         conv2d_32[0][0]                  
__________________________________________________________________________________________________
re_lu_31 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_48[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_17 (DepthwiseC (None, 7, 7, 232)    2088        re_lu_31[0][0]                   
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, 7, 7, 232)    928         depthwise_conv2d_17[0][0]        
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 7, 7, 232)    53824       batch_normalization_49[0][0]     
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, 7, 7, 232)    928         conv2d_33[0][0]                  
__________________________________________________________________________________________________
re_lu_32 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_50[0][0]     
__________________________________________________________________________________________________
tf.concat_14 (TFOpLambda)       (None, 7, 7, 464)    0           tf.split_11[0][0]                
                                                                 re_lu_32[0][0]                   
__________________________________________________________________________________________________
tf.reshape_28 (TFOpLambda)      (None, 7, 7, 2, 232) 0           tf.concat_14[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_14 (TFOp (None, 7, 7, 232, 2) 0           tf.reshape_28[0][0]              
__________________________________________________________________________________________________
tf.reshape_29 (TFOpLambda)      (None, 7, 7, 464)    0           tf.compat.v1.transpose_14[0][0]  
__________________________________________________________________________________________________
tf.split_12 (TFOpLambda)        [(None, 7, 7, 232),  0           tf.reshape_29[0][0]              
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 7, 7, 232)    53824       tf.split_12[0][1]                
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, 7, 7, 232)    928         conv2d_34[0][0]                  
__________________________________________________________________________________________________
re_lu_33 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_51[0][0]     
__________________________________________________________________________________________________
depthwise_conv2d_18 (DepthwiseC (None, 7, 7, 232)    2088        re_lu_33[0][0]                   
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, 7, 7, 232)    928         depthwise_conv2d_18[0][0]        
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 7, 7, 232)    53824       batch_normalization_52[0][0]     
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, 7, 7, 232)    928         conv2d_35[0][0]                  
__________________________________________________________________________________________________
re_lu_34 (ReLU)                 (None, 7, 7, 232)    0           batch_normalization_53[0][0]     
__________________________________________________________________________________________________
tf.concat_15 (TFOpLambda)       (None, 7, 7, 464)    0           tf.split_12[0][0]                
                                                                 re_lu_34[0][0]                   
__________________________________________________________________________________________________
tf.reshape_30 (TFOpLambda)      (None, 7, 7, 2, 232) 0           tf.concat_15[0][0]               
__________________________________________________________________________________________________
tf.compat.v1.transpose_15 (TFOp (None, 7, 7, 232, 2) 0           tf.reshape_30[0][0]              
__________________________________________________________________________________________________
tf.reshape_31 (TFOpLambda)      (None, 7, 7, 464)    0           tf.compat.v1.transpose_15[0][0]  
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 7, 7, 1024)   476160      tf.reshape_31[0][0]              
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 1024)         0           conv2d_36[0][0]                  
__________________________________________________________________________________________________
dense (Dense)                   (None, 1000)         1025000     global_average_pooling2d[0][0]   
==================================================================================================
Total params: 2,216,440
Trainable params: 2,203,236
Non-trainable params: 13,204
__________________________________________________________________________________________________

你可能感兴趣的:(TensorFlow神经网络,python,tensorflow,神经网络,cnn,深度学习)