【神经网络】(17) EfficientNet 代码复现，网络解析，附Tensorflow完整代码
各位同学好，今天和大家分享一下如何使用 Tensorflow 复现 EfficientNet 卷积神经网络模型。
EfficientNet 的网络结构和 MobileNetV3 比较相似，建议大家在学习 EfficientNet 之前先学习一下 MobileNetV3
MobileNetV3：https://blog.csdn.net/dgvv4/article/details/123476899
EfficientNet-B7在imagenet上准确率达到了当年最高的84.3%，与之前准确率最高的GPipe相比，参数量仅为其1/8.4，推理速度提高了6.1倍。
1. 引言
（1）根据以往的经验，增加网络的深度能得到更加丰富、复杂的特征，并且能够很好的应用到其他任务中。但网络的深度过深会面临梯度消失，训练困难的问题。
（2）增加网络的宽度能够获得更高细粒度的特征，并且也更容易训练，但对于宽度很大而深度很浅的网络往往很难学习到更深层次的特征。
（3）增加输入网络的图像分辨率能够潜在的获得更高细粒度的特征模板，但对于非常高的输入分辨率，准确率的增益也会减小。并且大分辨率的图像会增加计算量。
论文中就研究了如果同时增加网络的宽度、深度、分辨率，那会有什么样的效果。如下图所示，红色曲线就是同时增加网络的深度、宽度和分辨率，网络效果明显提高。
2. 网络核心模块
网络的核心模块大体上和MobileNetV3相似，这里再简单复习一下
2.1 深度可分离卷积
MobileNetV1 中主要使用了深度可分离卷积模块，大大减少了参数量和计算量。
普通卷积是一个卷积核处理所有的通道，输入特征图有多少个通道，卷积核就有几个通道，一个卷积核生成一张特征图。
深度可分离卷积 可理解为 深度卷积 + 逐点卷积
深度卷积只处理长宽方向的空间信息；逐点卷积只处理跨通道方向的信息。能大大减少参数量，提高计算效率
深度卷积： 一个卷积核只处理一个通道，即每个卷积核只处理自己对应的通道。输入特征图有多少个通道就有多少个卷积核。将每个卷积核处理后的特征图堆叠在一起。输入和输出特征图的通道数相同。
由于只处理长宽方向的信息会导致丢失跨通道信息，为了将跨通道的信息补充回来，需要进行逐点卷积。
逐点卷积： 是使用1x1卷积对跨通道维度处理，有多少个1x1卷积核就会生成多少个特征图。
2.2 逆转残差模块
逆转残差模块流程如下。输入图像，先使用1x1卷积提升通道数；然后在高维空间下使用深度卷积；再使用1x1卷积下降通道数，降维时采用线性激活函数(y=x)。当步长等于1且输入和输出特征图的shape相同时，使用残差连接输入和输出；当步长=2（下采样阶段）直接输出降维后的特征图。
对比 ResNet 的残差结构。输入图像，先使用1x1卷积下降通道数；然后在低维空间下使用标准卷积，再使用1x1卷积上升通道数，激活函数都是ReLU函数。当步长等于1且输入和输出特征图的shape相同时，使用残差连接输入和输出；当步长=2（下采样阶段）直接输出降维后的特征图。
2.3 SE注意力机制
（1）先将特征图进行全局平均池化，特征图有多少个通道，那么池化结果（一维向量）就有多少个元素，[h, w, c]==>[None, c]。
（2）然后经过两个全连接层得到输出向量。在EfficientNet中，第一个全连接层降维，输出通道数等于该逆转残差模块的输入图像的通道数的1/4；第二个全连接层升维，输出通道数等于全局平均池化前的特征图的通道数。
（3）全连接层的输出向量可理解为，向量的每个元素是对每张特征图进行分析得出的权重关系。比较重要的特征图就会赋予更大的权重，即该特征图对应的向量元素的值较大。反之，不太重要的特征图对应的权重值较小。
（4）经过两个全连接层得到一个由channel个元素组成的向量，每个元素是针对每个通道的权重，将权重和原特征图的像素值对应相乘，得到新的特征图数据
以下图为例，特征图经过两个全连接层之后，比较重要的特征图对应的向量元素的值就较大。将得到的权重和对应特征图中的所有元素相乘，得到新的输出特征图
2.4 总体流程
基本模块(stride=1)：图像输入，先经过1x1卷积上升通道数；然后在高纬空间下使用深度卷积；再经过SE注意力机制优化特征图数据；再经过1x1卷积下降通道数（使用线性激活函数）；若此时输入特征图的shape和输出特征图的shape相同，那么对1x1卷积降维后的特征图加一个Dropout层，防止过拟合；最后残差连接输入和输出
下采样模块(stride=2)：大致流程和基本模块相同，不采用Dropout层和残差连接，1x1卷积降维后直接输出特征图。
3. 代码复现
3.1 网络架构图
以 EfficientNet-B0 为例，网络结构如下图所示。
3.2 EfficientNet 系列网络参数
（1）width_coefficient 代表通道维度上的倍率因子。比如，在EfficientNet-B0中Stage1的3x3卷积层使用的卷积核个数是32个，那么EfficientNet-B6中Stage1的3x3卷积层使用卷积核个数是 32*1.8=57.6，取整到离57.6最近的8的倍数，即56
（2）depth_coefficient 代表深度维度上的倍率因子。比如，在EfficientNet-B0中Stage7的layers=4，即该模块重复4次。那么在EfficientNet-B6中Stage7的layers=4*2.6=10.4，向上取整为11。
（3）dropout_rate 代表Dropout的随机杀死神经元的概率
'''
Model | input_size | width_coefficient | depth_coefficient | dropout_rate
-------------------------------------------------------------------------------------------
EfficientNetB0 | 224x224 | 1.0 | 1.0 | 0.2
-------------------------------------------------------------------------------------------
EfficientNetB1 | 240x240 | 1.0 | 1.1 | 0.2
-------------------------------------------------------------------------------------------
EfficientNetB2 | 260x260 | 1.1 | 1.2 | 0.3
-------------------------------------------------------------------------------------------
EfficientNetB3 | 300x300 | 1.2 | 1.4 | 0.3
-------------------------------------------------------------------------------------------
EfficientNetB4 | 380x380 | 1.4 | 1.8 | 0.4
-------------------------------------------------------------------------------------------
EfficientNetB5 | 456x456 | 1.6 | 2.2 | 0.4
-------------------------------------------------------------------------------------------
EfficientNetB6 | 528x528 | 1.8 | 2.6 | 0.5
-------------------------------------------------------------------------------------------
EfficientNetB7 | 600x600 | 2.0 | 3.1 | 0.5
'''3.3 网络核心模块代码
（1）标准卷积块
一个标准卷积块由 普通卷积+批标准化+激活函数 组成
#（1）激活函数
def swish(x):
# swish激活函数
x = x*tf.nn.sigmoid(x)
return x
#（2）标准卷积
def conv_block(input_tensor, filters, kernel_size, stride, activation=True):
# 普通卷积+标准化+激活
x = layers.Conv2D(filters = filters, # 输出特征图个数
kernel_size = kernel_size, # 卷积核size
strides = stride, # 步长=2，size长宽减半
use_bias = False)(input_tensor) # 有BN层就不要偏置
x = layers.BatchNormalization()(x) # 批标准化
if activation: # 判断是否需要使用激活函数
x = swish(x) # 激活函数
return x（2）SE注意力机制
为了减少计算量，SE注意力机制中的全连接层可以换成1*1卷积层。这里要注意，第一个卷积层降维的通道数，是MBConv模块的输入特征图通道数的1/4，也就是在逆转残差模块中1*1卷积升维之前的特征图通道数的1/4
#（3）SE注意力机制
def squeeze_excitation(input_tensor, inputs_channel):
squeeze = inputs_channel / 4 # 通道数下降为输入该MBConv的特征图的1/4
excitation = input_tensor.shape[-1] # 通道数上升为深度卷积的输出特征图个数
# 全局平均池化 [h,w,c]==>[None,c]
x = layers.GlobalAveragePooling2D()(input_tensor)
# [None,c]==>[1,1,c]
x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)
# 1*1卷积降维，通道数变为输入MBblock模块的图像的通道数的1/4
x = layers.Conv2D(filters = squeeze,
kernel_size = (1,1),
strides = 1,
padding = 'same')(x)
x = swish(x) # swish激活函数
# 1*1卷积升维，通道数变为深度卷积的输出特征图个数
x = layers.Conv2D(filters = excitation,
kernel_size = (1,1),
strides = 1,
padding = 'same')(x)
x = tf.nn.sigmoid(x) # sigmoid激活函数
# 将深度卷积的输入特征图的每个通道和SE得到的针对每个通道的权重相乘
x = layers.multiply([input_tensor, x])
return x（3）逆转残差模块
以基本模块为例(stride=1)。如果需要提升特征图的通道数，那么先经过1x1卷积上升通道数；然后在高纬空间下使用深度卷积；再经过SE注意力机制优化特征图数据；再经过1x1卷积下降通道数（使用线性激活函数，y=x）；若此时输入特征图的shape和输出特征图的shape相同，那么对1x1卷积降维后的特征图加一个Dropout层，防止过拟合；最后残差连接输入和输出。
如第2.4小节所示。
#（4）逆转残差模块
def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):
'''
expansion代表第一个1*1卷积上升的通道数是输入图像通道数的expansion倍
out_channel代表MBConv模块输出通道数个数，即第二个1*1卷积的卷积核个数
dropout_rate代表dropout层随机杀死神经元的概率
'''
# 残差边
residual = x
# 输入的特征图的通道数
in_channel = x.shape[-1]
# ① 若expansion==1，1*1卷积升维就不用执行
if expansion != 1:
# 调用自定义的1*1标准卷积
x = conv_block(x,
filters=in_channel*expansion, # 通道数上升expansion倍
kernel_size=(1,1), stride=1, activation=True)
# ② 深度卷积
x = layers.DepthwiseConv2D(kernel_size = kernel_size,
strides = stride, # 步长=2下采样
padding = 'same', # 下采样时，特征图长宽减半
use_bias = False)(x) # 有BN层就不用偏置
x = layers.BatchNormalization()(x) # 批标准化
x = swish(x) # swish激活
# ③ SE注意力机制，传入深度卷积输出的tensor，和输入至MBConv模块的特征图通道数
x = squeeze_excitation(x, inputs_channel=in_channel)
# ④ 1*1卷积上升通道数，使用线性激活，即卷积+BN
x = conv_block(input_tensor = x,
filters = out_channel, # 1*1卷积输出通道数就是MBConv模块输出通道数
kernel_size=(1,1), stride=1,
activation = False)
# ⑤ 只有使用残差连接，并且dropout_rate>0时才会使用Dropout层
if stride == 1 and residual.shape == x.shape:
# 判断dropout_rate是否大于0
if dropout_rate > 0:
x = layers.Dropout(rate = dropout_rate)(x)
# 残差连接输入和输出
x = layers.Add()([residual, x])
return x
# 如果步长=2，直接输出1*1降维的结果
return x3.4 完整代码
以EfficientNet-B0为例，展示代码，如果需要使用其他EfficientNet系列的网络，只需要在主函数中（第9步）修改参数即可。
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Model, layers
import math
#（1）激活函数
def swish(x):
# swish激活函数
x = x*tf.nn.sigmoid(x)
return x
#（2）标准卷积
def conv_block(input_tensor, filters, kernel_size, stride, activation=True):
# 普通卷积+标准化+激活
x = layers.Conv2D(filters = filters, # 输出特征图个数
kernel_size = kernel_size, # 卷积核size
strides = stride, # 步长=2，size长宽减半
use_bias = False)(input_tensor) # 有BN层就不要偏置
x = layers.BatchNormalization()(x) # 批标准化
if activation: # 判断是否需要使用激活函数
x = swish(x) # 激活函数
return x
#（3）SE注意力机制
def squeeze_excitation(input_tensor, inputs_channel):
squeeze = inputs_channel / 4 # 通道数下降为输入该MBConv的特征图的1/4
excitation = input_tensor.shape[-1] # 通道数上升为深度卷积的输出特征图个数
# 全局平均池化 [h,w,c]==>[None,c]
x = layers.GlobalAveragePooling2D()(input_tensor)
# [None,c]==>[1,1,c]
x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)
# 1*1卷积降维，通道数变为输入MBblock模块的图像的通道数的1/4
x = layers.Conv2D(filters = squeeze,
kernel_size = (1,1),
strides = 1,
padding = 'same')(x)
x = swish(x) # swish激活函数
# 1*1卷积升维，通道数变为深度卷积的输出特征图个数
x = layers.Conv2D(filters = excitation,
kernel_size = (1,1),
strides = 1,
padding = 'same')(x)
x = tf.nn.sigmoid(x) # sigmoid激活函数
# 将深度卷积的输入特征图的每个通道和SE得到的针对每个通道的权重相乘
x = layers.multiply([input_tensor, x])
return x
#（4）逆转残差模块
def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):
'''
expansion代表第一个1*1卷积上升的通道数是输入图像通道数的expansion倍
out_channel代表MBConv模块输出通道数个数，即第二个1*1卷积的卷积核个数
dropout_rate代表dropout层随机杀死神经元的概率
'''
# 残差边
residual = x
# 输入的特征图的通道数
in_channel = x.shape[-1]
# ① 若expansion==1，1*1卷积升维就不用执行
if expansion != 1:
# 调用自定义的1*1标准卷积
x = conv_block(x,
filters=in_channel*expansion, # 通道数上升expansion倍
kernel_size=(1,1), stride=1, activation=True)
# ② 深度卷积
x = layers.DepthwiseConv2D(kernel_size = kernel_size,
strides = stride, # 步长=2下采样
padding = 'same', # 下采样时，特征图长宽减半
use_bias = False)(x) # 有BN层就不用偏置
x = layers.BatchNormalization()(x) # 批标准化
x = swish(x) # swish激活
# ③ SE注意力机制，传入深度卷积输出的tensor，和输入至MBConv模块的特征图通道数
x = squeeze_excitation(x, inputs_channel=in_channel)
# ④ 1*1卷积上升通道数，使用线性激活，即卷积+BN
x = conv_block(input_tensor = x,
filters = out_channel, # 1*1卷积输出通道数就是MBConv模块输出通道数
kernel_size=(1,1), stride=1,
activation = False)
# ⑤ 只有使用残差连接，并且dropout_rate>0时才会使用Dropout层
if stride == 1 and residual.shape == x.shape:
# 判断dropout_rate是否大于0
if dropout_rate > 0:
x = layers.Dropout(rate = dropout_rate)(x)
# 残差连接输入和输出
x = layers.Add()([residual, x])
return x
# 如果步长=2，直接输出1*1降维的结果
return x
#（5）一个stage模块是由多个MBConv模块组成
def stage(x, n, out_channel, expansion, kernel_size, stride, dropout_rate):
# 重复执行MBConv模块n次
for _ in range(n):
# 逆残差模块
x = MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate)
return x # 返回每个stage的输出特征图
#（6）通道数乘维度因子后，取8的倍数
def round_filters(filters, width_coefficient, divisor=8):
filters = filters * width_coefficient # 通道数乘宽度因子
# 新的通道数是距离远通道数最近的8的倍数
new_filters = max(divisor, int(filters + divisor/2) // divisor * divisor)
#
if new_filters < 0.9 * filters:
new_filters += filters
return new_filters
#（7）深度乘上深度因子后，向上取整
def round_repeats(repeats, depth_coefficient):
# 求得每一个卷积模块重复执行的次数
repeats = int(math.ceil(repeats * depth_coefficient)) #向上取整后小数部分=0，int()舍弃小数部分
return repeats
#（8）主干模型结构
def efficientnet(input_shape, classes, width_coefficient, depth_coefficient, dropout_rate):
'''
width_coefficient，通道维度上的倍率因子。与卷积核个数相乘，取整到离它最近的8的倍数
depth_coefficient，深度维度上的倍率因子。和模块重复次数相乘，向上取整
dropout_rate，dropout层杀死神经元的概率
'''
# 构建输入层
inputs = keras.Input(shape=input_shape)
# 标准卷积 [224,224,3]==>[112,112,32]
x = conv_block(inputs, filters=round_filters(32, width_coefficient), # 维度因子改变卷积核个数
kernel_size=(3,3), stride=2)
# [112,112,32]==>[112,112,16]
x = stage(x, n=round_repeats(1, depth_coefficient), expansion=1, out_channel=round_filters(16, width_coefficient),
kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)
# [112,112,16]==>[56,56,24]
x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(24, width_coefficient),
expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)
# [56,56,24]==>[28,28,40]
x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(40, width_coefficient),
expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)
# [28,28,40]==>[14,14,80]
x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(80, width_coefficient),
expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)
# [14,14,80]==>[14,14,112]
x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(112, width_coefficient),
expansion=6, kernel_size=(5,5), stride=1, dropout_rate=dropout_rate)
# [14,14,112]==>[7,7,192]
x = stage(x, n=round_repeats(4, depth_coefficient), out_channel=round_filters(192, width_coefficient),
expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)
# [7,7,192]==>[7,7,320]
x = stage(x, n=round_repeats(1, depth_coefficient), out_channel=round_filters(320, width_coefficient),
expansion=6, kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)
# [7,7,320]==>[7,7,1280]
x = layers.Conv2D(filters=1280, kernel_size=(1*1), strides=1,
padding='same', use_bias=False)(x)
x = layers.BatchNormalization()(x)
x = swish(x)
# [7,7,1280]==>[None,1280]
x = layers.GlobalAveragePooling2D()(x)
# [None,1280]==>[None,1000]
x = layers.Dropout(rate=dropout_rate)(x) # 随机杀死神经元防止过拟合
logits = layers.Dense(classes)(x) # 训练时再使用softmax
# 构建模型
model = Model(inputs, logits)
return model
#（9）接收网络模型
if __name__ == '__main__':
# 以efficientnetB0为例，输入参数
model = efficientnet(input_shape=[224,224,3], classes=1000, # 输入图象size，分类数
width_coefficient=1.0, depth_coefficient=1.0,
dropout_rate=0.2)
model.summary() # 参看网络模型结构3.5 查看网络架构
使用model.summary()查看网络架构，EfficientNet-B0有五百多万参数
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 111, 111, 32) 864 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 111, 111, 32) 128 conv2d[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
tf.math.multiply (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]
tf.math.sigmoid[0][0]
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 111, 111, 32) 288 tf.math.multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 111, 111, 32) 128 depthwise_conv2d[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
tf.math.multiply_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]
tf.math.sigmoid_1[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 32) 0 tf.math.multiply_1[0][0]
__________________________________________________________________________________________________
reshape (Reshape) (None, 1, 1, 32) 0 global_average_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 1, 1, 8) 264 reshape[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_2 (TFOpLambda) (None, 1, 1, 8) 0 conv2d_1[0][0]
__________________________________________________________________________________________________
tf.math.multiply_2 (TFOpLambda) (None, 1, 1, 8) 0 conv2d_1[0][0]
tf.math.sigmoid_2[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 1, 1, 32) 288 tf.math.multiply_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_3 (TFOpLambda) (None, 1, 1, 32) 0 conv2d_2[0][0]
__________________________________________________________________________________________________
multiply (Multiply) (None, 111, 111, 32) 0 tf.math.multiply_1[0][0]
tf.math.sigmoid_3[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 111, 111, 16) 512 multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 111, 111, 16) 64 conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 111, 111, 96) 1536 batch_normalization_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 111, 111, 96) 384 conv2d_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_4 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
tf.math.multiply_3 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]
tf.math.sigmoid_4[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 56, 56, 96) 864 tf.math.multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 56, 56, 96) 384 depthwise_conv2d_1[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_5 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
tf.math.multiply_4 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]
tf.math.sigmoid_5[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_1 (Glo (None, 96) 0 tf.math.multiply_4[0][0]
__________________________________________________________________________________________________
reshape_1 (Reshape) (None, 1, 1, 96) 0 global_average_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 1, 1, 4) 388 reshape_1[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_6 (TFOpLambda) (None, 1, 1, 4) 0 conv2d_5[0][0]
__________________________________________________________________________________________________
tf.math.multiply_5 (TFOpLambda) (None, 1, 1, 4) 0 conv2d_5[0][0]
tf.math.sigmoid_6[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 1, 1, 96) 480 tf.math.multiply_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_7 (TFOpLambda) (None, 1, 1, 96) 0 conv2d_6[0][0]
__________________________________________________________________________________________________
multiply_1 (Multiply) (None, 56, 56, 96) 0 tf.math.multiply_4[0][0]
tf.math.sigmoid_7[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 56, 56, 24) 2304 multiply_1[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 56, 56, 24) 96 conv2d_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 56, 56, 144) 3456 batch_normalization_5[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 56, 56, 144) 576 conv2d_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_8 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
tf.math.multiply_6 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]
tf.math.sigmoid_8[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 144) 1296 tf.math.multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 28, 28, 144) 576 depthwise_conv2d_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
tf.math.multiply_7 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]
tf.math.sigmoid_9[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 144) 0 tf.math.multiply_7[0][0]
__________________________________________________________________________________________________
reshape_2 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 1, 1, 6) 870 reshape_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_10 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_9[0][0]
__________________________________________________________________________________________________
tf.math.multiply_8 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_9[0][0]
tf.math.sigmoid_10[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_11 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_10[0][0]
__________________________________________________________________________________________________
multiply_2 (Multiply) (None, 28, 28, 144) 0 tf.math.multiply_7[0][0]
tf.math.sigmoid_11[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 28, 28, 24) 3456 multiply_2[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 28, 28, 24) 96 conv2d_11[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 28, 28, 144) 3456 batch_normalization_8[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 28, 28, 144) 576 conv2d_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_12 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
tf.math.multiply_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]
tf.math.sigmoid_12[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 14, 14, 144) 3600 tf.math.multiply_9[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 14, 14, 144) 576 depthwise_conv2d_3[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_13 (TFOpLambda) (None, 14, 14, 144) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
tf.math.multiply_10 (TFOpLambda (None, 14, 14, 144) 0 batch_normalization_10[0][0]
tf.math.sigmoid_13[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_3 (Glo (None, 144) 0 tf.math.multiply_10[0][0]
__________________________________________________________________________________________________
reshape_3 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_3[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 1, 1, 6) 870 reshape_3[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_14 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_13[0][0]
__________________________________________________________________________________________________
tf.math.multiply_11 (TFOpLambda (None, 1, 1, 6) 0 conv2d_13[0][0]
tf.math.sigmoid_14[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_15 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_14[0][0]
__________________________________________________________________________________________________
multiply_3 (Multiply) (None, 14, 14, 144) 0 tf.math.multiply_10[0][0]
tf.math.sigmoid_15[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 14, 14, 40) 5760 multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 14, 14, 40) 160 conv2d_15[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 14, 14, 240) 9600 batch_normalization_11[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 14, 14, 240) 960 conv2d_16[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_16 (TFOpLambda) (None, 14, 14, 240) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
tf.math.multiply_12 (TFOpLambda (None, 14, 14, 240) 0 batch_normalization_12[0][0]
tf.math.sigmoid_16[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 7, 7, 240) 6000 tf.math.multiply_12[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_17 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
tf.math.multiply_13 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_13[0][0]
tf.math.sigmoid_17[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_4 (Glo (None, 240) 0 tf.math.multiply_13[0][0]
__________________________________________________________________________________________________
reshape_4 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_4[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 1, 1, 10) 2410 reshape_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_18 (TFOpLambda) (None, 1, 1, 10) 0 conv2d_17[0][0]
__________________________________________________________________________________________________
tf.math.multiply_14 (TFOpLambda (None, 1, 1, 10) 0 conv2d_17[0][0]
tf.math.sigmoid_18[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_19 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_18[0][0]
__________________________________________________________________________________________________
multiply_4 (Multiply) (None, 7, 7, 240) 0 tf.math.multiply_13[0][0]
tf.math.sigmoid_19[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 7, 7, 40) 9600 multiply_4[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 7, 7, 40) 160 conv2d_19[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 7, 7, 240) 9600 batch_normalization_14[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 7, 7, 240) 960 conv2d_20[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_20 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
tf.math.multiply_15 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_15[0][0]
tf.math.sigmoid_20[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 4, 4, 240) 2160 tf.math.multiply_15[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 4, 4, 240) 960 depthwise_conv2d_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_21 (TFOpLambda) (None, 4, 4, 240) 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
tf.math.multiply_16 (TFOpLambda (None, 4, 4, 240) 0 batch_normalization_16[0][0]
tf.math.sigmoid_21[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_5 (Glo (None, 240) 0 tf.math.multiply_16[0][0]
__________________________________________________________________________________________________
reshape_5 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_5[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 1, 1, 10) 2410 reshape_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_22 (TFOpLambda) (None, 1, 1, 10) 0 conv2d_21[0][0]
__________________________________________________________________________________________________
tf.math.multiply_17 (TFOpLambda (None, 1, 1, 10) 0 conv2d_21[0][0]
tf.math.sigmoid_22[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_17[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_23 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_22[0][0]
__________________________________________________________________________________________________
multiply_5 (Multiply) (None, 4, 4, 240) 0 tf.math.multiply_16[0][0]
tf.math.sigmoid_23[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 4, 4, 80) 19200 multiply_5[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 4, 4, 80) 320 conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 4, 4, 480) 38400 batch_normalization_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 4, 4, 480) 1920 conv2d_24[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_24 (TFOpLambda) (None, 4, 4, 480) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
tf.math.multiply_18 (TFOpLambda (None, 4, 4, 480) 0 batch_normalization_18[0][0]
tf.math.sigmoid_24[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 2, 2, 480) 4320 tf.math.multiply_18[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 2, 2, 480) 1920 depthwise_conv2d_6[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_25 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
tf.math.multiply_19 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_19[0][0]
tf.math.sigmoid_25[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_6 (Glo (None, 480) 0 tf.math.multiply_19[0][0]
__________________________________________________________________________________________________
reshape_6 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_6[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 1, 1, 20) 9620 reshape_6[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_26 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_25[0][0]
__________________________________________________________________________________________________
tf.math.multiply_20 (TFOpLambda (None, 1, 1, 20) 0 conv2d_25[0][0]
tf.math.sigmoid_26[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_20[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_27 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_26[0][0]
__________________________________________________________________________________________________
multiply_6 (Multiply) (None, 2, 2, 480) 0 tf.math.multiply_19[0][0]
tf.math.sigmoid_27[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 2, 2, 80) 38400 multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 2, 2, 80) 320 conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 2, 2, 480) 38400 batch_normalization_20[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 2, 2, 480) 1920 conv2d_28[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_28 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
tf.math.multiply_21 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_21[0][0]
tf.math.sigmoid_28[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 1, 1, 480) 4320 tf.math.multiply_21[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_7[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_29 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
tf.math.multiply_22 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_22[0][0]
tf.math.sigmoid_29[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_7 (Glo (None, 480) 0 tf.math.multiply_22[0][0]
__________________________________________________________________________________________________
reshape_7 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_7[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 1, 1, 20) 9620 reshape_7[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_30 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_29[0][0]
__________________________________________________________________________________________________
tf.math.multiply_23 (TFOpLambda (None, 1, 1, 20) 0 conv2d_29[0][0]
tf.math.sigmoid_30[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_23[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_31 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_30[0][0]
__________________________________________________________________________________________________
multiply_7 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_22[0][0]
tf.math.sigmoid_31[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 1, 1, 80) 38400 multiply_7[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 1, 1, 80) 320 conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 1, 1, 480) 38400 batch_normalization_23[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 1, 1, 480) 1920 conv2d_32[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_32 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
tf.math.multiply_24 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_24[0][0]
tf.math.sigmoid_32[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 1, 1, 480) 12000 tf.math.multiply_24[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_33 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
tf.math.multiply_25 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_25[0][0]
tf.math.sigmoid_33[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_8 (Glo (None, 480) 0 tf.math.multiply_25[0][0]
__________________________________________________________________________________________________
reshape_8 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_8[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 1, 1, 20) 9620 reshape_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_34 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_33[0][0]
__________________________________________________________________________________________________
tf.math.multiply_26 (TFOpLambda (None, 1, 1, 20) 0 conv2d_33[0][0]
tf.math.sigmoid_34[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_26[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_35 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_34[0][0]
__________________________________________________________________________________________________
multiply_8 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_25[0][0]
tf.math.sigmoid_35[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 1, 1, 112) 53760 multiply_8[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 1, 1, 112) 448 conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_26[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 1, 1, 672) 2688 conv2d_36[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_36 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
tf.math.multiply_27 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_27[0][0]
tf.math.sigmoid_36[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 1, 1, 672) 16800 tf.math.multiply_27[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_9[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_37 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
tf.math.multiply_28 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_28[0][0]
tf.math.sigmoid_37[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_9 (Glo (None, 672) 0 tf.math.multiply_28[0][0]
__________________________________________________________________________________________________
reshape_9 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_9[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 1, 1, 28) 18844 reshape_9[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_38 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_37[0][0]
__________________________________________________________________________________________________
tf.math.multiply_29 (TFOpLambda (None, 1, 1, 28) 0 conv2d_37[0][0]
tf.math.sigmoid_38[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_29[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_39 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_38[0][0]
__________________________________________________________________________________________________
multiply_9 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_28[0][0]
tf.math.sigmoid_39[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 1, 1, 112) 75264 multiply_9[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 1, 1, 112) 448 conv2d_39[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_29[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 1, 1, 672) 2688 conv2d_40[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_40 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
tf.math.multiply_30 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_30[0][0]
tf.math.sigmoid_40[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_30[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_10[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_41 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
tf.math.multiply_31 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_31[0][0]
tf.math.sigmoid_41[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_10 (Gl (None, 672) 0 tf.math.multiply_31[0][0]
__________________________________________________________________________________________________
reshape_10 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_10[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 1, 1, 28) 18844 reshape_10[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_42 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_41[0][0]
__________________________________________________________________________________________________
tf.math.multiply_32 (TFOpLambda (None, 1, 1, 28) 0 conv2d_41[0][0]
tf.math.sigmoid_42[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_32[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_43 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_42[0][0]
__________________________________________________________________________________________________
multiply_10 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_31[0][0]
tf.math.sigmoid_43[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 1, 1, 112) 75264 multiply_10[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 1, 1, 112) 448 conv2d_43[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_32[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 1, 1, 672) 2688 conv2d_44[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_44 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
tf.math.multiply_33 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_33[0][0]
tf.math.sigmoid_44[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_33[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_45 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
tf.math.multiply_34 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_34[0][0]
tf.math.sigmoid_45[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_11 (Gl (None, 672) 0 tf.math.multiply_34[0][0]
__________________________________________________________________________________________________
reshape_11 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_11[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 1, 1, 28) 18844 reshape_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_46 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_45[0][0]
__________________________________________________________________________________________________
tf.math.multiply_35 (TFOpLambda (None, 1, 1, 28) 0 conv2d_45[0][0]
tf.math.sigmoid_46[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_35[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_47 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_46[0][0]
__________________________________________________________________________________________________
multiply_11 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_34[0][0]
tf.math.sigmoid_47[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 1, 1, 192) 129024 multiply_11[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 1, 1, 192) 768 conv2d_47[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_35[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 1, 1, 1152) 4608 conv2d_48[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_48 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
tf.math.multiply_36 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_36[0][0]
tf.math.sigmoid_48[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_36[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_49 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
tf.math.multiply_37 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_37[0][0]
tf.math.sigmoid_49[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_12 (Gl (None, 1152) 0 tf.math.multiply_37[0][0]
__________________________________________________________________________________________________
reshape_12 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_12[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, 1, 1, 48) 55344 reshape_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_50 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_49[0][0]
__________________________________________________________________________________________________
tf.math.multiply_38 (TFOpLambda (None, 1, 1, 48) 0 conv2d_49[0][0]
tf.math.sigmoid_50[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_38[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_51 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_50[0][0]
__________________________________________________________________________________________________
multiply_12 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_37[0][0]
tf.math.sigmoid_51[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, 1, 1, 192) 221184 multiply_12[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 1, 1, 192) 768 conv2d_51[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 1, 1, 1152) 4608 conv2d_52[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_52 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
tf.math.multiply_39 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_39[0][0]
tf.math.sigmoid_52[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_39[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_13[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_53 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
tf.math.multiply_40 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_40[0][0]
tf.math.sigmoid_53[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_13 (Gl (None, 1152) 0 tf.math.multiply_40[0][0]
__________________________________________________________________________________________________
reshape_13 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_13[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, 1, 1, 48) 55344 reshape_13[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_54 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_53[0][0]
__________________________________________________________________________________________________
tf.math.multiply_41 (TFOpLambda (None, 1, 1, 48) 0 conv2d_53[0][0]
tf.math.sigmoid_54[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_41[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_55 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_54[0][0]
__________________________________________________________________________________________________
multiply_13 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_40[0][0]
tf.math.sigmoid_55[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, 1, 1, 192) 221184 multiply_13[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 1, 1, 192) 768 conv2d_55[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_41[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 1, 1, 1152) 4608 conv2d_56[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_56 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
tf.math.multiply_42 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_42[0][0]
tf.math.sigmoid_56[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_42[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_57 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
tf.math.multiply_43 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_43[0][0]
tf.math.sigmoid_57[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_14 (Gl (None, 1152) 0 tf.math.multiply_43[0][0]
__________________________________________________________________________________________________
reshape_14 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_14[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, 1, 1, 48) 55344 reshape_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_58 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_57[0][0]
__________________________________________________________________________________________________
tf.math.multiply_44 (TFOpLambda (None, 1, 1, 48) 0 conv2d_57[0][0]
tf.math.sigmoid_58[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_44[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_59 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_58[0][0]
__________________________________________________________________________________________________
multiply_14 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_43[0][0]
tf.math.sigmoid_59[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, 1, 1, 192) 221184 multiply_14[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 1, 1, 192) 768 conv2d_59[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_44[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 1, 1, 1152) 4608 conv2d_60[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_60 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
tf.math.multiply_45 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_45[0][0]
tf.math.sigmoid_60[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_15 (DepthwiseC (None, 1, 1, 1152) 10368 tf.math.multiply_45[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_15[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_61 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
tf.math.multiply_46 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_46[0][0]
tf.math.sigmoid_61[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_15 (Gl (None, 1152) 0 tf.math.multiply_46[0][0]
__________________________________________________________________________________________________
reshape_15 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_15[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, 1, 1, 48) 55344 reshape_15[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_62 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_61[0][0]
__________________________________________________________________________________________________
tf.math.multiply_47 (TFOpLambda (None, 1, 1, 48) 0 conv2d_61[0][0]
tf.math.sigmoid_62[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_47[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_63 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_62[0][0]
__________________________________________________________________________________________________
multiply_15 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_46[0][0]
tf.math.sigmoid_63[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, 1, 1, 320) 368640 multiply_15[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 1, 1, 320) 1280 conv2d_63[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, 1, 1, 1280) 409600 batch_normalization_47[0][0]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 1, 1, 1280) 5120 conv2d_64[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_64 (TFOpLambda) (None, 1, 1, 1280) 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
tf.math.multiply_48 (TFOpLambda (None, 1, 1, 1280) 0 batch_normalization_48[0][0]
tf.math.sigmoid_64[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_16 (Gl (None, 1280) 0 tf.math.multiply_48[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1280) 0 global_average_pooling2d_16[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1000) 1281000 dropout[0][0]
==================================================================================================
Total params: 5,330,564
Trainable params: 5,288,548
Non-trainable params: 42,016
__________________________________________________________________________________________________