Tensorflow2.0：使用Keras自定义网络实战
tensorflow2.0建议使用tf.keras作为构建神经网络的高级API
接下来我就使用tensorflow实现VGG16去训练数据
背景介绍：
2012年 AlexNet 在 ImageNet 上显著的降低了分类错误率，深度神经网络进入迅速发展阶段。在2014年牛津大学机器人实验室尝试构建了更深的网络，文章中称为"VERY DEEP CONVOLUTIONAL NETWORKS"，如VGG16，有16层，虽然现在看起来稀疏平常，但与 AlexNet 相比，翻了几倍。这个阶段，主要是没有解决网络太深梯度反向传播消失的问题，且受限于GPU等硬件设备的性能，所以深度网络不易于训练。不过，VGG 显然是当时最好的图像分类模型，斩获 ILSVRC 比赛冠军。顺便说下，2012年之后，标准数据集主要是ImageNet，到后来又有微软的COCO数据集。
上图为VGG16的网络结构，我们可以从输入层开始数，数到最终的输出层，正好是16层，有兴趣的可以数一数，接下来我们就来实现这样的一个网络
实现：
- Stage 1
卷积层1：conv1
卷积层2：conv2
池化层1：pool1
激活函数1：relu1
self.conv1 = layers.Conv2D(64,3,1,'same')
self.conv2 = layers.Conv2D(64,3,1,'same')
self.pool1 = layers.MaxPool2D(2,1,'same')
self.relu1 = layers.ReLU()
- Stage 2
卷积层3：conv3
卷积层4：conv4
池化层2：pool2
激活函数2：relu2
self.conv3 = layers.Conv2D(128, 3, 1, 'same')
self.conv4 = layers.Conv2D(128, 3, 1, 'same')
self.pool2 = layers.MaxPool2D(2, 1, 'same')
self.relu2 = layers.ReLU()
- Stage 3
卷积层5：conv5
卷积层6：conv6
卷积层7：conv7
池化层3：pool3
激活函数3：relu3
self.conv5 = layers.Conv2D(256,3,1,'same')
self.conv6 = layers.Conv2D(256,3,1,'same')
self.conv7 = layers.Conv2D(256,3,1,'same')
self.pool3 = layers.MaxPool2D(2,1,'same')
self.relu3 = layers.ReLU()
- Stage 4
卷积层8：conv8
卷积层9：conv9
卷积层10：conv10
池化层4：pool4
激活函数4：relu4
self.conv8 = layers.Conv2D(512,3,1,'same')
self.conv9 = layers.Conv2D(512,3,1,'same')
self.conv10 = layers.Conv2D(512,3,1,'same')
self.pool4=layers.MaxPool2D(2,1,'same')
self.relu4=layers.ReLU()
- Statage 5
卷积层11：conv11
卷积层12：conv12
卷积层13：conv13
池化层5：pool5
激活函数5：relu5
self.conv8 = layers.Conv2D(512,3,1,'same')
self.conv9 = layers.Conv2D(512,3,1,'same')
self.conv10 = layers.Conv2D(512,3,1,'same')
self.pool4=layers.MaxPool2D(2,1,'same')
self.relu4=layers.ReLU()
接着实现call函数：
def call(self, inputs, training=None, mask=None):
out=self.conv1(inputs)
out=self.conv2(out)
out=self.relu1(out)
out=self.pool1(out)
out=self.conv3(out)
out=self.conv4(out)
out=self.relu2(out)
out=self.pool2(out)
out=self.conv5(out)
out=self.conv6(out)
out=self.conv7(out)
out=self.relu3(out)
out=self.pool3(out)
out=self.conv8(out)
out=self.conv9(out)
out=self.conv10(out)
out=self.relu4(out)
out=self.pool4(out)
out=self.conv11(out)
out=self.conv12(out)
out=self.conv13(out)
out=self.relu5(out)
out=self.pool5(out)
out=self.avgpool(out)
out=self.fc(out)
return out
网络搭建好了之后，我们可以使用model.summary()的方法查看一下网络结构和相关参数
查看网络结构：
训练
1. 数据的预处理
import os
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets,Sequential,layers,optimizers,metrics
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
def preprocess(x,y):
x=2*tf.cast(x,dtype=tf.float32)/255.-1.
y=tf.cast(y,dtype=tf.int32)
return x,y
batchsz=128
# load dataset
(x_train,y_train),(x_test,y_test)=datasets.cifar10.load_data()
y_train=tf.squeeze(y_train)
y_test=tf.squeeze(y_test)
y_train=tf.one_hot(y_train,depth=10)
y_test=tf.one_hot(y_test,depth=10)
2. 加载数据
这里为了训练方便，就使用CIFAR10的数据集了，获取该数据集很方便，只需keras.datasets.cifar10.load_data()即可获得
# train data
train_date=tf.data.Dataset.from_tensor_slices((x_train,y_train))
train_date=train_date.map(preprocess).shuffle(10000).batch(batchsz)
# test data
test_data=tf.data.Dataset.from_tensor_slices((x_test,y_test))
test_data=test_data.map(preprocess).batch(batchsz)
3. 搭建网络结构
class VGG(keras.Model):
def __init__(self,num_class=10):
super(VGG, self).__init__()
self.conv1 = layers.Conv2D(64,3,1,'same')
self.conv2 = layers.Conv2D(64,3,1,'same')
self.pool1 = layers.MaxPool2D(2,1,'same')
self.relu1 = layers.ReLU()
self.conv3 = layers.Conv2D(128, 3, 1, 'same')
self.conv4 = layers.Conv2D(128, 3, 1, 'same')
self.relu2 = layers.ReLU()
self.pool2 = layers.MaxPool2D(2, 1, 'same')
self.conv5 = layers.Conv2D(256,3,1,'same')
self.conv6 = layers.Conv2D(256,3,1,'same')
self.conv7 = layers.Conv2D(256,3,1,'same')
self.relu3 = layers.ReLU()
self.pool3 = layers.MaxPool2D(2,1,'same')
self.conv8 = layers.Conv2D(512,3,1,'same')
self.conv9 = layers.Conv2D(512,3,1,'same')
self.conv10 = layers.Conv2D(512,3,1,'same')
self.relu4=layers.ReLU()
self.pool4=layers.MaxPool2D(2,1,'same')
self.conv11 = layers.Conv2D(512,3,1,'same')
self.conv12 = layers.Conv2D(512,3,1,'same')
self.conv13 = layers.Conv2D(512,3,1,'same')
self.relu5 = layers.ReLU()
self.pool5 = layers.MaxPool2D(2,1,'same')
self.avgpool = layers.GlobalAveragePooling2D()
self.fc = layers.Dense(num_class)
def call(self, inputs, training=None, mask=None):
out=self.conv1(inputs)
out=self.conv2(out)
out=self.relu1(out)
out=self.pool1(out)
out=self.conv3(out)
out=self.conv4(out)
out=self.relu2(out)
out=self.pool2(out)
out=self.conv5(out)
out=self.conv6(out)
out=self.conv7(out)
out=self.relu3(out)
out=self.pool3(out)
out=self.conv8(out)
out=self.conv9(out)
out=self.conv10(out)
out=self.relu4(out)
out=self.pool4(out)
out=self.conv11(out)
out=self.conv12(out)
out=self.conv13(out)
out=self.relu5(out)
out=self.pool5(out)
out=self.avgpool(out)
out=self.fc(out)
return out
可以看到搭建网络的方式和pytorch很相似
4. 预训练
model=VGG()
model.build(input_shape=(None,32,32,3))
model.compile(optimizer=optimizers.Adam(1e-4),
loss=tf.losses.CategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(train_date, epochs=15, validation_data=test_data, validation_freq=1)
5. 训练数据
这里我们同样使用Tensorflow提供的一个接口compile实现训练，大家也可以改用其他的方法实现数据的更新。
下面为训练的数据，可以看到前半部的效果还是不错的，test_acc达到了82%，但是后面出现了过拟合的现象，笔者这里训练了30个epoch,这30个epoch训练了大概四个小时的样子；大家有兴趣的话，可以将参数进行优化，防止过拟合，再训练以达到更好的效果。最后希望大家多多实践，共同进步。
Epoch 1/30
391/391 [==============================] - 428s 1s/step - loss: 1.6934 - acc: 0.3565 - val_loss: 1.4043 - val_acc: 0.4945
Epoch 2/30
391/391 [==============================] - 424s 1s/step - loss: 1.2890 - acc: 0.5294 - val_loss: 1.1885 - val_acc: 0.5673
Epoch 3/30
391/391 [==============================] - 423s 1s/step - loss: 1.0865 - acc: 0.6059 - val_loss: 1.0395 - val_acc: 0.6334
Epoch 4/30
391/391 [==============================] - 423s 1s/step - loss: 0.9334 - acc: 0.6637 - val_loss: 0.8658 - val_acc: 0.6929
Epoch 5/30
391/391 [==============================] - 422s 1s/step - loss: 0.8382 - acc: 0.7040 - val_loss: 0.8632 - val_acc: 0.7012
Epoch 6/30
391/391 [==============================] - 422s 1s/step - loss: 0.7352 - acc: 0.7410 - val_loss: 0.8306 - val_acc: 0.7171
Epoch 7/30
391/391 [==============================] - 422s 1s/step - loss: 0.6655 - acc: 0.7671 - val_loss: 0.8549 - val_acc: 0.7071
Epoch 8/30
391/391 [==============================] - 421s 1s/step - loss: 0.5957 - acc: 0.7929 - val_loss: 0.6909 - val_acc: 0.7699
Epoch 9/30
391/391 [==============================] - 421s 1s/step - loss: 0.5478 - acc: 0.8103 - val_loss: 0.6670 - val_acc: 0.7755
Epoch 10/30
391/391 [==============================] - 421s 1s/step - loss: 0.5001 - acc: 0.8261 - val_loss: 0.5885 - val_acc: 0.8019
Epoch 11/30
391/391 [==============================] - 421s 1s/step - loss: 0.4494 - acc: 0.8442 - val_loss: 0.6598 - val_acc: 0.7872
Epoch 12/30
391/391 [==============================] - 420s 1s/step - loss: 0.4189 - acc: 0.8534 - val_loss: 0.6492 - val_acc: 0.7789
Epoch 13/30
391/391 [==============================] - 420s 1s/step - loss: 0.3793 - acc: 0.8680 - val_loss: 0.5678 - val_acc: 0.8087
Epoch 14/30
391/391 [==============================] - 421s 1s/step - loss: 0.3489 - acc: 0.8777 - val_loss: 0.6082 - val_acc: 0.8030
Epoch 15/30
391/391 [==============================] - 421s 1s/step - loss: 0.3194 - acc: 0.8881 - val_loss: 0.6338 - val_acc: 0.8107
Epoch 16/30
391/391 [==============================] - 421s 1s/step - loss: 0.2814 - acc: 0.9022 - val_loss: 0.6844 - val_acc: 0.7938
Epoch 17/30
391/391 [==============================] - 420s 1s/step - loss: 0.2697 - acc: 0.9045 - val_loss: 0.6159 - val_acc: 0.8151
Epoch 18/30
391/391 [==============================] - 421s 1s/step - loss: 0.2419 - acc: 0.9153 - val_loss: 0.6183 - val_acc: 0.8203
Epoch 19/30
391/391 [==============================] - 421s 1s/step - loss: 0.2186 - acc: 0.9232 - val_loss: 0.6346 - val_acc: 0.8121
Epoch 20/30
391/391 [==============================] - 420s 1s/step - loss: 0.2103 - acc: 0.9251 - val_loss: 0.7089 - val_acc: 0.8005
Epoch 21/30
391/391 [==============================] - 420s 1s/step - loss: 0.1888 - acc: 0.9328 - val_loss: 0.7297 - val_acc: 0.8097
Epoch 22/30
390/391 [============================>.] - ETA: 1s - loss: 0.1830 - acc: 0.9341Epoch 23/30
391/391 [==============================] - 419s 1s/step - loss: 0.1679 - acc: 0.9417 - val_loss: 0.7396 - val_acc: 0.8165
Epoch 24/30
391/391 [==============================] - 420s 1s/step - loss: 0.1646 - acc: 0.9419 - val_loss: 0.6877 - val_acc: 0.8199
Epoch 25/30
391/391 [==============================] - 420s 1s/step - loss: 0.1406 - acc: 0.9504 - val_loss: 0.7773 - val_acc: 0.8163
Epoch 26/30
391/391 [==============================] - 420s 1s/step - loss: 0.1590 - acc: 0.9438 - val_loss: 0.7408 - val_acc: 0.8143
Epoch 27/30
391/391 [==============================] - 420s 1s/step - loss: 0.1392 - acc: 0.9502 - val_loss: 0.7879 - val_acc: 0.8115
Epoch 28/30
391/391 [==============================] - 420s 1s/step - loss: 0.1309 - acc: 0.9537 - val_loss: 0.8269 - val_acc: 0.8189
Epoch 29/30
391/391 [==============================] - 420s 1s/step - loss: 0.1226 - acc: 0.9582 - val_loss: 0.7843 - val_acc: 0.8182
Epoch 30/30
391/391 [==============================] - 420s 1s/step - loss: 0.1167 - acc: 0.9591 - val_loss: 0.8827 - val_acc: 0.8187