准确率98%! 手把手教你用tensorflow 2对交通标志进行识别
文章目录
- 1 概述
- 2 数据集
-
- 2.1 下载数据
- 2.2 分析数据
- 3 使用LeNet进行分类
-
- 3.1 模型定义
- 3.2 模型训练
- 4 优化
-
- 4.1 Dropout
-
- 4.1.1 死亡的`ReLU`
- 4.2 图片增强(Image Augmentation)
- 4.3 模型优化
- 5 再优化
-
- 5.1 分类错误的图片
- 5.2 图片再增强
- 5.3 最后的训练
- 6 思考
1 概述
对图片进行分类是常见的计算机视觉任务，其中对交通标志图片的分类是自动驾驶识别算法中必不可少的，在本文，我们一起学习怎么用tensorflow2实现一个交通标志图片分类器并最终达到98%的准确率。
我在本文里使用的是jupyter notebook，读者朋友也可以使用其他编辑器。
2 数据集
本文使用的数据是一份德国的交通标志图片，数据地址为：https://bitbucket.org/jadslim/german-traffic-signs
2.1 下载数据
首先，我们把数据下载下来。
git clone https://bitbucket.org/jadslim/german-traffic-signs
cd german-traffic-signs
ls
signnames.csv test.p train.p valid.p
head signnames.csv
ClassId,SignName
0,Speed limit (20km/h)
1,Speed limit (30km/h)
2,Speed limit (50km/h)
3,Speed limit (60km/h)
4,Speed limit (70km/h)
5,Speed limit (80km/h)
6,End of speed limit (80km/h)
7,Speed limit (100km/h)
8,Speed limit (120km/h)
下载下来的文件有3个pickle序列化的文件和一个csv文件，可以看到csv文件是分类的id对应的名称；3个pickle文件分别为测试、训练和验证数据。我们接下加载数据并分析一下数据的构成。
2.2 分析数据
我们分别使用pickle和pandas来加载这些数据。
import pickle
import pandas as pd
with open('german-traffic-signs/train.p', 'rb') as f:
train_data = pickle.load(f)
with open('german-traffic-signs/valid.p', 'rb') as f:
val_data = pickle.load(f)
with open('german-traffic-signs/test.p', 'rb') as f:
test_data = pickle.load(f)
signnames = pd.read_csv('german-traffic-signs/signnames.csv')
print(len(signnames))
print(type(train_data))
print(train_data.keys()
43
dict_keys(['coords', 'labels', 'features', 'sizes'])
可以看到共有43个类别的交通标志，pickle文件反序列化后是dict对象，该对象有4个属性，对于我们的任务，只需要用到labels和features，分别为图片的分类id和图片本身。
我们看看数据集的大小：
X_train, y_train = train_data['features'], train_data['labels']
X_val, y_val = val_data['features'], val_data['labels']
X_test, y_test = test_data['features'], test_data['labels']
print('Training data:', X_train.shape)
print('Validation data:', X_val.shape)
print('Test data:', X_test.shape)
训练数据有34799个，验证数据有4410个，测试数据有12630个；图片的大小为32 x 32，有3个颜色channel：
Training data: (34799, 32, 32, 3)
Validation data: (4410, 32, 32, 3)
Test data: (12630, 32, 32, 3)
接着，我们随机显示每个类别的5张图片和对应的标签看看。
import matplotlib.pyplot as plt
import random
%matplotlib inline
num_of_samples = []
cols = 5
num_classes = 43
fig, axs = plt.subplots(nrows=num_classes, ncols=cols, figsize=(5, 50))
fig.tight_layout()
for i in range(cols):
for j, row in signnames.iterrows():
x_selected = X_train[y_train == j]
axs[j][i].imshow(x_selected[random.randint(0, (len(x_selected) - 1)), :, :], cmap=plt.get_cmap('gray'))
axs[j][i].axis("off")
if i == 2:
axs[j][i].set_title(str(j) + " - " + row["SignName"])
num_of_samples.append(len(x_selected))
图片的差异比较大：明暗不一、背景不一等等。
接下来看一下每个类别的图片的数量：
print(num_of_samples)
plt.figure(figsize=(12, 4))
plt.bar(range(0, num_classes), num_of_samples)
plt.title("各分类图片的分布")
plt.xlabel("类别id")
plt.ylabel("数量")
plt.show()
情况有点不太妙，有的类别只有不到200张图片，有的类别却有超过2000张图片，imbalanced dataset会对分类任务带来影响：
[180, 1980, 2010, 1260, 1770, 1650, 360, 1290, 1260, 1320, 1800, 1170, 1890, 1920, 690, 540, 360, 990, 1080, 180, 300, 270, 330, 450, 240, 1350, 540, 210, 480, 240, 390, 690, 210, 599, 360, 1080, 330, 180, 1860, 270, 300, 210, 210]
3 使用LeNet进行分类
我们先不对图片进行处理，先使用最简单的CNN网络对交通标志进行分类，在此基准上再进行各种优化来实现最终98%的准确率。
3.1 模型定义
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense, Dropout, Flatten
from tensorflow.keras.optimizers import Adam
def leNet_model():
model = Sequential()
model.add(Conv2D(30, (5, 5), input_shape=(32, 32, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(15, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(150, activation='relu'))
model.add(Dense(43, activation='softmax'))
model.compile(Adam(lr=0.001), loss='categorical_crossentropy', metrics=['accuracy'])
return model
model = leNet_model()
print(model.summary())
模型有两层CNN加MaxPooling，Flatten后经过一层Dense和一层softmax后得出43个分类每个分类的概率；优化器使用Adam，learning rate暂设为0.001。
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_2 (Conv2D) (None, 28, 28, 30) 2280
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 14, 14, 30) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 12, 12, 15) 4065
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 6, 6, 15) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 540) 0
_________________________________________________________________
dense_2 (Dense) (None, 150) 81150
_________________________________________________________________
dense_3 (Dense) (None, 43) 6493
=================================================================
Total params: 93,988
Trainable params: 93,988
Non-trainable params: 0
_________________________________________________________________
None
3.2 模型训练
首先，把43个分类做one hot处理：
from tensorflow.keras.utils import to_categorical
y_train = to_categorical(y_train, 43)
y_test = to_categorical(y_test, 43)
y_val = to_categorical(y_val, 43)
接着开始训练，使用20个epoch、每个batch为400个样本`：
history = model.fit(X_train, y_train, epochs=20, validation_data=(X_val, y_val), batch_size=400, verbose=1, shuffle=True)
Train on 34799 samples, validate on 4410 samples
Epoch 1/20
34799/34799 [==============================] - 2s 52us/sample - loss: 5.3459 - accuracy: 0.2106 - val_loss: 2.3427 - val_accuracy: 0.4367
Epoch 2/20
34799/34799 [==============================] - 1s 40us/sample - loss: 1.1822 - accuracy: 0.6883 - val_loss: 1.2369 - val_accuracy: 0.6941
Epoch 3/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.5687 - accuracy: 0.8515 - val_loss: 0.9957 - val_accuracy: 0.7728
Epoch 4/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.3659 - accuracy: 0.9042 - val_loss: 1.0038 - val_accuracy: 0.7932
Epoch 5/20
...
Epoch 15/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0461 - accuracy: 0.9879 - val_loss: 1.0673 - val_accuracy: 0.8642
Epoch 16/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0499 - accuracy: 0.9859 - val_loss: 1.0070 - val_accuracy: 0.8667
Epoch 17/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0564 - accuracy: 0.9843 - val_loss: 1.0201 - val_accuracy: 0.8610
Epoch 18/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0399 - accuracy: 0.9887 - val_loss: 1.0586 - val_accuracy: 0.8678
Epoch 19/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0346 - accuracy: 0.9905 - val_loss: 1.0694 - val_accuracy: 0.8773
Epoch 20/20
34799/34799 [==============================] - 1s 40us/sample - loss: 0.0379 - accuracy: 0.9895 - val_loss: 1.1620 - val_accuracy: 0.8710
def evaluate():
score = model.evaluate(X_test, y_test, verbose=0)
print('Test score:', score[0])
print('Test Accuracy:', score[1])
evaluate()
Test score: 1.5260828396982364
Test Accuracy: 0.8539984
结果出来了，可以看到在训练集上的准确率达到了98.9%，但在验证集上的准确率只有87.1%，存在严重的过拟合(overfitting)，在测试集上的成绩 – 准确率只有85.3%。我们再看看loss和accuracy的变化曲线：
def plot_history(history):
fig, axs = plt.subplots(1, 2, figsize=(20, 8))
axs[0].plot(history.history['loss'])
axs[0].plot(history.history['val_loss'])
axs[0].legend(['Training', 'Validation'])
axs[0].set_title('Loss')
axs[0].set_xlabel('epoch')
axs[1].plot(history.history['accuracy'])
axs[1].plot(history.history['val_accuracy'])
axs[1].legend(['Training', 'Validation'])
axs[1].set_title('Accuracy')
axs[1].set_xlabel('epoch')
plot_history(history)
验证曲线距离训练曲线还很远。
4 优化
接下来我们开始针对性的优化。
4.1 Dropout
针对过拟合，最简单的模型优化方式就是使用Dropout了，我们添加两个Dropout层看看：
def modified_model():
model = Sequential()
model.add(Conv2D(30, (5, 5), input_shape=(32, 32, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(15, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(150, activation='relu'))
model.add(Dropout(0.7))
model.add(Dense(43, activation='softmax'))
model.compile(Adam(lr=0.001), loss='categorical_crossentropy', metrics=['accuracy'])
return model
model = modified_model()
print(model.summary())
Model: "sequential_12"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_48 (Conv2D) (None, 28, 28, 30) 2280
_________________________________________________________________
max_pooling2d_48 (MaxPooling (None, 14, 14, 30) 0
_________________________________________________________________
conv2d_49 (Conv2D) (None, 12, 12, 15) 4065
_________________________________________________________________
max_pooling2d_49 (MaxPooling (None, 6, 6, 15) 0
_________________________________________________________________
dropout_18 (Dropout) (None, 6, 6, 15) 0
_________________________________________________________________
flatten_24 (Flatten) (None, 540) 0
_________________________________________________________________
dense_48 (Dense) (None, 150) 81150
_________________________________________________________________
dropout_19 (Dropout) (None, 150) 0
_________________________________________________________________
dense_49 (Dense) (None, 43) 6493
=================================================================
Total params: 93,988
Trainable params: 93,988
Non-trainable params: 0
_________________________________________________________________
None
可以看到参数数量是没有变化的。
我们开始训练了，可能你会遇到跟我一样的意外(只是有可能)：
model = modified_model()
history = model.fit(X_train, y_train, epochs=60, validation_data=(X_val, y_val), batch_size=400, verbose=1, shuffle=True)
Train on 34799 samples, validate on 4410 samples
Epoch 1/60
34799/34799 [==============================] - 2s 55us/sample - loss: 7.6550 - accuracy: 0.0534 - val_loss: 3.7177 - val_accuracy: 0.0544
Epoch 2/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.6863 - accuracy: 0.0563 - val_loss: 3.6767 - val_accuracy: 0.0544
Epoch 3/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.6388 - accuracy: 0.0558 - val_loss: 3.6372 - val_accuracy: 0.0544
Epoch 4/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.5953 - accuracy: 0.0557 - val_loss: 3.6030 - val_accuracy: 0.054
...
Epoch 16/60
34799/34799 [==============================] - 1s 40us/sample - loss: 3.4896 - accuracy: 0.0549 - val_loss: 3.5506 - val_accuracy: 0.0544
Epoch 17/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4873 - accuracy: 0.0558 - val_loss: 3.5507 - val_accuracy: 0.0544
Epoch 18/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4866 - accuracy: 0.0565 - val_loss: 3.5510 - val_accuracy: 0.0544
Epoch 19/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4847 - accuracy: 0.0567 - val_loss: 3.5511 - val_accuracy: 0.0544
Epoch 20/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4844 - accuracy: 0.0541 - val_loss: 3.5512 - val_accuracy: 0.0544
...
Epoch 56/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4770 - accuracy: 0.0578 - val_loss: 3.5547 - val_accuracy: 0.0544
Epoch 57/60
34799/34799 [==============================] - 1s 40us/sample - loss: 3.4770 - accuracy: 0.0578 - val_loss: 3.5548 - val_accuracy: 0.0544
Epoch 58/60
34799/34799 [==============================] - 1s 40us/sample - loss: 3.4770 - accuracy: 0.0578 - val_loss: 3.5547 - val_accuracy: 0.0544
Epoch 59/60
34799/34799 [==============================] - 1s 41us/sample - loss: 3.4770 - accuracy: 0.0578 - val_loss: 3.5546 - val_accuracy: 0.0544
Epoch 60/60
34799/34799 [==============================] - 1s 40us/sample - loss: 3.4770 - accuracy: 0.0578 - val_loss: 3.5547 - val_accuracy: 0.0544
训练了60个epoch后, loss并没有什么变化！这是怎么回事呢？
4.1.1 死亡的ReLU
聪明如你可能已经知道了，原因在于我们的激活函数(activation function) – ReLU在值小于0的时候是0，梯度(gradient)也是0，这个时候就不能有效训练网络了。我们可以使用eLU或Leaky ReLU来替代它。
这里选择eLU，eLU应该也能更快的converge，注意我把第2个Dropout的比例调成0.5，把learning rate减少一半，并增加训练次数到100个epoch：
def modified_model():
model = Sequential()
model.add(Conv2D(30, (5, 5), input_shape=(32, 32, 3), activation='elu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(15, (3, 3), activation='elu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Flatten())
model.add(Dense(150, activation='elu'))
model.add(Dropout(0.5))
model.add(Dense(43, activation='softmax'))
model.compile(Adam(lr=0.0005), loss='categorical_crossentropy', metrics=['accuracy'])
return model
model = modified_model()
history = model.fit(X_train, y_train, epochs=100, validation_data=(X_val, y_val), batch_size=400, verbose=1, shuffle=True)
Train on 34799 samples, validate on 4410 samples
Epoch 1/100
34799/34799 [==============================] - 2s 56us/sample - loss: 12.5261 - accuracy: 0.0355 - val_loss: 3.5446 - val_accuracy: 0.0655
Epoch 2/100
34799/34799 [==============================] - 1s 41us/sample - loss: 4.0887 - accuracy: 0.0599 - val_loss: 3.3880 - val_accuracy: 0.1345
Epoch 3/100
34799/34799 [==============================] - 1s 41us/sample - loss: 3.7705 - accuracy: 0.1050 - val_loss: 2.9809 - val_accuracy: 0.2524
Epoch 4/100
34799/34799 [==============================] - 1s 42us/sample - loss: 3.3804 - accuracy: 0.1718 - val_loss: 2.5046 - val_accuracy: 0.3574
Epoch 5/100
34799/34799 [==============================] - 1s 42us/sample - loss: 2.9944 - accuracy: 0.2417 - val_loss: 2.1266 - val_accuracy: 0.4383
...
Epoch 96/100
34799/34799 [==============================] - 1s 42us/sample - loss: 0.1113 - accuracy: 0.9660 - val_loss: 0.1910 - val_accuracy: 0.9535
Epoch 97/100
34799/34799 [==============================] - 2s 43us/sample - loss: 0.1144 - accuracy: 0.9645 - val_loss: 0.1795 - val_accuracy: 0.9571
Epoch 98/100
34799/34799 [==============================] - 1s 41us/sample - loss: 0.1140 - accuracy: 0.9643 - val_loss: 0.1817 - val_accuracy: 0.9560
Epoch 99/100
34799/34799 [==============================] - 1s 41us/sample - loss: 0.1120 - accuracy: 0.9661 - val_loss: 0.1751 - val_accuracy: 0.9546
Epoch 100/100
34799/34799 [==============================] - 1s 41us/sample - loss: 0.1102 - accuracy: 0.9652 - val_loss: 0.1683 - val_accuracy: 0.9578
Amazing! 仅仅加了Dropout和微调了一下参数，我们训练的准确率达到了95.7%，测试集上准确率达到了 96.3%(85.3% -> 96.3%) ：
plot_history(history)
evaluate()
Test score: 0.14850520864866457
Test Accuracy: 0.9631037
我们离目标98%不远啦！
4.2 图片增强(Image Augmentation)
由于训练集里某些类别的交通标志样本过少，同时为了增加训练样本的多样性，使网络学习到更稳定、更本质的特征，我们对训练图片做一些常见的增强，使用ImageDataGenerator即可。
我们将应用5种变换：
- width_shift: 水平方向移动，幅度10%
- height_shift: 垂直方向移动，幅度10%
- zoom: 放大与缩小，幅度20%
- shear: 平行四边形变换，可沿水平或者垂直方向变换，幅度为与坐标轴夹角0.1度
- rotation: 顺时针或逆时针旋转，幅度10度
from tensorflow.keras.preprocessing.image import ImageDataGenerator
X_train = X_train/255
X_val = X_val/255
X_test = X_test/255
datagen = ImageDataGenerator(width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=0.2,
shear_range=0.1,
rotation_range=10.)
datagen.fit(X_train)
随便抽几张图片来看看：
batches = datagen.flow(X_train, y_train, batch_size=10)
X_batch, y_batch = next(batches)
fig, axs = plt.subplots(1, 10, figsize=(15, 5))
fig.tight_layout()
for i in range(10):
axs[i].imshow(X_batch[i])
axs[i].axis("off")
可以看到图片确实有一些改变，有的图片可能应用了不止1个变换：
接着开始训练：
model = modified_model()
history = model.fit_generator(datagen.flow(X_train, y_train, batch_size=50),
steps_per_epoch=len(X_train)/50,
epochs=100,
validation_data=(X_val, y_val), shuffle=True)
Train for 695.98 steps, validate on 4410 samples
Epoch 1/100
696/695 [==============================] - 19s 27ms/step - loss: 2.7106 - accuracy: 0.2586 - val_loss: 1.7745 - val_accuracy: 0.4687
Epoch 2/100
696/695 [==============================] - 18s 26ms/step - loss: 1.8986 - accuracy: 0.4307 - val_loss: 1.3028 - val_accuracy: 0.5889
Epoch 3/100
696/695 [==============================] - 19s 27ms/step - loss: 1.5969 - accuracy: 0.5062 - val_loss: 1.1169 - val_accuracy: 0.6503
Epoch 4/100
696/695 [==============================] - 19s 27ms/step - loss: 1.4203 - accuracy: 0.5563 - val_loss: 0.9490 - val_accuracy: 0.6930
Epoch 5/100
696/695 [==============================] - 19s 27ms/step - loss: 1.2934 - accuracy: 0.5892 - val_loss: 0.8621 - val_accuracy: 0.7327
...
Epoch 96/100
696/695 [==============================] - 18s 25ms/step - loss: 0.4447 - accuracy: 0.8595 - val_loss: 0.2044 - val_accuracy: 0.9406
Epoch 97/100
696/695 [==============================] - 17s 25ms/step - loss: 0.4433 - accuracy: 0.8602 - val_loss: 0.1746 - val_accuracy: 0.9476
Epoch 98/100
696/695 [==============================] - 18s 25ms/step - loss: 0.4392 - accuracy: 0.8612 - val_loss: 0.1928 - val_accuracy: 0.9433
Epoch 99/100
696/695 [==============================] - 18s 26ms/step - loss: 0.4249 - accuracy: 0.8644 - val_loss: 0.2079 - val_accuracy: 0.9370
Epoch 100/100
696/695 [==============================] - 18s 26ms/step - loss: 0.4259 - accuracy: 0.8650 - val_loss: 0.2012 - val_accuracy: 0.9454
可以看到图片增强后准确率并没有提升，在测试集上的准确率甚至下降了，为 93.6%(96.3% -> 93.6%)：
evaluate()
Test score: 0.2087915445615258
Test Accuracy: 0.93634206
4.3 模型优化
万事开头难，然后中间难，最后最难。
首先，我们看到虽然应用了数据增强，但模型在训练集上的准确率是比较低的，只能达到86.5%，这说明模型可能有一点点欠拟合(underfitting)。因为我们的模型是最简单的LeNet结构，我们需要增加一些层次来抽取更复杂的特征。
说干就干：
def modified_model():
model = Sequential()
model.add(Conv2D(60, (5, 5), input_shape=(32, 32, 3), activation='elu'))
model.add(Conv2D(60, (5, 5), activation='elu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(30, (3, 3), activation='elu'))
model.add(Conv2D(30, (3, 3), activation='elu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(Conv2D(30, (3, 3), activation='elu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(500, activation='elu'))
model.add(Dropout(0.5))
model.add(Dense(43, activation='softmax'))
model.compile(Adam(lr=0.0005), loss='categorical_crossentropy', metrics=['accuracy'])
return model
m = modified_model()
print(m.summary())
Model: "sequential_62"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_161 (Conv2D) (None, 28, 28, 60) 4560
_________________________________________________________________
conv2d_162 (Conv2D) (None, 24, 24, 60) 90060
_________________________________________________________________
max_pooling2d_135 (MaxPoolin (None, 12, 12, 60) 0
_________________________________________________________________
conv2d_163 (Conv2D) (None, 10, 10, 30) 16230
_________________________________________________________________
conv2d_164 (Conv2D) (None, 8, 8, 30) 8130
_________________________________________________________________
max_pooling2d_136 (MaxPoolin (None, 4, 4, 30) 0
_________________________________________________________________
dropout_93 (Dropout) (None, 4, 4, 30) 0
_________________________________________________________________
conv2d_165 (Conv2D) (None, 2, 2, 30) 8130
_________________________________________________________________
max_pooling2d_137 (MaxPoolin (None, 1, 1, 30) 0
_________________________________________________________________
flatten_61 (Flatten) (None, 30) 0
_________________________________________________________________
dense_122 (Dense) (None, 500) 15500
_________________________________________________________________
dropout_94 (Dropout) (None, 500) 0
_________________________________________________________________
dense_123 (Dense) (None, 43) 21543
=================================================================
Total params: 164,153
Trainable params: 164,153
Non-trainable params: 0
_________________________________________________________________
None
加进了更多的CNN层，使用更多filter，Dense层的output也增加了，现在模型参数的数量从9万多增加到16万多了。
结果如何呢？且看：
model = modified_model()
history = model.fit_generator(datagen.flow(X_train, y_train, batch_size=50),
steps_per_epoch=len(X_train)/50,
epochs=200,
validation_data=(X_val, y_val), shuffle=True)
...
Epoch 196/200
696/695 [==============================] - 23s 33ms/step - loss: 0.8824 - accuracy: 0.7515 - val_loss: 0.0213 - val_accuracy: 0.9930
Epoch 197/200
696/695 [==============================] - 23s 33ms/step - loss: 0.8927 - accuracy: 0.7484 - val_loss: 0.0238 - val_accuracy: 0.9925
Epoch 198/200
696/695 [==============================] - 23s 33ms/step - loss: 0.8791 - accuracy: 0.7518 - val_loss: 0.0273 - val_accuracy: 0.9932
Epoch 199/200
696/695 [==============================] - 23s 33ms/step - loss: 0.8877 - accuracy: 0.7492 - val_loss: 0.0267 - val_accuracy: 0.9923
Epoch 200/200
696/695 [==============================] - 23s 33ms/step - loss: 0.9068 - accuracy: 0.7463 - val_loss: 0.0304 - val_accuracy: 0.9907
经过200轮训练，准确率终于有所上升了，为 97.6%！(93.6% -> 97.6%)
evaluate()
Test score: 0.09128276077196848
Test Accuracy: 0.9766429
plot_history(history)
5 再优化
虽然97.6%的准确率四舍五入可以认为已经达到98%了，但总感觉差点意思。毕竟还没看到0.98这几个数字呢。
5.1 分类错误的图片
我们先看看那分类错误的2.4%的图片是什么样的：
y_predicted = model.predict_classes(X_test)
y_test_classes = test_data['labels']
uncorrect = test_data['features'][y_test_classes != y_predicted]
print(uncorrect.shape)
(295, 32, 32, 3)
有295个是识别错误的。随便看10个吧：
import random
fig, axs = plt.subplots(1, 10, figsize=(15, 5))
fig.tight_layout()
for i in range(10):
axs[i].imshow(uncorrect[random.randint(0, len(uncorrect) - 1)])
axs[i].axis("off")
这些图片比我想象中的更难识别，有更多的变形，有的图片有噪音(第2与第4张)，有的图片有部分被遮挡(第8张)，最后一张似乎快变成黑白的了。
5.2 图片再增强
根据以上信息，我们可以继续尝试对训练图片增加噪音、遮挡和改变色调等增强处理。
这里用到功能强大的图片处理库imgaug，使用其中的2个增强器：
- Cutout: 随机切掉图片的一些部分，我们切掉2小方块
- AddToHueAndSaturation: 改变色调和饱和度，我们设定范围为(-50, 50)
以下代码先调整了一下ImageDataGenerator的参数，幅度更大些；然后创建两个图片增强器；为了和ImageDataGenerator搭配使用，我定义了一个自己的generator，在ImageDataGenerator产生的batch里挑50%的图片做cutout，再挑50%做改变色调饱和度：
import imgaug as ia
from imgaug import augmenters as iaa
import numpy as np
datagen = ImageDataGenerator(width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=0.2,
shear_range=10,
rotation_range=30.)
datagen.fit(X_train)
cutout = iaa.Cutout(nb_iterations=2)
change_hue_sat = iaa.AddToHueAndSaturation((-50, 50))
def my_generator():
batches = datagen.flow(X_train, y_train, batch_size=50)
while True:
X_batch, y_batch = next(batches)
for i in range(len(X_batch)):
if np.random.rand() < 0.5:
X_batch[i] = X_batch[i] * 255
X_batch[i] = cutout(image=X_batch[i].astype(np.uint8)) / 255
if np.random.rand() < 0.5:
X_batch[i] = X_batch[i] * 255
X_batch[i] = change_hue_sat(image=X_batch[i].astype(np.uint8)) / 255
yield (X_batch, y_batch)
5.3 最后的训练
model = modified_model()
history = model.fit_generator(my_generator(),
steps_per_epoch=len(X_train)/50,
epochs=200,
validation_data=(X_val, y_val), shuffle=True)
由于处理的步骤多了，训练的时间也长了很多。
Train for 695.98 steps, validate on 4410 samples
Epoch 1/200
696/695 [==============================] - 43s 62ms/step - loss: 2.5890 - accuracy: 0.2612 - val_loss: 1.3441 - val_accuracy: 0.5361
Epoch 2/200
696/695 [==============================] - 42s 61ms/step - loss: 1.6096 - accuracy: 0.4966 - val_loss: 0.7641 - val_accuracy: 0.7676
...
Epoch 196/200
696/695 [==============================] - 40s 57ms/step - loss: 0.1651 - accuracy: 0.9504 - val_loss: 0.0467 - val_accuracy: 0.9859
Epoch 197/200
696/695 [==============================] - 40s 58ms/step - loss: 0.1596 - accuracy: 0.9518 - val_loss: 0.0530 - val_accuracy: 0.9848
Epoch 198/200
696/695 [==============================] - 40s 57ms/step - loss: 0.1722 - accuracy: 0.9485 - val_loss: 0.0338 - val_accuracy: 0.9893
Epoch 199/200
696/695 [==============================] - 40s 57ms/step - loss: 0.1689 - accuracy: 0.9491 - val_loss: 0.0197 - val_accuracy: 0.9943
Epoch 200/200
696/695 [==============================] - 40s 57ms/step - loss: 0.1706 - accuracy: 0.9493 - val_loss: 0.0216 - val_accuracy: 0.9939
最终在训练集和验证集上都达到了比较高的准确率。
plot_history(history)
那么在测试集上是否达到目标了呢？
evaluate()
Test score: 0.0694863362028404
Test Accuracy: 0.9830562
是的，98.3%! (97.6% -> 98.3%)，超过98% 。功夫不负有心人呐！
6 思考
在图片处理环节，我并没有把图片转成灰度，虽然认为交通标志的形状才是最重要的，但我觉得其颜色也是标准的，应该也是识别的线索。
欢迎读者朋友对灰度图片进行试验，看能否达到98%的准确率，期待你的好消息！
对图片的处理还有很多的方法，本文只是抛砖引玉，欢迎尝试更多更好的方法来得到更高的准确率，也欢迎提出问题一起讨论学习，谢谢！