前面一篇Keras 基础学习的文章说的是多分类问题，稍微提及了如果一个样本属于多个类，比如在CelebA数据集中一张人脸照片是椭圆形的，戴眼镜的... 这个时候样本的label标注是multi-hot-encoding的，也就是会出现多个标签1。如[0,1,1,0]， 1代表属性出现，0代表属性不出现。当然其实这种问题也可以分解为多个二分类/多分类问题，比如人脸形状训练一个模型，有无戴眼镜训练一个模型，头发颜色训练一个模型，然后使用几个模型一起预测。缺点的话就是成本太高，一个模型可以进行多标签分类就容易多了。这里面最典型的问题应该是就CelebA数据集衍生出的一系列模型，电影类型分类等。
这篇文章我们会简单介绍一个多标签分类问题，便于大家理解。
我们使用Adrian Rosebrock 提供的衣服数据集, 这篇文章也是在他的基础上做了点修改。
先看下数据:
keras_multi_label_dataset.jpg
整个数据集有两种属性，一种是颜色(blue, red, black)，另一种是衣服的类型(dress, jeans, shirt) 。说明我们对每个衣服的label应该是长度为6的vector，其中两个值为1,其它为0。如假设one-hot-vector编码顺序是(blue, red, black, dress, jeans, shirt)则black jeans的 label就是[0,0,1,0,1,0]。
网络结构
Adrian Rosebrock 使用的网络结构是smallvggnet，结构如下:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 96, 96, 32) 896
_________________________________________________________________
activation_1 (Activation) (None, 96, 96, 32) 0
_________________________________________________________________
batch_normalization_1 (Batch (None, 96, 96, 32) 128
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 32) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 32, 32, 64) 18496
_________________________________________________________________
activation_2 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
conv2d_3 (Conv2D) (None, 32, 32, 64) 36928
_________________________________________________________________
activation_3 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
batch_normalization_3 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 64) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 16, 16, 128) 73856
_________________________________________________________________
activation_4 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
conv2d_5 (Conv2D) (None, 16, 16, 128) 147584
_________________________________________________________________
activation_5 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
batch_normalization_5 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 128) 0
_________________________________________________________________
dropout_3 (Dropout) (None, 8, 8, 128) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 8192) 0
_________________________________________________________________
dense_1 (Dense) (None, 1024) 8389632
_________________________________________________________________
activation_6 (Activation) (None, 1024) 0
_________________________________________________________________
batch_normalization_6 (Batch (None, 1024) 4096
_________________________________________________________________
dropout_4 (Dropout) (None, 1024) 0
_________________________________________________________________
dense_2 (Dense) (None, 6) 6150
_________________________________________________________________
activation_7 (Activation) (None, 6) 0
=================================================================
Total params: 8,679,302
Trainable params: 8,676,422
Non-trainable params: 2,880
_________________________________________________________________
实际上Adrian Rosebrock使用的是传统的CNN结构，最后几层是全连接层，参数很多。整个模型训练下来有100M。但是实际上现在轻量网络都是将全连接层改为pooling，减少参数。下面给出一个我改进的网络结构，增加两层卷积层和GlobalAveraePooling2D层，训练完只有14M，accuray和原来的网络差不多，甚至更好：
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_1 (Conv2D) (None, 96, 96, 32) 896
_________________________________________________________________
activation_1 (Activation) (None, 96, 96, 32) 0
_________________________________________________________________
batch_normalization_1 (Batch (None, 96, 96, 32) 128
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 32, 32, 32) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 32, 32, 32) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 32, 32, 64) 18496
_________________________________________________________________
activation_2 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
batch_normalization_2 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
conv2d_3 (Conv2D) (None, 32, 32, 64) 36928
_________________________________________________________________
activation_3 (Activation) (None, 32, 32, 64) 0
_________________________________________________________________
batch_normalization_3 (Batch (None, 32, 32, 64) 256
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 16, 16, 64) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 16, 16, 64) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 16, 16, 128) 73856
_________________________________________________________________
activation_4 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
conv2d_5 (Conv2D) (None, 16, 16, 128) 147584
_________________________________________________________________
activation_5 (Activation) (None, 16, 16, 128) 0
_________________________________________________________________
batch_normalization_5 (Batch (None, 16, 16, 128) 512
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 8, 8, 128) 0
_________________________________________________________________
dropout_3 (Dropout) (None, 8, 8, 128) 0
_________________________________________________________________
conv2d_6 (Conv2D) (None, 8, 8, 256) 295168
_________________________________________________________________
activation_6 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
batch_normalization_6 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
conv2d_7 (Conv2D) (None, 8, 8, 256) 590080
_________________________________________________________________
activation_7 (Activation) (None, 8, 8, 256) 0
_________________________________________________________________
batch_normalization_7 (Batch (None, 8, 8, 256) 1024
_________________________________________________________________
max_pooling2d_4 (MaxPooling2 (None, 4, 4, 256) 0
_________________________________________________________________
dropout_4 (Dropout) (None, 4, 4, 256) 0
_________________________________________________________________
global_average_pooling2d_1 ( (None, 256) 0
_________________________________________________________________
activation_8 (Activation) (None, 256) 0
_________________________________________________________________
batch_normalization_8 (Batch (None, 256) 1024
_________________________________________________________________
dropout_5 (Dropout) (None, 256) 0
_________________________________________________________________
dense_1 (Dense) (None, 6) 1542
_________________________________________________________________
activation_9 (Activation) (None, 6) 0
=================================================================
Total params: 1,169,286
Trainable params: 1,166,918
Non-trainable params: 2,368
_________________________________________________________________
损失函数
确定了网络结构之后就需要确定针对我们的问题选用什么样的loss function了。这也是多类别分类(multi-class classification)和 多标签(multi-label classification)的差别之处。
先看看多分类问题中用的softmax函数假设是网络最后的输出:
import math
def softmax(z):
z_exp = [math.exp(i) for i in z]
sum_z_exp = sum(z_exp)
return [i / sum_z_exp for i in z_exp]
经过softmax 层 之后得到的是一个多项式概率分布，所有的节点概率和为1, 这种情况下每个类别的输出是不独立的。
假设网络最后的输出为:z = [-1.0, 5.0, -0.5, 5.0, -0.5]
In [4]: z = [-1.0, 5.0, -0.5, 4.7, -0.5]
In [5]: softmax(z)
Out[5]:
[0.0014152405960574873,
0.5709488061694115,
0.002333337273878307,
0.4229692786867745,
0.002333337273878307]
则我们的预测值为概率只为0.57的第二类。
而sigmoid层为:
def sigmoid(z):
return [1 / (1 + math.exp(-n)) for n in z]
In [7]: z = [-1.0, 5.0, -0.5, 4.7, -0.5]
In [8]: sigmoid(z)
Out[8]:
[0.2689414213699951,
0.9933071490757153,
0.3775406687981454,
0.990986701347152,
0.3775406687981454]
在sigmoid函数下，假设我们的判断阈值为0.5，则我们的预测值应该是第2,4类。在这中情况下每个输出节点对网络预测都是独立的。这也是我们在多标签分类要解决的问题，我们希望网络的输出是独立的伯努利分布，每个节点对loss函数的贡献也是独立的，每个label的出现也是对立的，比如red, blue可能同时出现，可能同时不出现，也可能出现一个，但是最后的预测值最主要还是取决于你的训练样本，如果 你的训练样本没有红色和蓝色同时出现的，那么你的预测值也很大程度上不会有。
多标签编码(multi-hots-encoding)
可以使用scikit-learn.preprocessing的MultiLabelBinarizer来对多标签类进行类别编码, 具体用法如下:
In [9]: from sklearn.preprocessing import MultiLabelBinarizer
In [10]: labels = [['blue', 'shirt'],['red', 'jeans'],['blue', 'jeans']]
In [11]: mlb = MultiLabelBinarizer()
In [12]: labels = mlb.fit_transform(labels)
In [13]: labels
Out[13]:
array([[1, 0, 0, 1],
[0, 1, 1, 0],
[1, 1, 0, 0]])
In [14]: print(mlb.classes_)
['blue' 'jeans' 'red' 'shirt']
可以看到MLB是按照每个属性的字母序对当前的label进行编码的，出现编码为1， 不出现为0。
编码完成之后使用ImageDataGenerator 和图片同步输入到网络里进行训练就可以。
训练结果图:
Screen Shot 2018-12-13 at 11.48.50 AM.png
所有的代码在这: https://github.com/ItchyHiker/multi-label-classification-Keras
Reference
- https://github.com/keras-team/keras/issues/741
- https://www.depends-on-the-definition.com/guide-to-multi-label-classification-with-neural-networks/
- https://www.pyimagesearch.com/2018/05/07/multi-label-classification-with-keras/
- https://stackoverflow.com/questions/44164749/how-does-keras-handle-multilabel-classification
前两篇Keras实战文章:
Keras 实战 II: VGG16图片分类:https://www.jianshu.com/p/5a7df18498d4
Keras 实战 I: 卷积神经网络图片分类:
https://www.jianshu.com/p/dcb10f9a5b05