今天学习了一个多元分类的机器学习方法，在这里分享给大家。
说是多元分类其实就是把四个象限的坐标分成四类。
废话不多说，先贴代码。
from keras.layers import Dense, Activation
from keras.models import Sequential
from keras import optimizers
import numpy as np
# collect the data
x_train1 = 100*np.random.random((100, 2))
x_train2 = [-100, 100]*np.random.random((100, 2))
x_train3 = -100*np.random.random((100, 2))
x_train4 = [100, -100]*np.random.random((100, 2))
x_train = np.concatenate((x_train1, x_train2, x_train3, x_train4))
y_train = np.array([[1, 0, 0, 0]*100 + [0, 1, 0, 0]*100 + [0, 0, 1, 0]*100 + [0, 0, 0, 1]*100])
y_train = y_train.reshape((400, 4))
x_test1 = 100*np.random.random((100, 2))
x_test2 = [-100, 100]*np.random.random((100, 2))
x_test3 = -100*np.random.random((100, 2))
x_test4 = [100, -100]*np.random.random((100, 2))
x_test = np.concatenate((x_test1, x_test2, x_test3, x_test4))
y_test = y_train
# set the model
model = Sequential()
model.add(Dense(4, input_dim=2, activation=None, use_bias=False))
model.add(Activation('softmax'))
# compile the model and pick the loss function and optimizer
ada = optimizers.Adagrad(lr=0.1, epsilon=1e-8)
model.compile(optimizer=ada, loss='categorical_crossentropy', metrics=['accuracy'])
# training the model
ly = []
for i in range(10):
model.fit(x_train, y_train, batch_size=400, epochs=100, shuffle=False)
ly.append(model.layers[0].get_weights())
# test the model
score = model.evaluate(x_test, y_test, batch_size=400)
print 'loss:', score[0], '\t\taccuracy:', score[1]
for i in range(10):
print 'first weight:\t', ly[i][0][0], '\t\tsecond weight:\t', ly[i][0][1]
下面咱么开始分析
导入相关模块
from keras.layers import Dense, Activation
from keras.models import Sequential
from keras import optimizers
import numpy as np
因为之前介绍过这里就不多说了详情请了解线性回归|第一天的Keras、分类训练|第二天的Keras
收集训练数据
# collect the data
x_train1 = 100*np.random.random((100, 2))
x_train2 = [-100, 100]*np.random.random((100, 2))
x_train3 = -100*np.random.random((100, 2))
x_train4 = [100, -100]*np.random.random((100, 2))
x_train = np.concatenate((x_train1, x_train2, x_train3, x_train4))
y_train = np.array([[1, 0, 0, 0]*100 + [0, 1, 0, 0]*100 + [0, 0, 1, 0]*100 + [0, 0, 0, 1]*100])
y_train = y_train.reshape((400, 4))
x_test1 = 100*np.random.random((100, 2))
x_test2 = [-100, 100]*np.random.random((100, 2))
x_test3 = -100*np.random.random((100, 2))
x_test4 = [100, -100]*np.random.random((100, 2))
x_test = np.concatenate((x_test1, x_test2, x_test3, x_test4))
y_test = y_train
- 在这里用np.random.random((100,2))生成一百个大小在0～1之间的二维向量。为了增加其一般性，并扩大规模，我们乘上100后，就变成了在第一象限的100个坐标，范围为0～100。同理后面也是作相似处理。然后四个象限的坐标数据就收集好了，我们通过np.concatenate方法把这四个象限的数据合并到一起，x_train就完成了。
- y_train的处理比较简单，最后要把y_train进行重塑。变成4维的向量就是【0 ，1，0，0】1代表什么意思呢，就是第二类了。
- 好了，后面x_test和y_test就如同上面一样了。
设计训练模型
# set the model
model = Sequential()
model.add(Dense(4, input_dim=2, activation=None, use_bias=False))
model.add(Activation('softmax'))
在这里有一个新东西就是softmax下面解释一下：
softmax 是对上一层的所有输入例如y1，y2......yn.进行如下操作。
yi <- yi/(y1+y2+y3....+yn)
了解了吧。
编译模型
# compile the model and pick the loss function and optimizer
ada = optimizers.Adagrad(lr=0.1, epsilon=1e-8)
model.compile(optimizer=ada, loss='categorical_crossentropy', metrics=['accuracy'])
- 加速器选用AdaGrad，learing_rate=0.1,epsilon(小量)=1e-8(0.00000001)
- 值得一提的是
loss = 'categorical_crossentropy'在进行多分类时一定要用到这一个loss_function.并且在使用此损失函数时，我们的y_train一定要写成【0，1，0，0】类似的向量。即向量中只能有0、1.维度为要分成的类数。
训练模型
ly = []
for i in range(10):
model.fit(x_train, y_train, batch_size=400, epochs=100, shuffle=False)
ly.append(model.layers[0].get_weights())
- 这里我们分成10此训练每次训练100轮然后每次训练我们可以获得它的weight，追踪weight的变化。
- ly = [] 是用来存储每此训练后的weight数组。
- 因为只有一层，所以layers[0]就行了。
测试模型
score = model.evaluate(x_test, y_test, batch_size=400)
print 'loss:', score[0], '\t\taccuracy:', score[1]
for i in range(10):
print 'first weight:\t', ly[i][0][0], '\t\tsecond weight:\t', ly[i][0][1]
- 我们这里用evalute方法评估一下该训练好的模型。该模型返回loss，和accuracy。
- 然后我们再把weight打印出来。
得到数据
/home/kroossun/miniconda2/bin/python /home/kroossun/PycharmProjects/ML/multi_classfication.py
Using TensorFlow backend.
2017-09-02 19:01:10.912652: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912672: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912677: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912681: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-02 19:01:10.912684: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Epoch 1/100
400/400 [==============================] - 0s - loss: 14.7780 - acc: 0.0500
Epoch 2/100
400/400 [==============================] - 0s - loss: 13.3719 - acc: 0.1025
.........
Epoch 99/100
400/400 [==============================] - 0s - loss: 0.0181 - acc: 0.9975
Epoch 100/100
400/400 [==============================] - 0s - loss: 0.0181 - acc: 0.9975
400/400 [==============================] - 0s
loss: 0.0185921788216 accuracy: 0.997500002384
first weight: [ 0.15896426 -0.41137934 -0.40870011 0.18085584] second weight: [ 0.10717846 0.10917206 -0.71580416 -0.70930481]
first weight: [ 0.17234965 -0.42518327 -0.42248955 0.19431336] second weight: [ 0.11188302 0.11416188 -0.72089857 -0.71434933]
first weight: [ 0.18458323 -0.43780571 -0.43509614 0.20663606] second weight: [ 0.11647248 0.11900952 -0.72586221 -0.71925527]
first weight: [ 0.19587189 -0.44945827 -0.44673175 0.21802464] second weight: [ 0.12095292 0.12372655 -0.730703 -0.72403347]
first weight: [ 0.20636941 -0.46029857 -0.45755425 0.22862951] second weight: [ 0.12533006 0.12832268 -0.73542821 -0.72869295]
first weight: [ 0.21619371 -0.47044694 -0.46768427 0.23856631] second weight: [ 0.12960933 0.13280627 -0.74004406 -0.73324138]
first weight: [ 0.22543719 -0.47999874 -0.4772172 0.24792583] second weight: [ 0.13379583 0.13718463 -0.74455637 -0.73768568]
first weight: [ 0.23417395 -0.48902947 -0.4862287 0.25678098] second weight: [ 0.13789426 0.14146428 -0.7489711 -0.74203229]
first weight: [ 0.24246417 -0.49760085 -0.49478057 0.2651912 ] second weight: [ 0.14190902 0.14565097 -0.75329334 -0.74628681]
first weight: [ 0.25035757 -0.50576413 -0.50292408 0.27320552] second weight: [ 0.14584419 0.1497499 -0.75752771 -0.75045419]
Process finished with exit code 0
- 这里我们训练了1000轮其实100轮已经够了，我们这里训练的有点多。
- 可以看到，在训练集上accuracy为0.9975，测试集上，也是0.9975。说明我们的训练效果还是不错的。