Tensorflow2.0学习(19):Embedding——电影评论的情感分析

概述

  • 将电影中评论中的某个句子的所有词变成对应的维度的embedding向量,然后将多个句子组合成batch送入神经网络,标签是句子所代表的正面或者负面的标签。

实战

  • 前面同上一篇
  • 对句子进行paddding
# 设置句子的长度,长度高于500的会被截断,长度低于500的会被补全
max_length = 500

train_data = keras.preprocessing.sequence.pad_sequences(
    train_data,
    # 填充值
    value = word_index[''],
    # padding可取“post”和“pre”,post将padding放在句子后面,pre将放前面
    padding = 'post',
    maxlen = max_length)

test_data = keras.preprocessing.sequence.pad_sequences(
    test_data,
    value = word_index[''],
    padding = 'post',
    maxlen = max_length)

print(train_data[0])
[   1   14   22   16   43  530  973 1622 1385   65  458 4468   66 3941
    4  173   36  256    5   25  100   43  838  112   50  670    2    9
   35  480  284    5  150    4  172  112  167    2  336  385   39    4
  172 4536 1111   17  546   38   13  447    4  192   50   16    6  147
 2025   19   14   22    4 1920 4613  469    4   22   71   87   12   16
   43  530   38   76   15   13 1247    4   22   17  515   17   12   16
  626   18    2    5   62  386   12    8  316    8  106    5    4 2223
 5244   16  480   66 3785   33    4  130   12   16   38  619    5   25
  124   51   36  135   48   25 1415   33    6   22   12  215   28   77
   52    5   14  407   16   82    2    8    4  107  117 5952   15  256
    4    2    7 3766    5  723   36   71   43  530  476   26  400  317
   46    7    4    2 1029   13  104   88    4  381   15  297   98   32
 2071   56   26  141    6  194 7486   18    4  226   22   21  134  476
   26  480    5  144   30 5535   18   51   36   28  224   92   25  104
    4  226   65   16   38 1334   88   12   16  283    5   16 4472  113
  103   32   15   16 5345   19  178   32    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0    0    0    0    0
    0    0    0    0    0    0    0    0    0    0]
  • 构建模型:每个句子都有500个词,每个词对应一个16维的向量,也就是每个句子对应的是50016的矩阵。组合成batch后就是128500*16的三维矩阵
# 将每个词变成长度为16的embedding向量
embedding_dim = 16
batch_size = 128

model = keras.models.Sequential([
    # keras.layers.Embedding要做的几件事:
    # 1.定义矩阵:[vocab_size, embedding_dim]
    # 2.对于每一个句子/样本,如:[1,2,3,4...],都会取矩阵中查找对应的向量,最后变成成
    # max_length * embeddding_dim的矩阵
    # 3.最后输出的大小为一个三维矩阵:batch_size * max_length * embedding_dim
    keras.layers.Embedding(vocab_size, embedding_dim,
                          input_length = max_length),
    # 对输出做合并
    # 将三维矩阵变为:batch_size * embedding_dim
    keras.layers.GlobalAveragePooling1D(),
    keras.layers.Dense(64, activation = 'relu'),
    keras.layers.Dense(1, activation = 'sigmoid'),
])

model.summary()
model.compile(optimizer = 'adam', loss = 'binary_crossentropy',
             metrics = ['accuracy'])
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, 500, 16)           160000    
_________________________________________________________________
global_average_pooling1d (Gl (None, 16)                0         
_________________________________________________________________
dense (Dense)                (None, 64)                1088      
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 65        
=================================================================
Total params: 161,153
Trainable params: 161,153
Non-trainable params: 0
_________________________________________________________________
  • 训练模型
history = model.fit(train_data,
                    train_labels,
                    epochs=30,
                    batch_size=batch_size,
                    validation_split=0.2)
Train on 20000 samples, validate on 5000 samples
Epoch 1/30
20000/20000 [==============================] - 2s 110us/sample - loss: 0.6814 - accuracy: 0.6095 - val_loss: 0.6336 - val_accuracy: 0.7728
Epoch 2/30
20000/20000 [==============================] - 2s 85us/sample - loss: 0.5103 - accuracy: 0.8185 - val_loss: 0.4055 - val_accuracy: 0.8582
Epoch 3/30
20000/20000 [==============================] - 2s 85us/sample - loss: 0.3320 - accuracy: 0.8803 - val_loss: 0.3267 - val_accuracy: 0.8752
Epoch 4/30
20000/20000 [==============================] - 2s 85us/sample - loss: 0.2643 - accuracy: 0.9018 - val_loss: 0.2995 - val_accuracy: 0.8844
Epoch 5/30
20000/20000 [==============================] - 2s 116us/sample - loss: 0.2262 - accuracy: 0.9184 - val_loss: 0.2861 - val_accuracy: 0.8874
Epoch 6/30
20000/20000 [==============================] - 2s 118us/sample - loss: 0.2012 - accuracy: 0.9275 - val_loss: 0.2843 - val_accuracy: 0.8880
Epoch 7/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.1781 - accuracy: 0.9369 - val_loss: 0.2846 - val_accuracy: 0.8930
Epoch 8/30
20000/20000 [==============================] - 2s 116us/sample - loss: 0.1617 - accuracy: 0.9424 - val_loss: 0.2810 - val_accuracy: 0.8974
Epoch 9/30
20000/20000 [==============================] - 2s 118us/sample - loss: 0.1450 - accuracy: 0.9506 - val_loss: 0.2866 - val_accuracy: 0.8974
Epoch 10/30
20000/20000 [==============================] - 2s 116us/sample - loss: 0.1310 - accuracy: 0.9568 - val_loss: 0.2964 - val_accuracy: 0.8958
Epoch 11/30
20000/20000 [==============================] - 2s 114us/sample - loss: 0.1216 - accuracy: 0.9596 - val_loss: 0.3066 - val_accuracy: 0.8944
Epoch 12/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.1098 - accuracy: 0.9641 - val_loss: 0.3213 - val_accuracy: 0.8894
Epoch 13/30
20000/20000 [==============================] - 2s 111us/sample - loss: 0.0995 - accuracy: 0.9675 - val_loss: 0.3489 - val_accuracy: 0.8824
Epoch 14/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0940 - accuracy: 0.9700 - val_loss: 0.3454 - val_accuracy: 0.8882
Epoch 15/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0824 - accuracy: 0.9748 - val_loss: 0.3564 - val_accuracy: 0.8888
Epoch 16/30
20000/20000 [==============================] - 2s 112us/sample - loss: 0.0772 - accuracy: 0.9767 - val_loss: 0.3715 - val_accuracy: 0.8882
Epoch 17/30
20000/20000 [==============================] - 2s 114us/sample - loss: 0.0680 - accuracy: 0.9805 - val_loss: 0.3920 - val_accuracy: 0.8868
Epoch 18/30
20000/20000 [==============================] - 2s 113us/sample - loss: 0.0612 - accuracy: 0.9832 - val_loss: 0.4075 - val_accuracy: 0.8860
Epoch 19/30
20000/20000 [==============================] - 2s 116us/sample - loss: 0.0546 - accuracy: 0.9856 - val_loss: 0.4345 - val_accuracy: 0.8816
Epoch 20/30
20000/20000 [==============================] - 2s 113us/sample - loss: 0.0526 - accuracy: 0.9862 - val_loss: 0.4513 - val_accuracy: 0.8822
Epoch 21/30
20000/20000 [==============================] - 2s 113us/sample - loss: 0.0448 - accuracy: 0.9886 - val_loss: 0.4642 - val_accuracy: 0.8844
Epoch 22/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0401 - accuracy: 0.9903 - val_loss: 0.4846 - val_accuracy: 0.8824
Epoch 23/30
20000/20000 [==============================] - 2s 113us/sample - loss: 0.0349 - accuracy: 0.9922 - val_loss: 0.5072 - val_accuracy: 0.8806
Epoch 24/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0307 - accuracy: 0.9936 - val_loss: 0.5297 - val_accuracy: 0.8798
Epoch 25/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0275 - accuracy: 0.9949 - val_loss: 0.5534 - val_accuracy: 0.8790
Epoch 26/30
20000/20000 [==============================] - 2s 115us/sample - loss: 0.0239 - accuracy: 0.9954 - val_loss: 0.5783 - val_accuracy: 0.8802
Epoch 27/30
20000/20000 [==============================] - 2s 122us/sample - loss: 0.0209 - accuracy: 0.9962 - val_loss: 0.6157 - val_accuracy: 0.8778
Epoch 28/30
20000/20000 [==============================] - 2s 125us/sample - loss: 0.0195 - accuracy: 0.9966 - val_loss: 0.6278 - val_accuracy: 0.8780
Epoch 29/30
20000/20000 [==============================] - 2s 114us/sample - loss: 0.0170 - accuracy: 0.9973 - val_loss: 0.6506 - val_accuracy: 0.8762
Epoch 30/30
20000/20000 [==============================] - 2s 116us/sample - loss: 0.0150 - accuracy: 0.9980 - val_loss: 0.6712 - val_accuracy: 0.8756
def plot_learning_curves(history, label, epochs, min_value, max_value):
    data = {}
    data[label] = history.history[label]
    data['val_' + label] = history.history['val_' + label]
    pd.DataFrame(data).plot(figsize=(8, 5))
    plt.grid(True)
    plt.axis([0, epochs, min_value, max_value])
    plt.show()


plot_learning_curves(history, 'accuracy', 30, 0, 1)
plot_learning_curves(history, 'loss', 30, 0, 1)

Tensorflow2.0学习(19):Embedding——电影评论的情感分析_第1张图片

Tensorflow2.0学习(19):Embedding——电影评论的情感分析_第2张图片
可以看出验证集存在过拟合。

  • 测试
model.evaluate(
    test_data, test_labels,
    batch_size = batch_size)
25000/25000 [==============================] - 0s 10us/sample - loss: 0.7317 - accuracy: 0.8566

[0.7316589960098266, 0.85664]

你可能感兴趣的:(Tensorflow2.0学习(19):Embedding——电影评论的情感分析)