HCIA-AI_深度学习_TensorFlow2模块tf.keras基本用法

2 TensorFlow 2 模块tf.keras基本用法

2.1 实验介绍

• tf.keras - 用于构建和训练深度学习模型的高阶API

2.2 实验目的

• 掌握tf.keras中常用的深度学习建模接口

2.3 实验步骤

2.3.1 模型构建

2.3.1.1 模型堆叠(tf.keras.Sequential)

import tensorflow as tf
print(tf.__version__)
print(tf.keras.__version__)

2.2.0
2.3.0-tf

import tensorflow.keras.layers as layers

model = tf.keras.Sequential()
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(32, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))

model.build(input_shape=(None,32))
model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                multiple                  1056      
_________________________________________________________________
dense_1 (Dense)              multiple                  1056      
_________________________________________________________________
dense_2 (Dense)              multiple                  330       
=================================================================
Total params: 2,442
Trainable params: 2,442
Non-trainable params: 0
_________________________________________________________________

2.3.1.2 函数式模型(tf.keras.Model, tf.keras.Input)

函数式模型主要利用tf.keras.Input和tf.keras.Model构建，比tf.keras.Sequential模型要复杂，但是效果很好，可以同时/分阶段输入变量，分阶段输出数据；你的模型需要多于一个的输出，那么需要选择函数式模型。

模型堆叠（Sequential） vs 函数式模型（Model）:
tf.keras.Sequential模型是层的简单堆叠，无法表示任意模型。使用Keras的函数式模型一个构建复杂的模型拓扑，例如：

• 多输入模型；
• 多输出模型；
• 具有共享层的模型；
• 具有非序列数据流的模型（例如残差连接）。

x = tf.keras.Input(shape = (32,))
h1 = layers.Dense(32, activation='relu')(x)
h2 = layers.Dense(32, activation='relu')(h1)
y = layers.Dense(10, activation='softmax')(h2)
model_2 = tf.keras.Model(x, y)

# 打印模型信息
model_2.summary()

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 32)]              0         
_________________________________________________________________
dense_3 (Dense)              (None, 32)                1056      
_________________________________________________________________
dense_4 (Dense)              (None, 32)                1056      
_________________________________________________________________
dense_5 (Dense)              (None, 10)                330       
=================================================================
Total params: 2,442
Trainable params: 2,442
Non-trainable params: 0
_________________________________________________________________

2.3.1.3 网络层(tf.keras.layers)

2.3.2.3.1 tf.keras.layers.Dense 全连接层

tf.keras.layers.Dense可配置的参数，主要有：

• units - 神经元个数
• activation - 激活函数
• use_bias - 是否使用偏置项
• kernel_initializer - 核初始化
• kernel_regularizer - 核正则化
• bias_initializer - 偏置初始化
• bias_regularizer - 偏置正则化
• activity_regularizer - 输出正则项
• kernel_constraint - 核约束项
• bias_constraint - 偏置约束项

# 设置激活函数
layers.Dense(32, activation='sigmoid')
layers.Dense(32, activation=tf.sigmoid)

# 设置核初始化
layers.Dense(32, kernel_initializer=tf.keras.initializers.he_normal)
# 设置核正则化
layers.Dense(32, kernel_regularizer=tf.keras.regularizers.l2(0.01))

2.3.2.3.2 tf.keras.layers.Conv2D 卷积层

tf.keras.layers.Conv2D可配置的参数，主要有：

• filters - 卷积核的数目（即输出的维度）
• kernel_size - 卷积核的宽度和长度
• strides - 步长
• padding - 补0策略 “valid” “same”
• activation - 激活函数
• data_format - 数据格式，“channels_first” 或 “channels_last”

layers.Conv2D(64, [1, 1], 2, padding='same', activation='relu')

2.3.2.3.3 tf.keras.layers.MaxPooling2D/AveragePooling2D 池化层

tf.keras.layers.MaxPooling2D/AveragePooling2D可配置的参数，主要有：

• pool_size - 池化核的大小
• strides - 步长
• 其他参数还包括：padding, data_format

layers.MaxPooling2D(pool_size=(2, 2), strides=(2, 1))

2.3.2.3.4 tf.keras.layers.LSTM/LSTMCell 长短时记忆网络及单元

tf.keras.layers.LSTM/LSTMCell 可配置的参数，主要有：

• units - 输出维度
• input_shape(timestep, input_dim)
• activation - 激活函数
• recurrent_activation - 循环步激活函数
• return_sequences - True返回全部序列；False返回输出序列中的最后一个Cell的输出
• return_state - 是否返回最后一个状态
• dropout
• recurrent_dropout

import numpy as np
inputs = tf.keras.Input(shape=(3,1))
lstm = layers.LSTM(1, return_sequences=True)(inputs)
model_lstm_1 = tf.keras.Model(inputs=inputs, outputs=lstm)

inuputs = tf.keras.Input(shape=(3,1))
lstm = layers.LSTM(1, return_sequences=False)(inputs)
model_lstm_2 = tf.keras.Model(inputs=inputs, outputs=lstm)

data = [[[0.1],
        [0.2],
        [0.3]]]
print(data)
print(model_lstm_1.predict(data))
print(model_lstm_2.predict(data))

[[[0.1], [0.2], [0.3]]]
[[[0.00220224]
  [0.00551959]
  [0.00939013]]]
[[-0.103958]]

LSTMCell是LSTM层的实现单元

• LSTM是一个网络层
• LSTMCell是一个单步的计算单元

tf.keras.layers.LSTM(16, return_sequences=True)

x = tf.keras.Input((None, 3))
y = layers.RNN(layers.LSTMCell(16))(x)
model_lstm_3 = tf.keras.Model(x, y)

2.3.2 模型训练与评估

2.3.2.1 模型编译，确定训练流程（确定优化器、损失函数、评估指标）

构建好模型后，通过调用compile方法配置该模型的训练流程：

• compile(optimizer=‘rmsprop’, loss=None, metrics=None, loss_weights=None)

model = tf.keras.Sequential()
model.add(layers.Dense(10, activation='softmax'))
# 确定优化器、损失函数、评估指标
model.compile(optimizer=tf.keras.optimizers.Adam(0.001),
             loss=tf.keras.losses.categorical_crossentropy,
             metrics=tf.keras.metrics.categorical_accuracy)

2.3.2.2 模型训练

fit(x=None, y=None, batch_size=None, epochs=1, callbacks=None):

• x - 输入训练数据
• y - 目标（标签）数据
• batch_size - 每次梯度更新的样本数。默认为32
• epochs - 迭代轮次
• callbacks - 回调函数
• steps_per_epoch

import numpy as np
train_x = np.random.random((1000, 36))
train_y = np.random.random((1000, 10))

val_x = np.random.random((200, 36))
val_y = np.random.random((200, 10))

model.fit(train_x, train_y, epochs=10, batch_size=100, validation_data=(val_x, val_y))

Epoch 1/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6369 - categorical_accuracy: 0.1000 - val_loss: 12.4994 - val_categorical_accuracy: 0.0850
Epoch 2/10
10/10 [==============================] - 0s 6ms/step - loss: 12.6358 - categorical_accuracy: 0.1030 - val_loss: 12.4990 - val_categorical_accuracy: 0.0850
Epoch 3/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6343 - categorical_accuracy: 0.1030 - val_loss: 12.4975 - val_categorical_accuracy: 0.0850
Epoch 4/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6331 - categorical_accuracy: 0.1020 - val_loss: 12.4973 - val_categorical_accuracy: 0.0850
Epoch 5/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6318 - categorical_accuracy: 0.1020 - val_loss: 12.4956 - val_categorical_accuracy: 0.0850
Epoch 6/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6307 - categorical_accuracy: 0.1030 - val_loss: 12.4955 - val_categorical_accuracy: 0.0850
Epoch 7/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6299 - categorical_accuracy: 0.1030 - val_loss: 12.4954 - val_categorical_accuracy: 0.0850
Epoch 8/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6284 - categorical_accuracy: 0.1020 - val_loss: 12.4932 - val_categorical_accuracy: 0.0850
Epoch 9/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6271 - categorical_accuracy: 0.1020 - val_loss: 12.4933 - val_categorical_accuracy: 0.0850
Epoch 10/10
10/10 [==============================] - 0s 5ms/step - loss: 12.6268 - categorical_accuracy: 0.1010 - val_loss: 12.4933 - val_categorical_accuracy: 0.0850

对于大型数据集可以使用tf.data构建训练数据

dataset = tf.data.Dataset.from_tensor_slices((train_x, train_y))
dataset = dataset.batch(32)
dataset = dataset.repeat()
val_dataset = tf.data.Dataset.from_tensor_slices((val_x, val_y))
val_dataset = val_dataset.batch(32)
val_dataset = val_dataset.repeat()

model.fit(dataset, epochs=10, steps_per_epoch=30,
         validation_data=val_dataset, validation_steps=3)

Epoch 1/10
WARNING:tensorflow:Layer dense_14 is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2.  The layer has dtype float32 because it's dtype defaults to floatx.

If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.

To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.

30/30 [==============================] - 0s 6ms/step - loss: 12.6309 - categorical_accuracy: 0.1010 - val_loss: 12.3281 - val_categorical_accuracy: 0.0625
Epoch 2/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6322 - categorical_accuracy: 0.1026 - val_loss: 12.3300 - val_categorical_accuracy: 0.0625
Epoch 3/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6279 - categorical_accuracy: 0.0972 - val_loss: 12.3311 - val_categorical_accuracy: 0.0625
Epoch 4/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6389 - categorical_accuracy: 0.1058 - val_loss: 12.3321 - val_categorical_accuracy: 0.0625
Epoch 5/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6097 - categorical_accuracy: 0.0994 - val_loss: 12.3311 - val_categorical_accuracy: 0.0625
Epoch 6/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6285 - categorical_accuracy: 0.0983 - val_loss: 12.3299 - val_categorical_accuracy: 0.0625
Epoch 7/10
30/30 [==============================] - 0s 2ms/step - loss: 12.5989 - categorical_accuracy: 0.1036 - val_loss: 12.3289 - val_categorical_accuracy: 0.0625
Epoch 8/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6354 - categorical_accuracy: 0.1026 - val_loss: 12.3277 - val_categorical_accuracy: 0.0625
Epoch 9/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6225 - categorical_accuracy: 0.0994 - val_loss: 12.3269 - val_categorical_accuracy: 0.0625
Epoch 10/10
30/30 [==============================] - 0s 2ms/step - loss: 12.6334 - categorical_accuracy: 0.1004 - val_loss: 12.3261 - val_categorical_accuracy: 0.0625

2.3.2.3 回调函数(tf.keras.callbacks)

回调函数是传递给模型以自定义核扩展其在训练期间的行为的对象。我们可以编写自己的自定义回调，或使用tf.keras.callbacks中的内置函数，常用内置回调函数如下：

• tf.keras.callbacks.ModelCheckpoint - 定期保存模型
• tf.keras.callbacks.LearningRateScheduler - 动态更改学习率
• tf.keras.callbacks.EarlyStopping - 提前终止训练过程
• tf.keras.callbacks.TensorBoard - 使用TensorBoard

import os
logdir = os.path.join('logs')
if not os.path.exists(logdir):
    os.mkdir(logdir)
callbacks = [
    tf.keras.callbacks.ModelCheckpoint(
        filepath='testmodel_{epoch}.h5',
        save_best_only=True,
        monitor='val_loss'),
    tf.keras.callbacks.TensorBoard(log_dir=logdir)
]
model.fit(train_x, train_y, batch_size=16, epochs=10, callbacks=callbacks, validation_data=(val_x,val_y))

Epoch 1/10
63/63 [==============================] - 0s 6ms/step - loss: 12.6083 - categorical_accuracy: 0.0990 - val_loss: 12.4752 - val_categorical_accuracy: 0.0850
Epoch 2/10
63/63 [==============================] - 0s 3ms/step - loss: 12.6024 - categorical_accuracy: 0.1010 - val_loss: 12.4638 - val_categorical_accuracy: 0.0900
Epoch 3/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5830 - categorical_accuracy: 0.0970 - val_loss: 12.4533 - val_categorical_accuracy: 0.0850
Epoch 4/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5758 - categorical_accuracy: 0.0990 - val_loss: 12.4456 - val_categorical_accuracy: 0.0850
Epoch 5/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5659 - categorical_accuracy: 0.0950 - val_loss: 12.4299 - val_categorical_accuracy: 0.0850
Epoch 6/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5516 - categorical_accuracy: 0.0980 - val_loss: 12.4344 - val_categorical_accuracy: 0.0850
Epoch 7/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5474 - categorical_accuracy: 0.0950 - val_loss: 12.4227 - val_categorical_accuracy: 0.0900
Epoch 8/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5575 - categorical_accuracy: 0.0980 - val_loss: 12.4226 - val_categorical_accuracy: 0.0900
Epoch 9/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5475 - categorical_accuracy: 0.0950 - val_loss: 12.4121 - val_categorical_accuracy: 0.0900
Epoch 10/10
63/63 [==============================] - 0s 3ms/step - loss: 12.5291 - categorical_accuracy: 0.0970 - val_loss: 12.4053 - val_categorical_accuracy: 0.0850

2.3.2.4 模型评估与预测

# 模型评估
test_x = np.random.random((1000,36))
test_y = np.random.random((1000,10))
model.evaluate(test_x, test_y, batch_size=32)

32/32 [==============================] - 0s 1ms/step - loss: 12.5696 - categorical_accuracy: 0.1020





[12.569572448730469, 0.10199999809265137]

# 模型预测
pred_x = np.random.random((10, 36))
result = model.predict(test_x)
print(result)

[[0.08188314 0.11118438 0.09862817 ... 0.11808255 0.20492536 0.06566519]
 [0.0890706  0.06404206 0.12123366 ... 0.06613814 0.1619877  0.10920925]
 [0.02985656 0.09919958 0.09332599 ... 0.13977195 0.16148092 0.08238488]
 ...
 [0.07203209 0.05714086 0.05819634 ... 0.1054978  0.2895207  0.06621139]
 [0.05601608 0.0504833  0.16844288 ... 0.07254815 0.20934965 0.04347179]
 [0.03471439 0.07980331 0.12522417 ... 0.16443023 0.19968598 0.08617512]]

2.3.3 模型保存与恢复

2.3.3.1 保存和恢复整个模型

import numpy as np
logdir = './model'
if not os.path.exists(logdir):
    os.mkdir(logdir)

# 保存模型
model.save(logdir + '/the_save_model.h5')

# 加载模型
new_model = tf.keras.models.load_model(logdir+'/the_save_model.h5')
new_pred = new_model.predict(test_x)
np.testing.assert_allclose(result, new_pred, atol=1e-6)

2.3.3.2 保存和加载网络权重

model.save_weights('./model/model_weights')
model.save_weights('./model/model_weights.h5')

model.load_weights('./model/model_weights')
model.load_weights('./model/model_weights.h5')