keras开发者文档 8： 编写自己的回调（Writing your own callbacks）

介绍

回调是一种强大的工具，可以在训练，评估或推理期间自定义Keras模型的行为。 示例包括tf.keras.callbacks.TensorBoard通过TensorBoard可视化训练进度和结果，或tf.keras.callbacks.ModelCheckpoint在训练期间定期保存模型。

在本指南中，您将了解Keras回调是什么，它可以做什么以及如何构建自己的回调。 我们提供了一些简单的回调应用程序演示，以帮助您入门。

设置

import tensorflow as tf
from tensorflow import keras

Keras 回调概览

所有回调都将keras.callbacks.Callback类作为子类，并覆盖在训练，测试和预测的各个阶段调用的一组方法。 回调对于在训练期间了解模型的内部状态和统计信息很有用。

您可以将回调列表（作为关键字参数callbacks）传递给以下模型方法：

• keras.Model.fit()
• keras.Model.evaluate()
• keras.Model.predict()

回调方法概述

全局方法

on_(train|test|predict)_begin(self, logs=None)
Called at the beginning of fit/evaluate/predict.

on_(train|test|predict)_end(self, logs=None)
Called at the end of fit/evaluate/predict.

批级的训练/测试/预测方法

on_(train|test|predict)_batch_begin(self, batch, logs=None)
Called right before processing a batch during training/testing/predicting.

on_(train|test|predict)_batch_end(self, batch, logs=None)

在培训/测试/预测批次结束时调用。 在此方法中，logs 是包含指标结果的字典。

Epoch-level methods (training only)

on_epoch_begin(self, epoch, logs=None)
Called at the beginning of an epoch during training.

on_epoch_end(self, epoch, logs=None)
Called at the end of an epoch during training.

一个基本的例子

让我们看一个具体的例子。 首先，让我们导入tensorflow并定义一个简单的顺序Keras模型：

# Define the Keras model to add callbacks to
def get_model():
    model = keras.Sequential()
    model.add(keras.layers.Dense(1, input_dim=784))
    model.compile(
        optimizer=keras.optimizers.RMSprop(learning_rate=0.1),
        loss="mean_squared_error",
        metrics=["mean_absolute_error"],
    )
    return model

然后，从Keras数据集API加载MNIST数据以进行训练和测试：

# Load example MNIST data and pre-process it
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train.reshape(-1, 784).astype("float32") / 255.0
x_test = x_test.reshape(-1, 784).astype("float32") / 255.0

# Limit the data to 1000 samples
x_train = x_train[:1000]
y_train = y_train[:1000]
x_test = x_test[:1000]
y_test = y_test[:1000]

现在，定义一个简单的自定义回调来记录：

• When fit/evaluate/predict starts & ends
• When each epoch starts & ends
• When each training batch starts & ends
• When each evaluation (test) batch starts & ends
• When each inference (prediction) batch starts & ends

class CustomCallback(keras.callbacks.Callback):
    def on_train_begin(self, logs=None):
        keys = list(logs.keys())
        print("Starting training; got log keys: {}".format(keys))

    def on_train_end(self, logs=None):
        keys = list(logs.keys())
        print("Stop training; got log keys: {}".format(keys))

    def on_epoch_begin(self, epoch, logs=None):
        keys = list(logs.keys())
        print("Start epoch {} of training; got log keys: {}".format(epoch, keys))

    def on_epoch_end(self, epoch, logs=None):
        keys = list(logs.keys())
        print("End epoch {} of training; got log keys: {}".format(epoch, keys))

    def on_test_begin(self, logs=None):
        keys = list(logs.keys())
        print("Start testing; got log keys: {}".format(keys))

    def on_test_end(self, logs=None):
        keys = list(logs.keys())
        print("Stop testing; got log keys: {}".format(keys))

    def on_predict_begin(self, logs=None):
        keys = list(logs.keys())
        print("Start predicting; got log keys: {}".format(keys))

    def on_predict_end(self, logs=None):
        keys = list(logs.keys())
        print("Stop predicting; got log keys: {}".format(keys))

    def on_train_batch_begin(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Training: start of batch {}; got log keys: {}".format(batch, keys))

    def on_train_batch_end(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Training: end of batch {}; got log keys: {}".format(batch, keys))

    def on_test_batch_begin(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Evaluating: start of batch {}; got log keys: {}".format(batch, keys))

    def on_test_batch_end(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Evaluating: end of batch {}; got log keys: {}".format(batch, keys))

    def on_predict_batch_begin(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Predicting: start of batch {}; got log keys: {}".format(batch, keys))

    def on_predict_batch_end(self, batch, logs=None):
        keys = list(logs.keys())
        print("...Predicting: end of batch {}; got log keys: {}".format(batch, keys))

让我们尝试一下：

model = get_model()
model.fit(
    x_train,
    y_train,
    batch_size=128,
    epochs=1,
    verbose=0,
    validation_split=0.5,
    callbacks=[CustomCallback()],
)

res = model.evaluate(
    x_test, y_test, batch_size=128, verbose=0, callbacks=[CustomCallback()]
)

res = model.predict(x_test, batch_size=128, callbacks=[CustomCallback()])

logs字典的用法

logs字典包含损失值以及批次或纪元末尾的所有度量。 示例包括损耗和平均绝对误差。

class LossAndErrorPrintingCallback(keras.callbacks.Callback):
    def on_train_batch_end(self, batch, logs=None):
        print("For batch {}, loss is {:7.2f}.".format(batch, logs["loss"]))

    def on_test_batch_end(self, batch, logs=None):
        print("For batch {}, loss is {:7.2f}.".format(batch, logs["loss"]))

    def on_epoch_end(self, epoch, logs=None):
        print(
            "The average loss for epoch {} is {:7.2f} "
            "and mean absolute error is {:7.2f}.".format(
                epoch, logs["loss"], logs["mean_absolute_error"]
            )
        )


model = get_model()
model.fit(
    x_train,
    y_train,
    batch_size=128,
    epochs=2,
    verbose=0,
    callbacks=[LossAndErrorPrintingCallback()],
)

res = model.evaluate(
    x_test,
    y_test,
    batch_size=128,
    verbose=0,
    callbacks=[LossAndErrorPrintingCallback()],
)

self.model属性的用法

除了在调用其中一种方法时接收日志信息外，回调还可以访问与当前一轮训练/评估/推断相关的模型：self.model。

以下是您可以在回调中使用self.model进行的一些操作：

• Set self.model.stop_training = True to immediately interrupt training.
• Mutate hyperparameters of the optimizer (available as self.model.optimizer), such as self.model.optimizer.learning_rate.
• Save the model at period intervals.
• Record the output of model.predict() on a few test samples at the end of each epoch, to use as a sanity check during training.
• Extract visualizations of intermediate features at the end of each epoch, to monitor what the model is learning over time.
• etc.
  让我们在几个示例中看到这一点。

Keras回调应用程序示例

以最小的损失Early stopping

第一个示例显示了如何通过设置属性self.model.stop_training（boolean）来创建 Callback，该 Callback将在达到最小损失时停止训练。 （可选）您可以提供一个参数patience，以指定在达到局部最小值后停止之前应等待多少个时期。
tf.keras.callbacks.EarlyStopping提供了更完整和通用的实现。

import numpy as np


class EarlyStoppingAtMinLoss(keras.callbacks.Callback):
    """Stop training when the loss is at its min, i.e. the loss stops decreasing.

  Arguments:
      patience: Number of epochs to wait after min has been hit. After this
      number of no improvement, training stops.
  """

    def __init__(self, patience=0):
        super(EarlyStoppingAtMinLoss, self).__init__()
        self.patience = patience
        # best_weights to store the weights at which the minimum loss occurs.
        self.best_weights = None

    def on_train_begin(self, logs=None):
        # The number of epoch it has waited when loss is no longer minimum.
        self.wait = 0
        # The epoch the training stops at.
        self.stopped_epoch = 0
        # Initialize the best as infinity.
        self.best = np.Inf

    def on_epoch_end(self, epoch, logs=None):
        current = logs.get("loss")
        if np.less(current, self.best):
            self.best = current
            self.wait = 0
            # Record the best weights if current results is better (less).
            self.best_weights = self.model.get_weights()
        else:
            self.wait += 1
            if self.wait >= self.patience:
                self.stopped_epoch = epoch
                self.model.stop_training = True
                print("Restoring model weights from the end of the best epoch.")
                self.model.set_weights(self.best_weights)

    def on_train_end(self, logs=None):
        if self.stopped_epoch > 0:
            print("Epoch %05d: early stopping" % (self.stopped_epoch + 1))


model = get_model()
model.fit(
    x_train,
    y_train,
    batch_size=64,
    steps_per_epoch=5,
    epochs=30,
    verbose=0,
    callbacks=[LossAndErrorPrintingCallback(), EarlyStoppingAtMinLoss()],
)

学习率安排

在此示例中，我们展示了如何在训练过程中使用自定义的回调来动态更改优化器的学习率。

有关更一般的实现，请参见callbacks.LearningRateScheduler。

class CustomLearningRateScheduler(keras.callbacks.Callback):
    """Learning rate scheduler which sets the learning rate according to schedule.

  Arguments:
      schedule: a function that takes an epoch index
          (integer, indexed from 0) and current learning rate
          as inputs and returns a new learning rate as output (float).
  """

    def __init__(self, schedule):
        super(CustomLearningRateScheduler, self).__init__()
        self.schedule = schedule

    def on_epoch_begin(self, epoch, logs=None):
        if not hasattr(self.model.optimizer, "lr"):
            raise ValueError('Optimizer must have a "lr" attribute.')
        # Get the current learning rate from model's optimizer.
        lr = float(tf.keras.backend.get_value(self.model.optimizer.learning_rate))
        # Call schedule function to get the scheduled learning rate.
        scheduled_lr = self.schedule(epoch, lr)
        # Set the value back to the optimizer before this epoch starts
        tf.keras.backend.set_value(self.model.optimizer.lr, scheduled_lr)
        print("\nEpoch %05d: Learning rate is %6.4f." % (epoch, scheduled_lr))


LR_SCHEDULE = [
    # (epoch to start, learning rate) tuples
    (3, 0.05),
    (6, 0.01),
    (9, 0.005),
    (12, 0.001),
]


def lr_schedule(epoch, lr):
    """Helper function to retrieve the scheduled learning rate based on epoch."""
    if epoch < LR_SCHEDULE[0][0] or epoch > LR_SCHEDULE[-1][0]:
        return lr
    for i in range(len(LR_SCHEDULE)):
        if epoch == LR_SCHEDULE[i][0]:
            return LR_SCHEDULE[i][1]
    return lr


model = get_model()
model.fit(
    x_train,
    y_train,
    batch_size=64,
    steps_per_epoch=5,
    epochs=15,
    verbose=0,
    callbacks=[
        LossAndErrorPrintingCallback(),
        CustomLearningRateScheduler(lr_schedule),
    ],
)

内置Keras回调

请务必阅读API文档，以检查现有的Keras回调。 应用程序包括记录到CSV，保存模型，在TensorBoard中可视化指标等等！

参考文献

Writing your own callbacks.