tensorflow2自动调参（在fashion_mnist数据集上利用keras_tuner自动进行网格搜索优化网络结构）

Copyright 2020 The TensorFlow Authors.

#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

Introduction to the Keras Tuner

View on TensorFlow.org

Run in Google Colab

View source on GitHub

Download notebook

Overview

The Keras Tuner is a library that helps you pick the optimal set of hyperparameters for your TensorFlow program. The process of selecting the right set of hyperparameters for your machine learning (ML) application is called hyperparameter tuning or hypertuning.

Hyperparameters are the variables that govern the training process and the topology of an ML model. These variables remain constant over the training process and directly impact the performance of your ML program. Hyperparameters are of two types:

1. Model hyperparameters which influence model selection such as the number and width of hidden layers
2. Algorithm hyperparameters which influence the speed and quality of the learning algorithm such as the learning rate for Stochastic Gradient Descent (SGD) and the number of nearest neighbors for a k Nearest Neighbors (KNN) classifier

In this tutorial, you will use the Keras Tuner to perform hypertuning for an image classification application.

Setup

import tensorflow as tf
from tensorflow import keras

import IPython

Install and import the Keras Tuner.

!pip install -q -U keras-tuner
import kerastuner as kt

Download and prepare the dataset

In this tutorial, you will use the Keras Tuner to find the best hyperparameters for a machine learning model that classifies images of clothing from the Fashion MNIST dataset.

Load the data.

(img_train, label_train), (img_test, label_test) = keras.datasets.fashion_mnist.load_data()

# Normalize pixel values between 0 and 1
img_train = img_train.astype('float32') / 255.0
img_test = img_test.astype('float32') / 255.0

Define the model

When you build a model for hypertuning, you also define the hyperparameter search space in addition to the model architecture. The model you set up for hypertuning is called a hypermodel.

You can define a hypermodel through two approaches:

• By using a model builder function
• By subclassing the HyperModel class of the Keras Tuner API

You can also use two pre-defined HyperModel classes - HyperXception and HyperResNet for computer vision applications.

In this tutorial, you use a model builder function to define the image classification model. The model builder function returns a compiled model and uses hyperparameters you define inline to hypertune the model.

def model_builder(hp):
  model = keras.Sequential()
  model.add(keras.layers.Flatten(input_shape=(28, 28)))
  
  # Tune the number of units in the first Dense layer
  # Choose an optimal value between 32-512
  hp_units = hp.Int('units', min_value = 32, max_value = 512, step = 32)
  model.add(keras.layers.Dense(units = hp_units, activation = 'relu'))
  model.add(keras.layers.Dense(10))

  # Tune the learning rate for the optimizer 
  # Choose an optimal value from 0.01, 0.001, or 0.0001
  hp_learning_rate = hp.Choice('learning_rate', values = [1e-2, 1e-3, 1e-4]) 
  
  model.compile(optimizer = keras.optimizers.Adam(learning_rate = hp_learning_rate),
                loss = keras.losses.SparseCategoricalCrossentropy(from_logits = True), 
                metrics = ['accuracy'])
  
  return model

Instantiate the tuner and perform hypertuning

Instantiate the tuner to perform the hypertuning. The Keras Tuner has four tuners available - RandomSearch, Hyperband, BayesianOptimization, and Sklearn. In this tutorial, you use the Hyperband tuner.

To instantiate the Hyperband tuner, you must specify the hypermodel, the objective to optimize and the maximum number of epochs to train (max_epochs).

tuner = kt.Hyperband(model_builder,
                     objective = 'val_accuracy', 
                     max_epochs = 10,
                     factor = 3,
                     directory = 'my_dir',
                     project_name = 'intro_to_kt')                       

The Hyperband tuning algorithm uses adaptive resource allocation and early-stopping to quickly converge on a high-performing model. This is done using a sports championship style bracket. The algorithm trains a large number of models for a few epochs and carries forward only the top-performing half of models to the next round. Hyperband determines the number of models to train in a bracket by computing 1 + log_factor(max_epochs) and rounding it up to the nearest integer.

Before running the hyperparameter search, define a callback to clear the training outputs at the end of every training step.

class ClearTrainingOutput(tf.keras.callbacks.Callback):
  def on_train_end(*args, **kwargs):
    IPython.display.clear_output(wait = True)

Run the hyperparameter search. The arguments for the search method are the same as those used for tf.keras.model.fit in addition to the callback above.

tuner.search(img_train, label_train, epochs = 10, validation_data = (img_test, label_test), callbacks = [ClearTrainingOutput()])

# Get the optimal hyperparameters
best_hps = tuner.get_best_hyperparameters(num_trials = 1)[0]

print(f"""
The hyperparameter search is complete. The optimal number of units in the first densely-connected
layer is {best_hps.get('units')} and the optimal learning rate for the optimizer
is {best_hps.get('learning_rate')}.
""")

Trial complete

Trial summary

|-Trial ID: e078cd9e0ceb2d7b5a921e124ded1445

|-Score: 0.8884999752044678

|-Best step: 0

Hyperparameters:

|-learning_rate: 0.001

|-tuner/bracket: 0

|-tuner/epochs: 10

|-tuner/initial_epoch: 0

|-tuner/round: 0

|-units: 320

INFO:tensorflow:Oracle triggered exit

The hyperparameter search is complete. The optimal number of units in the first densely-connected
layer is 320 and the optimal learning rate for the optimizer
is 0.001.

To finish this tutorial, retrain the model with the optimal hyperparameters from the search.

# Build the model with the optimal hyperparameters and train it on the data
model = tuner.hypermodel.build(best_hps)
model.fit(img_train, label_train, epochs = 10, validation_data = (img_test, label_test))

Epoch 1/10
1875/1875 [==============================] - ETA: 0s - loss: 2.7466 - accuracy: 0.09 - ETA: 1s - loss: 1.0518 - accuracy: 0.62 - ETA: 1s - loss: 0.8328 - accuracy: 0.70 - ETA: 1s - loss: 0.7672 - accuracy: 0.72 - ETA: 1s - loss: 0.7209 - accuracy: 0.74 - ETA: 1s - loss: 0.6823 - accuracy: 0.75 - ETA: 1s - loss: 0.6542 - accuracy: 0.76 - ETA: 1s - loss: 0.6331 - accuracy: 0.77 - ETA: 1s - loss: 0.6194 - accuracy: 0.78 - ETA: 1s - loss: 0.6041 - accuracy: 0.78 - ETA: 1s - loss: 0.5902 - accuracy: 0.79 - ETA: 1s - loss: 0.5818 - accuracy: 0.79 - ETA: 1s - loss: 0.5732 - accuracy: 0.79 - ETA: 1s - loss: 0.5666 - accuracy: 0.79 - ETA: 0s - loss: 0.5571 - accuracy: 0.80 - ETA: 0s - loss: 0.5496 - accuracy: 0.80 - ETA: 0s - loss: 0.5431 - accuracy: 0.80 - ETA: 0s - loss: 0.5378 - accuracy: 0.80 - ETA: 0s - loss: 0.5331 - accuracy: 0.81 - ETA: 0s - loss: 0.5292 - accuracy: 0.81 - ETA: 0s - loss: 0.5247 - accuracy: 0.81 - ETA: 0s - loss: 0.5190 - accuracy: 0.81 - ETA: 0s - loss: 0.5152 - accuracy: 0.81 - ETA: 0s - loss: 0.5099 - accuracy: 0.81 - ETA: 0s - loss: 0.5058 - accuracy: 0.81 - ETA: 0s - loss: 0.5029 - accuracy: 0.82 - ETA: 0s - loss: 0.4996 - accuracy: 0.82 - ETA: 0s - loss: 0.4970 - accuracy: 0.82 - ETA: 0s - loss: 0.4944 - accuracy: 0.82 - ETA: 0s - loss: 0.4911 - accuracy: 0.82 - ETA: 0s - loss: 0.4885 - accuracy: 0.82 - ETA: 0s - loss: 0.4850 - accuracy: 0.82 - ETA: 0s - loss: 0.4823 - accuracy: 0.82 - 2s 999us/step - loss: 0.4803 - accuracy: 0.8286 - val_loss: 0.4143 - val_accuracy: 0.8565
Epoch 2/10
1875/1875 [==============================] - ETA: 0s - loss: 0.5416 - accuracy: 0.84 - ETA: 1s - loss: 0.3472 - accuracy: 0.87 - ETA: 1s - loss: 0.3490 - accuracy: 0.87 - ETA: 1s - loss: 0.3632 - accuracy: 0.86 - ETA: 1s - loss: 0.3757 - accuracy: 0.86 - ETA: 1s - loss: 0.3834 - accuracy: 0.86 - ETA: 1s - loss: 0.3834 - accuracy: 0.86 - ETA: 1s - loss: 0.3815 - accuracy: 0.86 - ETA: 1s - loss: 0.3809 - accuracy: 0.86 - ETA: 1s - loss: 0.3762 - accuracy: 0.86 - ETA: 1s - loss: 0.3769 - accuracy: 0.86 - ETA: 1s - loss: 0.3762 - accuracy: 0.86 - ETA: 1s - loss: 0.3768 - accuracy: 0.86 - ETA: 1s - loss: 0.3755 - accuracy: 0.86 - ETA: 1s - loss: 0.3776 - accuracy: 0.86 - ETA: 1s - loss: 0.3764 - accuracy: 0.86 - ETA: 0s - loss: 0.3775 - accuracy: 0.86 - ETA: 0s - loss: 0.3778 - accuracy: 0.86 - ETA: 0s - loss: 0.3771 - accuracy: 0.86 - ETA: 0s - loss: 0.3772 - accuracy: 0.86 - ETA: 0s - loss: 0.3759 - accuracy: 0.86 - ETA: 0s - loss: 0.3726 - accuracy: 0.86 - ETA: 0s - loss: 0.3700 - accuracy: 0.86 - ETA: 0s - loss: 0.3701 - accuracy: 0.86 - ETA: 0s - loss: 0.3691 - accuracy: 0.86 - ETA: 0s - loss: 0.3673 - accuracy: 0.86 - ETA: 0s - loss: 0.3670 - accuracy: 0.86 - ETA: 0s - loss: 0.3662 - accuracy: 0.86 - ETA: 0s - loss: 0.3666 - accuracy: 0.86 - ETA: 0s - loss: 0.3662 - accuracy: 0.86 - ETA: 0s - loss: 0.3648 - accuracy: 0.86 - ETA: 0s - loss: 0.3645 - accuracy: 0.86 - ETA: 0s - loss: 0.3633 - accuracy: 0.86 - ETA: 0s - loss: 0.3629 - accuracy: 0.86 - ETA: 0s - loss: 0.3616 - accuracy: 0.86 - ETA: 0s - loss: 0.3617 - accuracy: 0.86 - 2s 1ms/step - loss: 0.3612 - accuracy: 0.8675 - val_loss: 0.3633 - val_accuracy: 0.8670
Epoch 3/10
1875/1875 [==============================] - ETA: 0s - loss: 0.2202 - accuracy: 0.90 - ETA: 1s - loss: 0.3308 - accuracy: 0.88 - ETA: 1s - loss: 0.3422 - accuracy: 0.87 - ETA: 1s - loss: 0.3364 - accuracy: 0.87 - ETA: 1s - loss: 0.3338 - accuracy: 0.87 - ETA: 1s - loss: 0.3378 - accuracy: 0.87 - ETA: 1s - loss: 0.3388 - accuracy: 0.87 - ETA: 1s - loss: 0.3359 - accuracy: 0.87 - ETA: 1s - loss: 0.3375 - accuracy: 0.87 - ETA: 1s - loss: 0.3365 - accuracy: 0.87 - ETA: 1s - loss: 0.3352 - accuracy: 0.87 - ETA: 1s - loss: 0.3340 - accuracy: 0.87 - ETA: 1s - loss: 0.3374 - accuracy: 0.87 - ETA: 1s - loss: 0.3370 - accuracy: 0.87 - ETA: 1s - loss: 0.3375 - accuracy: 0.87 - ETA: 1s - loss: 0.3373 - accuracy: 0.87 - ETA: 1s - loss: 0.3370 - accuracy: 0.87 - ETA: 0s - loss: 0.3347 - accuracy: 0.87 - ETA: 0s - loss: 0.3362 - accuracy: 0.87 - ETA: 0s - loss: 0.3372 - accuracy: 0.87 - ETA: 0s - loss: 0.3364 - accuracy: 0.87 - ETA: 0s - loss: 0.3349 - accuracy: 0.87 - ETA: 0s - loss: 0.3325 - accuracy: 0.87 - ETA: 0s - loss: 0.3336 - accuracy: 0.87 - ETA: 0s - loss: 0.3330 - accuracy: 0.87 - ETA: 0s - loss: 0.3319 - accuracy: 0.87 - ETA: 0s - loss: 0.3315 - accuracy: 0.87 - ETA: 0s - loss: 0.3309 - accuracy: 0.87 - ETA: 0s - loss: 0.3302 - accuracy: 0.87 - ETA: 0s - loss: 0.3300 - accuracy: 0.87 - ETA: 0s - loss: 0.3290 - accuracy: 0.87 - ETA: 0s - loss: 0.3287 - accuracy: 0.87 - ETA: 0s - loss: 0.3274 - accuracy: 0.87 - ETA: 0s - loss: 0.3272 - accuracy: 0.87 - ETA: 0s - loss: 0.3273 - accuracy: 0.87 - ETA: 0s - loss: 0.3274 - accuracy: 0.87 - ETA: 0s - loss: 0.3266 - accuracy: 0.87 - 2s 1ms/step - loss: 0.3266 - accuracy: 0.8794 - val_loss: 0.3622 - val_accuracy: 0.8659
Epoch 4/10
1875/1875 [==============================] - ETA: 0s - loss: 0.2643 - accuracy: 0.90 - ETA: 1s - loss: 0.2906 - accuracy: 0.89 - ETA: 1s - loss: 0.3107 - accuracy: 0.88 - ETA: 1s - loss: 0.3090 - accuracy: 0.88 - ETA: 1s - loss: 0.3113 - accuracy: 0.88 - ETA: 1s - loss: 0.3091 - accuracy: 0.88 - ETA: 1s - loss: 0.3062 - accuracy: 0.88 - ETA: 1s - loss: 0.3064 - accuracy: 0.88 - ETA: 1s - loss: 0.3063 - accuracy: 0.88 - ETA: 1s - loss: 0.3022 - accuracy: 0.88 - ETA: 1s - loss: 0.3043 - accuracy: 0.88 - ETA: 1s - loss: 0.3026 - accuracy: 0.88 - ETA: 1s - loss: 0.3024 - accuracy: 0.88 - ETA: 1s - loss: 0.3021 - accuracy: 0.88 - ETA: 1s - loss: 0.3017 - accuracy: 0.88 - ETA: 1s - loss: 0.3012 - accuracy: 0.88 - ETA: 1s - loss: 0.2994 - accuracy: 0.88 - ETA: 0s - loss: 0.2986 - accuracy: 0.88 - ETA: 0s - loss: 0.2989 - accuracy: 0.88 - ETA: 0s - loss: 0.2995 - accuracy: 0.88 - ETA: 0s - loss: 0.3002 - accuracy: 0.88 - ETA: 0s - loss: 0.3019 - accuracy: 0.88 - ETA: 0s - loss: 0.3022 - accuracy: 0.88 - ETA: 0s - loss: 0.3027 - accuracy: 0.88 - ETA: 0s - loss: 0.3021 - accuracy: 0.88 - ETA: 0s - loss: 0.3028 - accuracy: 0.88 - ETA: 0s - loss: 0.3041 - accuracy: 0.88 - ETA: 0s - loss: 0.3034 - accuracy: 0.88 - ETA: 0s - loss: 0.3033 - accuracy: 0.88 - ETA: 0s - loss: 0.3027 - accuracy: 0.88 - ETA: 0s - loss: 0.3030 - accuracy: 0.88 - ETA: 0s - loss: 0.3039 - accuracy: 0.88 - ETA: 0s - loss: 0.3035 - accuracy: 0.88 - ETA: 0s - loss: 0.3042 - accuracy: 0.88 - ETA: 0s - loss: 0.3036 - accuracy: 0.88 - ETA: 0s - loss: 0.3030 - accuracy: 0.88 - ETA: 0s - loss: 0.3029 - accuracy: 0.88 - 2s 1ms/step - loss: 0.3029 - accuracy: 0.8875 - val_loss: 0.3478 - val_accuracy: 0.8750
Epoch 5/10
1875/1875 [==============================] - ETA: 0s - loss: 0.3725 - accuracy: 0.87 - ETA: 1s - loss: 0.2715 - accuracy: 0.90 - ETA: 1s - loss: 0.2688 - accuracy: 0.90 - ETA: 1s - loss: 0.2682 - accuracy: 0.90 - ETA: 1s - loss: 0.2670 - accuracy: 0.89 - ETA: 1s - loss: 0.2714 - accuracy: 0.89 - ETA: 1s - loss: 0.2769 - accuracy: 0.89 - ETA: 1s - loss: 0.2764 - accuracy: 0.89 - ETA: 1s - loss: 0.2761 - accuracy: 0.89 - ETA: 1s - loss: 0.2772 - accuracy: 0.89 - ETA: 1s - loss: 0.2769 - accuracy: 0.89 - ETA: 1s - loss: 0.2762 - accuracy: 0.89 - ETA: 1s - loss: 0.2797 - accuracy: 0.89 - ETA: 1s - loss: 0.2810 - accuracy: 0.89 - ETA: 1s - loss: 0.2842 - accuracy: 0.89 - ETA: 1s - loss: 0.2842 - accuracy: 0.89 - ETA: 1s - loss: 0.2840 - accuracy: 0.89 - ETA: 0s - loss: 0.2835 - accuracy: 0.89 - ETA: 0s - loss: 0.2834 - accuracy: 0.89 - ETA: 0s - loss: 0.2835 - accuracy: 0.89 - ETA: 0s - loss: 0.2841 - accuracy: 0.89 - ETA: 0s - loss: 0.2838 - accuracy: 0.89 - ETA: 0s - loss: 0.2832 - accuracy: 0.89 - ETA: 0s - loss: 0.2828 - accuracy: 0.89 - ETA: 0s - loss: 0.2829 - accuracy: 0.89 - ETA: 0s - loss: 0.2829 - accuracy: 0.89 - ETA: 0s - loss: 0.2834 - accuracy: 0.89 - ETA: 0s - loss: 0.2836 - accuracy: 0.89 - ETA: 0s - loss: 0.2845 - accuracy: 0.89 - ETA: 0s - loss: 0.2838 - accuracy: 0.89 - ETA: 0s - loss: 0.2824 - accuracy: 0.89 - ETA: 0s - loss: 0.2817 - accuracy: 0.89 - ETA: 0s - loss: 0.2820 - accuracy: 0.89 - ETA: 0s - loss: 0.2828 - accuracy: 0.89 - ETA: 0s - loss: 0.2830 - accuracy: 0.89 - ETA: 0s - loss: 0.2817 - accuracy: 0.89 - ETA: 0s - loss: 0.2821 - accuracy: 0.89 - 2s 1ms/step - loss: 0.2825 - accuracy: 0.8958 - val_loss: 0.3776 - val_accuracy: 0.8694
Epoch 6/10
1875/1875 [==============================] - ETA: 0s - loss: 0.1992 - accuracy: 0.93 - ETA: 1s - loss: 0.2680 - accuracy: 0.90 - ETA: 1s - loss: 0.2572 - accuracy: 0.90 - ETA: 1s - loss: 0.2626 - accuracy: 0.90 - ETA: 1s - loss: 0.2586 - accuracy: 0.90 - ETA: 1s - loss: 0.2544 - accuracy: 0.90 - ETA: 1s - loss: 0.2556 - accuracy: 0.90 - ETA: 1s - loss: 0.2582 - accuracy: 0.90 - ETA: 1s - loss: 0.2566 - accuracy: 0.90 - ETA: 1s - loss: 0.2594 - accuracy: 0.90 - ETA: 1s - loss: 0.2619 - accuracy: 0.90 - ETA: 1s - loss: 0.2626 - accuracy: 0.90 - ETA: 1s - loss: 0.2639 - accuracy: 0.90 - ETA: 1s - loss: 0.2660 - accuracy: 0.90 - ETA: 1s - loss: 0.2670 - accuracy: 0.90 - ETA: 1s - loss: 0.2675 - accuracy: 0.90 - ETA: 0s - loss: 0.2686 - accuracy: 0.90 - ETA: 0s - loss: 0.2679 - accuracy: 0.90 - ETA: 0s - loss: 0.2670 - accuracy: 0.90 - ETA: 0s - loss: 0.2672 - accuracy: 0.90 - ETA: 0s - loss: 0.2676 - accuracy: 0.90 - ETA: 0s - loss: 0.2691 - accuracy: 0.89 - ETA: 0s - loss: 0.2692 - accuracy: 0.89 - ETA: 0s - loss: 0.2693 - accuracy: 0.89 - ETA: 0s - loss: 0.2683 - accuracy: 0.89 - ETA: 0s - loss: 0.2695 - accuracy: 0.89 - ETA: 0s - loss: 0.2694 - accuracy: 0.89 - ETA: 0s - loss: 0.2682 - accuracy: 0.89 - ETA: 0s - loss: 0.2682 - accuracy: 0.90 - ETA: 0s - loss: 0.2684 - accuracy: 0.90 - ETA: 0s - loss: 0.2691 - accuracy: 0.90 - ETA: 0s - loss: 0.2686 - accuracy: 0.90 - ETA: 0s - loss: 0.2692 - accuracy: 0.90 - ETA: 0s - loss: 0.2692 - accuracy: 0.89 - ETA: 0s - loss: 0.2694 - accuracy: 0.89 - ETA: 0s - loss: 0.2698 - accuracy: 0.89 - 2s 1ms/step - loss: 0.2695 - accuracy: 0.8999 - val_loss: 0.3360 - val_accuracy: 0.8821
Epoch 7/10
1875/1875 [==============================] - ETA: 0s - loss: 0.2894 - accuracy: 0.90 - ETA: 2s - loss: 0.2688 - accuracy: 0.90 - ETA: 1s - loss: 0.2528 - accuracy: 0.90 - ETA: 1s - loss: 0.2558 - accuracy: 0.90 - ETA: 1s - loss: 0.2586 - accuracy: 0.90 - ETA: 1s - loss: 0.2573 - accuracy: 0.90 - ETA: 1s - loss: 0.2555 - accuracy: 0.90 - ETA: 1s - loss: 0.2577 - accuracy: 0.90 - ETA: 1s - loss: 0.2595 - accuracy: 0.90 - ETA: 1s - loss: 0.2600 - accuracy: 0.90 - ETA: 1s - loss: 0.2618 - accuracy: 0.90 - ETA: 1s - loss: 0.2605 - accuracy: 0.90 - ETA: 1s - loss: 0.2614 - accuracy: 0.90 - ETA: 1s - loss: 0.2606 - accuracy: 0.90 - ETA: 1s - loss: 0.2601 - accuracy: 0.90 - ETA: 1s - loss: 0.2589 - accuracy: 0.90 - ETA: 1s - loss: 0.2579 - accuracy: 0.90 - ETA: 0s - loss: 0.2593 - accuracy: 0.90 - ETA: 0s - loss: 0.2584 - accuracy: 0.90 - ETA: 0s - loss: 0.2567 - accuracy: 0.90 - ETA: 0s - loss: 0.2584 - accuracy: 0.90 - ETA: 0s - loss: 0.2583 - accuracy: 0.90 - ETA: 0s - loss: 0.2574 - accuracy: 0.90 - ETA: 0s - loss: 0.2574 - accuracy: 0.90 - ETA: 0s - loss: 0.2569 - accuracy: 0.90 - ETA: 0s - loss: 0.2572 - accuracy: 0.90 - ETA: 0s - loss: 0.2572 - accuracy: 0.90 - ETA: 0s - loss: 0.2579 - accuracy: 0.90 - ETA: 0s - loss: 0.2573 - accuracy: 0.90 - ETA: 0s - loss: 0.2573 - accuracy: 0.90 - ETA: 0s - loss: 0.2578 - accuracy: 0.90 - ETA: 0s - loss: 0.2574 - accuracy: 0.90 - ETA: 0s - loss: 0.2563 - accuracy: 0.90 - ETA: 0s - loss: 0.2565 - accuracy: 0.90 - ETA: 0s - loss: 0.2564 - accuracy: 0.90 - ETA: 0s - loss: 0.2556 - accuracy: 0.90 - ETA: 0s - loss: 0.2558 - accuracy: 0.90 - 2s 1ms/step - loss: 0.2559 - accuracy: 0.9048 - val_loss: 0.3272 - val_accuracy: 0.8841
Epoch 8/10
1875/1875 [==============================] - ETA: 0s - loss: 0.0789 - accuracy: 0.96 - ETA: 1s - loss: 0.2173 - accuracy: 0.92 - ETA: 1s - loss: 0.2359 - accuracy: 0.91 - ETA: 1s - loss: 0.2275 - accuracy: 0.91 - ETA: 1s - loss: 0.2347 - accuracy: 0.91 - ETA: 1s - loss: 0.2394 - accuracy: 0.90 - ETA: 1s - loss: 0.2367 - accuracy: 0.90 - ETA: 1s - loss: 0.2390 - accuracy: 0.90 - ETA: 1s - loss: 0.2360 - accuracy: 0.90 - ETA: 1s - loss: 0.2392 - accuracy: 0.90 - ETA: 1s - loss: 0.2376 - accuracy: 0.91 - ETA: 1s - loss: 0.2401 - accuracy: 0.90 - ETA: 1s - loss: 0.2389 - accuracy: 0.90 - ETA: 1s - loss: 0.2399 - accuracy: 0.90 - ETA: 1s - loss: 0.2402 - accuracy: 0.90 - ETA: 1s - loss: 0.2397 - accuracy: 0.91 - ETA: 1s - loss: 0.2399 - accuracy: 0.91 - ETA: 0s - loss: 0.2420 - accuracy: 0.90 - ETA: 0s - loss: 0.2440 - accuracy: 0.90 - ETA: 0s - loss: 0.2434 - accuracy: 0.90 - ETA: 0s - loss: 0.2427 - accuracy: 0.90 - ETA: 0s - loss: 0.2423 - accuracy: 0.90 - ETA: 0s - loss: 0.2426 - accuracy: 0.90 - ETA: 0s - loss: 0.2427 - accuracy: 0.90 - ETA: 0s - loss: 0.2434 - accuracy: 0.90 - ETA: 0s - loss: 0.2431 - accuracy: 0.90 - ETA: 0s - loss: 0.2435 - accuracy: 0.90 - ETA: 0s - loss: 0.2434 - accuracy: 0.90 - ETA: 0s - loss: 0.2419 - accuracy: 0.90 - ETA: 0s - loss: 0.2415 - accuracy: 0.90 - ETA: 0s - loss: 0.2413 - accuracy: 0.90 - ETA: 0s - loss: 0.2418 - accuracy: 0.90 - ETA: 0s - loss: 0.2421 - accuracy: 0.90 - ETA: 0s - loss: 0.2424 - accuracy: 0.90 - ETA: 0s - loss: 0.2425 - accuracy: 0.90 - ETA: 0s - loss: 0.2427 - accuracy: 0.90 - ETA: 0s - loss: 0.2434 - accuracy: 0.90 - 2s 1ms/step - loss: 0.2428 - accuracy: 0.9091 - val_loss: 0.3287 - val_accuracy: 0.8820
Epoch 9/10
1875/1875 [==============================] - ETA: 0s - loss: 0.2357 - accuracy: 0.93 - ETA: 1s - loss: 0.2086 - accuracy: 0.92 - ETA: 1s - loss: 0.2370 - accuracy: 0.91 - ETA: 1s - loss: 0.2388 - accuracy: 0.90 - ETA: 1s - loss: 0.2385 - accuracy: 0.90 - ETA: 1s - loss: 0.2397 - accuracy: 0.90 - ETA: 1s - loss: 0.2383 - accuracy: 0.91 - ETA: 1s - loss: 0.2415 - accuracy: 0.90 - ETA: 1s - loss: 0.2400 - accuracy: 0.90 - ETA: 1s - loss: 0.2384 - accuracy: 0.91 - ETA: 1s - loss: 0.2365 - accuracy: 0.91 - ETA: 1s - loss: 0.2348 - accuracy: 0.91 - ETA: 1s - loss: 0.2357 - accuracy: 0.91 - ETA: 1s - loss: 0.2354 - accuracy: 0.91 - ETA: 1s - loss: 0.2352 - accuracy: 0.91 - ETA: 0s - loss: 0.2340 - accuracy: 0.91 - ETA: 0s - loss: 0.2337 - accuracy: 0.91 - ETA: 0s - loss: 0.2330 - accuracy: 0.91 - ETA: 0s - loss: 0.2327 - accuracy: 0.91 - ETA: 0s - loss: 0.2316 - accuracy: 0.91 - ETA: 0s - loss: 0.2310 - accuracy: 0.91 - ETA: 0s - loss: 0.2303 - accuracy: 0.91 - ETA: 0s - loss: 0.2301 - accuracy: 0.91 - ETA: 0s - loss: 0.2313 - accuracy: 0.91 - ETA: 0s - loss: 0.2314 - accuracy: 0.91 - ETA: 0s - loss: 0.2315 - accuracy: 0.91 - ETA: 0s - loss: 0.2318 - accuracy: 0.91 - ETA: 0s - loss: 0.2322 - accuracy: 0.91 - ETA: 0s - loss: 0.2317 - accuracy: 0.91 - ETA: 0s - loss: 0.2325 - accuracy: 0.91 - ETA: 0s - loss: 0.2341 - accuracy: 0.91 - ETA: 0s - loss: 0.2338 - accuracy: 0.91 - ETA: 0s - loss: 0.2333 - accuracy: 0.91 - ETA: 0s - loss: 0.2335 - accuracy: 0.91 - ETA: 0s - loss: 0.2334 - accuracy: 0.91 - ETA: 0s - loss: 0.2332 - accuracy: 0.91 - 2s 1ms/step - loss: 0.2332 - accuracy: 0.9126 - val_loss: 0.3618 - val_accuracy: 0.8736
Epoch 10/10
1875/1875 [==============================] - ETA: 0s - loss: 0.2021 - accuracy: 0.90 - ETA: 1s - loss: 0.2054 - accuracy: 0.92 - ETA: 1s - loss: 0.2159 - accuracy: 0.91 - ETA: 1s - loss: 0.2214 - accuracy: 0.91 - ETA: 1s - loss: 0.2213 - accuracy: 0.91 - ETA: 1s - loss: 0.2235 - accuracy: 0.91 - ETA: 1s - loss: 0.2249 - accuracy: 0.91 - ETA: 1s - loss: 0.2242 - accuracy: 0.91 - ETA: 1s - loss: 0.2249 - accuracy: 0.91 - ETA: 1s - loss: 0.2260 - accuracy: 0.91 - ETA: 1s - loss: 0.2245 - accuracy: 0.91 - ETA: 1s - loss: 0.2222 - accuracy: 0.91 - ETA: 1s - loss: 0.2213 - accuracy: 0.91 - ETA: 1s - loss: 0.2215 - accuracy: 0.91 - ETA: 1s - loss: 0.2210 - accuracy: 0.91 - ETA: 0s - loss: 0.2208 - accuracy: 0.91 - ETA: 0s - loss: 0.2218 - accuracy: 0.91 - ETA: 0s - loss: 0.2229 - accuracy: 0.91 - ETA: 0s - loss: 0.2229 - accuracy: 0.91 - ETA: 0s - loss: 0.2224 - accuracy: 0.91 - ETA: 0s - loss: 0.2230 - accuracy: 0.91 - ETA: 0s - loss: 0.2233 - accuracy: 0.91 - ETA: 0s - loss: 0.2224 - accuracy: 0.91 - ETA: 0s - loss: 0.2229 - accuracy: 0.91 - ETA: 0s - loss: 0.2231 - accuracy: 0.91 - ETA: 0s - loss: 0.2228 - accuracy: 0.91 - ETA: 0s - loss: 0.2232 - accuracy: 0.91 - ETA: 0s - loss: 0.2246 - accuracy: 0.91 - ETA: 0s - loss: 0.2243 - accuracy: 0.91 - ETA: 0s - loss: 0.2251 - accuracy: 0.91 - ETA: 0s - loss: 0.2249 - accuracy: 0.91 - ETA: 0s - loss: 0.2254 - accuracy: 0.91 - ETA: 0s - loss: 0.2252 - accuracy: 0.91 - ETA: 0s - loss: 0.2258 - accuracy: 0.91 - 2s 1ms/step - loss: 0.2257 - accuracy: 0.9151 - val_loss: 0.3361 - val_accuracy: 0.8851

The my_dir/intro_to_kt directory contains detailed logs and checkpoints for every trial (model configuration) run during the hyperparameter search. If you re-run the hyperparameter search, the Keras Tuner uses the existing state from these logs to resume the search. To disable this behavior, pass an additional overwrite = True argument while instantiating the tuner.

Summary

In this tutorial, you learned how to use the Keras Tuner to tune hyperparameters for a model. To learn more about the Keras Tuner, check out these additional resources:

• Keras Tuner on the TensorFlow blog
• Keras Tuner website

Also check out the HParams Dashboard in TensorBoard to interactively tune your model hyperparameters.