使用TensorFlow Estimator构建卷积神经网络的参考示例

借助TensorFlow Estimator，可以轻松构建一个卷积神经网络，本文介绍一个用于MNIST手写数字分类的卷积神经网络示例，该示例来源于Google TensorFlow官方教程。

一、程序框架

创建一个名为cnn_mnist.py的文件 ，添加以下框架代码：

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

# Imports
import numpy as np
import tensorflow as tf

tf.logging.set_verbosity(tf.logging.INFO)

# Our application logic will be added here

if __name__ == "__main__":
  tf.app.run()

二、卷积神经网络简介

卷积神经网络（CNN）将一系列滤波器应用于高维原始输入数据，以提取数量较少的高级特征，然后利用这些数量较少的高级特征执行分类任务。CNN常用于图像分类，它一般包含三个组成部分：

• 卷积层（Convolutional Layer），将指定窗口尺寸的卷积滤镜应用于图像。对于每个子区域，图层执行一组数学运算并输出对应的卷积结果。卷积层通常使用非线性的ReLU激活函数输出结果。

• 池化层（Pooling Layer），对由卷积层提取的图像数据进行下采样，以减少特征映射的维数，从而减少处理时间。常用的池化算法是最大池化，在其提取特征图的子区域（例如，2×2像素块）内保持最大值，并丢弃所有其他值。

• 密集（全连接）层（Dense Layer or Fully Connected Layer），对卷积层提取的特征进行分类。

本文示例网络结构如下所示：

Created with Raphaël 2.2.0 开始 输入层(Input Layer, 批处理数x宽28x高28x图像通道数1) 卷积层1(Convolutional Layer, 32映射，卷积核：5x5) 最大池化层1(Max Pooling Layer, pool_size=2x2，strides=2) 卷积层2(Convolutional Layer, 32映射，卷积核：5x5) 最大池化层2(Max Pooling Layer, pool_size=2x2，strides=2) 扁平层(Flatten Layer, 将矩阵转换为向量) 密集层(Dense Layer, 1024个神经元) 丢弃层(Dropout Layer, 丢弃40%) 输出层(Output Layer, 10个输出，分别表示10个数字的概率) 结束

三、完整代码

""" Convolutional Neural Network Estimator for MNIST, build with tf.layers. """

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import tensorflow as tf

tf.logging.set_verbosity(tf.logging.INFO)


def cnn_model_fn(features, labels, mode):
    """ Model function for CNN. """
    # Input Layer
    # Reshape X to 4-D tensor: [batch_size, width, height, channels]
    # MNIST images are 28x28 pixels, and have one color channel
    input_layer = tf.reshape(features["x"], [-1, 28, 28, 1])

    # Convolutional Layer #1
    # Computes 32 features using a 5x5 filter with ReLU activation.
    # Padding is added to preserve width and height.
    # Input Tensor Shape: [batch_size, 28, 28, 1]
    # Output Tensor Shape: [batch_size, 28, 28, 32]
    conv1 = tf.layers.conv2d(
        inputs=input_layer,
        filters=32,
        kernel_size=[5, 5],
        padding="same",
        activation=tf.nn.relu)

    # Pooling Layer #1
    # First max pooling layer with a 2x2 filter and stride of 2
    # Input Tensor Shape: [batch_size, 28, 28, 32]
    # Output Tensor Shape: [batch_size, 14, 14, 32]
    pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)

    # Convolutional Layer #2
    # Computes 64 features using a 5x5 filter.
    # Padding is added to preserve width and height.
    # Input Tensor Shape: [batch_size, 14, 14, 32]
    # Output Tensor Shape: [batch_size, 14, 14, 64]
    conv2 = tf.layers.conv2d(
        inputs=pool1,
        filters=64,
        kernel_size=[5, 5],
        padding="same",
        activation=tf.nn.relu)

    # Pooling Layer #2
    # Second max pooling layer with a 2x2 filter and stride of 2
    # Input Tensor Shape: [batch_size, 14, 14, 64]
    # Output Tensor Shape: [batch_size, 7, 7, 64]
    pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)

    # Flatten tensor into a batch of vectors
    # Input Tensor Shape: [batch_size, 7, 7, 64]
    # Output Tensor Shape: [batch_size, 7 * 7 * 64]
    pool2_flat = tf.reshape(pool2, [-1, 7 * 7 * 64])

    # Dense Layer
    # Densely connected layer with 1024 neurons
    # Input Tensor Shape: [batch_size, 7 * 7 * 64]
    # Output Tensor Shape: [batch_size, 1024]
    dense = tf.layers.dense(
        inputs=pool2_flat, units=1024, activation=tf.nn.relu)

    # Add dropout operation
    # 0.6 probability that element will be kept
    dropout = tf.layers.dropout(
        inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN)

    # Logits Layer
    # Input Tensor Shape: [batch_size, 1024]
    # Output Tensor Shape: [batch_size, 10]
    logits = tf.layers.dense(inputs=dropout, units=10)

    predictions = {
        # Generate predictions (for PREDICT and EVAL mode)
        "classes": tf.argmax(input=logits, axis=1),
        # Add `softmax_tensor` to the graph. It is used for PREDICT and by the
        # `logging_hook`
        "probabilities": tf.nn.softmax(logits, name="softmax_tensor")
    }

    if mode == tf.estimator.ModeKeys.PREDICT:
        return tf.estimaotor.EstimatorSpec(mode=mode, predictions=predictions)

    # Calculate Loss (for both TRAIN and EVAL modes)
    loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)

    # Configure the Training Op (for TRAIN mode)
    if mode == tf.estimator.ModeKeys.TRAIN:
        optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
        train_op = optimizer.minimize(
            loss=loss,
            global_step=tf.train.get_global_step())
        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)

    # Add evaluation metrics (for EVAL mode)
    eval_metric_ops = {
        "accuracy": tf.metrics.accuracy(
            labels=labels, predictions=predictions["classes"])}
    return tf.estimator.EstimatorSpec(
        mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)


def main(unused_argv):
    # Load training and eval data
    mnist = tf.contrib.learn.datasets.load_dataset("mnist")
    train_data = mnist.train.images  # Returns np.array
    train_labels = np.asarray(mnist.train.labels, dtype=np.int32)
    eval_data = mnist.test.images  # Returns np.array
    eval_labels = np.asarray(mnist.test.labels, dtype=np.int32)

    # Create the Estimator
    mnist_classifier = tf.estimator.Estimator(
        model_fn=cnn_model_fn, model_dir="/tmp/mnist_convnet_model")

    # Set up logging for predictions
    # Log the values in the "softmax" tensor with label "probabilities"
    tensors_to_log = {"probabilities": "softmax_tensor"}
    logging_hook = tf.train.LoggingTensorHook(
        tensors=tensors_to_log, every_n_iter=50)

    # Train the model
    train_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"x": train_data},
        y=train_labels,
        batch_size=100,
        num_epochs=None,
        shuffle=True)
    mnist_classifier.train(
        input_fn=train_input_fn,
        steps=20000,
        hooks=[logging_hook])

    # Evaluate the model and print results.
    eval_input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"x": eval_data},
        y=eval_labels,
        num_epochs=1,
        shuffle=False)
    eval_results = mnist_classifier.evaluate(input_fn=eval_input_fn)
    print(eval_results)
    
    # Must return a value, otherwise the following error will occur:
    # TypeError: int() argument must be a string, a bytes-like object
    # or a number, not 'NoneType'
    # Terminated
    return 0

if __name__ == "__main__":
    tf.app.run()

四、运行结果

最终运行结果如下，正确率大约为98.87%，不算特别好：

...
INFO:tensorflow:Saving checkpoints for 120000 into /tmp/mnist_convnet_model/model.ckpt.
INFO:tensorflow:Loss for final step: 0.0599797.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2018-08-27-05:47:52
INFO:tensorflow:Graph was finalized.
2018-08-27 13:47:52.584618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2018-08-27 13:47:52.584648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-27 13:47:52.584654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0
2018-08-27 13:47:52.584657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N
2018-08-27 13:47:52.584729: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3108 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Restoring parameters from /tmp/mnist_convnet_model/model.ckpt-120000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2018-08-27-05:47:53
INFO:tensorflow:Saving dict for global step 120000: accuracy = 0.9887, global_step = 120000, loss = 0.0329527
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 120000: /tmp/mnist_convnet_model/model.ckpt-120000
{'accuracy': 0.98869997, 'global_step': 120000, 'loss': 0.03295273}