第二阶段-tensorflow程序图文详解（五） Saving and Restoring

This document explains how to save and restore variables and models.
这篇blog讨论变量，模型的存储和加载。

1，Saving and restoring variables

A TensorFlow variable provides the best way to represent shared, persistent state manipulated by your program. (See Variables for details.) This section explains how to save and restore variables. Note that Estimators automatically saves and restores variables (in the model_dir).
variables提供最好的共享，持久化状态。注意Estimators自动化保存，恢复变量。

The tf.train.Saver class provides methods for saving and restoring models. The tf.train.Saver constructor adds save and restore ops to the graph for all, or a specified list, of the variables in the graph. The Saver object provides methods to run these ops, specifying paths for the checkpoint files to write to or read from.
tf.train.Saver类提供了保存和恢复模型的方法。tf.train.Saver构造器对于graph，或者graph中的list,variables。保存对象提供这些操作，指定checkpoint的路径，去读写。

The saver will restore all variables already defined in your model. If you’re loading a model without knowing how to build its graph (for example, if you’re writing a generic program to load models), then read the Overview of saving and restoring models section later in this document.

TensorFlow saves variables in binary checkpoint files that, roughly speaking, map variable names to tensor values.
TensorFlow将变量保存在二进制检查点文件中，粗略地说，它将变量名称映射为张量值。

1.1 Saving variables

Create a Saver with tf.train.Saver() to manage all variables in the model. For example, the following snippet demonstrates how to call the tf.train.Saver.save method to save variables to a checkpoint file:
创建一个Saver去管理所有的variables。下面代码片段，演示保存variables为checkpoint文件。

#创建一些variables.
v1 = tf.get_variable("v1", shape=[3], initializer = tf.zeros_initializer)
v2 = tf.get_variable("v2", shape=[5], initializer = tf.zeros_initializer)

inc_v1 = v1.assign(v1+1)
dec_v2 = v2.assign(v2-1)


# 这个操作用来初始化所有variables
init_op = tf.global_variables_initializer()

# 这个操作用来保存所有variables
saver = tf.train.Saver()

# 之后，运行这个模型，并保存variables到硬盘上。
with tf.Session() as sess:
  sess.run(init_op)
  #让这个模型工作起来
  inc_v1.op.run()
  dec_v2.op.run()
  # 讲variables保存到硬盘
  save_path = saver.save(sess, "/tmp/model.ckpt")
  print("Model saved in file: %s" % save_path)

1.2，Restoring variables

The tf.train.Saver object not only saves variables to checkpoint files, it also restores variables. Note that when you restore variables from a file you do not have to initialize them beforehand. For example, the following snippet demonstrates how to call the tf.train.Saver.restore method to restore variables from a checkpoint file:
恢复模型示例

tf.reset_default_graph()

# 创建一些 variables.
v1 = tf.get_variable("v1", shape=[3])
v2 = tf.get_variable("v2", shape=[5])

# 添加一个saver
saver = tf.train.Saver()

# 之后，开始保存模型
with tf.Session() as sess:
  # 从硬盘中加载ckpt文件
  saver.restore(sess, "/tmp/model.ckpt")
  print("Model restored.")
  # 将variables值打印出。
  print("v1 : %s" % v1.eval())
  print("v2 : %s" % v2.eval())

1.3，Choosing which variables to save and restore

If you do not pass any arguments to tf.train.Saver(), the saver handles all variables in the graph. Each variable is saved under the name that was passed when the variable was created.
如果不传递任何参数给tf.train.Saver（），保存器将处理图中的所有变量。 每个变量都保存在创建变量时传递的名称下。

It is sometimes useful to explicitly specify names for variables in the checkpoint files. For example, you may have trained a model with a variable named “weights” whose value you want to restore into a variable named “params”.
显式指定检查点文件中变量的名称有时很有用。 例如，您可能已经训练了一个名为“weights”的变量，该变量的值要恢复到名为“params”的变量中。

It is also sometimes useful to only save or restore a subset of the variables used by a model. For example, you may have trained a neural net with five layers, and you now want to train a new model with six layers that reuses the existing weights of the five trained layers. You can use the saver to restore the weights of just the first five layers.
保存或恢复模型使用的变量的子集，有时也是有用的。 例如，您可能已经训练了一个五层的神经网络，现在您要训练一个六层的新模型，重新使用五个训练层的现有权重。 您可以使用保存程序恢复前五层的权重。

You can easily specify the names and variables to save or load by passing to the tf.train.Saver() constructor either of the following:

1. A list of variables (which will be stored under their own names).
2. A Python dictionary in which keys are the names to use and the
   values are the variables to manage.
   一个Python字典，其中键是要使用的名称和
   值是要管理的变量。
   Continuing from the save/restore examples shown earlier:

tf.reset_default_graph()
# 创建一些 variables.
v1 = tf.get_variable("v1", [3], initializer = tf.zeros_initializer)
v2 = tf.get_variable("v2", [5], initializer = tf.zeros_initializer)

# 添加一个存储，恢复的操作，将v2,映射到key为v2
saver = tf.train.Saver({"v2": v2})

# 使用saver对象
with tf.Session() as sess:
  # 初始化v1变量，当不会保存。
  v1.initializer.run()
  saver.restore(sess, "/tmp/model.ckpt")

  print("v1 : %s" % v1.eval())
  print("v2 : %s" % v2.eval())

Notes:

1. You can create as many Saver objects as you want if you need to save
   and restore different subsets of the model variables. The same
   variable can be listed in multiple saver objects; its value is only
   changed when the Saver.restore() method is run.
   如果需要保存和恢复模型变量的不同子集，可以根据需要创建任意多个Saver对象。 同一个变量可以列在多个保存对象中; 它的值只有在Saver.restore（）方法运行时才会改变。

2. If you only restore a subset of the model variables at the start of
   a session, you have to run an initialize op for the other variables.
   See tf.variables_initializer for more information.
   如果只在会话开始时恢复模型变量的子集，则必须为其他变量运行初始化操作。 有关更多信息，请参阅tf.variables_initializer。

3. To inspect the variables in a checkpoint, you can use the
   inspect_checkpoint library, particularly the
   print_tensors_in_checkpoint_file function.

   要检查检查点中的变量，可以使用inspect_checkpoint库，特别是print_tensors_in_checkpoint_file函数。

4. By default, Saver uses the value of the tf.Variable.name property
   for each variable. However, when you create a Saver object, you may
   optionally choose names for the variables in the checkpoint files.
   默认情况下，Saver使用每个变量的tf.Variable.name属性的值。 但是，当您创建一个Saver对象时，您可以选择为检查点文件中的变量选择名称。

2，Overview of saving and restoring models

---

When you want to save and load variables, the graph, and the graph’s metadata–basically, when you want to save or restore your model–we recommend using SavedModel. SavedModel is a language-neutral, recoverable, hermetic serialization format. SavedModel enables higher-level systems and tools to produce, consume, and transform TensorFlow models. TensorFlow provides several mechanisms for interacting with SavedModel, including tf.saved_model APIs, Estimator APIs and a CLI.
当你想要保存和加载变量，图形和图形的元数据 - 基本上，当你想保存或恢复你的模型 - 我们建议使用SavedModel。 SavedModel是一种语言中立，可恢复，密封的序列化格式。 SavedModel使更高级别的系统和工具能够生成，消耗和转换TensorFlow模型。 TensorFlow提供了多种与SavedModel进行交互的机制，包括tf.saved_model API，Estimator API和CLI。

3，APIs to build and load a SavedModel

This section focuses on the APIs for building and loading a SavedModel, particularly when using lower-level TensorFlow APIs.
使用底层API构建一个SavedModel
3.1，Building a SavedModel

We provide a Python implementation of the SavedModel builder. The SavedModelBuilder class provides functionality to save multiple MetaGraphDefs. A MetaGraph is a dataflow graph, plus its associated variables, assets, and signatures. A MetaGraphDef is the protocol buffer representation of a MetaGraph. A signature is the set of inputs to and outputs from a graph.
使用python接口构建SavedModel 。SavedModelBuilder 提供多个MetaGraphDefs的方法。
一个MetaGraph 就是一个数据流图，包括变量，资源，和signature 。
一个MetaGraphDef 就是一个MetaGraph 的控制协议。
一个signature 就是图的IO集合。

If assets need to be saved and written or copied to disk, they can be provided when the first MetaGraphDef is added. If multiple MetaGraphDefs are associated with an asset of the same name, only the first version is retained.
如果assets需要保存并写入或复制到磁盘，则可以在添加第一个MetaGraphDef时提供assets。 如果多个MetaGraphDefs与同名assets相关联，则只保留第一个版本。

Each MetaGraphDef added to the SavedModel must be annotated with user-specified tags. The tags provide a means to identify the specific MetaGraphDef to load and restore, along with the shared set of variables and assets. These tags typically annotate a MetaGraphDef with its functionality (for example, serving or training), and optionally with hardware-specific aspects (for example, GPU).
每个添加到SavedModel的MetaGraphDef都必须使用用户指定的标签注释。 这些标签提供了一种方法来识别要加载和恢复的特定MetaGraphDef，以及共享的一组变量和assets。 这些标签通常使用其功能（例如服务或训练）对MetaGraphDef进行注释，并可选地使用特定于硬件的方面（例如GPU）对其进行注释。

For example, the following code suggests a typical way to use SavedModelBuilder to build a SavedModel:

export_dir = ...
...
builder = tf.saved_model_builder.SavedModelBuilder(export_dir)
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph_and_variables(sess,
                                       [tag_constants.TRAINING],
                                       signature_def_map=foo_signatures,
                                       assets_collection=foo_assets)
...
# 添加第二个 MetaGraphDef for inference.
with tf.Session(graph=tf.Graph()) as sess:
  ...
  builder.add_meta_graph([tag_constants.SERVING])
...
builder.save()

3.2，Loading a SavedModel in Python

The Python version of the SavedModel loader provides load and restore capability for a SavedModel. The load operation requires the following information:
SavedModel加载器的Python版本为SavedModel提供加载和恢复功能。 加载操作需要以下信息：

• The session in which to restore the graph definition and variables.
• The tags used to identify the MetaGraphDef to load.
• The location (directory) of the SavedModel.

Upon a load, the subset of variables, assets, and signatures supplied as part of the specific MetaGraphDef will be restored into the supplied session.

export_dir = ...
...
with tf.Session(graph=tf.Graph()) as sess:
  tf.saved_model.loader.load(sess, [tag_constants.TRAINING], export_dir)
  ...

3.3，Loading a Savedmodel in C++

The C++ version of the SavedModel loader provides an API to load a SavedModel from a path, while allowing SessionOptions and RunOptions. You have to specify the tags associated with the graph to be loaded. The loaded version of SavedModel is referred to as SavedModelBundle and contains the MetaGraphDef and the session within which it is loaded.

const string export_dir = ...
SavedModelBundle bundle;
...
LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},
               &bundle);

3.4，Standard constants

SavedModel offers the flexibility to build and load TensorFlow graphs for a variety of use-cases. For the most common use-cases, SavedModel’s APIs provide a set of constants in Python and C++ that are easy to reuse and share across tools consistently.
SavedModel提供了为各种用例构建和加载TensorFlow图表的灵活性。 对于最常见的用例，SavedModel的API在Python和C ++中提供了一组易于重复使用和持续分享的常量。
Standard MetaGraphDef tags

You may use sets of tags to uniquely identify a MetaGraphDef saved in a SavedModel. A subset of commonly used tags is specified in:

Python
C++

Standard SignatureDef constants

A SignatureDef is a protocol buffer that defines the signature of a computation supported by a graph. Commonly used input keys, output keys, and method names are defined in:
SignatureDef是一个协议缓冲区，用于定义图形所支持的计算签名。 常用的输入键，输出键和方法名称定义如下：

Python
C++

4，Using SavedModel with Estimators

After training an Estimator model, you may want to create a service from that model that takes requests and returns a result. You can run such a service locally on your machine or deploy it scalably in the cloud.
训练Estimator模型，你可能先提供一个服务，通过发送请求，并返回一个结果。你可以在本地或云机上部署一个service。

To prepare a trained Estimator for serving, you must export it in the standard SavedModel format. This section explains how to:
为了将训练好的模型提供服务，你必须将模型导出成一个标准的SavedModel 模型，这部分介绍如何实现：

• Specify the output nodes and the corresponding APIs that can be
  served (Classify, Regress, or Predict).
• Export your model to the SavedModel format.
• Serve the model from a local server and request predictions.

4.1，Preparing serving inputs

During training, an input_fn() ingests data and prepares it for use by the model. At serving time, similarly, a serving_input_receiver_fn() accepts inference requests and prepares them for the model. This function has the following purposes:
在训练期间input_fn()方法用来准备填充数据给模型训练。在服务期间，serving_input_receiver_fn()方法为模型接收相关的请求，下面就是这个函数的目的：

1. To add placeholders to the graph that the serving system will feed
   with inference requests.
   将占位符添加到服务系统将要馈送的图形中
   有推理请求。

2. To add any additional ops needed to convert data from the input
   format into the feature Tensors expected by the model.
   添加所需的额外操作来转换来自输入的数据
   格式化为模型预期的特征张量。

The function returns a tf.estimator.export.ServingInputReceiver object, which packages the placeholders and the resulting feature Tensors together.
该函数返回一个tf.estimator.export.ServingInputReceiver对象，该对象将占位符和生成的特征张量打包在一起。
A typical pattern is that inference requests arrive in the form of serialized tf.Examples, so the serving_input_receiver_fn() creates a single string placeholder to receive them. The serving_input_receiver_fn() is then also responsible for parsing the tf.Examples by adding a tf.parse_example op to the graph.
一个典型的模式是相关请求以序列化的tf.Examples的形式到达，所以serving_input_receiver_fn（）创建一个单独的字符串占位符来接收它们。 然后，serving_input_receiver_fn（）还负责通过向图中添加一个tf.parse_example操作来解析tf.Examples。

When writing such a serving_input_receiver_fn(), you must pass a parsing specification to tf.parse_example to tell the parser what feature names to expect and how to map them to Tensors. A parsing specification takes the form of a dict from feature names to tf.FixedLenFeature, tf.VarLenFeature, and tf.SparseFeature. Note this parsing specification should not include any label or weight columns, since those will not be available at serving time—in contrast to a parsing specification used in the input_fn() at training time.
在编写这样的serving_input_receiver_fn（）时，必须将解析规范传递给tf.parse_example，以告诉parser 哪些特征名称以及如何将它们映射到Tensors。 解析规范采用从特征名称到tf.FixedLenFeature，tf.VarLenFeature和tf.SparseFeature的字典的形式。 请注意，此解析规范不应包含任何标签或权重列，因为这些列在服务时间将不可用 - 与训练时在input_fn（）中使用的解析规范contrast。

In combination, then:

feature_spec = {'foo': tf.FixedLenFeature(...),
                'bar': tf.VarLenFeature(...)}

def serving_input_receiver_fn():
  """一个输入接收器是一个序列化的 tf.Example."""
  serialized_tf_example = tf.placeholder(dtype=tf.string,
                                         shape=[default_batch_size],
                                         name='input_example_tensor')
  receiver_tensors = {'examples': serialized_tf_example}
  features = tf.parse_example(serialized_tf_example, feature_spec)
  return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

The tf.estimator.export.build_parsing_serving_input_receiver_fn utility function provides that input receiver for the common case.
tf.estimator.export.build_parsing_serving_input_receiver_fn实用程序函数为常见情况提供了输入接收器。

• Note: when training a model to be served using the Predict API with a
  local server, the parsing step is not needed because the model will
  receive raw feature data.

Even if you require no parsing or other input processing—that is, if the serving system will feed feature Tensors directly—you must still provide a serving_input_receiver_fn() that creates placeholders for the feature Tensors and passes them through. The tf.estimator.export.build_raw_serving_input_receiver_fn utility provides for this.
即使您不需要解析或其他输入处理，也就是说，如果服务系统将直接提供功能张量，您仍然必须提供一个serving_input_receiver_fn（），为功能张量创建占位符并将其传递。 tf.estimator.export.build_raw_serving_input_receiver_fn实用程序提供了这个。
If these utilities do not meet your needs, you are free to write your own serving_input_receiver_fn(). One case where this may be needed is if your training input_fn() incorporates some preprocessing logic that must be recapitulated at serving time. To reduce the risk of training-serving skew, we recommend encapsulating such processing in a function which is then called from both input_fn() and serving_input_receiver_fn().

如果这些实用程序不能满足您的需求，您可以自行编写自己的serving_input_receiver_fn（）。其中一种情况是，如果你的训练input_fn（）包含了一些预处理逻辑，那么这些逻辑必须在服务时间重演。为了减少训练服务偏斜的风险，我们建议将这样的处理封装在一个函数中，然后在input_fn（）和serving_input_receiver_fn（）中调用这个函数。

Note that the serving_input_receiver_fn() also determines the input portion of the signature. That is, when writing a serving_input_receiver_fn(), you must tell the parser what signatures to expect and how to map them to your model’s expected inputs. By contrast, the output portion of the signature is determined by the model.

请注意，serving_input_receiver_fn（）也决定签名的输入部分。也就是说，在编写一个serving_input_receiver_fn（）时，必须告诉解析器什么样的特征以及如何将它们映射到模型的期望输入。相比之下，签名的输出部分由模型确定。

4.2，Performing the export

To export your trained Estimator, call tf.estimator.Estimator.export_savedmodel with the export base path and the serving_input_receiver_fn.

estimator.export_savedmodel(export_dir_base, serving_input_receiver_fn)

This method builds a new graph by first calling the serving_input_receiver_fn() to obtain feature Tensors, and then calling this Estimator’s model_fn() to generate the model graph based on those features. It starts a fresh Session, and, by default, restores the most recent checkpoint into it. (A different checkpoint may be passed, if needed.) Finally it creates a time-stamped export directory below the given export_dir_base (i.e., export_dir_base/), and writes a SavedModel into it containing a single MetaGraphDef saved from this Session.
该方法通过首先调用serving_input_receiver_fn（）来获得特征张量，然后调用这个估计器的model_fn（）来基于这些特征来生成模型图来构建新的图。 它启动一个新的会话，并且默认情况下，将最近的检查点恢复到它。 （如果需要，可以传递不同的检查点）。最后，它在给定的export_dir_base（即，export_dir_base / ）下面创建一个带时间戳的导出目录，并将SavedModel写入其中，该SavedModel包含从此Session保存的单个MetaGraphDef。

• Note: It is your responsibility to garbage-collect old exports.
  Otherwise, successive exports will accumulate under export_dir_base.

4.3，Specifying the outputs of a custom model

When writing a custom model_fn, you must populate the export_outputs element of the tf.estimator.EstimatorSpec return value. This is a dict of {name: output} describing the output signatures to be exported and used during serving.
在编写自定义的model_fn时，必须填充tf.estimator.EstimatorSpec返回值的export_outputs元素。这是{name：output}的一个字典，用于描述在服务期间要输出和使用的输出签名。

In the usual case of making a single prediction, this dict contains one element, and the name is immaterial. In a multi-headed model, each head is represented by an entry in this dict. In this case the name is a string of your choice that can be used to request a specific head at serving time.

在通常的单一预测的情况下，这个字典包含一个元素，名字是不重要的。在一个多头模型中，每一个头都由这个词典中的一个条目表示。在这种情况下，名称是您选择的字符串，可用于请求服务时间的特定头像。

Each output value must be an ExportOutput object such as tf.estimator.export.ClassificationOutput, tf.estimator.export.RegressionOutput, or tf.estimator.export.PredictOutput.

每个输出值必须是ExportOutput对象，例如tf.estimator.export.ClassificationOutput，tf.estimator.export.RegressionOutput或tf.estimator.export.PredictOutput。

These output types map straightforwardly to the TensorFlow Serving APIs, and so determine which request types will be honored.
Note: In the multi-headed case, a SignatureDef will be generated for each element of the export_outputs dict returned from the model_fn, named using the same keys. These SignatureDefs differ only in their outputs, as provided by the corresponding ExportOutput entry. The inputs are always those provided by the serving_input_receiver_fn. An inference request may specify the head by name. One head must be named using signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY indicating which SignatureDef will be served when an inference request does not specify one.

这些输出类型直接映射到TensorFlow服务API，从而确定哪些请求类型将得到遵守。
注意：在多头情况下，将为从model_fn返回的export_outputs dict的每个元素生成一个SignatureDef，并使用相同的键命名。这些SignatureDefs仅在其输出方面有所不同，由相应的ExportOutput条目提供。输入始终是由serving_input_receiver_fn提供的。推理请求可以通过名称指定头部。必须使用signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY命名一个头，以指示在推理请求未指定哪个SignatureDef时将提供哪个SignatureDef。

4.4，Serving the exported model locally

For local deployment, you can serve your model using TensorFlow Serving, an open-source project that loads a SavedModel and exposes it as a gRPC service.

First, install TensorFlow Serving.

Then build and run the local model server, substituting $export_dir_base with the path to the SavedModel you exported above:

bazel build //tensorflow_serving/model_servers:tensorflow_model_server
bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_base_path=$export_dir_base

Now you have a server listening for inference requests via gRPC on port 9000!

4.5，Requesting predictions from a local server

The server responds to gRPC requests according to the PredictionService gRPC API service definition. (The nested protocol buffers are defined in various neighboring files).
服务器根据PredictionService gRPC API服务定义来响应gRPC请求。 （嵌套协议缓冲区在各种相邻文件中定义）。

From the API service definition, the gRPC framework generates client libraries in various languages providing remote access to the API. In a project using the Bazel build tool, these libraries are built automatically and provided via dependencies like these (using Python for example):
从API服务定义中，gRPC框架以各种语言生成客户端库，提供对API的远程访问。 在使用Bazel构建工具的项目中，这些库是自动构建的，并通过这些依赖关系来提供（例如使用Python）：

  deps = [
    "//tensorflow_serving/apis:classification_proto_py_pb2",
    "//tensorflow_serving/apis:regression_proto_py_pb2",
    "//tensorflow_serving/apis:predict_proto_py_pb2",
    "//tensorflow_serving/apis:prediction_service_proto_py_pb2"
  ]

Python client code can then import the libraries thus:

from tensorflow_serving.apis import classification_pb2
from tensorflow_serving.apis import regression_pb2
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2

• Note: prediction_service_pb2 defines the service as a whole and so is
  always required. However a typical client will need only one of
  classification_pb2, regression_pb2, and predict_pb2, depending on the
  type of requests being made.

Sending a gRPC request is then accomplished by assembling a protocol buffer containing the request data and passing it to the service stub. Note how the request protocol buffer is created empty and then populated via the generated protocol buffer API.
发送一个gRPC请求是通过组装一个包含请求数据的协议缓冲区并将其传递给服务存根来实现的。 请注意，请求协议缓冲区是如何创建的，然后通过生成的协议缓冲区API填充。

from grpc.beta import implementations

channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

request = classification_pb2.ClassificationRequest()
example = request.input.example_list.examples.add()
example.features.feature['x'].float_list.value.extend(image[0].astype(float))

result = stub.Classify(request, 10.0)  # 10 secs timeout

The returned result in this example is a ClassificationResponse protocol buffer.

This is a skeletal example; please see the Tensorflow Serving documentation and examples for more details.

• Note: ClassificationRequest and RegressionRequest contain a
  tensorflow.serving.Input protocol buffer, which in turn contains a
  list of tensorflow.Example protocol buffers. PredictRequest, by
  contrast, contains a mapping from feature names to values encoded via
  TensorProto. Correspondingly: When using the Classify and Regress
  APIs, TensorFlow Serving feeds serialized tf.Examples to the graph,
  so your serving_input_receiver_fn() should include a
  tf.parse_example() Op. When using the generic Predict API, however,
  TensorFlow Serving feeds raw feature data to the graph, so a pass
  through serving_input_receiver_fn() should be used.
  注意：ClassificationRequest和RegressionRequest包含一个
  tensorflow.serving.Input协议缓冲区，其中又包含一个
  张量列表。例子协议缓冲区。 预测请求，由
  对比，包含从功能名称到通过编码值的映射
  TensorProto。 相应地：使用分类和回归时
  API，TensorFlow Serving feeds序列化tf.Examples到图中，
  所以你的serving_input_receiver_fn（）应该包含一个
  tf.parse_example（）Op。 但是，使用通用预测API时，
  TensorFlow Serving将原始特征数据提供给图形，所以通过
  通过serving_input_receiver_fn（）应该被使用。
  

5，CLI to inspect and execute SavedModel

You can use the SavedModel Command Line Interface (CLI) to inspect and execute a SavedModel. For example, you can use the CLI to inspect the model’s SignatureDefs. The CLI enables you to quickly confirm that the input Tensor dtype and shape match the model. Moreover, if you want to test your model, you can use the CLI to do a sanity check by passing in sample inputs in various formats (for example, Python expressions) and then fetching the output.

5.1，Installing the SavedModel CLI

Broadly speaking, you can install TensorFlow in either of the following two ways:

• By installing a pre-built TensorFlow binary.
• By building TensorFlow from source code.

If you installed TensorFlow through a pre-built TensorFlow binary, then the SavedModel CLI is already installed on your system at pathname bin\saved_model_cli.

If you built TensorFlow from source code, you must run the following additional command to build saved_model_cli:

$ bazel build tensorflow/python/tools:saved_model_cli

5.2，Overview of commands

The SavedModel CLI supports the following two commands on a MetaGraphDef in a SavedModel:

show, which shows a computation on a MetaGraphDef in a SavedModel.
run, which runs a computation on a MetaGraphDef.

5.3，show command

A SavedModel contains one or more MetaGraphDefs, identified by their tag-sets. To serve a model, you might wonder what kind of SignatureDefs are in each model, and what are their inputs and outputs. The show command let you examine the contents of the SavedModel in hierarchical order. Here’s the syntax:

usage: saved_model_cli show [-h] --dir DIR [--all]
[--tag_set TAG_SET] [--signature_def SIGNATURE_DEF_KEY]

For example, the following command shows all available MetaGraphDef tag-sets in the SavedModel:

$ saved_model_cli show --dir /tmp/saved_model_dir
The given SavedModel contains the following tag-sets:
serve
serve, gpu

The following command shows all available SignatureDef keys in a MetaGraphDef:

$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve
The given SavedModel `MetaGraphDef` contains `SignatureDefs` with the
following keys:
SignatureDef key: "classify_x2_to_y3"
SignatureDef key: "classify_x_to_y"
SignatureDef key: "regress_x2_to_y3"
SignatureDef key: "regress_x_to_y"
SignatureDef key: "regress_x_to_y2"
SignatureDef key: "serving_default"

If a MetaGraphDef has multiple tags in the tag-set, you must specify all tags, each tag separated by a comma. For example:

$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve,gpu

To show all inputs and outputs TensorInfo for a specific SignatureDef, pass in the SignatureDef key to signature_def option. This is very useful when you want to know the tensor key value, dtype and shape of the input tensors for executing the computation graph later. For example:

$ saved_model_cli show --dir \
/tmp/saved_model_dir --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
inputs['x'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: x:0
The given SavedModel SignatureDef contains the following output(s):
outputs['y'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: y:0

Method name is: tensorflow/serving/predict

To show all available information in the SavedModel, use the –all option. For example:

$ saved_model_cli show --dir /tmp/saved_model_dir --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['classify_x2_to_y3']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: x2:0
The given SavedModel SignatureDef contains the following output(s):
outputs['scores'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: y3:0
Method name is: tensorflow/serving/classify

signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['x'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: x:0
The given SavedModel SignatureDef contains the following output(s):
outputs['y'] tensor_info:
    dtype: DT_FLOAT
    shape: (-1, 1)
    name: y:0
Method name is: tensorflow/serving/predict

5.4，run command

Invoke the run command to run a graph computation, passing inputs and then displaying (and optionally saving) the outputs. Here’s the syntax:

usage: saved_model_cli run [-h] –dir DIR –tag_set TAG_SET –signature_def
SIGNATURE_DEF_KEY [–inputs INPUTS]
[–input_exprs INPUT_EXPRS] [–outdir OUTDIR]
[–overwrite] [–tf_debug]

The run command provides the following two ways to pass inputs to the model:

--inputs option enables you to pass numpy ndarray in files.
--input_exprs option enables you to pass Python expressions.

–inputs

To pass input data in files, specify the –inputs option, which takes the following general format:

–inputs

where INPUTS is either of the following formats:

=
=[]

You may pass multiple INPUTS. If you do pass multiple inputs, use a semicolon to separate each of the INPUTS.

saved_model_cli uses numpy.load to load the filename. The filename may be in any of the following formats:

.npy
.npz
pickle format

A .npy file always contains a numpy ndarray. Therefore, when loading from a .npy file, the content will be directly assigned to the specified input tensor. If you specify a variable_name with that .npy file, the variable_name will be ignored and a warning will be issued.

When loading from a .npz (zip) file, you may optionally specify a variable_name to identify the variable within the zip file to load for the input tensor key. If you don’t specify a variable_name, the SavedModel CLI will check that only one file is included in the zip file and load it for the specified input tensor key.

When loading from a pickle file, if no variable_name is specified in the square brackets, whatever that is inside the pickle file will be passed to the specified input tensor key. Otherwise, the SavedModel CLI will assume a dictionary is stored in the pickle file and the value corresponding to the variable_name will be used.
–inputs_exprs

To pass inputs through Python expressions, specify the –input_exprs option. This can be useful for when you don’t have data files lying around, but still want to sanity check the model with some simple inputs that match the dtype and shape of the model’s SignatureDefs. For example:

input_key=[[1], [2], [3]]`

In addition to Python expressions, you may also pass numpy functions. For example:

input_key=np.ones((32, 32, 3))

(Note that the numpy module is already available to you as np.)
Save Output

By default, the SavedModel CLI writes output to stdout. If a directory is passed to –outdir option, the outputs will be saved as npy files named after output tensor keys under the given directory.

Use –overwrite to overwrite existing output files.
TensorFlow Debugger (tfdbg) Integration

If –tf_debug option is set, the SavedModel CLI will use the TensorFlow Debugger (tfdbg) to watch the intermediate Tensors and runtime graphs or subgraphs while running the SavedModel.
Full examples of run

Given:

    Your model simply adds x1 and x2 to get output y.
    All tensors in the model have shape (-1, 1).
    You have two npy files:
    /tmp/my_data1.npy, which contains a numpy ndarray [[1], [2], [3]].
    /tmp/my_data2.npy, which contains another numpy ndarray [[0.5], [0.5], [0.5]].

To run these two npy files through the model to get output y, issue the following command:

$ saved_model_cli run --dir /tmp/saved_model_dir --tag_set serve \
--signature_def x1_x2_to_y --inputs x1=/tmp/my_data1.npy;x2=/tmp/my_data2.npy \
--outdir /tmp/out
Result for output key y:
[[ 1.5]
 [ 2.5]
 [ 3.5]]

Let’s change the preceding example slightly. This time, instead of two .npy files, you now have an .npz file and a pickle file. Furthermore, you want to overwrite any existing output file. Here’s the command:

$ saved_model_cli run --dir /tmp/saved_model_dir --tag_set serve \
--signature_def x1_x2_to_y \
--inputs x1=/tmp/my_data1.npz[x];x2=/tmp/my_data2.pkl --outdir /tmp/out \
--overwrite
Result for output key y:
[[ 1.5]
 [ 2.5]
 [ 3.5]]

You may specify python expression instead of an input file. For example, the following command replaces input x2 with a Python expression:

$ saved_model_cli run --dir /tmp/saved_model_dir --tag_set serve \
--signature_def x1_x2_to_y --inputs x1=/tmp/my_data1.npz[x] \
--input_exprs 'x2=np.ones((3,1))'
Result for output key y:
[[ 2]
 [ 3]
 [ 4]]

To run the model with the TensorFlow Debugger on, issue the following command:

$ saved_model_cli run --dir /tmp/saved_model_dir --tag_set serve \
--signature_def serving_default --inputs x=/tmp/data.npz[x] --tf_debug

6，Structure of a SavedModel directory

When you save a model in SavedModel format, TensorFlow creates a SavedModel directory consisting of the following subdirectories and files:

assets/
assets.extra/
variables/
    variables.data-?????-of-?????
    variables.index
saved_model.pb|saved_model.pbtxt

where:

1. assets is a subfolder containing auxiliary (external) files, such as
   vocabularies. Assets are copied to the SavedModel location and can
   be read when loading a specific MetaGraphDef.

2. assets.extra is a subfolder where higher-level libraries and users
   can add their own assets that co-exist with the model, but are not
   loaded by the graph. This subfolder is not managed by the SavedModel
   libraries.

3. variables is a subfolder that includes output from tf.train.Saver.

4. saved_model.pb or saved_model.pbtxt is the SavedModel protocol
   buffer. It includes the graph definitions as MetaGraphDef protocol
   buffers.

A single SavedModel can represent multiple graphs. In this case, all the graphs in the SavedModel share a single set of checkpoints (variables) and assets. For example, the following diagram shows one SavedModel containing three MetaGraphDefs, all three of which share the same set of checkpoints and assets:
一个SavedModel可以表示多个图形。 在这种情况下，SavedModel中的所有图都共享一组检查点（变量）和资产。 例如，下图显示了一个包含三个MetaGraphDefs的SavedModel，其中三个都共享同一组检查点和资产：

SavedModel represents checkpoints, assets, and one or more MetaGraphDefs

Each graph is associated with a specific set of tags, which enables identification during a load or restore operation.
每个图形都与一组特定的标签相关联，从而可以在加载或恢复操作期间进行识别。