Tensorflow: Eager Execution 使用入门

常见术语：

特征(集)即某个样本的特点[对个特征描述就是特征集]；
标签即我们尝试预测的内容；
(一个)样本由一个特征集和标签组成；
模型是特征和标签之间的关系；
训练是一种机器学习阶段，在此阶段中，模型会逐渐得到优化(不断学习)
监督式机器学习即模型通过包含标签的样本加以训练;
非监督式机器学习中，样本不包含标签, 相反，模型通常会在特征中发现一些规律。
训练集包含我们用于训练模型的样本；
测试集包含我们用于评估训练后模型的效果的样本。

代码框架

1. 导入和解析数据集。
2. 创建特征列以描述数据。
3. 选择模型类型。
4. 训练模型。
5. 评估模型的效果。
6. 让经过训练的模型进行预测

1.tf.keras.utils.get_file:

下载 URL文件到本地(~/.keras/datasets/fname), 返回下载文件的路径
get_file in module tensorflow.python.keras.utils.data_utils:

get_file(fname, origin, 
    untar=False, md5_hash=None, file_hash=None, 
    cache_subdir='datasets', hash_algorithm='auto', 
    extract=False, archive_format='auto', cache_dir=None)

    Downloads a file from a URL if it not already in the cache.
    
    By default the file at the url `origin` is downloaded to the
    cache_dir `~/.keras`, placed in the cache_subdir `datasets`,
    and given the filename `fname`. The final location of a file
    `example.txt` would therefore be `~/.keras/datasets/example.txt`.
    
    Files in tar, tar.gz, tar.bz, and zip formats can also be extracted.
    Passing a hash will verify the file after download. The command line
    programs `shasum` and `sha256sum` can compute the hash.

    Returns:
        Path to the downloaded file

2.tf.decode_csv

转换为Tensor对象　list
decode_csv in module tensorflow.python.ops.parsing_ops:

decode_csv(records, record_defaults, 
    field_delim=',', use_quote_delim=True, 
    name=None, na_value='', select_cols=None)

    Convert CSV records to tensors. Each column maps to one tensor.  
    
    Args:
      records: A `Tensor` of type `string`.
        Each string is a record/row in the csv and all records should have
        the same format.
      record_defaults: A list of `Tensor` objects with specific types.
        Acceptable types are `float32`, `float64`, `int32`, `int64`, `string`.
        One tensor per column of the input record, with either a
        scalar default value for that column or empty if the column is required.
      field_delim: An optional `string`. Defaults to `","`.
        char delimiter to separate fields in a record.
      use_quote_delim: An optional `bool`. Defaults to `True`.
        If false, treats double quotation marks as regular
        characters inside of the string fields (ignoring RFC 4180, Section 2,
        Bullet 5).
      name: A name for the operation (optional).
      na_value: Additional string to recognize as NA/NaN.
      select_cols: Optional sorted list of column indices to select. If specified,
        only this subset of columns will be parsed and returned.
    
    Returns:
      A list of `Tensor` objects. Has the same type as `record_defaults`.
      Each tensor will have the same shape as records.

3.tf.data.TextLineDataSet

TextLineDataset in module tensorflow.python.data.ops.readers:

class TextLineDataset(tensorflow.python.data.ops.dataset_ops.Dataset)
   A `Dataset` comprising lines from one or more text files.

4.iter(train_dataset).next()

iter
built-in function iter in module __builtin__:

iter(...)
    iter(collection) -> iterator

5.sparse_softmax_cross_entropy

sparse_softmax_cross_entropy 标签实际值和通过模型预测结果间的差
sparse_softmax_cross_entropy in module tensorflow.python.ops.losses.losses_impl:

sparse_softmax_cross_entropy(labels, logits, 
    weights=1.0, scope=None, loss_collection='losses', 
    reduction='weighted_sum_by_nonzero_weights')

    Cross-entropy loss using `tf.nn.sparse_softmax_cross_entropy_with_logits`.
    
    Returns:
      Weighted loss `Tensor` of the same type as `logits`. If `reduction` is
      `NONE`, this has the same shape as `labels`; otherwise, it is scalar.

6.GradientTape

使用GradientTape对象，调用其gradient函数，gardient对loss函数对model.variables进行求导
GradientTape in module tensorflow.python.eager.backprop:

class GradientTape(__builtin__.object)
 |  Record operations for automatic differentiation.
 |  
 |  Operations are recorded if they are executed within this context manager and
 |  at least one of their inputs is being "watched".
 |  
 |  Trainable variables (created by `tf.contrib.eager.Variable` or
 |  @{tf.get_variable}, trainable=True is default in both cases) are automatically
 |  watched. Tensors can be manually watched by invoking the `watch` method on
 |  this context manager.
 |  
 |  For example, consider the function `y = x * x`. The gradient at `x = 3.0` can
 |  be computed as:
 |  
 |  ```python
 |  x = tf.constant(3.0)
 |  with tf.GradientTape() as g:
 |    g.watch(x)
 |    y = x * x
 |  dy_dx = g.gradient(y, x) # Will compute to 6.0
 |  ```
 |  
 |  GradientTapes can be nested to compute higher-order derivatives. For example,
 |  
 |  ```python
 |  x = tf.constant(3.0)
 |  with tf.GradientTape() as g:
 |    with tf.GradientTape() as gg:
 |      gg.watch(x)
 |      y = x * x
 |    dy_dx = gg.gradient(y, x)     # Will compute to 6.0
 |  d2y_dx2 = g.gradient(dy_dx, x)  # Will compute to 2.0
 |  ```
 |  
 |  By default, the resources held by a GradientTape are released as soon as
 |  GradientTape.gradient() method is called. To compute multiple gradients over
 |  the same computation, create a persistent gradient tape. This allows multiple
 |  calls to the gradient() method as resources are released when the tape object
 |  is garbage collected. For example:
 |  
 |  ```python
 |  x = tf.constant(3.0)
 |  with tf.GradientTape(persistent=True) as g:
 |    g.watch(x)
 |    y = x * x
 |    z = y * y
 |  dz_dx = g.gradient(z, x)  # 108.0 (4*x^3 at x = 3)
 |  dy_dx = g.gradient(y, x)  # 6.0
 |  del g  # Drop the reference to the tape
 |  ```
 |  
 |  Note that only tensors with real or complex dtypes are differentiable.
 |  
 |  Methods defined here:

 |  gradient(self, target, sources, output_gradients=None)
 |      Computes the gradient using operations recorded in context of this tape.
 |      
 |      Args:
 |        target: Tensor (or list of tensors) to be differentiated.

 |        sources: a list or nested structure of Tensors or Variables. `target`
 |          will be differentiated against elements in `sources`.

 |      
 |      Returns:
 |        a list or nested structure of Tensors (or IndexedSlices, or None),
 |        one for each element in `sources`. Returned structure is the same as
 |        the structure of `sources`.

7.GradientDescentOptimizer

怎样使用梯度？
GradientDescentOptimizer

GradientDescentOptimizer in module tensorflow.python.training.gradient_descent:

class GradientDescentOptimizer(tensorflow.python.training.optimizer.Optimizer)
 |  Optimizer that implements the gradient descent algorithm.

8.argmax

argmax in module tensorflow.python.ops.math_ops:

argmax(*args, **kwargs)
    Returns the index with the largest value across axes of a tensor.
    
    Args:
      input: A `Tensor`. Must be one of the following types: .

      axis: A `Tensor`. Must be one of the following types: `int32`, `int64`.
        int32 or int64, must be in the range `[-rank(input), rank(input))`.
        Describes which axis of the input Tensor to reduce across. 
    For vectors, use axis = 0.

      output_type: An optional `tf.DType` from: `tf.int32, tf.int64`. Defaults to `tf.int64`.
      name: A name for the operation (optional).
    
    Returns:
      A `Tensor` of type `output_type`.

/****************************************************************************************/
import os
import matplotlib.pyplot as plt

import tensorflow as tf
import tensorflow.contrib.eager as tfe

tf.enable_eager_execution()

print("TensorFlow version: {}".format(tf.VERSION))
print("Eager execution: {}".format(tf.executing_eagerly()))

# download training dataset
train_dataset_url = "http://download.tensorflow.org/data/iris_training.csv"
train_dataset_fp = tf.keras.utils.get_file(fname=os.path.basename(train_dataset_url),
                                           origin=train_dataset_url)
print("Local copy of the dataset file: {}".format(train_dataset_fp))

# parse data: for each line
def parse_csv(line):
  example_defaults = [[0.], [0.], [0.], [0.], [0]]  # sets field types
  parsed_line = tf.decode_csv(line, example_defaults)
  # First 4 fields are features, combine into single tensor
  features = tf.reshape(parsed_line[:-1], shape=(4,))
  # Last field is the label
  label = tf.reshape(parsed_line[-1], shape=())
  return features, label

#line -> dataset -> map[分开特征和标签]　-> shuffle -> batch
train_dataset = tf.data.TextLineDataset(train_dataset_fp)
train_dataset = train_dataset.skip(1)             # skip the first header row
train_dataset = train_dataset.map(parse_csv)      # parse each row
train_dataset = train_dataset.shuffle(buffer_size=1000)  # randomize
train_dataset = train_dataset.batch(32)

#　这里只是看看sample数据
# View a single example entry from a batch
features, label = iter(train_dataset).next()
print("example features:", features[0])
print("example label:", label[0])

# 构建model的层：输入特征input_shape
model = tf.keras.Sequential([
  tf.keras.layers.Dense(10, activation="relu", input_shape=(4,)),  # input shape required
  tf.keras.layers.Dense(10, activation="relu"),
  tf.keras.layers.Dense(3) //并没有softmat计算
])

# input -> model -> y_ 计算模式输出结果和实际结果间的 loss
def loss(model, x, y):
  y_ = model(x)
  return tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_)

#　计算损失函数对weight的梯度
def grad(model, inputs, targets):
  with tf.GradientTape() as tape:
    loss_value = loss(model, inputs, targets)
  return tape.gradient(loss_value, model.variables)

#　怎样使用梯度优化
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)

## Note: Rerunning this cell uses the same model variables

# keep results for plotting
train_loss_results = []
train_accuracy_results = []

num_epochs = 201

for epoch in range(num_epochs):
  epoch_loss_avg = tfe.metrics.Mean()
  epoch_accuracy = tfe.metrics.Accuracy()

  # Training loop - using batches of 32
  for x, y in train_dataset:
    # Optimize the model

    grads = grad(model, x, y)
    optimizer.apply_gradients(zip(grads, model.variables),
                              global_step=tf.train.get_or_create_global_step())
    # 到此生成模型的代码已经执行完了

    # Track progress　跟踪执行过程
    epoch_loss_avg(loss(model, x, y))  # add current batch loss
    # compare predicted label to actual label
    epoch_accuracy(tf.argmax(model(x), axis=1, output_type=tf.int32), y)

  # end epoch　每次epoch执行一遍训练集
  train_loss_results.append(epoch_loss_avg.result())
  train_accuracy_results.append(epoch_accuracy.result())

  if epoch % 50 == 0:
    print("Epoch {:03d}: Loss: {:.3f}, Accuracy: {:.3%}".format(epoch,
                                                                epoch_loss_avg.result(),
                                                                epoch_accuracy.result()))

#测试训练出的模型
test_url = "http://download.tensorflow.org/data/iris_test.csv"
test_fp = tf.keras.utils.get_file(fname=os.path.basename(test_url),
                                  origin=test_url)
test_dataset = tf.data.TextLineDataset(test_fp)
test_dataset = test_dataset.skip(1)             # skip header row
test_dataset = test_dataset.map(parse_csv)      # parse each row with the funcition created earlier
test_dataset = test_dataset.shuffle(1000)       # randomize
test_dataset = test_dataset.batch(32)           # use the same batch size as the training set

test_accuracy = tfe.metrics.Accuracy()

for (x, y) in test_dataset:
  prediction = tf.argmax(model(x), axis=1, output_type=tf.int32)
  test_accuracy(prediction, y)

print("Test set accuracy: {:.3%}".format(test_accuracy.result()))

#预测数据(泛化)
class_ids = ["Iris setosa", "Iris versicolor", "Iris virginica"]
predict_dataset = tf.convert_to_tensor([
    [5.1, 3.3, 1.7, 0.5,],
    [5.9, 3.0, 4.2, 1.5,],
    [6.9, 3.1, 5.4, 2.1]
])

predictions = model(predict_dataset)

for i, logits in enumerate(predictions):
  class_idx = tf.argmax(logits).numpy()
  name = class_ids[class_idx]
  print("Example {} prediction: {}".format(i, name))