深度学习笔记——深度学习框架TensorFlow(五)[TensorFlow大规模线性模型教程]

参考网站：

1. https://www.tensorflow.org/versions/r0.12/tutorials/linear/overview.html#large-scale-linear-models-with-tensorflow
2. http://www.jianshu.com/p/afe41ae17a16

Large-scale Linear Models with TensorFlow：

The tf.learn API provides (among other things) a rich set of tools for working with linear models in TensorFlow. This document provides an overview of those tools. It explains:
在Tensorflow里，tf.learn API 提供一个丰富的用于线性模型的工具集。这篇文档提供一个关于这些工具的概述，解释了：

1. what a linear model is.
   什么是线性模型？
2. why you might want to use a linear model.
   你为什么会想使用线性模型
3. how tf.learn makes it easy to build linear models in TensorFlow.
   如何在 Tensorflow 内使 tf.learn 方便的建立线性模型。
4. how you can use tf.learn to combine linear models with deep learning to get the advantages of both.
   如何使用 tf.learn 将线性模型和深度学习进行结合以获得其优势。

Read this overview to decide whether the tf.learn linear model tools might be useful to you. Then do the Linear Models tutorial to give it a try. This overview uses code samples from the tutorial, but the tutorial walks through the code in greater detail.
阅读这篇概述以决定 tf.learn 线性模型工具是否对你有用。然后尝试一下 线性模型教程（https://www.tensorflow.org/versions/r0.12/tutorials/wide/index.html）。这篇概述会使用该教程的示例代码，并详细介绍这些代码。

To understand this overview it will help to have some familiarity with basic machine learning concepts, and also with tf.learn.
理解这篇概述，它将有助于你熟悉基本的机器学习的概念和 tf.learn（https://www.tensorflow.org/versions/r0.12/tutorials/tflearn/index.html）.

目录：
TensorFlow下的大规模线性模型：
1. 什么是线性模型？
2. 为什么你想使用线性模型？
3. tf.learn如何帮助你建立线性模型？（特征列和转换，线性估计器，广度和深度学习）

什么是线性模型：

A linear model uses a single weighted sum of features to make a prediction. For example, if you have data on age, years of education, and weekly hours of work for a population, you can learn weights for each of those numbers so that their weighted sum estimates a person’s salary. You can also use linear models for classification.
一个线性模型使用特征的单个加权和来进行预测。例如，如果你有一些人口的年龄，受教育程度和每周工作时间的数据。你可以学习到每个参数的权重，学习这些数值的权重和，最后用其来估计一个人的薪水。你也同样可以使用线性模型来进行分类。

Some linear models transform the weighted sum into a more convenient form. For example, logistic regression plugs the weighted sum into the logistic function to turn the output into a value between 0 and 1. But you still just have one weight for each input feature.
一些线性模型会将加权和转换为一个更方便的形式。例如，logistic 回归将加权和注入进 logistic函数中并将输出转换为0和1之间的值。但对于每一个输入特征你仍然只有一个权重。

为什么你需要使用线性模型：
Why would you want to use so simple a model when recent research has demonstrated the power of more complex neural networks with many layers?
为什么你要在最近的研究展示了多层复杂的神经网络的（强大）能力的情况下，使用一个如此简单的模型？

Linear models：

1. train quickly, compared to deep neural nets.
2. can work well on very large feature sets.
3. can be trained with algorithms that don’t require a lot of fiddling with learning rates, etc.
4. can be interpreted and debugged more easily than neural nets. You can examine the weights assigned to each feature to figure out what’s having the biggest impact on a prediction.
5. provide an excellent starting point for learning about machine learning.
6. are widely used in industry.
   线性模型：
   
   1. 对比于神经网络，它的训练速度比较快
   2. 在极为庞大的特征集中，它表现的很好
   3. 可以训练算法，而不需要很多的调整学习速率的训练，等等。
   4. 比起神经网络，容易进行解释和调试。你可以为每个特征分配权重以获得其对于预测有着多大的影响。
   5. 为学习机器学习提供了一个极好的起步。
   6. 广泛应用于工业。

tf.learn如何帮助你建立线性模型:

You can build a linear model from scratch in TensorFlow without the help of a special API. But tf.learn provides some tools that make it easier to build effective large-scale linear models.
在Tensorflow下，你能够在不借助一些特别的API的情况下，从头开始建立一个线性模型。并且 tf.learn 提供一些工具使得能够的建立一个大规模线性模型。

特征列和转换：
Much of the work of designing a linear model consists of transforming raw data into suitable input features. tf.learn uses the FeatureColumn abstraction to enable these transformations.
设计一个线性模型有许多工作，包括转换原始数据成一个适合的输入特征。tf.learn使用FeatureColumn抽象这些变化

A FeatureColumn represents a single feature in your data. A FeatureColumn may represent a quantity like ‘height’, or it may represent a category like ‘eye_color’ where the value is drawn from a set of discrete possibilities like {‘blue’, ‘brown’, ‘green’}.
一个FeatureColumn代表你数据中的一个单一特征，FeatureColumn可以代表’高度’的值，或者它可以代表’眼睛颜色’，其值可能会被表示成{‘蓝色’,’褐色’,’绿色’}的离散概率

In the case of both continuous features like ‘height’ and categorical features like ‘eye_color’, a single value in the data might get transformed into a sequence of numbers before it is input into the model. The FeatureColumn abstraction lets you manipulate the feature as a single semantic unit in spite of this fact. You can specify transformations and select features to include without dealing with specific indices in the tensors you feed into the model.
在这两种连续特征，像 ‘高度’ 和 像 ‘眼睛颜色’ 的 ‘分类特征’ 情况下，单个值在输入这个模型前可能会被转换成一个数字序列。抽象出的FeatureColumn 使得你能够像操作语义一样来操作这些特征。你可以在不需要处理输入模型的张量中的特定指数的情况下，指定转换方式并选择要进行转换的特征。

Sparse Columns（稀疏列）

Categorical features in linear models are typically translated into a sparse vector in which each possible value has a corresponding index or id. For example, if there are only three possible eye colors you can represent ‘eye_color’ as a length 3 vector: ‘brown’ would become [1, 0, 0], ‘blue’ would become [0, 1, 0] and ‘green’ would become [0, 0, 1]. These vectors are called “sparse” because they may be very long, with many zeros, when the set of possible values is very large (such as all English words).
分类特征在线性模型通常被转化成一个稀疏向量，并且其对应值都有一个对应的序列或者id。例如，如果对于眼睛颜色只存在三种可能性的情况下，你可以表示‘眼睛颜色’为一个长度为3的向量：’棕色’应为[1, 0, 0], ‘蓝色’应为[0, 1, 0]然后 ‘绿色’ 为 [0, 0, 1]。当可能值的集合非常大时（如所有英语单词），这些向量被称为“”稀疏向量”，因为他们可能很长但是存在很多0值。

While you don’t need to use sparse columns to use tf.learn linear models, one of the strengths of linear models is their ability to deal with large sparse vectors. Sparse features are a primary use case for the tf.learn linear model tools.
当你不需要通过稀疏列来使用 tf.learn 的线性模型时， 线性模型的其中一个优势是他有能力去处理大规模的稀疏向量。稀疏特征是 tf.learn 线性模型工具的一个主要用例。

Encoding sparse columns（编码稀疏列）

FeatureColumn handles the conversion of categorical values into vectors automatically, with code like this:
FeatureColumn主要用于自动将分类值转化为向量值，代码如下：

eye_color = tf.contrib.layers.sparse_column_with_keys(column_name="eye_color0",keys=["blue","brown","green"])

where eye_color is the name of a column in your source data.
eye_color是你的源数据中一列的名称。

You can also generate FeatureColumns for categorical features for which you don’t know all possible values. For this case you would use sparse_column_with_hash_bucket(), which uses a hash function to assign indices to feature values.
当你不知道所有的可能值时，你同样能够为分类特征生成FeatureColumns。在这种情况下你应该使用sparse_column_with_hash_bucket()，其使用hash函数去为特征值分类序列。

education = tf.contrib.layers.sparse_column_with_hash_buket(\"education",hash_bucket_size=1000)

Feature Crosses(交叉特征)：

Because linear models assign independent weights to separate features, they can’t learn the relative importance of specific combinations of feature values. If you have a feature ‘favorite_sport’ and a feature ‘home_city’ and you’re trying to predict whether a person likes to wear red, your linear model won’t be able to learn that baseball fans from St. Louis especially like to wear red.
因为线性模型是给每个单独的特征分配独立的权重，所以他们无法学习特征在特定组合的重要性。如果你有一个特征 ‘最喜欢的运动’ 和特征 ‘居住城市’，然后并且你试图想要推断一个人在什么情况下会穿红色的衣服。你的线性模型将不会学习到来自St.Louis的棒球粉丝是很喜欢穿红色系的衣服。

You can get around this limitation by creating a new feature ‘favorite_sport_x_home_city’. The value of this feature for a given person is just the concatenation of the values of the two source features: ‘baseball_x_stlouis’, for example. This sort of combination feature is called a feature cross.
你可以通过创建一个新的叫‘最喜欢的运动x居住城市’ 特征来克服这个限制。这个特征的值对一个给定的人（的数据）来说只是关联了两个源特征的值。例如，这种组合特征被称为交叉特征。

The crossed_column() method makes it easy to set up feature crosses:
crossed_column()函数能够很容易就建立出交叉特征。

sport = tf.contrib.layers.sparse_column_with_hash_bucket(\"sport",hash_bucket_size=1000)
city = tf.contrib.layers.sparse_column)with_hash_bucket(\"city",hash_bucket_size=1000)
sport_x_city = tf.contrib.layers.crossed_column([sport,city],hash_bucket_size=int(1e4))

Continuous columns(连续列)

You can specify a continuous feature like so:
你可以通过如下方式定义一个特征：

age = tf.contrib.layers.real_valued_column("age")

Although, as a single real number, a continuous feature can often be input directly into the model, tf.learn offers useful transformations for this sort of column as well.
尽管对于一个实数，一个连续的特征一般是能够直接输入到模型中的。但tf.learn也提供对连续列进行转换。

Bucketization（桶化）
Bucketization turns a continuous column into a categorical column. This transformation lets you use continuous features in feature crosses, or learn cases where specific value ranges have particular importance.
桶化转换一个连续列为分类列。这种转换会让你在交叉特征中使用连续特征，或者是学习对于特定的范围值能够造成的重大影响的特征。

Bucketization divides the range of possible values into subranges called buckets:
桶化会将可能值的范围划分为一系列小的子范围，这样的子范围被称之为桶：

age_buckets = tf.contrib.layers.bucketized_column(
    age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])

Input function(输入函数)

FeatureColumns provide a specification for the input data for your model, indicating how to represent and transform the data. But they do not provide the data itself. You provide the data through an input function.
FeatureColumns为你的模型输入提供了一个规范格式，表示如何去表示和转换这些数据。但是其本身不会提供这些数据。你需要通过一个输入函数来提供这些数据。

The input function must return a dictionary of tensors. Each key corresponds to the name of a FeatureColumn. Each key’s value is a tensor containing the values of that feature for all data instances. See Building Input Functions with tf.contrib.learn for a more comprehensive look at input functions, and input_fn in the linear models tutorial code for an example implementation of an input function.
这个输入函数必须返回一个张量的目录。每一个键值对应一个FeatureColumn的名字。每一个键值对应的值是一个包含所有数据实例中某类特征值的张量。详情可见使用tf.contrib.learn建立输入函数（https://www.tensorflow.org/versions/r0.12/tutorials/input_fn/index.html）以便更全面的了解输入函数，并且input_fn在线性模型教程代码（https://github.com/tensorflow/tensorflow/blob/r0.12/tensorflow/examples/learn/wide_n_deep_tutorial.py）中有引入了一个输入函数的实例。

The input function is passed to the fit() and evaluate() calls that initiate training and testing, as described in the next section.
输入函数通过调用fit()和evaluate()来发起训练和测试，将会在下一节介绍他们。

Linear estimators：

tf.learn’s estimator classes provide a unified training and evaluation harness for regression and classification models. They take care of the details of the training and evaluation loops and allow the user to focus on model inputs and architecture.
tf.learn的估计器类为回归和分类模型提供了一个统一的训练和评估方法。他们主要处理训练和评估循环的细节及其允许用户能够更专注于模型输入与结构。

To build a linear estimator, you can use either the tf.contrib.learn.LinearClassifier estimator or the tf.contrib.learn.LinearRegressor estimator, for classification and regression respectively.
为了建立一个线性估计器，你可以使用tf.contrib.learn.LinearClassifier估计器或tf.contrib.learn.LinearRegressor估计器分别用于分类和回归。

As with all tf.learn estimators, to run the estimator you just:
1. Instantiate the estimator class. For the two linear estimator classes, you pass a list of FeatureColumns to the constructor.
2. Call the estimator’s fit() method to train it.
3. Call the estimator’s evaluate() method to see how it does.
所有的tf.learn估计器，只需要以下几步就可以运行：
1. 实例化估计器类。对于这两种估计器类，需要传递一个FeatureColumns列表到其构造器中。
2. 调用估计器的fit()方法去训练它。
3. 调用估计器的evaluate()方法去观察其运行。

e = tf.contrib.learn.LinearClassifier(feature_columns=[native_country,eduction,occupation,workclasss,marital_status,race,age_buckets,education_x_occupation,age_buckets_x_race_x_occupamodel_dir=YOUR_MODEL_DIRECTORY])
e.fit(input_fn=input_fn_train,steps = 2000)
#Evaluate for one step(one pass through the test data)
results = e.evaluate(input_fn = input_fn_test,steps = 1)
#Print the stats for the evaluation
for key in sorted(results)
    print "%s:%s"%(key,results[key])

Wide and deep learning ：

The tf.learn API also provides an estimator class that lets you jointly train a linear model and a deep neural network. This novel approach combines the ability of linear models to “memorize” key features with the generalization ability of neural nets. Use tf.contrib.learn.DNNLinearCombinedClassifier to create this sort of “wide and deep” model:
tf.learn API同样提供一个估计器类使得你将训练一个线性模型和一个深度神经网络连接起来。这种新颖的方法结合了线性模型“记忆”关键特征与神经网络的泛化能力。使用tf.contrib.learn.DNNLinearCombinedClassifier去创建这种“广度且深度”的模型：

e = tf.contrib.learn.DNNLinearCombinedClassifier(model_dir=YOUR_MODEL_DIR,linear_feature_columns=wide_columns,dnn_feature_columns=deep_columns,dnn_hidden_units=[100,50])

欲了解更多信息，可以查看这个广度和深度学习教程.(https://www.tensorflow.org/versions/r0.12/tutorials/wide_and_deep/index.html)