Tensorflow实现自编码器
本文参考了黄文坚和唐源所著的《TensorFlow实战》一书。
操作系统以及语言环境：MacOS Python3.5 TensorFlow1.1
下面开始实现最常用的自编码器。先导入NumPy库，Scikit-learn中的preprocessing模块，这个模块是对数据进行预处理的常用模块，之后会用到其中的数据标准化功能。
本文使用的数据集是MNIST数据集，导入TensorFlow中的MNIST数据加载模块。以下代码来自https://github.com/tensorflow/models.git的开源实现。可以使用git clone https://github.com/tensorflow/models.git 下载到本地。然后：
python3 pythitiveGaussianNoiseAutoe执行的结果：
Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.
Extracting MNIST_data/train-images-idx3-ubyte.gz
Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
2017-06-05 19:32:30.406118: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 19:32:30.406157: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 19:32:30.406166: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 19:32:30.406174: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-05 19:32:30.406191: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Epoch: 0001 cost= 19437.877923864
Epoch: 0002 cost= 12998.519503409
Epoch: 0003 cost= 11421.551217614
Epoch: 0004 cost= 10343.985351705
Epoch: 0005 cost= 9752.038456250
Epoch: 0006 cost= 9462.600794318
Epoch: 0007 cost= 8626.377535227
Epoch: 0008 cost= 8940.957032386
Epoch: 0009 cost= 8305.637163636
Epoch: 0010 cost= 8842.561927273
Epoch: 0011 cost= 8410.173371023
Epoch: 0012 cost= 8224.096524432
Epoch: 0013 cost= 8724.879144886
Epoch: 0014 cost= 7552.637163636
Epoch: 0015 cost= 7590.344242614
Epoch: 0016 cost= 7676.122007955
Epoch: 0017 cost= 7439.337359091
Epoch: 0018 cost= 8212.683217614
Epoch: 0019 cost= 7578.246875568
Epoch: 0020 cost= 7732.438578977
Total cost: 649715.0
import numpy as np
import sklearn.preprocessing as prep
import tensorflow as tf
from tensorflow.example.tutoraials.mnist import inout_data
我们使用了一种参数初始化方法xavier initialization，需要先定义好。Yoshua Bengio在一篇论文中指出，如果深度学习的初始权重初始化太小，那信号将在每层传递时候逐渐减小难以产生作用，但是如果初始化太大，那信号在传递时就会逐渐放大导致发散并失效。所以xavier所做的事情就是初始化合适的权重。从数学的角度分析，xavier就是让权重满足0均值，同时方差为：
代码如下所示：
def xavier_init(fan_in,fan_out,constant=1):
low = -constant*np.sqrt(6.0/fan_in+fan_out)
high = constant* np.squrt(6/fan_in+fan_out)
return tf.random_uniform(fan_in,fan_out),minval = low,macval=high,dtype = tf.float32)
这里实现的就是标准的均匀分布的xavier初始化器，其中fan_in是输入节点的数量，fan_out是输出节点的数量。
下面是一个自编码器的class。
class AdditiveGaussionNoiseAutoencoder(object):
def __init__(self,n_input,n_hidden,transfer_function=tf.nnsoftplus,optimizer = tf.train.AdamOptimizer(),scale=0.1):
self.n_input = n_input
self.n_hidden = n_hidden
self.transfer = transfer_function
self.scale = tf.placeholder(tf.float32)
self.training_scale = scale
network_weights = self._initialize_weights()
self.weights = network_weights
接下来定义网络结构，我们为输入x创建一个维度是n_input的placeorder.然后建立一个能提取特征的隐含层，先输入x加入噪声，self.x + scale*tf.random_normal((n_input,))
然后tf.matmul将加了噪声的输入与隐含层权重相乘，并使用tf.add加上隐含层的偏执B1,最后使用self.transfer对结果进行激活函数处理。由于经过银行层之后我们需要输出层进行重建工作，，这里我们不使用激活函数，直接将隐含层的输出self.hidden乘输出层的权重w2再加上偏置b2即可。
self.x = tf.placeholder(tf.float32, [None, self.n_input])
self.hidden = self.transfer(tf.add(tf.matmul(self.x, self.weights['w1']), self.weights['b1']))
self.reconstruction = tf.add(tf.matmul(self.hidden, self.weights['w2']), self.weights['b2'])
# cost
self.cost = 0.5 * tf.reduce_sum(tf.pow(tf.subtract(self.reconstruction, self.x), 2.0))
self.optimizer = optimizer.minimize(self.cost)
init = tf.global_variables_initializer()
self.sess = tf.Session()
self.sess.run(init)
ef _initialize_weights(self):
all_weights = dict()
all_weights['w1'] = tf.get_variable("w1", shape=[self.n_input, self.n_hidden],
initializer=tf.contrib.layers.xavier_initializer())
all_weights['b1'] = tf.Variable(tf.zeros([self.n_hidden], dtype=tf.float32))
all_weights['w2'] = tf.Variable(tf.zeros([self.n_hidden, self.n_input], dtype=tf.float32))
all_weights['b2'] = tf.Variable(tf.zeros([self.n_input], dtype=tf.float32))
return all_weights接下来，使用一个batch数据进行训练并返回当前的cost：
def partial_fit(self, X):
cost, opt = self.sess.run((self.cost, self.optimizer), feed_dict={self.x: X})
return cost
计算损失cost的函数calc_total_cost，只计算损失不进行训练：
def calc_total_cost(self, X):
return self.sess.run(self.cost, feed_dict = {self.x: X})
还定义一个transform函数，返回自编码器隐含层的输出。目的是提供一个接口来获取抽象后的特征，隐含层最主要的作用就是学习数据中的高阶特征。
def transform(self, X):
return self.sess.run(self.hidden, feed_dict={self.x: X})
def generate(self, hidden = None):
if hidden is None:
hidden = self.sess.run(tf.random_normal([1, self.n_hidden]))
return self.sess.run(self.reconstruction, feed_dict={self.hidden: hidden})
def reconstruct(self, X):
return self.sess.run(self.reconstruction, feed_dict={self.x: X})def getWeights(self):
return self.sess.run(self.weights['w1'])
def getBiases(self):
return self.sess.run(self.weights['b1'])接下来就是使用这个自编码器，进行数据的训练，见文章开始。
Enjoy It！