tensorflow入门(二)
TensorFlow实现卷积神经网络
学习Tensorflow实战个人笔记,共大家学习交流,欢迎拍砖
作者: 谢中朝
1. TensorFlow 实现简单的卷积神经网络
'''
载入MNIST数据集,并创建默认的Interactive Session
'''
from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
mnist = input_data.read_data_sets("MNIST_data/", one_hot = True)
sess = tf.InteractiveSession()Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
'''
创建卷及神经网络的权重和偏置
(1)我们需要给权重制造随机噪声来打破完全对称,比如截断的正态分布噪声,标准差设为0.1
(2)使用ReLU给偏置增加一些小的正值(0.1)用来避免死亡节点
'''
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape = shape)
return tf.Variable(initial)'''
创建卷积层和池化层
(1)参数x为输入,W为卷积参数,[5,5,1,32]:前面两个数字代表卷积尺寸,第三个数字代表有多少个channel,灰度单色为1,彩色RGB图片为3,
最后一个数字代表卷积数量.
(2)Strides代表模板卷积模板移动的步长,都是1代表会不遗漏地划过图片的每一个点.
(3)Padding代表边界的处理方式,这里的SAME代表给边界加上Padding让卷积的输出和输入保持同样的(SAME)的尺寸.
'''
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')'''
定义placeholder,x是特征,y_是真实的label
(1)卷积神经网络会利用空间信息,因此需要将1D的输入向量转化为2D图片结构,1*784转化原始的28*28,只有一个颜色通道,
最终尺寸[-1,28,28,1],前面的-1代表样本数量不固定,最后1代表颜色通道数量.
'''
x = tf.placeholder(tf.float32, [None, 784])
y_ = tf.placeholder(tf.float32, [None, 10])
x_image = tf.reshape(x, [-1,28,28,1])'''
定义第一个卷积层:
(1)参数初始化,包括weights和bias,[5,5,1,32]代表卷积核尺寸为5*5,1个颜色通道,32个不同卷积核.
(2)使用conv2d进行卷积,再加上偏置.
(3)接着使用ReLU激活函数进行非线性处理.
(4)最后用最大池化函数max_pool_2x2对卷积的结果进行池化.
'''
W_conv1 = weight_variable([5,5,1,32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)'''
定义第二个卷积层:
(1)参数初始化,包括weights和bias,[5,5,1,64]代表卷积核尺寸为5*5,1个颜色通道,64个不同卷积核.
(2)使用conv2d进行卷积,再加上偏置.
(3)接着使用ReLU激活函数进行非线性处理.
(4)最后用最大池化函数max_pool_2x2对卷积的结果进行池化.
'''
'''
经历两次步长为2*2的最大池化,图片尺寸28*28变成7*7,由于第二层卷积核数量为64,输出尺寸即为7*7*64
'''
W_conv2 = weight_variable([5,5,32,64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
print (h_pool2.shape)(?, 7, 7, 64)
'''
定义一个全连接层:
(1)将第二层卷积的输出tensor进行变形,将其转化为1D向量,然后连接一个全连接层,隐含节点为1024
(2)并使用ReLU激活函数
'''
W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
print (h_pool2_flat.shape)
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)
print (h_fc1.shape)(?, 3136)
(?, 1024)
'''
定义一个Dropout层:
(1)为了减轻过拟合,使用Dropout,通过一个placeholder传入keep_prob比率进行控制.
(2)在训练过程中,我们随机丢弃一部分节点的数据来减轻过拟合.
(3)预测时则保留全部的数据来追求最好的预测性能.
'''
keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
print h_fc1_drop.shape(?, 1024)
'''
最后将Dropout层的输出连接一个Softmax层,得到最后的概率输出
'''
W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
print y_conv.shape(?, 10)
'''
定义损失函数cross entropy, 优化器使用Adam, 并给予一个比较小的学习速率1e-4
'''
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)'''
定义评定准确率的操作:
'''
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))'''
开始训练过程:
(1)首先初始化所有参数
(2)设置训练时的Dropout的keep_prob比率为0.5
(3)然后使用大小为50的mini-batch,进行20000次的训练迭代,参与训练样本的数量总共为100万
(4)其中每训练100次,对准确率进行一次评测(评测时keep_prob设为1),用于实时监控模型性能
'''
tf.global_variables_initializer().run()
for i in range(20000):
batch = mnist.train.next_batch(50)#label:batch[1]代表50行*10列; input:batch[0]代表1024行*50列
if i%100 == 0:
train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_:batch[1], keep_prob: 1.0})
print ("step %d , training accuracy %g"%(i, train_accuracy))
train_step.run(feed_dict={x: batch[0], y_:batch[1], keep_prob: 0.5})step 0 , training accuracy 0.06
step 100 , training accuracy 0.8
step 200 , training accuracy 0.86
step 300 , training accuracy 0.92
step 400 , training accuracy 0.94
step 500 , training accuracy 0.9
step 600 , training accuracy 0.98
step 700 , training accuracy 0.92
step 800 , training accuracy 0.96
step 900 , training accuracy 0.98
step 1000 , training accuracy 0.96
step 1100 , training accuracy 0.94
step 1200 , training accuracy 0.98
step 1300 , training accuracy 1
step 1400 , training accuracy 0.96
step 1500 , training accuracy 0.98
step 1600 , training accuracy 0.98
step 1700 , training accuracy 1
step 1800 , training accuracy 0.94
step 1900 , training accuracy 0.98
step 2000 , training accuracy 1
step 2100 , training accuracy 1
step 2200 , training accuracy 1
step 2300 , training accuracy 0.98
step 2400 , training accuracy 0.96
step 2500 , training accuracy 1
step 2600 , training accuracy 0.94
step 2700 , training accuracy 0.98
step 2800 , training accuracy 1
step 2900 , training accuracy 0.94
step 3000 , training accuracy 0.96
step 3100 , training accuracy 0.96
step 3200 , training accuracy 1
step 3300 , training accuracy 0.98
step 3400 , training accuracy 1
step 3500 , training accuracy 0.96
step 3600 , training accuracy 1
step 3700 , training accuracy 1
step 3800 , training accuracy 1
step 3900 , training accuracy 0.98
step 4000 , training accuracy 1
step 4100 , training accuracy 1
step 4200 , training accuracy 0.96
step 4300 , training accuracy 0.98
step 4400 , training accuracy 0.98
step 4500 , training accuracy 0.96
step 4600 , training accuracy 1
step 4700 , training accuracy 1
step 4800 , training accuracy 1
step 4900 , training accuracy 1
step 5000 , training accuracy 0.98
step 5100 , training accuracy 1
step 5200 , training accuracy 1
step 5300 , training accuracy 1
step 5400 , training accuracy 1
step 5500 , training accuracy 0.98
step 5600 , training accuracy 1
step 5700 , training accuracy 1
step 5800 , training accuracy 0.98
step 5900 , training accuracy 1
step 6000 , training accuracy 1
step 6100 , training accuracy 1
step 6200 , training accuracy 1
step 6300 , training accuracy 1
step 6400 , training accuracy 1
step 6500 , training accuracy 0.98
step 6600 , training accuracy 1
step 6700 , training accuracy 0.98
step 6800 , training accuracy 1
step 6900 , training accuracy 1
step 7000 , training accuracy 1
step 7100 , training accuracy 0.98
step 7200 , training accuracy 0.98
step 7300 , training accuracy 1
step 7400 , training accuracy 1
step 7500 , training accuracy 1
step 7600 , training accuracy 1
step 7700 , training accuracy 1
step 7800 , training accuracy 0.98
step 7900 , training accuracy 0.98
step 8000 , training accuracy 1
step 8100 , training accuracy 0.96
step 8200 , training accuracy 1
step 8300 , training accuracy 0.98
step 8400 , training accuracy 1
step 8500 , training accuracy 1
step 8600 , training accuracy 1
step 8700 , training accuracy 1
step 8800 , training accuracy 1
step 8900 , training accuracy 1
step 9000 , training accuracy 1
step 9100 , training accuracy 1
step 9200 , training accuracy 0.98
step 9300 , training accuracy 0.98
step 9400 , training accuracy 1
step 9500 , training accuracy 0.98
step 9600 , training accuracy 0.98
step 9700 , training accuracy 0.98
step 9800 , training accuracy 1
step 9900 , training accuracy 1
step 10000 , training accuracy 1
step 10100 , training accuracy 1
step 10200 , training accuracy 1
step 10300 , training accuracy 0.98
step 10400 , training accuracy 1
step 10500 , training accuracy 1
step 10600 , training accuracy 1
step 10700 , training accuracy 1
step 10800 , training accuracy 1
step 10900 , training accuracy 0.98
step 11000 , training accuracy 1
step 11100 , training accuracy 1
step 11200 , training accuracy 1
step 11300 , training accuracy 1
step 11400 , training accuracy 1
step 11500 , training accuracy 1
step 11600 , training accuracy 1
step 11700 , training accuracy 1
step 11800 , training accuracy 1
step 11900 , training accuracy 1
step 12000 , training accuracy 1
step 12100 , training accuracy 1
step 12200 , training accuracy 1
step 12300 , training accuracy 1
step 12400 , training accuracy 1
step 12500 , training accuracy 1
step 12600 , training accuracy 1
step 12700 , training accuracy 1
step 12800 , training accuracy 1
step 12900 , training accuracy 1
step 13000 , training accuracy 1
step 13100 , training accuracy 1
step 13200 , training accuracy 1
step 13300 , training accuracy 1
step 13400 , training accuracy 1
step 13500 , training accuracy 1
step 13600 , training accuracy 1
step 13700 , training accuracy 1
step 13800 , training accuracy 1
step 13900 , training accuracy 0.96
step 14000 , training accuracy 1
step 14100 , training accuracy 1
step 14200 , training accuracy 1
step 14300 , training accuracy 1
step 14400 , training accuracy 1
step 14500 , training accuracy 1
step 14600 , training accuracy 1
step 14700 , training accuracy 1
step 14800 , training accuracy 1
step 14900 , training accuracy 1
step 15000 , training accuracy 1
step 15100 , training accuracy 1
step 15200 , training accuracy 1
step 15300 , training accuracy 1
step 15400 , training accuracy 1
step 15500 , training accuracy 1
step 15600 , training accuracy 0.98
step 15700 , training accuracy 1
step 15800 , training accuracy 1
step 15900 , training accuracy 1
step 16000 , training accuracy 1
step 16100 , training accuracy 1
step 16200 , training accuracy 1
step 16300 , training accuracy 1
step 16400 , training accuracy 0.98
step 16500 , training accuracy 1
step 16600 , training accuracy 1
step 16700 , training accuracy 1
step 16800 , training accuracy 1
step 16900 , training accuracy 1
step 17000 , training accuracy 1
step 17100 , training accuracy 1
step 17200 , training accuracy 1
step 17300 , training accuracy 1
step 17400 , training accuracy 1
step 17500 , training accuracy 1
step 17600 , training accuracy 1
step 17700 , training accuracy 1
step 17800 , training accuracy 1
step 17900 , training accuracy 1
step 18000 , training accuracy 1
step 18100 , training accuracy 1
step 18200 , training accuracy 1
step 18300 , training accuracy 1
step 18400 , training accuracy 1
step 18500 , training accuracy 1
step 18600 , training accuracy 1
step 18700 , training accuracy 1
step 18800 , training accuracy 1
step 18900 , training accuracy 1
step 19000 , training accuracy 1
step 19100 , training accuracy 1
step 19200 , training accuracy 1
step 19300 , training accuracy 1
step 19400 , training accuracy 1
step 19500 , training accuracy 1
step 19600 , training accuracy 1
step 19700 , training accuracy 1
step 19800 , training accuracy 1
step 19900 , training accuracy 1
'''
在最终的数据集上进行全面测试,得到整体的分类的准确率
'''
print("test_accuracy %g"%accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))test_accuracy 0.9923
2. TensorFlow 实现进阶的卷积神经网络
cd /home/xzz/code/python_tensorflow/models-master/tutorials/image/cifar10//home/xzz/code/python_tensorflow/models-master/tutorials/image/cifar10
#载入常用的库
import cifar10
import cifar10_input
import tensorflow as tf
import numpy as np
import time'''
定义batch_size, 训练轮数max_steps,以及下载CIFAR-10数据的默认路径
'''
max_step = 100000
batch_size = 128
data_dir = '/home/xzz/download/cifar-10-batches-bin/''''
(1)定义初始化weight的函数,使用tf.truncated_normal截断的正态分布来初始化权重,给weight加一个L2的loss,进行L2正则化处理.
(2)在机器学习中,不管是分类任务还是回归任务,都可能因特征过多而导致过拟合,一般可以通过减少特征或者惩罚不重要特征的权重来缓解这一问题.
(3)正则化就是帮组惩罚特征权重的,特征权重也会成为模型的损失函数的一部分.一般来说,L1正则会制造稀疏的特征,大部分无用的特征权重会被置为0,
而L2正则会让特征的权重不过大,使得特征的权重比较平均.
(4)我们使用w1控制L2 loss的大小, 使用tf.nn.l2_loss函数计算weight的L2 loss,再使用tf.multipy让L2 loss乘以w1得到weight loss.接着使用
tf.add_to_collection把weight loss统一存在一个collection,这个colloction称为'losses',在计算神经网络的总体loss时被用上.
'''
def variable_with_weight_loss(shape, stddev, wl):
var = tf.Variable(tf.truncated_normal(shape, stddev = stddev))
if wl is not None:
weight_loss = tf.multiply(tf.nn.l2_loss(var), wl, name = 'weight_loss')
tf.add_to_collection('losses', weight_loss)
return var #使用cifar10类下载数据集,并解压,展开到默认位置
cifar10.maybe_download_and_extract()'''
(1)使用cifar10_input类中的distorted_inputs函数产生训练需要使用的数据,包括特征及其对应的label.
(2)对图像的进行数据增强的操作耗费大量的CPU时间,因此disorted_inputs使用了16个独立的线程来加速任务,函数内部会产生线程池,
在需要使用的时候会通过TensorFlow queue进行调度
'''
images_train, labels_train = cifar10_input.distorted_inputs(data_dir = data_dir, batch_size = batch_size)Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.
'''
cifar10_input.inputs函数生成测试数据,不需要太多的处理,将图片裁剪为24*24大小的区块,进行数据的标准化处理
'''
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)'''
(1)创建输入数据placeholder,包括特征和label
(2)数据尺寸中的图片尺寸为24*24,即是裁剪后的大小,颜色通道数则设为3,代表图片是彩色有RGB三条通道
'''
image_holder = tf.placeholder(tf.float32, [batch_size, 24, 24, 3])
label_holder = tf.placeholder(tf.int32, [batch_size])'''
(1)开始创建第一个卷积层
(2)先对variable_with_weight_loss函数创建卷积核参数初始化,第一个卷积层使用5*5卷积核大小,3个颜色通道,64个卷积核
(3)不对第一个卷积层的weight进行L2正则,因此w1(weight loss)这项设为0
(4)使用tf.nn.conv2d函数对输入数据image_holder进行卷积操作,步长为1,padding模式为SAME,将这层bias全部初始化为0,
再将卷积结果加上bias,最后使用ReLU激活函数进行非线性化
(5)在激活函数后,使用3*3且步长为2*2最大池化层处理数据,这里最大池化尺寸和步长不一致可以增加数据的丰富性
(6)最后进行LRN,可以使其中响应比较大的值变得更大,并抑制其他反馈较小的神经元,增强模型的泛化能力,适合没有上界的激活函数
'''
weight1 = variable_with_weight_loss(shape=[5,5,3,64], stddev=5e-2, wl=0.0)
kernel1 = tf.nn.conv2d(image_holder, weight1, [1,1,1,1], padding='SAME')
print 'kernel', kernel1
bias1 = tf.Variable(tf.constant(0.0, shape=[64]))
print 'bias1', bias1
conv1 = tf.nn.relu(tf.nn.bias_add(kernel1, bias1))
print 'conv', conv1
pool1 = tf.nn.max_pool(conv1, ksize=[1,3,3,1], strides=[1,2,2,1],padding='SAME')
print 'pool', pool1
norm1 = tf.nn.lrn(pool1, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)
print 'norm', norm1kernel Tensor("Conv2D:0", shape=(128, 24, 24, 64), dtype=float32)
bias1 Tensor("Variable_1/read:0", shape=(64,), dtype=float32)
conv Tensor("Relu_2:0", shape=(128, 24, 24, 64), dtype=float32)
pool Tensor("MaxPool:0", shape=(128, 12, 12, 64), dtype=float32)
norm Tensor("LRN:0", shape=(128, 12, 12, 64), dtype=float32)
'''
(1)开始创建第二个卷积层
(2)64个通道,64个卷积核,注意bias全部初始化为0.1
(3)注意该层先进行LRN层处理,再使用最大池化层
'''
weight2 = variable_with_weight_loss(shape=[5,5,64,64], stddev=5e-2, wl=0.0)
kernel2 = tf.nn.conv2d(norm1, weight2, [1,1,1,1], padding='SAME')
bias2 = tf.Variable(tf.constant(0.0, shape=[64]))
conv2 = tf.nn.relu(tf.nn.bias_add(kernel2, bias2))
norm2 = tf.nn.lrn(conv2, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)
pool2 = tf.nn.max_pool(norm2, ksize=[1,3,3,1], strides=[1,2,2,1],padding='SAME')
print 'pool', pool2
print batch_sizepool Tensor("MaxPool_1:0", shape=(128, 6, 6, 64), dtype=float32)
128
'''
(1)使用全连接层,将两个卷积层的输入结果全部flatten,使用tf.shape函数把每一个样本都变成一维向量,get_shape可以获得长度
(2)variable_with_weight_loss函数对全连接层的weight进行初始化,隐含节点数384,正态分布标准差设为0.04,bias设置为0.1
(3)为了这个全连接层不要过拟合,设置了非零weight loss即w1 = 0.004,让这一层所有参数都被L2正则所约束,最后ReLU激活函数进行非线性化
'''
reshape = tf.reshape(pool2, [batch_size, -1])
print 'reshape', reshape.shape
dim = reshape.get_shape()[1].value
print reshape.get_shape()[0]
print reshape.get_shape()[1]
print 'dim', dim
weight3 = variable_with_weight_loss(shape=[dim, 384], stddev = 0.04, wl = 0.004)
print 'weight3', weight3
bias3 = tf.Variable(tf.constant(0.1, shape=[384]))
print 'bias3', bias3
local3 = tf.nn.relu(tf.matmul(reshape, weight3) + bias3)
print 'local3', local3reshape (128, 2304)
128
2304
dim 2304
weight3 Tensor("Variable_4/read:0", shape=(2304, 384), dtype=float32)
bias3 Tensor("Variable_5/read:0", shape=(384,), dtype=float32)
local3 Tensor("Relu_4:0", shape=(128, 384), dtype=float32)
'''
与前一个全连接层一样,就是隐藏节点下降一半,只有192个,其他的超参数保持不变
'''
weight4 = variable_with_weight_loss(shape=[384, 192], stddev = 0.04, wl = 0.004)
bias4 = tf.Variable(tf.constant(0.1, shape=[192]))
local4 = tf.nn.relu(tf.matmul(local3, weight4) + bias4)
print 'local4', local4local4 Tensor("Relu_5:0", shape=(128, 192), dtype=float32)
'''
最后一层,先创建这一层的weight,其正态分布的标准差设为上一个隐藏层的节点数的倒数,并且不计入L2的正则
'''
weight5 = variable_with_weight_loss(shape=[192, 10], stddev = 1/192.0, wl = 0.0)
print 'weight5',weight5
bias5 = tf.Variable(tf.constant(0.0, shape=[10]))
logits = tf.add(tf.matmul(local4, weight5), bias5)
print 'logits', logits.shapeweight5 Tensor("Variable_8/read:0", shape=(192, 10), dtype=float32)
logits (128, 10)
'''
为了完成模型inference部分构建,接下来计算CNN的loss,将全部的loss整合到一起,
其中包括cross entropy loss,还有后两个全连接层中的weight的L2 loss
'''
def loss(logits, labels):
labels = tf.cast(labels, tf.int64)
cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
logits = logits, labels = labels, name = 'cross_entropy_per_example')
cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')
tf.add_to_collection('losses', cross_entropy_mean)
return tf.add_n(tf.get_collection('losses'), name = 'total_loss' )#将logits节点和label_placeholder传入loss函数获得最终的loss
loss = loss(logits, label_holder)
#优化器Adam Optimizer, 学习速率设置为1e-3
train_op = tf.train.AdamOptimizer(1e-3).minimize(loss)
#使用tf.nn.in_top_k函数求出输出结果中top k的准确率,默认使用top 1,输出得分最高的那一类准确率
top_k_op = tf.nn.in_top_k(logits, label_holder, 1)
#tf.InteractiveSession创建默认的session,接着初始化全部模型参数
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()'''
启动图片数据增强线程队列,一共有16个线程来进行加速
'''
tf.train.start_queue_runners()[,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
]
'''
现在正式开始训练.
(1)在每一个step训练过程中,我们需要使用session的run方法执行images_train, labels_train
(2)获得一个batch的训练数据,在将这个batch的数据传入train_op和loss的计算
(3)我们记录,每一个step花费的时间,每隔10个step会计算和展示当前的loss,每秒训练的样本数量,
以及训练一个batch数据所花费的时间
'''
for step in range(max_step):
start_time = time.time()
image_batch, label_batch = sess.run([images_train, labels_train])
_, loss_value = sess.run([train_op, loss],
feed_dict={image_holder: image_batch, label_holder: label_batch})
duration = time.time() - start_time
if step%10 == 0 :
examples_per_sec = batch_size / duration
sec_per_batch = float(duration)
format_str=('step %d, loss=%.2f (%.1f examples/sec; %.3f sec/batch)' )
print(format_str % (step, loss_value, examples_per_sec, sec_per_batch))step 0, loss=4.68 (170.5 examples/sec; 0.751 sec/batch)
step 10, loss=3.66 (1896.5 examples/sec; 0.067 sec/batch)
step 20, loss=3.11 (1757.2 examples/sec; 0.073 sec/batch)
step 30, loss=2.75 (1859.3 examples/sec; 0.069 sec/batch)
step 40, loss=2.42 (1860.1 examples/sec; 0.069 sec/batch)
step 50, loss=2.25 (1932.3 examples/sec; 0.066 sec/batch)
step 60, loss=2.21 (1881.6 examples/sec; 0.068 sec/batch)
step 70, loss=2.02 (1929.8 examples/sec; 0.066 sec/batch)
step 80, loss=1.96 (1898.5 examples/sec; 0.067 sec/batch)
step 90, loss=2.01 (1902.7 examples/sec; 0.067 sec/batch)
step 100, loss=1.95 (1187.7 examples/sec; 0.108 sec/batch)
step 110, loss=2.03 (1928.6 examples/sec; 0.066 sec/batch)
step 120, loss=1.87 (1935.3 examples/sec; 0.066 sec/batch)
step 130, loss=1.88 (1802.0 examples/sec; 0.071 sec/batch)
step 140, loss=1.81 (1964.1 examples/sec; 0.065 sec/batch)
step 150, loss=1.86 (1899.4 examples/sec; 0.067 sec/batch)
step 160, loss=1.83 (1890.8 examples/sec; 0.068 sec/batch)
step 170, loss=1.99 (1129.3 examples/sec; 0.113 sec/batch)
step 180, loss=1.82 (1953.1 examples/sec; 0.066 sec/batch)
step 190, loss=1.75 (1931.8 examples/sec; 0.066 sec/batch)
step 200, loss=1.79 (1813.3 examples/sec; 0.071 sec/batch)
step 210, loss=1.55 (1550.7 examples/sec; 0.083 sec/batch)
step 220, loss=1.64 (1886.9 examples/sec; 0.068 sec/batch)
step 230, loss=1.71 (1889.5 examples/sec; 0.068 sec/batch)
step 240, loss=1.57 (1797.4 examples/sec; 0.071 sec/batch)
step 250, loss=1.78 (1909.6 examples/sec; 0.067 sec/batch)
step 260, loss=1.76 (1901.4 examples/sec; 0.067 sec/batch)
step 270, loss=1.73 (1283.8 examples/sec; 0.100 sec/batch)
step 280, loss=1.57 (1898.1 examples/sec; 0.067 sec/batch)
step 290, loss=1.53 (1974.6 examples/sec; 0.065 sec/batch)
step 300, loss=1.67 (2031.1 examples/sec; 0.063 sec/batch)
step 310, loss=1.69 (1917.7 examples/sec; 0.067 sec/batch)
step 320, loss=1.81 (1857.9 examples/sec; 0.069 sec/batch)
step 330, loss=1.52 (1961.6 examples/sec; 0.065 sec/batch)
step 340, loss=1.55 (1857.5 examples/sec; 0.069 sec/batch)
step 350, loss=1.63 (1730.8 examples/sec; 0.074 sec/batch)
step 360, loss=1.75 (1848.8 examples/sec; 0.069 sec/batch)
step 370, loss=1.59 (1582.6 examples/sec; 0.081 sec/batch)
step 380, loss=1.67 (1652.3 examples/sec; 0.077 sec/batch)
step 390, loss=1.64 (926.2 examples/sec; 0.138 sec/batch)
step 400, loss=1.62 (1666.3 examples/sec; 0.077 sec/batch)
step 410, loss=1.34 (1907.2 examples/sec; 0.067 sec/batch)
step 420, loss=1.40 (1929.3 examples/sec; 0.066 sec/batch)
step 430, loss=1.38 (1904.4 examples/sec; 0.067 sec/batch)
step 440, loss=1.57 (1899.8 examples/sec; 0.067 sec/batch)
step 450, loss=1.61 (1939.0 examples/sec; 0.066 sec/batch)
step 460, loss=1.42 (1908.8 examples/sec; 0.067 sec/batch)
step 470, loss=1.48 (1887.1 examples/sec; 0.068 sec/batch)
step 480, loss=1.44 (1865.2 examples/sec; 0.069 sec/batch)
step 490, loss=1.47 (1965.5 examples/sec; 0.065 sec/batch)
step 500, loss=1.62 (1778.1 examples/sec; 0.072 sec/batch)
step 510, loss=1.49 (1951.4 examples/sec; 0.066 sec/batch)
step 520, loss=1.41 (1848.1 examples/sec; 0.069 sec/batch)
step 530, loss=1.29 (1861.3 examples/sec; 0.069 sec/batch)
step 540, loss=1.49 (1864.5 examples/sec; 0.069 sec/batch)
step 550, loss=1.50 (1925.5 examples/sec; 0.066 sec/batch)
step 560, loss=1.59 (1965.4 examples/sec; 0.065 sec/batch)
step 570, loss=1.53 (1990.2 examples/sec; 0.064 sec/batch)
step 580, loss=1.39 (1908.6 examples/sec; 0.067 sec/batch)
step 590, loss=1.56 (1961.8 examples/sec; 0.065 sec/batch)
step 600, loss=1.53 (1902.4 examples/sec; 0.067 sec/batch)
step 610, loss=1.36 (1986.2 examples/sec; 0.064 sec/batch)
step 620, loss=1.41 (1975.8 examples/sec; 0.065 sec/batch)
step 630, loss=1.37 (1359.0 examples/sec; 0.094 sec/batch)
step 640, loss=1.42 (2006.3 examples/sec; 0.064 sec/batch)
step 650, loss=1.32 (1926.8 examples/sec; 0.066 sec/batch)
step 660, loss=1.38 (1856.8 examples/sec; 0.069 sec/batch)
step 670, loss=1.31 (1615.1 examples/sec; 0.079 sec/batch)
step 680, loss=1.42 (1897.6 examples/sec; 0.067 sec/batch)
step 690, loss=1.31 (1951.4 examples/sec; 0.066 sec/batch)
step 700, loss=1.45 (1058.2 examples/sec; 0.121 sec/batch)
step 710, loss=1.33 (1863.9 examples/sec; 0.069 sec/batch)
step 720, loss=1.41 (1924.8 examples/sec; 0.067 sec/batch)
step 730, loss=1.39 (1917.7 examples/sec; 0.067 sec/batch)
step 740, loss=1.60 (1858.7 examples/sec; 0.069 sec/batch)
step 750, loss=1.45 (1933.1 examples/sec; 0.066 sec/batch)
step 760, loss=1.30 (1843.2 examples/sec; 0.069 sec/batch)
step 770, loss=1.29 (1922.2 examples/sec; 0.067 sec/batch)
step 780, loss=1.39 (1924.9 examples/sec; 0.066 sec/batch)
step 790, loss=1.45 (1907.3 examples/sec; 0.067 sec/batch)
step 800, loss=1.30 (1913.0 examples/sec; 0.067 sec/batch)
step 810, loss=1.36 (1934.2 examples/sec; 0.066 sec/batch)
step 820, loss=1.34 (1886.5 examples/sec; 0.068 sec/batch)
step 830, loss=1.37 (1841.6 examples/sec; 0.070 sec/batch)
step 840, loss=1.35 (1912.6 examples/sec; 0.067 sec/batch)
step 850, loss=1.24 (1909.8 examples/sec; 0.067 sec/batch)
step 860, loss=1.41 (1917.0 examples/sec; 0.067 sec/batch)
step 870, loss=1.26 (1821.0 examples/sec; 0.070 sec/batch)
step 880, loss=1.29 (1823.2 examples/sec; 0.070 sec/batch)
step 890, loss=1.31 (1913.2 examples/sec; 0.067 sec/batch)
step 900, loss=1.44 (2066.2 examples/sec; 0.062 sec/batch)
step 910, loss=1.47 (1844.2 examples/sec; 0.069 sec/batch)
step 920, loss=1.38 (1914.7 examples/sec; 0.067 sec/batch)
step 930, loss=1.18 (1884.0 examples/sec; 0.068 sec/batch)
step 940, loss=1.33 (1877.6 examples/sec; 0.068 sec/batch)
step 950, loss=1.19 (1826.2 examples/sec; 0.070 sec/batch)
step 960, loss=1.38 (1940.5 examples/sec; 0.066 sec/batch)
step 970, loss=1.37 (1947.3 examples/sec; 0.066 sec/batch)
step 980, loss=1.26 (1921.8 examples/sec; 0.067 sec/batch)
step 990, loss=1.29 (1844.1 examples/sec; 0.069 sec/batch)
step 1000, loss=1.19 (2098.3 examples/sec; 0.061 sec/batch)
step 1010, loss=1.19 (1868.5 examples/sec; 0.069 sec/batch)
step 1020, loss=1.29 (1836.6 examples/sec; 0.070 sec/batch)
step 1030, loss=1.37 (1896.0 examples/sec; 0.068 sec/batch)
step 1040, loss=1.23 (1870.3 examples/sec; 0.068 sec/batch)
step 1050, loss=1.27 (1877.9 examples/sec; 0.068 sec/batch)
step 1060, loss=1.39 (1963.9 examples/sec; 0.065 sec/batch)
step 1070, loss=1.28 (1979.4 examples/sec; 0.065 sec/batch)
step 1080, loss=1.21 (1917.5 examples/sec; 0.067 sec/batch)
step 1090, loss=1.40 (1896.0 examples/sec; 0.068 sec/batch)
step 1100, loss=1.15 (1893.4 examples/sec; 0.068 sec/batch)
step 1110, loss=1.11 (1841.6 examples/sec; 0.070 sec/batch)
step 1120, loss=1.23 (1855.3 examples/sec; 0.069 sec/batch)
step 1130, loss=1.31 (1797.0 examples/sec; 0.071 sec/batch)
step 1140, loss=1.28 (1819.6 examples/sec; 0.070 sec/batch)
step 1150, loss=1.26 (1958.6 examples/sec; 0.065 sec/batch)
step 1160, loss=1.23 (1896.6 examples/sec; 0.067 sec/batch)
step 1170, loss=1.34 (1933.1 examples/sec; 0.066 sec/batch)
step 1180, loss=1.24 (1869.0 examples/sec; 0.068 sec/batch)
step 1190, loss=1.19 (1889.1 examples/sec; 0.068 sec/batch)
step 1200, loss=1.37 (1826.7 examples/sec; 0.070 sec/batch)
step 1210, loss=1.40 (1950.8 examples/sec; 0.066 sec/batch)
step 1220, loss=1.26 (1833.5 examples/sec; 0.070 sec/batch)
step 1230, loss=1.23 (1888.9 examples/sec; 0.068 sec/batch)
step 1240, loss=1.17 (1877.0 examples/sec; 0.068 sec/batch)
step 1250, loss=1.41 (1960.0 examples/sec; 0.065 sec/batch)
step 1260, loss=1.29 (1854.7 examples/sec; 0.069 sec/batch)
step 1270, loss=1.13 (2023.2 examples/sec; 0.063 sec/batch)
step 1280, loss=1.34 (1929.8 examples/sec; 0.066 sec/batch)
step 1290, loss=1.20 (1873.5 examples/sec; 0.068 sec/batch)
step 1300, loss=1.24 (1951.3 examples/sec; 0.066 sec/batch)
step 1310, loss=1.15 (1840.3 examples/sec; 0.070 sec/batch)
step 1320, loss=1.06 (1911.2 examples/sec; 0.067 sec/batch)
step 1330, loss=1.26 (1949.0 examples/sec; 0.066 sec/batch)
step 1340, loss=1.24 (1896.5 examples/sec; 0.067 sec/batch)
step 1350, loss=1.17 (1912.7 examples/sec; 0.067 sec/batch)
step 1360, loss=1.08 (1925.7 examples/sec; 0.066 sec/batch)
step 1370, loss=1.49 (1997.7 examples/sec; 0.064 sec/batch)
step 1380, loss=1.32 (1969.7 examples/sec; 0.065 sec/batch)
step 1390, loss=1.29 (1819.8 examples/sec; 0.070 sec/batch)
step 1400, loss=1.34 (2006.5 examples/sec; 0.064 sec/batch)
step 1410, loss=1.12 (1844.8 examples/sec; 0.069 sec/batch)
step 1420, loss=1.20 (1859.7 examples/sec; 0.069 sec/batch)
step 1430, loss=1.07 (1901.2 examples/sec; 0.067 sec/batch)
step 1440, loss=1.13 (1832.0 examples/sec; 0.070 sec/batch)
step 1450, loss=1.05 (1869.1 examples/sec; 0.068 sec/batch)
step 1460, loss=1.25 (1917.7 examples/sec; 0.067 sec/batch)
step 1470, loss=1.27 (1792.5 examples/sec; 0.071 sec/batch)
step 1480, loss=1.18 (1874.7 examples/sec; 0.068 sec/batch)
step 1490, loss=1.20 (1927.0 examples/sec; 0.066 sec/batch)
step 1500, loss=1.09 (1921.9 examples/sec; 0.067 sec/batch)
step 1510, loss=1.20 (1896.2 examples/sec; 0.068 sec/batch)
step 1520, loss=1.30 (1800.1 examples/sec; 0.071 sec/batch)
step 1530, loss=1.21 (1829.0 examples/sec; 0.070 sec/batch)
step 1540, loss=1.35 (1821.9 examples/sec; 0.070 sec/batch)
step 1550, loss=1.20 (1929.8 examples/sec; 0.066 sec/batch)
step 1560, loss=1.18 (1940.8 examples/sec; 0.066 sec/batch)
step 1570, loss=1.19 (1820.8 examples/sec; 0.070 sec/batch)
step 1580, loss=1.01 (1954.3 examples/sec; 0.065 sec/batch)
step 1590, loss=1.09 (1890.5 examples/sec; 0.068 sec/batch)
step 1600, loss=1.14 (1988.3 examples/sec; 0.064 sec/batch)
step 1610, loss=1.00 (1906.1 examples/sec; 0.067 sec/batch)
step 1620, loss=1.07 (1938.0 examples/sec; 0.066 sec/batch)
step 1630, loss=1.17 (1926.3 examples/sec; 0.066 sec/batch)
step 1640, loss=1.16 (1881.1 examples/sec; 0.068 sec/batch)
step 1650, loss=1.28 (1899.0 examples/sec; 0.067 sec/batch)
step 1660, loss=1.11 (1944.9 examples/sec; 0.066 sec/batch)
step 1670, loss=1.15 (1948.2 examples/sec; 0.066 sec/batch)
step 1680, loss=1.27 (1842.7 examples/sec; 0.069 sec/batch)
step 1690, loss=1.22 (1273.3 examples/sec; 0.101 sec/batch)
step 1700, loss=1.29 (1370.6 examples/sec; 0.093 sec/batch)
…………
…………
…………
…………
step 99850, loss=0.60 (1834.3 examples/sec; 0.070 sec/batch)
step 99860, loss=0.58 (1834.3 examples/sec; 0.070 sec/batch)
step 99870, loss=0.50 (2053.8 examples/sec; 0.062 sec/batch)
step 99880, loss=0.73 (1902.0 examples/sec; 0.067 sec/batch)
step 99890, loss=0.65 (1929.2 examples/sec; 0.066 sec/batch)
step 99900, loss=0.70 (1886.3 examples/sec; 0.068 sec/batch)
step 99910, loss=0.55 (1914.5 examples/sec; 0.067 sec/batch)
step 99920, loss=0.77 (1941.3 examples/sec; 0.066 sec/batch)
step 99930, loss=0.54 (1929.5 examples/sec; 0.066 sec/batch)
step 99940, loss=0.61 (1895.4 examples/sec; 0.068 sec/batch)
step 99950, loss=0.50 (1949.0 examples/sec; 0.066 sec/batch)
step 99960, loss=0.54 (1972.2 examples/sec; 0.065 sec/batch)
step 99970, loss=0.60 (1885.1 examples/sec; 0.068 sec/batch)
step 99980, loss=0.58 (1921.5 examples/sec; 0.067 sec/batch)
step 99990, loss=0.50 (1979.7 examples/sec; 0.065 sec/batch)
'''
评测模型在测试集上的准确率,测试集10000个样本
(1)使用固定的batch_size,然后一个batch一个batch输入测试数据
(2)计算一共多少个batch才能将全部样本测试完毕
(3)每一个step使用session的run方法获取images_test, labels_test的batch
(4)在执行top_k_top计算模型在这个batch的top 1上预测正确的样本数,最后汇总所有正确的预测结果
'''
num_examples = 10000
import math
num_iter = int(math.ceil(num_examples / batch_size))
true_count = 0
total_sample_count = num_iter * batch_size
step = 0
while step < num_iter :
image_batch, label_batch = sess.run([images_test, labels_test])
predictions = sess.run([top_k_op], feed_dict={image_holder: image_batch, label_holder: label_batch})
true_count += np.sum(predictions)
step += 1
print true_count
print total_sample_count8051
9984
'''
最后将准确率的评测结果计算并打印出来
'''
precision = float(true_count) / float(total_sample_count)
print true_count
print total_sample_count
print ("分类正确率是: %.3f" % precision)8051
9984
分类正确率是: 0.806