版权声明：本套技术专栏是作者（秦凯新）平时工作的总结和升华，通过从真实商业环境抽取案例进行总结和分享，并给出商业应用的调优建议和集群环境容量规划等内容，请持续关注本套博客。QQ邮箱地址：[email protected]，如有任何学术交流，可随时联系。
1 神经网络基本结构定义
- 28*28=784个像素点，第一层神经元256，第二层神经元128
2 基本神经网络构建
-
变量初始化
import numpy as np import tensorflow as tf import matplotlib.pyplot as plt import input_data mnist = input_data.read_data_sets('data/', one_hot=True) Extracting data/train-images-idx3-ubyte.gz Extracting data/train-labels-idx1-ubyte.gz Extracting data/t10k-images-idx3-ubyte.gz Extracting data/t10k-labels-idx1-ubyte.gz # NETWORK TOPOLOGIES #第一层神经元 n_hidden_1 = 256 #第二层神经元 n_hidden_2 = 128 #28*28 784像素点 n_input = 784 # 类别10 n_classes = 10 # INPUTS AND OUTPUTS x = tf.placeholder("float", [None, n_input]) y = tf.placeholder("float", [None, n_classes]) # NETWORK PARAMETERS stddev = 0.1 #初始化 weights = { 'w1': tf.Variable(tf.random_normal([n_input, n_hidden_1], stddev=stddev)), 'w2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2], stddev=stddev)), 'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes], stddev=stddev)) } #初始化 biases = { 'b1': tf.Variable(tf.random_normal([n_hidden_1])), 'b2': tf.Variable(tf.random_normal([n_hidden_2])), 'out': tf.Variable(tf.random_normal([n_classes])) } print ("NETWORK READY") 复制代码 -
前向传播（每一层增加激活函数sigmoid，最后一层不加sigmoid）
def multilayer_perceptron(_X, _weights, _biases): layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(_X, _weights['w1']), _biases['b1'])) layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, _weights['w2']), _biases['b2'])) return (tf.matmul(layer_2, _weights['out']) + _biases['out']) 复制代码 -
损失变量和优化器定义
-
softmax_cross_entropy_with_logits交叉熵损失函数（参数pred预测值），reduce_mean除以样本总数。
-
GradientDescentOptimizer采用梯度下降优化求解
# PREDICTION pred = multilayer_perceptron(x, weights, biases) # LOSS AND OPTIMIZER cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) optm = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(cost) #准确率求解 corr = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) accr = tf.reduce_mean(tf.cast(corr, "float")) # INITIALIZER init = tf.global_variables_initializer() print ("FUNCTIONS READY") 复制代码 -
按照Batch迭代
training_epochs = 20 batch_size = 100 display_step = 4 # LAUNCH THE GRAPH sess = tf.Session() sess.run(init) # OPTIMIZE for epoch in range(training_epochs): avg_cost = 0. total_batch = int(mnist.train.num_examples/batch_size) # ITERATION（按照Batch迭代，每一次迭代100） for i in range(total_batch): batch_xs, batch_ys = mnist.train.next_batch(batch_size) #填充值 feeds = {x: batch_xs, y: batch_ys} #sess.run（模型训练） sess.run(optm, feed_dict=feeds) avg_cost += sess.run(cost, feed_dict=feeds) avg_cost = avg_cost / total_batch # DISPLAY if (epoch+1) % display_step == 0: print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost)) feeds = {x: batch_xs, y: batch_ys} #sess.run（准确率求解） train_acc = sess.run(accr, feed_dict=feeds) print ("TRAIN ACCURACY: %.3f" % (train_acc)) feeds = {x: mnist.test.images, y: mnist.test.labels} test_acc = sess.run(accr, feed_dict=feeds) print ("TEST ACCURACY: %.3f" % (test_acc)) print ("OPTIMIZATION FINISHED") 复制代码
3 CNN神经网络
-
变量初始化
import numpy as np import tensorflow as tf import matplotlib.pyplot as plt import input_data mnist = input_data.read_data_sets('data/', one_hot=True) trainimg = mnist.train.images trainlabel = mnist.train.labels testimg = mnist.test.images testlabel = mnist.test.labels print ("MNIST ready") n_input = 784 n_output = 10 ##wc1 [3, 3, 1, 64] 中3表示Filter宽度和深度，1表示深度，64表示outchannl最后得到64张特征图。 14*14*128 ##wc2 [3, 3, 64, 128] 中3表示Filter宽度和深度，1表示深度，64表示输入64张特征图，输出128张特征图。7*7*128 输出1024向量 ## 卷积层没有减少挺像的大小。 ## polling层把图像减少到一半 ## wd1 输入7*7*128 输出1024向量 weights = { 'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1)), 'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.1)), 'wd1': tf.Variable(tf.random_normal([7*7*128, 1024], stddev=0.1)), 'wd2': tf.Variable(tf.random_normal([1024, n_output], stddev=0.1)) } biases = { 'bc1': tf.Variable(tf.random_normal([64], stddev=0.1)), 'bc2': tf.Variable(tf.random_normal([128], stddev=0.1)), 'bd1': tf.Variable(tf.random_normal([1024], stddev=0.1)), 'bd2': tf.Variable(tf.random_normal([n_output], stddev=0.1)) } 复制代码 -
help方法的使用
-
前向传播
def conv_basic(_input, _w, _b, _keepratio): # INPUT(转换格式，转换成4维 【n,h,w,c】 -1 batchSize大小，可以让TF推断 ，输出通道深度为1) _input_r = tf.reshape(_input, shape=[-1, 28, 28, 1]) # 第一层（nn模块CNN, RNN）（conv2 中 strides ->【n,h,w,c】表示在各个上面滑动窗的大小 # padding 两种选择 SAME=>滑动窗不够时填充，Valid不填充)。 _conv1 = tf.nn.conv2d(_input_r, _w['wc1'], strides=[1, 1, 1, 1], padding='SAME') #_mean, _var = tf.nn.moments(_conv1, [0, 1, 2]) #_conv1 = tf.nn.batch_normalization(_conv1, _mean, _var, 0, 1, 0.0001) # 激活函数relu _conv1 = tf.nn.relu(tf.nn.bias_add(_conv1, _b['bc1'])) # max_pool层，ksize表示Window -1 batchSize大小，2*2窗口 1表示，输出通道深度为1 _pool1 = tf.nn.max_pool(_conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME') # dropout不让所有的神经元参与计算比例 _pool_dr1 = tf.nn.dropout(_pool1, _keepratio) # 第二层 _conv2 = tf.nn.conv2d(_pool_dr1, _w['wc2'], strides=[1, 1, 1, 1], padding='SAME') #_mean, _var = tf.nn.moments(_conv2, [0, 1, 2]) #_conv2 = tf.nn.batch_normalization(_conv2, _mean, _var, 0, 1, 0.0001) _conv2 = tf.nn.relu(tf.nn.bias_add(_conv2, _b['bc2'])) _pool2 = tf.nn.max_pool(_conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME') _pool_dr2 = tf.nn.dropout(_pool2, _keepratio) #全连接层 # VECTORIZE _dense1 = tf.reshape(_pool_dr2, [-1, _w['wd1'].get_shape().as_list()[0]]) # FULLY CONNECTED LAYER 1 _fc1 = tf.nn.relu(tf.add(tf.matmul(_dense1, _w['wd1']), _b['bd1'])) _fc_dr1 = tf.nn.dropout(_fc1, _keepratio) # FULLY CONNECTED LAYER 2 _out = tf.add(tf.matmul(_fc_dr1, _w['wd2']), _b['bd2']) # RETURN out = { 'input_r': _input_r, 'conv1': _conv1, 'pool1': _pool1, 'pool1_dr1': _pool_dr1, 'conv2': _conv2, 'pool2': _pool2, 'pool_dr2': _pool_dr2, 'dense1': _dense1, 'fc1': _fc1, 'fc_dr1': _fc_dr1, 'out': _out } return out print ("CNN READY") 复制代码 -
模型训练和评估
a = tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1)) print (a) a = tf.Print(a, [a], "a: ") init = tf.global_variables_initializer() sess = tf.Session() sess.run(init) Tensor("Variable_28/read:0", shape=(3, 3, 1, 64), dtype=float32) #print (help(tf.nn.conv2d)) print (help(tf.nn.max_pool)) x = tf.placeholder(tf.float32, [None, n_input]) y = tf.placeholder(tf.float32, [None, n_output]) keepratio = tf.placeholder(tf.float32) # FUNCTIONS _pred = conv_basic(x, weights, biases, keepratio)['out'] cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(_pred, y)) optm = tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost) _corr = tf.equal(tf.argmax(_pred,1), tf.argmax(y,1)) accr = tf.reduce_mean(tf.cast(_corr, tf.float32)) init = tf.global_variables_initializer() # SAVER print ("GRAPH READY") sess = tf.Session() sess.run(init) training_epochs = 15 batch_size = 16 display_step = 1 for epoch in range(training_epochs): avg_cost = 0. #total_batch = int(mnist.train.num_examples/batch_size) total_batch = 10 # Loop over all batches for i in range(total_batch): batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Fit training using batch data sess.run(optm, feed_dict={x: batch_xs, y: batch_ys, keepratio:0.7}) # Compute average loss avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.})/total_batch # Display logs per epoch step if epoch % display_step == 0: print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost)) train_acc = sess.run(accr, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.}) print (" Training accuracy: %.3f" % (train_acc)) #test_acc = sess.run(accr, feed_dict={x: testimg, y: testlabel, keepratio:1.}) #print (" Test accuracy: %.3f" % (test_acc)) print ("OPTIMIZATION FINISHED") 复制代码 -
结果展示
Epoch: 000/015 cost: 30.928401661 Training accuracy: 0.500 Epoch: 001/015 cost: 12.954609606 Training accuracy: 0.700 Epoch: 002/015 cost: 10.392489696 Training accuracy: 0.700 Epoch: 003/015 cost: 7.254891634 Training accuracy: 0.800 Epoch: 004/015 cost: 4.977767670 Training accuracy: 0.900 Epoch: 005/015 cost: 5.414173813 Training accuracy: 0.600 Epoch: 006/015 cost: 3.057567777 Training accuracy: 0.700 Epoch: 007/015 cost: 4.929724103 Training accuracy: 0.600 Epoch: 008/015 cost: 3.192437538 Training accuracy: 0.600 Epoch: 009/015 cost: 3.224479928 Training accuracy: 0.800 Epoch: 010/015 cost: 2.720530389 Training accuracy: 0.400 Epoch: 011/015 cost: 3.000342276 Training accuracy: 0.800 Epoch: 012/015 cost: 0.639763238 Training accuracy: 1.000 Epoch: 013/015 cost: 1.897303332 Training accuracy: 0.900 Epoch: 014/015 cost: 2.295500937 Training accuracy: 0.800 OPTIMIZATION FINISHED 复制代码
4 模型持久化与加载
import tensorflow as tf
v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(init_op)
print ("V1:",sess.run(v1))
print ("V2:",sess.run(v2))
saver_path = saver.save(sess, "save/model.ckpt")
print ("Model saved in file: ", saver_path)
V1: [[-0.61912751 0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
[ 0.6146487 0.66980863 -1.00977123]]
Model saved in file: save/model.ckpt
import tensorflow as tf
v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
saver = tf.train.Saver()
with tf.Session() as sess:
saver.restore(sess, "save/model.ckpt")
print ("V1:",sess.run(v1))
print ("V2:",sess.run(v2))
print ("Model restored")
V1: [[-0.61912751 0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
[ 0.6146487 0.66980863 -1.00977123]]
Model restored
复制代码总结
基本的神经网络案例，在于真正的入门神经网络的构建。
版权声明：本套技术专栏是作者（秦凯新）平时工作的总结和升华，通过从真实商业环境抽取案例进行总结和分享，并给出商业应用的调优建议和集群环境容量规划等内容，请持续关注本套博客。QQ邮箱地址：[email protected]，如有任何学术交流，可随时联 秦凯新 于深圳 2018120892153