如何将Mnist数据分类器的准确率提高到98%以上?
技巧:
- 网络的层数以及每一层神经元的个数
- 优化器的选择:Adam,SGD, Adagrad, RMSprop, Adadelta
- 学习率的更新:随着迭代次的增加,指数下降
- 学习轮数的设定
程序:
mnist = input_data.read_data_sets("C:/Users/WangT/Desktop/MNIST_data",one_hot=True)
batch_size = 100
n_batch = mnist.train.num_examples // batch_size
x = tf.placeholder(tf.float32,[None,784])
y = tf.placeholder(tf.float32,[None,10])
keep_prob = tf.placeholder(tf.float32)
lr = tf.Variable(0.001, dtype = tf.float32)
W1 = tf.Variable(tf.truncated_normal([784,500],stddev=0.1))
b1 = tf.Variable(tf.zeros([500])+0.1)
L1 = tf.nn.tanh(tf.matmul(x,W1)+b1)
W2 = tf.Variable(tf.truncated_normal([500,300],stddev=0.1))
b2 = tf.Variable(tf.zeros([300])+0.1)
L2 = tf.nn.tanh(tf.matmul(L1,W2)+b2)
W3= tf.Variable(tf.truncated_normal([300,10],stddev=0.1))
b3 = tf.Variable(tf.zeros([10])+0.1)
prediction = tf.nn.softmax(tf.matmul(L2,W3)+b3)
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))
train_step = tf.train.AdamOptimizer(lr).minimize(loss)
init = tf.global_variables_initializer()
correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(prediction,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))
with tf.Session() as sess:
sess.run(init)
for epoch in range (51):
for batch in range(n_batch):
batch_xs,batch_ys = mnist.train.next_batch(batch_size)
sess.run(tf.assign(lr,0.001*(0.95**epoch)))
sess.run(train_step, feed_dict={x:batch_xs,y:batch_ys})
acc1 = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
acc2 = sess.run(accuracy,feed_dict={x:mnist.train.images,y:mnist.train.labels})
print("Iter"+str(epoch)+",Testing Accuracy"+str(acc1)+",Training accuracy"+str(acc2))
Extracting C:/Users/WangT/Desktop/MNIST_data\train-images-idx3-ubyte.gz
Extracting C:/Users/WangT/Desktop/MNIST_data\train-labels-idx1-ubyte.gz
Extracting C:/Users/WangT/Desktop/MNIST_data\t10k-images-idx3-ubyte.gz
Extracting C:/Users/WangT/Desktop/MNIST_data\t10k-labels-idx1-ubyte.gz
Iter0,Testing Accuracy0.9499,Training accuracy0.955691
Iter1,Testing Accuracy0.9627,Training accuracy0.970036
Iter2,Testing Accuracy0.9661,Training accuracy0.977127
Iter3,Testing Accuracy0.9705,Training accuracy0.981327
Iter4,Testing Accuracy0.9759,Training accuracy0.986345
Iter5,Testing Accuracy0.9755,Training accuracy0.988509
Iter6,Testing Accuracy0.9749,Training accuracy0.989545
Iter7,Testing Accuracy0.9783,Training accuracy0.991491
Iter8,Testing Accuracy0.9771,Training accuracy0.992618
Iter9,Testing Accuracy0.9793,Training accuracy0.992564
Iter10,Testing Accuracy0.9789,Training accuracy0.993691
Iter11,Testing Accuracy0.9796,Training accuracy0.994327
Iter12,Testing Accuracy0.9779,Training accuracy0.994382
Iter13,Testing Accuracy0.9788,Training accuracy0.995036
Iter14,Testing Accuracy0.9795,Training accuracy0.995073
Iter15,Testing Accuracy0.9801,Training accuracy0.995491
Iter16,Testing Accuracy0.9806,Training accuracy0.995727
新内容:
1.lr = tf.Variable(0.001, dtype = tf.float32)
2.sess.run(tf.assign(lr,0.001*(0.95**epoch)))
每批次迭代,学习率以初始值为0.001,指数下降,利用tf.assign()的方法赋值更新