[tf]完成一个简单的模型

• 当Session.run 时，将通过feed_dict 的字典提供一个mini-batch 的样本数据集，从而自动推导出tf.placeholder 的大小

x = tf.placeholder(tf.float32, [None, 28, 28, 1])
t = tf.placeholder(tf.float32, [None, 10])

• 定义变量

w = tf.Variable(tf.zeros([784, 10]))
w1 = tf.Variable(tf.truncated_normal([L, M], stddev=0.1))
b = tf.Variable(tf.zeros([10]))

• 定义init_op

init_op = tf.global_variables_initializer()

• 得到准确率

is_correct = tf.equal(tf.argmax(y,1), tf.argmax(t,1))
accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))

• 定义优化器和训练op

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.003)
train_step = optimizer.minimize(cross_entropy)

• 训练

with tf.Session() as sess:
    for step in range(1000):
        batch_xs, batch_ys = mnist.train.next_batch(100)
        sess.run(train_step, feed_dict={x: batch_xs, t: batch_ys})
        if step % 100 == 0:
            acc, loss = sess.run([accuracy, cross_entropy],
            feed_dict={x: batch_xs, t: batch_ys})
            acc, loss = sess.run([accuracy, cross_entropy],
            feed_dict={x: mnist.test.images, t: mnist.test.labels})

• tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=pred, name=None)，计算最后一层是softmax层的cross entropy，把softmax计算与cross entropy计算放到一起了，用一个函数来实现，用来提高程序的运行速度。
• 使得学习率指数衰减

lr = tf.placeholder(tf.float32)
train_step = tf.train.AdamOptimizer(lr).minimize(cross_entropy)

def lr(step):
    max_lr, min_lr, decay_speed = 0.003, 0.0001, 2000.0
    return min_lr + (max_lr - min_lr) * math.exp(-step/decay_speed)

with tf.Session() as sess:
    for step in range(10000):
        batch_xs, batch_ys = mnist.train.next_batch(100)
        sess.run(train_step,
        feed_dict={x: batch_xs, t: batch_ys, pkeep: 0.75, lr: lr(step)})

• 设置dropout，要注意，在训练的时候设置超参数pkeep的值小于1，而在推理的时候，设置超参数pkeep的值为1。
• 彻底明白了而，placeholder只是一个占位符，可以输入相应类型的数字，这个占位符是在图运行的时候需要定制数据的。

pkeep = tf.placeholder(tf.float32)
y1 = tf.nn.relu(tf.matmul(x, w1) + b1)
y1d = tf.nn.dropout(y1, pkeep)

with tf.Session() as sess:
    for step in range(10000):
        batch_xs, batch_ys = mnist.train.next_batch(100)
        sess.run(train_step,
            feed_dict={x: batch_xs, t: batch_ys, pkeep: 0.75, lr: lr(step)})
    if step % 100 == 0:
        acc, loss = sess.run([accuracy, cross_entropy],
            feed_dict={x: batch_xs, t: batch_ys, pkeep: 1})
        acc, loss = sess.run([accuracy, cross_entropy],
            feed_dict={x: mnist.test.images, t: mnist.test.labels, pkeep: 1})