tensorflow 模型保存与恢复

1. 模型保存

1.1生成文件

通过tf.train.Saver()来保存训练模型，保存后会生成四个文件：
1.checkpoint 文本文件，记录了模型文件的路径信息列表
2.data文件，记录了网络权重信息
3.index文件，记录了模型中变量参数信息
4.meta文件，记录了模型的网络结构信息

1.2 代码解释

saver=tf.save(sess, os.path.join('checkpoints/', 'CNN'),global_step = i)

其中global_step可以用来将当前迭代步数传进文件命中：

CNN-1000.index
CNN-1000.meta
CNN-1000.data-00000-of-00001
checkpoint

如果你想只保留最新的4个模型，并希望每2小时保存一次，可以使用max_to_keep和keep_checkpoint_every_n_hours：

#saves a model every 2 hours and maximum 4 latest models are saved.
saver = tf.train.Saver(max_to_keep=4, keep_checkpoint_every_n_hours=2)

2.模型恢复

2.1恢复网络结构，并计算准确率

1. 恢复相关变量参数：

#How to access saved variable/Tensor/placeholders 
w1 = graph.get_tensor_by_name("w1:0")

## How to access saved operation
op_to_restore = graph.get_tensor_by_name("op_to_restore:0")

2.恢复网络结构，计算准确率：

# access and create placeholders variables and create feed-dict to feed new data
with tf.Session() as sess:
  saver = tf.train.import_meta_graph(os.path.join(‘checkpoints/’, ‘CNN.meta’))
  saver.restore(sess, tf.train.latest_checkpoint(‘checkpoints/’))
  
  graph = tf.get_default_graph()
  x = graph.get_tensor_by_name('x:0')
  y_ = graph.get_tensor_by_name('y_:0')
  y_conv = graph.get_tensor_by_name('y_conv:0')
 
  correct_prediction = tf.equal(tf.argmax(y_conv,1),tf.argmax(y_,1))
  accuracy = tf.reduce_mean(tf.cast(correction, tf.float32))

  final_accuracy = sess.run(accuracy, feed_dict={x:mnist.test.images, y_:mnist.test.labels})
  print('accuracy is %g' %final_accuracy)

2.2 恢复网络结构，并增加增添层

1.初始结构：

import tensorflow as tf

#Prepare to feed input, i.e. feed_dict and placeholders
w1 = tf.placeholder("float", name="w1")
w2 = tf.placeholder("float", name="w2")
b1= tf.Variable(2.0,name="bias")
feed_dict ={w1:4,w2:8}

#Define a test operation that we will restore
w3 = tf.add(w1,w2)
w4 = tf.multiply(w3,b1,name="op_to_restore")
sess = tf.Session()
sess.run(tf.global_variables_initializer())

#Create a saver object which will save all the variables
saver = tf.train.Saver()

#Run the operation by feeding input
print sess.run(w4,feed_dict)
#Prints 24 which is sum of (w1+w2)*b1 

#Now, save the graph
saver.save(sess, 'my_test_model',global_step=1000)

2.增加层

with tf.Session() as sess:
        saver = tf.train.import_meta_graph(os.path.join(model_saving_path, model_name+'-1000.meta'))
        saver.restore(sess, tf.train.latest_checkpoint(model_saving_path))

        graph = tf.get_default_graph();
        w1 = graph.get_tensor_by_name('w1:0');
        w2 = graph.get_tensor_by_name('w2:0');
        feed_dict = {w1:13.0, w2:17.0};

        op_to_restore = graph.get_tensor_by_name('op_to_restore:0');
        # Add more to the current graph
        add_on_op = tf.multiply(op_to_restore, 2)
        print(sess.run(add_on_op, feed_dict))
        # This will print 120.

2.3 恢复模型中部分参数，并构建新的网络结构

只想恢复原来神经网络的一部分参数或者一部分算子，然后利用这一部分参数或者算子构建新的神经网络模型，怎么办？可以使用graph.get_tensor_by_name（）方法。下面是一个真实的例子。在这里，我们使用.meta加载VGG（预先训练好的网络），并做一些修改。

......
......
saver = tf.train.import_meta_graph('vgg.meta')
# Access the graph
graph = tf.get_default_graph()
## Prepare the feed_dict for feeding data for fine-tuning 

#Access the appropriate output for fine-tuning
fc7= graph.get_tensor_by_name('fc7:0')

#use this if you only want to change gradients of the last layer
fc7 = tf.stop_gradient(fc7) # It's an identity function
fc7_shape= fc7.get_shape().as_list()

new_outputs=2
weights = tf.Variable(tf.truncated_normal([fc7_shape[3], num_outputs], stddev=0.05))
biases = tf.Variable(tf.constant(0.05, shape=[num_outputs]))
output = tf.matmul(fc7, weights) + biases
pred = tf.nn.softmax(output)

# Now, you run this with fine-tuning data in sess.run()

参考：https://blog.csdn.net/liangyihuai/article/details/78515913

2.4 恢复之前保存的模型，并继续训练

1.原始模型

import numpy as np
import tensorflow as tf
x_data=np.random.rand(100).astype(np.float32)
 
y_data=x_data*0.1+0.3
weight=tf.Variable(tf.random_uniform([1],-1.0,1.0),name="w")
biases=tf.Variable(tf.zeros([1]),name="b")
y=weight*x_data+biases
 
loss=tf.reduce_mean(tf.square(y-y_data))
optimizer=tf.train.GradientDescentOptimizer(0.5)
train=optimizer.minimize(loss)
 
tf.add_to_collection("new_way",train)
init=tf.global_variables_initializer()
sess=tf.Session()
sess.run(init)
saver=tf.train.Saver(max_to_keep=0)
 
for step in range(10):
    sess.run(train)
    saver.save(sess,"./save_mode",global_step=step)
    print("当前进行：",step)

2.恢复训练

import numpy as np
import tensorflow as tf
sess=tf.Session()
saver=tf.train.import_meta_graph(r'save_mode-9.meta')
saver.restore(sess,tf.train.latest_checkpoint(r'./'))
print(sess.run("w:0"),sess.run("b:0"))
graph=tf.get_default_graph()
weight=graph.get_tensor_by_name("w:0")
biases=graph.get_tensor_by_name("b:0")
 
y=tf.get_collection("new_way")[0]
 
saver=tf.train.Saver(max_to_keep=0)
for step in range(10):
    sess.run(y)
    saver.save(sess,r"./save_new_mode",global_step=step)
    print("当前进行：",step," ",sess.run(weight),sess.run(biases))

原文：https://blog.csdn.net/by_side_with_sun/article/details/79829619