TensorFlow 预训练模型导入

TensorFlow 预训练模型导入

模型下载地址

Pre-trained Models和Index of /models/tf/给出了大部分较新的、性能较好的网络预训练模型，包括VGG、ResNet、各个版本的Inception等。
要测试AlexNet模型，可以到AlexNet implementation + weights in TensorFlow下载相应的.npy文件，并按照提供的.py文件重建网络，注意该网络需要特征分组训练，并且输入像素值范围为 [0,255] [ 0 , 255 ] 。根据提供者的描述，该模型实际上由caffemodel转化而来。

【注】tensorflow是不能直接导入caffemodel的，需要将caffemodel转成tensorflow可以读取的格式，具体参见GitHub项目caffe-tensorflow。

方法

对于不同后缀名的模型，需要不同的加载方式

1. .npy文件、.npz文件、.mat文件
   这些文件保存的仅仅是网络的权重，因此为了重建网络，需要根据网络结构一步步搭建，将相应的权重赋予其中的 W,b W , b
   如AlexNet

   alex_path = '...\\bvlc_alexnet.npy'
   alex_data = np.load(open(alex_path, "rb"), encoding="latin1").item()
   tf.reset_default_graph()
   x = tf.placeholder(tf.float32,shape = [None,227,227,3],name = 'im')
   mean = tf.constant([[[[ 123.68 ,  116.779,  103.939]]]],name = 'im_mean') #[0,255]
   x = tf.subtract(x,mean)
   x = tf.reverse(x,axis = [3]) # RGB to BGR
   with tf.variable_scope('conv1'):
   w0,b0 = alex_data['conv1']
   w = tf.get_variable('w',initializer = tf.constant(w0)) # use tensorflow variable rather than constant
   b = tf.get_variable('b',initializer = tf.constant(b0))
   conv = tf.nn.conv2d(x,w,[1,4,4,1],'SAME')
   conv = tf.add(conv,b)
   conv = tf.nn.relu(conv)
   conv = tf.nn.local_response_normalization(conv,depth_radius = 2,alpha = 2e-5,beta = 0.75,bias = 1.0)
   pass
   with tf.variable_scope('pool1'):
   pool = tf.nn.max_pool(conv,[1,3,3,1],[1,2,2,1],'VALID')
   pass
   with tf.variable_scope('conv2'):
   w0,b0 = alex_data['conv2']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   group = 2
   inputs = tf.split(pool,group,3)
   kernels = tf.split(w,group,3)
   
   # cannot use tf.constant(kernel0) because kernel0 is a tensorflow Tensor yet
   
   outputs = [tf.nn.conv2d(input0,kernel0,[1,1,1,1],'SAME') for input0,kernel0 in zip(inputs,kernels)]
   output = tf.concat(outputs,3)
   conv = tf.add(output,b)
   conv = tf.nn.relu(conv)
   conv = tf.nn.local_response_normalization(conv,depth_radius = 2,alpha = 2e-5,beta = 0.75,bias = 1.0)
   pass
   with tf.variable_scope('pool2'):
   pool = tf.nn.max_pool(conv,[1,3,3,1],[1,2,2,1],'VALID')
   pass
   with tf.variable_scope('conv3'):
   w0,b0 = alex_data['conv3']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   conv = tf.nn.conv2d(pool,w,[1,1,1,1],'SAME')
   conv = tf.add(conv,b)
   conv = tf.nn.relu(conv)
   pass
   with tf.variable_scope('conv4'):
   w0,b0 = alex_data['conv4']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   group = 2
   inputs = tf.split(conv,group,3)
   kernels = tf.split(w,group,3)
   outputs = [tf.nn.conv2d(input0,kernel0,[1,1,1,1],'SAME') for input0,kernel0 in zip(inputs,kernels)]
   output = tf.concat(outputs,3)
   conv = tf.add(output,b)
   conv = tf.nn.relu(conv)
   pass
   with tf.variable_scope('conv5'):
   w0,b0 = alex_data['conv5']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   group = 2
   inputs = tf.split(conv,group,3)
   kernels = tf.split(w,group,3)
   outputs = [tf.nn.conv2d(input0,kernel0,[1,1,1,1],'SAME') for input0,kernel0 in zip(inputs,kernels)]
   output = tf.concat(outputs,3)
   conv = tf.add(output,b)
   conv = tf.nn.relu(conv)
   pass
   with tf.variable_scope('pool5'):
   pool = tf.nn.max_pool(conv,[1,3,3,1],[1,2,2,1],'VALID')
   pass
   with tf.variable_scope('fc6'):
   w0,b0 = alex_data['fc6']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   pool = tf.layers.flatten(pool) # flatten the convolutional feature for dense connection
   fc = tf.matmul(pool,w)
   fc = tf.add(fc,b)
   fc = tf.nn.relu(fc)
   pass
   with tf.variable_scope('fc7'):
   w0,b0 = alex_data['fc7']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   fc = tf.matmul(fc,w)
   fc = tf.add(fc,b)
   fc = tf.nn.relu(fc)
   pass
   with tf.variable_scope('fc8'):
   w0,b0 = alex_data['fc8']
   w = tf.get_variable('w',initializer = tf.constant(w0))
   b = tf.get_variable('b',initializer = tf.constant(b0))
   fc = tf.matmul(fc,w)
   fc = tf.add(fc,b)
   pass
   with tf.variable_scope('prob'):
   prob = tf.nn.softmax(fc)
   pass
   y = tf.argmax(prob,axis = 1)

2. .pb文件
   .pb文件保存了网络的结构和权重，因此可以直接导入，不必自己重建网络结构
   如ResNet152

model_fn = r'...\resnet_v1_152.pb'
with tf.gfile.FastGFile(model_fn, 'rb') as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
    pass
g = tf.Graph()
sess = tf.InteractiveSession(graph = g)
x = tf.placeholder(tf.float32,shape = (None,224,224,3),name = 'im')
mean = tf.constant([[[[ 123.68 ,  116.779,  103.939]]]],name = 'im_mean') # [0,255]
x1 = tf.subtract(x,mean)
tf.import_graph_def(graph_def,{'InputImage':x1}) # match the input of VGG19 to the placeholder 'x1'
layer = 'import/resnet_v1_152/predictions/Softmax:0'
y = tf.argmax(g.get_tensor_by_name(layer),axis = 1) # get output

相比于前面的代码，这里的可简洁得多了。
3. .ckpt文件
.ckpt文件的导入尝试了多次没能成功，后来看了问题only publish pre-trained model’s checkpoint files are useless. #6954 才大概明白
.ckpt files are useless on their own, but are intended to be used by a script that builds a model, sets up saver, and loads them.
pre-trained checkpoint files are useless by themselves, you need to use the code that created the model that generated it
I'm guessing those .ckpt files are read by some .py script that sets up the model, and loads the checkpoint. They are not meant to be used directly.
因此，光有.ckpt文件是不够的，需要tensorflow/models中配套的.py文件导入。如VGG19

import sys
nets_path = r'...\nets'
sys.path.insert(0,nets_path)
from vgg import vgg_19

ckpt_path = r'...\vgg_19_2016_08_28\vgg_19.ckpt'

tf.reset_default_graph()
x = tf.placeholder(tf.float32,shape = [None,224,224,3],name = 'im')
mean = tf.constant([[[[ 123.68 ,  116.779,  103.939]]]],name = 'im_mean',dtype = tf.float32) # [0,255]
x = tf.subtract(x,mean)
net,endpoints = vgg_19(x)
saver = tf.train.Saver()
sess = tf.InteractiveSession()

saver.restore(sess,ckpt_path)

# layer = 'vgg_19/fc8/squeezed:0'#'vgg_19/fc8/BiasAdd:0'
# fc8 = sess.graph.get_tensor_by_name(layer)
prob = tf.nn.softmax(net,axis = 1)
y = tf.argmax(prob,axis = 1)

但是在这样导入NasNet时，却报错了，不知道为什么…