【Tool】Tensorflow移动端模型转换

Tags: Tool DeepLearning

---

Screen Shot 2018-09-17 at 2.31.54 PM.png

深度学习和机器学习的移动端化是未来的趋势，这两年各个大厂也在这方面发力，竞相推出自己移动端的推理框架。Google有Tensorflow Lite, Apple有CoreML，Facebook有Caffe2, Tencent的NCNN最近也是风头正盛，百度有Paddle Mobile， 小米有MACE... 总之各个平台之间不仅性能有差异，光是要把训练好的模型正确的转换过去，就得吐一口血。
扯远了，这篇文章主要记录自己在使用Tensorflow+Keras训练模型，同时将模型转换到移动端的一些经验，会持续更新, 现有的移动端框架主要是Tensorflow Lite和CoreML，未来考虑加入Caffe2和NCNN。

Tensorflow 转 Tensorflow Lite

在Android端Tensorflow Lite 会使用 Android Neural Networks API进行加速，但是需要Android 8.1以上才支持：

Screen Shot 2018-09-17 at 2.39.49 PM.png

Tensorflow提供官方转换工具toco可以直接将tensorflow .pb模型转换为.tflite模型。

Screen Shot 2018-09-17 at 3.33.43 PM.png

toco

usage: toco [-h] --output_file OUTPUT_FILE
            (--graph_def_file GRAPH_DEF_FILE | --saved_model_dir SAVED_MODEL_DIR | --keras_model_file KERAS_MODEL_FILE)
            [--output_format {TFLITE,GRAPHVIZ_DOT}]
            [--inference_type {FLOAT,QUANTIZED_UINT8}]
            [--inference_input_type {FLOAT,QUANTIZED_UINT8}]
            [--input_arrays INPUT_ARRAYS] [--input_shapes INPUT_SHAPES]
            [--output_arrays OUTPUT_ARRAYS]
            [--saved_model_tag_set SAVED_MODEL_TAG_SET]
            [--saved_model_signature_key SAVED_MODEL_SIGNATURE_KEY]
            [--std_dev_values STD_DEV_VALUES] [--mean_values MEAN_VALUES]
            [--default_ranges_min DEFAULT_RANGES_MIN]
            [--default_ranges_max DEFAULT_RANGES_MAX]
            [--quantize_weights QUANTIZE_WEIGHTS] [--drop_control_dependency]
            [--reorder_across_fake_quant] [--change_concat_input_ranges]
            [--allow_custom_ops] [--dump_graphviz_dir DUMP_GRAPHVIZ_DIR]
            [--dump_graphviz_video]

• output_file: 输出模型.tflite后缀
• graph_def_file: 静态图文件, saved_model_dir: 模型保存路径, keras_model_file: keras模型文件, 三者必须有且只能有一个
• output_format: 输出文件类型
• inference_type: 这个参数可以用来对模型进行量化，从而达到不同的accuracy/speed的平衡
• inference_input_type: 输入类型，在使用QUANTIZED_UINT8的时候也需要配置 --std_dev_values --mean_values, --default_ranges_min --default_ranges_max这些参数
  我自己试验了下使用inference_type和inference_input_type使用QUANTIZED_UINT8比float要快很多（有时间给一个例子）
• input_shape: 输出图片的大小
• input_array: 图输入节点
• output_array: 图输出节点
• allow_custom_ops: 定义了官方不支持的layer或者op的时候需要开启
  其他参数可以在用到的时候在官网参考。
  从toco的使用提示可以看到，我们需要提供的参数有，静态图.pb文件，生成.tflite文件，输出的格式，输入的节点shape, 输入的节点名称，输出节点名称。
  成功实例参考：

toco --graph_def_file=DeeplabV3++_portrait_384_1_05alpha.pb --output_file=DeeplabV3++_portrait_384_1_05alpha.tflite --output_format=TFLITE --input_shape=1,384,384,3 --input_array=input_1 --output_array=output_0 --inference_type=float --allow_custom_ops

下面介绍下如何查看tensorflow模型中节点名称

• 如何查看checkpoints中节点名称

saver = tf.train.import_meta_graph(/path/to/meta/graph)
sess = tf.Session()
saver.restore(sess, /path/to/checkpoints)
graph = sess.graph
print([node.name for node in graph.as_graph_def().node])

• 如何查看静态图.pb节点信息:
  以我自己的simplenet 静态图文件为例:

"""FIND GRAPH INFO"""
tf_model_path = "./simplenet_V2_8M.pb"
with open(tf_model_path , 'rb') as f:
    serialized = f.read()
tf.reset_default_graph()
original_gdef = tf.GraphDef()
original_gdef.ParseFromString(serialized)

with tf.Graph().as_default() as g:
    tf.import_graph_def(original_gdef, name ='')
    ops = g.get_operations()
    N = len(ops)
    for i in [0,1,2,N-3,N-2,N-1]: # for循环设置输出的节点信息
        print('\n\nop id {} : op type: "{}"'.format(str(i), ops[i].type))
        print('input(s):')
        for x in ops[i].inputs:
            print("name = {}, shape: {}, ".format(x.name, x.get_shape()))
        print('\noutput(s):'),
        for x in ops[i].outputs:
            print("name = {}, shape: {},".format(x.name, x.get_shape()))

输出如下:

op id 0 : op type: "Placeholder"
input(s):

output(s):
name = input_1:0, shape: (?, 32, 32, 3),


op id 1 : op type: "Const"
input(s):

output(s):
name = block1_conv/kernel:0, shape: (3, 3, 3, 128),


op id 2 : op type: "Identity"
input(s):
name = block1_conv/kernel:0, shape: (3, 3, 3, 128), 

output(s):
name = block1_conv/kernel/read:0, shape: (3, 3, 3, 128),


op id 190 : op type: "MatMul"
input(s):
name = global_average_pooling2d_1/Mean:0, shape: (?, 600), 
name = dense_1/kernel/read:0, shape: (600, 10), 

output(s):
name = dense_1/MatMul:0, shape: (?, 10),


op id 191 : op type: "BiasAdd"
input(s):
name = dense_1/MatMul:0, shape: (?, 10), 
name = dense_1/bias/read:0, shape: (10,), 

output(s):
name = dense_1/BiasAdd:0, shape: (?, 10),


op id 192 : op type: "Softmax"
input(s):
name = dense_1/BiasAdd:0, shape: (?, 10), 

output(s):
name = activation_1/Softmax:0, shape: (?, 10),

得到这些信息之后我就可以给toco提供合适的信息进行转换:

toco --graph_def_file=simplenet_V2_8M.pb --output_file=simplenet_v2_8M.tflite --output_format=TFLITE --input_shape=1,32,32,3 --input_arrays=input_1 --output_arrays=activation_1/Softmax

转换成功之后会生成.tflite文件，就可以用于移动端部署了。

tflite

实际上在直接用于移动端部署之前，你可能需要测试下你的tflite模型的准确度，这就需要直接在Python里面调用tflite:

import numpy as np
import tensorflow as tf

# Load TFLite model and allocate tensors.
interpreter = tf.contrib.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Test model on random input data.
input_shape = input_details[0]['shape']
# change the following line to feed into your own data.
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)

参考资料: https://stackoverflow.com/questions/50764572/how-can-i-test-a-tflite-model-to-prove-that-it-behaves-as-the-original-model-us

Tensorlfow 转 CoreML

TF-COREML

CoreML并不提供将tensorflow模型直接转换为mlmodel的工具，但是有keras接口。此外有第三方转换工具tf-coreml可以用来转换，这里看下如何使用这个工具转换。

首先将我们保存的checkpoints保存为静态图,.pb格式。

然后要输入的参数有：

• tf_model_path: .pb静态图模型路径
• mlmodel_path: 生成的CoreML模型路径地址
• input_name_shape_dict: 网络的输入名称和数据的大小(要根据原始的模型输入确定)
• output_feature_names: 网络输出的名称(要根据原始的模型输出确定)

此外也可以对输入的数据做一些归一化处理:

• image_scale:
• red_bias
• green_bias
• blue_bias

import tensorflow as tf
import tfcoreml
from coremltools.proto import FeatureTypes_pb2 as _FeatureTypes_pb2
import coremltools

""" FIND GRAPH INFO """
tf_model_path = "/tmp//retrained_graph.pb"
with open(tf_model_path , 'rb') as f:
    serialized = f.read()
tf.reset_default_graph()
original_gdef = tf.GraphDef()
original_gdef.ParseFromString(serialized)

with tf.Graph().as_default() as g:
    tf.import_graph_def(original_gdef, name ='')
    ops = g.get_operations()
    N = len(ops)
    for i in [0,1,2,N-3,N-2,N-1]:
        print('\n\nop id {} : op type: "{}"'.format(str(i), ops[i].type))
        print('input(s):')
        for x in ops[i].inputs:
            print("name = {}, shape: {}, ".format(x.name, x.get_shape()))
        print('\noutput(s):'),
        for x in ops[i].outputs:
            print("name = {}, shape: {},".format(x.name, x.get_shape()))

""" CONVERT TF TO CORE ML """
# Model Shape
input_tensor_shapes = {"input:0":[1,224,224,3]} 
# Input Name
image_input_name = ['input:0']
# Output CoreML model path
coreml_model_file = '/tmp/myModel.mlmodel'
# Output name
output_tensor_names = ['final_result:0']
# Label file for classification
class_labels = '/tmp/retrained_labels.txt'

#Convert Process
coreml_model = tfcoreml.convert(
        tf_model_path=tf_model_path,
        mlmodel_path=coreml_model_file,
        input_name_shape_dict=input_tensor_shapes,
        output_feature_names=output_tensor_names,
        image_input_names = image_input_name,
        class_labels = class_labels)

# Get image pre-processing parameters of a saved CoreML model
spec = coremltools.models.utils.load_spec(coreml_model_file)
if spec.WhichOneof('Type') == 'neuralNetworkClassifier':
  nn = spec.neuralNetworkClassifier
  print("neuralNetworkClassifier")
if spec.WhichOneof('Type') == 'neuralNetwork':
  nn = spec.neuralNetwork  
  print("neuralNetwork")
if spec.WhichOneof('Type') == 'neuralNetworkRegressor':
  nn = spec.neuralNetworkRegressor
  print("neuralNetworkClassifierRegressor")

preprocessing = nn.preprocessing[0].scaler
print('channel scale: ', preprocessing.channelScale)
print('blue bias: ', preprocessing.blueBias)
print('green bias: ', preprocessing.greenBias)
print('red bias: ', preprocessing.redBias)

inp = spec.description.input[0]
if inp.type.WhichOneof('Type') == 'imageType':
  colorspace = _FeatureTypes_pb2.ImageFeatureType.ColorSpace.Name(inp.type.imageType.colorSpace)
  print('colorspace: ', colorspace)

coreml_model = tfcoreml.convert(
        tf_model_path=tf_model_path,
        mlmodel_path=coreml_model_file,
        input_name_shape_dict=input_tensor_shapes,
        output_feature_names=output_tensor_names,
        image_input_names = image_input_name,
        class_labels = class_labels,
        red_bias = -1,
        green_bias = -1,
        blue_bias = -1,
        image_scale = 2.0/255.0)

COREMLTOOLS

下面介绍下苹果官方的转换工具。现在已经发布2.0版本了。支持很多平台：

Screen Shot 2018-09-18 at 10.26.47 AM.png

模型量化:

Screen Shot 2018-09-18 at 10.17.13 AM.png

提供一个Keras 模型转换 coreml模型脚本:

  import coremltools
  import keras
  from keras.models import load_model
  from keras.utils.generic_utils import CustomObjectScope
  class_labels = []
  for i in range(62):
       class_labels.append(str(i))

   with CustomObjectScope({'relu6': keras.applications.mobilenet.relu6}):
      keras_model = load_model('traffic_sign_with_class_weights.h5')
      coreml_model = coremltools.converters.keras.convert(keras_model,
                                                      input_names=['input_1'],
                                                      image_input_names='input_1',
                                                      output_names='activation_1',
                                                      image_scale=2/255.0,
                                                      red_bias=-1,
                                                      green_bias=-1,
                                                      blue_bias=-1,
                                                      class_labels=class_labels)
  
 coreml_model.save('traffic_sign_with_class_weights.mlmodel')

我自己实现的一个人像分割模型，然后使用keras间接移植到IOS设备上，可以实时进行检测，效果还不错哈。

head_segmentation.jpg

如果有想了解深度学习模型移动端移植问题的可以咨询我的微信: ItchHacker。欢迎推荐江浙和上海地区的计算机视觉，数字图像处理岗位。

Reference

1. https://qiita.com/hengsokvisal/items/dbb61851a8c76c96c700
2. https://www.appcoda.com/coreml2/
3. https://www.appcoda.com/coreml-introduction/
4. https://www.raywenderlich.com/577-core-ml-and-vision-machine-learning-in-ios-11-tutorial
5. https://sourcediving.com/machine-learning-on-mobile-fc34be69df1a
   6.https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite/toco
6. https://blog.csdn.net/u011511601/article/details/80262707
7. https://blog.algorithmia.com/machine-learning-and-mobile-deploying-models-on-the-edge/
8. https://github.com/tensorflow/tensorflow/issues/15122