keras模型部署 flask & tfserving

keras训练后的模型可以用两种方法实现。

• 利用keras的save方法保存模型成h5格式。利用python web框架加载h5模型并利用api请求对外提供http请求
• 生成pb格式，利用tfserving加载pb模型。提供http请求。

keras模型训练之后保存成h5格式部署。模型是一个文本分类模型

1.模型保存

# tb_callback = TensorBoard(log_dir='./logs', embeddings_freq=1,embeddings_data=x_train)
model = Sequential()
#实现自己的网络层
model.add ....
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])
print(model.summary())
model.fit(x_train, y_train, validation_split=0.25, epochs=15, batch_size=1024)
model.save_weights('build/mini_test_weights.h5')
model.save('build/mini_test_model.h5')
with open('build/mini_test_model.json', 'w') as outfile:
    outfile.write(model.to_json())

2.flask搭建加载h5模型

from flask import Flask
from flask import request
from keras.preprocessing import sequence
from keras.preprocessing.text import Tokenizer
from keras.models import load_model
import json

app = Flask(__name__)

#加载词对应的数字枚举 & 加载模型
model = load_model('D:\\mini_test_model.h5')
tokenizer = Tokenizer(filters='\t\n', char_level=True)
word_dict_file = 'D:\\dictionary_mini.json'
with open(word_dict_file, 'r') as outfile:
    tokenizer.word_index = json.load(outfile)

#处理成相同序列
def get_sequence(text):
    x_token = tokenizer.texts_to_sequences(text)
    x_processed = sequence.pad_sequences(x_token, maxlen=100, value=0)
    return x_processed

@app.route('/')
def hello_world():
    return 'Hello World!'

@app.route("/predict", methods=["GET","POST"])
def predict():
    data = request.get_json()
    if 'values' not in data:
        return {'result': 'no input'}
    arr = data['values']
    res = model.predict(get_sequence(arr))
    return {'result':res.tolist()}

if __name__ == '__main__':
    app.run()

使用postman请求示例：

3.使用tf servering 加载

keras模型保存成pb模型：

#转pb
import shutil 
import tensorflow as tf
tf.keras.backend.clear_session()
tf.keras.backend.set_learning_phase(0)
model = tf.keras.models.load_model('./mini_test_model.h5')

if os.path.exists('./model/1'):
    shutil.rmtree('./model/1')
    
export_path = './model/1'

# Fetch the Keras session and save the model
with tf.keras.backend.get_session() as sess:
    tf.saved_model.simple_save(
        sess,
        export_path,
        inputs={'inputs': model.input},
        outputs={t.name:t for t in model.outputs})
#生成之后目录结构
#.
#└── 1
#    ├── saved_model.pb
#    └── variables
#        ├── variables.data-00000-of-00001
#        └── variables.index

如果有自定义的方法 必须f1 score等函数，

例如模型compile时候：model.compile(loss=‘binary_crossentropy’, optimizer=‘adam’, metrics=[‘acc’,f1_m,precision_m, recall_m])

那么在加载的时候需要指定custom_objects，如果没有自定义方法可以忽略：

from keras.models import Sequential, load_model
model = load_model('build/category.h5', custom_objects={'f1_m':f1_m,'precision_m':precision_m,'recall_m':recall_m})
model.load_weights('build/category-weight.h5')
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics=['acc',f1_m,precision_m, recall_m])

生成pb模型后，开始搭建tfserving服务（使用docker）。

1. 首先docker pull tensorflow/serving 拉取docker镜像。如果支持GPU 可以使用gpu版本docker pull tensorflow/serving:latest-gpu

2. 执行指令

   docker run -p 8501:8501 -v <模型路径>:/models/<模型名> -e MODEL_NAME=<模型名> -t tensorflow/serving &
   # 模型路径 到model路径即可
   #实例：
   docker run -p 8501:8501 -v /mnt/build/model/:/models/mini -e MODEL_NAME=mini -t tensorflow/serving &
   

   docker ps 查看运行中的镜像

   docker stop停止某个镜像

3. 测试

   curl -d '{"instances": [{"inputs":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1]}]}' -X POST  "http://localhost:8501/v1/models/mini:predict"
   #返回：{
   #    "predictions": [[0.548769891]
   #    ]
   #}