使用Docker进行模型部署

配置

（1）Docker安装+WSL2

首先需要开启Hyper-V（win10家庭版没有自己安装）

https://www.jb51.net/article/182013.htm

安装docker

https://www.runoob.com/docker/windows-docker-install.html

docker run hello-wold 就会开启Linux（一般试个两次就可以）

（一）本地部署

https://www.jianshu.com/p/a09279e7f871

1）准备训练好的weights文件或者model文件（硬盘里

2）定义图结构后+load_weights,或者load_model直接一起加载

//网络结构定义
model = build_bert_model(config_path,checkpoint_path,class_nums)
model.compile(
        loss='sparse_categorical_crossentropy',
        optimizer=Adam(5e-6),
        metrics=['accuracy'],
    )
model.load_weights('C:\\Users\\delll\\PycharmProjects\\RenamePrediction\\prediction\\v3_best_model.weights')

3）预测

1. 直接预测

tokenizer = Tokenizer(dict_path)
textt1="public void test"
text2="public void test"
token_ids, segment_ids = tokenizer.encode(text1,text1, maxlen=512)
import numpy as np
print(model.predict([np.array([token_ids]), np.array([segment_ids])]))#概率结果
print(model.predict([np.array([token_ids]), np.array([segment_ids])]).argmax(axis=1))#最终结果

2. 使用docker部署

• 转化成saved model（docker可读取的格式）

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
import os
from tensorflow.keras.losses import categorical_crossentropy
from tensorflow.keras.optimizers import Adadelta

def export_model(model,
                 export_model_dir,
                 model_version
                 ):
    """
    :param export_model_dir: type string, save dir for exported model    url
    :param model_version: type int best
    :return:no return
    """
    with tf.get_default_graph().as_default():
        # prediction_signature
        tensor_info_input_0 = tf.saved_model.utils.build_tensor_info(model.input[0])
        tensor_info_input_1 = tf.saved_model.utils.build_tensor_info(model.input[1])
        tensor_info_output = tf.saved_model.utils.build_tensor_info(model.output)
        print(model.input)
        print(model.output.shape, '**', tensor_info_output)
        prediction_signature = (
            tf.saved_model.signature_def_utils.build_signature_def(
                inputs ={'input_0': tensor_info_input_0,'input_1': tensor_info_input_1}, # Tensorflow.TensorInfo
                outputs={'result': tensor_info_output},
                #method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)
                 method_name= "tensorflow/serving/predict")
               
        )
        print('step1 => prediction_signature created successfully')
        # set-up a builder
        os.mkdir(export_model_dir)
        export_path_base = export_model_dir
        export_path = os.path.join(
            tf.compat.as_bytes(export_path_base),
            tf.compat.as_bytes(str(model_version)))
        builder = tf.saved_model.builder.SavedModelBuilder(export_path)
        builder.add_meta_graph_and_variables(
            # tags:SERVING,TRAINING,EVAL,GPU,TPU
            sess=tf.compat.v1.keras.backend.get_session(),
            tags=[tf.saved_model.tag_constants.SERVING],
            signature_def_map={
                'predict':prediction_signature,
                tf.saved_model.signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: prediction_signature,

            },
            )
        print('step2 => Export path(%s) ready to export trained model' % export_path, '\n starting to export model...')
        #builder.save(as_text=True)
        builder.save()
        print('Done exporting!')

export_model(model,"bert-methodPredict",1)

由于我的bert模型定义如下：

• 输入token_id和segment_id两部分输入，

• 输出是dense层的输出
  所以我的API定义过程如下

  ###输入tensor  
  tensor_info_input_0 = tf.saved_model.utils.build_tensor_info(model.input[0])
  tensor_info_input_1 = tf.saved_model.utils.build_tensor_info(model.input[1])
  ###输出tensor
  tensor_info_output = tf.saved_model.utils.build_tensor_info(model.output)
  ### 定义api
  prediction_signature = (
       tf.saved_model.signature_def_utils.build_signature_def(
              inputs ={'input_0': tensor_info_input_0,'input_1': tensor_info_input_1}, # Tensorflow.TensorInfo
              outputs={'result': tensor_info_output},
              #method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)
              method_name= "tensorflow/serving/predict")
                 
          )
  

输出以下则成功

转换得到的模型文件

• 运行部署

用powershell

docker pull tensorflow/serving

docker run -p 8501:8501 --mount type=bind,source=F:/bert-methodPredict,target=/models/bert -e MODEL_NAME=bert --name bert -t tensorflow/serving(保持开启)

• 不要有空格

• 使用上面的docker命令启动TF Server ：
  (1)-p 8501:8501是端口映射，是将容器的8501端口映射到宿主机的8501端口，后面预测的时候使用该端口；
  (2)-e MODEL_NAME=bert 设置模型名称；
  (3)–mount type=bind,source=/opt/developer/wp/learn/bert, target=/models/bert 是将宿主机的路径/opt/developer/wp/learn/bert 挂载到容器的/models/bert 下。
  /opt/developer/wp/learn/bert是通过上述py脚本生成的Saved_model的路径。容器内部会根据绑定的路径读取模型文件；

  curl http://localhost:8501/v1/models/bert查看服务状态

• python使用request请求bert服务

(anaconda activate python36)

sent = "来玩英雄联盟"
tokenid_train =  tokenizer.encode(sent,max_len=200)[0] 
sen_id_train =   tokenizer.encode(sent,max_len=200)[1]
import requests
SERVER_URL = "http://192.168.0.100:8501/v1/models/bert:predict"
predict_request='{"signature_name": "predict", "instances":[{"input_0":%s,"input_1":%s}] }' %(tokenid_train,sen_id_train)
response = requests.post(SERVER_URL, data=predict_request)
response.content

b'{\n    "predictions": [[0.0405665115, 0.959433496]\n    ]\n}'

当然你也可以使用gRPC 的API在8500端口访问你的服务。

（二）服务器部署

在服务器上存模型文件，安装docker，用服务器IP访问，客户端用POST

（1）模型环境配置

1. docker配置

   https://blog.csdn.net/qq_35307209/article/details/97894277

   • 拉取tf-serving

   

   • 查看所需镜像是否成功下载
     

   • 运行容器

   docker run -p 8501:8501 --mount type=bind,source=/root/SIAL-LAB/rename_model/bert-methodPredict,target=/models/bert_rename_predict xxxx

   （文件夹路径到到/1的父文件夹）

   • 重启后台docker

     sudo systemctl start docker

2. winSCP上传模型文件

   模型文件bert-methodPredict和调用文件bert_preprocess

（2）python+spyne远程数据预处理

1. 环境配置（python库）

   需要使用pip3和python3，下载tf=2.2.0，keras=2.3.1（其他下次下载的时候根据提示全部依赖下载完）

2. 服务器端代码（server_web.py）

   0.0.0.0服务器设置比较方便，都可以访问

   from spyne import ServiceBase, Iterable, Unicode, Integer, Application, rpc
   from spyne.protocol.soap import Soap11
   from spyne.server.wsgi import WsgiApplication
   from wsgiref.simple_server import make_server
   from bert4keras.tokenizers import Tokenizer
   import requests
   
   class HelloWorldService(ServiceBase):
       @rpc(Unicode, _returns=Unicode)
       #------------------------------获取输入文本+传入模型+返回模型数据
       def say_hello(self, name):
           method_list=name.split("@@@@@")
           dict_path = '/root/SIAL-LAB/rename_model/bert_preprocess/vocab.txt'
           tokenizer = Tokenizer(dict_path)
           token_ids, segment_ids = tokenizer.encode(method_list[0],method_list[1], maxlen=512)
   	
           SERVER_URL = "http://server_url:8501/v1/models/bert_rename_predict:predict"
           predict_request='{"signature_name": "predict", "instances":[{"input_0":%s,"input_1":%s}] }' %(token_ids,segment_ids)
           response = requests.post(SERVER_URL, data=predict_request)
           return str(response.content)
   application = Application([HelloWorldService],
                             tns='spyne.examples.hello',
   
                             in_protocol=Soap11(validator='lxml'),
                             out_protocol=Soap11())
   if __name__ == '__main__':
   
       wsgi_app = WsgiApplication(application)
       server = make_server('0.0.0.0', 8000, wsgi_app)
       server.serve_forever()

3.客户端代码（插件中)

把选取得到的文本输入到数据预处理器

  //把选取得到的文本输入到数据预处理器
               String urlString = "http:/server_url:8000/?wsdl";//wsdl文档的地址
               URL url = new URL(urlString);
               HttpURLConnection httpConn = (HttpURLConnection) url.openConnection();//打开连接
   
   
               String params="\n" +
                       "\n" +
                       "\n" +
                       "\n" +
                       ""+histStmt.toString()+"@@@@@"+curStmt.toString()+"\n" +
                       "\n" +
                       "\n" +
                       "";
               //Content-Length长度会自动进行计算
               httpConn.setRequestProperty("Content-Type", "text/xml; charset=utf-8");
               httpConn.setRequestMethod("POST");
               httpConn.setDoOutput(true);
               httpConn.setDoInput(true);
   
               OutputStream out = httpConn.getOutputStream();
               //把传过去的数据封装到buf中
   
               out.write(params.getBytes("UTF-8"));
               out.close();
   
   
               if (httpConn.getResponseCode() == HttpURLConnection.HTTP_OK)
               {
                   System.out.println("Success!");
                   InputStreamReader is = new InputStreamReader(httpConn.getInputStream());
                   BufferedReader in = new BufferedReader(is);
                   String inputLine;
   
                   while ((inputLine = in.readLine()) != null)
                   {
   
                       System.out.println(inputLine);
                  
                   }
   
                   in.close();
               }else{
                   System.out.println("Fail!");
               }
               httpConn.disconnect();

4. 远程后台运行

为了插件随时可以调用，作为服务器端需要常开在后台

nohup python3 xxx.py

看后台运行程序

ps -ef | grep python3