tfserving

部署

docker run -dit -p 8501:8501 -p 8500:8500 -v /data1/minisearch/modelzoo/deepfm/models:/models/deepfm -e MODEL_NAME=deepfm -e TF_CPP_MIN_VLOG_LEVEL=1 $image_name

TF_CPP_MIN_VLOG_LEVEL=1设置打印日志类型，如下

2023-08-03 11:38:03.760198: I external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:946] Process node: 305 step 5 {{node save/Assign_460}} = Assign[T=DT_FLOAT, _class=["loc:@save/Assign_458"], _grappler_relax_allocator_constraints=true, _output_shapes=[[500,16]], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](input_layer/input_layer_1/D_embedding/embedding_weights/part_0/Adam_1, save/RestoreV2_1:195) device: /job:localhost/replica:0/task:0/device:CPU:0
2023-08-03 11:38:03.760879: I external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:946] Process node: 340 step 5 {{node save/Assign_475}} = Assign[T=DT_FLOAT, _grappler_relax_allocator_constraints=true, _output_shapes=[[100,16]], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](input_layer/input_layer_1/R_embedding/embedding_weights/part_0/Adam_1, save/RestoreV2_1:210) device: /job:localhost/replica:0/task:0/device:CPU:0
2023-08-03 11:38:03.760136: I external/org_tensorflow/tensorflow/core/common_runtime/executor.cc:946] Process node: 319 step 5 {{node save/Assign_326}} = Assign[T=DT_FLOAT, _class=["loc:@save/Assign_324"], _grappler_relax_allocator_constraints=true, _output_shapes=[[49,128]], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](final_dnn/final_dnn_0/kernel/part_0/Adam_1, save/RestoreV2_1:61) device: /job:localhost/replica:0/task:0/device:CPU:0

测试模型

查看端口占用

netstat -tunlp |grep 8500

# 查看模型版本
curl http://localhost:8501/v1/models/deepfm
# 查看模型参数
curl http://localhost:8501/v1/models/deepfm/metadata
# 请求分数
curl -d '{"instances": $input_data}' -X POST http://localhost:8501/v1/models/testnet:predict

传参

行模式 instances: [{}, {}]
列模式 inputs: {key1:value1, key2:value2}

问题

请求tfserving的分数与batchsize有关

与batchnorm有关

模型训练时分数与tfserving分数不同

检查导出的模型输入和传递的特征输入。

优化

基于TensorFlow Serving的深度学习在线预估
CTR模型加几个新特征训练后上线，tensorflow serving预测时间为什么比原来慢20多倍？
如何将TensorFlow Serving的性能提高超过70%？
tensorfow实现DeepFM，模型保存、部署并java请求调用（ctr预测）
tensorflow为什么测试时batch_size不同会影响结果？
tf-serving docker 中批处理配置的踩坑
Tensorflow Serving 踩坑指南