TensorRT trtexec 学习笔记

• trtexec 实现onnx转tensorrt

./trtexec --onnx=test.onnx --saveEngine=test.gie --verbose --dumpLayerInfo --dumpOutput --dumpProfile

其中

--verbose 打印log

--dumpLayerInfo将每个layer的信息都打印出来

--dumpOutput将每个输出都打印到终端上来

--dumpProfile打印一些中间分析信息，比如每层的 执行时间

• trtexec使用GIE进行推理

./my_trtexec --loadEngine=model_fp16.gie --fp16 --loadInputs=input:input_tensor.bin --exportOutput=output.json

其中
--loadEngine 加载已经转化好的plan文件
--fp16 表示半精度
--loadInputs 可以加载输入数据

    input:input_tensor.bin, 
    input是网络的输入名字，根据自己的网络进行修改
    input_tensor.bin是自己生成好的输入，比如将图像数据转换成float之后，存成文件

--exportOutput 将网络所有的输出保存成output.json文件

• trtexec 转换DynamicShpaes的ONNX

trtexec --onnx=my_model.onnx --saveEngine=my_model_fp16.trt --minShapes=input:1x3x320x320 --optShapes=input:1x3x640x640 --maxShapes=input:1x3x1280x1280 --fp16

其中
input:1x3x320x320, input是输入tensor的名字
--minShapes 是指输入“input"的最小尺寸
--optShapes 是当前输入的实际大小
--maxShapes 是指输入”input"的最大尺寸
--fp16 推理引擎的精度
官方解释：

  --minShapes=spec            Build with dynamic shapes using a profile with the min shapes provided
  --optShapes=spec            Build with dynamic shapes using a profile with the opt shapes provided
  --maxShapes=spec            Build with dynamic shapes using a profile with the max shapes provided
  --minShapesCalib=spec       Calibrate with dynamic shapes using a profile with the min shapes provided
  --optShapesCalib=spec       Calibrate with dynamic shapes using a profile with the opt shapes provided
  --maxShapesCalib=spec       Calibrate with dynamic shapes using a profile with the max shapes provided
                              Note: All three of min, opt and max shapes must be supplied.
                                    However, if only opt shapes is supplied then it will be expanded so
                                    that min shapes and max shapes are set to the same values as opt shapes.
                                    Input names can be wrapped with escaped single quotes (ex: \'Input:0\').
                              Example input shapes spec: input0:1x3x256x256,input1:1x3x128x128
                              Each input shape is supplied as a key-value pair where key is the input name and
                              value is the dimensions (including the batch dimension) to be used for that input.
                              Each key-value pair has the key and value separated using a colon (:).
                              Multiple input shapes can be provided via comma-separated key-value pairs.

• trtexec动态加载自定义插件

trtexec --onnx=my_model.onnx --plugins=build/my_plugin.so --saveEngine=my_model.engine

其中
--plugins 指定我们自定义插件动态库文件名
官方解释
--plugins                   Plugin library (.so) to load (can be specified multiple times)