Linux安装mmdeploy1.2

参考链接

mmdeploy/docs/en/01-how-to-build/linux-x86_64.md at main · open-mmlab/mmdeploy

Toolchains安装

  • 操作系统:Ubuntu 18.04
  • cuda:11.4

安装cuda之前需要确认自己机器的驱动是否支持该版本的cuda,cuda版本与驱动之间的关系可以打开链接查看,驱动下载的载链接为:https://www.nvidia.com/download/index.aspx。
确认上述信息或者更新自己驱动之后,就可以打开链接,根据自己机器选择合适的配置,得到下载命令,下载文件并安装。Linux安装mmdeploy1.2_第1张图片
Linux安装mmdeploy1.2_第2张图片
将cuda环境变量添加到系统中

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
  • cudnn:8.2

cudnn的下载链接为:https://developer.nvidia.com/rdp/cudnn-archive。本文中下载的cudnn版本为8.2.1,适配cuda 11.x的版本。具体安装过程请自行搜索。
Linux安装mmdeploy1.2_第3张图片

  • cmake:cmake版本需要大于等于3.14.0,安装方式如下:
wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/

依赖环境

MMdeploy可以分为两部分:Model Converter和SDK。其中Model Converter是将模型转换为目标推理引擎要求的格式文件,如TensorRT,则生成TensorRT Engine文件。SDK提供多种编程语言在工业生产来部署目标推理引擎。

为Model Conveter安装依赖

miniconda

  1. 安装miniconda
  • 下载miniconda文件,打开链接,找到适合自己目标机器的文件
  • 安装minconda

通过如下命令安装即可,过程中一直enter就行,默认情况下,conda的环境变量会写入到bashrc文件中。

sudo bash Miniconda3-latest-Linux-x86_64.sh
  • 激活conda环境

通过下面的命令可以激活conda环境,进入base虚拟环境。

source ~/.bashrc
  • 替换conda国内源
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --set show_channel_urls yes

如果只想在安装时使用国内源,则可以通过如下命令实现:

conda install -y -c https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/linux-64 opencv tensorboardX
  1. 为mmdeploy创建虚拟环境
conda create -n mmdeploy python=3.8 -y
conda activate mmdeploy

pytorch:1.10.0

通过如下的命令安装pytorch 1.10.0,对应的cudatoolkit为11.3,而安装的cuda为11.4,但是对于后续的测试没有什么影响。

conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge

mmcv

export cu_version=cu114 # cuda 11.4
export torch_version=torch1.10
pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0rc2"

为SDK安装依赖环境

  • OpenCV
sudo apt-get install libopencv-dev
  • pplcv
git clone https://github.com/openppl-public/ppl.cv.git
cd ppl.cv
export PPLCV_DIR=$(pwd)
git checkout tags/v0.7.0 -b v0.7.0
./build.sh cuda

安装推理引擎

Linux安装mmdeploy1.2_第4张图片
mmdeploy支持以不同的backend推理模型,在这篇文章中主要测试Tensor RT backend。本文以TensorRT推理引擎的安装为例,其他推理平台可以参考文档。

TensorRT

  1. 下载TensorRT tar文件
tar -xvf TensorRT-8.2.4.2.Linux.x86_64-gnu.cuda-11.4.cudnn8.2.tar.gz
  1. 添加环境变量
# <<< TensorRT 8 <<<

export PATH=/home/xxxx/local/TensorRT-8.2.4.2/bin:$PATH
export LD_LIBRARY_PATH=/home/xxxx/local/TensorRT-8.2.4.2/lib:$LD_LIBRARY_PATH
export C_INCLUDE_PATH=/home/xxxx/local/TensorRT-8.2.4.2/include:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/home/xxxx/local/TensorRT-8.2.4.2/include:$CPLUS_INCLUDE_PATH
  1. 安装TensorRT python 包
cd python
# 根据python的版本安装
pip install tensorrt-8.2.4.2-cp38-none-linux_x86_64.whl
  1. 安装pycuda
conda install -c conda-forge pycuda

编译mmdeploy

编译mmdeploy分为两部分,Model Converter和SDK。其中converter的作用是将pytorch模型转换为对应backend的推理引擎文件,并测试起在不同数据集上的性能;SDK提供不同编程语言来将backend的推理引擎部署到工业实际生成中。

Model Converter

编译model converter包含两个步骤,一个是编译backend的自定义算子,一个是安装mmdeploy-python库。

编译tensorrt算子

cd ${MMDEPLOY_DIR}
mkdir -p build && cd build
cmake -DCMAKE_CXX_COMPILER=g++-7 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${TENSORRT_DIR} -DCUDNN_DIR=${CUDNN_DIR} ..
make -j$(nproc) && make install

编译成功之后会生成libmmdeploy_tensorrt_ops.so动态库。mmdeploy-python会在进行模型转换时,修改模型的function、module和symbolic,需要链接相应的tensorrt自定义算子

安装mmdeploy-python

cd ${MMDEPLOY_DIR}
mim install -e .

安装完成之后,就可以对mmdeploy支持的模型进行相应的转换。其实后续的SDK可以不安装。

Build SDK and Demo

这个应该就是基于mmdeploy的部署sdk,通过mmdeploy- python得到tensorrt engine之后,通过这个sdk可以直接部署模型。

cmake -DCMAKE_CXX_COMPILER=g++-7 \
-DMMDEPLOY_TARGET_BACKENDS=trt \
-DTENSORRT_DIR=/home/xxxx/local/TensorRT-8.2.4.2 \
-DCUDNN_DIR=/home/xxxx/local/cuda/ \
-DMMDEPLOY_BUILD_SDK=ON -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
-DMMDEPLOY_BUILD_EXAMPLES=ON -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" ..

转换模型

本文测试了常见模型Faster R-CNN、Cascade R-CNN、DeepLabv3和Mask R- CNN转换之后性能之间的对比。转换模型和测试性能可以分别参考如下两个链接:

  • https://github.com/open-mmlab/mmdeploy/blob/main/docs/en/02-how-to-run/convert_model.md
  • https://github.com/open-mmlab/mmdeploy/blob/main/docs/en/02-how-to-run/profile_model.md

benchmark

mmdeploy支持不同类型模型的转换,具体运行时间和精度可以参考如下链接。

mmdet

对应mmdetection,测试了Yolov3、Faster R-CNN、Cascade R-CNN和Mask R-CNN。
mmdetection/docs/en/model_zoo.md at main · open-mmlab/mmdetection
在安装MM Detection遇到numpy的版本问题,解决方案源码编译安装cocoapi。

git clone https://github.com/pdollar/coco.git

Linux安装mmdeploy1.2_第5张图片

Yolov3

转换engine脚本如下(static shape转换正常,dynamic shape存在问题):

#!/bin/bash

export MMDET_ROOT=/home/xxxx/workspace/mmlab/mmdetection
python ./tools/deploy.py \
    configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
    $MMDET_ROOT/configs/yolo/yolov3_d53_8xb8-ms-608-273e_coco.py \
    $MMDET_ROOT/exp/yolov3_d53_mstrain-608_273e_coco_20210518_115020-a2c3acb8.pth \
    $MMDET_ROOT/demo/demo.jpg \
    --work-dir work_dir \
    --device cuda:0

Yolo v3就进行了Tensor RT Engine转换未对性能进行相应的对比。

Faster R-CNN

转换engine文件的脚本与Yolo v3转换类似,只需要替换对应的deploy_config和model_config文件路径。

#!/bin/bash

export MMDET_ROOT=/home/xxxx/workspace/mmlab/mmdetection
python ./tools/deploy.py \
    configs/mmdet/detection/faster_rcnn_tensorrt_static-800x1344.py \
    $MMDET_ROOT/configs/faster_rcnn/faster-rcnn_r50_fpn_2x_coco.py \
    $MMDET_ROOT/exp/faster_rcnn_r50_fpn_2x_coco_bbox_mAP-0.384_20200504_210434-a5d8aa15.pth \
    $MMDET_ROOT/demo/demo.jpg \
    --work-dir work_dir \
    --device cuda:0

测试其在COCO2017-val数据集上的性能。
  • pytorch
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.420
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.421
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.503
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.520
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.326
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.557
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.661
07/05 14:00:14 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.420 0.215 0.421 0.503
07/05 14:00:16 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/05 14:00:16 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  
coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4200  
coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4210  
coco/bbox_mAP_l: 0.5030  
data_time: 0.0028  time: 0.0402
  • tensorrt-fp32

通过onnx转换得到对应的engine文件之后,可以通过tools/test.py文件测试精度和速度。

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.419
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.421
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.502
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.325
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.556
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.661
07/05 14:38:00 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.419 0.215 0.421 0.502
07/05 14:38:01 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4190  coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4210  coco/bbox_mAP_l: 0.5020  
data_time: 0.0030  time: 0.0297
  • tensorrt-fp16
Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.384
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.590
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.418
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.215
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.420
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.501
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.519
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.325
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.556
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.659
07/05 14:58:28 - mmengine - INFO - bbox_mAP_copypaste: 0.384 0.590 0.418 0.215 0.420 0.501
07/05 14:58:29 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3840  coco/bbox_mAP_50: 0.5900  
coco/bbox_mAP_75: 0.4180  coco/bbox_mAP_s: 0.2150  
coco/bbox_mAP_m: 0.4200  coco/bbox_mAP_l: 0.5010  
data_time: 0.0028  time: 0.0172

运行时间测试
  • tensorrt-fp16(仅包含engine推理时间)
07/05 15:20:09 - mmengine - INFO - [tensorrt]-4750 times per count: 10.58 ms, 94.51 FPS
07/05 15:20:09 - mmengine - INFO - Epoch(test) [4750/5000]    eta: 0:00:04  time: 0.0146  data_time: 0.0017  memory: 38  
07/05 15:20:10 - mmengine - INFO - Epoch(test) [4800/5000]    eta: 0:00:03  time: 0.0208  data_time: 0.0048  memory: 39  
07/05 15:20:11 - mmengine - INFO - [tensorrt]-4850 times per count: 10.58 ms, 94.54 FPS
07/05 15:20:11 - mmengine - INFO - Epoch(test) [4850/5000]    eta: 0:00:02  time: 0.0173  data_time: 0.0031  memory: 39  
07/05 15:20:12 - mmengine - INFO - Epoch(test) [4900/5000]    eta: 0:00:01  time: 0.0142  data_time: 0.0016  memory: 38  
07/05 15:20:13 - mmengine - INFO - [tensorrt]-4950 times per count: 10.58 ms, 94.55 FPS
07/05 15:20:13 - mmengine - INFO - Epoch(test) [4950/5000]    eta: 0:00:00  time: 0.0197  data_time: 0.0024  memory: 39  
07/05 15:20:13 - mmengine - INFO - Epoch(test) [5000/5000]    eta: 0:00:00  time: 0.0148  data_time: 0.0016  memory: 39  

截屏2023-07-06 18.41.16.png
同样,当得到TensorRT Engine之后,我们可以采用trtexec工具来加载engine测试时间,trtexec推理命令:

trtexec --loadEngine=work_dir/faster_rcnn_dynamic_fp16/end2end.engine --iterations=5000 --plugins=mmdeploy/lib/libmmdeploy_tensorrt_ops.so --minShapes=input:1x3x320x320 --optShapes=input:1x3x800x1344 --maxShapes=input:1x3x1344x1344 --shapes=input:1x3x800x1344 --fp16 --workspace=8000
  • tensorrt-fp32
07/05 15:35:09 - mmengine - INFO - [tensorrt]-4750 times per count: 21.97 ms, 45.51 FPS
07/05 15:35:09 - mmengine - INFO - Epoch(test) [4750/5000]    eta: 0:00:07  time: 0.0252  data_time: 0.0016  memory: 38  
07/05 15:35:10 - mmengine - INFO - Epoch(test) [4800/5000]    eta: 0:00:06  time: 0.0262  data_time: 0.0016  memory: 39  
07/05 15:35:11 - mmengine - INFO - [tensorrt]-4850 times per count: 21.96 ms, 45.53 FPS
07/05 15:35:11 - mmengine - INFO - Epoch(test) [4850/5000]    eta: 0:00:04  time: 0.0283  data_time: 0.0024  memory: 39  
07/05 15:35:13 - mmengine - INFO - Epoch(test) [4900/5000]    eta: 0:00:03  time: 0.0262  data_time: 0.0018  memory: 38  
07/05 15:35:14 - mmengine - INFO - [tensorrt]-4950 times per count: 21.97 ms, 45.52 FPS
07/05 15:35:14 - mmengine - INFO - Epoch(test) [4950/5000]    eta: 0:00:01  time: 0.0304  data_time: 0.0029  memory: 39  
07/05 15:35:16 - mmengine - INFO - Epoch(test) [5000/5000]    eta: 0:00:00  time: 0.0363  data_time: 0.0033  memory: 39  

截屏2023-07-06 18.28.18.png

  • pytorch
07/06 14:49:56 - mmengine - INFO - (GB) mem_used: 81.76 | uss: 3.89 | pss: 3.96 | total_proc: 1
07/06 14:50:45 - mmengine - INFO - ==================================
07/06 14:50:45 - mmengine - INFO - Done image [1000/5000], fps: 27.6 img/s, times per image: 36.2 ms/img, cuda memory: 520 MB
07/06 14:50:45 - mmengine - INFO - (GB) mem_used: 78.25 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:51:32 - mmengine - INFO - ==================================
07/06 14:51:32 - mmengine - INFO - Done image [2000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 520 MB
07/06 14:51:33 - mmengine - INFO - (GB) mem_used: 84.05 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:52:21 - mmengine - INFO - ==================================
07/06 14:52:21 - mmengine - INFO - Done image [3000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 534 MB
07/06 14:52:22 - mmengine - INFO - (GB) mem_used: 83.13 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:09 - mmengine - INFO - ==================================
07/06 14:53:09 - mmengine - INFO - Done image [4000/5000], fps: 27.8 img/s, times per image: 36.0 ms/img, cuda memory: 520 MB
07/06 14:53:09 - mmengine - INFO - (GB) mem_used: 82.96 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:56 - mmengine - INFO - ==================================
07/06 14:53:56 - mmengine - INFO - Done image [5000/5000], fps: 27.7 img/s, times per image: 36.0 ms/img, cuda memory: 534 MB
07/06 14:53:56 - mmengine - INFO - (GB) mem_used: 83.42 | uss: 4.74 | pss: 4.81 | total_proc: 1
07/06 14:53:56 - mmengine - INFO - ============== Done ==================
07/06 14:53:56 - mmengine - INFO - Overall fps: 27.7 img/s, times per image: 36.1 ms/img
07/06 14:53:56 - mmengine - INFO - cuda memory: 163 MB
07/06 14:53:57 - mmengine - INFO - (GB) mem_used: 83.29 | uss: 4.74 | pss: 4.81 | total_proc: 1

截屏2023-07-06 18.36.43.png

统计
FasterR-CNN-R50 mAP latency(ms) memory(G)
pytorch 0.384 36.1 3.903
tensorrt-fp32 0.384 21.69 4.243
tensorrt-fp16 0.384 10.58 3.634

Cascade R- CNN

性能对比
  • pytorch

截屏2023-07-07 09.40.03.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.440
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.438
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.529
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.582
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.689
07/07 09:41:21 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.440 0.225 0.438 0.529
07/07 09:41:23 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/07 09:41:23 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4400  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4380  coco/bbox_mAP_l: 0.5290  
data_time: 0.0028  time: 0.0474
  • tensorrt-fp32

截屏2023-07-07 10.08.51.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.439
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.437
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.529
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.543
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.581
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.689
07/07 10:06:05 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.439 0.225 0.437 0.529
07/07 10:06:05 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4390  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4370  coco/bbox_mAP_l: 0.5290  
data_time: 0.0022  time: 0.0310
07/07 10:09:36 - mmengine - INFO - [tensorrt]-3050 times per count: 25.29 ms, 39.54 FPS
07/07 10:10:08 - mmengine - INFO - [tensorrt]-4050 times per count: 25.33 ms, 39.48 FPS
  • tensorrt-fp16

截屏2023-07-07 10.22.51.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.403
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.586
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.439
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.438
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.531
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.333
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.581
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.691
07/07 10:23:56 - mmengine - INFO - bbox_mAP_copypaste: 0.403 0.586 0.439 0.225 0.438 0.531
07/07 10:23:56 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.4030  coco/bbox_mAP_50: 0.5860  
coco/bbox_mAP_75: 0.4390  coco/bbox_mAP_s: 0.2250  
coco/bbox_mAP_m: 0.4380  coco/bbox_mAP_l: 0.5310  
data_time: 0.0026  time: 0.0192
07/07 10:22:42 - mmengine - INFO - [tensorrt]-3050 times per count: 13.16 ms, 75.96 FPS
07/07 10:23:01 - mmengine - INFO - [tensorrt]-4050 times per count: 13.15 ms, 76.03 FPS

统计
cascade rcnn-R50 mAP latency(ms) memory(G)
pytorch 0.4030 40.3 4.124
tensorrt-fp32 0.4030 25.33 4.347
tensorrt-fp16 0.4030 13.16 3.757

Mask R-CNN

在进行实例分割模型转换时,用的是MMdeploy/configs/mmdet/instance-seg中的配置文件,这一点很重要‼️。

性能对比
  • pytorch

截屏2023-07-07 16.14.18.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.382
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.525
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.515
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/07 16:21:43 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.382 0.525
07/07 16:21:45 - mmengine - INFO - Results has been saved to exp/results.pkl.
07/07 16:21:45 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3920  coco/bbox_mAP_50: 0.5960  
coco/bbox_mAP_75: 0.4280  coco/bbox_mAP_s: 0.2290  
coco/bbox_mAP_m: 0.4260  coco/bbox_mAP_l: 0.5120  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3820  coco/segm_mAP_l: 0.5250  
data_time: 0.0225  time: 0.1063
Overall fps: 26.4 img/s, times per image: 37.9 ms/img
  • tensorrt-fp32
    image.png
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.525
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.514
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/08 16:01:46 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.381 0.525
07/08 16:01:47 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3910  coco/bbox_mAP_50: 0.5960  
coco/bbox_mAP_75: 0.4280  coco/bbox_mAP_s: 0.2290  
coco/bbox_mAP_m: 0.4250  coco/bbox_mAP_l: 0.5110  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3810  coco/segm_mAP_l: 0.5250  data_time: 0.0206  time: 0.0796
07/08 15:55:20 - mmengine - INFO - [tensorrt]-1050 times per count: 25.77 ms, 38.80 FPS
07/08 15:56:37 - mmengine - INFO - [tensorrt]-2050 times per count: 25.80 ms, 38.76 FPS
07/08 15:59:15 - mmengine - INFO - [tensorrt]-4050 times per count: 25.85 ms, 38.69 FPS
  • tensorrt-fp16

image.png

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.564
Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.380
Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.166
Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.381
Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.524
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.481
Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.283
Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.514
Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.646
07/08 16:29:03 - mmengine - INFO - segm_mAP_copypaste: 0.354 0.564 0.380 0.166 0.381 0.524
07/08 16:29:04 - mmengine - INFO - Epoch(test) [5000/5000]    
coco/bbox_mAP: 0.3920  coco/bbox_mAP_50: 0.5950  
coco/bbox_mAP_75: 0.4290  coco/bbox_mAP_s: 0.2280  
coco/bbox_mAP_m: 0.4270  coco/bbox_mAP_l: 0.5100  
coco/segm_mAP: 0.3540  coco/segm_mAP_50: 0.5640  
coco/segm_mAP_75: 0.3800  coco/segm_mAP_s: 0.1660  
coco/segm_mAP_m: 0.3810  coco/segm_mAP_l: 0.5240  data_time: 0.0200  time: 0.0663
07/08 16:25:37 - mmengine - INFO - [tensorrt]-3050 times per count: 12.55 ms, 79.70 FPS
07/08 16:26:43 - mmengine - INFO - [tensorrt]-4050 times per count: 12.55 ms, 79.69 FPS

统计
Mask R-CNN R50 mAP segm_mAP latency(ms) memory(G)
pytorch 0.3920 0.3540 37.9 4.355
tensorrt-fp32 0.3910 0.3540 25.85 4.636
tensorrt-fp16 0.3920 0.3540 12.55 4.325

mmseg

cityscapes数据集准备

参考如下链接准备cityscapes数据集。
mmsegmentation/docs/en/user_guides/2_dataset_prepare.md at main · open-mmlab/mmsegmentation
此外对于cityscapes数据集,还需要进行如下操作:
Linux安装mmdeploy1.2_第6张图片

deeplabv3

测试cityscapes数据集上的结果
Tutorial 4: Train and test with existing models — MMSegmentation 1.1.0 documentation
官方给出的deeplabv3在不同配置下的性能如下:
mmsegmentation/configs/deeplabv3 at main · open-mmlab/mmsegmentation

  • pytorch

截屏2023-07-07 11.57.48.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.52 |
|      wall     | 53.12 | 58.34 |
|     fence     | 61.57 | 71.94 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.56 |
|  traffic sign | 79.58 | 87.34 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.42 | 72.11 |
|      sky      | 94.46 | 98.12 |
|     person    | 82.53 | 92.13 |
|     rider     | 62.77 | 75.76 |
|      car      | 95.37 | 97.82 |
|     truck     | 82.07 |  92.3 |
|      bus      | 89.74 | 93.15 |
|     train     | 84.59 | 89.49 |
|   motorcycle  | 69.78 |  81.5 |
|    bicycle    | 78.04 | 89.03 |
+---------------+-------+-------+
07/07 11:53:43 - mmengine - INFO - Iter(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  data_time: 0.0068  time: 0.1914

时间:fps: 6.01 img / s

  • tensorrt-fp32

截屏2023-07-07 14.31.43.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.52 |
|      wall     | 53.12 | 58.34 |
|     fence     | 61.57 | 71.94 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.56 |
|  traffic sign | 79.58 | 87.34 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.42 |  72.1 |
|      sky      | 94.46 | 98.12 |
|     person    | 82.53 | 92.13 |
|     rider     | 62.76 | 75.76 |
|      car      | 95.37 | 97.82 |
|     truck     | 82.07 |  92.3 |
|      bus      | 89.74 | 93.15 |
|     train     | 84.59 | 89.49 |
|   motorcycle  | 69.78 |  81.5 |
|    bicycle    | 78.04 | 89.03 |
+---------------+-------+-------+
07/07 14:28:39 - mmengine - INFO - Epoch(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  data_time: 0.0069  time: 0.1515
07/07 14:27:57 - mmengine - INFO - [tensorrt]-205 times per count: 131.08 ms, 7.63 FPS
07/07 14:28:11 - mmengine - INFO - [tensorrt]-305 times per count: 131.18 ms, 7.62 FPS
07/07 14:28:25 - mmengine - INFO - [tensorrt]-405 times per count: 131.46 ms, 7.61 FPS
  • tensorrt-fp16

截屏2023-07-07 14.44.53.png

+---------------+-------+-------+
|     Class     |  IoU  |  Acc  |
+---------------+-------+-------+
|      road     | 98.18 | 98.88 |
|    sidewalk   |  85.3 | 93.17 |
|    building   | 92.71 | 96.53 |
|      wall     | 53.15 | 58.38 |
|     fence     | 61.58 | 71.95 |
|      pole     | 65.34 | 77.46 |
| traffic light | 71.21 | 82.55 |
|  traffic sign | 79.58 | 87.33 |
|   vegetation  | 92.47 | 96.84 |
|    terrain    | 64.41 | 72.07 |
|      sky      | 94.47 | 98.12 |
|     person    | 82.53 | 92.12 |
|     rider     | 62.76 | 75.77 |
|      car      | 95.37 | 97.82 |
|     truck     |  82.1 | 92.31 |
|      bus      | 89.74 | 93.14 |
|     train     | 84.58 | 89.49 |
|   motorcycle  | 69.76 | 81.49 |
|    bicycle    | 78.04 | 89.01 |
+---------------+-------+-------+
07/07 14:45:04 - mmengine - INFO - Epoch(test) [500/500]    
aAcc: 96.1700  mIoU: 79.1200  mAcc: 86.5500  
data_time: 0.0361  time: 0.0830
07/07 14:44:40 - mmengine - INFO - [tensorrt]-205 times per count: 36.42 ms, 27.46 FPS
07/07 14:44:48 - mmengine - INFO - [tensorrt]-305 times per count: 36.49 ms, 27.40 FPS
07/07 14:44:56 - mmengine - INFO - [tensorrt]-405 times per count: 36.41 ms, 27.47 FPS
deeplabv3 mIoU latency(ms) memory(G)
pytorch 79.12 166.39 12,587
tensorrt-fp32 79.12 131.46 5.308
tensorrt-fp16 79.12 36.49 4.032

总结

本文介绍了如何安装MMDeploy,详细叙述了安装的流程,并给出了相应的版本号。之后对不同类型的算法在转TensorRT Engine之后的性能进行了对比,可以看到的是评价指标基本上没有掉点,运行速度在FP16下提升了3-4倍。个人感觉使用MMdeploy来部署MMDetection和MMsegmetation的模型效率较高,且兼容性很好,推荐使用哈。

你可能感兴趣的:(#,openmmlab,mmdeploy,ubuntu)