mmdeploy 部署 mmdetection3d 模型

先说环境

python			--- 3.8.16
CUDA			---	11.6
cudnn			---	8.4.1
TensorRT		---	8.4.3.1	GA
PyTorch			---	1.12.1

2023-03-24 13:25:17,447 - mmdeploy - INFO - 

2023-03-24 13:25:17,447 - mmdeploy - INFO - **********Environmental information**********
2023-03-24 13:25:17,612 - mmdeploy - INFO - sys.platform: linux
2023-03-24 13:25:17,612 - mmdeploy - INFO - Python: 3.8.16 (default, Mar  2 2023, 03:21:46) [GCC 11.2.0]
2023-03-24 13:25:17,612 - mmdeploy - INFO - CUDA available: True
2023-03-24 13:25:17,612 - mmdeploy - INFO - GPU 0: NVIDIA GeForce RTX 3060 Laptop GPU
2023-03-24 13:25:17,612 - mmdeploy - INFO - CUDA_HOME: /usr/local/cuda-11.6
2023-03-24 13:25:17,612 - mmdeploy - INFO - NVCC: Cuda compilation tools, release 11.6, V11.6.124
2023-03-24 13:25:17,612 - mmdeploy - INFO - GCC: gcc (Ubuntu 8.4.0-3ubuntu2) 8.4.0
2023-03-24 13:25:17,612 - mmdeploy - INFO - PyTorch: 1.12.1
2023-03-24 13:25:17,612 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.6
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.6.1
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.6, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

2023-03-24 13:25:17,612 - mmdeploy - INFO - TorchVision: 0.13.1
2023-03-24 13:25:17,612 - mmdeploy - INFO - OpenCV: 4.7.0
2023-03-24 13:25:17,612 - mmdeploy - INFO - MMCV: 1.6.2
2023-03-24 13:25:17,612 - mmdeploy - INFO - MMCV Compiler: GCC 9.3
2023-03-24 13:25:17,612 - mmdeploy - INFO - MMCV CUDA Compiler: 11.6
2023-03-24 13:25:17,612 - mmdeploy - INFO - MMDeploy: 0.13.0+39c3282
2023-03-24 13:25:17,612 - mmdeploy - INFO - 

2023-03-24 13:25:17,612 - mmdeploy - INFO - **********Backend information**********
2023-03-24 13:25:17,638 - mmdeploy - INFO - tensorrt:	8.4.3.1
2023-03-24 13:25:17,638 - mmdeploy - INFO - tensorrt custom ops:	Available
2023-03-24 13:25:17,732 - mmdeploy - INFO - ONNXRuntime:	1.14.1
2023-03-24 13:25:17,732 - mmdeploy - INFO - ONNXRuntime-gpu:	None
2023-03-24 13:25:17,732 - mmdeploy - INFO - ONNXRuntime custom ops:	NotAvailable
2023-03-24 13:25:17,733 - mmdeploy - INFO - pplnn:	None
2023-03-24 13:25:17,734 - mmdeploy - INFO - ncnn:	None
2023-03-24 13:25:17,734 - mmdeploy - INFO - snpe:	None
2023-03-24 13:25:17,735 - mmdeploy - INFO - openvino:	None
2023-03-24 13:25:17,735 - mmdeploy - INFO - torchscript:	1.12.1
2023-03-24 13:25:17,735 - mmdeploy - INFO - torchscript custom ops:	NotAvailable
2023-03-24 13:25:17,782 - mmdeploy - INFO - rknn-toolkit:	None
2023-03-24 13:25:17,782 - mmdeploy - INFO - rknn2-toolkit:	None
2023-03-24 13:25:17,783 - mmdeploy - INFO - ascend:	None
2023-03-24 13:25:17,783 - mmdeploy - INFO - coreml:	None
2023-03-24 13:25:17,783 - mmdeploy - INFO - tvm:	None
2023-03-24 13:25:17,783 - mmdeploy - INFO - 

2023-03-24 13:25:17,783 - mmdeploy - INFO - **********Codebase information**********
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmdet:	2.28.2
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmseg:	0.30.0
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmcls:	0.25.0
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmocr:	None
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmedit:	None
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmdet3d:	1.0.0rc6
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmpose:	None
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmrotate:	None
2023-03-24 13:25:17,785 - mmdeploy - INFO - mmaction:	None

最好使用以下结构，方便设置目录

1. 检查工具版本

Cmake版本大于>= 3.14.0

wget https://github.com/Kitware/CMake/releases/download/v3.20.0/cmake-3.20.0-linux-x86_64.tar.gz
tar -xzvf cmake-3.20.0-linux-x86_64.tar.gz
sudo ln -sf $(pwd)/cmake-3.20.0-linux-x86_64/bin/* /usr/bin/

GCC 7+

# 如果 Ubuntu 版本 < 18.04，需要加入仓库
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-7
sudo apt-get install g++-7

2. 创建环境

# 创建虚拟环境
conda create -n mmdeploy python=3.8
# 安装torch
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6 -c pytorch -c conda-forge
# 安装mmcv
pip install openmim
mim install mmcv-full
# cmake
pip install cmake

3. MMDeploy SDK 依赖

# Ubuntu 18.04 及以上版本 
sudo apt install libspdlog-dev
# opencv >= 3.0
sudo apt install libopencv-dev
# pplcv
git clone https://github.com/openppl-public/ppl.cv.git
cd ppl.cv
export PPLCV_DIR=$(pwd)
git checkout tags/v0.7.0 -b v0.7.0
./build.sh cuda

4. 推理引擎

下载TensorRT，8.4.3 GA

# 下载
cd TensorRT压缩包目录
tar -zxvf 压缩包文件
pip3 install TensorRT-8.4.3.1/python/tensorrt-8.4.3.1-cp38-none-linux_x86_64.whl
cd TensorRT-8.4.3.1/
# .bashrc中添加环境变量
export TENSORRT_DIR=/home/txz/MMD/mmdeploy/TRT/TensorRT-8.4.3.1
export LD_LIBRARY_PATH=$TENSORRT_DIR/lib:$LD_LIBRARY_PATH
pip install pycuda

下载cudnn8.2.1.32

cd 压缩包目录
tar -zxvf 压缩包
# .bashrc中添加环境变量
export CUDNN_DIR=/home/txz/MMD/mmdeploy/cudnn/cudnn-11.3-linux-x64-v8.2.1.32/cuda
export LD_LIBRARY_PATH=$CUDNN_DIR/lib64:$LD_LIBRARY_PATH

5. 编译MMDeploy

git clone https://github.com/open-mmlab/mmdeploy.git
cd mmdeploy
git submodule update --init --recursive
# # .bashrc中添加环境变量
export MMDEPLOY_DIR=/home/txz/MMD/mmdeploy/mmdeploy
source ~/.bashrc
mkdir -p build && cd build
cmake -DCMAKE_CXX_COMPILER=g++-9 -DMMDEPLOY_TARGET_BACKENDS=trt -DTENSORRT_DIR=${TENSORRT_DIR} -DCUDNN_DIR=${CUDNN_DIR} ..
make -j$(nproc) && make install
cd ${MMDEPLOY_DIR}
pip install -e .

6. 编译SDK

cd ${MMDEPLOY_DIR}
mkdir -p build && cd build
cmake .. \
    -DCMAKE_CXX_COMPILER=g++-9 \
    -DMMDEPLOY_BUILD_SDK=ON \
    -DMMDEPLOY_BUILD_EXAMPLES=ON \
    -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON \
    -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
    -DMMDEPLOY_TARGET_BACKENDS=trt \
    -Dpplcv_DIR=${PPLCV_DIR}/cuda-build/install/lib/cmake/ppl \
    -DTENSORRT_DIR=${TENSORRT_DIR} \
    -DCUDNN_DIR=${CUDNN_DIR}
make -j$(nproc) && make install

7.Python使用后端推理

# 导入库
import time
import torch
import numpy as np
from mmdeploy.utils import get_input_shape, load_config
from mmdeploy.apis.utils import build_task_processor

# 加载配置
device = "cuda:0"
model_cfg = "/home/txz/MMD/mmdeploy/mmdeploy/checkpoints_mmdet3d/hv_pointpillars_secfpn_6x8_160e_kitti-3d-3class.py"
deploy_cfg = "/home/txz/MMD/mmdeploy/mmdeploy/configs/mmdet3d/voxel-detection/voxel-detection_tensorrt_dynamic-kitti-32x4.py"
deploy_cfg, model_cfg = load_config(deploy_cfg, model_cfg)
task_processor = build_task_processor(model_cfg, deploy_cfg, device)

# 加载后端
backend_files = ["/home/txz/MMD/mmdeploy/mmdeploy/work_dir/end2end.engine"]
model = task_processor.init_backend_model(backend_files)

# 初次获得输入
points = "/home/txz/MMD/mmdeploy/mmdeploy/checkpoints_mmdet3d/test.bin"
input_shape = get_input_shape(deploy_cfg)
model_inputs, _ = task_processor.create_input(points, input_shape


# 后续的输入
points = np.fromfile("/home/txz/MMD/mmdeploy/mmdeploy/checkpoints_mmdet3d/test.bin", dtype=np.float32).reshape(-1,4)
while 1:
    start = time.time_ns()
    input_shape = get_input_shape(deploy_cfg)
    model_inputs['points'][0][0] = torch.tensor(points, device='cuda:0')
    # model_inputs, _ = task_processor.create_input(points, input_shape)
    with torch.no_grad():
    	result = task_processor.run_inference(model, model_inputs)
    end = time.time_ns()
    fps = 1 / ((end - start)/10**9)
    fps = round(fps, 2)
    print("FPS : ", fps)

部署centerpoin

pip3 install mmsegmentation==1.0.0rc0
pip3 install mmdet==3.0.0rc1
mim install mmcv==2.0.0rc1