YOLOV5之TensorRT模型部署

目录

一、环境版本

1.1 概述

1.2 Ubuntu版本

1.3 python、pytorch与tensorrt版本

1.4 cudnn版本

1.5  驱动版本

1.6 opencv版本

二、TensorRT与opencv环境安装

2.1 opencv安装

2.2 TensorRT安装

三、yolov5转tensorrt

3.1 下载yolov5项目

3.2 下载tensorrtx项目

3.3 开始准备编译

3.4 获取wts模型

3.5 编译命令

3.6 测试

四、附录

---

一、环境版本

1.1 概述

系统环境：Ubuntu18.04  python=3.8.5  torch=1.10.0  TensorRT=7.2.3.4  opencv=4.5.1

cuda=11.2   cudnn=8.2.2    Driver Version: 460.32.03   显卡：RTX 3090

1.2 Ubuntu版本

在终端运行： lsb_release -a

1.3 python、pytorch与tensorrt版本

在终端运行： lsb_release -a

1.4 cudnn版本

在终端运行： dpkg -L libcudnn8

​​

1.5  驱动版本

在终端运行： watch -n 1 nvidia-smi

1.6 opencv版本

在终端运行：  pkg-config --modversion opencv

​

二、TensorRT与opencv环境安装

2.1 opencv安装

1. 可参考网：Home - OpenCV
2. ippicv包：https://github.com/opencv/opencv_3rdparty/tree/ippicv/master_20191018/ippicv
3. 编译命令：cmake -D CMAKE_INSTALL_PREFIX=/usr/local -D CMAKE_BUILD_TYPE=Release -D BUILD_TIFF=ON

注：在conda环境中，由于安装pytorch，会自动安装“libtiff”，卸载后导致pytorch不能用，然后导致opencv在编译过程中会报错，具体如下：

/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFReadRGBAStrip@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFReadDirectory@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFWriteEncodedStrip@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFIsTiled@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFWriteScanline@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFGetField@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFScanlineSize@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFWriteDirectory@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFReadEncodedTile@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFReadRGBATile@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFClose@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFClientOpen@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFRGBAImageOK@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFOpen@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFReadEncodedStrip@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFSetField@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFSetWarningHandler@LIBTIFF_4.0'
/usr/local/lib/libopencv_imgcodecs.so.4.5.1: undefined reference to `TIFFSetErrorHandler@LIBTIFF_4.0

解决方法：

1. opencv源码安装编译时候，保证conda环境没有libtiff包名
2. 在cmake 编译过程中开启 BUILD_TIFF

参考链接：

1. LIBTIFF_4.0 link errors - OpenCV Q&A Forum
2. https://stackoverflow.com/questions/23934386/using-opencv-with-libtiff
3. OpenCV LIBTIFF_4.0 link errors - 普兒 - 博客园
4. Linux（Ubuntu 18.04）中安装OpenCV + OpenCV_Contrib - 花落未殇 - 博客园
5. https://blog.51cto.com/u_14587161/3000682
6. https://blog.csdn.net/jiaken2660/article/details/117036606

2.2 TensorRT安装

1. 可参考网：https://developer.nvidia.com/zh-cn/tensorrt，有tar包和deb两种安装方，本文选择tar包进行解压安装。
2. 根据tar包解压安装：tar -xvf TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1.tar.gz
3. 安装命令：分别进入python、uff和graphsurgeon根据python环境，进行pip安装相应的包
4. vi ~/.bashrc 添加 export LD_LIBRARY_PATH=/usr/local/TensorRT-7.2.3.4/lib:/usr/local/cuda/bin:$LD_LIBRARY_PATH
   export INCLUDE_PATH=/usr/local/TensorRT-7.2.3.4/include:/usr/local/cuda/lib64:$INCLUDE_PATH
5. source ~/.bashrc

三、yolov5转tensorrt

3.1 下载yolov5项目

GitHub - ultralytics/yolov5: YOLOv5 in PyTorch > ONNX > CoreML > TFLite

3.2 下载tensorrtx项目

https://github.com/wang-xinyu/tensorrtx

3.3 开始准备编译

在tensorrx项目中，进入yolov5目录，修改Cmakelists.txt:

修改自己tcuda和tensorrt路径：

# cuda

include_directories(/usr/local/cuda/include)

link_directories(/usr/local/cuda/lib64)

# tensorrt

include_directories(/usr/local/TensorRT-7.2.3.4/include)

link_directories(/usr/local/TensorRT-7.2.3.4/lib)

3.4 获取wts模型

在终端运行： python gen_wts.py 

3.5 编译命令

在终端运行：mkdir build && cd build:

在终端运行：cmake ..

 在终端运行：make -j128

在终端运行：./yolov5 -s    // serialize model to plan file i.e. 'yolov5s.engine' 序列化模型

3.6 测试

在终端运行：./yolov5 -d ../data/images  // deserialize plan file and run inference, the images in samples will be processed. 测试用例

 或者在终端运行：CUDA_VISIBLE_DEVICES=6 python yolov5_trt.py

四、附录

1. https://blog.csdn.net/xingtianyao/article/details/111387274
2. https://blog.csdn.net/mathlxj/article/details/107810548
3. https://www.cnblogs.com/hyj74-18/p/10218740.html
4. OpenCV fails to link libtiff when build with Matlab - Stack Overflow
5. https://blog.csdn.net/zong596568821xp/article/details/86077553
6. https://www.icode9.com/content-4-782484.html
7. https://blog.csdn.net/u013372308/article/details/100168651
8. https://www.pytorch123.com/
9. https://positive.blog.csdn.net/article/details/118735018
10. https://zhuanlan.zhihu.com/p/159591904
11. https://blog.csdn.net/linghu8812/article/details/109322729
12. https://blog.csdn.net/qq_42722197/article/details/119708795