【YOLOX训练部署】YOLOX ONNX 使用GPU进行推理

YOLOX训练自己的VOC数据集

【YOLOX训练部署】YOLOX训练自己的VOC数据集_乐亦亦乐的博客-CSDN博客

将自己训练的YOLOX权重转化成ONNX 并进行推理

【YOLOX训练部署】将自己训练的YOLOX权重转化成ONNX 并进行推理_乐亦亦乐的博客-CSDN博客

ONNX 在 CPU 上推理速度较慢，对比GPU效果，使用GPU对onnx进行推理。具体操作：

首先卸载onnxruntime，并安装onnxruntime-gpu

pip uninstall onnxruntime
pip install onnxruntime-gpu

# 注意到onnx官网查看onnx版本与cuda版本的对应关系

还是使用【YOLOX训练部署】将自己训练的YOLOX权重转化成ONNX 并进行推理_乐亦亦乐的博客-CSDN博客

中的onnx_inference_video.py 进行推理。

运行：

python onnx_inference_video.py -m /media/liqiang/新加卷/YOLOX/my_yolox_s.onnx -i ./4.mp4 -o /media/liqiang/新加卷/YOLOX -s 0.3 --input_shape 640,640

会出现如下问题：

解决：修改代码

session = onnxruntime.InferenceSession(
        args.model, providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'])

 完整推理代码：

'''
Descripttion: 
version: 
Author: LiQiang
Date: 2022-01-01 09:39:19
LastEditTime: 2022-01-01 10:23:07
'''
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.

import argparse
import os

import cv2
import numpy as np

import onnxruntime

from yolox.data.data_augment import preproc as preprocess
# from yolox.data.datasets import COCO_CLASSES
from yolox.data.datasets import VOC_CLASSES
from yolox.utils import mkdir, multiclass_nms, demo_postprocess, vis


def make_parser():
    parser = argparse.ArgumentParser("onnxruntime inference sample")
    parser.add_argument(
        "-m",
        "--model",
        type=str,
        default="yolox.onnx",
        help="Input your onnx model.",
    )
    parser.add_argument(
        "-i",
        "--video_path",
        type=str,
        # default='test_image.png',
        help="Path to your input image.",
    )
    parser.add_argument(
        "-o",
        "--output_dir",
        type=str,
        default='demo_output',
        help="Path to your output directory.",
    )
    parser.add_argument(
        "-s",
        "--score_thr",
        type=float,
        default=0.3,
        help="Score threshould to filter the result.",
    )
    parser.add_argument(
        "--input_shape",
        type=str,
        default="640,640",
        help="Specify an input shape for inference.",
    )
    parser.add_argument(
        "--with_p6",
        action="store_true",
        help="Whether your model uses p6 in FPN/PAN.",
    )
    return parser


if __name__ == '__main__':
    args = make_parser().parse_args()
    input_shape = tuple(map(int, args.input_shape.split(',')))
    # origin_img = cv2.imread(args.image_path)
    session = onnxruntime.InferenceSession(
        args.model, providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'])
    cap = cv2.VideoCapture(args.video_path)
    while True:
        ret, origin_img = cap.read()

        img, ratio = preprocess(origin_img, input_shape)

        ort_inputs = {session.get_inputs()[0].name: img[None, :, :, :]}
        output = session.run(None, ort_inputs)
        predictions = demo_postprocess(output[0], input_shape, p6=args.with_p6)[0]

        boxes = predictions[:, :4]
        scores = predictions[:, 4:5] * predictions[:, 5:]

        boxes_xyxy = np.ones_like(boxes)
        boxes_xyxy[:, 0] = boxes[:, 0] - boxes[:, 2]/2.
        boxes_xyxy[:, 1] = boxes[:, 1] - boxes[:, 3]/2.
        boxes_xyxy[:, 2] = boxes[:, 0] + boxes[:, 2]/2.
        boxes_xyxy[:, 3] = boxes[:, 1] + boxes[:, 3]/2.
        boxes_xyxy /= ratio
        dets = multiclass_nms(boxes_xyxy, scores, nms_thr=0.45, score_thr=0.1)
        if dets is not None:
            final_boxes, final_scores, final_cls_inds = dets[:, :4], dets[:, 4], dets[:, 5]
            origin_img = vis(origin_img, final_boxes, final_scores, final_cls_inds,
                             conf=args.score_thr, class_names=VOC_CLASSES)
        cv2.imshow('result', origin_img)
        c = cv2.waitKey(1)
        if c == 27:
            break
        # mkdir(args.output_dir)
        # output_path = os.path.join(args.output_dir, args.image_path.split("/")[-1])
        # cv2.imwrite(output_path, origin_img)

重新运行：

python onnx_inference_video.py -m /media/liqiang/新加卷/YOLOX/my_yolox_s.onnx -i ./4.mp4 -o /media/liqiang/新加卷/YOLOX -s 0.3 --input_shape 640,640

可以看出速度明显提升！