使用cuda和tensorrt加速对比-C++部署pytorch模型

1.网络模型和数据

SSD网络，图片大小（w,h)=(480, 640)

SSD检测结果

2.cuda加速

处理一张图片的时间

[ 0 ] 695.201 ms.
[ 1 ] 42.9589 ms.
[ 2 ] 42.2552 ms.
[ 3 ] 40.0333 ms.
[ 4 ] 40.5067 ms.
[ 5 ] 42.2043 ms.
[ 6 ] 42.2497 ms.
[ 7 ] 43.8587 ms.
[ 8 ] 41.9123 ms.
[ 9 ] 42.1796 ms.
[ 10 ] 43.3248 ms.
[ 11 ] 43.942 ms.
[ 12 ] 44.5019 ms.
[ 13 ] 42.3113 ms.
[ 14 ] 42.4571 ms.
[ 15 ] 43.9381 ms.
[ 16 ] 42.5341 ms.
[ 17 ] 44.3687 ms.
[ 18 ] 42.5123 ms.
[ 19 ] 42.4622 ms.
[ 20 ] 44.3038 ms.
[ 21 ] 44.1808 ms.
[ 22 ] 42.5432 ms.
[ 23 ] 43.9668 ms.
[ 24 ] 42.8493 ms.
[ 25 ] 44.0818 ms.
[ 26 ] 44.467 ms.
[ 27 ] 44.26 ms.
[ 28 ] 42.7377 ms.
[ 29 ] 42.4226 ms.
[ 30 ] 43.9711 ms.

3. tensorRT加速

[ 0 ] 119.975 ms.
[ 1 ] 33.7516 ms.
[ 2 ] 25.3176 ms.
[ 3 ] 21.5865 ms.
[ 4 ] 21.6411 ms.
[ 5 ] 21.0087 ms.
[ 6 ] 19.676 ms.
[ 7 ] 21.0764 ms.
[ 8 ] 20.6931 ms.
[ 9 ] 22.0367 ms.
[ 10 ] 22.5404 ms.
[ 11 ] 21.7802 ms.
[ 12 ] 21.2304 ms.
[ 13 ] 21.7144 ms.
[ 14 ] 21.545 ms.
[ 15 ] 20.5097 ms.
[ 16 ] 22.1281 ms.
[ 17 ] 19.8469 ms.
[ 18 ] 19.8201 ms.
[ 19 ] 20.6956 ms.
[ 20 ] 21.94 ms.
[ 21 ] 21.839 ms.
[ 22 ] 20.6588 ms.
[ 23 ] 21.4913 ms.
[ 24 ] 20.9667 ms.
[ 25 ] 20.0627 ms.
[ 26 ] 20.0018 ms.
[ 27 ] 19.7089 ms.
[ 28 ] 19.6951 ms.
[ 29 ] 20.2393 ms.
[ 30 ] 21.5587 ms.