图像处理系列05——Jetson Xavier NX平台JPEG硬解

引言

Jetson Xavier NX提供了Jpeg硬解的能力，其在Multimedia APIs中介绍了如何使用Jpeg decode，具体流程如下，示例代码可以在 /usr/src/jetson_multimedia_api/samples/06_jpeg_decode中找到。

用例

以下代码实现了如何利用硬解将Jpeg格式图片解码到cv的Mat格式，输入是四个ROS compressedImg 消息（Jpeg格式压缩），由于需要内存拷贝和转换，效率并不高，且需要cv::cvtcolor()，也耗费CPU

主要步骤

1. 将消息数据读入char* buffer：`unsigned char *in_buffer = const_cast(Camera_image.data.data())`
2. 解码器解码：`jpegdec->decodeToFd(fd[i], in_buffer, in_file_size, pixfmt, width, height) < 0)`
3. 解码后的数据通过buffer转换拷贝到CPU，格式为RGBA（Nvidia论坛说只能转换到这个格式，有待考证和改进）
4. 再通过opencv转换成BGR的Mat

依赖

头文件：

• helper_cuda.h
• helper_string.h
• nvbuf_utils.h
• nvJPEG.hpp
• NvJpegDecoder.h
• 以上可能不完全，可包含/usr/src/jetson_multimedia_api/include/ 和 /usr/local/cuda-10.2/targets/aarch64-linux/include/

cpp：

• NvBuffer.cpp
• NvElement.cpp
• NvElementProfiler.cpp
• NvLogging.cpp
• NvJpegDecoder.cpp

代码

#include "NvJpegDecoder.h"
#include "nvbuf_utils.h"

#define WIDTH 1280
#define HEIGHT 720

int main(int argc, char* argv[]){
   NvJPEGDecoder *jpegdec;
   uint64_t in_file_size;
   unsigned char *in_buffer;
   jpegdec = NvJPEGDecoder::createJPEGDecoder("jpegdec");
   
   const sensor_msgs::CompressedImageConstPtr Camera_msgs[4] = {Camera_msg, Camera_msg1, Camera_msg2, Camera_msg3};

   // Decode to YUV buffer
   uint32_t width, height, pixfmt;

   int fd[4] = {0};  // YUV DMA buffer after decode
   int  nvbuffer_ABGR_fd[4] = {0};  // One plane ABGR DMA buffer
   NvBufferCreateParams input_params = {0};
   input_params.width = WIDTH;
   input_params.height = HEIGHT;
   input_params.layout = NvBufferLayout_Pitch;
   input_params.colorFormat = NvBufferColorFormat_ABGR32;
   input_params.nvbuf_tag = NvBufferTag_NONE;
   
   for (int i = 0; i < 4; i++){
       const sensor_msgs::CompressedImage Camera_image = *Camera_msgs[i];
       in_file_size = Camera_image.data.size();
       in_buffer = const_cast<uchar *>(Camera_image.data.data());
       if (jpegdec->decodeToFd(fd[i], in_buffer, in_file_size, pixfmt, width, height) < 0)
           ROS_ERROR("Cannot decode MJPEG");

       /* Convert the decoded buffer to ABGR */
       NvBufferCreateEx(&nvbuffer_ABGR_fd[i], &input_params);  
       /* Init the NvBufferTransformParams */
       NvBufferTransformParams transParams;
       memset(&transParams, 0, sizeof(transParams));
       transParams.transform_flag = NVBUFFER_TRANSFORM_FILTER;
       transParams.transform_filter = NvBufferTransform_Filter_Smart;
       if (-1 == NvBufferTransform(fd[i], nvbuffer_ABGR_fd[i], &transParams))
           ROS_ERROR("Failed to convert the buffer");
       void *pdata = NULL;
       NvBufferParams params;
       NvBufferGetParams(nvbuffer_ABGR_fd[i], &params);
       NvBufferMemMap(nvbuffer_ABGR_fd[i], 0, NvBufferMem_Read, &pdata);
       NvBufferMemSyncForCpu(nvbuffer_ABGR_fd[i], 0, &pdata);
       // std::cout<
       cv::Mat imgbuf = cv::Mat(HEIGHT, WIDTH, CV_8UC4, pdata, params.pitch[0]);
       cv::Mat display_img;
       cvtColor(imgbuf, display_img, CV_RGBA2BGR);
       // cv::imshow("img", display_img);
       // cv::waitKey(0);
       NvBufferMemUnMap(nvbuffer_ABGR_fd[i], 0, &pdata);
   }
  
   for (int i = 0; i < 4; i++){
       NvBufferDestroy(nvbuffer_ABGR_fd[i]);
   }
   delete jpegdec;
}

注

对于桌面级GPU可以用 nvJPEG Libraries，Jetson Xavier NX平台没有该库

参考

https://docs.nvidia.com/jetson/l4t-multimedia/l4t_mm_06_jpeg_decode.html