Jetson Orin NX 开发指南（4）: 安装 cuda 和 realsense

一、硬件说明

可以在官网查看各个类型的相机说明

Intel® RealSense™ Computer Vision - Depth and Tracking cameras

目前实验室常用的是 D400 系列的相机，如 D435i相机，D455相机等，本文主要使用 D435i 相机

D435i 相机是一个RGB-D相机，也就是我们常说的深度相机，目前深度相机的实现原理主要分为三种：分别是结构光，Tof，双目成像。

Realsense 使用的是结构光的方案。正面的四个摄像头，从左向右以次是左红外相机，红外点阵投射仪，右红外相机，和RGB相机。

二、安装 cuda

由于 Jetson 的 CPU 性能较差，因此安装 Realsense 前建议先安装 cuda，从而在启动 Realsense 时可以通过 GPU 进行加速，因此这里先安装 cuda，由于我们在烧录 Jetson Orin NX 开发板时安装了相关组件，其中就包括 cuda 组件，因此可以直接通过 apt-get 安装 cuda，终端输入

sudo apt-get update
sudo apt-get install cuda-toolkit-11-4

安装完成后查看是否已经 export 安装路径

如果没有这两行则加入这两行

export PATH=/usr/local/cuda-11.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH

添加之后 source 一下，使其生效

source ~/.bashrc

最后检查 cuda 是否安装成功，终端输入

nvcc -V

这表明 cuda 安装成功！

三、安装 realsense 驱动

Jetson 系列有专门针对 realsense 安装方式，当然传统的 realsense 安装方式也是可行的

3.1 传统安装 librealsense

传统的安装方式可以参考下文

Ubuntu 20.04 配置 realsense_ubuntu安装realsense2-CSDN博客Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394Ubuntu 20.04 配置 realsense_ubuntu安装realsense2https://blog.csdn.net/qq_44998513/article/details/131517394

3.2 Jetson 方法安装 librealsense

但是这里我们采用专门针对 Jetson 的方式进行安装 librealsense，参考

GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development KitsInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDKInstall the Intel RealSense SDK on the NVIDIA Jetson Development Kits - GitHub - jetsonhacks/installRealSenseSDK: Install the Intel RealSense SDK on the NVIDIA Jetson Development Kitshttps://github.com/jetsonhacks/installRealSenseSDK

首先从 github 上下载安装包，终端输入

git clone https://github.com/jetsonhacks/installRealSenseSDK.git

下载完成后，进入安装包并开始安装，终端输入

cd ./installRealSenseSDK
./buildLibrealsense.sh

至此 Librealsense 就安装成功了，最后测试 realsense 驱动是否能够正常运行。

首先将 realsense 相机连接上 Jetson Orin NX 开发板（注意这里必须使用 USB 3.0 的数据线连接），然后终端输入

realsense-viewer

正常显示图片

这表明 Realsense 驱动安装成功！

四、安装 realsense-ros

作用：将 realsense 数据转换为 ros 数据并发布到 ros 话题中，相当于realsense 和 ros 的桥梁。

参考 realsense-ros 的 github 官方教程：

GitHub - IntelRealSense/realsense-ros at ros1-legacy

官方提供了直接方式和源码方式两种方法安装 realsense-ros

4.1 直接方式安装

终端输入：

sudo apt-get install ros-$ROS_DISTRO-realsense2-camera

其中 $ROS_DISTRO 为自己的 ros 版本， Ubuntu 18.04 对应 melodic，Ubuntu 20.04 对应 noetic，这种方法的安装路径为

/opt/ros/noetic/share/realsense2_camera/

4.2 源码方式安装

创建工作目录

mkdir -p ~/catkin_pkg/src/
cd ~/catkin_pkg/src/

下载源码

git clone https://github.com/IntelRealSense/realsense-ros.git
cd realsense-ros/
git checkout `git tag | sort -V | grep -P "^2.\d+\.\d+" | tail -1`
cd ..

请确保安装了 ros 软件包 ddynamic_reconfigure， 如果没有安装，可以通过 apt 安装

sudo apt-get install ros-$ROS_DISTRO-ddynamic-reconfigure

或者将其源码下载到当前工作目录

git clone https://github.com/pal-robotics/ddynamic_reconfigure.git

注意，这里如果通过 apt 安装了 ddynamic_reconfigure 就不要再下载源码到工作空间一起编译了，可能会冲突报错。

接下来编译工作空间

catkin_init_workspace
cd ..
catkin_make clean
catkin_make -DCATKIN_ENABLE_TESTING=False -DCMAKE_BUILD_TYPE=Release
catkin_make install

最后将其加到 ~/.basrc 文件中

echo "source ~/catkin_pkg/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

这两种方法选择一种安装就可以。

测试 realsense-ros 是否安装成功，通过 ros 启动相机，终端输入

roslaunch realsense2_camera rs_camera.launch

如果发现报错
[camera/realsense2_camera_manager-2] process has died [pid 3282, exit code 127, cmd /opt/ros/noetic/lib/nodelet/nodelet manager __name:=realsense2_camera_manager __log:=/home/onx001/.ros/log/064220d6-6818-11ee-82cd-347de4babe3e/camera-realsense2_camera_manager-2.log].

参考

No GPU usage on jetson xavier NX · Issue #2326 · IntelRealSense/realsense-ros · GitHub

librealsense2 camera.so: undefined symbol:ZN2cV3MatC1EV解决笔记记录-CSDN博客

查看能否收到相机数据。

 解决方案

修改 CMakeLists.txt 文件

添加 OpenCV 功能包，由于安装 ROS 时自带 OpenCV，因此不需要另外安装 OpenCV，直接修改 CMakeLists.txt 文件即可

上面红字为添加的内容，也可以复制下面写的

find_package(catkin REQUIRED COMPONENTS
    message_generation
    nav_msgs
    roscpp
    sensor_msgs
    std_msgs
    std_srvs
    nodelet
    cv_bridge
    image_transport
    tf
    ddynamic_reconfigure
    diagnostic_updater
    OpenCV
    )

include_directories(
    include
    ${realsense2_INCLUDE_DIR}
    ${catkin_INCLUDE_DIRS}
    ${OpenCV_INCLUDE_DIRS}
    )

target_link_libraries(${PROJECT_NAME}
    ${realsense2_LIBRARY}
    ${catkin_LIBRARIES}
    ${CMAKE_THREAD_LIBS_INIT}
    ${OpenCV_LIBRARIES}
    )

重新测试 realsense-ros 是否安装成功，通过 ros 启动相机，终端输入

roslaunch realsense2_camera rs_camera.launch

终端输入 rostopic list 显示话题

onx001@onx001:~$ rostopic list 
/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressed/parameter_descriptions
/camera/color/image_raw/compressed/parameter_updates
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/compressedDepth/parameter_descriptions
/camera/color/image_raw/compressedDepth/parameter_updates
/camera/color/image_raw/theora
/camera/color/image_raw/theora/parameter_descriptions
/camera/color/image_raw/theora/parameter_updates
/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/motion_module/parameter_descriptions
/camera/motion_module/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/auto_exposure_roi/parameter_descriptions
/camera/rgb_camera/auto_exposure_roi/parameter_updates
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/diagnostics
/rosout
/rosout_agg
/tf
/tf_static
onx001@onx001:~$ 

这表明 realsense-ros 安装成功！

4.3 修改配置文件

一般来说，安装完成后可能会出现 ros 能接收到话题但是收不到数据的情况。

这时需要修改 ~/catkin_pkg/src/realsense2_camera/rs_camera.launch 文件。

先备份 rs_camera.launch 文件，终端输入

cd ~/catkin_pkg/src/realsense-ros/realsense2_camera/launch
cp ./rs_camera.launch ./rs_camera.launch.bak

然后将 rs_camera.launch 文件内容替换为如下内容

  
  
  
  
  
  
  
  
  
  

  
  
  

  
  
  

  
  
  
  

  
  
  
  
  
  

  
  
  

  
  
  
  
  
  
  
  

  
  
  
  
  

  
  

  
  

  
  
  
  
  
  
  
  
  
  

  
  
  
  
  
  
  
	
  
  
    /camera/stereo_module/emitter_enabled: 0
  
  
    /camera/stereo_module/emitter_enabled: 1
  
  

  
    
      
      
      
      
      
      
      
      
      

      
      
      
      
      

      
      
      

      
      
      

      
      
      
      

      
      
      

      
      
      
      
      
      

      
      
      
      
      
      
      
      

      
      

      
      
      
      
      
      
      
      
      
      
      
      
      
      

      
      
      
    
  

最后终端输入

roslaunch realsense2_camera rs_camera.launch

此时输出 imu 等信息，并且屏蔽了结构光

onx001@onx001:~$ rostopic list 
/camera/accel/imu_info
/camera/gyro/imu_info
/camera/imu
/camera/infra1/camera_info
/camera/infra1/image_rect_raw
/camera/infra1/image_rect_raw/compressed
/camera/infra1/image_rect_raw/compressed/parameter_descriptions
/camera/infra1/image_rect_raw/compressed/parameter_updates
/camera/infra1/image_rect_raw/compressedDepth
/camera/infra1/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra1/image_rect_raw/compressedDepth/parameter_updates
/camera/infra1/image_rect_raw/theora
/camera/infra1/image_rect_raw/theora/parameter_descriptions
/camera/infra1/image_rect_raw/theora/parameter_updates
/camera/infra1/metadata
/camera/infra2/camera_info
/camera/infra2/image_rect_raw
/camera/infra2/image_rect_raw/compressed
/camera/infra2/image_rect_raw/compressed/parameter_descriptions
/camera/infra2/image_rect_raw/compressed/parameter_updates
/camera/infra2/image_rect_raw/compressedDepth
/camera/infra2/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra2/image_rect_raw/compressedDepth/parameter_updates
/camera/infra2/image_rect_raw/theora
/camera/infra2/image_rect_raw/theora/parameter_descriptions
/camera/infra2/image_rect_raw/theora/parameter_updates
/camera/infra2/metadata
/camera/motion_module/parameter_descriptions
/camera/motion_module/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/auto_exposure_roi/parameter_descriptions
/camera/stereo_module/auto_exposure_roi/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates
/diagnostics
/rosout
/rosout_agg
/tf
/tf_static

至此，Realsense 驱动就安装完成了，并且实现了与 ROS 系统的连接！