ROS yolo加速 ros_openvino_toolkit安装教程
ROS yolo加速 ros_openvino_toolkit安装教程
下载地址：https://github.com/intel/ros_openvino_toolkit
平台:intel ncs 2
系统:ubuntu 16.04
准备条件：
- ubuntu16 系统 :支持处理器：(第6代 - 第8代Intel®Core™、英特尔®至强®v5系列、英特尔®至强®v6系列)
- ROS Kinetic ，ROS Kinetic Desktop-Full
- OpenVINO™工具包开源：深度学习部署工具包和Open Model Zoo。
- RGB相机，例如RealSense D400系列或标准USB相机或视频/图像文件
环境设置
1.安装ROS
- 对于Ubuntu16.04，安装ROS Kinetic Desktop-Full（指南）
- 对于Ubuntu18.04，安装ROS Melodic Desktop-Full （指南）
2.安装OpenVINO™Toolkit开源
- Install OpenCV 3.x: 3.4 or later(guide)
sudo apt-get install build-essential
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
cd ~/code
git clone https://github.com/opencv/opencv.git
git clone https://github.com/opencv/opencv_contrib.git
cd opencv && git checkout 3.4.2 && cd ..
cd opencv_contrib && git checkout 3.4.2 && cd ..
cd opencv
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -D OPENCV_EXTRA_MODULES_PATH=$HOME/code/opencv_contrib/modules/ ..
make -j8
sudo make install- Install OpenCL Driver for GPU(guide)
cd ~/Downloads
wget http://registrationcenter-
download.intel.com/akdlm/irc_nas/11396/SRB5.0_linux64.zip
unzip SRB5.0_linux64.zip -d SRB5.0_linux64
cd SRB5.0_linux64
sudo apt-get install xz-utils
mkdir intel-opencl
tar -C intel-opencl -Jxf intel-opencl-r5.0-63503.x86_64.tar.xz
tar -C intel-opencl -Jxf intel-opencl-devel-r5.0-63503.x86_64.tar.xz
tar -C intel-opencl -Jxf intel-opencl-cpu-r5.0-63503.x86_64.tar.xz
sudo cp -R intel-opencl/* /
sudo ldconfig- Install Deep Learning Deployment Toolkit(guide)
mkdir ~/code && cd ~/code
git clone https://github.com/opencv/dldt.git
cd dldt/inference-engine/
git checkout 2018_R4
./install_dependencies.sh
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j8
sudo mkdir -p /opt/openvino_toolkit
sudo ln -s ~/code/dldt /opt/openvino_toolkit/dldt- Install Open Model Zoo(guide)
cd ~/code
git clone https://github.com/opencv/open_model_zoo.git
cd open_model_zoo/demos/
git checkout 2018_R4
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release /opt/openvino_toolkit/dldt/inference-engine
make -j8
sudo mkdir -p /opt/openvino_toolkit
sudo ln -s ~/code/open_model_zoo /opt/openvino_toolkit/open_model_zoo注意：很可能遇到git下载不全的问题，另外CMAKE 版本需要升级到3.5以上。
3.安装Intel® RealSense™ SDK 2.0 (tag v2.17.1)
参考地址：https://github.com/IntelRealSense/librealsense/blob/v2.17.1/doc/distribution_linux.md
Install from source code(Recommended)
具体步骤：
- 导航到librealsense根目录以运行以下脚本。拔下所有已连接的英特尔实感摄像头。(https://github.com/IntelRealSense/librealsense/tree/v2.17.1)
- 安装核心软件包：sudo apt-get install git libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
Ubuntu 16：sudo apt-get install libglfw3-dev
Ubuntu 18：sudo apt-get install libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev - 安装位于librealsense源目录中的Intel Realsense权限脚本：
sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/ sudo udevadm control --reload-rules && udevadm trigger - 构建和应用修补的内核模块：(这边需要看自己的内核)
./scripts/patch-realsense-ubuntu-lts.shPS:更新内核失败时，可以参考这篇文章。https://blog.csdn.net/taiyangwangzi/article/details/89195714
- TM1-specific:
echo 'hid_sensor_custom' | sudo tee -a /etc/modules
- librealsense2 SDK
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-5 g++-5
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 60 --slave /usr/bin/g++ g++ /usr/bin/g++-5
sudo update-alternatives --set gcc "/usr/bin/gcc-5"
mkdir build && cd build
cmake ../ -DBUILD_EXAMPLES=true -DCMAKE_BUILD_TYPE=Release
make && sudo make install
sudo make uninstall && make clean && make && sudo make installPS 可以制定运行内核数目：sudo make uninstall && make clean && make **-j8** && sudo make install
Remove all RealSense™ SDK-related packages with:
dpkg -l | grep "realsense" | cut -d " " -f 3 | xargs sudo dpkg --purge
4.其他依赖
# numpy and networkx
pip3 install numpy
pip3 install networkx
# libboost
sudo apt-get install -y --no-install-recommends libboost-all-dev
cd /usr/lib/x86_64-linux-gnu
sudo ln -s libboost_python-py35.so libboost_python3.so编译和安装
设置ENV InferenceEngine_DIR，CPU_EXTENSION_LIB和GFLAGS_LIB
export InferenceEngine_DIR = / opt / openvino_toolkit / dldt / inference-engine / build /
export CPU_EXTENSION_LIB = / opt / openvino_toolkit / dldt / inference-engine / bin / intel64 / Release / lib / libcpu_extension.so
export GFLAGS_LIB = / opt / openvino_toolkit / dldt /inference-engine/bin/intel64/Release/lib/libgflags_nothreads.a- 安装ROS_OpenVINO包
mkdir -p ~ / catkin_ws / src
cd ~ / catkin_ws / src
git clone https://github.com/intel/ros_openvino_toolkit
git clone https://github.com/intel/object_msgs
git clone https://github.com/ ros-perception / vision_opencv
git clone https://github.com/intel-ros/realsense cd realsense
git checkout 2.1.3- 构建包
# Ubuntu 16.04
source /opt/ros/kinetic/setup.bash
# Ubuntu 18.04
source /opt/ros/melodic/setup.bash
cd ~/catkin_ws
catkin_make
source devel/setup.bash
sudo mkdir -p /opt/openvino_toolkit
sudo ln -s ~/catkin_ws/src/ros_openvino_toolkit /opt/openvino_toolkit/ros_openvino_toolkitROS接口和输出
话题
订阅主题
- 图片主题： /camera/color/image_raw（sensor_msgs :: Image）
发布主题
- 人脸检测： /ros_openvino_toolkit/face_detection（object_msgs :: ObjectsInBoxes）
- 情感识别： /ros_openvino_toolkit/emotions_recognition（people_msgs :: EmotionsStamped）
- 年龄和性别认可： /ros_openvino_toolkit/age_genders_Recognition（people_msgs :: AgeGenderStamped）
- 头部姿势估计： /ros_openvino_toolkit/headposes_estimation（people_msgs :: HeadPoseStamped）
- 对象检测： /ros_openvino_toolkit/detected_objects（object_msgs :: ObjectsInBoxes）
- 对象分割： /ros_openvino_toolkit/segmented_obejcts（people_msgs :: ObjectsInMasks）
- 人员重新识别： /ros_openvino_toolkit/reidentified_persons（people_msgs :: ReidentificationStamped）
- Rviz输出： /ros_openvino_toolkit/image_rviz（sensor_msgs :: Image）
服务
- 对象检测服务： /detect_object（object_msgs :: DetectObject）
- 人脸检测服务： /detect_face（object_msgs :: DetectObject）
- 年龄和性别检测服务： /detect_age_gender（people_msgs :: AgeGender）
- 头部检测服务： /detect_head_pose（people_msgs :: HeadPose）
- 情绪检测服务： /detect_emotion（people_msgs :: Emotion）
RViz
具有推理结果的原始图像帧的合成主题也支持RViz显示。要在RViz工具中显示，请添加带有合成主题的图像标记：/ros_openvino_toolkit/image_rviz（sensor_msgs :: Image）
模型等下载
- 下载并转换经过训练的模型以生成模型的优化中间表示（IR）
#object segmentation model
cd /opt/openvino_toolkit/dldt/model-optimizer/install_prerequisites
sudo ./install_prerequisites.sh
mkdir -p ~/Downloads/models
cd ~/Downloads/models
wget http://download.tensorflow.org/models/object_detection/mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
tar -zxvf mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
cd mask_rcnn_inception_v2_coco_2018_01_28
python3 /opt/openvino_toolkit/dldt/model-optimizer/mo_tf.py --input_model frozen_inference_graph.pb --tensorflow_use_custom_operations_config /opt/openvino_toolkit/dldt/model-optimizer/extensions/front/tf/mask_rcnn_support.json --tensorflow_object_detection_api_pipeline_config pipeline.config --reverse_input_channels --output_dir ./output/
sudo mkdir -p /opt/models
sudo ln -s ~/Downloads/models/mask_rcnn_inception_v2_coco_2018_01_28 /opt/models/
#object detection model
cd /opt/openvino_toolkit/open_model_zoo/model_downloader
python3 ./downloader.py --name mobilenet-ssd
#FP32 precision model
sudo python3 /opt/openvino_toolkit/dldt/model-optimizer/mo.py --input_model /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32 --mean_values [127.5,127.5,127.5] --scale_values [127.5]
#FP16 precision model
sudo python3 /opt/openvino_toolkit/dldt/model-optimizer/mo.py --input_model /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16 --data_type=FP16 --mean_values [127.5,127.5,127.5] --scale_values [127.5]- 下载模型的优化中间表示（IR）（执行一次）
cd /opt/openvino_toolkit/open_model_zoo/model_downloader
python3 downloader.py --name face-detection-adas-0001
python3 downloader.py --name age-gender-recognition-retail-0013
python3 downloader.py --name emotions-recognition-retail-0003
python3 downloader.py --name head-pose-estimation-adas-0001
python3 downloader.py --name person-detection-retail-0013
python3 downloader.py --name person-reidentification-retail-0076- 复制标签文件（执行一次）
sudo cp /opt/openvino_toolkit/ros_openvino_toolkit/data/labels/emotions-recognition/FP32/emotions-recognition-retail-0003.labels /opt/openvino_toolkit/open_model_zoo/model_downloader/Retail/object_attributes/emotions_recognition/0003/dldt
sudo cp /opt/openvino_toolkit/ros_openvino_toolkit/data/labels/face_detection/face-detection-adas-0001.labels /opt/openvino_toolkit/open_model_zoo/model_downloader/Transportation/object_detection/face/pruned_mobilenet_reduced_ssd_shared_weights/dldt
sudo cp /opt/openvino_toolkit/ros_openvino_toolkit/data/labels/object_segmentation/frozen_inference_graph.labels /opt/models/mask_rcnn_inception_v2_coco_2018_01_28/output
sudo cp /opt/openvino_toolkit/ros_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32
sudo cp /opt/openvino_toolkit/ros_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/openvino_toolkit/open_model_zoo/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16- set ENV LD_LIBRARY_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/openvino_toolkit/dldt/inference-engine/bin/intel64/Release/lib使用方法：
- 从StandardCamera运行面部检测示例代码输入。
roslaunch vino_launch pipeline_people_oss.launch
- 从Image运行面部检测示例代码输入。
roslaunch vino_launch pipeline_image_oss.launch
- 从RealsensCamera输入运行对象检测示例代码。
roslaunch vino_launch pipeline_object_oss.launch
- 从RealsensCameraTopic运行对象检测示例代码。
roslaunch vino_launch pipeline_object_oss_topic.launch
- 从RealSenseCameraTopic运行对象分割示例代码输入。
roslaunch vino_launch pipeline_segmentation.launch
- 从视频中运行对象分割示例代码输入。
roslaunch vino_launch pipeline_video.launch
- 从StandardCamera运行人员reidentification示例代码输入。
roslaunch vino_launch pipeline_reidentification_oss.launch
- 从Image
Run图像处理服务输入运行对象检测服务示例代码：
roslaunch vino_launch image_object_server_oss.launch
在另一个控制台上运行带有图像绝对路径的示例应用程序：
rosrun dynamic_vino_sample image_object_client ~ /catkin_ws/src/ros_openvino_toolkit/data/images/car.png
- 从Image
Run图像处理服务运行面部检测服务示例代码输入：
roslaunch vino_launch image_people_server_oss.launch
在另一个控制台上运行带有图像绝对路径的示例应用程序：
rosrun dynamic_vino_sample image_people_client ~ /catkin_ws/src/ros_openvino_toolkit/data/images/team.jpg