安装ros2_object_analytics(下文简称OA)一共用了我个三个工作日时间,主要是自己对系统构架不够熟悉并且调试能力也不够,所以将自己遇到的问题和安装流程记录下来,方便后来参考,另外这些问题是我在安装时按照当前版本的readme遇到的问题,因为版本还会不不断更新,更新之后可能问题就不存在了。
按照readme安装完OpenCV3.3编译OA的时候会遇到
/usr/bin/ld: cannot find -llz4
这应该OpenCV3.3的依赖问题,解决办法如下:
sudo apt-get install liblz4-dev
devel版本中的readme安装realsense的驱动的方法是
sudo apt-get install ros-crystal-librealsense2 ros-crystal-realsense-ros2-camera
而在后面运行的OA的时候,master分支没有问题,但是devel分支的话会发现没有点云信息,后来发现ros-crystal-librealsense2中的librealsense的SDK版本没有更新是2.16.5版本的,这个版本中发布的点云节点是’/camera/depth/color/points’,而不是’/camera/aligned_depth_to_color/color/points’,而OA中master版本订阅(映射)的点云节点是’/camera/depth/color/points’,devel中变成了’/camera/aligned_depth_to_color/color/points’,因此按照上述命令安装realsense驱动的话只有master版本是好用的,而devel版本就没法用了。
我的解决办法是,先卸载上面的两个包
sudo apt-get remove ros-crystal-librealsense2 ros-crystal-realsense-ros2-camera
按照Linux Distribution中安装distribution版本的librealsense SDK2.0,安装2.17.1的或者最新的都可以,然后将ros-crystal-realsense-ros2-camera放到工作空间中和OA一起编译,之后就能解决上述没有点云的问题
目前maste版本的OA通过NCS是可以成功运行的,但是devel版本,如果按照readme的方式一步一步安装的话,插入的硬件是NCS2的话会报错
[ERROR] [movidius_ncs_stream]: Error: there is no device at the given index or name
如果插入硬件NCS1的话会报错
[ERROR] [movidius_ncs_stream]: Error: an unexpected error was encontered during the function call
后来了解到OA之后可能不再支持NCS版本,因此我就没再话时间在调试NCS的问题上,转战OpenVINO.
下面我记录一的是一台新电脑的OA从无到有的一个可行的安装步骤,主要是参照ros2_object_analytics
和ros2_openvino_toolkit的readme,中间会指出一些容易出问题的地方
参看ros-crystal-desktop,应该不会有什么问题
sudo apt-get install ros-crystal-cv-bridge ros-crystal-object-msgs ros-crystal-image-transport ros-crystal-realsense-camera-msgs
参考Linux Distribution,应该没什么问题
因为我没有试没有不安转NCS会怎样,因此我还是按照我原来的步骤安装
# build ncsdk
mkdir ~/code
cd ~/code
git clone https://github.com/movidius/ncsdk
git clone https://github.com/movidius/ncappzoo
cd ~/code/ncsdk
make install
ln -sf ~/code/ncappzoo /opt/movidius/ncappzoo
# build ros2_intel_movidius_ncs
mkdir ~/ros2_ws/src -p
cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_intel_movidius_ncs.git
cd ~/ros2_ws
source /opt/ros/crystal/setup.bash
colcon build --symlink-install (Install python3-colcon-common-extensions by apt-get if colcon command not exist)
# build CNN model (Please plugin NCS device on the host while compiling)
cd /opt/movidius/ncappzoo/caffe/SSD_MobileNet
make
# Copy object label file to NCSDK installation location.
cp ~/ros2_ws/src/ros2_intel_movidius_ncs/data/labels/* /opt/movidius/ncappzoo/data/ilsvrc12/
mkdir ${HOME}/opencv
cd ${HOME}/opencv
git clone https://github.com/opencv/opencv.git -b 3.3.0
git clone https://github.com/opencv/opencv_contrib.git -b 3.3.0
mkdir opencv/build -p
cd opencv/build
cmake -DOPENCV_EXTRA_MODULES_PATH=${HOME}/opencv/opencv_contrib/modules \
-DCMAKE_INSTALL_PREFIX=/usr/local -DBUILD_opencv_cnn_3dobj=OFF ..
make -j8
sudo make install
sudo ldconfig
这里补上依赖
sudo apt-get install liblz4-dev
下面就属于安装OpenVINO的
Download OpenVINO toolkit by following the guide
cd ~/Downloads
wget -c http://registrationcenter-download.intel.com/akdlm/irc_nas/15078/l_openvino_toolkit_p_2018.5.455.tgz
Install OpenVINO toolkit by following the guide
cd ~/Downloads
tar -xvf l_openvino_toolkit_p_2018.5.455.tgz
cd l_openvino_toolkit_p_2018.5.455
# root is required instead of sudo
sudo -E ./install_cv_sdk_dependencies.sh
sudo ./install_GUI.sh
# build sample code under OpenVINO toolkit
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
cd /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/
mkdir build
cd build
cmake ..
make
Configure the Neural Compute Stick USB Driver
cd ~/Downloads
cat < 97-usbboot.rules
SUBSYSTEM=="usb", ATTRS{idProduct}=="2150", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
SUBSYSTEM=="usb", ATTRS{idProduct}=="2485", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
SUBSYSTEM=="usb", ATTRS{idProduct}=="f63b", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
EOF
sudo cp 97-usbboot.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger
sudo ldconfig
rm 97-usbboot.rules
cd /opt/intel/computer_vision_sdk/install_dependencies
sudo ./install_NEO_OCL_driver.sh
sudo add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main"
sudo apt update
sudo apt install libjasper1 libjasper-dev
#librealsense dependency
sudo apt-get install -y libssl-dev libusb-1.0-0-dev pkg-config libgtk-3-dev
sudo apt-get install -y libglfw3-dev libgl1-mesa-dev libglu1-mesa-dev
# numpy and networkx
pip3 install numpy
pip3 install networkx
# libboost
sudo apt-get install -y --no-install-recommends libboost-all-dev
cd /usr/lib/x86_64-linux-gnu
sudo ln -sf libboost_python-py35.so libboost_python3.so
Build sample code under openvino toolkit
# root is required instead of sudo
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
cd /opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/
mkdir build
cd build
cmake ..
make
set ENV CPU_EXTENSION_LIB and GFLAGS_LIB
export CPU_EXTENSION_LIB=/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib/libcpu_extension.so
export GFLAGS_LIB=/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib/libgflags_nothreads.a
Install ROS2_OpenVINO packages
mkdir -p ~/ros2_overlay_ws/src
cd ~/ros2_overlay_ws/src
git clone https://github.com/intel/ros2_openvino_toolkit
git clone https://github.com/intel/ros2_object_msgs
git clone https://github.com/ros-perception/vision_opencv -b ros2
git clone https://github.com/ros2/message_filters.git
git clone https://github.com/ros-perception/image_common.git -b ros2
git clone https://github.com/intel/ros2_intel_realsense.git
Build package
source ~/ros2_ws/install/local_setup.bash
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
export OpenCV_DIR=$HOME/code/opencv/build
cd ~/ros2_overlay_ws
colcon build --symlink-install
source ./install/local_setup.bash
sudo mkdir -p /opt/openvino_toolkit
sudo ln -sf ~/ros2_overlay_ws/src/ros2_openvino_toolkit /opt/openvino_toolkit/ros2_openvino_toolkit
这里虽然是Running the Demo,但是不Running一下,OA就跑不起来,原因是这里会需要下一个model,而这些model也是OA需要的
download and convert a trained model to produce an optimized Intermediate Representation (IR) of the model
#object segmentation model
cd /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/install_prerequisites
sudo ./install_prerequisites.sh
mkdir -p ~/Downloads/models
cd ~/Downloads/models
wget http://download.tensorflow.org/models/object_detection/mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
tar -zxvf mask_rcnn_inception_v2_coco_2018_01_28.tar.gz
cd mask_rcnn_inception_v2_coco_2018_01_28
python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo_tf.py --input_model frozen_inference_graph.pb --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/extensions/front/tf/mask_rcnn_support.json --tensorflow_object_detection_api_pipeline_config pipeline.config --reverse_input_channels --output_dir ./output/
sudo mkdir -p /opt/models
sudo ln -sf ~/Downloads/models/mask_rcnn_inception_v2_coco_2018_01_28 /opt/models/
#object detection model
cd /opt/intel/computer_vision_sdk/deployment_tools/model_downloader
sudo python3 ./downloader.py --name mobilenet-ssd
#FP32 precision model
sudo python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo.py --input_model /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32 --mean_values [127.5,127.5,127.5] --scale_values [127.5]
#FP16 precision model
sudo python3 /opt/intel/computer_vision_sdk/deployment_tools/model_optimizer/mo.py --input_model /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/mobilenet-ssd.caffemodel --output_dir /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16 --data_type=FP16 --mean_values [127.5,127.5,127.5] --scale_values [127.5]
copy label files (excute once)
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/emotions-recognition/FP32/emotions-recognition-retail-0003.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/emotions-recognition-retail-0003/FP32
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/face_detection/face-detection-adas-0001.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP32
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/face_detection/face-detection-adas-0001.labels /opt/intel/computer_vision_sdk/deployment_tools/intel_models/face-detection-adas-0001/FP16
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_segmentation/frozen_inference_graph.labels ~/Downloads/models/mask_rcnn_inception_v2_coco_2018_01_28/output
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP32
sudo cp /opt/openvino_toolkit/ros2_openvino_toolkit/data/labels/object_detection/mobilenet-ssd.labels /opt/intel/computer_vision_sdk/deployment_tools/model_downloader/object_detection/common/mobilenet-ssd/caffe/output/FP16
set ENV LD_LIBRARY_PATH and environment
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib
# get code
mkdir ~/ros2_ws/src -p
cd ~/ros2_ws/src
git clone https://github.com/intel/ros2_object_analytics.git -b devel (devel branch is the latest code with 2D tracking features, while master branch is stable for ros2 bloom release)
# Build
cd ~/ros2_ws
source /opt/ros/crystal/setup.bash
colcon build --symlink-install
# Start OA demo with OpenVINO
source /opt/ros/crystal/setup.bash
source ~/ros2_ws/install/local_setup.bash
source /opt/intel/computer_vision_sdk/bin/setupvars.sh
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/build/intel64/Release/lib
Option1: if ros2_openvino_toolkit built from source code
ros2 launch object_analytics_node object_analytics_with_openvino_oss.launch.py
Option2: if ros2_openvino_toolkit got from Robotics_SDK
ros2 launch object_analytics_node object_analytics_with_openvino_sdk.launch.py
注意lanch之前前面四个必不可少的source和export,后两个是和OpenVINO有关的。
然后
…
就成功啦!!!哈哈哈…
整个效果看上去还是非常牛逼的,到这一步算是装好了,主要还是自己太菜,接下来终于可以开心地看源码了~