【复现笔记】SRT3D: Sparse Region-Based 3D Object Tracking

paper: SRT3D: A Sparse Region-Based 3D Object Tracking Approach for the Real World
Manuel Stoiber, Martin Pfanne, Klaus H. Strobl, Rudolph Triebel, and Alin Albu-Schäffer
International Journal of Computer Vision (IJCV) 2022

作者提供源代码：3DObjectTracking/SRT3D at master · DLR-RM/3DObjectTracking · GitHub

目录

目录

Description of Content

USAGE

开始复现

1、cmake

2、build

---

Description of Content

This repository contains everything necessary to reproduce the results presented in our paper. This includes the evaluation on the RBOT and OPT datasets as well as the conducted parameters study. Source files for the srt3d library are stored in src and include/srt3d. Source files for the executables are stored in examples. For the evaluation and parameter study on the RBOT and OPT datasets, the code in evaluate_rbot_dataset.cpp, parameters_study_rbot.cpp, evaluate_opt_dataset.cpp, and parameters_study_opt.cpp was used. To reproduce our experiments, please download the RBOT and OPT datasets and adjust the dataset_directory in the source code. Note that model files (e.g. ape_model.bin, bakingsoda_model.bin, ...) will be created automatically and are stored in the same folder as the .obj file of each object.

For real-world tracking, we provide code that allows either to directly track objects or to first record a sequence and then run the tracker on this recorded sequence. The code for this is provided in run_on_camera_sequence.cpp, record_camera_sequence.cpp and run_on_recorded_sequence.cpp. In our implementation, an Azure Kinect camera is used. If you don't want to use the Azure Kinect, you can disable the CMake option USE_AZURE_KINECT. For tracking, object parameters and directories in run_on_camera_sequence.cpp and run_on_recorded_sequence.cpp have to be adjusted. Note that by default, tracking will not start automatically. To start tracking, please press the T key on your keyboard. To stop the application press Q. If you would like to use another camera than the Azure Kinect, we encourage you to create a class similar to the AzureKinectCamera class in src/azure_kinect_camera.cpp.

 翻译：

这个存储库包含再现我们论文中给出的结果所需的所有内容。这包括对RBOT和OPT数据集的评价以及进行的参数研究。srt3d库的源文件存储在src中，包括/srt3d。可执行文件的源文件存储在示例中。对于RBOT和OPT数据集的评估和参数研究，使用了evaluate_rbot_dataset.cpp、parameters_study_rbot.cpp、evaluate_opt_dataset.cpp和parameters_study_opt.cpp中的代码。为了再现我们的实验，请下载RBOT和OPT数据集，并调整源代码中的dataset_directory。注意，模型文件(例如ape_model.bin, bakingsoda_model.bin，…)将被自动创建，并与每个对象的.obj文件存储在同一文件夹中。

对于真实世界的跟踪，我们提供的代码既可以直接跟踪对象，也可以先记录一个序列，然后在这个记录的序列上运行跟踪器。这方面的代码在run_on_camera_sequence.cpp、record_camera_sequence.cpp和run_on_recorded_sequence.cpp中提供。在我们的实现中，使用了Azure Kinect摄像头。如果你不想使用Azure Kinect，你可以禁用CMake选项USE_AZURE_KINECT。为了跟踪，必须调整run_on_camera_sequence.cpp和run_on_recorded_sequence.cpp中的对象参数和目录。注意，默认情况下，跟踪不会自动启动。要开始追踪，请按键盘上的T键。如果你想使用AzureKinect以外的其他摄像头，我们建议你在src/azure_kinect_camera.cpp中创建一个类似于AzureKinectCamera类的类。

USAGE

If you want to use our tracker for your own project, we would like to refer you to run_on_camera_sequence.cpp for how to set up the tracker and required objects. The following variables in run_on_camera_sequence.cpp should be considered in an initial set up:

• model_path: location where all object models, that are created automatically from an .obj file, are stored.
• body_ptr: contains all information associated with an object
  • geometry_path: path to .obj file
  • geometry2body_pose: transformation that allows to set a different frame of reference for the object than defined by the .obj file.
  • world2body_pose: initial transformation between camera and object.
  • geometry_unit_in_meter: scale factor to scale the unit used in the .obj file to meter.
  • geometry_counterclockwise: true if winding order of triangles in .obj is defined counter-clockwise.
  • geometry_enable_culling: true if faces that are not facing toward the camera should be culled.
  • maximum_body_diameter: maximum diameter in meter of a sphere that encapsulates the entire body.
  • occlusion_mask_id: unique number between 0 and 7 that is used to encode the object in occlusion_masks.
• model_ptr: contains all information associated with a model
  • sphere_radius: distance from camera to object center
  • n_divides: how often an icosahedron is devided (controls the number of template views)
  • n_points: number of contour points

翻译：

如果您想在自己的项目中使用我们的跟踪器，我们建议您参考run_on_camera_sequence.cpp，了解如何设置跟踪器和所需的对象。

在初始设置中应该考虑run_on_camera_sequence.cpp中的以下变量:

• Model_path:存储从.obj文件自动创建的所有对象模型的位置。
• Body_ptr:包含与对象相关的所有信息
• Geometry_path: .obj文件的路径
• Geometry2body_pose:转换，允许为对象设置不同的参考框架，而不是由.obj文件定义的。
• World2body_pose:相机和物体之间的初始转换。
• Geometry_unit_in_meter:缩放因子，将.obj文件中使用的单位缩放为米。
• geometry_逆时针:如果.obj中三角形的缠绕顺序定义为逆时针，则为true。
• Geometry_enable_culling:如果不面向相机的脸应该被剔除，则为true。
• Maximum_body_diameter:封装整个身体的球体的最大直径(米)。
• Occlusion_mask_id: 0到7之间的唯一数字，用于编码occlusion_masks中的对象。
• Model_ptr:包含与模型相关的所有信息
• Sphere_radius:摄像机到物体中心的距离
• n_partitions:划分二十面体的频率(控制模板视图的数量)
• N_points:轮廓点个数

开始复现

1、编译

（1）cmake

cmake -DUSE_AZURE_KINECT=OFF

输出：

~/3dTracking/3DObjectTracking/SRT3D$ cmake -DUSE_AZURE_KINECT=OFF
CMake Warning:
  No source or binary directory provided.  Both will be assumed to be the
  same as the current working directory, but note that this warning will
  become a fatal error in future CMake releases.


-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Warning (dev) at /usr/local/share/cmake-3.25/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  CMakeLists.txt:22 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- Found GLEW: /usr/include (found version "2.1.0") 
-- Found OpenCV: /usr/local (found suitable version "4.0.0", minimum required is "4.0.0") 
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Performing Test HAS_MARCH
-- Performing Test HAS_MARCH - Failed
-- Performing Test HAS_MTUNE
-- Performing Test HAS_MTUNE - Failed
-- Performing Test HAS_GGDB
-- Performing Test HAS_GGDB - Success
-- Performing Test HAS_Z7
-- Performing Test HAS_Z7 - Failed
-- Performing Test HAS_FTRAPV
-- Performing Test HAS_FTRAPV - Success
-- Performing Test HAS_OD
-- Performing Test HAS_OD - Failed
-- Performing Test HAS_OB3
-- Performing Test HAS_OB3 - Failed
-- Performing Test HAS_O3
-- Performing Test HAS_O3 - Success
-- Performing Test HAS_OB2
-- Performing Test HAS_OB2 - Failed
-- Performing Test HAS_O2
-- Performing Test HAS_O2 - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /home/r*/3dTracking/3DObjectTracking/SRT3D

（2）make

make

输出：

~/3dTracking/3DObjectTracking/SRT3D$ make
[  3%] Building CXX object src/CMakeFiles/srt3d.dir/common.cpp.o
[  6%] Building CXX object src/CMakeFiles/srt3d.dir/body.cpp.o
[ 10%] Building CXX object src/CMakeFiles/srt3d.dir/renderer_geometry.cpp.o
[ 13%] Building CXX object src/CMakeFiles/srt3d.dir/renderer.cpp.o
[ 17%] Building CXX object src/CMakeFiles/srt3d.dir/normal_renderer.cpp.o
[ 20%] Building CXX object src/CMakeFiles/srt3d.dir/occlusion_renderer.cpp.o
[ 24%] Building CXX object src/CMakeFiles/srt3d.dir/model.cpp.o
[ 27%] Building CXX object src/CMakeFiles/srt3d.dir/camera.cpp.o
[ 31%] Building CXX object src/CMakeFiles/srt3d.dir/loader_camera.cpp.o
[ 34%] Building CXX object src/CMakeFiles/srt3d.dir/viewer.cpp.o
[ 37%] Building CXX object src/CMakeFiles/srt3d.dir/image_viewer.cpp.o
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/image_viewer.cpp: In member function ‘virtual bool srt3d::ImageViewer::UpdateViewer(int)’:
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/image_viewer.cpp:30:1: warning: control reaches end of non-void function [-Wreturn-type]
   30 | }
      | ^
[ 41%] Building CXX object src/CMakeFiles/srt3d.dir/normal_viewer.cpp.o
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/normal_viewer.cpp: In member function ‘virtual bool srt3d::NormalViewer::UpdateViewer(int)’:
/home/r*/3dTracking/3DObjectTracking/SRT3D/src/normal_viewer.cpp:60:1: warning: control reaches end of non-void function [-Wreturn-type]
   60 | }
      | ^
[ 44%] Building CXX object src/CMakeFiles/srt3d.dir/region_modality.cpp.o
[ 48%] Building CXX object src/CMakeFiles/srt3d.dir/tracker.cpp.o
[ 51%] Linking CXX static library libsrt3d.a
[ 51%] Built target srt3d
[ 55%] Building CXX object examples/CMakeFiles/evaluate_rbot_dataset.dir/evaluate_rbot_dataset.cpp.o
[ 58%] Building CXX object examples/CMakeFiles/evaluate_rbot_dataset.dir/rbot_evaluator.cpp.o
[ 62%] Linking CXX executable evaluate_rbot_dataset
[ 62%] Built target evaluate_rbot_dataset
[ 65%] Building CXX object examples/CMakeFiles/parameter_study_rbot_dataset.dir/parameter_study_rbot_dataset.cpp.o
[ 68%] Building CXX object examples/CMakeFiles/parameter_study_rbot_dataset.dir/rbot_evaluator.cpp.o
[ 72%] Linking CXX executable parameter_study_rbot_dataset
[ 72%] Built target parameter_study_rbot_dataset
[ 75%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/evaluate_opt_dataset.cpp.o
[ 79%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/opt_evaluator.cpp.o
[ 82%] Linking CXX executable evaluate_opt_dataset
[ 82%] Built target evaluate_opt_dataset
[ 86%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/parameter_study_opt_dataset.cpp.o
[ 89%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/opt_evaluator.cpp.o
[ 93%] Linking CXX executable parameter_study_opt_dataset
[ 93%] Built target parameter_study_opt_dataset
[ 96%] Building CXX object examples/CMakeFiles/run_on_recorded_sequence.dir/run_on_recorded_sequence.cpp.o
[100%] Linking CXX executable run_on_recorded_sequence
[100%] Built target run_on_recorded_sequence

（3）切换到examples目录

（4）在examples中cmake

~/3dTracking/3DObjectTracking/SRT3D/examples$ cmake ..
CMake Warning (dev) at /usr/local/share/cmake-3.25/Modules/FindOpenGL.cmake:315 (message):
  Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
  available.  Run "cmake --help-policy CMP0072" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  FindOpenGL found both a legacy GL library:

    OPENGL_gl_LIBRARY: /usr/lib/x86_64-linux-gnu/libGL.so

  and GLVND libraries for OpenGL and GLX:

    OPENGL_opengl_LIBRARY: /usr/lib/x86_64-linux-gnu/libOpenGL.so
    OPENGL_glx_LIBRARY: /usr/lib/x86_64-linux-gnu/libGLX.so

  OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
  compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
  CMakeLists.txt:22 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/rakuli/3dTracking/3DObjectTracking/SRT3D

（5）make

~/3dTracking/3DObjectTracking/SRT3D/examples$ make
[ 51%] Built target srt3d
[ 62%] Built target evaluate_rbot_dataset
[ 72%] Built target parameter_study_rbot_dataset
[ 75%] Building CXX object examples/CMakeFiles/evaluate_opt_dataset.dir/evaluate_opt_dataset.cpp.o
[ 79%] Linking CXX executable evaluate_opt_dataset
[ 82%] Built target evaluate_opt_dataset
[ 86%] Building CXX object examples/CMakeFiles/parameter_study_opt_dataset.dir/parameter_study_opt_dataset.cpp.o
[ 89%] Linking CXX executable parameter_study_opt_dataset
[ 93%] Built target parameter_study_opt_dataset
[100%] Built target run_on_recorded_sequence

（6）将examples中与opt相关的6个文件复制到src文件夹中