【复现笔记】PoseCNN-PyTorch
PoseCNN-PyTorch: A PyTorch Implementation of the PoseCNN Framework for 6D Object Pose Estimation
代码github：https://github.com/NVlabs/PoseCNN-PyTorch
目录
PoseCNN-PyTorch: A PyTorch Implementation of the PoseCNN Framework for 6D Object Pose Estimation
1、下载代码
2、安装pytorch、Eigen、Sophus
3、pip install
4、Initialize the submodules
5、Compile the new layers under $ROOT/lib/layers introduced in PoseCNN
6、Compile cython components
7、Compile the ycb_render
8、下载数据集和预训练权重
9、运行demo
10、用YCB对象的合成数据训练自己的模型
11、Training and testing on the YCB-Video dataset
12、Training and testing on the DexYCB dataset
13、Running with ROS on a Realsense Camera for real-world pose estimation
环境：ubuntu 20.04 + cuda-10.2 + python3.6.7
1、下载代码
git clone https://github.com/NVlabs/PoseCNN-PyTorch.git2、安装pytorch、Eigen、Sophus
（1）pytorch：Start Locally | PyTorch
（2）EIgen 3：
作者给的eigen库链接，安装后查看版本是3.3.90：GitHub - eigenteam/eigen-git-mirror: THIS MIRROR IS DEPRECATED -- New url: https://gitlab.com/libeigen/eigen
（3）Sophus：
作者给的sophus库链接：https://github.com/yuxng/Sophus
3、pip install
cd PoseCNN-PyTorch
pip install -r requirement.txt4、Initialize the submodules
cd ycb_render
git submodule update --init --recursive过程：这一步需要下载pybind11，但它的git仓库下载很慢，所以我先下载了pybind11，再执行，如下所示
~/3dTracking/PoseCNN-PyTorch/ycb_render$ git clone https://github.com/pybind/pybind11.git
Cloning into 'pybind11'...
remote: Enumerating objects: 24975, done.
remote: Counting objects: 100% (183/183), done.
remote: Compressing objects: 100% (116/116), done.
remote: Total 24975 (delta 91), reused 109 (delta 52), pack-reused 24792
Receiving objects: 100% (24975/24975), 9.48 MiB | 440.00 KiB/s, done.
Resolving deltas: 100% (17492/17492), done.
~/3dTracking/PoseCNN-PyTorch/ycb_render$ git submodule update --init --recursive
Submodule path 'pybind11': checked out '028812ae7eee307dca5f8f69d467af7b92cc41c8'5、Compile the new layers under $ROOT/lib/layers introduced in PoseCNN
cd ../lib/layers
sudo /home/r*/anaconda3/envs/posecnn/bin/python setup.py install报错1：
hough_voting_kernel.cu:14:10: fatal error: Eigen/Geometry: No such file or directory
14 | #include
| ^~~~~~~~~~~~~~~~
compilation terminated.
error: command '/usr/local/cuda-11.2/bin/nvcc' failed with exit code 1 说是找不到Eigen/Geometry，查看Eigen/Geometry文件在/usr/include/eigen3/Eigen文件夹内
/usr/include/eigen3/Eigen$ ls
Cholesky IterativeLinearSolvers QtAlignedMalloc StdDeque
CholmodSupport Jacobi Sparse StdList
Core LU SparseCholesky StdVector
Dense MetisSupport SparseCore SuperLUSupport
Eigen OrderingMethods SparseLU SVD
Eigenvalues PardisoSupport SparseQR UmfPackSupport
Geometry PaStiXSupport SPQRSupport
Householder QR src于是参考https://www.cnblogs.com/long5683/p/13406666.html
建立软连接：
sudo ln -s /usr/include/eigen3/Eigen /usr/include/Eigen报错2：fatal error: math_functions.hpp: No such file or directory
参考PoseCNN代码复现CMake编译找不到math_functions.hpp_不完美的上帝的博客-CSDN博客_posecnn 复现
报错3：could not find ninja
sudo apt install ninja-build重新编译，详细日志如下：
(posecnn) r****@r****-MS-7D42:~/3dTracking/PoseCNN-PyTorch/lib/layers$ sudo /home/r****/anaconda3/envs/posecnn/bin/python setup.py install
[sudo] password for r****:
running install
running bdist_egg
running egg_info
writing posecnn.egg-info/PKG-INFO
writing dependency_links to posecnn.egg-info/dependency_links.txt
writing top-level names to posecnn.egg-info/top_level.txt
/home/r****/anaconda3/envs/posecnn/lib/python3.6/site-packages/torch/utils/cpp_extension.py:381: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'posecnn.egg-info/SOURCES.txt'
writing manifest file 'posecnn.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
creating build/bdist.linux-x86_64/egg
copying build/lib.linux-x86_64-3.6/posecnn_cuda.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
creating stub loader for posecnn_cuda.cpython-36m-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/posecnn_cuda.py to posecnn_cuda.cpython-36.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying posecnn.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying posecnn.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying posecnn.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying posecnn.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.posecnn_cuda.cpython-36: module references __file__
creating 'dist/posecnn-0.0.0-py3.6-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing posecnn-0.0.0-py3.6-linux-x86_64.egg
removing '/home/r****/anaconda3/envs/posecnn/lib/python3.6/site-packages/posecnn-0.0.0-py3.6-linux-x86_64.egg' (and everything under it)
creating /home/r****/anaconda3/envs/posecnn/lib/python3.6/site-packages/posecnn-0.0.0-py3.6-linux-x86_64.egg
Extracting posecnn-0.0.0-py3.6-linux-x86_64.egg to /home/r****/anaconda3/envs/posecnn/lib/python3.6/site-packages
posecnn 0.0.0 is already the active version in easy-install.pth
Installed /home/r****/anaconda3/envs/posecnn/lib/python3.6/site-packages/posecnn-0.0.0-py3.6-linux-x86_64.egg
Processing dependencies for posecnn==0.0.0
Finished processing dependencies for posecnn==0.0.06、Compile cython components
cd ../utils
python setup.py build_ext --inplace日志：
(posecnn) r***@r***-MS-7D42:~/3dTracking/PoseCNN-PyTorch/lib/utils$ python setup.py build_ext --inplace
running build_ext
cythoning bbox.pyx to bbox.c
/home/r***/anaconda3/envs/posecnn/lib/python3.6/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/r***/3dTracking/PoseCNN-PyTorch/lib/utils/bbox.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
building 'cython_bbox' extension
creating build
creating build/temp.linux-x86_64-3.6
gcc -pthread -B /home/r***/anaconda3/envs/posecnn/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/r***/anaconda3/envs/posecnn/lib/python3.6/site-packages/numpy/core/include -I/home/r***/anaconda3/envs/posecnn/include/python3.6m -c bbox.c -o build/temp.linux-x86_64-3.6/bbox.o -Wno-cpp -Wno-unused-function
gcc -pthread -shared -B /home/r***/anaconda3/envs/posecnn/compiler_compat -L/home/r***/anaconda3/envs/posecnn/lib -Wl,-rpath=/home/r***/anaconda3/envs/posecnn/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/bbox.o -o /home/r***/3dTracking/PoseCNN-PyTorch/lib/utils/cython_bbox.cpython-36m-x86_64-linux-gnu.so
/home/r***/anaconda3/envs/posecnn/compiler_compat/ld: warning: /home/r***/anaconda3/envs/posecnn/lib/libgcc_s.so.1: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010001
/home/r***/anaconda3/envs/posecnn/compiler_compat/ld: warning: /home/r***/anaconda3/envs/posecnn/lib/libgcc_s.so.1: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010002
/home/r***/anaconda3/envs/posecnn/compiler_compat/ld: warning: /home/r***/anaconda3/envs/posecnn/lib/libgcc_s.so.1: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010001
/home/r***/anaconda3/envs/posecnn/compiler_compat/ld: warning: /home/r***/anaconda3/envs/posecnn/lib/libgcc_s.so.1: unsupported GNU_PROPERTY_TYPE (5) type: 0xc0010002
第二次运行时日志是这样的
~/3dTracking/PoseCNN-PyTorch/lib/utils$ python setup.py build_ext --inplace
running build_ext
skipping 'bbox.c' Cython extension (up-to-date)7、Compile the ycb_render
cd ../../ycb_render
python setup.py develop报错1：fatal error: X11/Xlib.h: No such file or directory
sudo apt-get install libghc-x11-dev报错2：fatal error: GL/gl.h: No such file or directory
sudo apt-get install libgl1-mesa-dev成功运行的日志：
(posecnn) rakuli@rakuli-MS-7D42:~/3dTracking/PoseCNN-PyTorch/ycb_render$ sudo /home/rakuli/anaconda3/envs/posecnn/bin/python setup.py develop
running develop
running egg_info
writing CppYCBRenderer.egg-info/PKG-INFO
writing dependency_links to CppYCBRenderer.egg-info/dependency_links.txt
writing top-level names to CppYCBRenderer.egg-info/top_level.txt
reading manifest file 'CppYCBRenderer.egg-info/SOURCES.txt'
writing manifest file 'CppYCBRenderer.egg-info/SOURCES.txt'
running build_ext
CMake Warning (dev) at CMakeLists.txt:2 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run "cmake --help-policy CMP0048" for policy details. Use the cmake_policy
command to set the policy and suppress this warning.
The following variable(s) would be set to empty:
CMAKE_PROJECT_VERSION
CMAKE_PROJECT_VERSION_MAJOR
CMAKE_PROJECT_VERSION_MINOR
CMAKE_PROJECT_VERSION_PATCH
This warning is for project developers. Use -Wno-dev to suppress it.
-- pybind11 v2.6.2 dev1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rakuli/3dTracking/PoseCNN-PyTorch/ycb_render/build/temp.linux-x86_64-3.6
[ 33%] Built target query_devices
[ 66%] Built target test_device
[ 75%] Building CXX object CMakeFiles/CppYCBRenderer.dir/cpp/ycb_renderer.cpp.o
[ 83%] Linking CXX shared module ../../CppYCBRenderer.cpython-36m-x86_64-linux-gnu.so
[100%] Built target CppYCBRenderer
Creating /home/rakuli/anaconda3/envs/posecnn/lib/python3.6/site-packages/CppYCBRenderer.egg-link (link to .)
Adding CppYCBRenderer 0.0.1 to easy-install.pth file
Installed /home/rakuli/3dTracking/PoseCNN-PyTorch/ycb_render
Processing dependencies for CppYCBRenderer==0.0.1
Finished processing dependencies for CppYCBRenderer==0.0.18、下载数据集和预训练权重
-
3D models of YCB Objects下载并保存在PoseCNN-PyTorch/data 下：https://drive.google.com/file/d/1PTNmhd-eSq0fwSPv0nvQN8h_scR1v-UJ/view?usp=sharing
-
pre-trained checkpoints下载并保存在PoseCNN-PyTorch/data下：https://drive.google.com/file/d/1-ECAkkTRfa1jJ9YBTzf04wxCGw6-m5d4/view?usp=sharing
-
YCB-Video数据集：PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes – UW Robotics and State Estimation Lab
-
dexYCB：DexYCB: A Benchmark for Capturing Hand Grasping of Objects
-
real-world images with pose annotations for 20 YCB objects collected via robot interation：https://drive.google.com/file/d/1cQH_dnDzyrI0MWNx8st4lht_q0F6cUrE/view?usp=sharing
9、运行demo
从以下链接下载assimp库（不要用apt-get install！！）：
https://github.com/assimp/assimp/tree/v4.1.0
下载3D models和checkpoints，再运行：
./experiments/scripts/demo.sh成功运行
日志过长，故存放在以下链接：
https://blog.csdn.net/weixin_54470372/article/details/127775006?csdn_share_tail=%7B%22type%22%3A%22blog%22%2C%22rType%22%3A%22article%22%2C%22rId%22%3A%22127775006%22%2C%22source%22%3A%22weixin_54470372%22%7D