深度图+点云常用工具汇总

1. PCL (Point Cloud Library)

• 主要功能：支持2D/3D图像和点云处理
• PCL Android资源
• 主要功能模块
  
• PCL教材
• PCL依赖库

库名	最小版本	描述
Eigen	3.0	高层次的C ++库，有效支持线性代数，矩阵和矢量运算，数值分析及其相关的算法，功能列表： Core：Matrix和Array类，基础的线性代数运算和数组操作 Geometry：旋转、平移、缩放、2D/3D旋转(Quanternion, AngleAxis) LU：求逆、行列式和LU分解 Cholesky：LLT和LDLT Cholesky分解 SVD：SVD分解（JacobiSVD） QR：QR分解 Eigenvalues：求特征值和特征向量
FLANN	1.7.1	Fast Library for Approximate Nearest Neighbors (近似近邻快速库) 一个在高维空间中执行快速近似近邻搜索的库 最适合最近邻搜索的算法，以及一个根据数据集自动选择最佳算法和最佳参数的系统
Boost	1.40 (without OpenNI) 1.47 (with OpenNI)	为C++语言标准库提供扩展的一些C++程序库的总称 Boost库是一个可移植、提供源代码的C++库，作为标准库的后备，是C++标准化进程的开发引擎之一 现有的 Boost 包含大约150种不同的函数库
VTK	5.6	PCL使用的可视化工具 (Visualization Toolkit) 支持3D渲染、3D交互

• PCL编译

# flann库
sudo apt-get install libflann1.8 libflann-dev 
# eigen库
sudo apt-get install libeigen3-dev
# boost库  
sudo apt-get install libboost-all-dev 
# VTK 
sudo apt-get install libvtk5.10-qt4 libvtk5.10 libvtk5-dev

# 下载PCL
git clone https://github.com/PointCloudLibrary/pcl.git
#编译
cd pcl
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DBUILD_apps=ON ..
make -j4
# install
sudo make install

# test
pcl_viewer model.pcd

1.1 基于pcl::visualization::PCLVisualizer 实时显示点云流的代码

include <pcl/point_types.h>
#include 



boost::mutex updateModelMutex;

boost::shared_ptr<pcl::visualization::PCLVisualizer> simpleVis (pcl::PointCloud<pcl::PointXYZ>::ConstPtr cloud)
{
        // --------------------------------------------
        // -----Open 3D viewer and add point cloud-----
        // --------------------------------------------
        boost::shared_ptr<pcl::visualization::PCLVisualizer> viewer (new pcl::visualization::PCLVisualizer ("3D Viewer"));
        viewer->setCameraPosition(0, 0, 0, 0, 0, 0, 0, 0, -1);
        viewer->setBackgroundColor(0,0,0);
        viewer->addPointCloud<pcl::PointXYZ> (cloud, "sample cloud");
        viewer->setPointCloudRenderingProperties (pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 1, "sample cloud");
        viewer->initCameraParameters ();
        return (viewer);
}

void viewerRunner(boost::shared_ptr<pcl::visualization::PCLVisualizer> viewer)
{
        while (!viewer->wasStopped ())
        {
               viewer->spinOnce (100);
               boost::this_thread::sleep (boost::posix_time::microseconds (100));
        }
}

void main()
{      
	    pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_ptr (new pcl::PointCloud<pcl::PointXYZ>);
        pcl::PointCloud<pcl::PointXYZ> &pcloud1 = *cloud_ptr;
        boost::shared_ptr<pcl::visualization::PCLVisualizer> viewer (new pcl::visualization::PCLVisualizer ("3D Viewer"));
        
        pcloud1.points.push_back( pcl::PointXYZ(10, 10, 80) );
        pcloud1.width = cloud_ptr->size();
        pcloud1.height = 1;
        pcloud1.is_dense = true;
        viewer = simpleVis(cloud_ptr);

        boost::thread vthread(&viewerRunner,viewer);
       
        while(1)//循环抓取深度数据
        {
              pcloud1.clear();
              for ( int _row = 0; _row < disp.rows; _row++ )
              {
                   for ( int _col = 0; _col < disp.cols; _col++ )
                   {
                       float x, y, z;
                       pcl::PointXYZ ptemp(x, y, z);
                       pcloud1.points.push_back( ptemp );
                    }
               }
               pcloud1.width = cloud_ptr->size();
               pcloud1.height = 1;
               pcloud1.is_dense = true;
               boost::mutex::scoped_lock updateLock(updateModelMutex);
               viewer->updatePointCloud<pcl::PointXYZ>(cloud_ptr,"sample cloud");
               updateLock.unlock();
               boost::this_thread::sleep (boost::posix_time::microseconds (100));
       }
       vthread.joint();
}

• 或

int main()
{
	pcl::visualization::PCLVisualizer viewer(“Cloud”);
	pcl::PointCloud::Ptr cloud(new pcl::PointCloud);
	int frame=0;
	while(1)
	{
		char buf[233];
		if (frame != 0)
		{
			sprintf(buf, “frame%d”, frame);
			viewer.removePointCloud(buf);
		}
		sprintf(buf, “frame%d”, ++frame);
		viewer.addPointCloud(cloud, buf);
		viewer.spinOnce();
	}
	return 0;
}

2. Open3D

• Open3D是Intel继OpenCV又一开源巨作
• 官方文档
• 主要特点
  • 天生支持最爱的Python
  • 采用Google Filament作为渲染引擎，可在Android, Linux, MacOS和Windows上运行
  • 聚焦可视化和渲染

2.1 编译环境

• Ubuntu 18.04+: GCC 5+, Clang 7+
• macOS 10.14+: XCode 8.0+
• Windows 10 (64-bit): Visual Studio 2019+

2.2 编译过程 (Ubuntu 18.04)

• 下载代码

git clone --recursive https://github.com/intel-isl/Open3D

• 安装依赖

sudo apt install nasm  
util/scripts/install-deps-ubuntu.sh

2.3 设置Python绑定环境 (pybind11)

• 设置正确的Python执行文件

# Ubuntu 18.04默认安装了 Python 3.6.9
# 检查python版本
which python3
python3 -V
# 在CMake中增加
-DPYTHON_EXECUTABLE=/usr/bin/python3

• 支持Jupyter可视化

# 安装npm
sudo apt install nodejs
sudo apt install npm
#check npm
node -v
npm -v

2.4 编译过程

mkdir build; cd build

• Ubuntu 和 MacOS

cmake -DCMAKE_INSTALL_PREFIX=/usr/local/open3d -DGLIBCXX_USE_CXX11_ABI=OFF -DPYTHON_EXECUTABLE=/usr/bin/python3 ..      

• Windows

cmake -DCMAKE_INSTALL_PREFIX=/usr/local/open3d \ 
      -DGLIBCXX_USE_CXX11_ABI=OFF \
      -DPYTHON_EXECUTABLE=/usr/bin/python3 \
	  -DBUILD_EIGEN3=ON  \
      -DBUILD_FLANN=ON   \
      -DBUILD_GLEW=ON    \
      -DBUILD_GLFW=ON    \
      -DBUILD_PNG=ON     \
      ..      

• 执行编译

make -j4

• 安装和卸载C++库

# 安装
sudo make install
# 卸载
sudo make uninstall

• 安装和卸载Python包

# Create Python package in build/lib/python_package
make python-package
# Create pip wheel in build/lib/python_package/pip_package
make pip-package
# Create conda package in build/lib/python_package/conda_package
make conda-package
# Install pip wheel
sudo make install-pip-package

• 测试导入Open3D

python -c "import open3d"

2.5 阻塞和非阻塞示例代码

# -*-coding: utf-8 -*-
import open3d
import numpy as np
import cv2

# 绘制open3d坐标系
axis_pcd = open3d.create_mesh_coordinate_frame(size=0.5, origin=[0, 0, 0])
# 在3D坐标上绘制点：坐标点[x,y,z]对应R，G，B颜色
points = np.array([[0.1, 0.1, 0.1], [1, 0, 0], [0, 1, 0], [0, 0, 1]])
colors = [[1, 1, 1], [1, 0, 0], [0, 1, 0], [0, 0, 1]]
 
test_pcd = open3d.geometry.PointCloud()  # 定义点云
 
# 方法1（非阻塞显示）
vis = open3d.Visualizer()
vis.create_window(window_name="Open3D1")
vis.get_render_option().point_size = 3
first_loop = True
# 先把点云对象添加给Visualizer
vis.add_geometry(axis_pcd)
vis.add_geometry(test_pcd)
while True:
    # 给点云添加显示的数据
    points -= 0.001
    test_pcd.points = open3d.utility.Vector3dVector(points)  # 定义点云坐标位置
    test_pcd.colors = open3d.Vector3dVector(colors)  # 定义点云的颜色
    # update_renderer显示当前的数据
    vis.update_geometry()
    vis.poll_events()
    vis.update_renderer()
    cv2.waitKey(100)
 
# 方法2（阻塞显示）：调用draw_geometries直接把需要显示点云数据
test_pcd.points = open3d.utility.Vector3dVector(points)  # 定义点云坐标位置
test_pcd.colors = open3d.Vector3dVector(colors)  # 定义点云的颜色
open3d.visualization.draw_geometries([test_pcd] + [axis_pcd], window_name="Open3D2")

2.6 添加自定义的 draw_geometries 函数

import os
import numpy as np
from open3d import *
points = np.random.rand(10000, 3)
point_cloud = PointCloud()
point_cloud.points = Vector3dVector(points)
draw_geometries([point_cloud])

# or
def custom_draw_geometry_with_key_callback(pcd):
    def change_background_to_black(vis):
        opt = vis.get_render_option()
        opt.background_color = np.asarray([0, 0, 0])
        return False
    def load_render_option(vis):
        vis.get_render_option().load_from_json(
                "../../TestData/renderoption.json")
        return False
    def capture_depth(vis):
        depth = vis.capture_depth_float_buffer()
        plt.imshow(np.asarray(depth))
        plt.show()
        return False
    def capture_image(vis):
        image = vis.capture_screen_float_buffer()
        plt.imshow(np.asarray(image))
        plt.show()
        return False
    key_to_callback = {}
    key_to_callback[ord("K")] = change_background_to_black
    key_to_callback[ord("R")] = load_render_option
    key_to_callback[ord(",")] = capture_depth
    key_to_callback[ord(".")] = capture_image
    draw_geometries_with_key_callbacks([pcd], key_to_callback)

custom_draw_geometry_with_key_callback(point_cloud)

2.7 体素下采样 (Voxel downsampling)

print("Downsample the point cloud with a voxel of 0.05")
downpcd = pcd.voxel_down_sample(voxel_size=0.05)
o3d.visualization.draw_geometries([downpcd], zoom=0.3412,
                                  front=[0.4257, -0.2125, -0.8795],
                                  lookat=[2.6172, 2.0475, 1.532],
                                  up=[-0.0694, -0.9768, 0.2024])

2.8 平面分割 (Plane segmentation)

• 使用RANSAC进行平面分割

pcd = o3d.io.read_point_cloud("../../TestData/fragment.pcd")
plane_model, inliers = pcd.segment_plane(distance_threshold=0.01,
                                         ransac_n=3,
                                         num_iterations=1000)
[a, b, c, d] = plane_model
print(f"Plane equation: {a:.2f}x + {b:.2f}y + {c:.2f}z + {d:.2f} = 0")

inlier_cloud = pcd.select_by_index(inliers)
inlier_cloud.paint_uniform_color([1.0, 0, 0])
outlier_cloud = pcd.select_by_index(inliers, invert=True)
o3d.visualization.draw_geometries([inlier_cloud, outlier_cloud], zoom=0.8,
                                  front=[-0.4999, -0.1659, -0.8499],
                                  lookat=[2.1813, 2.0619, 2.0999],
                                  up=[0.1204, -0.9852, 0.1215])

2.9 删除隐藏点

print("Convert mesh to a point cloud and estimate dimensions")
pcd = o3dtut.get_armadillo_mesh().sample_points_poisson_disk(5000)
diameter = np.linalg.norm(np.asarray(pcd.get_max_bound()) - np.asarray(pcd.get_min_bound()))
o3d.visualization.draw_geometries([pcd])

print("Define parameters used for hidden_point_removal")
camera = [0, 0, diameter]
radius = diameter * 100

print("Get all points that are visible from given view point")
_, pt_map = pcd.hidden_point_removal(camera, radius)

print("Visualize result")
pcd = pcd.select_by_index(pt_map)
o3d.visualization.draw_geometries([pcd])

3. 安装cmake 3.12.2或更高版本

# 删除已经安装的CMake旧版本（非必需项）：
apt-get autoremove cmake1
#下载cmake-3.12.2-Linux-x86_64.tar.gz安装包：
wget https://cmake.org/files/v3.12/cmake-3.12.2-Linux-x86_64.tar.gz
# 解压请运行以下命令：
tar zxvf cmake-3.12.2-Linux-x86_64.tar.gz
# 创建软链接
#   注: 文件路径是可以指定的，一般选择在/opt或者是/usr路径下，这里选择/opt：
mv cmake-3.12.2-Linux-x86_64 /opt/cmake-3.12.2
ln -sf /opt/cmake-3.12.2/bin/*  /usr/bin/
# 然后执行命令检查一下：
cmake --version

# cmake gui
cmake-gui