PCL 点云超体素分割-SupervoxelClustering

 一、概述

原始文档与点云

Clustering of Pointclouds into Supervoxels - Theoretical primer — Point Cloud Library 0.0 documentation 

 超像素 Superpixels 

Segmentation algorithms aim to group pixels in images into perceptually meaningful regions which conform to object boundaries. Graph-based approaches, such as Markov Random Field (MRF) and Conditional Random Field (CRF), have become popular, as they merge relational low-level context within the image with object level class knowledge. The cost of solving pixel-level graphs led to the development of mid-level inference schemes which do not use pixels directly, but rather use groupings of pixels, known as superpixels, as the base level for nodes. Superpixels are formed by over-segmenting the image into small regions based on local low-level features, reducing the number of nodes which must be considered for inference。

重点：超像素是一个像素群体而不是但个像素点

超体素Supervoxels 

Voxel Cloud Connectivity Segmentation (VCCS) is a recent “superpixel” method which generates volumetric over-segmentations of 3D point cloud data, known as supervoxels. Supervoxels adhere to object boundaries better than state-of-the-art 2D methods, while remaining efficient enough to use in online applications. VCCS uses a region growing variant of k-means clustering for generating its labeling of points directly within a voxel octree structure. Supervoxels have two important properties; they are evenly distributed across the 3D space, and they cannot cross boundaries unless the underlying voxels are spatial connected. The former is accomplished by seeding supervoxels directly in the cloud, rather than the projected plane, while the latter uses an octree structure which maintains adjacency information of leaves. Supervoxels maintain adjacency relations in voxelized 3D space; specifically, 26-adjacency- that is neighboring voxels are those that share a face, edge, or vertex, as seen below.

Voxel Cloud Connectivity Segmentation (VCCS)使用K-均值分类的区域生长方法来对点云进行超体速分割，超体速有两个重要的特征 ：在3D的空间中是均匀分布的，前者是通过直接在云中而不是投影平面中播种超体素来完成的，而后者使用八叉树结构来维护叶子的邻接信息。超级体素在体素化 3D 空间中维持邻接关系；具体来说，26-邻接-即相邻体素是那些共享面、边或顶点的体素，

如图：

VCCS is a region growing method which incrementally expand supervoxels from a set of seed points distributed evenly in space on a grid with resolution R_seed. To maintain efficiency, VCCS does not search globally, but rather only considers points within R_seed of the seed center. Additionally, seeds which are isolated are filtered out by establishing a small search radius R_search around each seed and removing seeds which do not have sufficient neighbor voxels connected to them.

VCCS 是一种区域生长方法，它从分辨率为 R_seed 的网格上均匀分布在空间中的一组种子点增量扩展超体素。为了保持效率，VCCS不会全局搜索，而是只考虑种子中心R_seed内的点。此外，通过在每个种子周围建立小搜索半径 R_search 并去除没有足够的相邻体素与其连接的种子，来过滤掉被隔离的种子。

影响超体素聚类的各种尺寸参数。 R_seed 和 R_voxel 都必须由用户设置

Expansion from the seed points is governed by a distance measure calculated in a feature space consisting of spatial extent, color, and normals. The spatial distance D_s is normalized by the seeding resolution, color distance D_c is the euclidean distance in normalized RGB space, and normal distance D_n measures the angle between surface normal vectors.

Supervoxels are grown iteratively, using a local k-means clustering which considers connectivity and flow. The general process is as follows. Beginning at the voxel nearest the cluster center, we flow outward to adjacent voxels and compute the distance from each of these to the supervoxel center using the distance equation above. If the distance is the smallest this voxel has seen, its label is set, and using the adjacency graph, we add its neighbors which are further from the center to our search queue for this label. We then proceed to the next supervoxel, so that each level outwards from the center is considered at the same time for all supervoxels (a 2d version of this is seen in the figure below). We proceed iteratively outwards until we have reached the edge of the search volume for each supervoxel (or have no more neighbors to check) 

超级体素使用考虑连通性和流量的局部 k 均值聚类迭代生长。大致流程如下。从距离聚类中心最近的体素开始，我们向外流动到相邻的体素，并使用上面的距离方程计算每个体素到超体素中心的距离。如果距离是该体素所见过的最小距离，则设置其标签，并使用邻接图，我们将距离中心更远的邻居添加到该标签的搜索队列中。然后我们继续处理下一个超级体素，以便对于所有超级体素同时考虑从中心向外的每个级别（下图中可以看到其二维版本）。我们向外迭代，直到到达每个超体素的搜索体积的边缘（或者没有更多的邻居需要检查）

 超体素分割过程

超体素分割（过分割）详细过程-CSDN博客

官方code 

#include 
#include 
#include 
#include 
#include 
#include 

//VTK include needed for drawing graph lines
#include 

// Types
typedef pcl::PointXYZRGBA PointT;
typedef pcl::PointCloud PointCloudT;
typedef pcl::PointNormal PointNT;
typedef pcl::PointCloud PointNCloudT;
typedef pcl::PointXYZL PointLT;
typedef pcl::PointCloud PointLCloudT;

void addSupervoxelConnectionsToViewer(PointT& supervoxel_center,
    PointCloudT& adjacent_supervoxel_centers,
    std::string supervoxel_name,
    pcl::visualization::PCLVisualizer::Ptr& viewer);

int   main(int argc, char** argv)
{
    PointCloudT::Ptr cloud(new PointCloudT);
    pcl::console::print_highlight("Loading point cloud...\n");
    if (pcl::io::loadPCDFile("D:\\work\\Pointclouds\\clouds\\milk_cartoon_all_small_clorox.pcd", *cloud))
    {
        pcl::console::print_error("Error loading cloud file!\n");
        return (1);
    }


    bool disable_transform = true;
    float voxel_resolution = 0.008f;  // 体素分辨率
    float seed_resolution = 0.1f;  // 种子 分辨率
    float color_importance = 0.2f;  // 颜色权重
    float spatial_importance = 0.2f;  // 空间权重
    float normal_importance = 2.0f;   // 法向量角度差 



    pcl::SupervoxelClustering super(voxel_resolution, seed_resolution);
    if (disable_transform)
        super.setUseSingleCameraTransform(false);
    super.setInputCloud(cloud);
    super.setColorImportance(color_importance);
    super.setSpatialImportance(spatial_importance);
    super.setNormalImportance(normal_importance);

    std::map ::Ptr > supervoxel_clusters;

    pcl::console::print_highlight("Extracting supervoxels!\n");
    super.extract(supervoxel_clusters);
    pcl::console::print_info("Found %d supervoxels\n", supervoxel_clusters.size());

    pcl::visualization::PCLVisualizer::Ptr viewer(new pcl::visualization::PCLVisualizer("3D Viewer"));
    viewer->setBackgroundColor(0, 0, 0);

    PointCloudT::Ptr voxel_centroid_cloud = super.getVoxelCentroidCloud();
    viewer->addPointCloud(voxel_centroid_cloud, "voxel centroids");
    viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 2.0, "voxel centroids");
    viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_OPACITY, 0.95, "voxel centroids");

    PointLCloudT::Ptr labeled_voxel_cloud = super.getLabeledVoxelCloud();
    viewer->addPointCloud(labeled_voxel_cloud, "labeled voxels");
    viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_OPACITY, 0.8, "labeled voxels");

    PointNCloudT::Ptr sv_normal_cloud = super.makeSupervoxelNormalCloud(supervoxel_clusters);
    //We have this disabled so graph is easy to see, uncomment to see supervoxel normals
    //viewer->addPointCloudNormals (sv_normal_cloud,1,0.05f, "supervoxel_normals");

    pcl::console::print_highlight("Getting supervoxel adjacency\n");
    std::multimap supervoxel_adjacency;
    super.getSupervoxelAdjacency(supervoxel_adjacency);
    //To make a graph of the supervoxel adjacency, we need to iterate through the supervoxel adjacency multimap
    for (auto label_itr = supervoxel_adjacency.cbegin(); label_itr != supervoxel_adjacency.cend(); )
    {
        //First get the label
        std::uint32_t supervoxel_label = label_itr->first;
        //Now get the supervoxel corresponding to the label
        pcl::Supervoxel::Ptr supervoxel = supervoxel_clusters.at(supervoxel_label);

        //Now we need to iterate through the adjacent supervoxels and make a point cloud of them
        PointCloudT adjacent_supervoxel_centers;
        for (auto adjacent_itr = supervoxel_adjacency.equal_range(supervoxel_label).first; adjacent_itr != supervoxel_adjacency.equal_range(supervoxel_label).second; ++adjacent_itr)
        {
            pcl::Supervoxel::Ptr neighbor_supervoxel = supervoxel_clusters.at(adjacent_itr->second);
            adjacent_supervoxel_centers.push_back(neighbor_supervoxel->centroid_);
        }
        //Now we make a name for this polygon
        std::stringstream ss;
        ss << "supervoxel_" << supervoxel_label;
        //This function is shown below, but is beyond the scope of this tutorial - basically it just generates a "star" polygon mesh from the points given
        addSupervoxelConnectionsToViewer(supervoxel->centroid_, adjacent_supervoxel_centers, ss.str(), viewer);
        //Move iterator forward to next label
        label_itr = supervoxel_adjacency.upper_bound(supervoxel_label);
    }

    while (!viewer->wasStopped())
    {
        viewer->spinOnce(100);
    }
    return (0);
}

void addSupervoxelConnectionsToViewer(PointT& supervoxel_center,
    PointCloudT& adjacent_supervoxel_centers,
    std::string supervoxel_name,
    pcl::visualization::PCLVisualizer::Ptr& viewer)
{
    vtkSmartPointer points = vtkSmartPointer::New();
    vtkSmartPointer cells = vtkSmartPointer::New();
    vtkSmartPointer polyLine = vtkSmartPointer::New();

    //Iterate through all adjacent points, and add a center point to adjacent point pair
    for (auto adjacent_itr = adjacent_supervoxel_centers.begin(); adjacent_itr != adjacent_supervoxel_centers.end(); ++adjacent_itr)
    {
        points->InsertNextPoint(supervoxel_center.data);
        points->InsertNextPoint(adjacent_itr->data);
    }
    // Create a polydata to store everything in
    vtkSmartPointer polyData = vtkSmartPointer::New();
    // Add the points to the dataset
    polyData->SetPoints(points);
    polyLine->GetPointIds()->SetNumberOfIds(points->GetNumberOfPoints());
    for (unsigned int i = 0; i < points->GetNumberOfPoints(); i++)
        polyLine->GetPointIds()->SetId(i, i);
    cells->InsertNextCell(polyLine);
    // Add the lines to the dataset
    polyData->SetLines(cells);
    viewer->addModelFromPolyData(polyData, supervoxel_name);
}

显示：

修改参数：

    bool disable_transform = true;
    float voxel_resolution = 0.08f;  // 体素分辨率
    float seed_resolution = 0.1f;  // 种子 分辨率
    float color_importance = 0.2f;  // 颜色权重
    float spatial_importance = 0.5f;  // 空间权重
    float normal_importance = 12.0f;   // 法向量角度差