(6)点云数据处理学习——RGBD图

1、主要参考

(1)官网的地址

RGBD images — Open3D 0.16.0 documentation

2、实现和测试

 2.1 功能说明

Open3D有一个用于图像的数据结构。它支持各种函数,如read_image、write_image、filter_image和draw_geometries。一个Open3D图像可以直接转换为numpy数组。

一个Open3D RGBDImage由两个图像rgbimage .depth和rgbimage .color组成。我们要求两个图像注册到相同的相机帧和具有相同的分辨率。下面的教程展示了如何从许多著名的RGBD数据集中读取和使用RGBD图像。

2.2Redwood 数据集(Redwood dataset) 

(1)数据集说明

在本节中,我们将展示如何从Redwood数据集[Choi2015]_读取和可视化RGBDImage。

Redwood格式在16位单通道图像中存储深度。整数值表示以毫米为单位的深度测量。它是Open3D解析深度图像的默认格式。

(2)使用的函数

create_from_color_and_depth(  color_raw, depth_raw)

功能说明:默认的转换函数create_rgbd_image_from_color_and_depth从一对彩色和深度图像创建一个RGBDImage。彩色图像被转换为灰度图像,存储在浮动范围[0,1]中。深度图像存储在float中,以米为单位表示深度值。

(3)测试代码

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt


print("Read Redwood dataset")
redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    color_raw, depth_raw)
print(rgbd_image)

注意open3d自动下载解压后在下面位置

(6)点云数据处理学习——RGBD图_第1张图片

 几张椅子的图挺漂亮的

(6)点云数据处理学习——RGBD图_第2张图片

2.2.1显示一下RGB图和深度图

(1)可以通过np.asarray直接对深度图和彩色图进行处理

color_img = np.asarray(rgbd_image.color)
depth_img = np.asarray(rgbd_image.depth)
print(np.size(color_img))
print(color_img[0][0])
print(depth_img[0][0])

(2)测试代码

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt


print("Read Redwood dataset")
redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    color_raw, depth_raw)
print(rgbd_image)

#640*480的图
color_img = np.asarray(rgbd_image.color)
depth_img = np.asarray(rgbd_image.depth)
print(np.size(color_img))
print(color_img[0][0])
print(depth_img[0][0])


plt.subplot(1, 2, 1)
plt.title('Redwood grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('Redwood depth image')
plt.imshow(rgbd_image.depth)
plt.show()

(3)测试的结果

(6)点云数据处理学习——RGBD图_第3张图片

2.2.2RGBD图像可以转换为点云图

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt


print("Read Redwood dataset")
redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    color_raw, depth_raw)
print(rgbd_image)

#640*480的图
# color_img = np.asarray(rgbd_image.color)
# depth_img = np.asarray(rgbd_image.depth)
# print(np.size(color_img))
# print(color_img[0][0])
# print(depth_img[0][0])


# plt.subplot(1, 2, 1)
# plt.title('Redwood grayscale image')
# plt.imshow(rgbd_image.color)
# plt.subplot(1, 2, 2)
# plt.title('Redwood depth image')
# plt.imshow(rgbd_image.depth)
# plt.show()

pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
# Flip it, otherwise the pointcloud will be upside down
# pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
# o3d.visualization.draw_geometries([pcd], zoom=0.5)

o3d.visualization.draw_geometries([pcd])

(6)点云数据处理学习——RGBD图_第4张图片

 这里使用了PinholeCameraIntrinsicParameters。PrimeSenseDefault作为默认相机参数。它的图像分辨率为640x480,焦距(fx, fy) =(525.0, 525.0),光心(cx, cy) =(319.5, 239.5)。单位矩阵被用作默认的外部参数。pcd.transform在点云上应用上下翻转转换,以达到更好的可视化目的。

2.3 SUN数据集(SUN dataset)

 (1)说明

在本节中,我们将展示如何读取和可视化SUN数据集[Song2015]_的RGBDImage。

本教程与上面处理Redwood数据集的教程几乎相同。唯一的区别是我们使用了转换函数create_rgbd_image_from_sun_format来解析SUN数据集中的深度图像。

(2)测试

1)下载后的数据集如下图所示

(6)点云数据处理学习——RGBD图_第5张图片

 2)测试代码

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt



print("Read SUN dataset")
sun_rgbd = o3d.data.SampleSUNRGBDImage()
color_raw = o3d.io.read_image(sun_rgbd.color_path)
depth_raw = o3d.io.read_image(sun_rgbd.depth_path)
rgbd_image = o3d.geometry.RGBDImage.create_from_sun_format(color_raw, depth_raw)
print(rgbd_image)


plt.subplot(1, 2, 1)
plt.title('SUN grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('SUN depth image')
plt.imshow(rgbd_image.depth)
plt.show()

pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
# Flip it, otherwise the pointcloud will be upside down
# pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
# o3d.visualization.draw_geometries([pcd], zoom=0.5)
o3d.visualization.draw_geometries([pcd])

(6)点云数据处理学习——RGBD图_第6张图片

 转变成的点云图如下图所示

(6)点云数据处理学习——RGBD图_第7张图片

2.4 NYU数据集(NYU dataset)

本部分展示如何从NYU数据集[Silberman2012]_读取和可视化RGBDImage。

(1)说明 

本部分教程与上面处理Redwood数据集的教程几乎相同,但有两点不同。首先,纽约大学的图片不是标准的jpg或png格式。因此,我们使用mpimg。imread将彩色图像作为numpy数组读取,并将其转换为Open3D图像。调用另一个辅助函数read_nyu_pgm从NYU数据集中使用的特殊大端pgm格式中读取深度图像。其次,我们使用一个不同的转换函数create_rgbd_image_from_nyu_format来解析SUN数据集中的深度图像

(2)测试代码

import open3d as o3d
import numpy as np

import matplotlib.image as mpimg
import re


# This is special function used for reading NYU pgm format
# as it is written in big endian byte order.
def read_nyu_pgm(filename, byteorder='>'):
    with open(filename, 'rb') as f:
        buffer = f.read()
    try:
        header, width, height, maxval = re.search(
            b"(^P5\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    except AttributeError:
        raise ValueError("Not a raw PGM file: '%s'" % filename)
    img = np.frombuffer(buffer,
                        dtype=byteorder + 'u2',
                        count=int(width) * int(height),
                        offset=len(header)).reshape((int(height), int(width)))
    img_out = img.astype('u2')
    return img_out


print("Read NYU dataset")
# Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
# MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
nyu_rgbd = o3d.data.SampleNYURGBDImage()
color_raw = mpimg.imread(nyu_rgbd.color_path)
depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
color = o3d.geometry.Image(color_raw)
depth = o3d.geometry.Image(depth_raw)
rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
print(rgbd_image)

下载解压后的图片如下图所示

(6)点云数据处理学习——RGBD图_第8张图片

 (3)显示一下

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt
import matplotlib.image as mpimg
import re


# This is special function used for reading NYU pgm format
# as it is written in big endian byte order.
def read_nyu_pgm(filename, byteorder='>'):
    with open(filename, 'rb') as f:
        buffer = f.read()
    try:
        header, width, height, maxval = re.search(
            b"(^P5\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    except AttributeError:
        raise ValueError("Not a raw PGM file: '%s'" % filename)
    img = np.frombuffer(buffer,
                        dtype=byteorder + 'u2',
                        count=int(width) * int(height),
                        offset=len(header)).reshape((int(height), int(width)))
    img_out = img.astype('u2')
    return img_out


print("Read NYU dataset")
# Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
# MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
nyu_rgbd = o3d.data.SampleNYURGBDImage()
color_raw = mpimg.imread(nyu_rgbd.color_path)
depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
color = o3d.geometry.Image(color_raw)
depth = o3d.geometry.Image(depth_raw)
rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
print(rgbd_image)

plt.subplot(1, 2, 1)
plt.title('NYU grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('NYU depth image')
plt.imshow(rgbd_image.depth)
plt.show()

(6)点云数据处理学习——RGBD图_第9张图片

 (4)转换为点云图像显示

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt
import matplotlib.image as mpimg
import re


# This is special function used for reading NYU pgm format
# as it is written in big endian byte order.
def read_nyu_pgm(filename, byteorder='>'):
    with open(filename, 'rb') as f:
        buffer = f.read()
    try:
        header, width, height, maxval = re.search(
            b"(^P5\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n])*"
            b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    except AttributeError:
        raise ValueError("Not a raw PGM file: '%s'" % filename)
    img = np.frombuffer(buffer,
                        dtype=byteorder + 'u2',
                        count=int(width) * int(height),
                        offset=len(header)).reshape((int(height), int(width)))
    img_out = img.astype('u2')
    return img_out


print("Read NYU dataset")
# Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
# MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
nyu_rgbd = o3d.data.SampleNYURGBDImage()
color_raw = mpimg.imread(nyu_rgbd.color_path)
depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
color = o3d.geometry.Image(color_raw)
depth = o3d.geometry.Image(depth_raw)
rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
print(rgbd_image)

plt.subplot(1, 2, 1)
plt.title('NYU grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('NYU depth image')
plt.imshow(rgbd_image.depth)
plt.show()


pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
# Flip it, otherwise the pointcloud will be upside down
# pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
# o3d.visualization.draw_geometries([pcd], zoom=0.5)
o3d.visualization.draw_geometries([pcd])

(6)点云数据处理学习——RGBD图_第10张图片

2.5 TUM数据集(TUM dataset)

(1)说明

本节展示如何从TUM数据集[Strum2012]_读取和可视化RGBDImage。

本教程与上面处理Redwood数据集的教程几乎相同。唯一的区别是我们使用转换函数create_rgbd_image_from_tum_format来解析TUM数据集中的深度图像。

(2)测试

注意:官网教程有误

# tum_rgbd = o3d.data.SampleSUNRGBDImage()

tum_rgbd = o3d.data.SampleTUMRGBDImage()

或者手动下载

 https://github.com/isl-org/open3d_downloads/releases/download/20220201-data/SampleTUMRGBDImage.zip 

import open3d as o3d
import numpy as np
from matplotlib import pyplot as plt


print("Read TUM dataset")
# 方法一:官网下载
#tum_rgbd = o3d.data.SampleTUMRGBDImage()
# color_raw = o3d.io.read_image(tum_rgbd.color_path)
# depth_raw = o3d.io.read_image(tum_rgbd.depth_path)
## 方法二 本地下载后使用 https://github.com/isl-org/open3d_downloads/releases/download/20220201-data/SampleTUMRGBDImage.zip 
tum_rgbd_color_path = "D:/RGBD_CAMERA/python_3d_process/SampleTUMRGBDImage/TUM_color.png"
tum_rgbd_depth_path = "D:/RGBD_CAMERA/python_3d_process/SampleTUMRGBDImage/TUM_depth.png"
color_raw = o3d.io.read_image(tum_rgbd_color_path)
depth_raw = o3d.io.read_image(tum_rgbd_depth_path)
rgbd_image = o3d.geometry.RGBDImage.create_from_tum_format(color_raw, depth_raw)
print(rgbd_image)

plt.subplot(1, 2, 1)
plt.title('TUM grayscale image')
plt.imshow(rgbd_image.color)
plt.subplot(1, 2, 2)
plt.title('TUM depth image')
plt.imshow(rgbd_image.depth)
plt.show()

pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    rgbd_image,
    o3d.camera.PinholeCameraIntrinsic(
        o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
# # Flip it, otherwise the pointcloud will be upside down
# pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
# o3d.visualization.draw_geometries([pcd], zoom=0.35)
o3d.visualization.draw_geometries([pcd])

(6)点云数据处理学习——RGBD图_第11张图片

 转换后的点云图像如下

(6)点云数据处理学习——RGBD图_第12张图片

 

 

 

你可能感兴趣的:(深度学习,学习)