【计算机图形学】【代码复现】A-SDF中的数据集制作与数据生成

Follow A-SDF 的Data Generation部分：
We follow
(1) ANSCH to create URDF for shape2motion dataset
(1-2) URDF2OBJ（本人认为是1-2之间需要进行的重要的过渡部分）
(2) Manifold to create watertight meshes
(3) and modified mesh_to_sdf for generating sampled points and sdf values.

1. ANSCH to create URDF for shape2motion dataset

follow这个github: ANSCH

（1）克隆数据集，编辑自己的路径信息

git clone https://github.com/dragonlong/articulated-pose.git # 克隆仓库
# global_info主要放路径信息，通过下行命令编辑路径信息
vim global_info.py

在global_info.py中，主要修改192行的self.base_path路径，改成克隆下来文件夹的位置，我这里改成/mnt/d/sdc/liutong/codes/articulated-pose。

（2）Shape2Motion数据集的解压和使用

因为这里需要使用到Shape2Motion数据集，所以下载一下，Shape2Motion项目主页：Shape2Motion，Shape2Motion下载链接：Shape2Motion下载链接

下载好Shape2Motion数据集后，在zip文件路径做如下操作

unzip 'Motion Dataset v0.zip' # 解压数据集
mv 'Motion Dataset v0' shape2motion # 重命名数据集
mv shape2motion /mnt/d/sdc/liutong/codes/articulated-pose/dataset/ # 将数据集移动到目标目录下

（3）json2urdf

follow github上的操作：

cd tools # 进入工具文件夹
python json2urdf.py	# 执行python2urdf

可能会出现以下报错，根据报错进行修改后重复运行python json2urdf.py

这时候报错1：

Traceback (most recent call last):
  File "/mnt/d/sdc/liutong/codes/articulated-pose/tools/json2urdf.py", line 62, in <module>
    all_objs     = os.listdir( base_path  + dataset  + '/objects' )
FileNotFoundError: [Errno 2] No such file or directory: '/mnt/d/sdc/liutong/codes/articulated-pose/dataset/shape2motion/objects'
(gapartnet) root@szu-SYS-7048GR-TR:/mnt/d/sdc/liutong/codes/articulated-pose/tools# ls -l ../dadtaset/shape2motion

经检查，是Shape2Motion下没有objects文件夹，所以我将报错的62行进行更改如下，确保这部分可以正常运行：

# 从
all_objs     = os.listdir( base_path  + dataset  + '/objects' )
# 更改至
all_objs     = os.listdir( base_path  + dataset )

同理，73行中的路径代码因为也有objects，所以肯定也会出问题，同样进行修改：

# 从
instances_per_obj = sorted(glob.glob(base_path  + dataset  + '/objects/' + obj_name + '/*'))
# 更改至
instances_per_obj = sorted(glob.glob(base_path  + dataset + '/' + obj_name + '/*'))

接着会 报错2：

Traceback (most recent call last):
  File "/mnt/d/sdc/liutong/codes/articulated-pose/tools/json2urdf.py", line 87, in <module>
    with open(json_name[0]) as json_file:
IndexError: list index out of range

这是因为前面运行的时候在dataset/shape2motion/下生成了一个urdf文件夹，这里面是生成的结果，所以会导致读取不到.json文件，所以只需要执行：

rm -rf dataset/shape2motion/urdf

即可。或者一劳永逸地改掉urdf的保存路径（前面的更改我没有把旧代码删掉，而注释旧代码，在下面添加新代码，所以我这里保存路径是在85行）。这里有个小小的担心是会不会后续条件下也要对urdf的路径进行修改：

# 从
save_dir     = base_path + '/' + dataset  + '/urdf/{}/{}'.format(obj_name, instance_name) # todo
# 更改至
save_dir     = base_path + 'urdf/{}/{}'.format(obj_name, instance_name)

接着会 报错3：

Traceback (most recent call last):
  File "/mnt/d/sdc/liutong/codes/articulated-pose/tools/json2urdf.py", line 224, in <module>
    f.write('{}/{}\t'.format(obj_j['revolute'], obj_j['prismatic']))
KeyError: 'revolute'

发现是因为有的obj_j为空造成的，故更改如下：

# 从
with open(base_path + '/' + dataset + '/statistics.txt', "a+") as f:
	for obj_j in object_joints:
		f.write('{}/{}\t'.format(obj_j['revolute'], obj_j['prismatic']))
    f.write('\n')
# 更改至
with open(base_path + '/' + dataset + '/statistics.txt', "a+") as f:
	for obj_j in object_joints:
		if not len(obj_j) == 0:
			f.write('{}/{}\t'.format(obj_j['revolute'], obj_j['prismatic']))
		else:
			f.write('NAN/NAN\t')
    f.write('\n')

虽然ANSCH后续还有一些渲染数据的操作，但是这里我们只需要拿到URDF就足够了

1-2. URDF2OBJ

在这一步，我follow了一个已经写好的github代码：urdf_to_obj
由于我们得到的文件形式是这样的：


我们要取的是这下边的syn.urdf来进行最终结果的合成，我根据这个文件格式改了一下我上面提及的github的代码，更改后的代码urdf_to_obj如下，参数释义：
--urdf_file_dir：urdf文件夹的路径
--output_dir：输出文件夹的路径
--angle：用于给输出obj文件命名时候用的，你生成的是多少度就写多少度就好了（并不是在这里指定角度就能够实现生成）

from urdfpy import URDF
import numpy as np
import trimesh
import argparse
import meshes_extracted
import os

def merge_meshes(mesh_list):
    """Merge a list of meshes into a single mesh
    
    Parameters:
    - mesh_list (list): list of trimesh.Trimesh objects
    
    Returns:
    - mesh (trimesh.Trimesh): merged mesh"""

    # Get vertices and faces
    verts_list = [mesh.vertices for mesh in mesh_list]
    faces_list = [mesh.faces for mesh in mesh_list]

    # Num of faces per mesh
    faces_offset = np.cumsum([v.shape[0] for v in verts_list], dtype=np.float32) 

    # Compute offset for faces, otherwise they all start from 0
    faces_offset = np.insert(faces_offset, 0, 0)[:-1]            

    verts = np.vstack(verts_list)
    faces = np.vstack([face + offset for face, offset in zip(faces_list, faces_offset)])

    # Create single mesh
    mesh = trimesh.Trimesh(verts, faces)

    return mesh


def main(args):
    """Extract meshes from a URDF file and save them as .obj files"""

    # Load urdf file dir
    urdf_file_dir = args.urdf_file_dir

    output_dir = args.output_dir

    # Load other parameters
    object_angle = args.angle

    # Get the sub_dir under urdf_file_dir, and then 0001/0002/0003/0004/0005
    urdf_root = sorted(os.listdir(urdf_file_dir))

    for urdf_dir in urdf_root:
        # urdf_file_path be like 0001/syn.urdf, 0002/syn.urdf, 0003/syn.urdf, etc
        urdf_file_path = os.path.join(urdf_file_dir, urdf_dir, 'syn.urdf')
        output_path = os.path.join(output_dir, f'{urdf_dir}art{object_angle}.obj')

        # Load urdf file
        robot = URDF.load(urdf_file_path)

        meshes = robot.visual_trimesh_fk()

        mesh_list = []

        for idx, mesh in enumerate(meshes):

            pose = meshes[mesh]   # 4 x 4 : rotation + translation

            translation = pose[:3, 3][:, None]
            
            # Add a column of zeros to the vertices
            verts = np.array(mesh.vertices)
            zeros = np.zeros((verts.shape[0], 1))
            new_verts = np.hstack((verts, zeros))

            # Apply pose to the vertices
            verts_pose = pose @ new_verts.transpose(1, 0) 
            verts_pose = verts_pose[:3, :] + translation   
            verts_pose = verts_pose.transpose(1, 0)
                
            mesh_extracted = trimesh.Trimesh(verts_pose, mesh.faces)

            mesh_list.append(mesh_extracted)

        # Merge meshes
        mesh_merged = merge_meshes(mesh_list)

        # Save merged meshes
        print('save ' + str(output_path))
        trimesh.exchange.export.export_mesh(mesh_merged, output_path, file_type='obj')


if __name__=='__main__':
    parser = argparse.ArgumentParser()

    parser.add_argument(
        "--urdf_file_dir", default='', type=str, help="Path to the .urdf file", required=True
    )
    parser.add_argument(
        "--output_dir", default='', type=str, help="Path to the output dir", required=True
    )
    parser.add_argument(
        "--angle", default='', type=str, help="Angle of objects, be used to name the file", required=True
    )
    args = parser.parse_args()

    main(args)

运行示例：

 python urdf_to_obj_group.py --urdf_file_dir /mnt/d/sdc/liutong/codes/articulated-pose/dataset/shape2motion/urdf/door --output_dir obj_tmp --angle 90

报错如下：

Traceback (most recent call last):
  File "urdf_to_obj_group.py", line 104, in <module>
    main(args)
  File "urdf_to_obj_group.py", line 56, in main
    robot = URDF.load(urdf_file_path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 3729, in load
    return URDF._from_xml(node, path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 3926, in _from_xml
    kwargs = cls._parse(node, path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 161, in _parse
    kwargs.update(cls._parse_simple_elements(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 137, in _parse_simple_elements
    v = [t._from_xml(n, path) for n in vs]
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 137, in <listcomp>
    v = [t._from_xml(n, path) for n in vs]
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 181, in _from_xml
    return cls(**cls._parse(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 161, in _parse
    kwargs.update(cls._parse_simple_elements(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 137, in _parse_simple_elements
    v = [t._from_xml(n, path) for n in vs]
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 137, in <listcomp>
    v = [t._from_xml(n, path) for n in vs]
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 1146, in _from_xml
    kwargs = cls._parse(node, path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 161, in _parse
    kwargs.update(cls._parse_simple_elements(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 127, in _parse_simple_elements
    v = t._from_xml(v, path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 181, in _from_xml
    return cls(**cls._parse(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 161, in _parse
    kwargs.update(cls._parse_simple_elements(node, path))
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 127, in _parse_simple_elements
    v = t._from_xml(v, path)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/urdf.py", line 581, in _from_xml
    meshes = load_meshes(fn)
  File "/home/liutong/anaconda3/envs/gapartnet/lib/python3.8/site-packages/urdfpy/utils.py", line 235, in load_meshes
    raise ValueError('At least one mesh must be pmeshesent in file')
ValueError: At least one mesh must be pmeshesent in file

其实使用原始github连接中的python urdf_to_obj.py --urdf_path relative/path/to/urdf来进行试验的话，会发现0001的syn.urdf文件会报这样的错误，而0002的syn.urdf文件就不会报这样的错误。
经过仔细地检查发现，实际上是因为0001的syn.urdf中，mesh - none_motion.obj的vertices数量为空所导致的，但是因为这个错误是包里的错误，所以我也不知道应该怎么进行修改比较合适。
最后我是直接删掉会产生这个异常的文件夹，最后不会产生异常的文件夹应该只有51个。

2. Manifold to create watertight meshes

这一步实际上就是缩减网格数量，因为原本的网格太密集了
follow这个github: Manifold
follow里面的Install进行安装就可以了
然后我根据它的命令，用python写了一个自动执行减少网格数量的程序，其中要用到的参数如下：
--obj_file_dir：放置了URDF2OBJ的那些obj文件的文件夹
--target_dir：输出文件夹路径

import os
import argparse

# res = os.popen('ls').read()
# print(res)

def main(args):
    # Load obj file dir
    obj_file_dir = args.obj_file_dir
    target_dir = args.target_dir

    obj_files = sorted(os.listdir(obj_file_dir))

    for obj in obj_files:
        source_obj = os.path.join(obj_file_dir, obj)
        obj = "manifold_" + obj
        target_obj = os.path.join(target_dir, obj)
        
        res = os.popen('./manifold ' + source_obj + ' ' + target_obj).read()
        print(res)
        

if __name__=='__main__':
    parser = argparse.ArgumentParser()

    parser.add_argument(
        "--obj_file_dir", default='', type=str, help="Path to the .obj file dir", required=True
    )
    parser.add_argument(
        "--target_dir", default='', type=str, help="Path to the target dir", required=True
    )
    args = parser.parse_args()


    main(args)

这是得到的结果：

3. modified mesh_to_sdf for generating sampled points and sdf values

follow 这个github: mesh_to_sdf
按照里面的步骤安装好
这一步实际上就是做sdf采样了，像A-SDF里的话采样好像是保存成一些.npy还是.npz文件的，但是我这里根据我自己的需要，需要采样并保存到.mat文件中，下面提供一个保存到.mat文件中的代码：
--manifold_obj_dir: 上一步manifold减少网格数量后的结果
--output_dir: .mat文件的保存路径

import os

import mesh_to_sdf
# from mesh_to_sdf import sample_sdf_near_surface

import trimesh
import inspect
import pyrender
import numpy as np
import scipy
from scipy.io import savemat
from plyfile import PlyData
import argparse
os.environ['PYOPENGL_PLATFORM'] = 'egl'


def calculate_ply_center(mesh):
    # 获取顶点数据
    vertices = mesh.vertices

    # 提取顶点坐标
    x_coords = vertices[:, 0]
    y_coords = vertices[:, 1]
    z_coords = vertices[:, 2]

    # 计算顶点坐标的平均值
    center_x = np.mean(x_coords)
    center_y = np.mean(y_coords)
    center_z = np.mean(z_coords)

    return center_x, center_y, center_z


def scale_to_specific_sphere(mesh):
    vertices = mesh.vertices - calculate_ply_center(mesh)
    distances = np.linalg.norm(vertices, axis=1)
    vertices /= np.max(distances) / 1.03
    return trimesh.Trimesh(vertices=vertices, faces=mesh.faces)


if __name__ == "__main__":

    print(inspect.getfile(mesh_to_sdf))

    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--manifold_obj_dir", default='./Manifold_result', type=str, help="Path to .obj file dir"
    )
    parser.add_argument(
        "--output_dir", default='./All_result', type=str, help="Path to the .mat dir"
    )
    args = parser.parse_args()

    manifold_obj_dir = args.manifold_obj_dir
    save_dir = args.output_dir

    if not os.path.exists(os.path.join(save_dir, 'surface_pts_n_normal')):
        os.mkdir(os.path.join(save_dir, 'surface_pts_n_normal'))
    if not os.path.exists(os.path.join(save_dir, 'free_space_pts')):
        os.mkdir(os.path.join(save_dir, 'free_space_pts'))
    save_surface_dir = os.path.join(save_dir, 'surface_pts_n_normal')
    save_free_dir = os.path.join(save_dir, 'free_space_pts')

    manifold_dirs = os.listdir(manifold_obj_dir)
    for dirs in manifold_dirs:
        files = os.listdir(os.path.join(manifold_obj_dir, dirs))
        for file_ in files:
            print("Process " + str(file_) + " ......")
            mesh = trimesh.load(os.path.join(manifold_obj_dir, dirs, file_))
            mesh = scale_to_specific_sphere(mesh)

            surface = mesh_to_sdf.get_surface_point_cloud(mesh, surface_point_method='scan', sample_point_count=1000)
            surface_points = surface.points
            surface_normals = surface.normals
            surface_data = np.concatenate((surface_points, surface_normals), axis=1)
            surface_data = {'p': surface_data}

            free_p, free_sdf = mesh_to_sdf.sample_sdf_in_cube(mesh, surface_point_method='scan', number_of_points=1000,
                                                              sample_point_count=1000, sign_method='depth')
            free_sdf = free_sdf.reshape(-1, 1)
            free_data = np.concatenate((free_p, free_sdf), axis=1)
            free_data = {'p_sdf': free_data}

            print("Saving " + str(str(file_.split("_")[-1]).split('.')[0])+'.mat'  + " to " + str(os.path.join(save_surface_dir, str(str(file_.split("_")[-1]).split('.')[0])+'.mat')))
            savemat(os.path.join(save_surface_dir, str(str(file_.split("_")[-1]).split('.')[0])+'.mat'), surface_data)
            print("Saving " + str(str(file_.split("_")[-1]).split('.')[0])+'.mat'  + " to " + str(os.path.join(save_free_dir, str(str(file_.split("_")[-1]).split('.')[0])+'.mat')))
            savemat(os.path.join(save_free_dir, str(str(file_.split("_")[-1]).split('.')[0])+'.mat'), free_data)

# # 加载DIF原本的free_space_pts，查看可视化效果
# result = scipy.io.loadmat('free_space_cbc47018135fc1b1462977c6d3c24550.mat')['p_sdf']
# free_p = result[:, :3]
# free_sdf = result[:, 3:]
# free_sdf = free_sdf.reshape(-1)
# print(free_p.shape)
# print(free_sdf.shape)


# # 可视化free_space_pts的点
# colors = np.zeros(free_p.shape)
# colors[free_sdf < 0, 2] = 1
# colors[free_sdf > 0, 0] = 1
# cloud = pyrender.Mesh.from_points(free_p, colors=colors)
# scene = pyrender.Scene()
# scene.add(cloud)
# viewer = pyrender.Viewer(scene, use_raymond_lighting=True, point_size=2)

得到的结果如下：

最终得到的这些SDF对文件，只需要放到目标路径文件夹下进行使用就好了