机器人技能学习--数据集剖析
文章目录
- 前言
- 数据总览
- 数据介绍
-
- actions
- robot0_eef_pose
- robot0_eef_quat
- states
- object
- 参考资料
前言
一切为了能自己构建属于自己的数据集，所以，从现有数据集剖析入手。
目前，基于 MimicGen 官方提供的数据集，初始数据集有11组，其中，每组数据中，包含10个demo，每个 demo 的数据构成是一样的，只是轨迹和物体的姿态不一样。
数据总览
为简化工作流程，在本文中，若无特殊标记，均为针对 stack.hdf5 数据集的解析：
-data
-- attr: env_args
-- attr: total
-- demo_0 # 10 组
--- attr: model_file
--- arrt: num_samples
--- actions
--- dones
--- rewards
--- states
--- obs
---- agentview_image
---- object
---- robot0_eef_pos
---- robot0_eef_quat
---- robot0_eef_vel_ang
---- robot0_eef_vel_lin
---- robot0_eye_in_hand_image
---- robot0_gripper_qpos
---- robot0_gripper_qvel
---- robot0_joint_pos
---- robot0_joint_pos_cos
---- robot0_joint_pos_sin
---- robot0_joint_vel
-- demo_1
-- ......
-- demo_9
-mask
-- first_10
针对不同关键词及对应数据的解释，其中 / 表示未知：
| Key | Description | Dim | Get | Robomimic使用方法 | Note |
|---|---|---|---|---|---|
| env_args(attr) | 基础配置信息 | string | 人工定义 | / | 包含场景名称、控制器（OSC_POSE）等信息 |
| total(attr) | / | 1001 | / | / | / |
| model_file(attr) | 场景信息 | string | 人工定义 | / | 详细配置参数，包括机器人位置，物体位姿、属性等 |
| num_samples(attr) | 轨迹点数量 | 87 | 计算 | / | / |
| actions | 动作 | 87 * 7 | 示教 | / | 与控制器类型对应，表示(x, y, z, r, p, y, -1/1)，前6位表姿态，最后一位表示夹具状态，-1表示打开状态，1表示闭合状态 |
| dones | 是否完成任务 | 87 | / | / | 前82位为0，后5位为1 |
| rewards | 奖励参数 | 87 | / | / | 前82位为0.0，后5位为1.0 |
| states | 状态 | 87 * 45 | / | / | Mujoco表示状态的方式 * |
| object | 物体表示 | 87 * 23 | / | / | / |
| robot0_eef_pos | 机器人末端位置 | 87 * 3 | 示教 | / | x, y, z |
| robot0_eef_quat | 末端姿态 | 87 * 4 | 示教 | / | qw, qx, qy, qz |
| robot0_eef_vel_ang | 末端角速度 | 87 * 3 | / | / | |
| robot0_eef_vel_lin | 末端线速度 | 87 * 3 | / | / | |
| robot0_gripper_qpos | 夹具(关节)位置 | 87 * 2 | |||
| robot0_gripper_qvel | 夹具速度 | 87 * 2 | |||
| robot0_joint_pos | 7Dof关节位置 | 87 * 7 | |||
| robot0_joint_pos_cos | / | 87 * 7 | |||
| robot0_joint_pos_sin | / | 87 * 7 | |||
| robot0_joint_vel | 关节速度 | 87 * 7 |
数据介绍
actions
前 10 组数据为：
[0.11900000000000001, -0.027, -0.163, -0.0027573707047849894, 0.09558416903018951, 0.018667636439204216, -1.0]
[0.22200000000000003, -0.004, -0.247, -0.001246497966349125, 0.09782034158706665, 0.02653670310974121, -1.0]
[0.35500000000000004, -0.158, -0.35200000000000004, 0.01000046357512474, 0.08761297911405563, 0.04865993559360504, -1.0]
[0.398, -0.08, -0.515, 0.008693302050232887, 0.07150734215974808, 0.08536489307880402, -1.0]
[0.45799999999999996, -0.056999999999999995, -0.545, 0.006539663765579462, 0.05647878348827362, 0.11847471445798874, -1.0]
[0.439, 0.076, -0.594, -0.00130382867064327, 0.04076612368226051, 0.14484722912311554, -1.0]
[0.439, 0.11199999999999999, -0.599, -0.009466821327805519, 0.025265417993068695, 0.16476278007030487, -1.0]
[0.384, 0.181, -0.542, -0.00946812517940998, 0.02268858812749386, 0.1813860535621643, -1.0]
[0.344, 0.13699999999999998, -0.543, -0.010094424709677696, 0.01842835173010826, 0.19724716246128082, -1.0]
[0.303, 0.131, -0.553, -0.008746521547436714, 0.00791255198419094, 0.21429434418678284, -1.0]
robot0_eef_pose
前 10 组数据为：
[-0.10677228640382701, -0.007912818970765859, 0.9978511242350278]
[-0.10621354303360744, -0.006935816058219822, 0.9979395750460012]
[-0.10536093958873115, -0.006315994512394012, 0.9957368956011524]
[-0.10292113662390937, -0.006428708338244041, 0.9913538586666457]
[-0.09888318372052782, -0.006433863197485929, 0.985309696980578]
[-0.09367487660808188, -0.006261336940084016, 0.9782275947352779]
[-0.08798568739163559, -0.005638058018654422, 0.9705599359873655]
[-0.08209873483921205, -0.004743362303315956, 0.962652752983167]
[-0.07657093954923748, -0.003287089779160534, 0.9550401634889241]
[-0.07158081623646902, -0.001681889698740893, 0.9477366260562331]
robot0_eef_quat
前 10 组数据为：
[0.9987497020281128, -0.005340784012888566, 0.04969201885270823, 0.0011009029139912377]
[0.998764328193982, -0.0037890852616592205, 0.049544501455921285, 0.0008955085639633759]
[0.9989091769069497, -0.0020599392656113213, 0.04664085209166352, 0.0009186171488014434]
[0.999086829166481, -0.0004595555938569085, 0.04271319705277585, 0.0009377591661813914]
[0.9992429301494844, 0.0028395374951761805, 0.038795429035699124, 0.0006467294813540437]
[0.9993372250884551, 0.008313254230825684, 0.0354395485249257, 0.00019686784783459234]
[0.9993433766030909, 0.015947470957228708, 0.03253290289028111, -0.0003225509772582462]
[0.9992261832040216, 0.025115127092460332, 0.030262966085433277, -0.0006465867030544439]
[0.9989703212844835, 0.03539576655809851, 0.028364696315765554, -0.0009385656368913736]
[0.9985707083451394, 0.04633552838823607, 0.02661470086373277, -0.0011031509432972032]
states
ipdb> f['data']['demo_0']['states'][0]
array([ 0. , 0.00566507, 0.23094458, -0.01965714, -2.60890608,
0.01534579, 2.93922448, 0.7684602 , 0.020833 , -0.020833 ,
-0.07533379, 0.00950491, 0.83 , -0.89364334, 0. ,
0. , 0.44877787, -0.02757463, 0.07990465, 0.835 ,
0.59937795, 0. , 0. , 0.80046616, 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. ])
ipdb> f['data']['demo_0']['states'][-1]
array([ 4.30000000e+00, 5.59799918e-02, 7.21294844e-01, 7.20511426e-02,
-2.15581180e+00, -1.09002756e-01, 2.89116760e+00, 6.91851452e-01,
3.40614251e-02, -3.39791898e-02, -1.38892392e-02, 8.35409363e-02,
8.69026681e-01, -8.13647281e-01, 1.67608256e-03, 2.73918668e-03,
5.81349971e-01, -2.75145818e-02, 7.99156301e-02, 8.24690121e-01,
5.99382437e-01, 8.13167534e-04, 8.85709985e-04, 8.00461897e-01,
5.07794596e-02, -3.80039429e-03, -2.87840718e-02, -5.71935216e-02,
-4.75354677e-02, 1.49040575e-01, 9.27308822e-02, 5.80022004e-02,
-5.79495064e-02, 2.90350009e-03, -1.17890268e-03, 4.38212894e-03,
-1.20456301e-01, 1.09043320e-01, 2.78845797e-04, -2.27485074e-05,
4.12241415e-05, -1.23992816e-04, -4.21637708e-04, 1.86013180e-03,
-6.15372096e-06])
object
ipdb> f['data']['demo_0']['obs']['object'][0]
array([-0.07533379, 0.00950491, 0.83 , 0. , 0. ,
0.44877787, -0.89364334, -0.02757463, 0.07990465, 0.835 ,
0. , 0. , 0.80046616, 0.59937795, 0.03143849,
0.01741772, -0.16785112, 0.07919765, 0.08781747, -0.16285112,
0.04775916, 0.07039975, 0.005 ])
ipdb> f['data']['demo_0']['obs']['object'][-1]
array([-1.38892392e-02, 8.35409363e-02, 8.69026681e-01, 1.67608256e-03,
2.73918668e-03, 5.81349971e-01, -8.13647281e-01, -2.75145818e-02,
7.99156301e-02, 8.24690121e-01, 8.13167534e-04, 8.85709985e-04,
8.00461897e-01, 5.99382437e-01, 9.96283912e-03, 2.85510509e-03,
-8.71079446e-03, -3.66250354e-03, -7.70201109e-04, -5.30473545e-02,
-1.36253427e-02, -3.62530620e-03, -4.43365600e-02])
参考资料
robomimic-study