UR+RealSense手眼标定（eye-to-hand）

01 手眼标定的原理

基坐标系（base_tree）和相机（camera_tree）两个坐标系属于不同的tree，通过将标签贴到手上，相机识别出标签的position和orention，并通过easy_handeye标定包得到tool0(机械手)，进一步得到相对于base的位置关系。即子坐标系（camera_rgb_optical_frame）到父坐标系（base_link）之间的关系。

在之后如果摄像头识别到物体的位置（在camera坐标系下），即可通过transform(这种转换关系)，转化为base(也就是机器人知道的自己的位置坐标系)坐标系下的位置，这样机器人就通过转化关系得到相机识别到的位置实际在空间中的位置。

---

对于手眼标定，场景主要有以下两种，

• eye-to-hand，眼在手外。
  这种场景下我们已知机械臂终端end_link与base_link、相机camera_link与识别物体object_link之间的关系；
  需要求解camera_link与base_link之间的变换。

• eye-in-hand，眼在手上。
  这种场景base_link和机械臂各关节joint_link、end_link已经通过URDF发布了；
  只需要求解camera_link与end_link之间的变换。
  

02 准备工作

所用系统及硬件版本：

• Ubuntu18.04（ROS Melodic）
• UR3机械臂（CB3.12）
• RealSense D435i

---

• 安装ur功能包（Universal_Robots_ROS_Driver驱动）
• 安装realsense-ros

2.1 安装aruco_ros

cd ~/ur_ws/src
git clone -b melodic-devel https://github.com/pal-robotics/aruco_ros.git
cd ..
catkin_make

2.2 安装vision_visp / visp_hand2eye_calibration

cd ~/ur_ws/src
sudo apt-get install ros-melodic-visp
git clone -b melodic-devel https://github.com/lagadic/vision_visp.git
cd ..
catkin_make --pkg visp_hand2eye_calibration
catkin_make

2.3 安装easy_handeye

cd ~/ur_ws/src
git clone https://github.com/IFL-CAMP/easy_handeye
cd ..
catkin_make

03 眼在手外

3.1 修改标定 launch 文件

标定过程需启动ur3机械臂的相关节点，realsense节点，aruco节点，easy_handeye节点，可以写一个 launch 文件同时启动上述节点，也可以分别启动。

easy_handeye 包中给出了用一个 launch 文件实现的示例，在如下的目录中：/home/guyue/ur_ws/src/easy_handeye/docs/example_launch/ur5_kinect_calibration.launch，这里只有ur5+kinect的，ur3+realsense的修改即可。

在ur5_kinect_calibration.launch基础上进行修改：

• 复制launch文件
  将launch文件拷贝到easy_handeye功能包的launch目录中，并修改文件名字

  cd ~/ur_ws/src/easy_handeye/docs/example_launch
  cp ur5_kinect_calibration.launch ~/ur_ws/src/easy_handeye/easy_handeye/launch/ur3_eye_to_hand_calibration.launch
  

• 修改launch文件
  修改launch文件如下：
  注意：realsense和ur机械臂最好分开启动，否则会有报错

  <launch>
      <arg name="namespace_prefix" default="ur3_realsense_handeyecalibration" />
  
      <arg name="robot_ip" doc="The IP address of the UR3 robot" />
  
      <arg name="marker_size" doc="Size of the ArUco marker used, in meters" default="0.1" />
      <arg name="marker_id" doc="The ID of the ArUco marker used" default="323" />
  
      
      
  
      
      <node name="aruco_tracker" pkg="aruco_ros" type="single">
          <remap from="/camera_info" to="/camera/color/camera_info" />
          <remap from="/image" to="/camera/color/image_raw" />
          <param name="image_is_rectified" value="true"/>
          <param name="marker_size"        value="$(arg marker_size)"/>
          <param name="marker_id"          value="$(arg marker_id)"/>
          <param name="reference_frame"    value="camera_color_frame"/>
          <param name="camera_frame"       value="camera_color_frame"/>
          <param name="marker_frame"       value="camera_marker" />
      node>
  
      
      
  
      
      <include file="$(find easy_handeye)/launch/calibrate.launch" >
          <arg name="namespace_prefix" value="$(arg namespace_prefix)" />
          <arg name="eye_on_hand" value="false" />
  
          <arg name="tracking_base_frame" value="camera_color_frame" />
          <arg name="tracking_marker_frame" value="camera_marker" />
          <arg name="robot_base_frame" value="base" />
          <arg name="robot_effector_frame" value="tool0_controller" />
  
          <arg name="freehand_robot_movement" value="false" />
          <arg name="robot_velocity_scaling" value="0.5" />
          <arg name="robot_acceleration_scaling" value="0.2" />
      include>
      	
  launch>
  

• 分析launch文件
  这里主要是启动realsense相机，启动ArUco，启动UR3机械臂，启动easy_handeye 四部分：

  • 1/ Realsense435节点
    把rs_camera.launch文件导入
  • 2/ ArUco节点
    • 修改：/camera_info / /image / reference_frame 和 camera_frame
    • 从https://chev.me/arucogen/中下载aruco二维码并打印出来
    • 注意：
      ❤ Dictionary 一定要选 Original ArUco
      ❤ Marker ID 和 Marker size 自选，在launch 文件中做相应的修改
      ❤ 打印时注意选择原始大小，否则要测量一下打印出来的真实大小
  • 3/ UR3节点
    • 这里用了 ur_robot_driver 包，而没有用原始的 ur_bringup 包
    • 修改机器人的真实 ip
  • 4/ easy_handeye节点
    • ：眼在手外时，value 为 false
    • tracking_base_frame ：为相机坐标系 camera_color_frame
    • robot_base_frame ：为机器人基座坐标系，示例里写的是 base_link，我在 rviz 中查看 base 才是真实的基座坐标系
    • robot_effector_frame：为工具坐标系，因为我安装了 robotiq相机/力传感器和夹爪，所以TCP 改变了

3.2 启动 launch 文件，开始标定

3.2.1 启动realsense

roslaunch realsense2_camera rs_camera.launch

3.2.2 启动ur机械臂

• ① 启动机械臂

  roslaunch ur_robot_driver ur3_bringup.launch limited:=true robot_ip:=192.168.56.10
  

• ② 启动示教器

• ③ 启动moveit

  roslaunch ur3_moveit_config ur3_moveit_planning_execution.launch limited:=true
  

3.2.3 启动手眼标定的其他程序

roslaunch easy_handeye ur3_eye_to_hand_calibration.launch

3.3 标定

launch文件启动后，会出现3个窗口。



标定过程：

• 首先打开一个终端，输入rqt，点击菜单栏的 Plugins -> Visulization -> Image View，选择 /aruco_tracker/result 话题。当识别出aruco码时，则可以进行下一步。
• 在第三个屏幕中点击check starting pose，若检查成功，界面会出现： 0/17，ready to start
  
• 在第三个窗口点击next pose -> plan -> execute，当点完 plan ，出现绿色框，则说明规划成功，然后可以点击 execute让机械臂执行动作
  
• 然后在第二个窗口，点击take sample采样
  
• 然后再次回到第三个窗口使机械臂执行规划动作。
  当17个动作执行完成，回到第二个界面，点击compute，然后出现结果的姿态矩阵，然后可以点击save保存
  
  

04 报错

以下的报错主要需要注意3点：

• ① 单独启动ur机械臂和realsense相机，不要放到launch文件里一起启动

• ② 三个标定窗口都启动后，注意再打开一个rqt窗口，确定识别出aruco码

• ③ 如果有关于opencv的报错，需要升级opencv的版本

  pip2 install opencv-python==4.2.0.32
  

下面是详细的报错信息及解决方法

4.1 unused args [limited] for include

如果不注释掉lauch文件中ur机械臂启动的部分，会遇到下述问题，所以最好的办法是ur机械臂单独启动

• 报错1：

  guyue@guyue:~/ur_ws$ roslaunch easy_handeye ur3_eye_to_hand_calibration.launch
  ... logging to /home/guyue/.ros/log/7697ce46-6c91-11ec-9d22-38fc98e4336a/roslaunch-guyue-23663.log
  Checking log directory for disk usage. This may take a while.
  Press Ctrl-C to interrupt
  Done checking log file disk usage. Usage is <1GB.
  
  RLException: unused args [limited] for include of [/home/guyue/ur_ws/src/Universal_Robots_ROS_Driver/ur_robot_driver/launch/ur3_bringup.launch]
  The traceback for the exception was written to the log file
  

  解决： 将ur机械臂启动中的limited注释
• 报错2：

  guyue@guyue:~/ur_ws$ roslaunch easy_handeye ur3_eye_to_hand_calibration.launch
  ... logging to /home/guyue/.ros/log/ecddcd44-6c91-11ec-9d22-38fc98e4336a/roslaunch-guyue-23704.log
  Checking log directory for disk usage. This may take a while.
  Press Ctrl-C to interrupt
  Done checking log file disk usage. Usage is <1GB.
  
  RLException: unused args [limited] for include of [/home/guyue/ur_ws/src/fmauch_universal_robot/ur3_moveit_config/launch/ur3_moveit_planning_execution.launch]
  The traceback for the exception was written to the log file
  

  解决： 将ur机械臂启动中moveit启动的部分的limited注释

4.2 关于opencv版本的问题

• 报错3：

  [ WARN] [1641220271.220611210]: normalizeImageIllumination is unimplemented!
  [ INFO] [1641220271.250652945]: rviz version 1.13.17
  [ INFO] [1641220271.250695191]: compiled against Qt version 5.9.5
  [ INFO] [1641220271.250704120]: compiled against OGRE version 1.9.0 (Ghadamon)
  [ INFO] [1641220271.253321240]: Forcing OpenGl version 0.
  [ INFO] [1641220271.336844200]: Stereo is NOT SUPPORTED
  [ INFO] [1641220271.336903845]: OpenGL device: Mesa DRI Intel(R) UHD Graphics (CML GT2)
  [ INFO] [1641220271.336918119]: OpenGl version: 3.0 (GLSL 1.3).
  Traceback (most recent call last):
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/scripts/calibrate.py", line 5, in <module>
      from easy_handeye.handeye_server import HandeyeServer
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_server.py", line 13, in <module>
      from easy_handeye.handeye_calibration_backend_opencv import HandeyeCalibrationBackendOpenCV
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_calibration_backend_opencv.py", line 4, in <module>
      import transforms3d as tfs
  ImportError: No module named transforms3d
  [ur3_realsense_handeyecalibration_eye_on_base/easy_handeye_calibration_server-4] process has died [pid 27827, exit code 1, cmd /home/guyue/ur_ws/src/easy_handeye/easy_handeye/scripts/calibrate.py __name:=easy_handeye_calibration_server __log:=/home/guyue/.ros/log/5cfd9712-6ca1-11ec-9d22-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-easy_handeye_calibration_server-4.log].
  log file: /home/guyue/.ros/log/5cfd9712-6ca1-11ec-9d22-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-easy_handeye_calibration_server-4*.log
  arguments:  Namespace(quiet=False)
  unknowns:  []
  [INFO] [1641220272.193446]: Configuring for calibration with namespace: /ur3_realsense_handeyecalibration_eye_on_base/
  [INFO] [1641220272.194252]: Loading parameters for calibration /ur3_realsense_handeyecalibration_eye_on_base/ from the parameters server
  [INFO] [1641220272.523661]: Loading parameters for calibration ur3_realsense_handeyecalibration_eye_on_base/ from the parameters server
  [ INFO] [1641220272.533509978]: Loading robot model 'ur3_robot'...
  [ WARN] [1641220272.579945030]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
  Please remove the parameter '/robot_description_kinematics/manipulator/kinematics_solver_attempts' from your configuration.
  [ INFO] [1641220273.754739224]: Ready to take commands for planning group manipulator.
  [ INFO] [1641220274.520914331]: Loading robot model 'ur3_robot'...
  [ WARN] [1641220274.558909017]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
  Please remove the parameter '/robot_description_kinematics/manipulator/kinematics_solver_attempts' from your configuration.
  [ INFO] [1641220274.701571379]: Starting planning scene monitor
  
  

  解决： 安装transforms3d

  guyue@guyue:~$ pip install transforms3d
  
  Command 'pip' not found, but can be installed with:
  
  sudo apt install python-pip
  
  guyue@guyue:~$ sudo apt install python-pip
  
  guyue@guyue:~$ pip install transforms3d
  
  

• 报错4： 依然报错

  [ WARN] [1641220801.405651818]: normalizeImageIllumination is unimplemented!
  [ INFO] [1641220801.437421516]: Stereo is NOT SUPPORTED
  [ INFO] [1641220801.437463026]: OpenGL device: Mesa DRI Intel(R) UHD Graphics (CML GT2)
  [ INFO] [1641220801.437478491]: OpenGl version: 3.0 (GLSL 1.3).
  Traceback (most recent call last):
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/scripts/calibrate.py", line 5, in <module>
      from easy_handeye.handeye_server import HandeyeServer
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_server.py", line 13, in <module>
      from easy_handeye.handeye_calibration_backend_opencv import HandeyeCalibrationBackendOpenCV
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_calibration_backend_opencv.py", line 10, in <module>
      class HandeyeCalibrationBackendOpenCV(object):
    File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_calibration_backend_opencv.py", line 15, in HandeyeCalibrationBackendOpenCV
      'Tsai-Lenz': cv2.CALIB_HAND_EYE_TSAI,
  AttributeError: 'module' object has no attribute 'CALIB_HAND_EYE_TSAI'
  [ur3_realsense_handeyecalibration_eye_on_base/easy_handeye_calibration_server-4] process has died [pid 29462, exit code 1, cmd /home/guyue/ur_ws/src/easy_handeye/easy_handeye/scripts/calibrate.py __name:=easy_handeye_calibration_server __log:=/home/guyue/.ros/log/e9da5296-6ca2-11ec-9d22-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-easy_handeye_calibration_server-4.log].
  log file: /home/guyue/.ros/log/e9da5296-6ca2-11ec-9d22-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-easy_handeye_calibration_server-4*.log
  arguments:  Namespace(quiet=False)
  unknowns:  []
  [INFO] [1641220802.356186]: Configuring for calibration with namespace: /ur3_realsense_handeyecalibration_eye_on_base/
  [INFO] [1641220802.356995]: Loading parameters for calibration /ur3_realsense_handeyecalibration_eye_on_base/ from the parameters server
  [INFO] [1641220802.691195]: Loading parameters for calibration ur3_realsense_handeyecalibration_eye_on_base/ from the parameters server
  [ INFO] [1641220802.701319432]: Loading robot model 'ur3_robot'...
  [ WARN] [1641220802.748617616]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
  Please remove the parameter '/robot_description_kinematics/manipulator/kinematics_solver_attempts' from your configuration.
  [ INFO] [1641220803.836626902]: Ready to take commands for planning group manipulator.
  [ INFO] [1641220804.638232086]: Loading robot model 'ur3_robot'...
  [ WARN] [1641220804.672640533]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
  Please remove the parameter '/robot_description_kinematics/manipulator/kinematics_solver_attempts' from your configuration.
  
  

  [easy_handeye_calibration_server_robot-3] killing on exit
  PluginHandler.save_settings() plugin "rqt_easy_handeye/Hand-eye Calibration automatic movement#0" raised an exception:
  Traceback (most recent call last):
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 191, in save_settings
      self._save_settings(plugin_settings, instance_settings)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler_direct.py", line 114, in _save_settings
      self.emit_save_settings_completed()
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 207, in emit_save_settings_completed
      callback(self._instance_id)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 459, in _close_application_save_callback
      self._close_application_shutdown_plugins()
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 467, in _close_application_shutdown_plugins
      info['instance_id'], self._close_application_shutdown_callback)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 353, in _shutdown_plugin
      handler.close_signal.disconnect(self.unload_plugin)
  TypeError: disconnect() failed between 'close_signal' and 'unload_plugin'
  
  PluginHandler.save_settings() plugin "rqt_easy_handeye/Hand-eye Calibration#0" raised an exception:
  Traceback (most recent call last):
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 191, in save_settings
      self._save_settings(plugin_settings, instance_settings)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler_direct.py", line 114, in _save_settings
      self.emit_save_settings_completed()
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 207, in emit_save_settings_completed
      callback(self._instance_id)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 459, in _close_application_save_callback
      self._close_application_shutdown_plugins()
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 467, in _close_application_shutdown_plugins
      info['instance_id'], self._close_application_shutdown_callback)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 353, in _shutdown_plugin
      handler.close_signal.disconnect(self.unload_plugin)
  TypeError: disconnect() failed between 'close_signal' and 'unload_plugin'
  
  Traceback (most recent call last):
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 454, in close_application
      global_settings, perspective_settings, self._close_application_save_callback)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 429, in _save_settings
      self._save_plugin_settings(info['instance_id'], callback)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_manager.py", line 341, in _save_plugin_settings
      handler.save_settings(plugin_settings, instance_settings, callback)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 195, in save_settings
      self.emit_save_settings_completed()
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 202, in emit_save_settings_completed
      self._call_method_on_all_dock_widgets('save_settings', self.__instance_settings)
    File "/opt/ros/melodic/lib/python2.7/dist-packages/qt_gui/plugin_handler.py", line 213, in _call_method_on_all_dock_widgets
      settings = instance_settings.get_settings(name)
  AttributeError: 'NoneType' object has no attribute 'get_settings'
  [ur3_realsense_handeyecalibration_eye_on_base/calibration_mover-6] escalating to SIGTERM
  shutting down processing monitor...
  ... shutting down processing monitor complete
  
  

  解决：
  AttributeError: 'module' object has no attribute'CALIB_HAND_EYE_TSAI'
  出现这个问题的原因在于python的opencv版本过低，低版本的opencv中没有手眼标定的函数，因此需要更新opencv版本即可。

  pip2 install opencv-python==4.2.0.32
  

  参考：
  - https://github.com/IFL-CAMP/easy_handeye/issues/78
  - https://blog.csdn.net/m0_53621852/article/details/121021402

4.3 关于camera_marker的报错

[ERROR] [1641266714.990937]: Error processing request: "camera_marker" passed to lookupTransform argument source_frame does not exist. 
['Traceback (most recent call last):\n', '  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n    response = convert_return_to_response(self.handler(request), self.response_class)\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_server.py", line 88, in take_sample\n    self.sampler.take_sample()\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_sampler.py", line 88, in take_sample\n    transforms = self._get_transforms()\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_sampler.py", line 78, in _get_transforms\n    Duration(10))\n', '  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_ros/buffer.py", line 87, in lookup_transform\n    return self.lookup_transform_core(target_frame, source_frame, time)\n', 'LookupException: "camera_marker" passed to lookupTransform argument source_frame does not exist. \n']
Traceback (most recent call last):
  File "/home/guyue/ur_ws/src/easy_handeye/rqt_easy_handeye/src/rqt_easy_handeye/rqt_easy_handeye.py", line 132, in handle_take_sample
    sample_list = self.client.take_sample()
  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_client.py", line 76, in take_sample
    return self.take_sample_proxy().samples
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 442, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 522, in call
    responses = transport.receive_once()
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 735, in receive_once
    p.read_messages(b, msg_queue, sock) 
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 360, in read_messages
    self._read_ok_byte(b, sock)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 343, in _read_ok_byte
    raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str))
rospy.service.ServiceException: service [/ur3_realsense_handeyecalibration_eye_on_base/take_sample] responded with an error: error processing request: "camera_marker" passed to lookupTransform argument source_frame does not exist.
[ur3_realsense_handeyecalibration_eye_on_base/namespace_guyue_11030_7952019406960363886_rqt-5] process has died [pid 11069, exit code -6, cmd /home/guyue/ur_ws/src/easy_handeye/rqt_easy_handeye/scripts/rqt_easy_handeye __name:=namespace_guyue_11030_7952019406960363886_rqt __log:=/home/guyue/.ros/log/4f4a0756-6d0d-11ec-b452-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-namespace_guyue_11030_7952019406960363886_rqt-5.log].
log file: /home/guyue/.ros/log/4f4a0756-6d0d-11ec-b452-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-namespace_guyue_11030_7952019406960363886_rqt-5*.log

• 解决：
  打开rqt，对准二维码，然后让rviz中出现了这个坐标
  注意realsense需要单独启动

4.4 在仿真环境运行的报错

[INFO] [1641298197.817943, 69.092000]: Taking a sample...
[ERROR] [1641298207.884633, 79.146000]: Error processing request: Lookup would require extrapolation into the past.  Requested time 69.094000000 but the earliest data is at time 1641298197.859954119, when looking up transform from frame [camera_marker] to frame [camera_color_frame]
['Traceback (most recent call last):\n', '  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n    response = convert_return_to_response(self.handler(request), self.response_class)\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_server.py", line 88, in take_sample\n    self.sampler.take_sample()\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_sampler.py", line 88, in take_sample\n    transforms = self._get_transforms()\n', '  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_sampler.py", line 78, in _get_transforms\n    Duration(10))\n', '  File "/opt/ros/melodic/lib/python2.7/dist-packages/tf2_ros/buffer.py", line 87, in lookup_transform\n    return self.lookup_transform_core(target_frame, source_frame, time)\n', 'ExtrapolationException: Lookup would require extrapolation into the past.  Requested time 69.094000000 but the earliest data is at time 1641298197.859954119, when looking up transform from frame [camera_marker] to frame [camera_color_frame]\n']
Traceback (most recent call last):
  File "/home/guyue/ur_ws/src/easy_handeye/rqt_easy_handeye/src/rqt_easy_handeye/rqt_easy_handeye.py", line 132, in handle_take_sample
    sample_list = self.client.take_sample()
  File "/home/guyue/ur_ws/src/easy_handeye/easy_handeye/src/easy_handeye/handeye_client.py", line 76, in take_sample
    return self.take_sample_proxy().samples
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 442, in __call__
    return self.call(*args, **kwds)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 522, in call
    responses = transport.receive_once()
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_base.py", line 735, in receive_once
    p.read_messages(b, msg_queue, sock) 
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 360, in read_messages
    self._read_ok_byte(b, sock)
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 343, in _read_ok_byte
    raise ServiceException("service [%s] responded with an error: %s"%(self.resolved_name, str))
rospy.service.ServiceException: service [/ur3_realsense_handeyecalibration_eye_on_base/take_sample] responded with an error: error processing request: Lookup would require extrapolation into the past.  Requested time 69.094000000 but the earliest data is at time 1641298197.859954119, when looking up transform from frame [camera_marker] to frame [camera_color_frame]
[ur3_realsense_handeyecalibration_eye_on_base/namespace_guyue_2228_1559384220386469985_rqt-5] process has died [pid 2270, exit code -6, cmd /home/guyue/ur_ws/src/easy_handeye/rqt_easy_handeye/scripts/rqt_easy_handeye __name:=namespace_guyue_2228_1559384220386469985_rqt __log:=/home/guyue/.ros/log/119cbc42-6d57-11ec-b452-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-namespace_guyue_2228_1559384220386469985_rqt-5.log].
log file: /home/guyue/.ros/log/119cbc42-6d57-11ec-b452-38fc98e4336a/ur3_realsense_handeyecalibration_eye_on_base-namespace_guyue_2228_1559384220386469985_rqt-5*.log
^C[rviz_guyue_2228_7026475105225641117-7] killing on exit
[ur3_realsense_handeyecalibration_eye_on_base/calibration_mover-6] killing on exit
[easy_handeye_calibration_server_robot-3] killing on exit
[ur3_realsense_handeyecalibration_eye_on_base/easy_handeye_calibration_server-4] killing on exit
[aruco_tracker-1] killing on exit
[dummy_handeye-2] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

• 解决：
  因为前面的报错太多，所以就准备先在gazebo中运行没有错误后再连接真实的机械臂，然后前面的错误报完以后出现这个错误，这个错误不必在意，连接真实机械臂后就不报这个错误了。

  使用gazebo调试的方法：
  把连接真实ur机械臂ip地址那句换为启动ur机械臂gazebo的语句；并且启动ur机械臂moveit的语句后面标记sim为true

05 总结

1. 安装功能包

2. 修改lauch文件（放入启动aruco和easy_handeye部分），并放到easy_handeye功能包下面

3. 启动realsense

   roslaunch realsense2_camera rs_camera.launch
   

4. 启动ur机械臂

   # 1. 启动机械臂
   roslaunch ur_robot_driver ur3_bringup.launch limited:=true robot_ip:=192.168.56.10
   # 2. 启动示教器
   # 3. 启动moveit
   roslaunch ur3_moveit_config ur3_moveit_planning_execution.launch limited:=true
   

5. 启动手眼标定程序

   roslaunch easy_handeye ur3_eye_to_hand_calibration.launch
   

6. 启动rqt查看是否能识别到aruco码（点击菜单栏的 Plugins -> Visulization -> Image View，选择 /aruco_tracker/result 话题）

7. 在窗口3检测当前位置是否可行check starting pose，依次点击next pose -> plan -> execute（注意plan完是绿色才可以execute）
   

8. 每次执行完机械臂动作，在窗口2点击take sample，共17次，然后点击compute计算，结果显示在右下方

9. 注： 如果手眼标定launch文件启动有问题，可能是opencv版本不对：

   pip2 install opencv-python==4.2.0.32