AI实战营：MMPose开源算法库

• 目录

  RTMPose关键点检测全流程

  MMPose官方可视化工具visualizer

  代码实现：

  ---

• RTMPose关键点检测全流程

  • 
• MMPose预训练模型预测-命令行
  • 预测单张图

    • # HRNet
      python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\configs\body_2d_keypoint\topdown_heatmap\coco\td-hm_hrnet-w32_8xb64-210e_coco-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --input .\data\test\multi-person.jpeg --output-root .\outputs\B1_HRNet_1 --dev cuda:0 --bbox-thr 0.5 --kpt-thr 0.2 --nms-thr 0.3 --radius 8 --thickness 4 --draw-bbox --draw-heatmap --show-kpt-idx

    • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
      Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth
      06/03 20:47:33 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.  

    • #RTMPose
      python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\projects\rtmpose\rtmpose\body_2d_keypoint\rtmpose-s_8xb256-420e_coco-256x192.py https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_
      420e-256x192-fcb2599b_20230126.pth --input .\data\test\multi-person.jpeg --output-root .\outputs\B1_RTM_1 --d
      evice cuda:0 --bbox-thr 0.5 --kpt-thr 0.5 --nms-thr 0.3 --radius 8 --thickness 4 --draw-bbox --draw-heatmap --show-kpt-idx
      

    • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
      Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
      Downloading: "https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth" to C:\Users\16798/.cache\torch\hub\checkpoints\rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
      100%|██████████████████████████████████████████████████████████████████| 60.8M/60.8M [00:37<00:00, 1.71MB/s]
      06/03 23:04:00 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
      d:\workspace\.git\mmpose\mmpose\models\heads\coord_cls_heads\rtmcc_head.py:217: UserWarning: The predicted si
      mcc values are normalized for visualization. This may cause discrepancy between the keypoint scores and the 1D heatmaps.
        warnings.warn('The predicted simcc values are normalized for '
  • 预测视频：直接将--input 换成视频路径即可

    • #HRNet 
      python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\configs\body_2d_keypoint\topdown_heatmap\coco\td-hm_hrnet-w32_8xb64-210e_coco-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --input .\data\test\mother_wx.mp4 --output-root .\outputs\B1_HRNet_2 --device cuda:0 --bbox-thr 0.5 --kpt-thr 0.2 --nms-thr 0.3 --radius 5 --thickness 2 --draw-bbox --draw-heatmap --show-kpt-idx
      

    • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
      Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth
      06/03 23:11:03 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
       

    • #TRMPose
      python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-s_8xb256-420e_coco-256x192.py https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth --input data/test/mother_wx.mp4 --output-root outputs/B1_RTM_2 --device cuda:0 --bbox-thr 0.5 --kpt-thr 0.5 --nms-thr 0.3 --radius 5 --thickness 2 --draw-bbox --draw-heatmap  --show-kpt-idx
      

    • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
      Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
      06/03 23:14:39 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
      d:\workspace\.git\mmpose\mmpose\models\heads\coord_cls_heads\rtmcc_head.py:217: UserWarning: The predicted si
      mcc values are normalized for visualization. This may cause discrepancy between the keypoint scores and the 1D heatmaps.
        warnings.warn('The predicted simcc values are normalized for '
• MMPose预测训练模型预测-Python API

  • import cv2
    import numpy as np
    from PIL import Image
    import matplotlib.pyplot as plt
    import torch
    import mmcv
    from mmcv import imread
    import mmengine
    from mmengine.registry import init_default_scope
    from mmpose.apis import inference_topdown
    from mmpose.apis import init_model as init_pose_estimator
    from mmpose.evaluation.functional import nms
    from mmpose.registry import VISUALIZERS
    from mmpose.structures import merge_data_samples
    from mmdet.apis import inference_detector, init_detector
    
    # 设备
    device = torch.device('cuda') if torch.cuda.is_available() else torch.device(
        'cpu')
    print('device:', device)
    # 载入待测图片
    img_path = 'data/test/multi-person.jpeg'
    # 打开图片
    Image.open(img_path)
    # 构建目标检测模型
    # Faster RCNN
    detector = init_detector(
        'demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py',
        'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth',
        device=device)
    # RTMPose-Tiny
    detector = init_detector(
        'projects/rtmpose/rtmdet/person/rtmdet_m_640-8xb32_coco-person.py',
        'https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth',
        device=device)
    # 构建人体姿态估计模型
    pose_estimator = init_pose_estimator(
        'configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py',
        'https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth',
        device=device,
        cfg_options={'model': {
            'test_cfg': {
                'output_heatmaps': True
            }
        }})
    # 预测-目标检测
    init_default_scope(detector.cfg.get('default_scope', 'mmdet'))
    #获取目标检测预测结果
    detect_result = inference_detector(detector, img_path)
    detect_result.keys()
    # 预测类别
    print(detect_result.pred_instances.labels)
    # 置信度
    print(detect_result.pred_instances.scores)
    # 框坐标：左上角X坐标、左上角Y坐标、右下角X坐标、右下角Y坐标
    print(detect_result.pred_instances.bboxes)
    # 置信度阈值过滤，获得最终目标检测预测结果
    # 置信度阈值
    CONF_THRES = 0.5
    pred_instance = detect_result.pred_instances.cpu().numpy()
    bboxes = np.concatenate((pred_instance.bboxes, pred_instance.scores[:, None]), axis=1)
    bboxes = bboxes[np.logical_and(pred_instance.labels == 0, pred_instance.scores > CONF_THRES)]
    bboxes = bboxes[nms(bboxes, 0.3)][:, :4]
    print(bboxes)
    # 预测-关键点
    # 获取每个 bbox 的关键点预测结果
    pose_results = inference_topdown(pose_estimator, img_path, bboxes)
    print(len(pose_results))
    # 把多个bbox的pose结果打包到一起
    data_samples = merge_data_samples(pose_results)
    print(data_samples.keys())
    # 预测结果-关键点坐标
    print(data_samples.pred_instances.keypoints.shape)
    # 索引为 0 的人，每个关键点的坐标
    print(data_samples.pred_instances.keypoints[0,:,:])
    # 预测结果-关键点热力图
    # 每一类关键点的预测热力图
    print(data_samples.pred_fields.heatmaps.shape)
    idx_point = 13
    heatmap = data_samples.pred_fields.heatmaps[idx_point,:,:]
    print(heatmap.shape)
    # 索引为 idx 的关键点，在全图上的预测热力图
    plt.imshow(heatmap)
    plt.show()

    

  • MMPose官方可视化工具visualizer

    • # 半径
      pose_estimator.cfg.visualizer.radius = 10
      # 线宽
      pose_estimator.cfg.visualizer.line_width = 8
      visualizer = VISUALIZERS.build(pose_estimator.cfg.visualizer)
      # 元数据
      visualizer.set_dataset_meta(pose_estimator.dataset_meta)
      # 可视化
      img = mmcv.imread(img_path)
      img = mmcv.imconvert(img, 'bgr', 'rgb')
      
      img_output = visualizer.add_datasample(
          'result',
          img,
          data_sample=data_samples,
          draw_gt=False,
          draw_heatmap=True,
          draw_bbox=True,
          show_kpt_idx=True,
          show=False,
          wait_time=0,
          out_file='outputs/B2.jpg')
      print(img_output.shape)
      plt.figure(figsize=(10, 10))
      plt.imshow(img_output)
      plt.show()

• 多人检测问题
  • 

• 代码实现：

  • GitHub - guwuyue/OpenMMLabCamp: OpenMMLabCamp