AI实战营:MMPose开源算法库

  • 目录

    RTMPose关键点检测全流程

    MMPose官方可视化工具visualizer

    代码实现:


  • RTMPose关键点检测全流程

    • AI实战营:MMPose开源算法库_第1张图片
  • MMPose预训练模型预测-命令行
    • 预测单张图
      • # HRNet
        python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\configs\body_2d_keypoint\topdown_heatmap\coco\td-hm_hrnet-w32_8xb64-210e_coco-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --input .\data\test\multi-person.jpeg --output-root .\outputs\B1_HRNet_1 --dev cuda:0 --bbox-thr 0.5 --kpt-thr 0.2 --nms-thr 0.3 --radius 8 --thickness 4 --draw-bbox --draw-heatmap --show-kpt-idx
      • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
        Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth
        06/03 20:47:33 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.  
      • #RTMPose
        python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\projects\rtmpose\rtmpose\body_2d_keypoint\rtmpose-s_8xb256-420e_coco-256x192.py https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_
        420e-256x192-fcb2599b_20230126.pth --input .\data\test\multi-person.jpeg --output-root .\outputs\B1_RTM_1 --d
        evice cuda:0 --bbox-thr 0.5 --kpt-thr 0.5 --nms-thr 0.3 --radius 8 --thickness 4 --draw-bbox --draw-heatmap --show-kpt-idx
        
      • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
        Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
        Downloading: "https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth" to C:\Users\16798/.cache\torch\hub\checkpoints\rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
        100%|██████████████████████████████████████████████████████████████████| 60.8M/60.8M [00:37<00:00, 1.71MB/s]
        06/03 23:04:00 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
        d:\workspace\.git\mmpose\mmpose\models\heads\coord_cls_heads\rtmcc_head.py:217: UserWarning: The predicted si
        mcc values are normalized for visualization. This may cause discrepancy between the keypoint scores and the 1D heatmaps.
          warnings.warn('The predicted simcc values are normalized for '
    • 预测视频:直接将--input 换成视频路径即可
      • #HRNet 
        python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth .\configs\body_2d_keypoint\topdown_heatmap\coco\td-hm_hrnet-w32_8xb64-210e_coco-256x192.py https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth --input .\data\test\mother_wx.mp4 --output-root .\outputs\B1_HRNet_2 --device cuda:0 --bbox-thr 0.5 --kpt-thr 0.2 --nms-thr 0.3 --radius 5 --thickness 2 --draw-bbox --draw-heatmap --show-kpt-idx
        
      • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
        Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth
        06/03 23:11:03 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
         
      • #TRMPose
        python .\demo\topdown_demo_with_mmdet.py .\demo\mmdetection_cfg\faster_rcnn_r50_fpn_coco.py https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth projects/rtmpose/rtmpose/body_2d_keypoint/rtmpose-s_8xb256-420e_coco-256x192.py https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth --input data/test/mother_wx.mp4 --output-root outputs/B1_RTM_2 --device cuda:0 --bbox-thr 0.5 --kpt-thr 0.5 --nms-thr 0.3 --radius 5 --thickness 2 --draw-bbox --draw-heatmap  --show-kpt-idx
        
      • Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
        Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-s_simcc-aic-coco_pt-aic-coco_420e-256x192-fcb2599b_20230126.pth
        06/03 23:14:39 - mmengine - WARNING - `Visualizer` backend is not initialized because save_dir is None.
        d:\workspace\.git\mmpose\mmpose\models\heads\coord_cls_heads\rtmcc_head.py:217: UserWarning: The predicted si
        mcc values are normalized for visualization. This may cause discrepancy between the keypoint scores and the 1D heatmaps.
          warnings.warn('The predicted simcc values are normalized for '
  • MMPose预测训练模型预测-Python API
    • import cv2
      import numpy as np
      from PIL import Image
      import matplotlib.pyplot as plt
      import torch
      import mmcv
      from mmcv import imread
      import mmengine
      from mmengine.registry import init_default_scope
      from mmpose.apis import inference_topdown
      from mmpose.apis import init_model as init_pose_estimator
      from mmpose.evaluation.functional import nms
      from mmpose.registry import VISUALIZERS
      from mmpose.structures import merge_data_samples
      from mmdet.apis import inference_detector, init_detector
      
      # 设备
      device = torch.device('cuda') if torch.cuda.is_available() else torch.device(
          'cpu')
      print('device:', device)
      # 载入待测图片
      img_path = 'data/test/multi-person.jpeg'
      # 打开图片
      Image.open(img_path)
      # 构建目标检测模型
      # Faster RCNN
      detector = init_detector(
          'demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py',
          'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth',
          device=device)
      # RTMPose-Tiny
      detector = init_detector(
          'projects/rtmpose/rtmdet/person/rtmdet_m_640-8xb32_coco-person.py',
          'https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_m_8xb32-100e_coco-obj365-person-235e8209.pth',
          device=device)
      # 构建人体姿态估计模型
      pose_estimator = init_pose_estimator(
          'configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w32_8xb64-210e_coco-256x192.py',
          'https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth',
          device=device,
          cfg_options={'model': {
              'test_cfg': {
                  'output_heatmaps': True
              }
          }})
      # 预测-目标检测
      init_default_scope(detector.cfg.get('default_scope', 'mmdet'))
      #获取目标检测预测结果
      detect_result = inference_detector(detector, img_path)
      detect_result.keys()
      # 预测类别
      print(detect_result.pred_instances.labels)
      # 置信度
      print(detect_result.pred_instances.scores)
      # 框坐标:左上角X坐标、左上角Y坐标、右下角X坐标、右下角Y坐标
      print(detect_result.pred_instances.bboxes)
      # 置信度阈值过滤,获得最终目标检测预测结果
      # 置信度阈值
      CONF_THRES = 0.5
      pred_instance = detect_result.pred_instances.cpu().numpy()
      bboxes = np.concatenate((pred_instance.bboxes, pred_instance.scores[:, None]), axis=1)
      bboxes = bboxes[np.logical_and(pred_instance.labels == 0, pred_instance.scores > CONF_THRES)]
      bboxes = bboxes[nms(bboxes, 0.3)][:, :4]
      print(bboxes)
      # 预测-关键点
      # 获取每个 bbox 的关键点预测结果
      pose_results = inference_topdown(pose_estimator, img_path, bboxes)
      print(len(pose_results))
      # 把多个bbox的pose结果打包到一起
      data_samples = merge_data_samples(pose_results)
      print(data_samples.keys())
      # 预测结果-关键点坐标
      print(data_samples.pred_instances.keypoints.shape)
      # 索引为 0 的人,每个关键点的坐标
      print(data_samples.pred_instances.keypoints[0,:,:])
      # 预测结果-关键点热力图
      # 每一类关键点的预测热力图
      print(data_samples.pred_fields.heatmaps.shape)
      idx_point = 13
      heatmap = data_samples.pred_fields.heatmaps[idx_point,:,:]
      print(heatmap.shape)
      # 索引为 idx 的关键点,在全图上的预测热力图
      plt.imshow(heatmap)
      plt.show()

      AI实战营:MMPose开源算法库_第2张图片

    • MMPose官方可视化工具visualizer

      • # 半径
        pose_estimator.cfg.visualizer.radius = 10
        # 线宽
        pose_estimator.cfg.visualizer.line_width = 8
        visualizer = VISUALIZERS.build(pose_estimator.cfg.visualizer)
        # 元数据
        visualizer.set_dataset_meta(pose_estimator.dataset_meta)
        # 可视化
        img = mmcv.imread(img_path)
        img = mmcv.imconvert(img, 'bgr', 'rgb')
        
        img_output = visualizer.add_datasample(
            'result',
            img,
            data_sample=data_samples,
            draw_gt=False,
            draw_heatmap=True,
            draw_bbox=True,
            show_kpt_idx=True,
            show=False,
            wait_time=0,
            out_file='outputs/B2.jpg')
        print(img_output.shape)
        plt.figure(figsize=(10, 10))
        plt.imshow(img_output)
        plt.show()

  • 多人检测问题

    • AI实战营:MMPose开源算法库_第3张图片
  • 代码实现:

    • GitHub - guwuyue/OpenMMLabCamp: OpenMMLabCamp

你可能感兴趣的:(深度学习,人工智能,深度学习,计算机视觉)