计算YOLOv3在COCO数据集上的mAP值

安装darknet

1. 详细步骤和解释请参考YOLOv3的官网，这里给出相应的命令行和需要特别注意的地方

2. 从github上下载项目源码

   git clone https://github.com/pjreddie/darknet
   cd darknet

3. 对源码进行编译(编译默认运行YOLOv3时是不使用GPU的，我们假设跑YOLOv3时需要使用GPU，所以需要修改编译选项)
   1. 只需要将当前文件夹下（darknet/）的Makefile文件中第一行的GPU=0改为GPU=1，改完后的Makefile：

      GPU=1
      CUDNN=0
      OPENCV=0
      OPENMP=0
      DEBUG=0
      
      ...
      ...
      ...

   2. 编译

      make

4. 将权重文件（yolov3.weights）下载到当前目录下

   wget https://pjreddie.com/media/files/yolov3.weights

5. 测试

   #命令行中的‘-i 0’代表使用第0号显卡，假如不想使用GPU可以用‘-nogpu’替代‘-i 0’
   ./darknet -i 0 detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg 

COCO数据集的下载和配置

1. 通过脚本文件（scripts/get_coco_dataset.sh）获取COCO数据集（数据集下载速度会很慢，建议从Function的数据集下载站下载，然后逐行运行scripts/get_coco_dataset.sh中的每一行命令）

   cp scripts/get_coco_dataset.sh data
   cd data
   bash get_coco_dataset.sh

2. 修改COCO的配置文件（cfg/coco.data）

   classes= 80
   train  = /trainvalno5k.txt # 注意这里的需要填入COCO数据集的文件夹路径
   valid  = /5k.txt # 注意这里的需要填入COCO数据集的文件夹路径
   names = data/coco.names
   backup = backup
   eval = coco #注意这行是为了生成json格式的检测结果，官网里并没有说，如果去掉这一行生成的文件是classes(80)类个txt文件

3. 修改YOLOv3模型的配置文件（cfg/yolov3.cfg）

   [net]
   # Testing
   # batch=1
   # subdivisions=1
   # Training
   batch=64 #注意这里的batch有可能使得显存爆掉,这时需要修改,GeForce RTX 2080Ti需要修改为8
   subdivisions=8
   ....

对COCO验证集进行检测

1. 在darknet目录下运行下面的命令行

   ./darknet -i 0 detector valid cfg/coco.data cfg/yolov3.cfg yolov3.weights # '-i 0'代表使用0号GPU

   运行完上述命令后会生成results/coco_results.json文件，该文件保存了检测结果

计算mAP

方法源于：https://blog.csdn.net/xidaoliang/article/details/88397280

1. 安装pycocotools

   pip install pycocotools

2. 在darknet目录下编写一个python脚本(compute_coco_mAP.py)用于计算mAP（注意根据自己的实际情况修改两个文件路径）

   #-*- coding:utf-8 -*-
   import matplotlib.pyplot as plt 
   from pycocotools.coco import COCO 
   from pycocotools.cocoeval import COCOeval 
   import numpy as np 
   import skimage.io as io 
   import pylab,json 
   pylab.rcParams['figure.figsize'] = (10.0, 8.0) 
   def get_img_id(file_name): 
       ls = [] 
       myset = [] 
       annos = json.load(open(file_name, 'r')) 
       for anno in annos: 
         ls.append(anno['image_id']) 
       myset = {}.fromkeys(ls).keys() 
       return myset 
   if __name__ == '__main__': 
       annType = ['segm', 'bbox', 'keypoints']#set iouType to 'segm', 'bbox' or 'keypoints'
       annType = annType[1] # specify type here
       cocoGt_file = 'data/coco/annotations/instances_val2014.json' #需要根据自己的实际情况配置该路径
       cocoGt = COCO(cocoGt_file)#取得标注集中coco json对象
       cocoDt_file = 'results/coco_results.json' #需要根据自己的实际情况配置该路径
       imgIds = get_img_id(cocoDt_file) 
       print(len(imgIds))
       cocoDt = cocoGt.loadRes(cocoDt_file)#取得结果集中image json对象
       imgIds = sorted(imgIds)#按顺序排列coco标注集image_id
       imgIds = imgIds[0:5000]#标注集中的image数据
       cocoEval = COCOeval(cocoGt, cocoDt, annType) 
       cocoEval.params.imgIds = imgIds#参数设置
       cocoEval.evaluate()#评价
       cocoEval.accumulate()#积累
       cocoEval.summarize()#总结

3. 运行compute_coco_mAP.py脚本

   python compute_coco_mAP.py

4. 有可能在运行compute_coco_mAP.py脚本的过程中会报错，可能的解决方法：

   1. 升级scikit-image

      pip install -U scikit-image

   2. 修改Numpy版本为1.16

      pip install numpy==1.16

结果展示

1. img_size为608*608（img_size的设置可以通过修改cfg/yolov3.cfg文件中的width和height来实现）时：

    Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
    Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.585
    Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.345
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.194
    Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.365
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.439
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.291
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.446
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.470
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.304
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.502
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.593
   

2. img_size为416*416时：

    Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.314
    Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.559
    Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.318
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.142
    Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.341
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.464
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.278
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.419
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.239
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.482
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611
   

3. img_size为320*320时：

    Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286
    Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.521
    Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.284
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.103
    Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.315
    Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.449
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.260
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.389
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.189
    Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.454
    Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.601
   

    

关于结果的解释请参考：https://blog.csdn.net/u014734886/article/details/78831884