map评价吗 voc数据集可以用coco_[数据集][COCO]目标检测任务评估
Image Localization Dataset是一个目标检测数据集，其标注文件使用了PASCAL VOC定义的格式，将其转换成COCO格式
Image Localization Dataset
其标注文件和图像文件放置在同一目录下，部分文件结构如下：1
2
3
4
5
6├── cucumber_10.jpg
├── cucumber_10.xml
├── cucumber_11.jpg
├── cucumber_11.xml
。。。
。。。
共包含了3个类别：cucumber、eggplant和mushroom
COCO格式
输入数据json文件包含了3个节点1
2
3
4
5
6{
"images": [image],
"annotations": [annotation],
"categories": [category],
}
其中image节点的数据结构如下：1
2
3
4
5
6{
"id": int,
"width": int,
"height": int,
"file_name": str,
}
annotation节点的数据结构如下：1
2
3
4
5
6
7
8
9{
"id": int,
"image_id": int,
"category_id": int,
"area": float,
"bbox": [x,y,width,height],
"iscrowd": 0,
"segmentation": []
}
category节点的数据结构如下：1
2
3
4
5{
"id": int,
"name": str,
"supercategory": "",
}]
实现
实现步骤：将标注文件转换成COCO格式的json文件 - gt.json
将训练结果转换成COCO格式的json文件 - dt.json
载入gt.json和dt.json，计算评价指标1
2
3
4
5
6
7
8# initialize CocoEval object
E = COCOeval(cocoGt, cocoDt, iouType='bbox')
# run per image evaluation
E.evaluate()
# accumulate per image results
E.accumulate()
# display summary metrics of results
E.summarize()
参数解析
COCO提供了一个参数类Param来调整计算1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28class Params:
'''
Params for coco evaluation api
'''
def setDetParams(self):
self.imgIds = []
self.catIds = []
# np.arange causes trouble. the data point on arange is slightly larger than the true value
self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
self.recThrs = np.linspace(.0, 1.00, int(np.round((1.00 - .0) / .01)) + 1, endpoint=True)
self.maxDets = [1, 10, 100]
self.areaRng = [[0 ** 2, 1e5 ** 2], [0 ** 2, 32 ** 2], [32 ** 2, 96 ** 2], [96 ** 2, 1e5 ** 2]]
self.areaRngLbl = ['all', 'small', 'medium', 'large']
self.useCats = 1
。。。
。。。
def __init__(self, iouType='segm'):
if iouType == 'segm' or iouType == 'bbox':
self.setDetParams()
elif iouType == 'keypoints':
self.setKpParams()
else:
raise Exception('iouType not supported')
self.iouType = iouType
# useSegm is deprecated
self.useSegm = None
IoU设置self.iouThrs = np.linspace(.5, 0.95, int(np.round((0.95 - .5) / .05)) + 1, endpoint=True)
输出：array([0.5 , 0.55, 0.6 , 0.65, 0.7 , 0.75, 0.8 , 0.85, 0.9 , 0.95])
可以输入固定的IoU进行计算1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23# initialize CocoEval object
E = COCOeval(cocoGt, cocoDt, iouType='bbox')
# set parameters as desired
E.params.iouThrs = [0.5]
# run per image evaluation
E.evaluate()
# accumulate per image results
E.accumulate()
# display summary metrics of results
E.summarize()
###################### 输出
Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.976
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=100 ] = 0.751
Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=100 ] = 0.984
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets= 1 ] = 0.973
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets= 10 ] = 0.989
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.989
Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=100 ] = 0.917
Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=100 ] = 0.994
从运行结果可以发现AP/AR大幅提高了
maxDets设置self.maxDets = [1, 10, 100]
使用3个不同的maxDets参数，可以调整最大检测数目1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24# initialize CocoEval object
E = COCOeval(cocoGt, cocoDt, iouType='bbox')
# set parameters as desired
E.params.maxDets = [10, 100, 300]
E.params.iouThrs = [0.5]
# run per image evaluation
E.evaluate()
# accumulate per image results
E.accumulate()
# display summary metrics of results
E.summarize()
############################# 输出
Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.976
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=300 ] = 0.751
Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=300 ] = 0.984
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets= 10 ] = 0.989
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.989
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=300 ] = 0.989
Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=300 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=300 ] = 0.917
Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=300 ] = 0.994
如何计算某一类别的AP/ARself.catIds = []
可以指定哪个类别的AP/AR计算，实现如下：1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25# initialize CocoEval object
E = COCOeval(cocoGt, cocoDt, iouType='bbox')
# set parameters as desired
E.params.catIds = [1]
E.params.maxDets = [10, 100, 300]
E.params.iouThrs = [0.5]
# run per image evaluation
E.evaluate()
# accumulate per image results
E.accumulate()
# display summary metrics of results
E.summarize()
################################ 输出
Average Precision (AP) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 0.990
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area= small | maxDets=300 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area=medium | maxDets=300 ] = 1.000
Average Precision (AP) @[ IoU=0.50:0.50 | area= large | maxDets=300 ] = 0.994
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets= 10 ] = 1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=100 ] = 1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area= all | maxDets=300 ] = 1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area= small | maxDets=300 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area=medium | maxDets=300 ] = 1.000
Average Recall (AR) @[ IoU=0.50:0.50 | area= large | maxDets=300 ] = 1.000