【2017-2018】COCO Challenges分类整理
本文记录了博主在研究COCO（Common Objects in Context） Challenges过程中关于挑战赛分类的笔记。更新于2018.12.21。
COCO官方网址
2017挑战赛官网
2018挑战赛官网
官网中包括所有talk的ppt。
【2017-2018】COCO Challenges分类整理
- 2018挑战赛
- COCO检测挑战
- COCO全景分割挑战
- COCO关键点检测
- COCO稠密姿态检测
- Mapillary Challenges
- Mapillary Vistas 目标识别任务
- Mapillary Vistas全景分割任务
- 2017挑战赛
- COCO检测挑战
- COCO关键点挑战
- COCO实物挑战
- 场景挑战
- 其他相关挑战赛
- VQA挑战赛
2018挑战赛
挑战赛日期：
Workshop时间： 2018年9月9
COCO 2018挑战赛包括：
相比较2017挑战赛，2018挑战赛在内容上做了改动。具体如下：
- 以segmentation masks的目标识别（实例分割）；
- 全景分割；
- 人体关键点估计；
- 稠密姿态。
The specific tracks in the COCO 2018 Challenges are (1) object detection with segmentation masks (instance segmentation), (2) panoptic segmentation, (3) person keypoint estimation, and (4) DensePose. We describe each next. Note: neither object detection with bounding-box outputs nor stuff segmentation will be featured at the COCO 2018 challenge (but evaluation servers for both tasks remain open).
Speaker：
下面是每个类别的详细介绍。
COCO检测挑战
The COCO Object Detection Task is designed to push the state of the art in object detection forward. Note: only the detection task with object segmentation output (that is, instance segmentation) will be featured at the COCO 2018 challenge. For full details of this task please see the COCO Object Detection Task.
与2017年不同，2018年的挑战赛仅接收以目标分割（实例分割）为输出的算法。
COCO2018检测挑战网址
COCO针对两类目标识别任务：
- bounding box output
- object segmentation output（instance segmentation）
但COCO2018仅接收分割类输出的算法。
算法评价指标：
具体指标等参看detection evaluation页面。
COCO的训练、验证和测试集包含超过200,000张图片和80个目标类别，可以从下载界面下载。所有的实例都被详细标注了segmentation mask，但只公开训练集和验证集（超500,000个已分割目标实例）的标注。2018年所使用的数据、度量和指导都与2017年目标识别任务的相同。二者唯一的区别是，2018年只接收分割类结果。
COCO测试集分为两个部分：test-dev和test-challenge。其中在挑战结束后，test-dev作为默认测试集用于维护公开的leaderboard，而test-challenge只用于挑战赛，结果在workshop上公布。
COCO2018检测挑战网址上还说明了提交数据的格式、上传和评估要求。
COCO提供用于数据、标注和评估代码的API，可以从他们的GitHub上下载。API的使用说明在下载界面可以找到。COCO还提供了所有步骤的说明，包括下载、数据类型、结果格式、指导、上传和评估。
COCO全景分割挑战
COCO把检测任务分为两类：thing（人、车、象）和stuff（玻璃、墙、天空）。新出的全景任务（panoptic task）同时包含上面两类。
The COCO Panoptic Segmentation Task has the goal of advancing the state of the art in scene segmentation. Panoptic segmentation addresses both stuff and thing classes, unifying the typically distinct semantic and instance segmentation tasks. For full details of this task please see the COCO Panoptic Segmentation Task.
全景挑战数据库和该数据库对应的论文。数据库中包括80个来自于检测任务的thing类别和91个来自于stuff任务的stuff类别。关于全景分割的更多内容可以看这篇论文。
COCO2018全景分割挑战网址
具体而言，things类别中是包括可数的目标，如人、动物、工具等；而stuff类别中是具有相似材质或纹理的无固定形态的一个区域，比如玻璃、天空、路等等。
The definition of ‘panoptic’ is “including everything visible in one view”, in our context panoptic refers to a unified, global view of segmentation.
致力于的应用场景：autonomous driving、augmented reality等。
评估标准：
具体评估内容可以参见全景分割评估网址。其中包括了类别说明、质量检测公式、度量和评价代码等。
目前，COCO API不支持全景分割下的评估，但是可以从这里找到数据、评估代码和评估服务，leaderboard也可以找到。
COCO关键点检测
The COCO Keypoint Detection Task requires localization of person keypoints in challenging, uncontrolled conditions. The keypoint task involves simultaneously detecting people and localizing their keypoints (person locations are not given at test time). For full details of this task please see the COCO Keypoint Detection Task.
COCO2018关键点检测网址
COCO关键点检测要求算法能够在有挑战、不受控的条件下检测出人的关键点。关键点检测任务包括人物检测和其关键点定位（测试时不提供人的位置信息）两项工作。
评估标准：
具体见全景风格评估网址。
数据集包括超200,000张图片和250,000个标注有关键点信息的人物实例（COCO中大部分人物都是中等或大尺寸），这里是下载地址。其中，训练集和验证集（超150,000个人和170万已标注关键点）的标注是公开的。
COCO稠密姿态检测
The COCO DensePose Task requires localization of dense person keypoints in challenging, uncontrolled conditions. The DensePose task involves simultaneously detecting people and localizing their dense keypoints, mapping all human pixels to a 3D surface of the human body. For full details of this task please see the COCO DensePose Task.
COCO2018稠密姿态检测网址。其中包括数据集的下载地址和DensePose-RCNN系统说明等信息。
Mapillary Challenges
2018年，Mapillary Research携Mapillary Vistas数据库加入了COCO识别任务。
Vistas is a diverse, pixel-accurate street-level image dataset for empowering autonomous mobility and transport at global scale. It has been designed and collected to cover diversity in appearance, richness of annotation detail, and geographic extent. The Mapillary challenges are based on the publicly available Vistas Research dataset, featuring:
- 28 stuff classes, 37 thing classes (w instance-specific annotations), and 1 void class
- 25K high-resolution images (18K train, 2K val, 5K test; w average resolution of ~9 megapixels)
- Global geographic coverage including North and South America, Europe, Africa, Asia, and Oceania
Highly variable weather conditions (sun, rain, snow, fog, haze) and capture times (dawn, daylight, dusk, night)- Broad range of camera sensors, varying focal length, image aspect ratios, and different types of camera noise
- Different capturing viewpoints (road, sidewalks, off-road)
基于Mapillary Vistas数据库将会被分别归入目标识别和全景分割这两类任务中：
Challenge tracks based on the Mapillary Vistas dataset will be (1) object detection with segmentation masks (instance segmentation) and (2) panoptic segmentation, in line with COCO’s detection and panoptic segmentation tasks, respectively.
leaderboard
Mapillary Vistas 目标识别任务
The Mapillary Vistas Object Detection Task emphasizes recognizing individual instances of both static street-image objects (like street lights, signs, poles) but also dynamic street participants (like cars, pedestrians, cyclists). This task aims to push the state-of-the-art in instance segmentation, targeting critical perception tasks for autonomously acting agents like cars or transportation robots. For full details of this task please see the Mapillary Vistas Object Detection Task.
Mapillary Vistas目标识别任务网址
Mapillary Vistas全景分割任务
The Mapillary Vistas Panoptic Segmentation Task targets the full perception stack for scene segmentation in street-images. Panoptic segmentation addresses both stuff and thing classes, unifying the typically distinct semantic and instance segmentation tasks. For full details of this task please see the Mapillary Vistas Panoptic Segmentation Task.
Mapillary Vistas全景分割任务网址
2017挑战赛
举行时间：
Workshop时间： 2017年10月29
COCO 2017挑战赛包括：
- 以bounding boxes和segmentation masks实现的目标检测；
- 关节检测和人类关键点估计；
- 实物分割。
The specific tracks in the COCO 2017 Challenges are (1) object detection with bounding boxes and segmentation masks, (2) joint detection and person keypoint estimation, and (3) stuff segmentation. We describe each next.
Speakers：
下面是每个类别的详细介绍。
COCO检测挑战
The COCO 2017 Detection Challenge is designed to push the state of the art in object detection forward. Teams are encouraged to compete in either (or both) of two object detection challenges: using bounding box output or object segmentation output. For full details of this task please see the COCO Detection Challenge page.
COCO2017检测挑战网址
COCO关键点挑战
The COCO 2017 Keypoint Challenge requires localization of person keypoints in challenging, uncontrolled conditions. The keypoint challenge involves simultaneously detecting people and localizing their keypoints (person locations are not given at test time). For full details of this task please see the COCO Keypoints Challenge page.
COCO2017关键点挑战网址
注意，对于这一类挑战，人的位置在测试时是不提供的，也就是说算法自身要具备估计目标位置的能力。
COCO实物挑战
The COCO 2017 Stuff Segmentation Challenge is designed to push the state of the art in semantic segmentation of stuff classes. Whereas the COCO 2017 Detection Challenge addresses thing classes (person, car, elephant), this challenge focuses on stuff classes (grass, wall, sky). For full details of this task please see the COCO Stuff Challenge page.
COCO2017实物挑战网址
检测挑战主要针对的是thing（如人、车、大象等），而实物挑战针对的是stuff（如杯子、墙、天空等）。
场景挑战
The Places Challenge will host three tracks meant to complement the COCO Challenges. The data for the 2017 Places Challenge is from the pixel-wise annotated image dataset ADE20K, in which there are 20K images for training, 2K validation images, and 3K testing images. The three specific tracks in the Places Challenge 2017 are: (1) scene parsing, (2) instance segmentation, and (3) semantic boundary detection. See the Places Challenge Page for detailed information.
COCO2017场景挑战网址
COCO2017场景挑战数据库
其他相关挑战赛
VQA挑战赛
官方网址
更多内容，欢迎加入星球讨论。