[图像分割] PointRend: Image Segmentation as Rendering

转载请注明作者和出处： http://blog.csdn.net/john_bh/

论文链接：PointRend: Image Segmentation as Rendering
作者及团队：Facebook 人工智能实验室(Alexander Kirillov/吴育昕/何恺明/RossGirshick)
会议及时间： CVPR 2020 Oral
code：https://github.com/facebookresearch/detectron2/tree/master/projects/PointRend

Abstract

We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over- and undersampling challenges faced in pixel labeling tasks, we develop a unique perspective of image segmentation as a rendering problem. From this vantage,we present the PointRend (Point-based Rendering) neural network module: a module that performs point-based segmentation predictions at adaptively selected locations based on an iterative subdivision algorithm. PointRend can be flexibly applied to both instance and semantic segmentation tasks by building on top of existing state-ofthe-art models. While many concrete implementations of the general idea are possible, we show that a simple design already achieves excellent results. Qualitatively,PointRend outputs crisp object boundaries in regions that are oversmoothed by previous methods. Quantitatively,PointRend yields significant gains on COCO and Cityscapes, for both instance and semantic segmentation. PointRend’s efficiency enables output resolutions that are otherwise impractical in terms of memory or computation compared to existing approaches. Code has been made available at https://github.com/facebookresearch/detectron2/tree/master/projects/PointRend

我们提出了一种新的方法来对物体和场景进行有效的高质量图像分割。通过将经典的计算机图形方法模拟为有效渲染，以解决像素标记任务面临的过采样和欠采样问题，我们开发了一种独特的视角，将图像分割视为渲染问题。从这个角度出发，我们提出了PointRend（基于点的渲染）神经网络模块：该模块基于迭代细分算法在自适应选择的位置执行基于点的分段预测。通过在现有最新模型的基础上构建，PointRend可以灵活地应用于实例和语义分割任务。尽管可以实现该总体思想的许多具体实现，但我们表明，简单的设计已经可以实现出色的结果。定性地，PointRend在先前方法过度平滑的区域中输出清晰的对象边界。从数量上来说，PointRend在实例和语义分割方面都大大提高了COCO和Cityscapes的收益。 PointRend的效率可以实现与现有方法相比在内存或计算方面不可行的输出分辨率。代码已在https://github.com/facebookresearch/detectron2/tree/master/projects/PointRend