one/few-shot segmentation 笔记

Task

对某一类别c1，选择已有标注的样本作为support，待测样本为 query，model 以 support 为参考去分割 query。
C-way K-shot：C为类别数；分割每个 query 使用 K 个 support images

Solution

提取support 和 query 的 feature，将两者的 feature 进行比对。训练时借助其他类别的标注数据去训练model。

数据集

pascal-5i

每个sub-dataset中，trainset来自pascal trainset，testset来自pascal valset；trainset 和 testset 的类别无交集

 

One-Shot Learning for Semantic Segmentation   (2017.09)

dataset： pascal-5i
train时，support 和 query 均从Dtrain中抽样，class相同
test时，sample N = 1000 examples 
Metric：mIoU

结构：conditioning branch 输出参数 θ 用于segmentation branch 输出feature 的分类
for k-shot：对每个support单独得到的mask做logical OR

 

SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation （2018.11）

query image 送入 Guidance Branch 和 Segmentation Branch，通过concate 来使 Guidance Branch 输出指导Segmentation Branch。
CosineSimilarty map ：as guidance to teach the segmentation branch to discover the desired object regions.

for k-shot：
1、  Y is the predicted semantic label
2、use the averaged vector to guide the segmentation process

 

CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning （2019.03）

Dense Comparison Module：

Feature Extractor：ResNet-50，使用中间 block 的 feature（middle-level features that may
constitute object parts shared by unseen classes）
use dilated convolutions in layers after block2
Dense Comparison：concatenated feature maps go through another convolutional block with 256 3 × 3 convolutional filters for comparison.

Iterative Optimization Module
循环迭代，alternatively use predicted masks in the last epoch and empty masks as the input to IOM（dropout）

for k-shot：The result from the attention branch serves as the weight λ, λ 通过 softmax 作为 sum 权重

使用弱标签

 

PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment (2019.08 ICCV)

与检测的 PANet 无关，重名了
support 和 query image 公用 feature extractor，对 support feature 做 mask GAP 得到 prototypes。
query feature 与 prototypes 计算 cosine distance
Prototype alignment regularization (PAR)：将support 和 query 互换，taking the query and the predicted mask as the new support to learn to segment the support images.

for k-shot：所有 support 一起做 Mask GAP