【cvpr2018】Solving Arbitrary Jigsaw Puzzles for Unsupervised Representation Learning（自监督）

解决问题：

• Learning visual features from unlabeled image data
• a new approach in which the number of configurations can be arbitrarily large while the number of parameters remains
  unchanged.

 

相关工作：

• 聚类；
• all images can be represented in a low-level space and trained encoders and/or decoders to recover the image and/or representation

 

Problem and Baseline Solution：

• an image is partitioned into a grid (e.g., 3*3) of patches and then disordered, and the task is to recover the original configuration. 
• the network needs to understand what a patch contains as well as how two or more patches are related to each other
• we expect this task to teach a network both intra-patch and inter-patch information

 

Strategy

• 相对于9！的排列方式，作者提出新的算法
• unary term：对每一个patch预测它的位置（1~9），分类问题
• binary term：第一个patch在第二个patch的top, bottom, left, right, top-left, top-right, bottomleft, bottom-right或其他位置（9种可能性）

 

structure

• 输入两个序列
• relative position of two patches （蓝色）
• absolute position of a patch（绿色）

Experiment

• adding binary terms improves puzzle recognition accuracy
• 3*3优于2*2
• puzzle recognition accuracy低但transfer效果好
• transfer recognition performance goes up with the power of network backbones
• mirror augmentation harms puzzle recognition but improves transfer learning
• using smaller numbers of configurations leads to slightly worse performance

Discussion：

• forcing the network to discriminate very similar puzzles (e.g., only a pair of patches are reversed) often leads to accuracy drop because the model can focus too much on local patterns
• it is indeed the larger number of parameters in a plain classifier, rather than solving very similar puzzles, that causes transfer performance drop

 

 

参考：Iterative Reorganization withWeak Spatial Constraints: Solving Arbitrary Jigsaw Puzzles for Unsupervised Representation Learning