【CVPR 2021】 Lifelong Person Re-Identification via Adaptive Knowledge Accumulation
方法概述
1，文章提出了一种终身学习的person re-ID方法，该方法可以持续不断地跨多域学习。
2， 文章提出了用于上述终身学习方法的AKA框架，该框架包含一个可学习的知识图用于更新之前的知识， 同时，该框架转移知识来提高看不见领域上的泛化性。
3， 文章为LReID提供了一个基线和评估策略。
文章目录
- 方法概述
- 内容概要
-
- 工作概述
- 成果概述
- 方法详解
-
- 方法框架
- 具体实现
- 实验结果
- 总体评价
- 引用格式
- 参考文献
内容概要
| 论文名称 | 简称 | 会议/期刊 | 出版年份 | baseline | backbone | 数据集 |
|---|---|---|---|---|---|---|
| Lifelong Person Re-Identification via Adaptive Knowledge Accumulation | - | CVPR | 2021 | - | ResNet-50 | Market、SYSU、Duke、MSMT17、CUHK03 |
在线链接：https://openaccess.thecvf.com/content/CVPR2021/html/Pu_Lifelong_Person_Re-Identification_via_Adaptive_Knowledge_Accumulation_CVPR_2021_paper.html
源码链接： https://github.com/TPCD/LifelongReID
工作概述
1，lifelong person re-identification (LReID), which enables to learn continuously across multiple domains and even generalise on new and unseen domains.
2，Following the cognitive pro- cesses in the human brain, we design an Adaptive Knowl- edge Accumulation (AKA) framework that is endowed with two crucial abilities: knowledge representation and knowl- edge operation.
成果概述
1，Our method alleviates catastrophic for- getting on seen domains and demonstrates the ability to generalize to unseen domains.
2，we also provide a new and large-scale benchmark for LReID. Ex- tensive experiments demonstrate our method outperforms other competitors by a margin of 5.8% mAP in general- ising evaluation.
方法详解
方法框架
Figure 1: Pipeline of the proposed lifelong person re- identification task. The person identities among the in- volved domains are completely disjoint.
具体实现
1，文章采取了一种不断加入新的域来进行学习的 终生学习方法，并假设中的域的个数是 T。
2， 每个域对应的数据集都包含了训练集和测试集两个部分。且都有标注。
3，baseline的基本损失是 交叉熵损失，如公式1所示。 考虑到前后域的知识保留，文章还设计了一个知识蒸馏损失，如公式2所示。 总的损失函数是二者加权，如公式3所示。
4， 知识累积的过程，包含了 知识表达和知识操作两个部分。如图2所示。其中知识表达 由两个图组成，一个是基于实体的相似性图（ISG），另一个是累积知识图（AKG）。
5， ISG 由一个mini-batch的样本特征的全连接图构成，其边的权重计算如公式4所示。 AKG 的构成和ISG相似，但是其数据来源以及如何更新文章中没有交代很清楚，其边权重计算如公式5所示。
6，知识操作分为两个步骤：知识转移和知识累积。 在知识转移中，将上述两图全连接起来，其连街边的权重的计算如公式6所示。 最后组成的新的图称为联合图，其计算公式为公式7。
7，通过图卷积网络（GCN）进行相关知识传播（公式8）
8，知识的累积下降 原始的特征和上步骤求得的传播后的知识进行平均，得到表达F，然后引入一个可塑性目标（plasticity objective），如公式9所示。
9，为了避免遗忘的问题，文章又提出了一个稳定性损失，如公式10所示。 公式9和10被用来优化AKG 的参数。
10，总的损失函数，公式11.
11，提出新的基线，LReID-Seen 和 LReID-Unseen。
实验结果
总体评价
1，可想而知，是个体量非常大的训练过程。属于有监督行人重识别。
2，知识累积过程中的两个图都比较容易想到，全连接有点暴力，都不用考虑计算复杂性和时间开销的吗
3，文章的部分策略受到了人类学习过程的启发，这一点和hpla类似。
4，需要好好学习一下人家公式的表达。
小样本学习与智能前沿（下方↓公众号）后台回复“LLAKA"，即可获得论文电子资源。
引用格式
@inproceedings{DBLP:conf/cvpr/Pu0LBL21,
author = {Nan Pu and
Wei Chen and
Yu Liu and
Erwin M. Bakker and
Michael S. Lew},
title = {Lifelong Person Re-Identification via Adaptive Knowledge Accumulation},
booktitle = {{CVPR}},
pages = {7901–7910},
publisher = {Computer Vision Foundation / {IEEE}},
year = {2021}
}
参考文献
[1] Abien Fred Agarap. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375, 2018. 4
[2] Francisco M Castro, Manuel J Mar´ın-Jim´enez, Nicol´as Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incre- mental learning. In ECCV, pages 233–248, 2018. 2
[3] Wei Chen, Yu Liu, Weiping Wang, Tinne Tuytelaars, Er- win M Bakker, and Michael Lew. On the exploration of incremental learning for fine-grained image retrieval. In BMVC, 2020. 2
[4] Rosemary A Cowell, Morgan D Barense, and Patricnow S Sadil. A roadmap for understanding memory: Decompos- ing cognitive processes into operations and representations. Eneuro, 6(4), 2019. 1, 3, 5
[5] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. arXiv preprint arXiv:1704.01212, 2017. 4
[6] Douglas Gray and Hai Tao. Viewpoint invariant pedestrian recognition with an ensemble of localized features. In ECCV, pages 262–275, 2008. 6
[7] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In de- fense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017. 6
[8] Martin Hirzer, Csaba Beleznai, Peter M Roth, and Horst Bischof. Person re-identification by descriptive and discrim- inative classification. In scandinavian conference on image analysis, pages 91–102. Springer, 2011. 6
[9] Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439, 2020. 5
[10] Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, and Li Zhang. Style normalization and restitution for generalizable person re-identification. In CVPR, pages 3143–3152, 2020. 1
[11] Thomas N Kipf and Max Welling. Semi-supervised classi- fication with graph convolutional networks. In ICLR, 2017. 4
[12] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, et al. Overcoming catastrophic forgetting in neu- ral networks. Proceedings of the national academy of sci- ences, 114(13):3521–3526, 2017. 5
[13] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Uni- versity of Toronto, 2009. 1, 2
[14] Yann LeCun, L´eon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recog- nition. Proceedings of the IEEE, 86(11):2278–2324, 1998. 2
[15] Qingming Leng, Mang Ye, and Qi Tian. A survey of open- world person re-identification. IEEE Trans. Circuit Syst. Video Technol., 30(4):1092–1108, 2019. 1
[16] Wei Li and Xiaogang Wang. Locally aligned feature trans- forms across views. In CVPR, pages 3594–3601, 2013. 6
[17] Wei Li, Rui Zhao, and Xiaogang Wang. Human reidentifica- tion with transferred metric learning. In ACCV, 2012. 6
[18] Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. Deep- reid: Deep filter pairing neural network for person re- identification. In CVPR, pages 152–159, 2014. 6
[19] Wei-Hong Li, Zhuowei Zhong, and Wei-Shi Zheng. One- pass person re-identification by sketch online discriminant analysis. Pattern Recognition, 93:237–250, 2019. 2
[20] Yujia Li, Chenjie Gu, Thomas Dullien, Oriol Vinyals, and Pushmeet Kohli. Graph matching networks for learning the similarity of graph structured objects. In ICML, pages 3835– 3845. PMLR, 2019. 4
[21] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell., 40(12):2935–2947, 2017. 2, 3, 6
[22] Yutian Lin, Xuanyi Dong, Liang Zheng, Yan Yan, and Yi Yang. A bottom-up clustering approach to unsupervised per- son re-identification. In AAAI, volume 33, pages 8738–8745, 2019. 2
[23] Jialun Liu, Yifan Sun, Chuchu Han, Zhaopeng Dou, and Wenhui Li. Deep representation learning on long-tailed data: A learnable embedding augmentation perspective. In CVPR, pages 2970–2979, 2020. 2
[24] Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. arXiv preprint arXiv:1705.03550, 2017. 2
[25] Chen Change Loy, Tao Xiang, and Shaogang Gong. Time- delayed correlation analysis for multi-camera activity under- standing. Int. J. Comput. Vis., 90(1):106–129, 2010. 6
[26] Chuanchen Luo, Yuntao Chen, Naiyan Wang, and Zhaoxi- ang Zhang. Spectral feature transformation for person re- identification. In ICCV, pages 4976–4985, 2019. 3
[27] Michael McCloskey and Neal J Cohen. Catastrophic inter- ference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, vol- ume 24, pages 109–165. Elsevier, 1989. 1
[28] German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and StefanWermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. 2
[29] Anastasia Pentina and Christoph Lampert. A pac-bayesian bound for lifelong learning. In ICML, pages 991–999, 2014. 2
[30] Angelo Porrello, Luca Bergamini, and Simone Calderara. Robust re-identification by multiple views knowledge distil- lation. In ECCV, pages 93–110, 2020. 1, 2
[31] Nan Pu, Wei Chen, Yu Liu, Erwin M Bakker, and Michael S Lew. Dual gaussian-based variational subspace disentangle- ment for visible-infrared person re-identification. In ACM MM, pages 2149–2158, 2020. 1, 2
[32] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classi- fier and representation learning. In CVPR, pages 2001–2010, 2017. 1, 2
[33] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large
scale visual recognition challenge. Int. J. Comput. Vis., 115(3):211–252, 2015. 1, 2
[34] Konstantin Shmelkov, Cordelia Schmid, and Karteek Ala- hari. Incremental learning of object detectors without catas- trophic forgetting. In ICCV, pages 3400–3409, 2017. 2
[35] Jifei Song, Yongxin Yang, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. Generalizable person re- identification by domain-invariant mapping network. In CVPR, pages 719–728, 2019. 1, 2
[36] Frederick Tung and Greg Mori. Similarity-preserving knowl- edge distillation. In ICCV, pages 1365–1374, 2019. 6
[37] C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Re- port CNS-TR-2011-001, California Institute of Technology, 2011. 2
[38] DongkaiWang and Shiliang Zhang. Unsupervised person re- identification via multi-label classification. In CVPR, pages 10981–10990, 2020. 1, 2
[39] Wei-Chun Wang, Nadia M Brashier, Erik A Wing, Eliza- beth J Marsh, and Roberto Cabeza. Knowledge supports memory retrieval through familiarity, not recollection. Neu- ropsychologia, 113:14–21, 2018. 1, 5
[40] Longhui Wei, Shiliang Zhang, Wen Gao, and Qi Tian. Person transfer gan to bridge domain gap for person re- identification. In CVPR, pages 79–88, 2018. 1, 2, 6
[41] Zheng Wei-Shi, Gong Shaogang, and Xiang Tao. Associat- ing groups of people. In BMVC, pages 23–1, 2009. 6
[42] ChenshenWu, Luis Herranz, Xialei Liu, Joost van deWeijer, Bogdan Raducanu, et al. Memory replay gans: Learning to generate new categories without forgetting. In NeurIPS, pages 5962–5972, 2018. 2
[43] Tong Xiao, Shuang Li, Bochao Wang, Liang Lin, and Xi- aogang Wang. End-to-end deep learning for person search. arXiv preprint arXiv:1604.01850, 2(2), 2016. 6
[44] Mang Ye, Jianbing Shen, Xu Zhang, Pong C Yuen, and Shih- Fu Chang. Augmentation invariant and instance spreading feature for softmax embedding. IEEE Trans. Pattern Anal. Mach. Intell., 2020. 4
[45] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable net- works. arXiv preprint arXiv:1708.01547, 2017. 2
[46] Hong-Xing Yu and Wei-Shi Zheng. Weakly supervised dis- criminative feature learning with state information for person identification. In CVPR, pages 5527–5537, 2020. 1
[47] Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Xin Jin, and Zhibo Chen. Relation-aware global attention for person re- identification. In CVPR, pages 3186–3195, 2020. 1, 2
[48] Bo Zhao, Shixiang Tang, Dapeng Chen, Hakan Bilen, and Rui Zhao. Continual representation learning for biometric identification. arXiv preprint arXiv:2006.04455, 2020. 2, 5, 6, 7
[49] Fang Zhao, Shengcai Liao, Guo-Sen Xie, Jian Zhao, Kai- hao Zhang, and Ling Shao. Unsupervised domain adap- tation with noise resistible mutual-training for person re- identification. In ECCV, pages 526–544, 2020. 1, 2
[50] Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang. Spindle
net:net: Person re-identification with human body region guided feature decomposition and fusion. In CVPR, pages 1077– 1085, 2017. 6
[51] Feng Zheng, Cheng Deng, Xing Sun, Xinyang Jiang, Xi- aowei Guo, Zongqiao Yu, Feiyue Huang, and Rongrong Ji. Pyramidal person re-identification via multi-loss dynamic training. In CVPR, pages 8514–8522, 2019. 7
[52] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jing- dong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In ICCV, pages 1116–1124, 2015. 1, 6
[53] Liang Zheng, Yi Yang, and Alexander G Hauptmann. Per- son re-identification: Past, present and future. arXiv preprint arXiv:1610.02984, 2016. 1, 2
[54] Zhedong Zheng, Liang Zheng, and Yi Yang. Unlabeled sam- ples generated by gan improve the person re-identification baseline in vitro. In ICCV, pages 3754–3762, 2017. 1, 2, 6
[55] Yang Zou, Xiaodong Yang, Zhiding Yu, BVK Kumar, and Jan Kautz. Joint disentangling and adaptation for cross- domain person re-identification. In ECCV, pages 87–104, 2020. 1, 2