半监督学习/模型蒸馏汇总

半监督学习汇总

自训练：Self-training with Noisy Student improves ImageNet classification
论文：https://arxiv.org/abs/1911.04252
代码：https://github.com/google-research/noisystudent
解读：https://zhuanlan.zhihu.com/p/164597142、 https://blog.csdn.net/qq_39426225/article/details/105571340

SimCLRv1：A Simple Framework for Contrastive Learning of Visual Representations
论文：https://arxiv.org/abs/2002.05709
代码：https://github.com/google-research/simclr、https://github.com/Spijkervet/SimCLR、https://github.com/sthalles/SimCLR
解读：https://blog.csdn.net/u011984148/article/details/106233313/、
https://blog.csdn.net/m0_37929824/article/details/104440694
SimCLRv2：Big Self-Supervised Models are Strong Semi-Supervised Learners
论文：https://arxiv.org/abs/2006.10029
代码：https://github.com/google-research/simclr
解读：https://blog.csdn.net/qq_33204646/article/details/107667230
MoCov1：Momentum Contrast for Unsupervised Visual Representation Learning
论文：https://arxiv.org/pdf/1911.05722.pdf
代码：https://github.com/facebookresearch/moco
解读：https://www.zhihu.com/question/355779873、
https://blog.csdn.net/FatMigo/article/details/103211622
MoCov2：Improved Baselines with Momentum Contrastive Learning
论文：https://arxiv.org/pdf/2003.04297.pdf
代码：https://github.com/facebookresearch/moco
解读：https://zhuanlan.zhihu.com/p/114048224

Meta Pseudo Labels
论文：https://arxiv.org/pdf/2003.10580.pdf
代码：https://github.com/google-research/google-research/tree/master/meta_pseudo_labels
解读：https://blog.csdn.net/BAAIBeijing/article/details/112856194

模型蒸馏汇总

汇总：https://github.com/dkozlov/awesome-knowledge-distillation
综述：https://zhuanlan.zhihu.com/p/93287223?from_voters_page=true

PaddleClas-SSLD
文档：https://paddleclas.readthedocs.io/zh_CN/latest/advanced_tutorials/distillation/index.html
https://github.com/PaddlePaddle/PaddleClas/blob/master/docs/zh_CN/advanced_tutorials/distillation/distillation.md
代码：https://github.com/PaddlePaddle/PaddleClas
方法梳理：https://blog.csdn.net/u012526003/article/details/106160464
MEAL: Multi-Model Ensemble via Adversarial Learning
论文：https://arxiv.org/abs/1812.02425
代码：https://github.com/AaronHeee/MEAL#imagenet-model
解读：https://www.cnblogs.com/White-xzx/p/10313982.html

MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks
论文：https://arxiv.org/abs/2009.08453
代码：https://github.com/szq0214/MEAL-V2
解读：https://cloud.tencent.com/developer/article/1703632