知识蒸馏经典论文详解

蒸馏论文一（knowledge distillation）：利用soft target

蒸馏论文二（Attention Transfer）：利用注意力图

蒸馏论文三（Similarity-Preserving）：利用特征图内在相似性

蒸馏论文四（Relational Knowledge Distillation）：利用样本间距离/角度差异

蒸馏论文五（Neuron Selectivity Transfer）：利用教师网络中间层神经元的激活分布

蒸馏论文六（Probabilistic Knowledge Transfer）：利用特征向量分布概率信息

蒸馏论文七（Variational Information Distillation）：利用互信息

蒸馏论文八（分类蒸馏论文八篇）

1. Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
   
2. MSD: MULTI-SELF-DISTILLATION LEARNING VIA MULTI-CLASSIFIERS WITHIN DEEP NEURAL NET- WORKS
   
3. FEED: Feature-level Ensemble for Knowledge Distillation
   
4. Adaptive Multi-Teacher Multi-level Knowledge Distillation
   
5. Deep Mutual Learning
   
6. Training convolutional neural networks with cheap convolutions and online distillation
   
7. Cascaded channel pruning using hierarchical self-distillation
   
8. Cascaded channel pruning using hierarchical self-distillation
   

其他：
目标检测论文解读四（检测蒸馏论文三篇）

1. Mimicking Very Efficient Network for Object Detection
   
2. General Instance Distillation for Object Detection （2021.4）
   
3. Multi-Scale Aligned Distillation for Low-Resolution Detection