模型特征蒸馏:Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation
https://github.com/SwinTransformer/Feature-Distillation.在FeatureWhitening中使用不带参数的LayerNorm:nn.LayerNorm(embed_dim,elementwise_affine=False)在Projector中使用1x1卷积进行特征对齐:VIT中使用self.decoder=nn.Sequential(nn.