深度学习经典论文及开源代码(转)

---

目录

综述

Deep Learning

图像分类

LeNet

AlexNet

ZFNet

Inception系列

VGG

ResNet

DenseNet

ResNeXt

SENet

EfficientNet

目标检测

RCNN

OverFeat

SPP

Fast RCNN

Faster RCNN

R-FCN

FPN

SSD系列

YOLO系列

RetinaNet

CornerNet

CenterNet

CVPR2018

图像分割

FCN

Mask RCNN

梯度下降

方法综述

Momentum

Nesterov

Adagrad

Adadelta

Adam

Nesterov+Adam

Adabound

生成对抗网络

GAN

DCGAN

LSGAN

Wasserstein GAN

Age Conditional GAN

Cycle GAN

StarGAN

CartoonGAN

双目匹配

归一化方法

Batch Normalization

Layer Normalization

Instance Normalization

Group Normalization

模型剪枝

---

综述

Deep Learning

论文题目：Deep Learning

论文地址：https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf

DeepLearning 3大佬Yann LeCun, Yoshua Bengio & Geoffrey Hinton在Nature上发表的关于DeepLearning的综述文章。

 

图像分类

LeNet

论文题目：Gradient-Based Learning Applied to Document Recognition

论文地址：http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf

第一个卷积神经网络

开源代码：

1. class Lenet(nn.Module):

2. def __init__(self, bool_bn=False, num_classes=10):

3. super(Lenet, self).__init__()

4. self.features = nn.Sequential(

5. nn.Conv2d(3, 6, kernel_size=5),

6. nn.ReLU(inplace=True),

7. nn.MaxPool2d(kernel_size=2, stride=2),

8. nn.Conv2d(6, 16, kernel_size=5),

9. nn.ReLU(inplace=True),

10. nn.MaxPool2d(kernel_size=2, stride=2),

11. )

12. self.features_bn = nn.Sequential(

13. nn.Conv2d(3, 6, kernel_size=5),

14. nn.BatchNorm2d(6),

15. nn.ReLU(inplace=True),

16. nn.MaxPool2d(kernel_size=2, stride=2),

17. nn.Conv2d(6, 16, kernel_size=5),

18. nn.BatchNorm2d(16),

19. nn.ReLU(inplace=True),

20. nn.MaxPool2d(kernel_size=2, stride=2),

21. )

22. self.classifier = nn.Sequential(

23. nn.Linear(16 * 5 * 5, 120),

24. nn.ReLU(inplace=True),

25. nn.Dropout(),

26. nn.Linear(120, 84),

27. nn.ReLU(inplace=True),

28. nn.Dropout(),

29. nn.Linear(84, num_classes),

30. )

31. self.bool_bn = bool_bn

32.  

33. def forward(self, x):

34. if self.bool_bn:

35. x = self.features_bn(x)

36. else:

37. x = self.features(x)

38. x = x.view(x.size(0), 16 * 5 * 5)

39. x = self.classifier(x)

40. return x

 

AlexNet

论文题目：Imagenet classification with deep convolutional neural networks

论文地址：https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

2012年ILSVRC分类比赛冠军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py

 

ZFNet

论文题目：Visualizing and Understanding Convolutional Networks

论文地址：https://arxiv.org/pdf/1311.2901.pdf

2013年ILSVRC分类比赛冠军

 

Inception系列

论文题目：Going deeper with convolutions

论文地址：https://arxiv.org/pdf/1409.4842v1.pdf

2014年ILSVRC分类比赛冠军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/googlenet.py

https://github.com/pytorch/vision/blob/master/torchvision/models/inception.py

 

论文题目：Rethinking the Inception Architecture for Computer Vision

论文地址：https://arxiv.org/pdf/1512.00567.pdf

Inception v2-v3
 

论文题目：Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

论文地址：https://arxiv.org/pdf/1602.07261.pdf

Inception-v4, Inception-ResNet-v1, Inception-ResNet-v2

 

VGG

论文题目：Very Deep Convolutional Networks for Large-Scale Image Recognition

论文地址：https://arxiv.org/pdf/1409.1556.pdf

2014年ILSVRC分类比赛亚军

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/vgg.py

 

ResNet

论文题目：Deep Residual Learning for Image Recognition

论文地址：https://arxiv.org/pdf/1512.03385.pdf

2015年1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation

 

论文题目：Identity Mappings in Deep Residual Networks

论文地址：https://arxiv.org/pdf/1603.05027.pdf

修改了ResNet中卷积、批量归一化和Relu层的顺序提高分类效果

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py

 

DenseNet

论文题目：Densely Connected Convolutional Networks

论文地址：https://arxiv.org/pdf/1608.06993.pdf

CVPR2017最佳论文

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py

 

ResNeXt

论文题目：Aggregated Residual Transformations for Deep Neural Networks

论文地址：https://arxiv.org/pdf/1611.05431.pdf

ResNet分组卷积，提高检测准确率；Faster-RCNN Backbone首选网络

开源代码：

https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py

 

SENet

论文题目：Squeeze-and-Excitation Networks

论文地址：https://arxiv.org/pdf/1709.01507.pdf

在网络中增加了Squeeze和Excitation分支，提高检测准确率

 

EfficientNet

论文题目：EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

论文地址：https://arxiv.org/pdf/1905.11946.pdf

 

目标检测

RCNN

论文题目：Rich feature hierarchies for accurate object detection and semantic segmentation

论文地址：https://arxiv.org/pdf/1311.2524.pdf

通过Selective Search选择存在目标的区域，通过SVM实现分类，通过Bounding Box Regression实现定位

 

OverFeat

论文题目：OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks

论文地址：https://arxiv.org/pdf/1312.6229.pdf

通过全卷积网络实现目标定位，winner of the ILSVRC13 localization competition

 

SPP

论文题目：Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition

论文地址：https://arxiv.org/pdf/1406.4729.pdf

提出了空间金字塔池化的方法，进行图像分类和目标检测

 

Fast RCNN

论文题目：Fast R-CNN

论文地址：https://arxiv.org/pdf/1504.08083.pdf

提出了ROI Pooling的方法

 

Faster RCNN

论文题目：Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

论文地址：https://arxiv.org/pdf/1506.01497.pdf

使用了Region Proposal Net替代Selective Search的方法，实现端到端检测并提高检测速度。

 

R-FCN

论文题目：R-FCN: Object Detection via Region-based Fully Convolutional Networks

论文地址：https://arxiv.org/pdf/1605.06409.pdf

在ROI中使用k*k的feature map投票预测种类

 

FPN

论文题目：Feature Pyramid Networks for Object Detection

论文地址：https://arxiv.org/pdf/1612.03144.pdf

提出了图像特征金字塔的方法，提高对图像中小目标的检测精度

 

SSD系列

论文题目：SSD: Single Shot MultiBox Detector

论文地址：https://arxiv.org/pdf/1512.02325.pdf

一种One Stage Detector

 

论文题目：DSSD : Deconvolutional Single Shot Detector

论文地址：https://arxiv.org/pdf/1701.06659.pdf

改进了SSD，提高检测精度

 

YOLO系列

论文题目：You Only Look Once: Unified, Real-Time Object Detection

论文地址：https://arxiv.org/pdf/1506.02640.pdf

代码地址：https://github.com/pjreddie/darknet

                  https://github.com/AlexeyAB/darknet

一种One Stage Detector，YOLO v1

论文题目：YOLO9000: Better, Faster, Stronger

论文地址：https://arxiv.org/pdf/1612.08242.pdf

YOLO v2， Backbone为Darknet19

论文题目：YOLOv3: An Incremental Improvement

论文地址：https://pjreddie.com/media/files/papers/YOLOv3.pdf

YOLOv3，Backbone为Darknet53，与v2相比，采用了残差模块和FPN结构

论文题目：Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving

论文地址：https://arxiv.org/pdf/1904.04620v2.pdf

代码地址：https://github.com/jwchoi384/Gaussian_YOLOv3

论文题目：YOLOv4: Optimal Speed and Accuracy of Object Detection

论文地址：https://arxiv.org/pdf/2004.10934.pdf

YOLOv4，CSPDarknet53作为骨干网络，SPP作为附加模块，PANet作为特征融合的Neck，YOLOv3检测器作为Header。详细解析见https://blog.csdn.net/linghu8812/article/details/105729693。

 

 

RetinaNet

论文题目：Focal Loss for Dense Object Detection

论文地址：https://arxiv.org/pdf/1708.02002.pdf

对于训练集中正负样本不平均的情况，采用了focal loss的方法，减小负样本数量过多的影响。

 

CornerNet

论文题目：CornerNet: Detecting Objects as Paired Keypoints

论文地址：https://arxiv.org/pdf/1808.01244.pdf

基于关键点的方法进行目标检测，使用了角池化的方法检测不在检测框的左上角或者右下角的目标。

论文题目：CornerNet-Lite: Efficient Keypoint Based Object Detection

论文地址：https://arxiv.org/pdf/1904.08900.pdf

使用了CornerNet Saccade和CornerNet Squeez对CornerNet检测进行提速。

 

CenterNet

论文题目：CenterNet: Keypoint Triplets for Object Detection

论文地址：https://arxiv.org/pdf/1904.08189.pdf

对CornerNet进行了中心池化和级联角池化的优化，提高了mAP。

论文题目：Objects as Points

论文地址：https://arxiv.org/pdf/1904.07850.pdf

采用了不同的Backbone进行检测比较检测速度和mAP。

 

CVPR2018

论文题目：Cascade R-CNN Delving into High Quality Object Detection
论文地址：https://arxiv.org/pdf/1712.00726.pdf

 

论文题目：Relation Networks for Object Detection
论文地址：https://arxiv.org/pdf/1711.11575.pdf

 

论文题目：Single-Shot Refinement Neural Network for Object Detection

论文地址：https://arxiv.org/pdf/1711.06897.pdf

 

论文题目：An Analysis of Scale Invariance in Object Detection – SNIP
论文地址：https://arxiv.org/pdf/1711.08189.pdf

 

论文题目：R-FCN-3000 at 30fps: Decoupling Detection and Classification
论文地址：https://arxiv.org/pdf/1712.01802.pdf

 

论文题目：Single-Shot Object Detection with Enriched Semantics
论文链接：https://arxiv.org/pdf/1712.00433.pdf

 

论文题目：Scale-Transferrable Object Detection
论文地址：http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/1376.pdf

 

图像分割

FCN

论文题目：Fully Convolutional Networks for Semantic Segmentation

论文地址：https://arxiv.org/pdf/1411.4038.pdf

使用全连接网络进行图像语义分割

 

Mask RCNN

论文题目：Mask R-CNN

论文地址：https://arxiv.org/pdf/1703.06870.pdf

ICCV2017最佳论文，在faster rcnn的基础上增加了mask分支，进行实例分割

 

论文题目：Learning to Segment Every Thing

论文地址：https://arxiv.org/abs/1711.10370

Mask RCNN的升级版，设计了从bbox到mask的weight transfer function

 

梯度下降

方法综述

论文题目：An overview of gradient descent optimization algorithms

论文地址：https://arxiv.org/pdf/1609.04747.pdf

介绍各种梯度下降方法，比较性能。

 

Momentum

论文题目：On the momentum term in gradient descent learning algorithms

论文地址：http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=11C50B032D6D31F8BF95E865E769B4BC?doi=10.1.1.65.3526&rep=rep1&type=pdf

带动量的SGD

 

Nesterov

论文题目：A method for unconstrained convex minimization problem with the rate of convergence  o(1/k2)

论文地址：http://www.cis.pku.edu.cn/faculty/vision/zlin/1983-A%20Method%20of%20Solving%20a%20Convex%20Programming%20Problem%20with%20Convergence%20Rate%20O(k%5E(-2))_Nesterov.pdf

带Nesterov动量的SGD

 

Adagrad

论文题目：Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

论文地址：http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf

Adagrad算法

 

Adadelta

论文题目：ADADELTA: An Adaptive Learning Rate Method

论文地址：https://arxiv.org/pdf/1212.5701.pdf

Adagrad算法

 

Adam

论文题目：Adam: a Method for Stochastic Optimization

论文地址：https://arxiv.org/pdf/1412.6980.pdf

Adam算法

 

Nesterov+Adam

论文题目：Incorporating Nesterov Momentum into Adam

论文地址：https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ

带Nesterov动量的Adam算法

 

Adabound

论文题目：Adaptive Gradient Methods with Dynamic Bound of Learning Rate

论文地址：https://arxiv.org/pdf/1902.09843.pdf

ICLR2019最新梯度下降论文

 

生成对抗网络

GAN

论文题目：Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1406.2661.pdf

通过对抗训练，生成与训练集类似的数据

 

DCGAN

论文题目：Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1511.06434.pdf

通过卷积和反卷积网络生成数据

 

LSGAN

论文题目：Least Squares Generative Adversarial Networks

论文地址：https://arxiv.org/pdf/1611.04076.pdf

损失函数为二次函数训练网络

 

Wasserstein GAN

论文题目：Wasserstein GAN

论文地址：https://arxiv.org/pdf/1701.07875.pdf

通过修改损失函数，提高生成效果

 

Age Conditional GAN

论文题目：FACE AGING WITH CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS

论文地址：https://arxiv.org/pdf/1702.01983.pdf

生成每个人不同年龄段的照片

 

Cycle GAN

论文题目：Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

论文地址：https://arxiv.org/pdf/1703.10593.pdf

斑马图像和马的图像、苹果图像和橘子图像相互转换

 

StarGAN

论文题目：StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation

论文地址：https://arxiv.org/pdf/1711.09020.pdf

生成明星不同肤色、发型、表情等特征的GAN

 

CartoonGAN

论文题目：CartoonGAN: Generative Adversarial Networks for Photo Cartoonization

论文地址：http://openaccess.thecvf.com/content_cvpr_2018/papers/Chen_CartoonGAN_Generative_Adversarial_CVPR_2018_paper.pdf

照片转卡通风格

 

双目匹配

论文题目：Computing the Stereo Matching Cost with a Convolutional Neural Network

论文地址：https://arxiv.org/pdf/1409.4326.pdf

在9x9的图片patch上判断两点是否匹配

 

论文题目：Learning to Compare Image Patches via Convolutional Neural Networks

论文地址：https://arxiv.org/pdf/1504.03641.pdf

 

论文题目：Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches

论文地址：https://arxiv.org/pdf/1510.05970v2.pdf

 

论文题目：Efficient Deep Learning for Stereo Matching

论文地址：https://www.cs.toronto.edu/~urtasun/publications/luo_etal_cvpr16.pdf

 

论文题目：End-to-End Learning of Geometry and Context for Deep Stereo Regression

论文地址：https://arxiv.org/pdf/1703.04309v1.pdf

 

论文题目：Learning for Disparity Estimation through Feature Constancy

论文地址：https://arxiv.org/pdf/1712.01039.pdf

 

论文题目：Pyramid Stereo Matching Network

论文地址：https://arxiv.org/pdf/1803.08669.pdf

 

归一化方法

       四种归一化方法说明——Group Normalization, Yuxin Wu, Kaiming He, https://arxiv.org/pdf/1803.08494.pdf

Batch Normalization

论文题目：Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

论文地址：https://arxiv.org/pdf/1502.03167.pdf

Batch Normalization批量归一化方法

 

Layer Normalization

论文题目：Layer Normalization

论文地址：https://arxiv.org/pdf/1607.06450.pdf

 

Instance Normalization

论文题目：Instance Normalization: The Missing Ingredient for Fast Stylization

论文地址：https://arxiv.org/pdf/1607.08022.pdf

 

Group Normalization

论文题目：Group Normalization

论文地址：https://arxiv.org/pdf/1803.08494.pdf

组归一化方法

 

模型剪枝

论文题目：Pruning Filters for Efficient ConvNets

论文地址：https://arxiv.org/pdf/1608.08710.pdf

基于卷积层权重的L1正则化大小作为标准进行剪枝

 

论文题目：Learning Efficient Convolutional Networks through Network Slimming

论文地址：https://arxiv.org/pdf/1708.06519.pdf

将Batch Normalization层的gamma系数作为标准进行剪枝。

 

论文题目：Rethinking the Value of Network Pruning

论文地址：https://arxiv.org/pdf/1810.05270.pdf

比较了各种剪枝算法，及剪枝后进行fine tune和train from scratch准确率的比较。