ImageNet
AlexNet
ImageNet Classification with Deep Convolutional Neural Networks
- nips-page: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-
- paper: http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
- slides: http://www.image-net.org/challenges/LSVRC/2012/supervision.pdf
GoogLeNet
Going Deeper with Convolutions
- paper: http://arxiv.org/abs/1409.4842
- code: https://github.com/google/inception
- blog(“Building a deeper understanding of images”):http://googleresearch.blogspot.jp/2014/09/building-deeper-understanding-of-images.html
VGGNet
Very Deep Convolutional Networks for Large-Scale Image Recognition
- homepage: http://www.robots.ox.ac.uk/~vgg/research/very_deep/
- arxiv: http://arxiv.org/abs/1409.1556
- slides: http://llcao.net/cu-deeplearning15/presentation/cc3580_Simonyan.pptx
- slides: http://www.robots.ox.ac.uk/~karen/pdf/ILSVRC_2014.pdf
- slides: http://deeplearning.cs.cmu.edu/slides.2015/25.simonyan.pdf
Inception-v3
Rethinking the Inception Architecture for Computer Vision
- intro: “21.2% top-1 and 5.6% top-5 error for single frame evaluation using a network; 3.5% top-5 error and 17.3% top-1 error With an ensemble of 4 models and multi-crop evaluation.”
- arXiv: http://arxiv.org/abs/1512.00567
- github(“Torch port of Inception V3”): https://github.com/Moodstocks/inception-v3.torch
ResNet
Deep Residual Learning for Image Recognition
- arxiv: http://arxiv.org/abs/1512.03385
- slides: http://research.microsoft.com/en-us/um/people/kahe/ilsvrc15/ilsvrc2015_deep_residual_learning_kaiminghe.pdf
- gitxiv: http://gitxiv.com/posts/LgPRdTY3cwPBiMKbm/deep-residual-learning-for-image-recognition
- github(by KaimingHe): https://github.com/KaimingHe/deep-residual-networks
- github: https://github.com/alrojo/lasagne_residual_network
- github: https://github.com/shuokay/resnet
- github: https://github.com/gcr/torch-residual-networks
- blog(“Highway Networks and Deep Residual Networks”):http://yanran.li/peppypapers/2016/01/10/highway-networks-and-deep-residual-networks.html
- blog(“Interpretating Deep Residual Learning Blocks as Locally Recurrent Connections”):https://matrixmashing.wordpress.com/2016/01/29/interpretating-deep-residual-learning-blocks-as-locally-recurrent-connections/
- github: https://github.com/apark263/cfmz
- github:https://github.com/NervanaSystems/neon/blob/master/examples/cifar10_msra.py
- blog(“Training and investigating Residual Nets”):http://torch.ch/blog/2016/02/04/resnets.html
- github: https://github.com/facebook/fb.resnet.torch
Training and investigating Residual Nets
http://torch.ch/blog/2016/02/04/resnets.html
Inception-V4
Inception-V4, Inception-Resnet And The Impact Of Residual Connections On Learning (Workshop track - ICLR 2016)
- intro: “achieve 3.08% top-5 error on the test set of the ImageNet classification (CLS) challenge”
- arxiv: http://arxiv.org/abs/1602.07261
- paper: http://beta.openreview.net/pdf?id=q7kqBkL33f8LEkD3t7X9
Network In Network
- arxiv: http://arxiv.org/abs/1312.4400
Striving for Simplicity: The All Convolutional Net
- arxiv: http://arxiv.org/abs/1412.6806
Batch-normalized Maxout Network in Network
- arxiv: http://arxiv.org/abs/1511.02583
Tensor
Tensorizing Neural Networks
- paper: http://arxiv.org/abs/1509.06569v1
- github(TensorNet): https://github.com/Bihaqo/TensorNet
On the Expressive Power of Deep Learning: A Tensor Analysis
- paper: http://arxiv.org/abs/1509.05009
Convolutional neural networks with low-rank regularization
- arxiv: http://arxiv.org/abs/1511.06067
- github: https://github.com/chengtaipu/lowrankcnn
Deep Learning And Bayesian
Scalable Bayesian Optimization Using Deep Neural Networks (ICML 2015)
- paper: http://jmlr.org/proceedings/papers/v37/snoek15.html
- arxiv: http://arxiv.org/abs/1502.05700
- github: https://github.com/bshahr/torch-dngo
Bayesian Dark Knowledge
- paper: http://arxiv.org/abs/1506.04416
- notes: Notes on Bayesian Dark Knowledge
Memory-based Bayesian Reasoning with Deep Learning(2015.Google DeepMind)
- slides: http://blog.shakirm.com/wp-content/uploads/2015/11/CSML_BayesDeep.pdf
Autoencoders
Importance Weighted Autoencoders
- paper: http://arxiv.org/abs/1509.00519
- code: https://github.com/yburda/iwae
Review of Auto-Encoders(by Piotr Mirowski, Microsoft Bing London, 2014)
- slides:https://piotrmirowski.files.wordpress.com/2014/03/piotrmirowski_2014_reviewautoencoders.pdf
- github: https://github.com/piotrmirowski/Tutorial_AutoEncoders/
Stacked What-Where Auto-encoders
- arxiv: http://arxiv.org/abs/1506.02351
Semi-Supervised Learning
Semi-Supervised Learning with Graphs (Label Propagation)
- paper: http://pages.cs.wisc.edu/~jerryzhu/pub/thesis.pdf
- blog(“标签传播算法(Label Propagation)及Python实现”):http://blog.csdn.net/zouxy09/article/details/49105265
Unsupervised Learning
Unsupervised Learning of Spatiotemporally Coherent Metrics
- paper: http://arxiv.org/abs/1412.6056
- code: https://github.com/jhjin/flattened-cnn
Unsupervised Learning on Neural Network Outputs
- intro: “use CNN trained on the ImageNet of 1000 classes to the ImageNet of over 20000 classes”
- arXiv: http://arxiv.org/abs/1506.00990
- github: https://github.com/yaolubrain/ULNNO
Deep Learning Networks
Deeply-supervised Nets (DSN)
- arxiv: http://arxiv.org/abs/1409.5185
- homepage: http://vcl.ucsd.edu/~sxie/2014/09/12/dsn-project/
- github: https://github.com/s9xie/DSN
- notes: http://zhangliliang.com/2014/11/02/paper-note-dsn/
Striving for Simplicity: The All Convolutional Net
- arxiv: http://arxiv.org/abs/1412.6806
Highway Networks
- arxiv: http://arxiv.org/abs/1505.00387
- blog(“Highway Networks with TensorFlow”): https://medium.com/jim-fleming/highway-networks-with-tensorflow-1e6dfa667daa#.71fgztsb6
Training Very Deep Networks (highway networks)
- arxiv: http://arxiv.org/abs/1507.06228
Very Deep Learning with Highway Networks
- homepage(papers, code, FAQ): http://people.idsia.ch/~rupesh/very_deep_learning/
Rectified Factor Networks
- arXiv: http://arxiv.org/abs/1502.06464
- github: https://github.com/untom/librfn
Correlational Neural Networks
- arxiv: http://arxiv.org/abs/1504.07225
Semi-Supervised Learning with Ladder Networks
- arxiv: http://arxiv.org/abs/1507.02672
- github: https://github.com/CuriousAI/ladder
Diversity Networks
- arxiv: http://arxiv.org/abs/1511.05077
A Unified Approach for Learning the Parameters of Sum-Product Networks (SPN)
- intro: “The Sum-Product Network (SPN) is a new type of machine learning model with fast exact probabilistic inference over many layers.”
- arxiv: http://arxiv.org/abs/1601.00318
- homepage: http://spn.cs.washington.edu/index.shtml
- code: http://spn.cs.washington.edu/code.shtml
Learning Discriminative Features via Label Consistent Neural Network
- arxiv: http://arxiv.org/abs/1602.01168
Binarized Neural Networks
- arxiv: http://arxiv.org/abs/1602.02505
BinaryConnect: Training Deep Neural Networks with binary weights during propagations
- paper: http://papers.nips.cc/paper/5647-shape-and-illumination-from-shading-using-the-generic-viewpoint-assumption
- github: https://github.com/MatthieuCourbariaux/BinaryConnect
BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- arxiv: http://arxiv.org/abs/1602.02830
- github: https://github.com/MatthieuCourbariaux/BinaryNet
A Theory of Generative ConvNet
- arxiv: http://arxiv.org/abs/1602.03264
- project page: http://www.stat.ucla.edu/~ywu/GenerativeConvNet/main.html
Value Iteration Networks
- arxiv: http://arxiv.org/abs/1602.02867
How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks
- arxiv: http://arxiv.org/abs/1602.02282
Group Equivariant Convolutional Networks (G-CNNs)
- arxiv: http://arxiv.org/abs/1602.07576
Deep Spiking Networks
- arxiv: http://arxiv.org/abs/1602.08323
- github: https://github.com/petered/spiking-mlp
Low-rank passthrough neural networks
- arxiv: http://arxiv.org/abs/1603.03116
- github: https://github.com/Avmb/lowrank-gru
Distributed System
SparkNet: Training Deep Networks in Spark
- arXiv: http://arxiv.org/abs/1511.06051
- github: https://github.com/amplab/SparkNet
- blog: http://www.kdnuggets.com/2015/12/spark-deep-learning-training-with-sparknet.html
A Scalable Implementation of Deep Learning on Spark (Alexander Ulanov)
- page: http://www.slideshare.net/AlexanderUlanov1/a-scalable-implementation-of-deep-learning-on-spark-alexander-ulanov
- slides: http://pan.baidu.com/s/1jHiNW5C
Deep Learning For Driving
- project: http://deepdriving.cs.princeton.edu/
- paper: http://deepdriving.cs.princeton.edu/paper.pdf
- code: http://deepdriving.cs.princeton.edu/DeepDriving.zip
Recurrent Neural Networks for Driver Activity Anticipation via Sensory-Fusion Architecture
- paper: http://arxiv.org/abs/1509.05016
- homepage: http://www.brain4cars.com/
Eyes on the Road: How Autonomous Cars Understand What They’re Seeing
- blog: http://blogs.nvidia.com/blog/2016/01/05/eyes-on-the-road-how-autonomous-cars-understand-what-theyre-seeing/
Deep Learning’s Accuracy
- blog: http://deeplearning4j.org/accuracy.html
GPU Programming
An Introduction to GPU Programming using Theano
- youtube: https://www.youtube.com/watch?v=eVd2TqEkVp0
- video: http://pan.baidu.com/s/1c1i6LtI#path=%252F
Deep Learning and Traditional ML
Decision Forests, Convolutional Networks and the Models in-Between
- arxiv: http://arxiv.org/abs/1603.01250
Papers
Reweighted Wake-Sleep
- paper: http://arxiv.org/abs/1406.2751
- code: https://github.com/jbornschein/reweighted-ws
Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
- paper: http://arxiv.org/abs/1502.05336
- code: https://github.com/HIPS/Probabilistic-Backpropagation
Deeply-Supervised Nets
- paper: http://arxiv.org/abs/1409.5185
- code: https://github.com/mbhenaff/spectral-lib
STDP
A biological gradient descent for prediction through a combination of STDP and homeostatic plasticity
- arxiv: http://arxiv.org/abs/1206.4812
An objective function for STDP(Yoshua Bengio)
- arxiv: http://arxiv.org/abs/1509.05936
Towards a Biologically Plausible Backprop
- arxiv: http://arxiv.org/abs/1602.05179
Bitwise Neural Networks
- paper: http://paris.cs.illinois.edu/pubs/minje-icmlw2015.pdf
- demo: http://minjekim.com/demo_bnn.html
Understanding and Predicting Image Memorability at a Large Scale (MIT. ICCV2015)
- homepage: http://memorability.csail.mit.edu/
- paper: https://people.csail.mit.edu/khosla/papers/iccv2015_khosla.pdf
- code: http://memorability.csail.mit.edu/download.html
- reviews: http://petapixel.com/2015/12/18/how-memorable-are-times-top-10-photos-of-2015-to-a-computer/
A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
- arxiv: http://arxiv.org/abs/1512.06293
Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition
- arxiv: http://arxiv.org/abs/1601.02970
- demo: http://brainmodels.csail.mit.edu/dnn/drawCNN/
Deep-Spying: Spying using Smartwatch and Deep Learning
- arxiv: http://arxiv.org/abs/1512.05616
- github: https://github.com/tonybeltramelli/Deep-Spying
A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
- arxiv: http://arxiv.org/abs/1512.06293
Understanding Deep Convolutional Networks
- arxiv: http://arxiv.org/abs/1601.04920
DeepCare: A Deep Dynamic Memory Model for Predictive Medicine
- arxiv: http://arxiv.org/abs/1602.00357
Exploiting Cyclic Symmetry in Convolutional Neural Networks
- arxiv: http://arxiv.org/abs/1602.02660
- github(Winning solution for the National Data Science Bowl competition on Kaggle (plankton classification)): https://github.com/benanne/kaggle-ndsb
- ref(use Cyclic pooling): http://benanne.github.io/2015/03/17/plankton.html
Cross-dimensional Weighting for Aggregated Deep Convolutional Features
- arxiv: http://arxiv.org/abs/1512.04065
- github: https://github.com/yahoo/crow
Understanding Visual Concepts with Continuation Learning
- project page: http://willwhitney.github.io/understanding-visual-concepts/
- arxiv: http://arxiv.org/abs/1602.06822
- github: https://github.com/willwhitney/understanding-visual-concepts
Learning Efficient Algorithms with Hierarchical Attentive Memory
- arxiv: http://arxiv.org/abs/1602.03218
Resnet in Resnet: Generalizing Residual Architectures
- paper: http://beta.openreview.net/forum?id=lx9l4r36gU2OVPy8Cv9g
Codes
deepnet: Implementation of some deep learning algorithms
- github: https://github.com/nitishsrivastava/deepnet
DeepNeuralClassifier(Julia): Deep neural network using rectified linear units to classify hand written digits from the MNIST dataset
- github: https://github.com/jostmey/DeepNeuralClassifier
Using deep learning to break a Captcha system
- intro: “Using Torch code to break simplecaptcha with 92% accuracy”
- blog: https://deepmlblog.wordpress.com/2016/01/03/how-to-break-a-captcha-system/
- github: https://github.com/arunpatala/captcha
Breaking reddit captcha with 96% accuracy
- blog: https://deepmlblog.wordpress.com/2016/01/05/breaking-reddit-captcha-with-96-accuracy/
- github: https://github.com/arunpatala/reddit.captcha
Clarifai Node.js Demo
- github: https://github.com/patcat/Clarifai-Node-Demo
- blog(“How to Make Your Web App Smarter with Image Recognition”):http://www.sitepoint.com/how-to-make-your-web-app-smarter-with-image-recognition/
Visual Search Server
- intro: “A simple implementation of Visual Search using features extracted from Tensor Flow inception model”
- github: https://github.com/AKSHAYUBHAT/VisualSearchServer
Deep Learning in Rust: baby steps
- blog: https://medium.com/@tedsta/deep-learning-in-rust-7e228107cccc#.t0pskuwkm
- github: https://github.com/tedsta/deeplearn-rs
Readings and Questions
What are the toughest neural networks and deep learning interview questions?
https://www.quora.com/What-are-the-toughest-neural-networks-and-deep-learning-interview-questions
26 Things I Learned in the Deep Learning Summer School
http://www.marekrei.com/blog/26-things-i-learned-in-the-deep-learning-summer-school/
http://www.csdn.net/article/2015-09-16/2825716
What you wanted to know about AI
http://fastml.com/what-you-wanted-to-know-about-ai/
Epoch vs iteration when training neural networks
- stackoverflow: http://stackoverflow.com/questions/4752626/epoch-vs-iteration-when-training-neural-networks
Questions to Ask When Applying Deep Learning
http://deeplearning4j.org/questions.html
Resources
Awesome Deep Learning
- github: https://github.com/ChristosChristofidis/awesome-deep-learning
Deep Learning Libraries by Language
- website: http://www.teglor.com/b/deep-learning-libraries-language-cm569/
Deep Learning Resources
http://yanirseroussi.com/deep-learning-resources/
Turing Machine: musings on theory & code(DEEP LEARNING REVOLUTION, summer 2015, state of the art & topnotch links)
https://vzn1.wordpress.com/2015/09/01/deep-learning-revolution-summer-2015-state-of-the-art-topnotch-links/
BICV Group: Biologically Inspired Computer Vision research group
http://www.bicv.org/deep-learning/
Learning Deep Learning
http://rt.dgyblog.com/ref/ref-learning-deep-learning.html
Summaries and notes on Deep Learning research papers
- github: https://github.com/dennybritz/deeplearning-papernotes
Deep Learning Glossary
- intro: “Simple, opinionated explanations of various things encountered in Deep Learning / AI / ML.”
- author: Ryan Dahl, author of NodeJS.
- github: https://github.com/ry/deep_learning_glossary
DNNGraph - A deep neural network model generation DSL in Haskell
- homepage: http://ajtulloch.github.io/dnngraph/
Books
Deep Learning (by Ian Goodfellow, Aaron Courville and Yoshua Bengio)
- homepage: http://www.deeplearningbook.org/
- website: http://goodfeli.github.io/dlbook/
Blogs
Neural Networks and Deep Learning
http://neuralnetworksanddeeplearning.com
Deep Learning Reading List
http://deeplearning.net/reading-list/
WILDML: A BLOG ABOUT MACHINE LEARNING, DEEP LEARNING AND NLP.
http://www.wildml.com/
Andrej Karpathy blog
http://karpathy.github.io/
Rodrigob’s github page
http://rodrigob.github.io/
colah’s blog
http://colah.github.io/
Competitions
Classifying plankton with deep neural networks
- blog: http://benanne.github.io/2015/03/17/plankton.html
- github: https://github.com/benanne/kaggle-ndsb
参考:http://handong1587.github.io/deep_learning/2015/10/09/dl-resources.html#alexnet