Literatures on Deep Learning 关于Deep Learning的一些文章

Recently I engaged in studying Deep Learning, which was motivated by G. E. Hinton from University of Toronto in 2006. The striking paper was A FAST LEARNING ALGORITHM FOR DEEP BELIEF NETS in Neural Computation and a corresonding paper by Hinton appeared on Nature at the same time. Since then, Deep Learning welcomes its new spring.

    Deep Learning is to explore the visual mechanism which employs hierarchical representations in parsing natural scenes from V1,V2,...,V5. Each higher level representation is composed of lower level representations, and thus is more abstract and robust in visual tasks. Deep Learning methods, which make use of multilayer perceptions to imitate the visual mechanism, provide a universal framework for representing complex functions which is the shortcomings of shallower networks or classifiers. Based on these reasons, Deep Learning has been a new frontier in machine learning research.

    Here is a reading list below on Deep Learning since the breakthrough by Hinton.

1. G. E. Hinton,et al..Reducing the dimensionality of data with neural networks. Nature,2006.

2. G. E. Hinton,et al..A Fast Learning Algorithm for Deep Belief Nets. Neural Computaion,2006.

3. Marc' Aurelio Ranzato, Yann LeCun. Efficient Learning of Sparse Representations with an Energy-Based Model. ICML,2007.

4. Yoshua Bengio. Greedy Layer-Wise Training of Deep Networks.ICML,2007.

5. Jason Weston. Deep Learning via Semi-supervised Embedding. ICML,2008.

6. Yoshua Bengio. Learning Deep Architectures for AI. Foundations and Trends in Machine Learning,2009. (Review)

7. Hugo Larochelle,Yoshua Bengio. Exploring Strategies for Training Deep Neural Networks. Journal of Machine Learning Research,2009.

8. Dumitru Erhan,Yoshua Bengio. The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training. ICML,2009.

9. Pascal Vincent,Yohusa Bengio. Stacked Denoising Autoencoders- Learning Useful Representations in a Deep Network with a Local Denoising Criterion. Journal of Machine Learning Research,2010.

10. Itamar Arel. Deep Machine Learning: A New Friontier in Artificial Interlligence Research. IEEE Computation Intelligence Magazine,2010.

11. Dumitru Erhan,Yoshua Bengio. Why Does Unsupervised Pre-training Help Deep Learning. Journal of Machine Learning Research,2010.

12. Xavier Glorot,Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. AISTATS,2010.