Deep Learning论文阅读-Abstract阶段

深度学习文章阅读

---

前言

• 按照Deep Learning思维导图中目前阶段的优先级进行文献阅读；
• 论文阅读方法按照彭明辉教授的研究生指导手册进行。

---

Review

1、几篇中文综述：

• 深度学习研究研究与进展-孙志远-中科院计算技术研究所-2016.02-计算机科学
• 深度学习研究综述-尹宝才-北京工业大学-2015.01-京工业大学学报
• 深度学习的昨天、今天和明天-余凯-百度-2013-计算机研究与发展
• 深度学习研究综述-孙志军-电子工程学院-2012.08-计算机应用研究
  • 高效的特征提取方法（特征学习）；

好像没什么用？

2、Deep Learning of Representations: Looking Forward_Yoshua Bengio_2013.06

Abstract:

深度学习目的，分层学习数据的特征，越高层级可以表达越抽象的概念；

Challenges:

scaling to larger models and datasets;
reduce optimization difficulties due to ill-conditioning or local minima;
design more efficient and powerful inference and sampling procedures;
disentangle the factors of variation underlying the observed data.

3、Deep Learning of Representations for Unsupervised and Transfer Learning_Yoshua Bengio_2012.

Abstract:

特征学习；

Hypothesis:

input distribution P(x) is structurally related to some task of interest, say predicting P(y|x);

Challenges:

why unsupervised pre-training of representations can be useful;
How it can be exploited in the transfer learning scenario, where we care about predictions on examples that are not from the same distribution as the training distribution.

4、Deep learning in neural networks: An overview_Jurgen Schmidhuber_2015_The Swiss AI Lab IDSIA

Abstract:

介绍了神经网络的历史；

5、Deep learning_Yann LeCun,Yoshua Bengio & Geoffrey Hinton_2015.05

Abstract:

对数据进行逐层表示逐层抽象。

6、Representation Learning: A Review and New Prespectives_Yoshua Bengio_2014.04

Abstract:

unsupervised feature learning and deep learning;
Probabilistic models, auto-encoders, manifold learning, deep networks.

Questions:

appropriate objectives for learning good representations, for computing representations;
Geometrical connections between representation learning, density estimation and manifold learning.

---

Application

---

Optimization

GD

1、An overview of gradient descent optimization algorithms

---

Tools

TensorFlow

1、TensorFlow: A system for large-scale machine learning_Google Brain_2016.05

Abstract: