【Deep Learning】1、AutoEncoder

Deep Learning 第一战：

完成：UFLDL教程 稀疏自编码器-Exercise：Sparse Autoencoder

Code:

学习到的稀疏参数W1：

参考资料：

1. UFLDL教程 稀疏自编码器

Autoencoders相关文章阅读:

• [3] Hinton, G. E., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets
• [4] Hinton, G. E. and Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 2006.
  • If you want to play with the code, you can also find it at [5].
• [6] Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. Greedy Layer-Wise Training of Deep Networks. NIPS 2006
• [7] Pascal Vincent, Hugo Larochelle, Yoshua Bengio and Pierre-Antoine Manzagol. Extracting and Composing Robust Features with Denoising Autoencoders. ICML 2008.
  • (They have a nice model, but then backwards rationalize it into a probabilistic model. Ignore the backwards rationalized probabilistic model [Section 4].)

【4】Reducing the dimensionality of data with neural networks，Hinton用的是RBM来pre-training参数 

【5】

【6】Greedy Layer-Wise Training of Deep Networks中，Bengio例证了 RBM可以用autoencoder来替换，能得到相当的performance；探索了DBN的训练、对连续数值输入的适用问题、Dealing with uncooperative input distributions等。

【7】Extracting and Composing Robust Features with Denoising Autoencoders 处理带噪声/遮挡的图像