无监督学习特征--稀疏编码、深度学习、ICA部分代表文献-------之一

l         学习映射函数及在行为识别/图像分类中应用的文献（模型与非模型之间存在关联，算法相互采用，没有明确的区分，含仿生学文献）

% 研究重点放到ICA模型及深度学习兼顾稀疏编码

1）稀疏编码（稀疏编码、自动编码、递归编码）：

[1] B. Olshausen and D. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 1996.

[2] H. Lee, A. Battle, R. Raina, and A. Y. Ng. Efficient sparse coding algorithms. In NIPS, 2007.

[3] B. A. Olshausen. Sparse coding of time-varying natural images. In ICA, 2000.

[4] Dean, T., Corrado, G., Washington, R.: Recursive sparse spatiotemporal coding.In: Proc. IEEE Int. Workshop on Mult. Inf. Proc. and Retr. (2009).

[5] J. Yang, K. Yu, Y. Gong, and T. Huang. Linear spatial pyramid matching using sparse coding for image

classification. In CVPR, 2009.

[6] S. Wang, L. Zhang, Y. Liang and Q. Pan.Semi-Coupled Dictionary Learning with Applications to Image Super-Resolution and Photo-Sketch Image Synthesis. in CVPR 2012.

[7] Yan Zhu, Xu Zhao,Yun Fu,Yuncai Liu. Sparse Coding on Local Spatial-temporal Volumes for Human action Recognition.ACCV2010,Part II,LNCS 6493.(上海交大，采用3DHOG特征描述，3DSift稀疏编码未注意)。

2）ICA（ISA）模型：

[1] A. Hyvarinen, J. Hurri, and P. Hoyer. Natural Image Statistics. Springer, 2009.

[2]Alireza Fathi and Greg Mori. Action Recognition by Learning Mid-level Motion Features. IEEE,2008,978-1-4244-2243.

[3] A. Hyvarinen and P. Hoyer. Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neu. Comp., 2000.

[4] A. Coates, H. Lee, and A. Y. Ng. An analysis of single-layer networks in unsupervised feature learning.

In AISTATS 14, 2011.（该篇采用采用的特征，基于BOW方法不需要检测。形成BOW时采用图像块相似聚类，跟据离BOW距离将图像块特征非线性判决，之后将正副图像以一种稀疏形式表示）

[5] Q. V. Le, W. Zou, S. Y. Yeung, and A. Y. Ng. Learning hierarchical spatio-temporal features for action

recognition with independent subspace analysis. In CVPR, 2011.

[6] Q. V. Le, J. Ngiam, Z. Chen, D. Chia, P. W. Koh, and A. Y. Ng. Tiled convolutional neural networks. In

NIPS, 2010.

[7] M. S. Lewicki and T. J. Sejnowski. Learning overcomplete representations. Neural Computation, 2000.

[8] L. Ma and L. Zhang. Overcomplete topographic independent component analysis. Elsevier, 2008.

[9] A. Krizhevsky. Learning multiple layers of features from tiny images. Technical report, U. Toronto, 2009.

%非分类识别文献，引入copula估计子空间，新特征组合

[10]Nicolas Brunel, Wojciech Pieczynski,Stephane Derrode.Copulas in vectorial hidden markov chains for multicomponent images segmentation.ICASSP’05,Philadelphia,USA,March 19-23,2005.（非识别分类文献，但是涉及到一种算法，对估计子空间很有用，可以引入ICA模型。）

[11] Xiaomei Qu. Feature Extraction by Combining Independent Subspaces Analysis and Copula Techniques. International Conference on system Science and Engineering,2012.

[12] Pietro Berkes, Frank Wood and Jonathan Pillow. Characterizing neural dependencies with copula models. In NIPS, 2008.

[13] Y-Lan Boureau, Jean Ponce, Yann LeCun. A theoretical Analysis of Feature Pooling in Visual Recognition. In Proceedings of the 27’th International Conference on machine Learning, Haifa, Israel,2010.（介绍多样池及概念，可以形成稀疏表示及产生鲁棒性特征）

3）深度学习（与ICA、RBM关联性强，属于多层学习）：

[1] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle.Greedy layerwise training of deep networks. In NIPS, 2006.

[2] Alessio Plebe. A model of the response of visual area V2 to combinations of orientations. Network: Computation in Neural Systems, September 2012; 23(3): 105–122.(涉及到模拟人类大脑皮层感知（v1、v2、v3、v4、v5），此类文献多，主要以猴子猫动物实验)

[3] G. Hinton, S. Osindero, and Y. Teh. A fast learning algorithms for deep belief nets. Neu. Comp., 2006

[4] H. Lee, R. Grosse, R. Ranganath, and A. Ng. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In ICML, 2009.

[5]Yann Lecun, Koray Kavukcuoglu, and Clement Farabet. Convolutional Networks and Applications in Vision. In Proc. International Symposium on Circuits and Systems (ISCAS'10), 2010.

[6] Pierre Sermanet, Soumith Chintala and Yann LeCun. Convolutional Neural Networks Applied to House Numbers Digit Classification. Computer Vision and Pattern Recognition,2012.

[7] Quoc V. Le. Marc’Aurelio Ranzato. Rajat Monga. Matthieu Devin. Kai Chen. Greg S. Corrado. Jeff Dean. Andrew Y. Ng. Building High-level Features Using Large Scale Unsupervised Learning. the 29’th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012.

[8] A. Hyvarinen and P. Hoyer. Topographic independent component analysis as a model of v1 organization and receptive fields. Neu. Comp., 2001

[9]Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle. Greedy layerwise training of deep networks. In

NIPS, 2007.

[10] Q. V. Le, J. Ngiam, A. Coates, A. Lahiri, B. Prochnow, and A. Y. Ng. On optimization methods for deep learning. In ICML, 2011.

[11] H. Lee, C. Ekanadham, and A. Y. Ng. Sparse deep belief net model for visual area V2. In NIPS, 2008.

[12] G. E. Hinton, S. Osindero, and Y. W. Teh. A fast learning algorithm for deep belief nets. Neural Computation, 2006.

[13] Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multistage architecture for object recognition? In: ICCV. (2009) 2146-2153.

[14] Lee, H., Ekanadham, C., Ng., A.: Sparse deep belief net model for visual area V2.In: NIPS. (2008) 873-880.

[15] Bo Chen .Deep Learning of Invariant Spatio-Temporal Feature from Video.[D].2010.

[16] Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh,Andrew Y.Ng.Learning Deep Energy Models.in Proceedings of the 28’th international Conference on Machine Learning,Bellevue,WA,USA,2011.

%以下（CRBM、SF）这些模型参考，不做重点研究，可借鉴算法。

4）CRBM(文献多，有博士论文)：

[1] G. Hinton. A practical guide to training restricted boltzmann machines. Technical report, U. of Toronto,

2010．

[2] G. Taylor, R. Fergus, Y. Lecun, and C. Bregler. Convolutional learning of spatio-temporal features. In ECCV, 2010.

[3] Norouzi, M., Ranjbar, M., Mori, G.: Stacks of convolutional restricted Boltzmann machines for shift-invariant feature learning. In: CVPR. (2009).

[4] Memisevic, R., Hinton, G.: Learning to represent spatial transformations with factored higher-order Boltzmann machines. Neural Comput 2010.

 

5）Slow Feature（慢特征学习分析（德国），代表文献）

这种新方法以邻帧图像为基础研究，是一种新思路。

[1] P. Berkes and L. Wiskott. Slow feature analysis yields arich repertoire of complex cell properties. Journal of Vision,2005