矩阵的奇异值分解

定义

矩阵的奇异值分解，顾名思义，是矩阵分解的一种。定义如下：

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It has many useful applications in signal processing and statistics.

Formally, the singular value decomposition of an m × n real or complex matrix M is a factorization of the form M = UΣV∗, where U is an m × m real or complex unitary matrix, Σ is an m × n rectangular diagonal matrix with non-negative real numbers on the diagonal, and V∗ (the conjugate transpose of V, or simply the transpose of V if V is real) is an n × n real or complex unitary matrix. The diagonal entries Σi,i of Σ are known as the singular values of M. The m columns of U and the n columns of V are called the left-singular vectors and right-singular vectors of M, respectively.

那么奇异值分解和特征值分解是什么关系呢？

The singular value decomposition and the eigendecomposition are closely related. Namely:

• The left-singular vectors of M are eigenvectors of MM∗.

• The right-singular vectors of M are eigenvectors of M∗M.

• The non-zero singular values of M (found on the diagonal entries of Σ) are the square roots of the non-zero eigenvalues of both M∗M and MM∗.

思路总结如下：

尤其需要注意外积形式的SVD

A=u1*c1*v1'+u2*c2*v2'+……+uk*ck*vk'

应用前景

在机器学习领域，有相当多的应用与奇异值都可以扯上关系，比如做feature reduction的PCA，做数据压缩（以图像压缩为代表）的算法，还有做搜索引擎语义层次检索的LSI（Latent Semantic Indexing）

调用代码

在matlab中可以直接调用svd（）函数即可。

>> A=[1 2;3 4; 5 6;7 8]

A =

     1     2
     3     4
     5     6
     7     8

>> size(A)

ans =

     4     2

>> svd(A)

ans =

   14.2691
    0.6268

>> [U,S,V]=svd(A)

U =

   -0.1525   -0.8226   -0.3945   -0.3800
   -0.3499   -0.4214    0.2428    0.8007
   -0.5474   -0.0201    0.6979   -0.4614
   -0.7448    0.3812   -0.5462    0.0407


S =

   14.2691         0
         0    0.6268
         0         0
         0         0


V =

   -0.6414    0.7672
   -0.7672   -0.6414