Eigen 矩阵的SVD分解

一、SVD分解原理

  奇异值分解是将一个非零的实数矩阵$A_{m\times n}$分解成由三个是矩阵乘积形式的运算，即进行矩阵的因子分解：
$$A=U\Sigma V^T$$
其中，$U$为$m\times m$的单位正交阵，$V$为$n\times n$的单位正交阵，即有$U{U}^T=I,V{V}^T=I$。$\Sigma$是$m\times n$维的对角矩阵其对角线上的数值即为奇异值，并且按照降序排列，如：
$$\Sigma ={\left[\begin{array}{cccc}\sqrt{{\sigma }_1}& & & \\ & \sqrt{{\sigma }_2}& & \\ & & \ddots & \\ & & & \ddots \end{array}\right]}_{m\times n}$$
  $U\Sigma V^T$为矩阵$A$的奇异值分解，$\sqrt{{\sigma }_i}$为矩阵的奇异值，$U$称为左奇异矩阵，$V$称为右奇异矩阵。各矩阵的维度分别为$U\in R^{m\times m}$,$\Sigma \in R^{m\times n}$,$V\in R^{n\times n}$。
  为求解上述$U$,$\Sigma$,$V$，我们可以利用如下性质：
$$\begin{gathered} A{A}^T=U\Sigma V^T(U\Sigma V^T{)}^T=U\Sigma V^{T}V{\Sigma }^T{U}^T=U\Sigma {\Sigma }^T{U}^T \\ {A}^{T}A=(U\Sigma V^T{)}^{T}U\Sigma V^T=V{\Sigma }^T{U}^{T}U\Sigma V^T=V{\Sigma }^T\Sigma V^T \end{gathered}$$
其中$\Sigma {\Sigma }^T\in R^{m\times m}$,${\Sigma }^T\Sigma \in R^{n\times n}$,即：

$$\Sigma {\Sigma }^T={\left[\begin{array}{cccc}{\sigma }_1& & & \\ & {\sigma }_2& & \\ & & \ddots & \\ & & & \ddots \end{array}\right]}_{m\times m}$$

$${\Sigma }^T\Sigma ={\left[\begin{array}{cccc}{\sigma }_1& & & \\ & {\sigma }_2& & \\ & & \ddots & \\ & & & \ddots \end{array}\right]}_{n\times n}$$
可以看到$\Sigma {\Sigma }^T$和${\Sigma }^T\Sigma$的形式非常接近，进一步分析，我们可以发现$A{A}^T$和$A^{T}A$也是对称矩阵，那么就可以做特征值分解(EVD)。对$A{A}^T$特征值分解，得到的特征矩阵即为$U$ 。对$A^{T}A$特征值分解，得到的特征矩阵即为$V$ 。对$A{A}^T$或$A^{T}A$中的特征值开方，可以得到所有的奇异值。
由此可求得特征值为$\sqrt{{\sigma }_1}$ 、$\sqrt{{\sigma }_2}$ 、…、$\sqrt{{\sigma }_k}$ 。所以矩阵$A$：
$$A=U\Sigma V^T=\left[\begin{array}{cccc}{u}_1& u_2& \cdots & u_m\end{array}\right]{\left[\begin{array}{cccc}\sqrt{{\sigma }_1}& & & \\ & \sqrt{{\sigma }_2}& & \\ & & \ddots & \\ & & & \ddots \end{array}\right]}_{m\times n}\left[\begin{array}{c}{v}_1\\ v_2\\ ⋮\\ v_n\end{array}\right]$$
进一步化简得到
$$A=\sqrt{{\sigma }_1}{u}_1{v}_1^T+\sqrt{{\sigma }_2}{u}_2{v}_2^T+\cdots +\sqrt{{\sigma }_k}{u}_k{v}_k^T$$
即：
$$A={\lambda }_1{u}_1{v}_1^T+{\lambda }_2{u}_2{v}_2^T+\cdots +{\lambda }_k{u}_k{v}_k^T$$
矩阵$A$被分解为$k$个小矩阵，每个矩阵都是$\lambda$乘以$u_i{v}_i^T$。此时可以看到，若$\lambda$取值较大，其对应的矩阵在矩阵$A$的占比也大，所以取以去前面主要的$\lambda$来近似代表系统，近似的系统可以表示为：
$$A_{m\times n}=U_{m\times m}{\Sigma }_{m\times n}{V}_{n\times n}^T\approx U_{m\times k}{\Sigma }_{k\times k}{V}_{k\times n}^T$$

二、SVD分解举例

$$A=\left[\begin{array}{cc}0& 1\\ 1& 1\\ 1& 0\end{array}\right]$$
求解$A^{T}A$和$A^{T}A$:
$$A^{T}A=\left[\begin{array}{ccc}0& 1& 1\\ 1& 1& 0\end{array}\right]\left[\begin{array}{cc}0& 1\\ 1& 1\\ 1& 0\end{array}\right]=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]$$

$$A{A}^T=\left[\begin{array}{cc}0& 1\\ 1& 1\\ 1& 0\end{array}\right]\left[\begin{array}{ccc}0& 1& 1\\ 1& 1& 0\end{array}\right]=\left[\begin{array}{ccc}1& 1& 0\\ 1& 2& 1\\ 0& 1& 1\end{array}\right]$$
进而求$A^{T}A$的特征值和特征向量：
$$\begin{gathered} {\lambda }_1=3;v_1=\left[\begin{array}{c}1/\sqrt{2}\\ 1/\sqrt{2}\end{array}\right] \\ {\lambda }_2=1;v_2=\left[\begin{array}{c}1/\sqrt{2}\\ -1/\sqrt{2}\end{array}\right] \end{gathered}$$
$A^{T}A$的特征值和特征向量
$$\begin{gathered} {\lambda }_1=3;v_1=\left[\begin{array}{c}1/\sqrt{6}\\ 2/\sqrt{6}\\ 1/\sqrt{6}\end{array}\right] \\ {\lambda }_2=1;v_2=\left[\begin{array}{c}-1/\sqrt{2}\\ 0\\ 1/\sqrt{2}\end{array}\right] \\ {\lambda }_3=0;v_3=\left[\begin{array}{c}1/\sqrt{3}\\ -1/\sqrt{3}\\ 1/\sqrt{3}\end{array}\right] \end{gathered}$$
利用${\sigma }_i=\sqrt{{\lambda }_i}$，可得奇异值为$\sqrt{3}$和$1$。最终得$A$的奇异值分解为：
$$A=U\Sigma V^T=\left[\begin{array}{ccc}1/\sqrt{6}& -1/\sqrt{2}& 1/\sqrt{3}\\ 2/\sqrt{6}& 0& -1/\sqrt{3}\\ 1/\sqrt{6}& 1/\sqrt{2}& 1/\sqrt{3}\end{array}\right]\left[\begin{array}{cc}\sqrt{3}& 0\\ 0& 1\\ 0& 0\end{array}\right]\left[\begin{array}{cc}1/\sqrt{2}& 1/\sqrt{2}\\ 1/\sqrt{2}& -1/\sqrt{2}\end{array}\right]$$

三、用Eigen库实现SVD分解

1.C++代码

#include 
#include 

using namespace std;
using namespace Eigen;

int main()
{
	MatrixXf m = MatrixXf::Zero(3, 2);
	m << 0,1,1,1,1,0;
	cout << "Here is the matrix m:" << endl << m << endl;
	JacobiSVD<MatrixXf> svd(m, ComputeFullU | ComputeFullV);
	cout << "Its singular values are:" << endl << svd.singularValues() << endl;
	cout << "Its left singular vectors are the columns of the thin U matrix:" << endl << endl << svd.matrixU() << endl;
	cout << "Its right singular vectors are the columns of the thin V matrix:" << endl << endl << svd.matrixV() << endl;
	system("pause");
	return 0;
}

2.输出结果

Here is the matrix m:
0 1
1 1
1 0
Its singular values are:
1.73205
      1
Its left singular vectors are the columns of the thin U matrix:

   0.408248   -0.707107     0.57735
   0.816496 5.96046e-08    -0.57735
   0.408248    0.707107     0.57735
Its right singular vectors are the columns of the thin V matrix:

 0.707107  0.707107
 0.707107 -0.707107