伪逆总结

参考书籍

方保镕, 周继东, 李医民. 矩阵论.第2版[M]. 清华大学出版社, 2013.7714
Convex optimization[M]. 2013.

1. 加号逆的定义
   矩阵$\text{A}\in R^{m\times n}$，若存在$\text{G}\in R^{m\times n}$满足：
   (1) $\text{AGA}=\text{A}$;
   (2) $\text{GAG}=\text{G}$;
   (3) $(\text{AG}{)}^T=\text{AG}$;
   (4) $(\text{GA}{)}^T=\text{GA}$;
   则称$G$为$A$的加号逆，或伪逆，或摩尔-彭诺斯逆，记为${\text{A}}^{+}$。

2. 加号逆的求法
   任何矩阵的加号逆具有唯一性。假设$A\in {\text{R}}^{m\times n}$，$\text{rank }A=r$。那么$A$可以因式分解为
   $$A=U\Sigma V^T$$
   其中$U\in {\text{R}}^{m\times r}$满足$U^{T}U=I$，$V\in {\text{R}}^{n\times r}$满足$V^{T}V=I$，而$\Sigma =\text{diag}({\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_r)$满足
   $${\sigma }_1\ge {\sigma }_2\ge \cdots \ge {\sigma }_r$$
   $U$的列向量称为$A$的左奇异向量，$V$的列向量称为$A$的右奇异向量，而${\sigma }_i$则称为奇异值。奇异值分解可以写成
   $$A=\sum _{i=1}^r{\sigma }_i{u}_i{v}_i^T，$$
   其中$u_i\in {\text{R}}^m$是左奇异向量，$v_i\in {\text{R}}^n$是右奇异向量。

矩阵$A$的伪逆如下：
$$A^{+}=V{\Sigma }^{-1}{U}^T\in {\text{R}}^{n\times m}.$$
根据伪逆的定义验证通过。
可将$U$扩展成${\text{R}}^{m\times m}$酉矩阵，将$V$扩展成${\text{R}}^{n\times n}$酉矩阵，$A$的奇异分解可以写成：
$$A=U_{m\times m}{\left[\begin{array}{cc}{\Sigma }_{r\times r}& 0\\ 0& 0\end{array}\right]}_{m\times n}{V}_{n\times n}^T$$
矩阵$A$的伪逆如下：
$$A^{+}=V_{n\times n}{\left[\begin{array}{cc}{\Sigma }_{r\times r}^{-1}& 0\\ 0& 0\end{array}\right]}_{n\times m}{U}_{m\times m}^T$$
根据伪逆的定义验证通过。

3. 等价表达式
   $$A^{+}=\underset{ϵ\to 0}{\lim }(A^{T}A+ϵI{)}^{-1}{A}^T=\underset{ϵ\to 0}{\lim }{A}^T(A{A}^T+ϵI{)}^{-1}$$
   证明：
   $$\begin{array}{rl}\underset{ϵ\to 0}{\lim }(A^{T}A+ϵI{)}^{-1}{A}^T& =\underset{ϵ\to 0}{\lim }(V_{n\times n}{\left[\begin{array}{cc}{\Sigma }_{r\times r}^2& 0\\ 0& 0\end{array}\right]}_{n\times n}{V}_{n\times n}^T+ϵI{)}^{-1}{A}^T\\ & =\underset{ϵ\to 0}{\lim }(V{\left[\begin{array}{cccc}{\sigma }_1^2+ϵ& & & \\ & \ddots & & \\ & & {\sigma }_r^2+ϵ& \\ & & & ϵI_{(n-r)\times (n-r)}\end{array}\right]}_{n\times n}{V}^T{)}^{-1}{A}^T\\ & =\underset{ϵ\to 0}{\lim }V{\left[\begin{array}{cccc}{\sigma }_1^2+ϵ& & & \\ & \ddots & & \\ & & {\sigma }_r^2+ϵ& \\ & & & ϵI\end{array}\right]}_{n\times n}^{-1}{V}^T{A}^T\\ & =\underset{ϵ\to 0}{\lim }V{\left[\begin{array}{cccc}\frac{1}{{\sigma }_1^2+ϵ}& & & \\ & \ddots & & \\ & & \frac{1}{{\sigma }_r^2+ϵ}& \\ & & & \frac{1}{ϵ}I\end{array}\right]}_{n\times n}{V}^{T}V{\left[\begin{array}{cc}{\Sigma }_{r\times r}& 0\\ 0& 0\end{array}\right]}_{n\times n}{U}^T\\ & =\underset{ϵ\to 0}{\lim }V\left[\begin{array}{cccc}\frac{{\sigma }_1}{{\sigma }_1^2+ϵ}& & & \\ & \ddots & & \\ & & \frac{{\sigma }_r}{{\sigma }_r^2+ϵ}& \\ & & & 0\end{array}\right]U^T\\ & =V\left[\begin{array}{cccc}\frac{1}{{\sigma }_1}& & & \\ & \ddots & & \\ & & \frac{1}{{\sigma }_r}& \\ & & & 0\end{array}\right]U^T\\ & =V\left[\begin{array}{cc}{\Sigma }_{r\times r}^{-1}& 0\\ 0& 0\end{array}\right]U^T\end{array}$$
   得证。

（2）
$$\begin{array}{rl}\underset{ϵ\to 0}{\lim }{A}^T(A{A}^T+ϵI{)}^{-1}& =\underset{ϵ\to 0}{\lim }{A}^T(U_{m\times m}{\left[\begin{array}{cc}{\Sigma }_{r\times r}^2& 0\\ 0& 0\end{array}\right]}_{m\times m}{U}_{m\times m}^T+ϵI{)}^{-1}\\ & =\underset{ϵ\to 0}{\lim }{A}^T(U_{m\times m}{\left[\begin{array}{cccc}{\sigma }_1^2+ϵ& & & \\ & \ddots & & \\ & & {\sigma }_r^2+ϵ& \\ & & & ϵI_{(m-r)\times (m-r)}\end{array}\right]}_{m\times m}{U}_{m\times m}^T{)}^{-1}\\ & =\underset{ϵ\to 0}{\lim }{A}^{T}U{\left[\begin{array}{cccc}{\sigma }_1^2+ϵ& & & \\ & \ddots & & \\ & & {\sigma }_r^2+ϵ& \\ & & & ϵI\end{array}\right]}^{-1}{U}^T\\ & =\underset{ϵ\to 0}{\lim }V\left[\begin{array}{cc}{\Sigma }_{r\times r}& 0\\ 0& 0\end{array}\right]U^{T}U\left[\begin{array}{cccc}\frac{1}{{\sigma }_1^2+ϵ}& & & \\ & \ddots & & \\ & & \frac{1}{{\sigma }_r^2+ϵ}& \\ & & & \frac{1}{ϵ}I\end{array}\right]U^T\\ & =\underset{ϵ\to 0}{\lim }V{\left[\begin{array}{cccc}\frac{{\sigma }_1}{{\sigma }_1^2+ϵ}& & & \\ & \ddots & & \\ & & \frac{{\sigma }_r}{{\sigma }_r^2+ϵ}& \\ & & & 0\end{array}\right]}_{n\times n}{U}^T\\ & =V\left[\begin{array}{cccc}\frac{1}{{\sigma }_1}& & & \\ & \ddots & & \\ & & \frac{1}{{\sigma }_r}& \\ & & & 0\end{array}\right]U^T\\ & =V\left[\begin{array}{cc}{\Sigma }_{r\times r}^{-1}& 0\\ 0& 0\end{array}\right]U^T\end{array}$$