【矩阵论】2. 矩阵分解——SVD

矩阵论
1. 准备知识——复数域上的矩阵与换位公式)
1. 准备知识——复数域上的内积域正交阵
1. 准备知识——相似对角化与合同&正定阵
2. 矩阵分解—— SVD准备知识——奇异值
2. 矩阵分解——SVD
2. 矩阵分解——QR分解
2. 矩阵分解——乔利斯分解&平方根公式
2. 矩阵分解——正规谱分解——正规阵
2. 矩阵分解——正规分解
2. 矩阵分解——单阵及特征值特征向量一些求法

---

3.2 奇异值分解SVD

3.2.1 正SVD

$$\begin{array}{rl}& 设A=A_{m\times n}，r(A)>0,正奇值\sqrt{{\lambda }_1},\cdots ,\sqrt{{\lambda }_r},则有分解A=P\Delta Q^H\\ & 其中\Delta =\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)，P_{m\times r}，Q_{n\times r}为半U阵,P^{H}P=I_r=Q^{H}Q\\ & 可写正SVD公式A=P\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)Q^H\end{array}$$

---

证明

1. $P_{m\times r}，Q_{n\times r}$ 的构造

$$\begin{array}{rl}& A^{H}A为Hermite阵，由Hermite分解定理，存在U阵\\ & 使U^H(A^{H}A)U={\left(\begin{array}{cccc}& {\lambda }_1& & \\ & & \ddots & \\ & & & {\lambda }_n\end{array}\right)}_{n\times n}，\\ & ,且A^{H}A为半正定阵，有{\lambda }_1,\cdots ,{\lambda }_r>0,{\lambda }_{r+1}=\cdots ={\lambda }_n=0,r(A)=r\\ & 而U的列向量(q_1,q_2,\cdots ,q_n)为A^{H}A的特征向量\\ & 且A^{H}A{q}_1={\lambda }_1{q}_1,\cdots ,A^{H}A{q}_r={\lambda }_r{q}_r,A^{H}A{q}_{r+1}=0,A^{H}A{q}_n=0\\ & \\ & 令Q=(q_1,q_2,\cdots ,q_r{)}_{n\times r}，P=(\frac{A{q}_1}{|A{q}_1|},\cdots ,\frac{A{q}_r}{|A{q}_r|}{)}_{m\times r}\end{array}$$

2. P与Q为半U阵
   $$\begin{array}{rl}& 已知Q中列向量为U阵的r个非零列向量，则Q为半U阵\\ & 而(A{q}_1,A{q}_2)=(A{q}_2{)}^H(A{q}_1)=q_2{A}^{H}A{q}_1={\lambda }_1{q}_2^H{q}_1=({\lambda }_1{q}_1,q_2)=0\\ & 同理，A{q}_i与A{q}_j都正交\\ & |A{q}_1{|}^2=(A{q}_1{)}^H(A{q}_1)=q_1^H{A}^{H}A{q}_1={\lambda }_1{q}_1^H{q}_1={\lambda }_1\ge 0\\ & \Rightarrow P=\left(\frac{A{q}_1}{|A{q}_1|},\frac{A{q}_2}{|A{q}_2|},\cdots ,\frac{A{q}_r}{|A{q}_r|}\right)=\left(\frac{A{q}_1}{\sqrt{{\lambda }_1}},\frac{A{q}_2}{\sqrt{{\lambda }_2}},\cdots ,\frac{A{q}_r}{\sqrt{{\lambda }_r}}\right)\\ & 即P阵为半U阵\end{array}$$

3. 代入P,Q
   $$\begin{array}{rl}& P\Delta Q^H=\left(\frac{A{q}_1}{\sqrt{{\lambda }_1}},\frac{A{q}_2}{\sqrt{{\lambda }_2}},\cdots ,\frac{A{q}_r}{\sqrt{{\lambda }_r}}\right)\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)\left(\begin{array}{c}{q}_1^H\\ q_2^H\\ ⋮\\ q_r^H\end{array}\right)\\ & =(A{q}_1,A{q}_2,\cdots ,A{q}_r)\left(\begin{array}{c}{q}_1^H\\ q_2^H\\ ⋮\\ q_r^H\end{array}\right)=\left(A{q}_1{q}_1^H,A{q}_2{q}_2^H,，\cdots ,A{q}_r{q}_r^H,\right)\end{array}$$

4. 验证 $\left(A{q}_1{q}_1^H,A{q}_2{q}_2^H,，\cdots ,A{q}_r{q}_r^H,\right)=A$
   $$\begin{array}{rl}& 由A^{H}A为半正定Hermite阵，r(A^{H}A)=r\\ & 由同解定理，A^{H}Ax=0⟺Ax=0\\ & A^{H}A{q}_{r+1}=\cdots =A^{H}A{q}_n=0\Rightarrow A{q}_{r+1}=\cdots =A{q}_n=0\\ & \Rightarrow A(q_{r+1}{q}_{r+1}^H+\cdots +q_n{q}_n^H)=0\\ & 而A(q_1{q}_1^H+\cdots +q_r{q}_r^H+q_{r+1}{q}_{r+1}^H+\cdots +q_n{q}_n^H)=AI=A\\ & 即A(q_1{q}_1^H+\cdots +q_r{q}_r^H)=A\end{array}$$

最后得证 正SVD公式，$A=P\Delta Q^H=P\left(\begin{array}{cccc}& \sqrt{{\lambda }_1}& & \\ & & \ddots & \\ & & & \sqrt{{\lambda }_r}\end{array}\right)Q^H$

分解方法

1. 求 $A^{H}A$ 的特征值，${\lambda }_1\ge \cdots ,\ge {\lambda }_r>0$

   正奇值为 $\sqrt{{\lambda }_1},\cdots \sqrt{{\lambda }_r}$

2. 求 ${\lambda }_1,\cdots ,{\lambda }_r$ 的正交特征向量（不必单位化）

3. 令列半U阵 $Q=\left(\frac{X_1}{|X_1|},\cdots ,\frac{X_r}{|X_r|}\right),P=\left(\frac{A{X}_1}{|A{X}_1|},\cdots ,\frac{A{X}_r}{|A{X}_r|}\right)$

eg

$$\begin{array}{r}A=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 1& 1\end{array}\right)\end{array}$$

---

$$\begin{array}{rl}& A^{H}A=\left(\begin{array}{cc}2& 2\\ 2& 2\end{array}\right),由A^{H}A为秩1矩阵，得\lambda =\{4,0\}，正奇值为\sqrt{\lambda }=\sqrt{4}=2\\ & 而{\lambda }_1=4的特征向量为X_1=\left(\begin{array}{c}1\\ 1\end{array}\right),则\Delta =\left(2\right)\\ & 令Q=\left(\frac{X_1}{|X_1|}\right)=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}\end{array}\right),P=\left(\begin{array}{c}\frac{A{X}_1}{|A{X}_1|}\end{array}\right)=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ 0\\ \frac{1}{\sqrt{2}}\end{array}\right)\\ & 则A的正SVD为A=P\Delta Q^H=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ 0\\ \frac{1}{\sqrt{2}}\end{array}\right)\left(2\right)\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)\end{array}$$

考虑 $B=\left(\begin{array}{ccc}1& 0& 1\\ 1& 0& 1\end{array}\right)$ 的正SVD

---

$$\begin{array}{rl}& 已知B=A^H,且有A的正SVD，A=P\Delta Q^H=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ 0\\ \frac{1}{\sqrt{2}}\end{array}\right)\left(2\right)\left(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}}\right)\\ & B=(P\Delta Q^H{)}^H=Q\Delta P^H=\left(\begin{array}{c}\frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}\end{array}\right)\left(2\right)\left(\frac{1}{\sqrt{2}},0,\frac{1}{\sqrt{2}}\right)\end{array}$$

3.2.2 SVD

$$\begin{array}{rl}& 设A=A_{m\times n}，r(A)>0,正奇值\sqrt{{\lambda }_1},\cdots ,\sqrt{{\lambda }_r}>0,则有分解A=W\Delta V^H\\ & 其中D={\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)}_{m\times n}，W_{m\times m}=(P,P_1),V_{n\times n}=(Q,Q_1)\\ & 可写SVD公式A=W\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)V^H=P\Delta Q^H,其中\left(\begin{array}{ccc}\sqrt{{\lambda }_1}& & \\ & \ddots & \\ & & \sqrt{{\lambda }_r}\end{array}\right)\end{array}$$

证明:
$$\begin{gathered} \begin{array}{rl}& 由正SVD：A=P\Delta Q^H,分别把P，Q扩充为U阵，即W=(P,P_1),V=(Q,Q_1)，\\ & V^H=\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right),则WD{V}^H=(P,P_1)\left(\begin{array}{cc}\Delta & 0\\ 0& 0\end{array}\right)\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right)=\left(P\Delta 0 \\ \right)\left(\begin{array}{c}{Q}^H\\ A_1^H\end{array}\right)=P\Delta Q^H\end{array} \end{gathered}$$

分解方法

1. 求 $A^{H}A$ 的特征值，${\lambda }_1\ge \cdots ,\ge {\lambda }_r>0$

   正奇值为 $\sqrt{{\lambda }_1},\cdots \sqrt{{\lambda }_r}$

2. 求 ${\lambda }_1,\cdots ,{\lambda }_r$ 的正交特征向量（不必单位化）

3. 令列U半阵 $Q=\left(\frac{X_1}{|X_1|},\cdots ,\frac{X_r}{|X_r|}\right),P=\left(\frac{A{X}_1}{|A{X}_1|},\cdots ,\frac{A{X}_r}{|A{X}_r|}\right)$ ，则有正SVD：$A=P\Delta Q^H$

4. 将P，Q扩充为W，V 扩充方法不唯一 由证明可知，不管 $P_1,Q_1$ 为何，都会被消去

   得SVD公式 $A=WD{V}^H$

eg

$$\begin{array}{r}A=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 1& 1\end{array}\right)\end{array}$$

---

$$\begin{array}{rl}& 扩充为两个U阵V=\left(Q,X\right)=\left(\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right),W=\left(P,Y\right)=\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& 0& \frac{1}{\sqrt{2}}\\ 0& 1& 0\\ \frac{1}{\sqrt{2}}& 0& -\frac{1}{\sqrt{2}}\end{array}\right)\\ & 则有奇异分解SVD，A=\left(\begin{array}{ccc}\frac{1}{\sqrt{2}}& 0& \frac{1}{\sqrt{2}}\\ 0& 10& 0\\ \frac{1}{\sqrt{2}}& 0& -\frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{cc}2& 0\\ 0& 0\\ 0& 0\end{array}\right)\left(\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right)\end{array}$$

3.2.3 SVD例题

$$\begin{array}{r}A=\left(\begin{array}{cc}2i& 1\\ 1& i\\ 1& i\end{array}\right)\end{array}$$

---

$$\begin{array}{rl}& A^{H}A=\left(\begin{array}{ccc}-2i& 1& 1\\ 1& -i& -i\end{array}\right)\left(\begin{array}{cc}2i& 1\\ 1& i\\ 1& i\end{array}\right)=\left(\begin{array}{cc}6& 0\\ 0& 3\end{array}\right)，可知正奇值S(A{)}_{+}=\{\sqrt{6},\sqrt{3}\}\\ & {\lambda }_1=6,X_1=\left(\begin{array}{c}1\\ 0\end{array}\right),A{X}_1=\left(\begin{array}{c}2i\\ 1\\ 1\end{array}\right)|A{X}_1|=\sqrt{6};{\lambda }_2=3,X_2=\left(\begin{array}{c}0\\ 1\end{array}\right),A{X}_2=\left(\begin{array}{c}1\\ i\\ i\end{array}\right)\\ & 令Q=\left(\begin{array}{c}{X}_1,X_2\end{array}\right)=\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right),P=\left(\begin{array}{c}\frac{A{X}_1}{|A{X}_1|},\frac{A{X}_2}{|A{X}_2|}\end{array}\right)=\left(\begin{array}{cc}\frac{2i}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}\end{array}\right)\\ & 则有正SVD，A=P\Delta Q^H=\left(\begin{array}{cc}\frac{2i}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}\end{array}\right)\left(\begin{array}{cc}\sqrt{6}& \\ & \sqrt{3}\end{array}\right){\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)}^H\\ & 可知SVD，A=WDV=\left(\begin{array}{ccc}\frac{2i}{\sqrt{6}}& \frac{1}{\sqrt{3}}& 0\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{6}}& \frac{i}{\sqrt{3}}& -\frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{cc}\sqrt{6}& \\ & \sqrt{3}\end{array}\right){\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)}^H\end{array}$$

当A是向量时

$$\begin{array}{rl}& A=\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)\ne 0,则其正SVD\end{array}$$

---

$$\begin{array}{rl}& A^{H}A=\left(\overline{a_1},\cdots ,\overline{a_n}\right)\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)=|a_1{|}^2+\cdots +|a_n{|}^2>0\\ & {\lambda }_1=|a_1{|}^2+\cdots +|a_n{|}^2，令\Delta =\left(\sqrt{{\lambda }_1}\right),Q=(1)\\ & X_1=(1)，P=\frac{1}{\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n{|}^n}}\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right),\\ & 正SVD：A=P\Delta Q^H=\\ & \frac{1}{\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n{|}^n}}\left(\begin{array}{c}{a}_1\\ ⋮\\ a_n\end{array}\right)\left(\sqrt{|a_1{|}^2+|a_2{|}^2+\cdots +|a_n}\right)\left(1\right)\end{array}$$

$A^H$与A的SVD只需求一个

$$若已知正SVD，A=P\Delta Q^H,可得A^H的正SVD=(P\Delta Q^H{)}^H$$

$$\begin{array}{r}A=\left(\begin{array}{ccc}2& 0& 0\\ 3& 0& 0\end{array}\right)与B=A^H的正SVD与SVD\end{array}$$

---

一个SVD解答

$$\begin{array}{r}A=\left(\begin{array}{cc}i& 2\\ 1& i\\ 1& i\end{array}\right)，求正SVD与SVD\end{array}$$

---

$$\begin{array}{rl}& \because A^{H}A=\left(\begin{array}{ccc}-i& 1& 1\\ 2& -i& -i\end{array}\right)\left(\begin{array}{cc}i& 2\\ 1& i\\ 1& i\end{array}\right)=\left(\begin{array}{cc}3& 0\\ 0& 6\end{array}\right)为对角阵，令{\lambda }_1=3,{\lambda }_2=6\\ & A^H有两个特向X_1=\left(\begin{array}{c}1\\ 0\end{array}\right),X_2=\left(\begin{array}{c}0\\ 1\end{array}\right)(互正交)，正奇值为\sqrt{{\lambda }_1}=\sqrt{3},\sqrt{{\lambda }_2}=\sqrt{6}\\ & A{X}_1=\left(\begin{array}{c}i\\ 1\\ 1\end{array}\right),A{X}_2=\left(\begin{array}{c}2\\ i\\ i\end{array}\right),|A{X}_1|=\sqrt{3},|A{X}_2|=\sqrt{6}\\ & 令P=\left(\frac{A{X}_1}{|A{X}_1|},\frac{A{X}_2}{|A{X}_2|}\right)=\left(\begin{array}{cc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}\end{array}\right),则正SVD\\ & A=P\Delta Q^H=\left(\begin{array}{cc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}\end{array}\right)\left(\begin{array}{cc}\sqrt{3}& \\ & \sqrt{6}\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\end{array}$$

---

$$\begin{array}{rl}& 令W=(P,P_1)=\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{1}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{1}{\sqrt{2}}\end{array}\right),或\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{i}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{i}{\sqrt{2}}\end{array}\right),\\ & 有SVD，A=\left(\begin{array}{ccc}\frac{i}{\sqrt{3}}& \frac{2}{\sqrt{6}}& 0\\ \frac{1}{\sqrt{3}}& \frac{i}{\sqrt{6}}& \frac{1}{\sqrt{2}}\\ \frac{i}{\sqrt{3}}& \frac{i}{\sqrt{6}}& -\frac{1}{\sqrt{2}}\end{array}\right)\left(\begin{array}{cc}\sqrt{3}& 0\\ 0& \sqrt{6}\\ 0& 0\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 1\end{array}\right)\end{array}$$

3.2.4 正SVD的等价写法

$$\begin{array}{rl}& A=P\Delta Q^H,\Delta =\left(\begin{array}{ccc}{s}_1& & \\ & \ddots & \\ & & s_r\end{array}\right),令P=(\frac{A}{|A{X}_1|}\\ & \\ & \end{array}$$