4.5 QR分解一：施密特正交化

计算方法

  矩阵的正交三角分解QR decomposition，简称QR分解，是找到两个矩阵Q和R使得$A=QR$,Q是正交矩阵（就是$Q^{T}Q=E$的矩阵，也就是列向量之间互相正交的矩阵），R是对角线上没有0的上三角矩阵。
  QR分解主要有三种方法：

1. 施密特正交化
2. Householder变换
3. Givens旋转

  本文只讲施密特正交化Gram-Schmidt orthogonalization，Gram-Schmit正交分解步骤很简单，是先根据原矩阵获取一组正交基，再求原矩阵在这组正交基下的坐标。以$3\times 3$矩阵$(w_1,w_2,w_3)$为例子。
u 1 = w 1 u 2 = w 2 − ( u 1 , w 2 ) ( u 1 , u 1 ) u 1 u 3 = w 3 − ( u 1 , w 3 ) ( u 1 , u 1 ) u 1 − ( u 2 , w 3 ) ( u 2 , u 2 ) u 2 u i = w i − ∑ j = 1 j < i ( u j , w i ) ( u j , u j ) u j ( w 1 , w 2 , w 3 ) = ( u 1 , u 2 , u 3 ) ( 1 ( u 1 , w 2 ) ( u 1 , u 1 ) ( u 1 , w 3 ) ( u 1 , u 1 ) 0 1 u 2 ⋅ w 3 ( u 2 , u 2 ) 0 0 1 ) Q = ( u 1 , u 2 , u 3 ) R = ( 1 ( u 1 , w 2 ) ( u 1 , u 1 ) ( u 1 , w 3 ) ( u 1 , u 1 ) 0 1 ( u 2 , w 3 ) ( u 2 , u 2 ) 0 0 1 ) \\ u_1=w_1\\ u_2=w_2-\frac{(u_1, w_2)}{(u_1,u_1)} u_1\\ u_3=w_3-\frac{(u_1, w_3)}{(u_1,u_1)} u_1-\frac{(u_2,w_3)}{(u_2,u_2)} u_2\\ u_i=w_i-\sum_{j=1}^{ju1​=w1​u2​=w2​−(u1​,u1​)(u1​,w2​)​u1​u3​=w3​−(u1​,u1​)(u1​,w3​)​u1​−(u2​,u2​)(u2​,w3​)​u2​ui​=wi​−j=1∑j<i​(uj​,uj​)(uj​,wi​)​uj​(w1​,w2​,w3​)=(u1​,u2​,u3​) ​100​(u1​,u1​)(u1​,w2​)​10​(u1​,u1​)(u1​,w3​)​(u2​,u2​)u2​⋅w3​​1​ ​Q=(u1​,u2​,u3​)R= ​100​(u1​,u1​)(u1​,w2​)​10​(u1​,u1​)(u1​,w3​)​(u2​,u2​)(u2​,w3​)​1​ ​
  公式里的$u_1,u_2,\cdots ,u_i$是一组正交基。Gram-Schmit正交分解获得的两个矩阵，右边的是上三角矩阵没错，但一般在上面的步骤做完之后，还需要对得到的Q和R两个矩阵进行单位化。以$3\times 3$矩阵为例子，单位化是做如下处理：
Q = ( u 1 ( u 1 , u 1 ) u 2 ( u 2 , u 2 ) u 3 u 3 ⋅ u 3 ) R = ( ( u 1 , u 1 ) ( u 1 , w 2 ) ( u 1 , u 1 ) ( u 1 , w 3 ) ( u 1 , u 1 ) 0 ( u 2 , u 2 ) ( u 2 , w 3 ) ( u 2 , u 2 ) 0 0 u 3 ⋅ u 3 ) Q=\begin{pmatrix} \frac{u_1}{\sqrt{(u_1,u_1)}} &\frac{ u_2}{\sqrt{(u_2,u_2)}} & \frac{u_3}{\sqrt{u_3\cdot u_3}} \end{pmatrix}\\ R=\begin{pmatrix} \sqrt{(u_1,u_1)} & \frac{(u_1, w_2)}{\sqrt{(u_1,u_1)}}& \frac{(u_1, w_3)}{\sqrt{(u_1,u_1)}} \\ 0 & \sqrt{(u_2,u_2)} & \frac{(u_2,w_3)}{\sqrt{(u_2,u_2)}}\\ 0 & 0 & \sqrt{u_3\cdot u_3} \end{pmatrix} Q=((u1​,u1​) ​u1​​​(u2​,u2​) ​u2​​​u3​⋅u3​ ​u3​​​)R= ​(u1​,u1​) ​00​(u1​,u1​) ​(u1​,w2​)​(u2​,u2​) ​0​(u1​,u1​) ​(u1​,w3​)​(u2​,u2​) ​(u2​,w3​)​u3​⋅u3​ ​​ ​
  以下列矩阵为例子：
( 2 1 − 1 1 2 − 1 − 1 − 1 2 ) Q = ( 2 − 0.667 0.364 1 1.167 0.364 − 1 − 0.167 1.091 ) R = ( 1 0.833 − 0.833 0 1 − 0.455 0 0 1 ) \begin{pmatrix} 2& 1& -1\\ 1& 2& -1\\ -1& -1& 2 \end{pmatrix}\\ Q= \begin{pmatrix}2 & -0.667 & 0.364\\ 1 & 1.167 & 0.364\\ -1 & -0.167 & 1.091\\ \end{pmatrix}\\ R= \begin{pmatrix}1 & 0.833 & -0.833\\ 0 & 1 & -0.455\\ 0 & 0 & 1\\ \end{pmatrix} ​21−1​12−1​−1−12​ ​Q= ​21−1​−0.6671.167−0.167​0.3640.3641.091​ ​R= ​100​0.83310​−0.833−0.4551​ ​
  单位化之后，就是下面的结果：
Q = ( 0.816 − 0.492 0.302 0.408 0.862 0.302 − 0.408 − 0.123 0.905 ) R = ( 2.449 2.041 − 2.041 0.0 1.354 − 0.615 0.0 0.0 1.206 ) Q= \begin{pmatrix}0.816 & -0.492 & 0.302\\ 0.408 & 0.862 & 0.302\\ -0.408 & -0.123 & 0.905\\ \end{pmatrix}\\ R= \begin{pmatrix}2.449 & 2.041 & -2.041\\ 0.0 & 1.354 & -0.615\\ 0.0 & 0.0 & 1.206\\ \end{pmatrix} Q= ​0.8160.408−0.408​−0.4920.862−0.123​0.3020.3020.905​ ​R= ​2.4490.00.0​2.0411.3540.0​−2.041−0.6151.206​ ​

python实现

  实现代码就是严格按照公式：

    def gram_schmidt(self, matrix = None):
        # q矩阵先复制原矩阵
        q = [[i for i in vector] for vector in self.__vectors]
        # 从第二个向量开始做减法
        # q的长度数组
        q_square = [0 for _ in q]
        q_square[0] = inner_product(q[0], q[0], matrix)
        q_length = [0 for _ in q]
        q_length[0] = math.sqrt(q_square[0])
        columns = len(self.__vectors)
        # r 初始化为全部为0
        r = [[0 for _ in vector] for vector in self.__vectors]
        # r对角线为1
        for i, vector in enumerate(r):
            vector[i] = 1
        # i 是列
        for i in range(1, columns):
            q[i] = self.__vectors[i]
            # sum其实是一个向量
            for j in range(0, i):
                frac = inner_product(q[j], self.__vectors[i], matrix) / q_square[j]
                # 把系数存储在r中
                r[i][j] = frac
                q[i] = sub(q[i], mul_num(q[j], frac))
            q_square[i] = inner_product(q[i], q[i], matrix)
            q_length[i] = math.sqrt(q_square[i])
        # Q单位化
        # R乘回来
        for i, vector in enumerate(q):
            for j, e in enumerate(vector):
                vector[j] = e / q_length[i]
                # vector[j] = round(vector[j] * 10000) / 10000.0
        # i 代表向量，也就是列
        for i, vector in enumerate(r):
            # j才代表行
            for j, e in enumerate(vector):
                # 每一行乘以长度
                vector[j] = e * q_length[j]
                # vector[j] = round(vector[j] * 10000) / 10000.0
        return Matrix(q), Matrix(r)

数据测试

  以下矩阵，在标准内积下分解为：
( 2 1 − 1 1 2 − 1 − 1 − 1 2 ) = ( 0.816 − 0.493 0.303 0.408 0.862 0.301 − 0.408 − 0.123 0.906 ) ( 2.449 2.041 − 2.041 0 1.353 − 0.614 0 0 1.204 ) \begin{pmatrix}2 & 1 & -1\\ 1 & 2 & -1\\ -1 & -1 & 2\\ \end{pmatrix}\\= \begin{pmatrix}0.816 & -0.493 & 0.303\\ 0.408 & 0.862 & 0.301\\ -0.408 & -0.123 & 0.906\\ \end{pmatrix} \begin{pmatrix}2.449 & 2.041 & -2.041\\ 0 & 1.353 & -0.614\\ 0 & 0 & 1.204\\ \end{pmatrix} ​21−1​12−1​−1−12​ ​= ​0.8160.408−0.408​−0.4930.862−0.123​0.3030.3010.906​ ​ ​2.44900​2.0411.3530​−2.041−0.6141.204​ ​
  定义内积为：
( x , y ) = x T ( 1 0 0 0 2 0 0 0 3 ) y (x,y)=x^T\begin{pmatrix}1 & 0 & 0\\ 0 & 2 & 0\\ 0 & 0 & 3\\ \end{pmatrix}y (x,y)=xT ​100​020​003​ ​y
  在这个内积下，QR分解为：
( 2 1 − 1 1 2 − 1 − 1 − 1 2 ) = ( 0.667 − 0.577 0.471 0.333 0.577 0.236 − 0.333 0 0.471 ) ( 3 3 − 3.333 0 1.732 − 0.577 0 0 1.887 ) \begin{pmatrix}2 & 1 & -1\\ 1 & 2 & -1\\ -1 & -1 & 2\\ \end{pmatrix} \\=\begin{pmatrix}0.667 & -0.577 & 0.471\\ 0.333 & 0.577 & 0.236\\ -0.333 & 0 & 0.471\\ \end{pmatrix} \begin{pmatrix}3 & 3 & -3.333\\ 0 & 1.732 & -0.577\\ 0 & 0 & 1.887\\ \end{pmatrix} ​21−1​12−1​−1−12​ ​= ​0.6670.333−0.333​−0.5770.5770​0.4710.2360.471​ ​ ​300​31.7320​−3.333−0.5771.887​ ​