矩阵二范数的求导问题

现有目标函数:
f ( x ) = 1 2 ∥ A x − b ∥ 2 2 f(x)=\frac{1}{2} \parallel Ax-b \parallel_2^2 f(x)=21Axb22
其中 A ∈ R N × m A\in \mathbb{R}^{N \times m} ARN×m, x ∈ R m x \in \mathbb{R}^m xRm, b ∈ R n b\in \mathbb{R}^n bRn
则求 f ′ ( x ) f^{'}(x) f(x)的推导如下:
f ( x ) = 1 2 ∥ A x − b ∥ 2 2 = 1 2 ( A x − b ) T ( A x − b ) = 1 2 ( x T A T − b T ) ( A x − b ) = 1 2 ( x T A T A x − x T A T b − b T A x + b T b ) \begin {aligned} f(x)&=\frac{1}{2} \parallel Ax-b \parallel_2^2 \\ &=\frac{1}{2}(Ax-b)^T(Ax-b)\\ &=\frac{1}{2}(x^TA^T-b^T)(Ax-b)\\ &=\frac{1}{2}(x^TA^TAx-x^TA^Tb-b^TAx+b^Tb) \end {aligned} f(x)=21Axb22=21(Axb)T(Axb)=21(xTATbT)(Axb)=21(xTATAxxTATbbTAx+bTb)
利用如下性质:
∂   x T D x ∂ x = ( D + D T ) x ∂ D T x ∂ x = D ∂ x T D ∂ x = D \begin {aligned} \frac{\partial \ x^TDx}{\partial x}&=(D+D^T)x \\ \frac{\partial D^Tx}{\partial x}&=D\\ \frac{\partial x^TD}{\partial x}&=D \end {aligned} x xTDxxDTxxxTD=(D+DT)x=D=D
则有
∂ f ( x ) ∂ x = 1 2 { [ A T A + ( A T A ) T ] x − A T b − A T b } = 1 2 { 2 A T A x − 2 A T b } = A T A x − 2 A T b = A T ( A x − b ) \begin {aligned} \frac{\partial f(x)}{\partial x}&=\frac{1}{2} \{[A^TA+(A^TA)^T]x-A^Tb-A^Tb \} \\ &=\frac{1}{2} \{ 2A^TAx-2A^Tb \}\\ &=A^TAx-2A^Tb\\ &=A^T(Ax-b) \end {aligned} xf(x)=21{[ATA+(ATA)T]xATbATb}=21{2ATAx2ATb}=ATAx2ATb=AT(Axb)


∂ f ( x ) ∂ x = A T ( A x − b ) \frac{\partial f(x)}{\partial x}=A^T(Ax-b) xf(x)=AT(Axb)

你可能感兴趣的:(sparse)