ADMM算法求解二次项目标函数+l1正则项问题

ADMM算法求解二次项目标函数+l1正则项问题

问题描述

min ⁡ x f ( x ) + λ ∣ ∣ x ∣ ∣ 1 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ( 1 ) \min_x f(x)+\lambda||x||_1······(1) xminf(x)+λx11
其中,f(x)为二次项函数,λ>0,x是 R n R^n Rn上的列向量。

分析

由于(1)带l1范数不可导,考虑将(1)转化为以下问题(2):

min ⁡ f ( x ) + g ( z ) x − z = 0 \min f(x)+g(z)\\ x-z=0 minf(x)+g(z)xz=0
其中,
g ( z ) = λ ∣ ∣ z ∣ ∣ 1 g(z)=\lambda||z||_1 g(z)=λz1
则增广拉格朗日函数为
L ( x , z , y ) = f ( x ) + g ( z ) + y T ( x − z ) + n 2 ∣ ∣ x − z ∣ ∣ 2 2 L(x,z,y)=f(x)+g(z)+y^{T}(x-z)+\frac{n}{2}||x-z||_{2}^{2} L(x,z,y)=f(x)+g(z)+yT(xz)+2nxz22
迭代算法为
x k + 1 : = arg min ⁡ x L ( x , z k , y k ) z k + 1 : = arg min ⁡ z L ( x k + 1 , z , y k ) y k + 1 : = y k + n ( x k + 1 − z k + 1 ) x^{k+1}:=\argmin_xL(x,z^k,y^k)\\z^{k+1}:=\argmin_zL(x^{k+1},z,y^k)\\y^{k+1}:=y^k+n(x^{k+1}-z^{k+1}) xk+1:=xargminL(x,zk,yk)zk+1:=zargminL(xk+1,z,yk)yk+1:=yk+n(xk+1zk+1)

简化

r = x − z r=x-z r=xz,则
y T ( x − z ) + n 2 ∣ ∣ x − z ∣ ∣ 2 2 = y T r + n 2 ∣ ∣ r ∣ ∣ 2 2 = n 2 ∣ ∣ r + 1 n y ∣ ∣ 2 2 − n 2 ∣ ∣ 1 n y ∣ ∣ 2 2 = n 2 ∣ ∣ x − z + u ∣ ∣ 2 2 − n 2 ∣ ∣ u ∣ ∣ 2 2 ( u = 1 n y ) y^T(x-z)+\frac{n}{2}||x-z||_2^2=y^Tr+\frac{n}{2}||r||_2^2\\=\frac{n}{2}||r+\frac{1}{n}y||_2^2-\frac{n}{2}||\frac{1}{n}y||_2^2\\=\frac{n}{2}||x-z+u||_2^2-\frac{n}{2}||u||^2_2\\(u=\frac{1}{n}y) yT(xz)+2nxz22=yTr+2nr22=2nr+n1y222nn1y22=2nxz+u222nu22(u=n1y
迭代算法可转化为
x k + 1 : = arg min ⁡ x f ( x ) + n 2 ∣ ∣ x − z k + u k ∣ ∣ 2 2 z k + 1 : = arg min ⁡ z g ( z ) + n 2 ∣ ∣ x k + 1 − z + u k ∣ ∣ 2 2 u k + 1 : = u k + x k + 1 − z k + 1 x^{k+1}:=\argmin_xf(x)+\frac{n}{2}||x-z^k+u^k||^2_2\\z^{k+1}:=\argmin_zg(z)+\frac{n}{2}||x^{k+1}-z+u^k||^2_2\\u^{k+1}:=u^k+x^{k+1}-z^{k+1} xk+1:=xargminf(x)+2nxzk+uk22zk+1:=zargming(z)+2nxk+1z+uk22uk+1:=uk+xk+1zk+1

问题的求解

1.x-update的求取

对于x-update,由于f(x)是二次项目标函数,不妨假设
f ( x ) = n 2 ∣ ∣ A x − b ∣ ∣ 2 2 f(x)=\frac{n}{2}||Ax-b||^2_2 f(x)=2nAxb22A为n阶方阵,b是与x同规模的矩阵。
那么x-update的求解即寻求超定线性方程组
A x − b = 0 x − z k + u k = 0 Ax-b=0\\x-z^k+u^k=0 Axb=0xzk+uk=0的最小二乘解。此外,还可以通过以下方法求取x-update
f ( x ) + n 2 ∣ ∣ x − z k + u k ∣ ∣ 2 2 = n 2 ( ∣ ∣ A x − b ∣ ∣ 2 2 + ∣ ∣ x − z k + u k ∣ ∣ 2 2 ) f(x)+\frac{n}{2}||x-z^k+u^k||^2_2=\frac{n}{2}(||Ax-b||^2_2+||x-z^k+u^k||^2_2) f(x)+2nxzk+uk22=2n(Axb22+xzk+uk22)
对x求导,并令导数为0,有
A T A x − A T b + x − z k + u k = ( A T A + I ) x − ( A T b + z k − u k ) = 0 A^TAx-A^Tb+x-z^k+u^k=(A^TA+I)x-(A^Tb+z^k-u^k)=0 ATAxATb+xzk+uk=(ATA+I)x(ATb+zkuk)=0
( A T A + I ) x = A T b + z k − u k (A^TA+I)x=A^Tb+z^k-u^k ATA+I)x=ATb+zkuk

2.z-update的求取

对于z-update的求取,
g ( z ) + n 2 ∣ ∣ x k + 1 − z + u k ∣ ∣ 2 2 = Σ i = 1 n ( λ ∣ z i ∣ + n 2 ( x k + 1 − z + u k ) 2 ) g(z)+\frac{n}{2}||x^{k+1}-z+u^k||_2^2=\Sigma_{i=1}^{n}(\lambda|z_i|+\frac{n}{2}(x^{k+1}-z+u^k)^2) g(z)+2nxk+1z+uk22=Σi=1nλzi+2n(xk+1z+uk)2)
对于第i个分量
x i k + 1 + u i k > λ n x^{k+1}_i+u^k_i>\frac{\lambda}{n} xik+1+uik>nλ时, z i k + 1 = x i k + 1 + u i k − λ n z^{k+1}_i=x^{k+1}_i+u^k_i-\frac{\lambda}{n} zik+1=xik+1+uiknλ
x i k + 1 + u i k < − λ n x^{k+1}_i+u^k_i<-\frac{\lambda}{n} xik+1+uik<nλ时, z i k + 1 = x i k + 1 + u i k + λ n z^{k+1}_i=x^{k+1}_i+u^k_i+\frac{\lambda}{n} zik+1=xik+1+uik+nλ
其他, z i k + 1 = 0. z^{k+1}_i=0. zik+1=0.

3.u-update的求取

u k + 1 = u k + x k + 1 − z k + 1 u^{k+1}=u^k+x^{k+1}-z^{k+1} uk+1=uk+xk+1zk+1

你可能感兴趣的:(算法)