线性规划求解——交替方向乘子法（ADMM）

原问题

$$\begin{gathered} \underset{x}{\min }{c}^{T}x \\ s.t.Ax=b \\ x\ge 0\text{\hspace{1em}(1)} \end{gathered}$$

对偶问题

$$\begin{gathered} \underset{y}{\max }{b}^{T}y \\ s.t.A^{T}y+s=c \\ s\ge 0\text{\hspace{1em}(2)} \end{gathered}$$
$A\in {\mathbb{R}}^{m\times n},x\in {\mathbb{R}}^n,s\in {\mathbb{R}}^n,y\in {\mathbb{R}}^m$

---

本文用交替方向乘子法（Alternating Direction Method of Multipilers）实现线性规划问题的求解器。

首先，写出对偶问题的增广拉格朗日函数，$$\begin{gathered} L_t(x,y,s)=-b^{T}y+x^T(A^{T}y+s-c)+\frac{t}{2}||A^{T}y+s-c|{|}_2^2 \\ s.t.s\ge 0 \end{gathered}$$
基于ADMM的迭代规则：$$\begin{gathered} y^{k+1}=arg\underset{y}{\min }{L}_t(y;x^k,s^k) \\ {s}^{k+1}=arg\underset{s}{\min }{L}_t(s;x^k,y^{k+1}),s\ge 0 \\ {x}^{k+1}=x^k+t\ast (A^T{y}^{k+1}+s^{k+1}-c) \end{gathered}$$直到结果收敛。针对该问题，其增广拉格朗日函数非常简单，可以写出迭代步的显式形式：$$\begin{gathered} y^{k+1}=arg\underset{y}{\min }{L}_t(y;x^k,s^k) \\ =-(A{A}^T{)}^{-1}((A{x}^k-b)/t+A(s^k-c)) \end{gathered}$$
$$\begin{gathered} s^{k+1}=arg\underset{s}{\min }{L}_t(s;x^k,y^{k+1}),s\ge 0 \\ =max(c-A^T{y}^{k+1}-\frac{x^k}{t},0) \end{gathered}$$
$$x^{k+1}=x^k+t\ast (A^T{y}^{k+1}+s^{k+1}-c)$$

代码如下：

function [x] = LP_ADMM_dual(c, A, b, opts, x0)

m = size(A,1);
n = size(A,2);

% 随机初始化
y = randn(m,1);
x = x0; 
s = randn(n,1);

t = 0.1;  % ALM rate

S = A*A';
U = chol(S);
L = U'; %cholesky decomposition: S = L*U = U'*U

err = 1;
x_old = x;
while(err > 1e-6)
    
    y = -U\(L\((A*x-b)/t+A*(s-c)));  % 固定 x,s, 更新 y
    
    s = max(c-A'*y-x/t,0); % 固定 y,x, 更新 s
    
    x = x + (A'*y+s-c)*t;  % 固定 y,s, 更新 x
    
    err = norm(x-x_old);
    x_old = x;
end

end

测试：

% 生成数据
n = 100;
m = 20;
A = rand(m,n);
xs = full(abs(sprandn(n,1,m/n)));
b = A*xs;
y = randn(m,1);
s = rand(n,1).*(xs==0);
c = A'*y + s;

% 计算误差
errfun = @(x1, x2) norm(x1-x2)/(1+norm(x1));

% 标准答案
figure(1);
subplot(2,1,1);
stem(xs,'fill','k-.')
title('exact solu');

% ADMM 求解
opts = [];
tic;
[x1, out] = LP_ADMM_dual(c, A, b, opts, x0);
t1 = toc;
subplot(2,1,2);
stem(x1,'fill','k-.');
title('lp_admm_dual');

fprintf('lp-alm-dual: cpu: %5.2f, err-to-exact: %3.2e\n', t2, errfun(x1, xs));

lp-admm-dual: cpu:  0.08, err-to-exact: 1.17e-04

又快又准！！！