图Lasso求逆协方差矩阵(Graphical Lasso for inverse covariance matrix)

图Lasso求逆协方差矩阵(Graphical Lasso for inverse covariance matrix)

作者：凯鲁嘎吉 - 博客园 http://www.cnblogs.com/kailugaji/

1. 图Lasso方法的基本理论

2. 坐标下降算法

3. 图Lasso算法

4. MATLAB程序

    数据见参考文献[2]

4.1 demo.m

load SP500
data = normlization(data);
S = cov(data);  %样本协方差
[X, W] = glasso_1(double(S), 0.5);
%X:sigma^(-1), W:sigma
[~, idx] = sort(info(:,3));
colormap gray
imagesc(X(idx, idx) == 0)
axis off

%% Data Normalization
function data = normlization(data)
data = bsxfun(@minus, data, mean(data));
data = bsxfun(@rdivide, data, std(data));
end

4.2 glasso_1.m

function [X, W] = glasso_1(S, lambda)
%% Graphical Lasso - Friedman et. al, Biostatistics, 2008
% Input:
%   S - 样本的协方差矩阵
%   lambda - 罚参数
% Output:
%   X - 精度矩阵    sigma^(-1)
%   W - 协方差矩阵   sigma
%%
p = size(S,1);  %数据维度
W = S + lambda * eye(p);  %W=S+λI
beta = zeros(p) - lambda * eye(p);   %β=-λI
eps = 1e-4;
finished = false(p);   %finished:p*p的逻辑0矩阵
while true
    for j = 1 : p
        idx = 1 : p; idx(j) = [];
        beta(idx, j) = lasso(W(idx, idx), S(idx, j), lambda, beta(idx, j));
        W(idx, j) = W(idx,idx) * beta(idx, j);  %W=W*β
        W(j, idx) = W(idx, j);
    end
    index = (beta == 0);
    finished(index) = (abs(W(index) - S(index)) <= lambda);
    finished(~index) = (abs(W(~index) -S(~index) + lambda * sign(beta(~index))) < eps);
    if finished
        break;
    end
end
X = zeros(p);
for j = 1 : p
    idx = 1 : p; idx(j) = [];
    X(j,j) = 1 / (W(j,j) - dot(W(idx,j), beta(idx,j)));
    X(idx, j) = -1 * X(j, j) * beta(idx,j);
end
% X = sparse(X);
end

4.3 lasso.m

function w = lasso(A, b, lambda, w)
% Lasso 
p = size(A,1);
df = A * w - b;
eps = 1e-4;
finished = false(1, p);
while true
    for j = 1 : p
        wtmp = w(j);
        w(j) = soft(wtmp - df(j) / A(j,j), lambda / A(j,j));
        if w(j) ~= wtmp
            df = df + (w(j) - wtmp) * A(:, j); % update df
        end
    end
    index = (w == 0);
    finished(index) = (abs(df(index)) <= lambda);
    finished(~index) = (abs(df(~index) + lambda * sign(w(~index))) < eps);
    if finished
        break;
    end
end
end
%% Soft thresholding
function x = soft(x, lambda)
x = sign(x) * max(0, abs(x) - lambda);
end

4.4 结果

注意：罚参数lamda的设定对逆协方差的稀疏性的影响很大，可以用交叉验证方式得到。

5. 参考文献

[1] 林祝莹. 图Lasso及相关方法的研究与应用[D].燕山大学,2016.

[2] Graphical Lasso for sparse inverse covariance selection