图Lasso求逆协方差矩阵(Graphical Lasso for inverse covariance matrix)
作者：凯鲁嘎吉 - 博客园 http://www.cnblogs.com/kailugaji/
1. 图Lasso方法的基本理论
2. 坐标下降算法
3. 图Lasso算法
4. MATLAB程序
数据见参考文献[2]
4.1 demo.m
load SP500 data = normlization(data); S = cov(data); %样本协方差 [X, W] = glasso_1(double(S), 0.5); %X:sigma^(-1), W:sigma [~, idx] = sort(info(:,3)); colormap gray imagesc(X(idx, idx) == 0) axis off %% Data Normalization function data = normlization(data) data = bsxfun(@minus, data, mean(data)); data = bsxfun(@rdivide, data, std(data)); end
4.2 glasso_1.m
function [X, W] = glasso_1(S, lambda)
%% Graphical Lasso - Friedman et. al, Biostatistics, 2008
% Input:
% S - 样本的协方差矩阵
% lambda - 罚参数
% Output:
% X - 精度矩阵 sigma^(-1)
% W - 协方差矩阵 sigma
%%
p = size(S,1); %数据维度
W = S + lambda * eye(p); %W=S+λI
beta = zeros(p) - lambda * eye(p); %β=-λI
eps = 1e-4;
finished = false(p); %finished:p*p的逻辑0矩阵
while true
for j = 1 : p
idx = 1 : p; idx(j) = [];
beta(idx, j) = lasso(W(idx, idx), S(idx, j), lambda, beta(idx, j));
W(idx, j) = W(idx,idx) * beta(idx, j); %W=W*β
W(j, idx) = W(idx, j);
end
index = (beta == 0);
finished(index) = (abs(W(index) - S(index)) <= lambda);
finished(~index) = (abs(W(~index) -S(~index) + lambda * sign(beta(~index))) < eps);
if finished
break;
end
end
X = zeros(p);
for j = 1 : p
idx = 1 : p; idx(j) = [];
X(j,j) = 1 / (W(j,j) - dot(W(idx,j), beta(idx,j)));
X(idx, j) = -1 * X(j, j) * beta(idx,j);
end
% X = sparse(X);
end
4.3 lasso.m
function w = lasso(A, b, lambda, w)
% Lasso
p = size(A,1);
df = A * w - b;
eps = 1e-4;
finished = false(1, p);
while true
for j = 1 : p
wtmp = w(j);
w(j) = soft(wtmp - df(j) / A(j,j), lambda / A(j,j));
if w(j) ~= wtmp
df = df + (w(j) - wtmp) * A(:, j); % update df
end
end
index = (w == 0);
finished(index) = (abs(df(index)) <= lambda);
finished(~index) = (abs(df(~index) + lambda * sign(w(~index))) < eps);
if finished
break;
end
end
end
%% Soft thresholding
function x = soft(x, lambda)
x = sign(x) * max(0, abs(x) - lambda);
end
4.4 结果
注意：罚参数lamda的设定对逆协方差的稀疏性的影响很大，可以用交叉验证方式得到。
5. 参考文献
[1] 林祝莹. 图Lasso及相关方法的研究与应用[D].燕山大学,2016.
[2] Graphical Lasso for sparse inverse covariance selection