稳态视觉诱发电位(SSVEP)识别| Extended Canonical Correlation Analysis, eCCA

前言

如果要说目前SSVEP识别中最流行的几个方法，那么Extended Canonical Correlation Analysis(eCCA)¹绝对是其中之一。目前来看，如果研究SSVEP算法，那么对比算法基本是TRCA或者eCCA^{2 3}。此外，很多在线BCI系统也是采用该方法进行频率识别。与TRCA这种具有自己数学模型的方法不同，eCCA仅是对CCA的一种计算结构调整。我们知道受试者的template中包含着一些潜在可利用的分类信息（相位等），eCCA在CCA基础上考虑到了受试者template项。

算法

我们知道典型相关分析(CCA)是一种降维和评估两组多维数据相关性的方法，它利用典型系数将两矩阵线性组合为一对典型变量，使得两者之间的相关系数最大。可以参考博客1、2。
CCA应用在SSVEP识别上得到了很好的效果，当已有受试者训练数据的时候，我们能够获得三种多通道信号：测试数据$X(t)\in R^{N_c\times N_s\times N_t}$，受试者训练数据的平均${\hat{X}}_k^{N_c\times N_s}$以及构造的正余弦参考信号$Y_{f_k}$，其中$N_c$，$N_s$，$N_t$分别表示通道数、采样点数和测试集trial数。任意两种信号可以根据CCA计算出空间滤波器。
按照排列组合方式来看，如果我们要根据其中两个信号求空间滤波器，那么会有六种空间滤波器形式。六种空间滤波器会产生10个典型变量，然后每两个典型变量之间又可以计算相关系数，那么就有45个相关系数。这个在Mohammad Hadi MehdizavarehI等人2020年发表在PLOS ONE上的文章⁴有详述。但实际应用过程中，我们通常只取了三种空间滤波器形式（这里以Chen X等人PANS上的文章为例，第三项做了修正）分别为

• $W_X(X{\hat{X}}_k)$，$W_{{\hat{X}}_k}(X{\hat{X}}_k)$：测试数据和template之间；
• $W_X(X{Y}_{f_k})$：测试数据和参考信号之间；
• $W_{{\hat{X}}_k}({\hat{X}}_k{Y}_{f_k})$：template和参考信号之间。

由上述空间滤波器，我们可以得到5个相关系数组合（这里的数量在不同的论文中不一致，最初提出eCCA的时候并没有对为什么采用下述相关系数组合做解释。），如下

注：上述公式来自Chen X等人的论文，其中$r_k(4)$所对应的空间滤波器形式有误，应该为$W_{{\hat{X}}_k}({\hat{X}}_k{Y}_{f_k})$。如果仅考虑第一项，那么算法变为标准的CCA；如果对相关系数进行融合，便可以得到k-th刺激下的相关系数。最后通过确认最大相关系数对应的刺激便可以实现分类

整个算法的计算结构为

Matlab代码

CCA

function [output1, output2] = CCA(signal1, signal2)
% Canonical Correlation Analysis, CCA
% Input:
%   signal: #channels, #points
% Output:
% output1: spatial filter of signal1
% output2: spatial filter of signal2
%
X=signal1;
Y=signal2;
T=size(signal2, 2);
%compute covariance matrix 
meanx=mean(X,2);
meany=mean(Y,2);
s11=0;s22=0;s12=0;s21=0;
for i1=1:T
  s11=s11+(X(:,i1)-meanx)*(X(:,i1)-meanx)';
  s22=s22+(Y(:,i1)-meany)*(Y(:,i1)-meany)';
  s12=s12+(X(:,i1)-meanx)*(Y(:,i1)-meany)';
  s21=s21+(Y(:,i1)-meany)*(X(:,i1)-meanx)';
end
s11=s11/(T-1);         
s22=s22/(T-1); 
s12=s12/(T-1); 
s21=s21/(T-1);
%compute eigvalue and eigvector
[eigvectora,eigvaluea]=eig(inv(s11)*s12*inv(s22)*s21);
[eigvectorb,eigvalueb]=eig(inv(s22)*s21*inv(s11)*s12);

evaluea= diag(eigvaluea);
evalueb = diag(eigvalueb);
% correlation coefficient & canonical variates of signal1
[corrcoef1, index]= max(sqrt(evaluea));
output1 = eigvectora(:, index);
% correlation coefficient & canonical variates of signal2
[corrcoef2, index]= max(sqrt(evalueb));
output2 = eigvectorb(:, index);    
end

对上述程序做简要解释

• 上述CCA求解采用的是特征值分解的方法，另外还可以参考SVD分解的做法；
• 通过CCA的推导过程可以看到，corrcoef1和corrcoef2相等，表示两个输入多通道信号的相关系数。在SSVEP中，标准CCA可采用该值直接输出；
• 这个函数文件输出的是两个多通道信号的典型变量，即空间滤波器。

eCCA

% -------------------------------------------------------------------------
% Main for Extended Canonical Correlation Analysis[1]
%
% Dataset (Sx.mat):
%   A 40-target SSVEP dataset recorded from a single subject. The stimuli
%   were generated by the j oint frequency-phase modulation (JFPM)
%     - Stimulus frequencies    : 8.0 - 15.8 Hz with an interval of 0.2 Hz
%     - Stimulus phases         : 0pi, 0.5pi, 1.0pi, and 1.5pi
%     - # of channels           : Oz
%     - # of recording blocks   : 6
%     - Data length of epochs   : 1.5 [seconds]
%     - Sampling rate           : 250 [Hz]
%     - Data format             : # channels, # points, # targets, # blocks
% 
% See also:
%   CCA.m
%
% Reference:
%   [1] Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S (2015)
%       High-speed spelling with a noninvasive brain-computer interface. 
%       PNAS 112:E6058-6067.
% -------------------------------------------------------------------------

clear all
close all

load ('Freq_Phase.mat')
load('subject1.mat')
eeg = subject1;
[N_channel, N_point, N_target, N_block] = size(eeg);
% sample rate
fs = 250;
t=1/fs:1/fs:N_point/fs;   
%% ------------classification-------------
tic
% LOO cross-validation
for loocv_i = 1:N_block
     Testdata = eeg(:, :, :, loocv_i);
     Traindata = eeg;
     Traindata(:, :, :, loocv_i) = [];
     % number of harmonics
     N_harmonic = 2;
    for targ_i = 1:N_target
        % Template
        Template(:, :, targ_i) = mean(squeeze(Traindata(:,:,targ_i,:)),3);
        % Reference
        Y=[];
        for har_i=1:N_harmonic
             Y=cat(1,Y,cat(1, sin(2*pi*freqs(targ_i)*har_i*t), ...
                 cos(2*pi*freqs(targ_i)*har_i*t)));        
        end
        Reference(:, :, targ_i) = Y;
    end
    
    % labels assignment according to testdata
    truelabels=freqs;
    
    N_testTrial=size(Testdata, 3);
    for trial_i=1:N_testTrial
        Allcoefficience = [];
        for targ_j=1:length(freqs)             
            % ρ1 (Filter: Test data & Reference)
            [wn1, wn2]= CCA(Testdata(:,:,trial_i), Reference(:, :, targ_j));
            weighted_train = wn2'*Reference(:,:,targ_j);
            weighted_test = wn1'*Testdata(:,:,trial_i);
            coefficienceMatrix = corrcoef(weighted_test,weighted_train);
            coefficience(1) = abs(coefficienceMatrix(1,2));
            % ρ2 (Filter: Test data & Template)
            [wn, ~] = CCA(Testdata(:,:,trial_i), Template(:, :, targ_j));
            weighted_train = wn'*Template(:,:,targ_j);
            weighted_test = wn'*Testdata(:,:,trial_i);
            coefficienceMatrix = corrcoef(weighted_test,weighted_train);
            coefficience(2) = coefficienceMatrix(1,2);
             % ρ3 (Filter: Test data & Reference)
            [wn, ~] = CCA(Testdata(:,:,trial_i), Reference(:, :, targ_j));
            weighted_train = wn'*Template(:,:,targ_j);
            weighted_test = wn'*Testdata(:,:,trial_i);
            coefficienceMatrix = corrcoef(weighted_test,weighted_train);
            coefficience(3) = coefficienceMatrix(1,2);
            % ρ4 (Filter: Template & Reference)
            [wn, ~] = CCA(Template(:, :, targ_j), Reference(:, :, targ_j));
            weighted_train = wn'*Template(:,:,targ_j);
            weighted_test = wn'*Testdata(:,:,trial_i);
            coefficienceMatrix = corrcoef(weighted_test,weighted_train);
            coefficience(4) = coefficienceMatrix(1,2);
%              % ρ5 (Filter: Test data & Template)
%             [wn1, wn2] = CCA(Testdata(:,:,trial_i), Template(:, :, targ_j));
%             weighted_train = wn2'*Template(:,:,targ_j);
%             weighted_test = wn1'*Template(:,:,targ_j);
%             coefficienceMatrix = corrcoef(weighted_test,weighted_train);
%             coefficience(5) = coefficienceMatrix(1,2);
            % compute correlation values
            Allcoefficience(targ_j) = sum(sign(coefficience).*coefficience.^2);
        end % end targ_i
            % target detection
            [~, index] = max(Allcoefficience);
            outputlabels(trial_i) = freqs(index);
            
    end % end trial_i
    trueNum = sum((outputlabels-truelabels)==0);
    acc(loocv_i) = trueNum/length(truelabels);
    fprintf('The %d-th CV accuracy is: %.4f, samples: %d/%d\n',loocv_i,...
        acc(loocv_i),trueNum, N_testTrial)
end % end looCv_i
t=toc;
% data visualization
fprintf('\n-----------------------------------------\n')
disp(['total time: ',num2str(t),' s']);
fprintf('6-fold CV average accuracy is: %.4f\n',mean(acc))

对上述程序做简要解释

• 采用的数据为清华benchmark dataset 受试者1，数据已经经过滤波等预处理，数据长度为1.5s；
• 实验中发现第五项的加入使得正确率降低，故只采用了四项的融合，和万峰老师论文保持一致² ；
• 完整代码见我的仓库。

前四项相关系数融合运行结果为

The 1-th CV accuracy is: 0.9250, samples: 37/40
The 2-th CV accuracy is: 0.9750, samples: 39/40
The 3-th CV accuracy is: 0.9750, samples: 39/40
The 4-th CV accuracy is: 0.9750, samples: 39/40
The 5-th CV accuracy is: 0.9750, samples: 39/40
The 6-th CV accuracy is: 0.9750, samples: 39/40

-----------------------------------------
total time: 175.7503 s
6-fold CV average accuracy is: 0.9667

五项相关系数融合运行结果为

The 1-th CV accuracy is: 0.5750, samples: 23/40
The 2-th CV accuracy is: 0.5750, samples: 23/40
The 3-th CV accuracy is: 0.7750, samples: 31/40
The 4-th CV accuracy is: 0.4750, samples: 19/40
The 5-th CV accuracy is: 0.6000, samples: 24/40
The 6-th CV accuracy is: 0.6500, samples: 26/40

-----------------------------------------
total time: 216.9206 s
6-fold CV average accuracy is: 0.6083

关于取哪些相关系数进行融合的问题，本文参考了四篇较新的文献，其中采用5个相关系数融合是常用的选择（都是清华的文章…）。但是在本文实验后发现，第五项加入使正确率下降了…所以采用了万峰老师的做法，只取了前四项，如果有问题希望各位能及时提出来。

References

1. Chen X, Wang Y, Nakanishi M, Gao X, Jung TP, Gao S (2015) High-speed spelling with a noninvasive brain-computer interface. PNAS 112:E6058-6067.
2. Wong CM, Wan F, Wang B, Wang Z, Nan W, Lao KF, Mak PU, Vai MI, Rosa A (2020) Learning across multi-stimulus enhances target recognition methods in SSVEP-based BCIs. Journal of neural engineering 17:016026.
3. Xu M, Han J, Wang Y, Jung TP, Ming D (2020) Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features. IEEE Transactions on Biomedical Engineering.
4. Mehdizavareh MH, Hemati S, Soltanian-Zadeh H (2020) Enhancing performance of subject-specific models via subject-independent information for SSVEP-based BCIs. PloS one 15:e0226048.