整理一些网络上的PCA算法，供大家参考
最近一段时间在做pca降维，就把网络上的资源看了不少，这里做个总结。
http://blog.csdn.net/weixingstudio/article/details/8234766
上面的连接中有我对pca的一个详细的介绍。这个文档里只给出一些其他的自己实现的pca程序。
上面的文档有正确运行pca降维的matlab程序，大家可以参考上面文档里面的。
这个文档里面的程序仅供大家学习研究用。。
一下内容，都来自网络资源。
算法一
这个算法注释很清楚，但是因为我没有具体的学习pca的原理，所以里面的白化矩阵我不知道做什么用的，还望大家多多指教
%程序说明：y = pca(mixedsig)，程序中mixedsig为 n*T 阶混合数据矩阵，n为信号个数，T为采样点数
% y为 m*T 阶主分量矩阵。
function y = pca(mixedsig)
if nargin == 0
error('You must supply the mixed data as input argument.');
end
if length(size(mixedsig))>2
error('Input data can not have more than two dimensions. ');
end
if any(any(isnan(mixedsig)))
error('Input data contains NaN''s.');
end
%——————————————去均值————————————
meanValue = mean(mixedsig')';
mixedsig = mixedsig - meanValue * ones(1,size(meanValue,2));
[Dim,NumofSampl] = size(mixedsig);
oldDimension = Dim;
fprintf('Number of signals: %d\n',Dim);
fprintf('Number of samples: %d\n',NumofSampl);
fprintf('Calculate PCA...');
firstEig = 1;
lastEig = Dim;
covarianceMatrix = cov(mixedsig',1); %计算协方差矩阵
[E,D] = eig(covarianceMatrix); %计算协方差矩阵的特征值和特征向量
%———计算协方差矩阵的特征值大于阈值的个数lastEig———
rankTolerance = 1e-5;
maxLastEig = sum(diag(D)) > rankTolerance;
lastEig = maxLastEig;
%——————————降序排列特征值——————————
eigenvalues = flipud(sort(diag(D)));
%—————————去掉较小的特征值——————————
if lastEig < oldDimension
lowerLimitValue = (eigenvalues(lastEig) + eigenvalues(lastEig + 1))/2;
else
lowerLimitValue = eigenvalues(oldDimension) - 1;
end
lowerColumns = diag(D) > lowerLimitValue;
%—————去掉较大的特征值(一般没有这一步)——————
if firstEig > 1
higherLimitValue = (eigenvalues(firstEig - 1) + eigenvalues(firstEig))/2;
else
higherLimitValue = eigenvalues(1) + 1;
end
higherColumns = diag(D) < higherLimitValue;
%—————————合并选择的特征值——————————
selectedColumns =lowerColumns & higherColumns;
%—————————输出处理的结果信息—————————
fprintf('Selected[ %d ] dimensions.\n',sum(selectedColumns));
fprintf('Smallest remaining (non-zero) eigenvalue[ %g ]\n',eigenvalues(lastEig));
fprintf('Largest remaining (non-zero) eigenvalue[ %g ]\n',eigenvalues(firstEig));
fprintf('Sum of removed eigenvalue[ %g ]\n',sum(diag(D) .* (~selectedColumns)));
%———————选择相应的特征值和特征向量———————
E = selcol(E,selectedColumns);
D = selcol(selcol(D,selectedColumns)',selectedColumns);
%——————————计算白化矩阵———————————
whiteningMatrix = inv(sqrt(D)) * E';
dewhiteningMatrix = E * sqrt(D);
%——————————提取主分量————————————
y = whiteningMatrix * mixedsig;
%——————————行选择子程序———————————
function newMatrix = selcol(oldMatrix,maskVector)
if size(maskVector,1)~ = size(oldMatrix,2)
error('The mask vector and matrix are of uncompatible size.');
end
numTaken = 0;
for i = 1:size(maskVector,1)
if maskVector(i,1) == 1
takingMask(1,numTaken + 1) == i;
numTaken = numTaken + 1;
end
end
newMatrix = oldMatrix(:,takingMask);算法二
这个算法是国外比较详细的算法，还有其他的文件，程序的原名叫：'Eigenface' Face Recognition System
function [m, A, Eigenfaces] = EigenfaceCore(T)
% Use Principle Component Analysis (PCA) to determine the most
% discriminating features between images of faces.
%
% Description: This function gets a 2D matrix, containing all training image vectors
% and returns 3 outputs which are extracted from training database.
%
% Argument: T - A 2D matrix, containing all 1D image vectors.
% Suppose all P images in the training database
% have the same size of MxN. So the length of 1D
% column vectors is M*N and 'T' will be a MNxP 2D matrix.
%
% Returns: m - (M*Nx1) Mean of the training database
% Eigenfaces - (M*Nx(P-1)) Eigen vectors of the covariance matrix of the training database
% A - (M*NxP) Matrix of centered image vectors
%
% See also: EIG
% Original version by Amir Hossein Omidvarnia, October 2007
% Email: [email protected]
%%%%%%%%%%%%%%%%%%%%%%%% Calculating the mean image
m = mean(T,2); % Computing the average face image m = (1/P)*sum(Tj's) (j = 1 : P)
Train_Number = size(T,2);
%%%%%%%%%%%%%%%%%%%%%%%% Calculating the deviation of each image from mean image
A = [];
for i = 1 : Train_Number
temp = double(T(:,i)) - m; % Computing the difference image for each image in the training set Ai = Ti - m
A = [A temp]; % Merging all centered images
end
%%%%%%%%%%%%%%%%%%%%%%%% Snapshot method of Eigenface methos
% We know from linear algebra theory that for a PxQ matrix, the maximum
% number of non-zero eigenvalues that the matrix can have is min(P-1,Q-1).
% Since the number of training images (P) is usually less than the number
% of pixels (M*N), the most non-zero eigenvalues that can be found are equal
% to P-1. So we can calculate eigenvalues of A'*A (a PxP matrix) instead of
% A*A' (a M*NxM*N matrix). It is clear that the dimensions of A*A' is much
% larger that A'*A. So the dimensionality will decrease.
L = A'*A; % L is the surrogate of covariance matrix C=A*A'.
[V D] = eig(L); % Diagonal elements of D are the eigenvalues for both L=A'*A and C=A*A'.
%%%%%%%%%%%%%%%%%%%%%%%% Sorting and eliminating eigenvalues
% All eigenvalues of matrix L are sorted and those who are less than a
% specified threshold, are eliminated. So the number of non-zero
% eigenvectors may be less than (P-1).
L_eig_vec = [];
for i = 1 : size(V,2)
if( D(i,i)>1 )
L_eig_vec = [L_eig_vec V(:,i)];
end
end
%%%%%%%%%%%%%%%%%%%%%%%% Calculating the eigenvectors of covariance matrix 'C'
% Eigenvectors of covariance matrix C (or so-called "Eigenfaces")
% can be recovered from L's eiegnvectors.
Eigenfaces = A * L_eig_vec; % A: centered image vectors'Eigenface' Face Recognition System
Written by: Amir Hossein Omidvarnia
Email: [email protected]
This package implements a well-known PCA-based face recognition
method, which is called 'Eigenface' [1].
All functions are easy to use, as they are heavy commented.
Furthermore, a sample script is included to show their usage.
In general, you should follow this order:
1. Select training and test database paths.
2. Select path of the test image.
3. Run 'CreateDatabase' function to create 2D matrix of all training images.
4. Run 'EigenfaceCore' function to produce basis's of facespace.
5. Run 'Recognition' function to get the name of equivalent image in training database.
For your convenience, I have prepared sample training and test databases, which are parts
of 'face94' Essex face database [2]. You just need to copy the above functions, along with
the training and test databases into a specified path (for example 'work' path of your
MATLAB root). Then follow dialog boxes, which will appear upon running 'example.m'.
Enjoy it!
References:
[1] P. N. Belhumeur, J. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition
using class specific linear projection. In ECCV (1), pages 45--58, 1996.
[2] Available at:
http://cswww.essex.ac.uk/mv/allfaces/faces94.zip为了方便理解，我把其他的几段程序也复制上来：
function T = CreateDatabase(TrainDatabasePath)
% Align a set of face images (the training set T1, T2, ... , TM )
%
% Description: This function reshapes all 2D images of the training database
% into 1D column vectors. Then, it puts these 1D column vectors in a row to
% construct 2D matrix 'T'.
%
%
% Argument: TrainDatabasePath - Path of the training database
%
% Returns: T - A 2D matrix, containing all 1D image vectors.
% Suppose all P images in the training database
% have the same size of MxN. So the length of 1D
% column vectors is MN and 'T' will be a MNxP 2D matrix.
%
% See also: STRCMP, STRCAT, RESHAPE
% Original version by Amir Hossein Omidvarnia, October 2007
% Email: [email protected]
%%%%%%%%%%%%%%%%%%%%%%%% File management
TrainFiles = dir(TrainDatabasePath);
Train_Number = 0;
for i = 1:size(TrainFiles,1)
if not(strcmp(TrainFiles(i).name,'.')|strcmp(TrainFiles(i).name,'..')|strcmp(TrainFiles(i).name,'Thumbs.db'))
Train_Number = Train_Number + 1; % Number of all images in the training database
end
end
%%%%%%%%%%%%%%%%%%%%%%%% Construction of 2D matrix from 1D image vectors
T = [];
for i = 1 : Train_Number
% I have chosen the name of each image in databases as a corresponding
% number. However, it is not mandatory!
str = int2str(i);
str = strcat('\',str,'.jpg');
str = strcat(TrainDatabasePath,str);
img = imread(str);
img = rgb2gray(img);
[irow icol] = size(img);
temp = reshape(img',irow*icol,1); % Reshaping 2D images into 1D image vectors
T = [T temp]; % 'T' grows after each turn 行扩展
endfunction OutputName = Recognition(TestImage, m, A, Eigenfaces)
% Recognizing step....
%
% Description: This function compares two faces by projecting the images into facespace and
% measuring the Euclidean distance between them.
%
% Argument: TestImage - Path of the input test image
%
% m - (M*Nx1) Mean of the training
% database, which is output of 'EigenfaceCore' function.
%
% Eigenfaces - (M*Nx(P-1)) Eigen vectors of the
% covariance matrix of the training
% database, which is output of 'EigenfaceCore' function.
%
% A - (M*NxP) Matrix of centered image
% vectors, which is output of 'EigenfaceCore' function.
%
% Returns: OutputName - Name of the recognized image in the training database.
%
% See also: RESHAPE, STRCAT
% Original version by Amir Hossein Omidvarnia, October 2007
% Email: [email protected]
%%%%%%%%%%%%%%%%%%%%%%%% Projecting centered image vectors into facespace
% All centered images are projected into facespace by multiplying in
% Eigenface basis's. Projected vector of each face will be its corresponding
% feature vector.
ProjectedImages = [];
Train_Number = size(Eigenfaces,2);
for i = 1 : Train_Number
temp = Eigenfaces'*A(:,i); % Projection of centered images into facespace
ProjectedImages = [ProjectedImages temp];
end
%%%%%%%%%%%%%%%%%%%%%%%% Extracting the PCA features from test image
InputImage = imread(TestImage);
temp = InputImage(:,:,1);
[irow icol] = size(temp);
InImage = reshape(temp',irow*icol,1);
Difference = double(InImage)-m; % Centered test image
ProjectedTestImage = Eigenfaces'*Difference; % Test image feature vector
%%%%%%%%%%%%%%%%%%%%%%%% Calculating Euclidean distances
% Euclidean distances between the projected test image and the projection
% of all centered training images are calculated. Test image is
% supposed to have minimum distance with its corresponding image in the
% training database.
Euc_dist = [];
for i = 1 : Train_Number
q = ProjectedImages(:,i);
temp = ( norm( ProjectedTestImage - q ) )^2;
Euc_dist = [Euc_dist temp];
end
[Euc_dist_min , Recognized_index] = min(Euc_dist);
OutputName = strcat(int2str(Recognized_index),'.jpg');
这里面的降维思想和我上一篇博客写的内容是一致的。
算法三
%--------------------------------------------------------------------------
% 函数说明：（2D）2 PCA算法
% 接收参数
% trainingSet:训练数据集，height*width*num的矩阵
% testingSet:测试数据集，3维矩阵
% numClass:训练样本和测试样本所拥有的类别数
% thresthold:阈值，决定特征向量个数的选择，既决定d,q的取值，
% 若传递的thresthold为小数，如0.95，自动根据阈值计算，d,q的取值；
% 若传递的thresthold为二维向量，如[30,20],则会设d=30，q=20
%
% 函数返回
% accuracy:分类准确率
% right:正确分类的样本数目
% qz:投影到Z上的个数
% dx:投影到X上的个数
%
%author:潘世瑞
%Date:2008-10-22
%--------------------------------------------------------------------------
function [accuracy,right,qz,dx]=PCA_2D_2_Classifier(trainingSet,testingSet,numClass,thresthold)
numTrainInstance = size(trainingSet,3); %训练样本数
numTestInstance = size(testingSet,3); %测试样本数
height = size(trainingSet,1); %图像高度
width = size(trainingSet,2); %图像宽度
perClassTrainLen = numTrainInstance/numClass;%每个类别的训练样本数
perClassTestLen = numTestInstance/numClass;%每个类别的测试样本数
dx = 3; %投影到多少维向量上，变量d的取值,thresthold)
break;
end
end;
dx = xi;
sumEigenValueZ = 0;
tmpZ = 0;
for xi=1:size(eigenValueZ,2)
sumEigenValueZ = sumEigenValueZ+eigenValueZ(xi,xi);
end
for qi=1:size(eigenValueZ,2)
tmpZ = tmpZ+eigenValueZ(qi,qi);
if(tmpZ/sumEigenValueZ>thresthold)
break;
end
end;
qz = qi;
disp('u set thresthold');
elseif(size(thresthold,2)==2)%接收为一个矩阵，分别表示q,d的取值
disp('u set d,q value');
qz = thresthold(2);
dx = thresthold(1);
end;
projection = zeros(width,dx); %投影矩阵,X
projectionZ = zeros(height,qz);%投影矩阵,Z
allprojectionFace = zeros(qz,dx,numTrainInstance);
end;
%求投影向量projection,即图像的特征矩阵或特征图像
for k=1:dx
projection(:,k) = eigenFace(:,k);
end
for k=1:qz
projectionZ(:,k) = eigenFaceZ(:,k);
end
%保存每个训练样本投影后的特征
for inum = 1:numTrainInstance
allprojectionFace(:,:,inum) = projectionZ'*trainingSet(:,:,inum)*projection;
end;
%以下为测试
right = 0;
for x=1:numTestInstance
%构造测试样本的特征向量
afterProjection = projectionZ'*testingSet(:,:,x)*projection;
error = zeros(numTrainInstance,1);
for i=1:numTrainInstance
%计算重构图像矩阵到各个类别图像矩阵间的距离
miss = afterProjection -allprojectionFace(:,:,i);
for j=1:size(miss,2)
error(i) =error(i)+ norm(miss(:,j));
end
end;
[errorS,errorIndex] = sort(error); %对距离进行排序
class = floor((errorIndex(1)-1)/perClassTrainLen)+1;%将图像分到距离最小的类别中去,预测的类别
oriclass = floor((x-1)/perClassTestLen)+1 ; %实际的类别
if(class == oriclass)
right = right+1;
end
end
accuracy = right/numTestInstance;
end 算法四
%--------------------------------------------------------------------------
% 函数说明：2DPCA算法
% 接收参数
% trainingSet:训练数据集，height*width*num的矩阵
% testingSet:测试数据集，3维矩阵
% numClass:训练样本和测试样本所拥有的类别数
% thresthold:阈值，决定特征向量个数的选择，既决定d取值，
% 若传递的thresthold为小数，如0.95，自动根据阈值计算，d,q的取值；
% 若传递的thresthold为>=1的整数，30,则会设d=30,numShape=d
%
% 函数返回
% accuracy:分类准确率
% right:正确分类的样本数目
%
%author:潘世瑞
%Date:2008-10-22
%--------------------------------------------------------------------------
function [accuracy,right,numShape]=PCA_2D_Classifier(trainingSet,testingSet,numClass,thresthold)
numTrainInstance = size(trainingSet,3); %训练样本数
numTestInstance = size(testingSet,3); %测试样本数
height = size(trainingSet,1); %图像高度
width = size(trainingSet,2); %图像宽度
perClassTrainLen = numTrainInstance/numClass;%每个类别的训练样本数
perClassTestLen = numTestInstance/numClass;%每个类别的测试样本数
numShape = 3;%训练的时候投影到多少维向量上，numShapethresthold)
break;
end
end;
numShape = xi;
else %否则认为设置的阈值为d的值，即投影到多少维上
%disp('u set the d value...');
numShape = thresthold;
end
projection = zeros(width,numShape);
allprojectionFace = zeros(height,numShape,numTrainInstance);
end
%求投影向量projection,即图像的特征矩阵或特征图像
for k=1:numShape
projection(:,k) = eigenFace(:,k);
end
%保存每个训练样本投影后的特征
for inum = 1:numTrainInstance
allprojectionFace(:,:,inum) = trainingSet(:,:,inum)*projection;
end;
%以下为测试
right = 0;
for x=1:numTestInstance
afterProjection = testingSet(:,:,x)*projection;
error = zeros(numTrainInstance,1);
for i=1:numTrainInstance
%计算重构图像矩阵到各个类别图像矩阵间的距离
miss = afterProjection -allprojectionFace(:,:,i);
for j=1:size(miss,2)
error(i) =error(i)+ norm(miss(:,j));
end
end;
[errorS,errorIndex] = sort(error); %对距离进行排序
class = floor((errorIndex(1)-1)/perClassTrainLen)+1;%将图像分到距离最小的类别中去,预测的类别
oriclass = floor((x-1)/perClassTestLen)+1 ; %实际的类别
if(class == oriclass)
right = right+1;
end
end
accuracy = right/numTestInstance;
end 算法五
function [C y] = myPCA(x)
%对矩阵用主成分分析法（PCA）进行降维，得到白化矩阵和降维后的矩阵
%输入参量：
%x-----待处理的矩阵
%输出参量：
%C-----白化矩阵
%y-----降维后的矩阵
[m,n] = size(mat1);
ref_mean = mean(mat1,2); %求矩阵mat1每行的均值
X = mat1 - repmat(ref_mean,1,n); %mat1的每个元素减去对应行的均值，以便求协方差
A = X'*X;
[V D] = eig(A); %求特征值和特征向量
D_diag = diag(D);
[numd] = find(D_diag<0); %去除因扰动造成的负特征值及其对应的特征向量
D_diag(numd) = [];
V(:,numd) = [];
[D_sort D_index] = sort(-D_diag); %特征值排序
D_sort = -D_sort;
%以总方差的70%为阈值，选取特征向量和特征值
D_pro = cumsum(D_sort)./sum(D_sort);
num = find(D_pro>0.9); %找到累计方差大于总方差70%的值所在的位置
T_num = num(1); %阈值所在的位置（以排序后的特征值为参照）
D_select = D_sort(1:T_num); %去除较小的特征值，得到有效特征值（已排序）
D_index = D_index(1:T_num); %得到有效特征值的序号
V_select = zeros(size(V,1),T_num);
for i = 1:T_num
V_select(:,i) = V(:,D_index(i)); %得到有效特征向量
end
C = X*V_select*diag(1./sqrt(D_select)); %求白化矩阵,X*V_select是X*X'的特征向量，乘以diag(1./sqrt(D_select))是为了归一化
%求投影矩阵，即经PCA降维后的矩阵
y = C'* X;
算法六
对于一个训练集，100个sample，特征是10维，那么它可以建立一个100*10的矩阵，作为样本。求这个样本的协方差矩阵，得到一个10*10的协方差矩阵，然后求出这个协方差矩阵的特征值和特征向量，应该有10个特征值和特征向量，我们根据特征值的大小，取前四个特征值所对应的特征向量，构成一个10*4的矩阵，这个矩阵就是我们要求的特征矩阵，100*10的样本矩阵乘以这个10*4的特征矩阵，就得到了一个100*4的新的降维之后的样本矩阵，每个sample的维数下降了。
当给定一个测试的特征集之后，比如1*10维的特征，乘以上面得到的10*4的特征矩阵，便可以得到一个1*4的特征，用这个特征去分类。
所以做PCA实际上是求得这个投影矩阵，用高维的特征乘以这个投影矩阵，便可以将高维特征的维数下降到指定的维数。
load yale.mat
m = mean(fea,2);
Train_Number = size(fea,2);
A = [];
for i = 1 : Train_Number
temp = fea(:,i) - m;
A = [A temp];
end
L = A'*A;
[V D] = eig(L);
L_eig_vec = [];
for i = 1 : 500
L_eig_vec = [L_eig_vec V(:,i)];
end
PCA = A * L_eig_vec;算法七
%PCA人脸识别
imgdata=[];%训练图像矩阵
for i=1:40
for j=1:5
a=imread(strcat('C:\Documents and Settings\Administrator\桌面\orl\s',num2str(i),'\',num2str(j),'.jpg'));
% imshow(a);
b=a(1:112*92); % b是行矢量 1×N，其中N＝10304
b=double(b);
imgdata=[imgdata; b]; % imgdata 是一个M * N 矩阵，imgdata中每一行数据一张图片，M＝200
end;
end;
imgdata=imgdata'; %每一列为一张图片
imgmean=mean(imgdata,2); % 平均图片，N维列向量 % 每一列是一张图像，但是为什么求一行的均值？有点怪异的程序
for i=1:200
minus(:,i) = imgdata(:,i)-imgmean; % minus是一个N*M矩阵，是训练图和平均图之间的差值
end;
covx=minus'* minus; % M * M 阶协方差矩阵
[COEFF, latent,explained] = pcacov(covx'); %PCA，用协方差矩阵的转置来计算以减小计算量
%选择构成95%的能量的特征值
i=1;
proportion=0;
while(proportion < 95)
proportion=proportion+explained(i);
i=i+1;
end;
p=i-1;
% 训练得到特征脸坐标系
i=1;
while (i<=p && latent(i)>0)
base(:,i) = latent(i)^(-1/2)*minus * COEFF(:,i); % base是N×p阶矩阵，用来进行投影，除以latent(i)^(1/2)是对人脸图像的标准化
i = i + 1;
end
% 将训练样本对坐标系上进行投影,得到一个 p*M 阶矩阵为参考
reference = base'*minus;
accu = 0; %计算准确度
% 测试过程
for i=1:40
for j=6:10 %读入40 x 5 副测试图像
a=imread(strcat('C:\Documents and Settings\Administrator\桌面\orl\s',num2str(i),'\',num2str(j),'.jpg'));
b=a(1:10304);
b=double(b);
b=b';
object = base'*(b-imgmean);
distance=100000;
%最小距离法，寻找和待识别图片最为接近的训练图片
for k=1:200
temp= norm(object - reference(:,k));
if (distance > temp)
which = k;
distance = temp;
end;
end;
temp1 = which/5;
if (temp1 == floor(temp1))
left = (temp1-1)*5;
right = temp1*5;
else
left = floor(temp1)*5;
right = ceil(temp1)*5;
end;
if (((i-1)*5+j-5>left)&&((i-1)*5+j-5<=right)) %正确识别
accu = accu+1;
end;
end;
end;
accuracy=accu/200 %输出识别率算法八
function [descriptors] = ComputePCA(array, numcomps)
array=double(array);
array=array';
if nargin < 2
error( 'Not enough input arguments.' );
end
meanvec = mean(array, 2);
meanarray = repmat(meanvec, 1, size(array,2));
A = array-meanarray;
covA = A*A';
disp('Computing principle components.');
[V, D] = eig(covA);
P = V(:, (size(V, 2) - numcomps + 1) : size(V, 2));
P = fliplr(P);%左右翻转特征向量，即从大到小的顺序排列
descriptors=P'*array;
end
以上这些算法呢，都基本上符合pca的运算原理，但是能不能保证是正确的pca，我也不确定，不过我测试的时候貌似有一两个是不正确的，仅供参考，了解原理。