LDA matlab code

% prophet Mohammed said [ALLAH will help any one helped his/her brother/sister] PBUH
%This code to apply LDA (Linear Discriminant Analysis) 
% for any information please send to engalaatharwat@hotmail.com
%Egypt - HICIT - +20106091638

% This example deals with 2 classes
c1=[1 2;2 3;3 3;4 5;5 5]  % the first class 5 observations
c2=[1 0;2 1;3 1;3 2;5 3;6 5] % the second class 6 observations
scatter(c1(:,1),c1(:,2),6,'r'),hold on;
scatter(c2(:,1),c2(:,2),6,'b');


% Number of observations of each class
n1=size(c1,1)
n2=size(c2,1)

%Mean of each class
mu1=mean(c1)
mu2=mean(c2)

% Average of the mean of all classes
mu=(mu1+mu2)/2

% Center the data (data-mean)
d1=c1-repmat(mu1,size(c1,1),1)
d2=c2-repmat(mu2,size(c2,1),1)


% Calculate the within class variance (SW)
s1=d1'*d1
s2=d2'*d2
sw=s1+s2
invsw=inv(sw)

% in case of two classes only use v
% v=invsw*(mu1-mu2)'

% if more than 2 classes calculate between class variance (SB)
sb1=n1*(mu1-mu)'*(mu1-mu)
sb2=n2*(mu2-mu)'*(mu2-mu)
SB=sb1+sb2
v=invsw*SB

% find eigne values and eigen vectors of the (v)
[evec,eval]=eig(v)

% Sort eigen vectors according to eigen values (descending order) and
% neglect eigen vectors according to small eigen values
% v=evec(greater eigen value)
% or use all the eigen vectors

% project the data of the first and second class respectively
y2=c2*v
y1=c1*v

运行结果：

c1 =

     1     2
     2     3
     3     3
     4     5
     5     5


c2 =

     1     0
     2     1
     3     1
     3     2
     5     3
     6     5


n1 =

     5


n2 =

     6


mu1 =

    3.0000    3.6000


mu2 =

    3.3333    2.0000


mu =

    3.1667    2.8000


d1 =

   -2.0000   -1.6000
   -1.0000   -0.6000
         0   -0.6000
    1.0000    1.4000
    2.0000    1.4000


d2 =

   -2.3333   -2.0000
   -1.3333   -1.0000
   -0.3333   -1.0000
   -0.3333         0
    1.6667    1.0000
    2.6667    3.0000


s1 =

   10.0000    8.0000
    8.0000    7.2000


s2 =

   17.3333   16.0000
   16.0000   16.0000


sw =

   27.3333   24.0000
   24.0000   23.2000


invsw =

    0.3991   -0.4128
   -0.4128    0.4702


sb1 =

    0.1389   -0.6667
   -0.6667    3.2000


sb2 =

    0.1667   -0.8000
   -0.8000    3.8400


SB =

    0.3056   -1.4667
   -1.4667    7.0400


v =

    0.7274   -3.4917
   -0.8157    3.9156


evec =

   -0.9790    0.6656
   -0.2040   -0.7463


eval =

         0         0
         0    4.6430


y2 =

    0.7274   -3.4917
    0.6391   -3.0679
    1.3666   -6.5596
    0.5508   -2.6440
    1.1900   -5.7119
    0.2859   -1.3725


y1 =

   -0.9041    4.3394
   -0.9924    4.7633
   -0.2649    1.2716
   -1.1690    5.6110
   -0.4415    2.1193

我的理解：

将特征值降序排列，对应的特征向量为：

w1：evec(:,2)

w2：evec(:,1)

 z1=c1*evec
z2=c2*evec
z1 =

   -1.3869   -0.8271
   -2.5698   -0.9079
   -3.5488   -0.2424
   -4.9357   -1.0695
   -5.9147   -0.4040

z2 =

   -0.9790    0.6656
   -2.1619    0.5848
   -3.1409    1.2503
   -3.3448    0.5040
   -5.5068    1.0887
   -6.8937    0.2616

c1*w1>c1*w2,c1属于类别1

c2*w2>c2*w1,c2属于类别2