python线性判别分析_（数据科学学习手札17）线性判别分析的原理简介&Python与R实现...

之前数篇博客我们比较了几种具有代表性的聚类算法，但现实工作中，最多的问题是分类与定性预测，即通过基于已标注类型的数据的各显著特征值，通过大量样本训练出的模型，来对新出现的样本进行分类，这也是机器学习中最多的问题，而本文便要介绍分类算法中比较古老的线性判别分析：

线性判别

最早提出合理的判别分析法者是R.A.Fisher(1936)，Fisher提出将线性判别函数用于花卉分类上，将花卉的各种特征利用线性组合方法变成单变量值，即将高维数据利用线性判别函数进行线性变化投影到一条直线上，再利用单值比较方法来对新样本进行分类，主要步骤如下：

Step1：求线性判别函数；

Step2：计算判别界值；

Step3：建立判别标准(这里与模糊分类中的隶属度有些相似，即离哪一类的投影中心最近，就将样本判别为哪一类)

下面分别利用Python，R，基于著名的花卉分类数据集iris进行演示：

Python

我们利用sklearn包中封装的LinearDiscriminantAnalysis对iris构建线性判别模型，因为LDA实际上是将高维数据尽可能分开的投影到一条直线上，因此LDA也可以对特定数据进行降维转换：

'''Fisher线性判别分析'''

importnumpy as npimportmatplotlib.pyplot as pltfrom sklearn importdatasetsfrom sklearn.discriminant_analysis importLinearDiscriminantAnalysisfrom matplotlib.pyplot importstylefrom sklearn.model_selection importtrain_test_split

style.use('ggplot')

iris=datasets.load_iris()

X=iris.data

y=iris.target'''展示LDA的降维功能'''target_names=iris.target_names'''设置压缩到1维'''lda= LinearDiscriminantAnalysis(n_components=1)'''利用线性判别函数将四维的样本数据压缩到一条直线上'''X_r2=lda.fit(X,y).transform(X)

X_Zero=np.zeros(X_r2.shape)'''绘制降维效果图'''

for c,i,target_names in zip('ryb',[0,1,2],target_names):

plt.scatter(X_r2[y== i],X_Zero[y == i],c=c,label=target_names,s=5)

plt.legend()

plt.grid()

降维后的效果图如下：

下面正式对iris数据集进行LDA分类，这里用到一个新的方法，是sklearn.model_selection.train_test_split，它的作用是根据设置的训练集与测试集的比例进行随机分割，我们利用从样本集中分割的7成数据作为训练集，3成数据进行测试，过程及结果如下：

'''利用sklearn自带的样本集划分方法进行分类，这里选择训练集测试集73开'''X_train,X_test,y_train,y_test= train_test_split(X,y,test_size=0.3)'''搭建LDA模型'''lda= LinearDiscriminantAnalysis(n_components=1)'''利用分割好的训练集进行模型训练并对测试集进行预测'''ld=lda.fit(X_train,y_train).predict(X_test)'''比较预测结果与真实分类结果'''

print(np.array([ld,y_test]))'''打印正确率'''

print('正确率：',str(lda.score(X_test,y_test)))

结果如下：

可以看出，在iris上取得了非常高的准确率。

R

在R中做LDA需要用到MASS包中的lda(formula~feature1+feature2+...+featuren,data=df)，其中formula表示数据集中表示分类标注的列，右边的各种feature表示将要使用到的分类特征值，也即是构建线性判别函数要用到的基础变量，data指保存全部数据的数据框，具体过程如下：

> #Fisher线性判别

> rm(list=ls())>library(MASS)>data(iris)> data data$Species =as.character(data$Species)> #创造类别变量

> data$type[data$Species == 'setosa'] = 1

> data$type[data$Species == 'versicolor'] = 2

> data$type[data$Species == 'virginica'] = 3

> #利用简单随机抽样将样本集划分为训练集与验证集

> sam train test #根据样本数据创建线性判别分析模型

> ld #将样本集作为验证集求分类结果

> Z #保存预测类别

> newType

> #与真实分类结果进行比较

>cbind(train$type,Z$x,newType)

LD1 LD2 newType74 2 -2.0419740 -1.16551052 2

93 2 -1.2286978 -1.34868982 2

124 3 -4.2056288 -0.39378496 3

43 1 7.0499296 -0.38479505 1

141 3 -6.3297144 1.43240686 3

112 3 -5.2009805 -0.39665336 3

75 2 -1.2076317 -0.53028083 2

83 2 -0.8607055 -1.04255122 2

128 3 -3.7958024 0.38283281 3

134 3 -3.5521695 -0.79338125 2

84 2 -4.1930475 -0.86304028 3

80 2 0.2455854 -1.48598565 2

16 1 8.9754828 2.99336222 1

35 1 6.7862663 -0.74053169 1

64 2 -2.4399568 -0.53150309 2

79 2 -2.4024680 -0.24830622 2

142 3 -4.9647448 1.48158101 3

54 2 -2.2217845 -1.93554530 2

14 1 7.3092968 -0.99420385 1

119 3 -8.8253015 -0.67042900 3

71 2 -3.3701925 0.94839271 3

44 1 6.2098650 0.99672773 1

29 1 7.6841973 0.08156876 1

62 2 -1.7099434 0.15332464 2

59 2 -1.6648086 -0.67372500 2

77 2 -2.3932456 -0.84052244 2

27 1 6.6078477 0.36272030 1

101 3 -7.2690741 1.94879260 3

91 2 -2.1739793 -1.53761667 2

40 1 7.4330661 0.03443993 1

103 3 -5.9833783 0.46642759 3

121 3 -5.9399677 1.45493705 3

51 2 -1.3784535 0.24058778 2

88 2 -2.5153892 -2.12899905 2

6 1 7.5334551 1.60652494 1

148 3 -4.7085798 0.61397476 3

76 2 -1.3919721 -0.13234144 2

10 1 7.0733099 -0.92823629 1

7 1 7.0416936 0.27174257 1

104 3 -5.1966351 -0.20937105 3

24 1 6.0120553 0.24387476 1

137 3 -6.0383747 2.20984770 3

61 2 -1.2257443 -3.03469429 2

41 1 7.6466593 0.57623503 1

26 1 6.4775174 -1.04708182 1

70 2 -1.0443574 -1.74662921 2

31 1 6.5351351 -0.78766052 1

85 2 -2.5818237 0.01276122 2

36 1 7.5972776 -0.33972390 1

4 1 6.6085362 -0.73929708 1

130 3 -4.2970292 -0.42496657 3

95 2 -1.8418994 -0.99664464 2

111 3 -4.1644831 1.17871159 3

117 3 -4.7101562 0.09594447 3

48 1 6.9765285 -0.43315849 1

106 3 -7.0303766 0.13158736 3

69 2 -3.5167119 -2.05931691 2

131 3 -5.9675456 -0.52249341 3

92 2 -2.0719645 -0.22536449 2

22 1 7.3872921 1.18564382 1

39 1 6.6977209 -0.90199151 1

125 3 -5.3082628 1.33894916 3

47 1 7.9455957 1.02129249 1

108 3 -5.9474167 -0.54626897 3

123 3 -7.2281863 -0.62126561 3

25 1 6.4877846 -0.15448692 1

96 2 -0.9672993 -0.40896608 2

149 3 -5.4267988 2.11763536 3

58 2 -0.1967945 -1.90480909 2

100 2 -1.4146638 -0.69091758 2

107 3 -4.3326495 -0.90276305 3

89 2 -1.1216984 -0.17330958 2

67 2 -2.4633370 0.01193815 2

45 1 6.7958450 1.25408059 1

138 3 -4.5932952 0.35495424 3

30 1 6.6519961 -0.52865075 1

17 1 8.3010066 1.79668640 1

9 1 6.3297286 -1.20813011 1

135 3 -4.6952607 -1.73516107 3

18 1 7.5140148 0.52828312 1

145 3 -6.4564371 2.08976755 3

68 2 -0.6703941 -1.51304115 2

12 1 7.0634482 -0.01186583 1

23 1 8.4484974 0.79139590 1

20 1 7.8504400 1.25653745 1

15 1 9.4800594 1.72576966 1

147 3 -5.0367690 -0.77081719 3

46 1 6.4557628 -0.76347342 1

60 2 -1.7902528 -0.66467281 2

19 1 7.8221244 1.15898751 1

144 3 -6.3829867 1.36026786 3

1 1 7.8010583 0.34057853 1

90 2 -1.8695758 -1.41834884 2

53 2 -2.2846205 0.07502495 2

115 3 -6.4317784 0.89801781 3

146 3 -5.4512238 1.17626549 3

78 2 -3.3451866 0.14511863 2

99 2 0.3864166 -1.31670824 2

118 3 -6.0412297 2.34012590 3

38 1 8.1457196 0.41229523 1

139 3 -3.6631579 0.43078471 3

105 3 -6.4339941 0.70414177 3

132 3 -4.7729922 2.10651473 3

110 3 -6.3994587 2.67334311 3

72 2 -0.9858025 -0.64502336 2

> #打印混淆矩阵

> (tab

newType1 2 3

1 350 02 0 33 1

3 0 2 34

> #显示正确率

> cat('Accuracy:',sum(diag(tab))/length(train[,1]))

Accuracy:0.9714286

> #将验证集代入训练好的模型中计算近似泛化误差

> T #与真实分类结果进行比较

> cbind(test$type,T$x,T$class)

LD1 LD22 1 6.8020498 -0.95158956 1

3 1 7.2276597 -0.38602966 1

5 1 7.9179194 0.59958829 1

8 1 7.3738228 0.03485147 1

11 1 8.1391093 0.80900002 1

13 1 7.0298500 -1.13888262 1

21 1 7.2270204 -0.06187541 1

28 1 7.6684138 0.29262662 1

32 1 7.0367090 0.40861452 1

33 1 9.0120836 1.65651141 1

34 1 9.2707624 2.14912000 1

37 1 8.2299196 0.38647275 1

42 1 5.2371900 -2.52488607 1

49 1 8.0798659 0.80941155 1

50 1 7.3896062 -0.17620640 1

52 2 -1.6371815 0.52584233 2

55 2 -2.4742435 -0.55650250 2

56 2 -2.1822152 -0.88107904 2

57 2 -2.1911397 0.87747596 2

63 2 -1.2405414 -2.75931501 2

65 2 -0.3383635 -0.19420599 2

66 2 -1.1566243 0.12584525 2

73 2 -3.6967069 -1.47409522 2

81 2 -1.0878173 -1.95727554 2

82 2 -0.6088858 -2.09743977 2

86 2 -1.8089897 1.23238953 2

87 2 -2.0193315 0.17092876 2

94 2 -0.3136555 -2.16381886 2

97 2 -1.4304473 -0.47985972 2

98 2 -1.3261184 -0.52945776 2

102 3 -5.1726649 -0.29910341 3

109 3 -6.0478550 -1.34049085 3

113 3 -5.3935569 0.65782366 3

114 3 -5.6792727 -0.58064337 3

116 3 -5.4686329 1.64715619 3

120 3 -4.5946379 -2.29619567 3

122 3 -5.0183151 0.24310322 3

126 3 -4.9026833 0.37255836 3

127 3 -3.8968800 -0.08723483 3

129 3 -6.1746269 0.09473298 3

133 3 -6.4616705 0.28243757 3

136 3 -6.5857811 0.74428684 3

140 3 -4.9663213 0.96355072 3

143 3 -5.1726649 -0.29910341 3

150 3 -4.2980648 0.28857515 3

> #打印混淆矩阵

> (tab

1 150 02 0 1503 0 0 15

> #显示正确率

> cat('Accuracy:',sum(diag(tab))/length(test[,1]))

Accuracy:1

> #Fisher线性判别

> rm(list=ls())>library(MASS)>data(iris)> data data$Species =as.character(data$Species)> #创造类别变量

> data$type[data$Species == 'setosa'] = 1

> data$type[data$Species == 'versicolor'] = 2

> data$type[data$Species == 'virginica'] = 3

> #利用简单随机抽样将样本集划分为训练集与验证集

> sam train test #根据样本数据创建线性判别分析模型

> ld #将样本集作为验证集求分类结果

> Z #保存预测类别

> newType

> #与真实分类结果进行比较

>cbind(train$type,Z$x,newType)

LD1 LD2 newType44 1 6.5356943 1.380095755 1

150 3 -5.0777607 0.353229454 3

48 1 7.2092979 -0.233679182 1

17 1 8.7810490 1.832128789 1

11 1 8.5106371 0.591364581 1

13 1 7.4261015 -0.949875862 1

82 2 -0.7168056 -1.805629506 2

19 1 8.2170070 0.870266056 1

123 3 -7.9020752 -1.233817765 3

23 1 8.7554448 0.980676590 1

59 2 -1.8788410 -0.915972039 2

70 2 -1.2367635 -1.516788339 2

146 3 -5.6852127 1.788579457 3

142 3 -5.0723482 2.082228707 3

115 3 -6.9597642 1.869494439 3

7 1 7.2188221 0.472577805 1

112 3 -5.6667730 -0.156736416 3

117 3 -5.4232144 -0.112017744 3

26 1 6.8888800 -0.860736454 1

111 3 -4.6405840 1.346460523 3

29 1 8.1491362 0.125965565 1

81 2 -1.2242456 -1.597693375 2

14 1 7.6286098 -0.598455540 1

110 3 -7.1444731 2.585994755 3

40 1 7.7798631 0.022078382 1

39 1 6.9782244 -0.504346286 1

68 2 -1.0104685 -1.671771426 2

20 1 8.0502422 1.117799195 1

31 1 6.7929797 -0.656064848 1

134 3 -4.1319678 -1.006493240 2

73 2 -4.0110318 -1.372974920 2

4 1 6.8227612 -0.537268125 1

108 3 -6.6972047 -1.217364668 3

54 2 -2.3695531 -1.376553027 2

148 3 -5.1575482 0.848139667 3

141 3 -6.8305530 1.864676411 3

109 3 -6.6173517 -1.422188092 3

50 1 7.7923809 -0.058826655 1

91 2 -2.7513769 -1.544578568 2

97 2 -1.8728790 -0.435815301 2

96 2 -1.4911207 -0.557876549 2

77 2 -2.5464899 -1.021564544 2

66 2 -1.2439344 0.003029162 2

145 3 -7.0771380 2.462076368 3

38 1 8.3218303 0.213545770 1

45 1 6.7744745 0.999164236 1

58 2 -0.3713519 -1.340382308 2

35 1 7.1622529 -0.552177665 1

36 1 8.1741719 -0.035844508 1

57 2 -2.7067243 0.719387605 2

27 1 6.9079284 0.551777520 1

132 3 -5.6031449 1.030132762 3

128 3 -4.3392145 0.561003821 3

87 2 -2.2635929 -0.006748767 2

88 2 -2.4886790 -1.851739918 2

118 3 -7.1004619 1.347087687 3

119 3 -9.4291409 -0.890616169 3

46 1 6.9234400 -0.316289540 1

33 1 9.0573501 1.063438766 1

10 1 7.4135836 -0.868970825 1

78 2 -3.6650096 0.105534552 2

32 1 7.6166932 0.640755164 1

139 3 -4.1962691 0.674830697 3

103 3 -6.5226603 0.373114540 3

43 1 7.2390794 -0.114882460 1

121 3 -6.4785762 1.623818440 3

15 1 10.1229028 1.482252025 1

125 3 -6.0718150 1.194903725 3

92 2 -2.5655637 -0.379597733 2

21 1 7.6071362 -0.210545218 1

116 3 -6.0199587 2.084095792 3

95 2 -2.2468978 -0.820309281 2

93 2 -1.3874483 -1.124059988 2

135 3 -5.6483660 -2.247095684 3

52 2 -1.9604386 0.420606745 2

37 1 8.8751646 0.414644969 1

127 3 -4.2307963 0.275427178 3

137 3 -6.8919261 2.468751536 3

98 2 -1.5631688 -0.569521563 2

72 2 -1.0384324 -0.432712540 2

80 2 0.2825956 -1.208391313 2

67 2 -3.1266046 0.070901690 2

120 3 -4.9979151 -2.051118149 3

114 3 -6.2027782 0.131952933 3

69 2 -3.4910411 -1.516772693 2

130 3 -4.8967334 -1.106964081 3

42 1 5.9270652 -1.555646363 1

64 2 -2.9521004 -0.683186676 2

86 2 -2.4035699 1.146743118 2

75 2 -1.3368410 -0.579461256 2

143 3 -5.8335378 0.090796722 3

140 3 -5.3380144 1.122071295 3

1 1 8.1663998 0.325667325 1

83 2 -1.0009115 -0.820471045 2

28 1 8.0234544 0.211840448 1

85 2 -3.3529324 0.080841383 2

89 2 -1.5995061 -0.127256512 2

18 1 7.9150690 0.642460485 1

147 3 -5.2724641 -0.214659306 3

105 3 -7.1968652 0.828583809 3

41 1 8.0580144 0.756287362 1

63 2 -1.1801689 -2.546513654 2

122 3 -5.6685504 0.829975549 3

107 3 -5.0735507 -0.234382628 3

47 1 8.0454637 0.692149004 1

> #打印混淆矩阵

> (tab

newType1 2 3

1 360 02 0 33 1

3 0 0 35

> #显示正确率

> cat('Accuracy:',sum(diag(tab))/length(train[,1]))

Accuracy:0.9904762> #将验证集代入训练好的模型中计算近似泛化误差

> T #与真实分类结果进行比较

> cbind(test$type,T$x,T$class)

LD1 LD22 1 7.2879346 -0.63805255 1

3 1 7.5785711 -0.12979200 1

5 1 8.1836634 0.52536908 1

6 1 7.7566121 1.39670067 1

8 1 7.6666992 0.02704823 1

9 1 6.5916877 -0.80793523 1

12 1 7.1842622 -0.07186911 1

16 1 9.2604596 2.57316476 1

22 1 7.6684840 1.23986044 1

24 1 6.3832248 0.56001189 1

25 1 6.4159345 -0.39844020 1

30 1 6.8102433 -0.45636309 1

34 1 9.5320477 1.66891133 1

49 1 8.3974733 0.59633443 1

51 2 -1.5423430 -0.14371955 2

53 2 -2.5494836 -0.23440252 2

55 2 -2.6250939 -0.47214778 2

56 2 -2.7716342 -0.95711830 2

60 2 -2.1825564 -0.15706564 2

61 2 -1.2921165 -2.34199387 2

62 2 -2.0187853 0.38256324 2

65 2 -0.4493874 0.22229672 2

71 2 -4.0485780 1.06926437 3

74 2 -2.5798663 -1.51150491 2

76 2 -1.4875258 -0.18673290 2

79 2 -2.8043766 -0.14370961 2

84 2 -4.8532177 -0.86952245 3

90 2 -2.1086982 -0.98708920 2

94 2 -0.3886155 -1.54008407 2

99 2 0.5024002 -0.51222584 2

100 2 -1.7471972 -0.52169018 2

101 3 -8.2981213 2.15538466 3

102 3 -5.8335378 0.09079672 3

104 3 -6.0360789 -0.40566699 3

106 3 -7.7496056 -0.41373390 3

113 3 -5.8377150 0.82345220 3

124 3 -4.5041691 -0.03313161 3

126 3 -5.6507584 -0.30162799 3

129 3 -6.8073348 0.34501073 3

131 3 -6.4535806 -0.88255921 3

133 3 -7.0586655 0.66180389 3

136 3 -6.8585571 0.75916772 3

138 3 -5.4059508 0.08768402 3

144 3 -7.1039586 1.41107423 3

149 3 -6.2415407 2.37464228 3

> #打印混淆矩阵

> (tab

1 140 02 0 1503 0 2 14

> #显示正确率

> cat('Accuracy:',sum(diag(tab))/length(test[,1]))

Accuracy:0.9555556

可以看出，和Python中的效果相差无几。

以上就是关于线性判别的基本内容，如有意见望提出。