机器学习之PCA降维

PCA之降维与可视化

一、PCA 降维

1、使用使用sklearn自带数据集load_breast_cancer

from sklearn import datasets
cancer = datasets.load_breast_cancer()
X = cancer.data
y = cancer.target

2、划分训练测试集

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 1/4, random_state = 202

3、定义PCA

from sklearn.decomposition import PCA
n_components = 20
pca = PCA(n_components = n_components).fit(X_train)
X_train_pca = pca.transform(X_train)
X_test_pca = pca.transform(X_test)

4、使用XGB分别对降维前和降维后的训练集进行训练，并对测试集进行预测

import xgboost as xgb
num_round = 50
bst = xgb.train(params, X_train_xgb_origin, num_round, [(X_train_xgb_origin,‘train’),(X_test_xgb_origin,‘test’)])

降维后

num_round = 50
bst = xgb.train(params, X_train_xgb_pca, num_round, [(X_train_xgb_pca,‘train’),(X_test_xgb_pca,‘test’)])
由于数据量较少，因此在速度提升上不明显，但是训练效果有所下降…

二、PCA可视化

1、#将30维特征降到2维

n_components = 2
pca = PCA(n_components = n_components)
X_pca = pca.fit_transform(X)
import matplotlib.pyplot as plt
import numpy as np

#正样本用红色圆形表示，负样本用蓝色三角形表示
plt.scatter(pos[:,0],pos[:,1],s=60,marker = ‘o’,c= ‘r’)
plt.scatter(neg[:,0],neg[:,1],s=60,marker = ‘^’,c= ‘b’)
plt.title(u’PCA降维’)
plt.xlabel(u’维度1’)
plt.ylabel(u’维度2’)
plt.show()
从上图可以看出，正负样本的分布差异比较明显，尤其对于正样本，分布较为集中。