递归特征消除和K折交叉验证(以决策树回归为例)

与特征选择不同的是递归特征消除后不会输出权值

导入相应模块

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.tree import DecisionTreeRegressor
from sklearn.feature_selection import RFECV

读取数据

X=pd.read_excel('boston.xlsx')
del X['Unnamed: 0']
Y=X['target'].copy()
del X['target']
names=X.columns

指标选择为均方误差

scoring='neg_root_mean_squared_error'

用决策树模型做特征选择

Tree = DecisionTreeRegressor()
rfecv = RFECV(estimator=Tree, step=1, cv=10,scoring=scoring)
rfecv.fit(X,Y)

RFECV(cv=10, estimator=DecisionTreeRegressor(),
      scoring='neg_root_mean_squared_error')

输出最佳的特征个数

print("Optimal number of features : %d" % rfecv.n_features_)

Optimal number of features : 12

查看一下这些特征的排序

rfe_rank_list=rfecv.ranking_
print('特征\t\t排序')
for i in range(len(rfe_rank_list)):
    print('%s\t\t%d'%(names[i],rfe_rank_list[i]))

特征		排序
CRIM		1
ZN		2
INDUS		1
CHAS		1
NOX		1
RM		1
AGE		1
DIS		1
RAD		1
TAX		1
PTRATIO		1
B		1
LSTAT		1

要删除掉排序不是1的特征

留下的特征为

for i in range(len(rfe_rank_list)):
    if(rfe_rank_list[i]==1):
        print(names[i])

CRIM
INDUS
CHAS
NOX
RM
AGE
DIS
RAD
TAX
PTRATIO
B
LSTAT

plt.figure()
plt.xlabel("Number of features selected")
plt.ylabel("Cross validation score")
plt.plot(range(1, len(rfecv.grid_scores_) + 1), rfecv.grid_scores_)
plt.show()