基于SVM的RFE

特征选择方法-RFE(包装式模型)

(一)RFE基本思想

    1.将全部特征纳入模型中,得到特征对应的系数(即权重);
    2.将取值最小的系数平方和对应的特征从模型中移除;

    3.用剩下的特征在进行模型训练,在进行特征移除,直至没有特征;

(二)基于SVM的RFE

1.模拟算法

import numpy as np
import pandas as pd
from sklearn import datasets
from sklearn import preprocessing
from sklearn.feature_selection import RFE
from sklearn.svm import LinearSVC

datasets = datasets.load_wine()

featureNames = datasets.feature_names

feat,label = datasets.data,datasets.target

scaler = preprocessing.StandardScaler()
inputVec = scaler.fit_transform(feat)

tmp,feat = inputVec.copy(),featureNames.copy() 
rank = []
while(tmp.shape[1]):
       clf = LinearSVC()
       clf.fit(tmp,label)
       coef = clf.coef_
       scores = np.sum(coef**2,axis=0)
       _id_ = np.argmin(scores)
       rank.append(feat[_id_])
       feat.pop(_id_)

       tmp = np.delete(tmp,_id_,axis=1)

print(rank)

2.函数实现

clf = LinearSVC()
model = RFE(clf,n_features_to_select=8)
model.fit(inputVec,label)
feats = list(np.array(featureNames)[model.support_])
print('choosed features:\n',feats)