树类模型特征重要性计算方法

我们在使用GBDT、RF、Xgboost等树类模型建模时，都会有一个feature_importance的方法来返回特征重要性。下面总结了不同树类模型计算特征重要性的原理：

• Random Foreast
  • 袋外数据错误率
  • 基尼指数
• GBDT
  • 基尼指数
• Xgboost
  • gain：is the average gain of splits which use the feature。就是特征用于分割的平均增益 
  • weight：is the number of times a feature appears in a tree。就是在所有树中特征用来分割的节点个数总和
  • cover：is the average coverage of splits which use the feature where coverage is defined as the number of samples affected by the split。可以理解为被分到该节点的样本的二阶导数之和，而特征度量的标准就是平均的coverage值
• Lightgbm
  • split：result contains numbers of times the feature is used in a mode
  • gain：result contains total gains of splits which use the feature

参考文献：

https://blog.csdn.net/zhangbaoanhadoop/article/details/81840656

https://www.cnblogs.com/xinping-study/archive/2018/04/10/8780817.html