XGBoost 实现保险赔偿预测：数据预处理(一）

XGBoost的原理过程不作介绍
保险赔偿预测项目是Kaggle比赛上的，相关数据资料可以在kaggle下载。
实现过程是B站up主：Tesra-AI不错哟
视频链接：
【我在B站学习 技能分享】机器学习提升篇

整个流程包含对数据预处理分析，再到XGBoost相关调参。
视频中是在Jupyter上实现，我这里是基于Pycharm+Python3.7
所以遇到有一些报错的代码我会基于环境做一些调整。
比如grid_socres_这个参数，在我的版本变成cv_results_,对应的代码都要调整。

下面是数据预处理过程的代码

import pandas as pd
import  numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import  LogisticRegression
from sklearn.ensemble  import  RandomForestClassifier
from  sklearn.metrics  import  roc_auc_score as AUC
from  sklearn.metrics import  mean_absolute_error
from sklearn.decomposition import  PCA
from sklearn.preprocessing import  LabelEncoder,LabelBinarizer
# from sklearn.cross_validation import  cross_val_score
from  sklearn.model_selection import  cross_val_score
from scipy import  stats
import  seaborn as sns
from copy import  deepcopy

# %matplotlib inline

# %config InlineBackeng.figure_format='retina'
train =pd.read_csv('D:/机器学习算法/allstate-claims-severity/train.csv')
print(train.shape)
print(train.head())
test =pd.read_csv('D:/机器学习算法/allstate-claims-severity/test.csv')
print(test.shape)
print(test.head())

#数据查看
print('First 20 columns:',list(train.columns[:20]))
print('Last 20 columns:',list(train.columns[-20:]))

train.describe()

#查看缺失值
pd.isnull(train).values.any()
train.info()


cat_features=list(train.select_dtypes(include=['object']).columns)
print("Categorical:{} features".format(len(cat_features)))


cont_features=[cont for cont in  list(train.select_dtypes(include=['float64','int64']).columns)
               if cont not in ['loss','id']]
print("Continuous:{} features".format(len(cont_features)))

id_col=list(train.select_dtypes(include=['int64']).columns)
print("A column of int64:{}".format(id_col))


#类别中属性的个数
cat_uniques=[]
for cat in cat_features:
    cat_uniques.append(len(train[cat].unique()))

data={'cat_name':cat_features,'unique_values':cat_uniques}
uniq_values_in_categories=pd.DataFrame.from_dict(data)
print(uniq_values_in_categories.head())


fig,(ax1,ax2)=plt.subplots(1,2)
fig.set_size_inches(16,5)
ax1.hist(uniq_values_in_categories.unique_values,bins=50)
ax1.set_title("Amount of categorical features with X distinct values")
ax1.set_xlabel("Distinct values in a feature")
ax1.set_ylabel("Features")
ax1.annotate('A features with 326 vals',xy=(322,2),xytext=(200,38),
             arrowprops=dict(facecolor='black'))

ax2.set_xlim(2,30)
ax2.set_title("Zooming in the [0,30] part of left histogram")
ax2.set_xlabel('Distinct values in a feature')
ax2.set_ylabel('Features')
ax2.grid(True)
ax2.hist(uniq_values_in_categories[uniq_values_in_categories.unique_values<=30].unique_values,bins=30)
ax2.annotate('Binary features',xy=(3,71),xytext=(7,71),arrowprops=dict(facecolor='black'))
fig.show()
plt.show()

#赔偿值
plt.figure(figsize=(16,8))
plt.plot(train['id'],train['loss'])
plt.title('Loss values per id')
plt.xlabel('id')
plt.ylabel('loss')
plt.legend()
plt.show()

#偏度
stats.mstats.skew(train['loss']).data

#对数变换
stats.mstats.skew(np.log(train['loss'])).data

fig,(ax1,ax2)=plt.subplots(1,2)
fig.set_size_inches(16,5)
ax1.hist(train['loss'],bins=50)
ax1.set_title('Train loss target histogram')
ax1.grid(True)

ax2.hist(np.log(train['loss']),bins=50,color='g')
ax2.set_title('Train log loass target histogram')
ax2.grid(True)
plt.show()


#连续值画图
train[cont_features].hist(bins=50,figsize=(16,12))
plt.show()

#特征之间的相关性
plt.subplots(figsize=(16,9))
correlation_mat=train[cont_features].corr()
sns.heatmap(correlation_mat,annot=True, cmap="OrRd")
plt.show()

类别型变量：

Target目标变量分布：

Target对数转换前后分布对比

连续型特征的分布图

连续型变量相关系数矩阵：

其他参考资料链接：
1.AttributeError : ‘GridSearchCV’ object has no attribute ‘grid_scores_’
2.xgboost 调参经验
3.【Python】绘制热力图seaborn.heatmap，cmap设置颜色的参数
4. XGBoost超详细推导