雪龙无敌

集成学习01_xgboost参数讲解与实战

本章分以下几块来讲解

一.xgboost 模型参数介绍

二.xgboost 两种方式实现

三. 网格搜索最优xgboost参数

一.XGBoost的参数

XGBoost的作者把所有的参数分成了三类，这里只介绍我们常用的一些参数，不常用的不做介绍

通用参数：宏观函数控制。
Booster参数：控制每一步的booster(tree/regression)。
学习目标参数：控制训练目标的表现。

1 通用参数

1）booster[默认gbtree]

选择每次迭代的模型，有两种选择：
gbtree：基于树的模型
gbliner：线性模型

2）silent[默认0]

当这个参数值为1时，静默模式开启，不会输出任何信息。
一般这个参数就保持默认的0，因为这样能帮我们更好地理解模型。

3）nthread[默认值为最大可能的线程数]

这个参数用来进行多线程控制，应当输入系统的核数。
如果你希望使用CPU全部的核，那就不要输入这个参数，算法会自动检测它。

4）num_feature [set automatically by xgboost, no need to be set by user]

boosting过程中用到的特征维数，设置为特征个数。
XGBoost会自动设置，不需要手工设置。

2 booster参数

尽管有两种booster可供选择，我这里只介绍tree booster，因为它的表现远远胜过linear booster，所以linear booster很少用到。

1）eta[默认0.3]

和GBM中的 learning rate 参数类似。
通过减少每一步的权重，可以提高模型的鲁棒性。
典型值为0.01-0.2。

2）min_child_weight[默认1]

*决定最小叶子节点样本权重和。

和GBM的 min_child_leaf 参数类似，但不完全一样。XGBoost的这个参数是最小样本权重的和，而GBM参数是最小样本总数。
这个参数用于避免过拟合。当它的值较大时，可以避免模型学习到局部的特殊样本。
但是如果这个值过高，会导致欠拟合。这个参数需要使用CV来调整。

3）max_depth[默认6]

和GBM中的参数相同，这个值为树的最大深度。
这个值也是用来避免过拟合的。max_depth越大，模型会学到更具体更局部的样本。
需要使用CV函数来进行调优。
典型值：3-10

4）max_leaf_nodes

树上最大的节点或叶子的数量。
可以替代max_depth的作用。因为如果生成的是二叉树，一个深度为n的树最多生成
如果定义了这个参数，GBM会忽略max_depth参数。

5）gamma[默认0]

在节点分裂时，只有分裂后损失函数的值下降了，才会分裂这个节点。
Gamma指定了节点分裂所需的最小损失函数下降值。这个参数的值越大，算法越保守。这个参数的值和损失函数息息相关，所以是需要调整的。
模型在默认情况下，对于一个节点的划分只有在其loss function 得到结果大于0的情况下才进行，而gamma 给定了所需的最低loss function的值
gamma值使得算法更conservation，且其值依赖于loss function ，在模型中应该进行调参.

6）max_delta_step[默认0]

这参数限制每棵树权重改变的最大步长。如果这个参数的值为0，那就意味着没有约束。如果它被赋予了某个正值，那么它会让这个算法更加保守。
通常，这个参数不需要设置。但是当各类别的样本十分不平衡时，它对逻辑回归是很有帮助的。
这个参数一般用不到，但是你可以挖掘出来它更多的用处。

7）subsample[默认1]

和GBM中的subsample参数一模一样。这个参数控制对于每棵树，随机采样的比例。
减小这个参数的值，算法会更加保守，避免过拟合。但是，如果这个值设置得过小，它可能会导致欠拟合。
典型值：0.5-1

8）colsample_bytree[默认1]

和GBM里面的max_features参数类似。用来控制每棵随机采样的列数的占比(每一列是一个特征)。
典型值：0.5-1

9）colsample_bylevel[默认1]

用来控制树的每一级的每一次分裂，对列数的采样的占比。
一般不太用这个参数，因为subsample参数和colsample_bytree参数可以起到相同的作用。但是如果感兴趣，可以挖掘这个参数更多的用处。

10）lambda[默认1]

权重的L2正则化项。(和Ridge regression类似)。
这个参数是用来控制XGBoost的正则化部分的。

11）alpha[默认1]

权重的L1正则化项。(和Lasso regression类似)。
可以应用在很高维度的情况下，使得算法的速度更快。

12）scale_pos_weight[默认1]

在各类别样本十分不平衡时，把这个参数设定为一个正值，可以使算法更快收敛。
大于0的取值可以处理类别不平衡的情况。帮助模型更快收敛。

13) Parameter for Linear Booster

lambda_bias

在偏置上的L2正则。缺省值为0（在L1上没有偏置项的正则，因为L1时偏置不重要）

3 学习目标参数

这个参数用来控制理想的优化目标和每一步结果的度量方法。

1）objective[默认reg:linear]

这个参数定义需要被最小化的损失函数。最常用的值有：定义学习任务及相应的学习目标，可选的目标函数如下：
“reg:linear” –线性回归。
“reg:logistic” –逻辑回归。
“binary:logistic” –二分类的逻辑回归问题，输出为概率。
“binary:logitraw” –二分类的逻辑回归问题，输出的结果为wTx。
“count:poisson” –计数问题的poisson回归，输出结果为poisson分布。
在poisson回归中，max_delta_step的缺省值为0.7。(used to safeguard optimization)
“multi:softmax” –让XGBoost采用softmax目标函数处理多分类问题，同时需要设置参数num_class（类别个数）
“multi:softprob” –和softmax一样，但是输出的是ndata * nclass的向量，可以将该向量reshape成ndata行nclass列的矩阵。每行数据表示样本所属于每个类别的概率。
“rank:pairwise” –set XGBoost to do ranking task by minimizing the pairwise loss

2）eval_metric[默认值取决于objective参数的取值]

对于有效数据的度量方法。
对于回归问题，默认值是rmse，对于分类问题，默认值是error。
典型值有：
rmse 均方根误差
mae 平均绝对误差
logloss 负对数似然函数值
error 二分类错误率(阈值为0.5)
merror 多分类错误率
mlogloss 多分类logloss损失函数
auc 曲线下面积

3）seed(默认0)

随机数的种子
设置它可以复现随机数据的结果，也可以用于调整参数
如果你比较习惯scikit-learn的参数形式，那么XGBoost的Python 版本也提供了sklearn形式的接口 XGBClassifier。

4）sklearn 参数对照

它使用sklearn形式的参数命名方式，对应关系如下：

1、eta -> learning_rate
2、lambda -> reg_lambda
3、alpha -> reg_alpha

4.平台控制参数 Console Parameters

The following parameters are only used in the console version of xgboost

use_buffer [ default=1 ]
是否为输入创建二进制的缓存文件，缓存文件可以加速计算。缺省值为1
num_round
boosting迭代计算次数。
data
输入数据的路径
test:data
测试数据的路径
save_period [default=0]
表示保存第i*save_period次迭代的模型。例如save_period=10表示每隔10迭代计算XGBoost将会保存中间结果，设置为0表示每次计算的模型都要保持。
task [default=train] options: train, pred, eval, dump
train：训练模型
pred：对测试数据进行预测
eval：通过eval[name]=filenam定义评价指标
dump：将学习模型保存成文本格式
model_in [default=NULL]
指向模型的路径在test, eval, dump都会用到，如果在training中定义XGBoost将会接着输入模型继续训练
model_out [default=NULL]
训练完成后模型的保存路径，如果没有定义则会输出类似0003.model这样的结果，0003是第三次训练的模型结果。
model_dir [default=models]
输出模型所保存的路径。
fmap
feature map, used for dump model
name_dump [default=dump.txt]
name of model dump file
name_pred [default=pred.txt]
预测结果文件
pred_margin [default=0]
输出预测的边界，而不是转换后的概率

二.xgboost 实现

本章以优惠券推荐数据为例对xgboost结合skleran与直接采用xgboost进行实现

1.导入相关包


import pandas as pd, numpy as np
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn import metrics
import catboost as cb
import xgboost as xgb
from xgboost.sklearn import XGBClassifier
import os
import joblib
from sklearn.preprocessing import LabelEncoder
from collections import defaultdict

data=pd.read_excel('car_coupon.xlsx')
data.head(5)

	ID	destination	passanger	toCoupon_GEQ15min	direction_same	direction_opp	gender	age	maritalStatus	...	Bar	CoffeeHouse	CarryAway	RestaurantLessThan20	Restaurant20To50	weather	time	coupon	expiration	Y
0	11263	No Urgent Place	Friend(s)	0	0	1	Male	55	Widowed	...	0	0	1	1	1	Sunny	14	Coffee House	24	1
1	20136	Work	Alone	1	1	0	Female	26	Married partner	...	0	0	3	3	3	Sunny	7	Bar	24	0
2	14763	Work	Alone	1	0	1	Female	55	Single	...	0	0	1	1	1	Sunny	7	Coffee House	24	0
3	12612	No Urgent Place	Kid(s)	1	0	1	Female	41	Married partner	...	0	3	3	3	3	Sunny	10	Carry out & Take away	2	0
4	17850	No Urgent Place	Partner	1	0	1	Female	31	Married partner	...	1	1	10	10	10	Snowy	14	Coffee House	2	0

5 rows × 23 columns

2.数据处理

1. 对类别数据进行编码

d = defaultdict(LabelEncoder)
data[['destination', 'passanger','gender','maritalStatus','education', 'occupation', \
    'weather','coupon' ]]=data[['destination', 'passanger','gender','maritalStatus','education', 'occupation', \
        'weather','coupon' ]].apply(lambda x: d[x.name].fit_transform(x))
data.head(5)

	ID	destination	passanger	toCoupon_GEQ15min	direction_same	direction_opp	gender	age	maritalStatus	...	Bar	CoffeeHouse	CarryAway	RestaurantLessThan20	Restaurant20To50	weather	time	coupon	expiration	Y
0	11263	1	1	0	0	1	1	55	4	...	0	0	1	1	1	2	14	2	24	1
1	20136	2	0	1	1	0	0	26	1	...	0	0	3	3	3	2	7	0	24	0
2	14763	2	0	1	0	1	0	55	2	...	0	0	1	1	1	2	7	2	24	0
3	12612	1	2	1	0	1	0	41	1	...	0	3	3	3	3	2	10	1	2	0
4	17850	1	3	1	0	1	0	31	1	...	1	1	10	10	10	1	14	2	2	0

5 rows × 23 columns

切分训练集与测试集

train, test, y_train, y_test = train_test_split(data.drop(["Y"], axis=1), data["Y"],
                                                random_state=10, test_size=0.3)

注意下 data ，train, test, y_train, y_test的数据格式

print(type(data))
print(type(train))
print(type( test))
print(type(y_train))
print(type(y_test))

撰写评价函数

def model_eval2(m, train, test):
    print('train_roc_auc_score:',metrics.roc_auc_score(y_train, m.predict_proba(train)[:, 1]))
    print('test_roc_auc_score:',metrics.roc_auc_score(y_test, m.predict_proba(test)[:, 1]))
    print('train_accuracy_score:',metrics.accuracy_score(y_train,  m.predict(train)))
    print('test_accuracy_score:',metrics.accuracy_score(y_test, m.predict(test)))
    print('train_precision_score:',metrics.precision_score(y_train, m.predict(train)))
    print('test__precision_score:',metrics.precision_score(y_test, m.predict(test)))
    print('train_recall_score:',metrics.recall_score(y_train, m.predict(train)))
    print('test_recall_score:',metrics.recall_score(y_test, m.predict(test)))
    print('train_f1_score:',metrics.f1_score(y_train, m.predict(train)))
    print('test_f1_score:',metrics.f1_score(y_test, m.predict(test)))

3.结合sklearn的xgboot模型

step01-拟合模型

from xgboost.sklearn import XGBClassifier
xgboost_model = XGBClassifier()
eval_set = [(test.values,  y_test.values)]
#拟合模型
xgboost_model.fit(train.values, 
                  y_train.values, 
                  early_stopping_rounds=300, 
                  eval_metric="logloss",  # 损失函数的类型，分类一般都是用对数作为损失函数
                  eval_set=eval_set,
                  verbose=False)

D:\dprograme\Anaconda3\lib\site-packages\xgboost\sklearn.py:793: UserWarning: `eval_metric` in `fit` method is deprecated for better compatibility with scikit-learn, use `eval_metric` in constructor or`set_params` instead.
  warnings.warn(
D:\dprograme\Anaconda3\lib\site-packages\xgboost\sklearn.py:793: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(

XGBClassifier(base_score=0.5, booster='gbtree', callbacks=None,
          colsample_bylevel=1, colsample_bynode=1, colsample_bytree=1,
          early_stopping_rounds=None, enable_categorical=False,
          eval_metric=None, gamma=0, gpu_id=-1, grow_policy='depthwise',
          importance_type=None, interaction_constraints='',
          learning_rate=0.300000012, max_bin=256, max_cat_to_onehot=4,
          max_delta_step=0, max_depth=6, max_leaves=0, min_child_weight=1,
          missing=nan, monotone_constraints='()', n_estimators=100,
          n_jobs=0, num_parallel_tree=1, predictor='auto', random_state=0,
          reg_alpha=0, reg_lambda=1, ...)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. 
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

step02-评价模型

model_eval2(xgboost_model, train.values, test.values)

train_roc_auc_score: 0.890295988831706
test_roc_auc_score: 0.7178983466569767
train_accuracy_score: 0.8007142857142857
test_accuracy_score: 0.6683333333333333
train_precision_score: 0.7965116279069767
test__precision_score: 0.704225352112676
train_recall_score: 0.8681875792141952
test_recall_score: 0.7267441860465116
train_f1_score: 0.8308065494238933
test_f1_score: 0.7153075822603719

step03-利用模型预测

xgboost_model.predict 预测结果是0或1的int型
xgboost_model.predict_proba预测结果是0到1之间的float型

y_test_pred = xgboost_model.predict( test.values )
y_trian_prod = xgboost_model.predict_proba( train.values )

step04-保存和调用模型

joblib.dump(xgboost_model , r'D:\Ensemble_Learning\xgboostinfo\xgboostsklearnsingle.model')
load_model=joblib.load(r'D:\Ensemble_Learning\xgboostinfo\xgboostsklearnsingle.model')
load_model.predict( test.values )

array([0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1,
       1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1,
       1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0,
       1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0,
       1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1,
       0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0,
       1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1,
       0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0,
       1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0,
       1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0,
       1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1,
       1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0,
       1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1,
       1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0,
       1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,
       0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0,
       1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1,
       1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0,
       0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1,
       1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0,
       1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1,
       0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
       1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1,
       1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0,
       1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1,
       0, 1, 1, 0, 1, 0])

注意点

上面xgboost_model.fit传入的是train.values和y_train.values,数据类型为numpy.ndarray
上面* xgboost_model.predict与xgboost_model.predict_proba传入的数据类型为numpy.ndarray

print(type(data.values))
print(type(train.values))
print(type( test.values))
print(type(y_train.values))
print(type(y_test.values))

4.直接采用xgboost的模型

step-01 构建参数

params={'alpha': 0.09,
 'booster': 'gbtree',
 'colsample_bylevel': 0.4,
 'colsample_bytree': 0.7,
 'eval_metric': 'logloss',
 'gamma': 0.85,
 'learning_rate': 0.1,
 'max_depth': 7,
 'min_child_weight': 20,
 'n_estimator': 40,
 'objective': 'binary:logistic',
 'reg_lambda': 0.1,
 'seed': 1,
 'subsample': 0.6}

step-02 处理数据

dtrain = xgb.DMatrix(train, label=y_train,feature_names=list(train.columns))
dtest = xgb.DMatrix(test)
validation = xgb.DMatrix(test,y_test)
watchlist = [(validation,'train')]

step-03 拟合模型

model = xgb.train(params,
                  dtrain,
                  num_boost_round= 2000, # 迭代的次数，及弱学习器的个数
                  evals= watchlist)

[21:06:30] WARNING: C:/Users/administrator/workspace/xgboost-win64_release_1.6.0/src/learner.cc:627: 
Parameters: { "n_estimator" } might not be used.

  This could be a false alarm, with some parameters getting used by language bindings but
  then being mistakenly passed down to XGBoost core, or some parameter actually being used
  but getting flagged wrongly here. Please open an issue if you find any such cases.


[0]	train-logloss:0.68835
[1]	train-logloss:0.68565
[2]	train-logloss:0.68298
[3]	train-logloss:0.67752
[4]	train-logloss:0.67465
[5]	train-logloss:0.67235
[6]	train-logloss:0.66660
[7]	train-logloss:0.66280
[8]	train-logloss:0.66026
[9]	train-logloss:0.65894
[10]	train-logloss:0.65901
[11]	train-logloss:0.65892
[12]	train-logloss:0.65751
[13]	train-logloss:0.65512
[14]	train-logloss:0.65389
[15]	train-logloss:0.65229
[16]	train-logloss:0.64792
[17]	train-logloss:0.64436
[18]	train-logloss:0.64343
[19]	train-logloss:0.64374
[20]	train-logloss:0.64223
[21]	train-logloss:0.63890
[22]	train-logloss:0.63934
[23]	train-logloss:0.63531
[24]	train-logloss:0.63163
[25]	train-logloss:0.63014
[26]	train-logloss:0.62985
[27]	train-logloss:0.62939
[28]	train-logloss:0.62872
[29]	train-logloss:0.62832
[30]	train-logloss:0.62718
[31]	train-logloss:0.62531
[32]	train-logloss:0.62274
[33]	train-logloss:0.62034
[34]	train-logloss:0.61853
[35]	train-logloss:0.61825
[36]	train-logloss:0.61698
[37]	train-logloss:0.61518
[38]	train-logloss:0.61462
[39]	train-logloss:0.61375
[40]	train-logloss:0.61137
[41]	train-logloss:0.61013
[42]	train-logloss:0.61013
[43]	train-logloss:0.61091
[44]	train-logloss:0.60978
[45]	train-logloss:0.60987
[46]	train-logloss:0.60909
[47]	train-logloss:0.60926
[48]	train-logloss:0.60889
[49]	train-logloss:0.60833
[50]	train-logloss:0.60849
[51]	train-logloss:0.60889
[52]	train-logloss:0.60871
[53]	train-logloss:0.60861
[54]	train-logloss:0.60935
[55]	train-logloss:0.60868
[56]	train-logloss:0.60836
[57]	train-logloss:0.60862
[58]	train-logloss:0.60933
[59]	train-logloss:0.60926
[60]	train-logloss:0.60929
[61]	train-logloss:0.60936
[62]	train-logloss:0.60876
[63]	train-logloss:0.60862
[64]	train-logloss:0.60866
[65]	train-logloss:0.60921
[66]	train-logloss:0.60946
[67]	train-logloss:0.60896
[68]	train-logloss:0.60919
[69]	train-logloss:0.60852
[70]	train-logloss:0.60873
[71]	train-logloss:0.60902
[72]	train-logloss:0.60903
[73]	train-logloss:0.60881
[74]	train-logloss:0.60862
[75]	train-logloss:0.60658
[76]	train-logloss:0.60641
[77]	train-logloss:0.60657
[78]	train-logloss:0.60661
[79]	train-logloss:0.60736
[80]	train-logloss:0.60740
[81]	train-logloss:0.60726
[82]	train-logloss:0.60717
[83]	train-logloss:0.60745
[84]	train-logloss:0.60663
[85]	train-logloss:0.60681
[86]	train-logloss:0.60718
[87]	train-logloss:0.60616
[88]	train-logloss:0.60682
[89]	train-logloss:0.60632
[90]	train-logloss:0.60609
[91]	train-logloss:0.60548
[92]	train-logloss:0.60544
[93]	train-logloss:0.60522
[94]	train-logloss:0.60536
[95]	train-logloss:0.60596
[96]	train-logloss:0.60680
[97]	train-logloss:0.60665
[98]	train-logloss:0.60742
[99]	train-logloss:0.60716
[100]	train-logloss:0.60704
[101]	train-logloss:0.60628
[102]	train-logloss:0.60648
[103]	train-logloss:0.60658
[104]	train-logloss:0.60748
[105]	train-logloss:0.60746
[106]	train-logloss:0.60750
[107]	train-logloss:0.60736
[108]	train-logloss:0.60640
[109]	train-logloss:0.60703
[110]	train-logloss:0.60651
[111]	train-logloss:0.60647
[112]	train-logloss:0.60556
[113]	train-logloss:0.60544
[114]	train-logloss:0.60372
[115]	train-logloss:0.60246
[116]	train-logloss:0.60285
[117]	train-logloss:0.60266
[118]	train-logloss:0.60286
[119]	train-logloss:0.60331
[120]	train-logloss:0.60429
[121]	train-logloss:0.60428
[122]	train-logloss:0.60386
[123]	train-logloss:0.60349
[124]	train-logloss:0.60357
[125]	train-logloss:0.60228
[126]	train-logloss:0.60228
[127]	train-logloss:0.60304
[128]	train-logloss:0.60288
[129]	train-logloss:0.60234
[130]	train-logloss:0.60196
[131]	train-logloss:0.60220
[132]	train-logloss:0.60163
[133]	train-logloss:0.60118
[134]	train-logloss:0.60188
[135]	train-logloss:0.60089
[136]	train-logloss:0.60052
[137]	train-logloss:0.60121
[138]	train-logloss:0.60029
[139]	train-logloss:0.59980
[140]	train-logloss:0.60066
[141]	train-logloss:0.60037
[142]	train-logloss:0.60084
[143]	train-logloss:0.60068
[144]	train-logloss:0.60141
[145]	train-logloss:0.60053
[146]	train-logloss:0.60028
[147]	train-logloss:0.60044
[148]	train-logloss:0.59957
[149]	train-logloss:0.60004
[150]	train-logloss:0.59962
[151]	train-logloss:0.59961
[152]	train-logloss:0.59938
[153]	train-logloss:0.59880
[154]	train-logloss:0.59873
[155]	train-logloss:0.59878
[156]	train-logloss:0.59905
[157]	train-logloss:0.59885
[158]	train-logloss:0.59913
[159]	train-logloss:0.59885
[160]	train-logloss:0.59845
[161]	train-logloss:0.59908
[162]	train-logloss:0.59909
[163]	train-logloss:0.59804
[164]	train-logloss:0.59788
[165]	train-logloss:0.59796
[166]	train-logloss:0.59915
[167]	train-logloss:0.59874
[168]	train-logloss:0.59868
[169]	train-logloss:0.59866
[170]	train-logloss:0.59915
[171]	train-logloss:0.59945
[172]	train-logloss:0.59978
[173]	train-logloss:0.59945
[174]	train-logloss:0.59956
[175]	train-logloss:0.59835
[176]	train-logloss:0.59840
[177]	train-logloss:0.59836
[178]	train-logloss:0.59825
[179]	train-logloss:0.59791
[180]	train-logloss:0.59836
[181]	train-logloss:0.59813
[182]	train-logloss:0.59832
[183]	train-logloss:0.59790
[184]	train-logloss:0.59847
[185]	train-logloss:0.59873
[186]	train-logloss:0.59886
[187]	train-logloss:0.59942
[188]	train-logloss:0.59865
[189]	train-logloss:0.59852
[190]	train-logloss:0.59852
[191]	train-logloss:0.59848
[192]	train-logloss:0.59884
[193]	train-logloss:0.59845
[194]	train-logloss:0.59827
[195]	train-logloss:0.59773
[196]	train-logloss:0.59742
[197]	train-logloss:0.59782
[198]	train-logloss:0.59742
[199]	train-logloss:0.59765
[200]	train-logloss:0.59699
[201]	train-logloss:0.59748
[202]	train-logloss:0.59788
[203]	train-logloss:0.59799
[204]	train-logloss:0.59756
[205]	train-logloss:0.59685
[206]	train-logloss:0.59746
[207]	train-logloss:0.59756
[208]	train-logloss:0.59718
[209]	train-logloss:0.59742
[210]	train-logloss:0.59784
[211]	train-logloss:0.59826
[212]	train-logloss:0.59800
[213]	train-logloss:0.59736
[214]	train-logloss:0.59694
[215]	train-logloss:0.59707
[216]	train-logloss:0.59706
[217]	train-logloss:0.59695
[218]	train-logloss:0.59711
[219]	train-logloss:0.59697
[220]	train-logloss:0.59773
[221]	train-logloss:0.59839
[222]	train-logloss:0.59860
[223]	train-logloss:0.59783
[224]	train-logloss:0.59776
[225]	train-logloss:0.59783
[226]	train-logloss:0.59780
[227]	train-logloss:0.59815
[228]	train-logloss:0.59765
[229]	train-logloss:0.59831
[230]	train-logloss:0.59830
[231]	train-logloss:0.59818
[232]	train-logloss:0.59829
[233]	train-logloss:0.59806
[234]	train-logloss:0.59734
[235]	train-logloss:0.59763
[236]	train-logloss:0.59748
[237]	train-logloss:0.59630
[238]	train-logloss:0.59615
[239]	train-logloss:0.59571
[240]	train-logloss:0.59605
[241]	train-logloss:0.59521
[242]	train-logloss:0.59485
[243]	train-logloss:0.59427
[244]	train-logloss:0.59476
[245]	train-logloss:0.59555
[246]	train-logloss:0.59568
[247]	train-logloss:0.59555
[248]	train-logloss:0.59653
[249]	train-logloss:0.59710
[250]	train-logloss:0.59722
[251]	train-logloss:0.59678
[252]	train-logloss:0.59689
[253]	train-logloss:0.59721
[254]	train-logloss:0.59773
[255]	train-logloss:0.59789
[256]	train-logloss:0.59814
[257]	train-logloss:0.59722
[258]	train-logloss:0.59697
[259]	train-logloss:0.59736
[260]	train-logloss:0.59678
[261]	train-logloss:0.59661
[262]	train-logloss:0.59701
[263]	train-logloss:0.59634
[264]	train-logloss:0.59628
[265]	train-logloss:0.59599
[266]	train-logloss:0.59570
[267]	train-logloss:0.59623
[268]	train-logloss:0.59656
[269]	train-logloss:0.59578
[270]	train-logloss:0.59617
[271]	train-logloss:0.59549
[272]	train-logloss:0.59521
[273]	train-logloss:0.59510
[274]	train-logloss:0.59484
[275]	train-logloss:0.59461
[276]	train-logloss:0.59496
[277]	train-logloss:0.59509
[278]	train-logloss:0.59511
[279]	train-logloss:0.59475
[280]	train-logloss:0.59425
[281]	train-logloss:0.59337
[282]	train-logloss:0.59408
[283]	train-logloss:0.59440
[284]	train-logloss:0.59461
[285]	train-logloss:0.59478
[286]	train-logloss:0.59540
[287]	train-logloss:0.59601
[288]	train-logloss:0.59565
[289]	train-logloss:0.59641
[290]	train-logloss:0.59619
[291]	train-logloss:0.59652
[292]	train-logloss:0.59666
[293]	train-logloss:0.59647
[294]	train-logloss:0.59690
[295]	train-logloss:0.59681
[296]	train-logloss:0.59674
[297]	train-logloss:0.59613
[298]	train-logloss:0.59633
[299]	train-logloss:0.59615
[300]	train-logloss:0.59657
[301]	train-logloss:0.59685
[302]	train-logloss:0.59679
[303]	train-logloss:0.59676
[304]	train-logloss:0.59651
[305]	train-logloss:0.59599
[306]	train-logloss:0.59591
[307]	train-logloss:0.59589
[308]	train-logloss:0.59606
[309]	train-logloss:0.59680
[310]	train-logloss:0.59755
[311]	train-logloss:0.59776
[312]	train-logloss:0.59839
[313]	train-logloss:0.59982
[314]	train-logloss:0.60061
[315]	train-logloss:0.60068
[316]	train-logloss:0.60074
[317]	train-logloss:0.60003
[318]	train-logloss:0.59996
[319]	train-logloss:0.59952
[320]	train-logloss:0.59922
[321]	train-logloss:0.59896
[322]	train-logloss:0.59843
[323]	train-logloss:0.59792
[324]	train-logloss:0.59771
[325]	train-logloss:0.59799
[326]	train-logloss:0.59850
[327]	train-logloss:0.59840
[328]	train-logloss:0.59858
[329]	train-logloss:0.59830
[330]	train-logloss:0.59859
[331]	train-logloss:0.59892
[332]	train-logloss:0.59962
[333]	train-logloss:0.59948
[334]	train-logloss:0.59957
[335]	train-logloss:0.59921
[336]	train-logloss:0.59992
[337]	train-logloss:0.60011
[338]	train-logloss:0.60025
[339]	train-logloss:0.60013
[340]	train-logloss:0.59981
[341]	train-logloss:0.59978
[342]	train-logloss:0.59933
[343]	train-logloss:0.59936
[344]	train-logloss:0.59835
[345]	train-logloss:0.59806
[346]	train-logloss:0.59652
[347]	train-logloss:0.59686
[348]	train-logloss:0.59685
[349]	train-logloss:0.59660
[350]	train-logloss:0.59550
[351]	train-logloss:0.59544
[352]	train-logloss:0.59591
[353]	train-logloss:0.59621
[354]	train-logloss:0.59615
[355]	train-logloss:0.59651
[356]	train-logloss:0.59627
[357]	train-logloss:0.59743
[358]	train-logloss:0.59777
[359]	train-logloss:0.59810
[360]	train-logloss:0.59777
[361]	train-logloss:0.59743
[362]	train-logloss:0.59659
[363]	train-logloss:0.59644
[364]	train-logloss:0.59640
[365]	train-logloss:0.59634
[366]	train-logloss:0.59636
[367]	train-logloss:0.59684
[368]	train-logloss:0.59731
[369]	train-logloss:0.59742
[370]	train-logloss:0.59739
[371]	train-logloss:0.59784
[372]	train-logloss:0.59729
[373]	train-logloss:0.59773
[374]	train-logloss:0.59768
[375]	train-logloss:0.59806
[376]	train-logloss:0.59811
[377]	train-logloss:0.59777
[378]	train-logloss:0.59874
[379]	train-logloss:0.59870
[380]	train-logloss:0.59868
[381]	train-logloss:0.59937
[382]	train-logloss:0.59917
[383]	train-logloss:0.59956
[384]	train-logloss:0.59952
[385]	train-logloss:0.59952
[386]	train-logloss:0.59907
[387]	train-logloss:0.59934
[388]	train-logloss:0.59920
[389]	train-logloss:0.59938
[390]	train-logloss:0.59972
[391]	train-logloss:0.59959
[392]	train-logloss:0.59966
[393]	train-logloss:0.59993
[394]	train-logloss:0.59983
[395]	train-logloss:0.60023
[396]	train-logloss:0.60025
[397]	train-logloss:0.60012
[398]	train-logloss:0.59959
[399]	train-logloss:0.59971
[400]	train-logloss:0.59964
[401]	train-logloss:0.59952
[402]	train-logloss:0.59944
[403]	train-logloss:0.59939
[404]	train-logloss:0.59934
[405]	train-logloss:0.59978
[406]	train-logloss:0.59954
[407]	train-logloss:0.59956
[408]	train-logloss:0.59985
[409]	train-logloss:0.59924
[410]	train-logloss:0.59999
[411]	train-logloss:0.60040
[412]	train-logloss:0.60098
[413]	train-logloss:0.60030
[414]	train-logloss:0.60028
[415]	train-logloss:0.59985
[416]	train-logloss:0.60055
[417]	train-logloss:0.60067
[418]	train-logloss:0.60093
[419]	train-logloss:0.60046
[420]	train-logloss:0.60099
[421]	train-logloss:0.60128
[422]	train-logloss:0.60063
[423]	train-logloss:0.60044
[424]	train-logloss:0.60062
[425]	train-logloss:0.60075
[426]	train-logloss:0.60039
[427]	train-logloss:0.60039
[428]	train-logloss:0.60120
[429]	train-logloss:0.60134
[430]	train-logloss:0.60121
[431]	train-logloss:0.60132
[432]	train-logloss:0.60147
[433]	train-logloss:0.60110
[434]	train-logloss:0.60113
[435]	train-logloss:0.60103
[436]	train-logloss:0.60065
[437]	train-logloss:0.60031
[438]	train-logloss:0.60043
[439]	train-logloss:0.60048
[440]	train-logloss:0.60005
[441]	train-logloss:0.59975
[442]	train-logloss:0.59958
[443]	train-logloss:0.59946
[444]	train-logloss:0.59932
[445]	train-logloss:0.59964
[446]	train-logloss:0.59884
[447]	train-logloss:0.59847
[448]	train-logloss:0.59863
[449]	train-logloss:0.59869
[450]	train-logloss:0.59856
[451]	train-logloss:0.59894
[452]	train-logloss:0.59901
[453]	train-logloss:0.59873
[454]	train-logloss:0.59953
[455]	train-logloss:0.59953
[456]	train-logloss:0.59972
[457]	train-logloss:0.59962
[458]	train-logloss:0.59994
[459]	train-logloss:0.60006
[460]	train-logloss:0.60028
[461]	train-logloss:0.60110
[462]	train-logloss:0.60111
[463]	train-logloss:0.60122
[464]	train-logloss:0.60074
[465]	train-logloss:0.60093
[466]	train-logloss:0.60080
[467]	train-logloss:0.60120
[468]	train-logloss:0.60122
[469]	train-logloss:0.60124
[470]	train-logloss:0.60122
[471]	train-logloss:0.60116
[472]	train-logloss:0.60101
[473]	train-logloss:0.60090
[474]	train-logloss:0.60111
[475]	train-logloss:0.60109
[476]	train-logloss:0.60151
[477]	train-logloss:0.60201
[478]	train-logloss:0.60160
[479]	train-logloss:0.60101
[480]	train-logloss:0.60132
[481]	train-logloss:0.60067
[482]	train-logloss:0.60054
[483]	train-logloss:0.60041
[484]	train-logloss:0.60017
[485]	train-logloss:0.60025
[486]	train-logloss:0.60024
[487]	train-logloss:0.59967
[488]	train-logloss:0.59935
[489]	train-logloss:0.59868
[490]	train-logloss:0.59907
[491]	train-logloss:0.59912
[492]	train-logloss:0.59919
[493]	train-logloss:0.59890
[494]	train-logloss:0.59955
[495]	train-logloss:0.59947
[496]	train-logloss:0.59907
[497]	train-logloss:0.59937
[498]	train-logloss:0.59933
[499]	train-logloss:0.59960
[500]	train-logloss:0.60029
[501]	train-logloss:0.60047
[502]	train-logloss:0.60013
[503]	train-logloss:0.59989
[504]	train-logloss:0.60059
[505]	train-logloss:0.60072
[506]	train-logloss:0.60102
[507]	train-logloss:0.60086
[508]	train-logloss:0.60060
[509]	train-logloss:0.60126
[510]	train-logloss:0.60112
[511]	train-logloss:0.60126
[512]	train-logloss:0.60129
[513]	train-logloss:0.60059
[514]	train-logloss:0.59989
[515]	train-logloss:0.60005
[516]	train-logloss:0.59968
[517]	train-logloss:0.60008
[518]	train-logloss:0.60084
[519]	train-logloss:0.60062
[520]	train-logloss:0.60111
[521]	train-logloss:0.60070
[522]	train-logloss:0.60063
[523]	train-logloss:0.60065
[524]	train-logloss:0.60044
[525]	train-logloss:0.60053
[526]	train-logloss:0.60099
[527]	train-logloss:0.60125
[528]	train-logloss:0.60105
[529]	train-logloss:0.60155
[530]	train-logloss:0.60176
[531]	train-logloss:0.60249
[532]	train-logloss:0.60304
[533]	train-logloss:0.60372
[534]	train-logloss:0.60326
[535]	train-logloss:0.60391
[536]	train-logloss:0.60371
[537]	train-logloss:0.60472
[538]	train-logloss:0.60431
[539]	train-logloss:0.60337
[540]	train-logloss:0.60355
[541]	train-logloss:0.60365
[542]	train-logloss:0.60295
[543]	train-logloss:0.60268
[544]	train-logloss:0.60312
[545]	train-logloss:0.60293
[546]	train-logloss:0.60275
[547]	train-logloss:0.60344
[548]	train-logloss:0.60334
[549]	train-logloss:0.60411
[550]	train-logloss:0.60460
[551]	train-logloss:0.60409
[552]	train-logloss:0.60423
[553]	train-logloss:0.60366
[554]	train-logloss:0.60341
[555]	train-logloss:0.60364
[556]	train-logloss:0.60365
[557]	train-logloss:0.60316
[558]	train-logloss:0.60353
[559]	train-logloss:0.60382
[560]	train-logloss:0.60396
[561]	train-logloss:0.60426
[562]	train-logloss:0.60465
[563]	train-logloss:0.60500
[564]	train-logloss:0.60502
[565]	train-logloss:0.60465
[566]	train-logloss:0.60496
[567]	train-logloss:0.60545
[568]	train-logloss:0.60523
[569]	train-logloss:0.60451
[570]	train-logloss:0.60424
[571]	train-logloss:0.60479
[572]	train-logloss:0.60501
[573]	train-logloss:0.60472
[574]	train-logloss:0.60399
[575]	train-logloss:0.60399
[576]	train-logloss:0.60339
[577]	train-logloss:0.60306
[578]	train-logloss:0.60286
[579]	train-logloss:0.60302
[580]	train-logloss:0.60266
[581]	train-logloss:0.60206
[582]	train-logloss:0.60216
[583]	train-logloss:0.60172
[584]	train-logloss:0.60186
[585]	train-logloss:0.60173
[586]	train-logloss:0.60159
[587]	train-logloss:0.60130
[588]	train-logloss:0.60173
[589]	train-logloss:0.60182
[590]	train-logloss:0.60176
[591]	train-logloss:0.60224
[592]	train-logloss:0.60242
[593]	train-logloss:0.60209
[594]	train-logloss:0.60148
[595]	train-logloss:0.60173
[596]	train-logloss:0.60187
[597]	train-logloss:0.60157
[598]	train-logloss:0.60219
[599]	train-logloss:0.60211
[600]	train-logloss:0.60197
[601]	train-logloss:0.60217
[602]	train-logloss:0.60158
[603]	train-logloss:0.60171
[604]	train-logloss:0.60143
[605]	train-logloss:0.60067
[606]	train-logloss:0.60052
[607]	train-logloss:0.60008
[608]	train-logloss:0.59992
[609]	train-logloss:0.60023
[610]	train-logloss:0.60063
[611]	train-logloss:0.60079
[612]	train-logloss:0.60056
[613]	train-logloss:0.60045
[614]	train-logloss:0.60035
[615]	train-logloss:0.60040
[616]	train-logloss:0.60038
[617]	train-logloss:0.60047
[618]	train-logloss:0.60006
[619]	train-logloss:0.60058
[620]	train-logloss:0.60048
[621]	train-logloss:0.60130
[622]	train-logloss:0.60134
[623]	train-logloss:0.60108
[624]	train-logloss:0.60107
[625]	train-logloss:0.60103
[626]	train-logloss:0.60110
[627]	train-logloss:0.60111
[628]	train-logloss:0.60118
[629]	train-logloss:0.60107
[630]	train-logloss:0.60026
[631]	train-logloss:0.60035
[632]	train-logloss:0.60089
[633]	train-logloss:0.60139
[634]	train-logloss:0.60136
[635]	train-logloss:0.60107
[636]	train-logloss:0.60094
[637]	train-logloss:0.60075
[638]	train-logloss:0.60102
[639]	train-logloss:0.60164
[640]	train-logloss:0.60075
[641]	train-logloss:0.60064
[642]	train-logloss:0.60051
[643]	train-logloss:0.60076
[644]	train-logloss:0.60053
[645]	train-logloss:0.60062
[646]	train-logloss:0.60055
[647]	train-logloss:0.60115
[648]	train-logloss:0.60093
[649]	train-logloss:0.60052
[650]	train-logloss:0.60054
[651]	train-logloss:0.60064
[652]	train-logloss:0.60126
[653]	train-logloss:0.60113
[654]	train-logloss:0.60096
[655]	train-logloss:0.60108
[656]	train-logloss:0.60129
[657]	train-logloss:0.60122
[658]	train-logloss:0.60162
[659]	train-logloss:0.60155
[660]	train-logloss:0.60163
[661]	train-logloss:0.60166
[662]	train-logloss:0.60170
[663]	train-logloss:0.60317
[664]	train-logloss:0.60358
[665]	train-logloss:0.60430
[666]	train-logloss:0.60406
[667]	train-logloss:0.60419
[668]	train-logloss:0.60394
[669]	train-logloss:0.60423
[670]	train-logloss:0.60479
[671]	train-logloss:0.60492
[672]	train-logloss:0.60493
[673]	train-logloss:0.60458
[674]	train-logloss:0.60413
[675]	train-logloss:0.60381
[676]	train-logloss:0.60380
[677]	train-logloss:0.60329
[678]	train-logloss:0.60327
[679]	train-logloss:0.60334
[680]	train-logloss:0.60352
[681]	train-logloss:0.60370
[682]	train-logloss:0.60361
[683]	train-logloss:0.60389
[684]	train-logloss:0.60361
[685]	train-logloss:0.60419
[686]	train-logloss:0.60502
[687]	train-logloss:0.60500
[688]	train-logloss:0.60507
[689]	train-logloss:0.60466
[690]	train-logloss:0.60461
[691]	train-logloss:0.60461
[692]	train-logloss:0.60505
[693]	train-logloss:0.60527
[694]	train-logloss:0.60532
[695]	train-logloss:0.60534
[696]	train-logloss:0.60565
[697]	train-logloss:0.60592
[698]	train-logloss:0.60541
[699]	train-logloss:0.60534
[700]	train-logloss:0.60509
[701]	train-logloss:0.60491
[702]	train-logloss:0.60503
[703]	train-logloss:0.60507
[704]	train-logloss:0.60564
[705]	train-logloss:0.60548
[706]	train-logloss:0.60611
[707]	train-logloss:0.60603
[708]	train-logloss:0.60553
[709]	train-logloss:0.60522
[710]	train-logloss:0.60433
[711]	train-logloss:0.60431
[712]	train-logloss:0.60441
[713]	train-logloss:0.60433
[714]	train-logloss:0.60479
[715]	train-logloss:0.60464
[716]	train-logloss:0.60522
[717]	train-logloss:0.60565
[718]	train-logloss:0.60521
[719]	train-logloss:0.60472
[720]	train-logloss:0.60502
[721]	train-logloss:0.60541
[722]	train-logloss:0.60551
[723]	train-logloss:0.60531
[724]	train-logloss:0.60464
[725]	train-logloss:0.60453
[726]	train-logloss:0.60449
[727]	train-logloss:0.60426
[728]	train-logloss:0.60378
[729]	train-logloss:0.60523
[730]	train-logloss:0.60574
[731]	train-logloss:0.60550
[732]	train-logloss:0.60547
[733]	train-logloss:0.60580
[734]	train-logloss:0.60546
[735]	train-logloss:0.60541
[736]	train-logloss:0.60566
[737]	train-logloss:0.60568
[738]	train-logloss:0.60556
[739]	train-logloss:0.60546
[740]	train-logloss:0.60533
[741]	train-logloss:0.60570
[742]	train-logloss:0.60580
[743]	train-logloss:0.60562
[744]	train-logloss:0.60563
[745]	train-logloss:0.60553
[746]	train-logloss:0.60570
[747]	train-logloss:0.60584
[748]	train-logloss:0.60632
[749]	train-logloss:0.60628
[750]	train-logloss:0.60637
[751]	train-logloss:0.60680
[752]	train-logloss:0.60716
[753]	train-logloss:0.60663
[754]	train-logloss:0.60630
[755]	train-logloss:0.60617
[756]	train-logloss:0.60614
[757]	train-logloss:0.60527
[758]	train-logloss:0.60568
[759]	train-logloss:0.60560
[760]	train-logloss:0.60595
[761]	train-logloss:0.60631
[762]	train-logloss:0.60588
[763]	train-logloss:0.60584
[764]	train-logloss:0.60627
[765]	train-logloss:0.60617
[766]	train-logloss:0.60665
[767]	train-logloss:0.60641
[768]	train-logloss:0.60655
[769]	train-logloss:0.60689
[770]	train-logloss:0.60710
[771]	train-logloss:0.60707
[772]	train-logloss:0.60664
[773]	train-logloss:0.60689
[774]	train-logloss:0.60732
[775]	train-logloss:0.60677
[776]	train-logloss:0.60677
[777]	train-logloss:0.60719
[778]	train-logloss:0.60771
[779]	train-logloss:0.60774
[780]	train-logloss:0.60803
[781]	train-logloss:0.60886
[782]	train-logloss:0.60919
[783]	train-logloss:0.60931
[784]	train-logloss:0.60956
[785]	train-logloss:0.60928
[786]	train-logloss:0.60890
[787]	train-logloss:0.60871
[788]	train-logloss:0.60884
[789]	train-logloss:0.60840
[790]	train-logloss:0.60815
[791]	train-logloss:0.60824
[792]	train-logloss:0.60808
[793]	train-logloss:0.60843
[794]	train-logloss:0.60818
[795]	train-logloss:0.60906
[796]	train-logloss:0.60931
[797]	train-logloss:0.60894
[798]	train-logloss:0.60874
[799]	train-logloss:0.60895
[800]	train-logloss:0.60818
[801]	train-logloss:0.60806
[802]	train-logloss:0.60856
[803]	train-logloss:0.60939
[804]	train-logloss:0.60937
[805]	train-logloss:0.60924
[806]	train-logloss:0.60880
[807]	train-logloss:0.60893
[808]	train-logloss:0.60851
[809]	train-logloss:0.60872
[810]	train-logloss:0.60823
[811]	train-logloss:0.60924
[812]	train-logloss:0.60916
[813]	train-logloss:0.60913
[814]	train-logloss:0.60906
[815]	train-logloss:0.60876
[816]	train-logloss:0.60875
[817]	train-logloss:0.60929
[818]	train-logloss:0.60952
[819]	train-logloss:0.60933
[820]	train-logloss:0.60891
[821]	train-logloss:0.60856
[822]	train-logloss:0.60921
[823]	train-logloss:0.60961
[824]	train-logloss:0.60921
[825]	train-logloss:0.60899
[826]	train-logloss:0.60953
[827]	train-logloss:0.61011
[828]	train-logloss:0.60985
[829]	train-logloss:0.60952
[830]	train-logloss:0.60889
[831]	train-logloss:0.60909
[832]	train-logloss:0.60925
[833]	train-logloss:0.60953
[834]	train-logloss:0.60918
[835]	train-logloss:0.60896
[836]	train-logloss:0.60951
[837]	train-logloss:0.60939
[838]	train-logloss:0.60935
[839]	train-logloss:0.60904
[840]	train-logloss:0.60951
[841]	train-logloss:0.61017
[842]	train-logloss:0.61034
[843]	train-logloss:0.61009
[844]	train-logloss:0.61010
[845]	train-logloss:0.61063
[846]	train-logloss:0.61112
[847]	train-logloss:0.61078
[848]	train-logloss:0.61036
[849]	train-logloss:0.61058
[850]	train-logloss:0.61066
[851]	train-logloss:0.61041
[852]	train-logloss:0.61029
[853]	train-logloss:0.60977
[854]	train-logloss:0.60990
[855]	train-logloss:0.60954
[856]	train-logloss:0.60964
[857]	train-logloss:0.60979
[858]	train-logloss:0.60995
[859]	train-logloss:0.60974
[860]	train-logloss:0.60945
[861]	train-logloss:0.60979
[862]	train-logloss:0.61024
[863]	train-logloss:0.61075
[864]	train-logloss:0.61087
[865]	train-logloss:0.61062
[866]	train-logloss:0.61108
[867]	train-logloss:0.61132
[868]	train-logloss:0.61127
[869]	train-logloss:0.61123
[870]	train-logloss:0.61163
[871]	train-logloss:0.61160
[872]	train-logloss:0.61153
[873]	train-logloss:0.61156
[874]	train-logloss:0.61207
[875]	train-logloss:0.61186
[876]	train-logloss:0.61301
[877]	train-logloss:0.61300
[878]	train-logloss:0.61276
[879]	train-logloss:0.61250
[880]	train-logloss:0.61269
[881]	train-logloss:0.61302
[882]	train-logloss:0.61330
[883]	train-logloss:0.61256
[884]	train-logloss:0.61219
[885]	train-logloss:0.61190
[886]	train-logloss:0.61175
[887]	train-logloss:0.61211
[888]	train-logloss:0.61195
[889]	train-logloss:0.61177
[890]	train-logloss:0.61180
[891]	train-logloss:0.61172
[892]	train-logloss:0.61242
[893]	train-logloss:0.61320
[894]	train-logloss:0.61337
[895]	train-logloss:0.61354
[896]	train-logloss:0.61354
[897]	train-logloss:0.61361
[898]	train-logloss:0.61390
[899]	train-logloss:0.61390
[900]	train-logloss:0.61439
[901]	train-logloss:0.61473
[902]	train-logloss:0.61455
[903]	train-logloss:0.61482
[904]	train-logloss:0.61491
[905]	train-logloss:0.61608
[906]	train-logloss:0.61604
[907]	train-logloss:0.61654
[908]	train-logloss:0.61628
[909]	train-logloss:0.61609
[910]	train-logloss:0.61661
[911]	train-logloss:0.61665
[912]	train-logloss:0.61649
[913]	train-logloss:0.61661
[914]	train-logloss:0.61669
[915]	train-logloss:0.61661
[916]	train-logloss:0.61669
[917]	train-logloss:0.61610
[918]	train-logloss:0.61622
[919]	train-logloss:0.61678
[920]	train-logloss:0.61674
[921]	train-logloss:0.61652
[922]	train-logloss:0.61651
[923]	train-logloss:0.61610
[924]	train-logloss:0.61625
[925]	train-logloss:0.61607
[926]	train-logloss:0.61634
[927]	train-logloss:0.61619
[928]	train-logloss:0.61594
[929]	train-logloss:0.61565
[930]	train-logloss:0.61541
[931]	train-logloss:0.61557
[932]	train-logloss:0.61549
[933]	train-logloss:0.61504
[934]	train-logloss:0.61500
[935]	train-logloss:0.61530
[936]	train-logloss:0.61608
[937]	train-logloss:0.61571
[938]	train-logloss:0.61553
[939]	train-logloss:0.61567
[940]	train-logloss:0.61549
[941]	train-logloss:0.61562
[942]	train-logloss:0.61594
[943]	train-logloss:0.61611
[944]	train-logloss:0.61579
[945]	train-logloss:0.61624
[946]	train-logloss:0.61548
[947]	train-logloss:0.61579
[948]	train-logloss:0.61570
[949]	train-logloss:0.61623
[950]	train-logloss:0.61624
[951]	train-logloss:0.61583
[952]	train-logloss:0.61581
[953]	train-logloss:0.61566
[954]	train-logloss:0.61573
[955]	train-logloss:0.61590
[956]	train-logloss:0.61602
[957]	train-logloss:0.61595
[958]	train-logloss:0.61607
[959]	train-logloss:0.61633
[960]	train-logloss:0.61581
[961]	train-logloss:0.61588
[962]	train-logloss:0.61593
[963]	train-logloss:0.61603
[964]	train-logloss:0.61550
[965]	train-logloss:0.61553
[966]	train-logloss:0.61595
[967]	train-logloss:0.61583
[968]	train-logloss:0.61558
[969]	train-logloss:0.61575
[970]	train-logloss:0.61599
[971]	train-logloss:0.61579
[972]	train-logloss:0.61623
[973]	train-logloss:0.61584
[974]	train-logloss:0.61529
[975]	train-logloss:0.61515
[976]	train-logloss:0.61492
[977]	train-logloss:0.61465
[978]	train-logloss:0.61481
[979]	train-logloss:0.61462
[980]	train-logloss:0.61420
[981]	train-logloss:0.61395
[982]	train-logloss:0.61406
[983]	train-logloss:0.61360
[984]	train-logloss:0.61340
[985]	train-logloss:0.61345
[986]	train-logloss:0.61342
[987]	train-logloss:0.61302
[988]	train-logloss:0.61285
[989]	train-logloss:0.61300
[990]	train-logloss:0.61285
[991]	train-logloss:0.61253
[992]	train-logloss:0.61262
[993]	train-logloss:0.61249
[994]	train-logloss:0.61250
[995]	train-logloss:0.61245
[996]	train-logloss:0.61260
[997]	train-logloss:0.61251
[998]	train-logloss:0.61306
[999]	train-logloss:0.61383
[1000]	train-logloss:0.61397
[1001]	train-logloss:0.61455
[1002]	train-logloss:0.61472
[1003]	train-logloss:0.61494
[1004]	train-logloss:0.61473
[1005]	train-logloss:0.61453
[1006]	train-logloss:0.61421
[1007]	train-logloss:0.61468
[1008]	train-logloss:0.61430
[1009]	train-logloss:0.61480
[1010]	train-logloss:0.61528
[1011]	train-logloss:0.61538
[1012]	train-logloss:0.61550
[1013]	train-logloss:0.61584
[1014]	train-logloss:0.61590
[1015]	train-logloss:0.61605
[1016]	train-logloss:0.61570
[1017]	train-logloss:0.61538
[1018]	train-logloss:0.61533
[1019]	train-logloss:0.61534
[1020]	train-logloss:0.61527
[1021]	train-logloss:0.61568
[1022]	train-logloss:0.61605
[1023]	train-logloss:0.61607
[1024]	train-logloss:0.61542
[1025]	train-logloss:0.61558
[1026]	train-logloss:0.61556
[1027]	train-logloss:0.61553
[1028]	train-logloss:0.61594
[1029]	train-logloss:0.61582
[1030]	train-logloss:0.61594
[1031]	train-logloss:0.61604
[1032]	train-logloss:0.61639
[1033]	train-logloss:0.61661
[1034]	train-logloss:0.61689
[1035]	train-logloss:0.61686
[1036]	train-logloss:0.61699
[1037]	train-logloss:0.61677
[1038]	train-logloss:0.61704
[1039]	train-logloss:0.61679
[1040]	train-logloss:0.61639
[1041]	train-logloss:0.61661
[1042]	train-logloss:0.61671
[1043]	train-logloss:0.61707
[1044]	train-logloss:0.61705
[1045]	train-logloss:0.61700
[1046]	train-logloss:0.61702
[1047]	train-logloss:0.61658
[1048]	train-logloss:0.61620
[1049]	train-logloss:0.61636
[1050]	train-logloss:0.61652
[1051]	train-logloss:0.61664
[1052]	train-logloss:0.61641
[1053]	train-logloss:0.61597
[1054]	train-logloss:0.61604
[1055]	train-logloss:0.61616
[1056]	train-logloss:0.61564
[1057]	train-logloss:0.61594
[1058]	train-logloss:0.61626
[1059]	train-logloss:0.61589
[1060]	train-logloss:0.61572
[1061]	train-logloss:0.61588
[1062]	train-logloss:0.61573
[1063]	train-logloss:0.61585
[1064]	train-logloss:0.61614
[1065]	train-logloss:0.61631
[1066]	train-logloss:0.61634
[1067]	train-logloss:0.61673
[1068]	train-logloss:0.61688
[1069]	train-logloss:0.61712
[1070]	train-logloss:0.61709
[1071]	train-logloss:0.61696
[1072]	train-logloss:0.61791
[1073]	train-logloss:0.61820
[1074]	train-logloss:0.61861
[1075]	train-logloss:0.61900
[1076]	train-logloss:0.61834
[1077]	train-logloss:0.61826
[1078]	train-logloss:0.61791
[1079]	train-logloss:0.61792
[1080]	train-logloss:0.61756
[1081]	train-logloss:0.61741
[1082]	train-logloss:0.61676
[1083]	train-logloss:0.61664
[1084]	train-logloss:0.61645
[1085]	train-logloss:0.61573
[1086]	train-logloss:0.61622
[1087]	train-logloss:0.61672
[1088]	train-logloss:0.61692
[1089]	train-logloss:0.61723
[1090]	train-logloss:0.61650
[1091]	train-logloss:0.61586
[1092]	train-logloss:0.61588
[1093]	train-logloss:0.61634
[1094]	train-logloss:0.61671
[1095]	train-logloss:0.61643
[1096]	train-logloss:0.61593
[1097]	train-logloss:0.61576
[1098]	train-logloss:0.61546
[1099]	train-logloss:0.61495
[1100]	train-logloss:0.61523
[1101]	train-logloss:0.61544
[1102]	train-logloss:0.61590
[1103]	train-logloss:0.61593
[1104]	train-logloss:0.61564
[1105]	train-logloss:0.61594
[1106]	train-logloss:0.61570
[1107]	train-logloss:0.61605
[1108]	train-logloss:0.61652
[1109]	train-logloss:0.61626
[1110]	train-logloss:0.61620
[1111]	train-logloss:0.61637
[1112]	train-logloss:0.61701
[1113]	train-logloss:0.61639
[1114]	train-logloss:0.61580
[1115]	train-logloss:0.61562
[1116]	train-logloss:0.61616
[1117]	train-logloss:0.61612
[1118]	train-logloss:0.61586
[1119]	train-logloss:0.61648
[1120]	train-logloss:0.61633
[1121]	train-logloss:0.61633
[1122]	train-logloss:0.61712
[1123]	train-logloss:0.61759
[1124]	train-logloss:0.61791
[1125]	train-logloss:0.61720
[1126]	train-logloss:0.61710
[1127]	train-logloss:0.61720
[1128]	train-logloss:0.61675
[1129]	train-logloss:0.61666
[1130]	train-logloss:0.61628
[1131]	train-logloss:0.61601
[1132]	train-logloss:0.61628
[1133]	train-logloss:0.61608
[1134]	train-logloss:0.61602
[1135]	train-logloss:0.61527
[1136]	train-logloss:0.61503
[1137]	train-logloss:0.61488
[1138]	train-logloss:0.61479
[1139]	train-logloss:0.61432
[1140]	train-logloss:0.61408
[1141]	train-logloss:0.61431
[1142]	train-logloss:0.61440
[1143]	train-logloss:0.61479
[1144]	train-logloss:0.61484
[1145]	train-logloss:0.61439
[1146]	train-logloss:0.61438
[1147]	train-logloss:0.61478
[1148]	train-logloss:0.61462
[1149]	train-logloss:0.61460
[1150]	train-logloss:0.61440
[1151]	train-logloss:0.61477
[1152]	train-logloss:0.61534
[1153]	train-logloss:0.61534
[1154]	train-logloss:0.61508
[1155]	train-logloss:0.61530
[1156]	train-logloss:0.61556
[1157]	train-logloss:0.61549
[1158]	train-logloss:0.61548
[1159]	train-logloss:0.61577
[1160]	train-logloss:0.61552
[1161]	train-logloss:0.61577
[1162]	train-logloss:0.61566
[1163]	train-logloss:0.61610
[1164]	train-logloss:0.61608
[1165]	train-logloss:0.61612
[1166]	train-logloss:0.61637
[1167]	train-logloss:0.61638
[1168]	train-logloss:0.61655
[1169]	train-logloss:0.61646
[1170]	train-logloss:0.61632
[1171]	train-logloss:0.61654
[1172]	train-logloss:0.61617
[1173]	train-logloss:0.61593
[1174]	train-logloss:0.61582
[1175]	train-logloss:0.61604
[1176]	train-logloss:0.61593
[1177]	train-logloss:0.61602
[1178]	train-logloss:0.61590
[1179]	train-logloss:0.61559
[1180]	train-logloss:0.61554
[1181]	train-logloss:0.61582
[1182]	train-logloss:0.61582
[1183]	train-logloss:0.61576
[1184]	train-logloss:0.61592
[1185]	train-logloss:0.61615
[1186]	train-logloss:0.61567
[1187]	train-logloss:0.61549
[1188]	train-logloss:0.61548
[1189]	train-logloss:0.61619
[1190]	train-logloss:0.61626
[1191]	train-logloss:0.61679
[1192]	train-logloss:0.61673
[1193]	train-logloss:0.61731
[1194]	train-logloss:0.61746
[1195]	train-logloss:0.61761
[1196]	train-logloss:0.61761
[1197]	train-logloss:0.61751
[1198]	train-logloss:0.61805
[1199]	train-logloss:0.61834
[1200]	train-logloss:0.61812
[1201]	train-logloss:0.61811
[1202]	train-logloss:0.61823
[1203]	train-logloss:0.61798
[1204]	train-logloss:0.61777
[1205]	train-logloss:0.61818
[1206]	train-logloss:0.61818
[1207]	train-logloss:0.61824
[1208]	train-logloss:0.61831
[1209]	train-logloss:0.61811
[1210]	train-logloss:0.61812
[1211]	train-logloss:0.61833
[1212]	train-logloss:0.61835
[1213]	train-logloss:0.61837
[1214]	train-logloss:0.61841
[1215]	train-logloss:0.61840
[1216]	train-logloss:0.61836
[1217]	train-logloss:0.61805
[1218]	train-logloss:0.61808
[1219]	train-logloss:0.61835
[1220]	train-logloss:0.61845
[1221]	train-logloss:0.61870
[1222]	train-logloss:0.61850
[1223]	train-logloss:0.61854
[1224]	train-logloss:0.61863
[1225]	train-logloss:0.61899
[1226]	train-logloss:0.61892
[1227]	train-logloss:0.61846
[1228]	train-logloss:0.61747
[1229]	train-logloss:0.61741
[1230]	train-logloss:0.61723
[1231]	train-logloss:0.61720
[1232]	train-logloss:0.61760
[1233]	train-logloss:0.61721
[1234]	train-logloss:0.61750
[1235]	train-logloss:0.61749
[1236]	train-logloss:0.61791
[1237]	train-logloss:0.61784
[1238]	train-logloss:0.61782
[1239]	train-logloss:0.61761
[1240]	train-logloss:0.61788
[1241]	train-logloss:0.61803
[1242]	train-logloss:0.61798
[1243]	train-logloss:0.61792
[1244]	train-logloss:0.61842
[1245]	train-logloss:0.61798
[1246]	train-logloss:0.61819
[1247]	train-logloss:0.61888
[1248]	train-logloss:0.61904
[1249]	train-logloss:0.61933
[1250]	train-logloss:0.61934
[1251]	train-logloss:0.61989
[1252]	train-logloss:0.61986
[1253]	train-logloss:0.61987
[1254]	train-logloss:0.62028
[1255]	train-logloss:0.62067
[1256]	train-logloss:0.62057
[1257]	train-logloss:0.62052
[1258]	train-logloss:0.62099
[1259]	train-logloss:0.62093
[1260]	train-logloss:0.62084
[1261]	train-logloss:0.62128
[1262]	train-logloss:0.62201
[1263]	train-logloss:0.62241
[1264]	train-logloss:0.62245
[1265]	train-logloss:0.62252
[1266]	train-logloss:0.62243
[1267]	train-logloss:0.62244
[1268]	train-logloss:0.62245
[1269]	train-logloss:0.62248
[1270]	train-logloss:0.62249
[1271]	train-logloss:0.62313
[1272]	train-logloss:0.62362
[1273]	train-logloss:0.62363
[1274]	train-logloss:0.62333
[1275]	train-logloss:0.62393
[1276]	train-logloss:0.62373
[1277]	train-logloss:0.62412
[1278]	train-logloss:0.62350
[1279]	train-logloss:0.62284
[1280]	train-logloss:0.62233
[1281]	train-logloss:0.62190
[1282]	train-logloss:0.62219
[1283]	train-logloss:0.62188
[1284]	train-logloss:0.62152
[1285]	train-logloss:0.62160
[1286]	train-logloss:0.62161
[1287]	train-logloss:0.62144
[1288]	train-logloss:0.62174
[1289]	train-logloss:0.62205
[1290]	train-logloss:0.62258
[1291]	train-logloss:0.62214
[1292]	train-logloss:0.62211
[1293]	train-logloss:0.62220
[1294]	train-logloss:0.62162
[1295]	train-logloss:0.62190
[1296]	train-logloss:0.62167
[1297]	train-logloss:0.62130
[1298]	train-logloss:0.62131
[1299]	train-logloss:0.62069
[1300]	train-logloss:0.62077
[1301]	train-logloss:0.62085
[1302]	train-logloss:0.62065
[1303]	train-logloss:0.62093
[1304]	train-logloss:0.62098
[1305]	train-logloss:0.62133
[1306]	train-logloss:0.62180
[1307]	train-logloss:0.62205
[1308]	train-logloss:0.62153
[1309]	train-logloss:0.62135
[1310]	train-logloss:0.62109
[1311]	train-logloss:0.62135
[1312]	train-logloss:0.62126
[1313]	train-logloss:0.62143
[1314]	train-logloss:0.62136
[1315]	train-logloss:0.62137
[1316]	train-logloss:0.62184
[1317]	train-logloss:0.62164
[1318]	train-logloss:0.62177
[1319]	train-logloss:0.62198
[1320]	train-logloss:0.62296
[1321]	train-logloss:0.62289
[1322]	train-logloss:0.62195
[1323]	train-logloss:0.62224
[1324]	train-logloss:0.62239
[1325]	train-logloss:0.62226
[1326]	train-logloss:0.62231
[1327]	train-logloss:0.62226
[1328]	train-logloss:0.62208
[1329]	train-logloss:0.62160
[1330]	train-logloss:0.62211
[1331]	train-logloss:0.62208
[1332]	train-logloss:0.62155
[1333]	train-logloss:0.62138
[1334]	train-logloss:0.62145
[1335]	train-logloss:0.62141
[1336]	train-logloss:0.62144
[1337]	train-logloss:0.62210
[1338]	train-logloss:0.62197
[1339]	train-logloss:0.62169
[1340]	train-logloss:0.62142
[1341]	train-logloss:0.62128
[1342]	train-logloss:0.62129
[1343]	train-logloss:0.62180
[1344]	train-logloss:0.62237
[1345]	train-logloss:0.62215
[1346]	train-logloss:0.62250
[1347]	train-logloss:0.62197
[1348]	train-logloss:0.62196
[1349]	train-logloss:0.62166
[1350]	train-logloss:0.62169
[1351]	train-logloss:0.62127
[1352]	train-logloss:0.62157
[1353]	train-logloss:0.62163
[1354]	train-logloss:0.62116
[1355]	train-logloss:0.62129
[1356]	train-logloss:0.62164
[1357]	train-logloss:0.62179
[1358]	train-logloss:0.62193
[1359]	train-logloss:0.62255
[1360]	train-logloss:0.62253
[1361]	train-logloss:0.62186
[1362]	train-logloss:0.62189
[1363]	train-logloss:0.62179
[1364]	train-logloss:0.62182
[1365]	train-logloss:0.62170
[1366]	train-logloss:0.62147
[1367]	train-logloss:0.62138
[1368]	train-logloss:0.62146
[1369]	train-logloss:0.62147
[1370]	train-logloss:0.62220
[1371]	train-logloss:0.62200
[1372]	train-logloss:0.62165
[1373]	train-logloss:0.62146
[1374]	train-logloss:0.62162
[1375]	train-logloss:0.62167
[1376]	train-logloss:0.62154
[1377]	train-logloss:0.62150
[1378]	train-logloss:0.62163
[1379]	train-logloss:0.62158
[1380]	train-logloss:0.62126
[1381]	train-logloss:0.62109
[1382]	train-logloss:0.62034
[1383]	train-logloss:0.62063
[1384]	train-logloss:0.61993
[1385]	train-logloss:0.62037
[1386]	train-logloss:0.62061
[1387]	train-logloss:0.62109
[1388]	train-logloss:0.62067
[1389]	train-logloss:0.62111
[1390]	train-logloss:0.62117
[1391]	train-logloss:0.62114
[1392]	train-logloss:0.62100
[1393]	train-logloss:0.62126
[1394]	train-logloss:0.62121
[1395]	train-logloss:0.62034
[1396]	train-logloss:0.62015
[1397]	train-logloss:0.61977
[1398]	train-logloss:0.61984
[1399]	train-logloss:0.61980
[1400]	train-logloss:0.62001
[1401]	train-logloss:0.62021
[1402]	train-logloss:0.61998
[1403]	train-logloss:0.61985
[1404]	train-logloss:0.62000
[1405]	train-logloss:0.61983
[1406]	train-logloss:0.62019
[1407]	train-logloss:0.62021
[1408]	train-logloss:0.62011
[1409]	train-logloss:0.62013
[1410]	train-logloss:0.62020
[1411]	train-logloss:0.62035
[1412]	train-logloss:0.62013
[1413]	train-logloss:0.62051
[1414]	train-logloss:0.62023
[1415]	train-logloss:0.61969
[1416]	train-logloss:0.61964
[1417]	train-logloss:0.62012
[1418]	train-logloss:0.61977
[1419]	train-logloss:0.62004
[1420]	train-logloss:0.61985
[1421]	train-logloss:0.62022
[1422]	train-logloss:0.62018
[1423]	train-logloss:0.62115
[1424]	train-logloss:0.62131
[1425]	train-logloss:0.62105
[1426]	train-logloss:0.62091
[1427]	train-logloss:0.62092
[1428]	train-logloss:0.62157
[1429]	train-logloss:0.62142
[1430]	train-logloss:0.62116
[1431]	train-logloss:0.62139
[1432]	train-logloss:0.62133
[1433]	train-logloss:0.62163
[1434]	train-logloss:0.62205
[1435]	train-logloss:0.62173
[1436]	train-logloss:0.62203
[1437]	train-logloss:0.62223
[1438]	train-logloss:0.62139
[1439]	train-logloss:0.62153
[1440]	train-logloss:0.62179
[1441]	train-logloss:0.62182
[1442]	train-logloss:0.62184
[1443]	train-logloss:0.62181
[1444]	train-logloss:0.62181
[1445]	train-logloss:0.62172
[1446]	train-logloss:0.62191
[1447]	train-logloss:0.62234
[1448]	train-logloss:0.62249
[1449]	train-logloss:0.62289
[1450]	train-logloss:0.62289
[1451]	train-logloss:0.62240
[1452]	train-logloss:0.62203
[1453]	train-logloss:0.62179
[1454]	train-logloss:0.62148
[1455]	train-logloss:0.62208
[1456]	train-logloss:0.62211
[1457]	train-logloss:0.62210
[1458]	train-logloss:0.62212
[1459]	train-logloss:0.62243
[1460]	train-logloss:0.62176
[1461]	train-logloss:0.62173
[1462]	train-logloss:0.62273
[1463]	train-logloss:0.62274
[1464]	train-logloss:0.62264
[1465]	train-logloss:0.62251
[1466]	train-logloss:0.62216
[1467]	train-logloss:0.62178
[1468]	train-logloss:0.62177
[1469]	train-logloss:0.62123
[1470]	train-logloss:0.62158
[1471]	train-logloss:0.62149
[1472]	train-logloss:0.62120
[1473]	train-logloss:0.62089
[1474]	train-logloss:0.62088
[1475]	train-logloss:0.62042
[1476]	train-logloss:0.62060
[1477]	train-logloss:0.62094
[1478]	train-logloss:0.62070
[1479]	train-logloss:0.62138
[1480]	train-logloss:0.62191
[1481]	train-logloss:0.62263
[1482]	train-logloss:0.62314
[1483]	train-logloss:0.62297
[1484]	train-logloss:0.62304
[1485]	train-logloss:0.62302
[1486]	train-logloss:0.62320
[1487]	train-logloss:0.62371
[1488]	train-logloss:0.62408
[1489]	train-logloss:0.62425
[1490]	train-logloss:0.62483
[1491]	train-logloss:0.62470
[1492]	train-logloss:0.62468
[1493]	train-logloss:0.62445
[1494]	train-logloss:0.62364
[1495]	train-logloss:0.62281
[1496]	train-logloss:0.62235
[1497]	train-logloss:0.62246
[1498]	train-logloss:0.62299
[1499]	train-logloss:0.62292
[1500]	train-logloss:0.62292
[1501]	train-logloss:0.62397
[1502]	train-logloss:0.62421
[1503]	train-logloss:0.62474
[1504]	train-logloss:0.62482
[1505]	train-logloss:0.62449
[1506]	train-logloss:0.62440
[1507]	train-logloss:0.62389
[1508]	train-logloss:0.62370
[1509]	train-logloss:0.62357
[1510]	train-logloss:0.62330
[1511]	train-logloss:0.62317
[1512]	train-logloss:0.62402
[1513]	train-logloss:0.62354
[1514]	train-logloss:0.62335
[1515]	train-logloss:0.62294
[1516]	train-logloss:0.62292
[1517]	train-logloss:0.62292
[1518]	train-logloss:0.62291
[1519]	train-logloss:0.62241
[1520]	train-logloss:0.62281
[1521]	train-logloss:0.62292
[1522]	train-logloss:0.62264
[1523]	train-logloss:0.62284
[1524]	train-logloss:0.62344
[1525]	train-logloss:0.62342
[1526]	train-logloss:0.62341
[1527]	train-logloss:0.62322
[1528]	train-logloss:0.62380
[1529]	train-logloss:0.62396
[1530]	train-logloss:0.62362
[1531]	train-logloss:0.62355
[1532]	train-logloss:0.62339
[1533]	train-logloss:0.62331
[1534]	train-logloss:0.62320
[1535]	train-logloss:0.62286
[1536]	train-logloss:0.62317
[1537]	train-logloss:0.62443
[1538]	train-logloss:0.62493
[1539]	train-logloss:0.62527
[1540]	train-logloss:0.62483
[1541]	train-logloss:0.62509
[1542]	train-logloss:0.62480
[1543]	train-logloss:0.62506
[1544]	train-logloss:0.62635
[1545]	train-logloss:0.62708
[1546]	train-logloss:0.62721
[1547]	train-logloss:0.62686
[1548]	train-logloss:0.62723
[1549]	train-logloss:0.62748
[1550]	train-logloss:0.62745
[1551]	train-logloss:0.62808
[1552]	train-logloss:0.62749
[1553]	train-logloss:0.62703
[1554]	train-logloss:0.62705
[1555]	train-logloss:0.62714
[1556]	train-logloss:0.62733
[1557]	train-logloss:0.62796
[1558]	train-logloss:0.62826
[1559]	train-logloss:0.62826
[1560]	train-logloss:0.62829
[1561]	train-logloss:0.62839
[1562]	train-logloss:0.62812
[1563]	train-logloss:0.62794
[1564]	train-logloss:0.62794
[1565]	train-logloss:0.62733
[1566]	train-logloss:0.62713
[1567]	train-logloss:0.62760
[1568]	train-logloss:0.62765
[1569]	train-logloss:0.62734
[1570]	train-logloss:0.62715
[1571]	train-logloss:0.62716
[1572]	train-logloss:0.62697
[1573]	train-logloss:0.62685
[1574]	train-logloss:0.62616
[1575]	train-logloss:0.62604
[1576]	train-logloss:0.62584
[1577]	train-logloss:0.62552
[1578]	train-logloss:0.62563
[1579]	train-logloss:0.62520
[1580]	train-logloss:0.62522
[1581]	train-logloss:0.62523
[1582]	train-logloss:0.62511
[1583]	train-logloss:0.62505
[1584]	train-logloss:0.62541
[1585]	train-logloss:0.62588
[1586]	train-logloss:0.62578
[1587]	train-logloss:0.62553
[1588]	train-logloss:0.62557
[1589]	train-logloss:0.62467
[1590]	train-logloss:0.62473
[1591]	train-logloss:0.62508
[1592]	train-logloss:0.62497
[1593]	train-logloss:0.62453
[1594]	train-logloss:0.62384
[1595]	train-logloss:0.62420
[1596]	train-logloss:0.62446
[1597]	train-logloss:0.62479
[1598]	train-logloss:0.62449
[1599]	train-logloss:0.62449
[1600]	train-logloss:0.62423
[1601]	train-logloss:0.62411
[1602]	train-logloss:0.62388
[1603]	train-logloss:0.62411
[1604]	train-logloss:0.62443
[1605]	train-logloss:0.62469
[1606]	train-logloss:0.62507
[1607]	train-logloss:0.62572
[1608]	train-logloss:0.62554
[1609]	train-logloss:0.62555
[1610]	train-logloss:0.62558
[1611]	train-logloss:0.62570
[1612]	train-logloss:0.62653
[1613]	train-logloss:0.62706
[1614]	train-logloss:0.62691
[1615]	train-logloss:0.62700
[1616]	train-logloss:0.62672
[1617]	train-logloss:0.62688
[1618]	train-logloss:0.62700
[1619]	train-logloss:0.62699
[1620]	train-logloss:0.62742
[1621]	train-logloss:0.62767
[1622]	train-logloss:0.62734
[1623]	train-logloss:0.62717
[1624]	train-logloss:0.62756
[1625]	train-logloss:0.62705
[1626]	train-logloss:0.62695
[1627]	train-logloss:0.62633
[1628]	train-logloss:0.62619
[1629]	train-logloss:0.62691
[1630]	train-logloss:0.62652
[1631]	train-logloss:0.62642
[1632]	train-logloss:0.62627
[1633]	train-logloss:0.62633
[1634]	train-logloss:0.62699
[1635]	train-logloss:0.62705
[1636]	train-logloss:0.62704
[1637]	train-logloss:0.62736
[1638]	train-logloss:0.62731
[1639]	train-logloss:0.62708
[1640]	train-logloss:0.62668
[1641]	train-logloss:0.62663
[1642]	train-logloss:0.62660
[1643]	train-logloss:0.62673
[1644]	train-logloss:0.62695
[1645]	train-logloss:0.62719
[1646]	train-logloss:0.62804
[1647]	train-logloss:0.62804
[1648]	train-logloss:0.62861
[1649]	train-logloss:0.62823
[1650]	train-logloss:0.62817
[1651]	train-logloss:0.62793
[1652]	train-logloss:0.62743
[1653]	train-logloss:0.62737
[1654]	train-logloss:0.62774
[1655]	train-logloss:0.62777
[1656]	train-logloss:0.62778
[1657]	train-logloss:0.62840
[1658]	train-logloss:0.62773
[1659]	train-logloss:0.62748
[1660]	train-logloss:0.62749
[1661]	train-logloss:0.62737
[1662]	train-logloss:0.62715
[1663]	train-logloss:0.62719
[1664]	train-logloss:0.62730
[1665]	train-logloss:0.62723
[1666]	train-logloss:0.62722
[1667]	train-logloss:0.62713
[1668]	train-logloss:0.62705
[1669]	train-logloss:0.62717
[1670]	train-logloss:0.62800
[1671]	train-logloss:0.62689
[1672]	train-logloss:0.62649
[1673]	train-logloss:0.62711
[1674]	train-logloss:0.62687
[1675]	train-logloss:0.62650
[1676]	train-logloss:0.62633
[1677]	train-logloss:0.62623
[1678]	train-logloss:0.62646
[1679]	train-logloss:0.62636
[1680]	train-logloss:0.62612
[1681]	train-logloss:0.62655
[1682]	train-logloss:0.62635
[1683]	train-logloss:0.62605
[1684]	train-logloss:0.62646
[1685]	train-logloss:0.62708
[1686]	train-logloss:0.62742
[1687]	train-logloss:0.62785
[1688]	train-logloss:0.62789
[1689]	train-logloss:0.62822
[1690]	train-logloss:0.62799
[1691]	train-logloss:0.62868
[1692]	train-logloss:0.62901
[1693]	train-logloss:0.62901
[1694]	train-logloss:0.62914
[1695]	train-logloss:0.62889
[1696]	train-logloss:0.62889
[1697]	train-logloss:0.62943
[1698]	train-logloss:0.63000
[1699]	train-logloss:0.63004
[1700]	train-logloss:0.63026
[1701]	train-logloss:0.63075
[1702]	train-logloss:0.63076
[1703]	train-logloss:0.63148
[1704]	train-logloss:0.63152
[1705]	train-logloss:0.63151
[1706]	train-logloss:0.63170
[1707]	train-logloss:0.63178
[1708]	train-logloss:0.63160
[1709]	train-logloss:0.63154
[1710]	train-logloss:0.63216
[1711]	train-logloss:0.63176
[1712]	train-logloss:0.63144
[1713]	train-logloss:0.63144
[1714]	train-logloss:0.63135
[1715]	train-logloss:0.63146
[1716]	train-logloss:0.63145
[1717]	train-logloss:0.63156
[1718]	train-logloss:0.63085
[1719]	train-logloss:0.63143
[1720]	train-logloss:0.63115
[1721]	train-logloss:0.63196
[1722]	train-logloss:0.63176
[1723]	train-logloss:0.63173
[1724]	train-logloss:0.63226
[1725]	train-logloss:0.63247
[1726]	train-logloss:0.63249
[1727]	train-logloss:0.63195
[1728]	train-logloss:0.63201
[1729]	train-logloss:0.63176
[1730]	train-logloss:0.63183
[1731]	train-logloss:0.63172
[1732]	train-logloss:0.63126
[1733]	train-logloss:0.63168
[1734]	train-logloss:0.63187
[1735]	train-logloss:0.63190
[1736]	train-logloss:0.63153
[1737]	train-logloss:0.63155
[1738]	train-logloss:0.63142
[1739]	train-logloss:0.63193
[1740]	train-logloss:0.63273
[1741]	train-logloss:0.63286
[1742]	train-logloss:0.63302
[1743]	train-logloss:0.63290
[1744]	train-logloss:0.63289
[1745]	train-logloss:0.63304
[1746]	train-logloss:0.63250
[1747]	train-logloss:0.63258
[1748]	train-logloss:0.63193
[1749]	train-logloss:0.63185
[1750]	train-logloss:0.63220
[1751]	train-logloss:0.63252
[1752]	train-logloss:0.63256
[1753]	train-logloss:0.63246
[1754]	train-logloss:0.63227
[1755]	train-logloss:0.63293
[1756]	train-logloss:0.63271
[1757]	train-logloss:0.63324
[1758]	train-logloss:0.63333
[1759]	train-logloss:0.63328
[1760]	train-logloss:0.63319
[1761]	train-logloss:0.63320
[1762]	train-logloss:0.63344
[1763]	train-logloss:0.63367
[1764]	train-logloss:0.63343
[1765]	train-logloss:0.63387
[1766]	train-logloss:0.63410
[1767]	train-logloss:0.63494
[1768]	train-logloss:0.63479
[1769]	train-logloss:0.63492
[1770]	train-logloss:0.63518
[1771]	train-logloss:0.63433
[1772]	train-logloss:0.63369
[1773]	train-logloss:0.63367
[1774]	train-logloss:0.63371
[1775]	train-logloss:0.63405
[1776]	train-logloss:0.63410
[1777]	train-logloss:0.63479
[1778]	train-logloss:0.63420
[1779]	train-logloss:0.63421
[1780]	train-logloss:0.63344
[1781]	train-logloss:0.63337
[1782]	train-logloss:0.63343
[1783]	train-logloss:0.63341
[1784]	train-logloss:0.63357
[1785]	train-logloss:0.63359
[1786]	train-logloss:0.63375
[1787]	train-logloss:0.63367
[1788]	train-logloss:0.63314
[1789]	train-logloss:0.63308
[1790]	train-logloss:0.63310
[1791]	train-logloss:0.63399
[1792]	train-logloss:0.63392
[1793]	train-logloss:0.63406
[1794]	train-logloss:0.63405
[1795]	train-logloss:0.63456
[1796]	train-logloss:0.63486
[1797]	train-logloss:0.63499
[1798]	train-logloss:0.63507
[1799]	train-logloss:0.63509
[1800]	train-logloss:0.63491
[1801]	train-logloss:0.63487
[1802]	train-logloss:0.63536
[1803]	train-logloss:0.63584
[1804]	train-logloss:0.63591
[1805]	train-logloss:0.63588
[1806]	train-logloss:0.63546
[1807]	train-logloss:0.63529
[1808]	train-logloss:0.63565
[1809]	train-logloss:0.63558
[1810]	train-logloss:0.63572
[1811]	train-logloss:0.63561
[1812]	train-logloss:0.63598
[1813]	train-logloss:0.63634
[1814]	train-logloss:0.63634
[1815]	train-logloss:0.63663
[1816]	train-logloss:0.63615
[1817]	train-logloss:0.63646
[1818]	train-logloss:0.63635
[1819]	train-logloss:0.63620
[1820]	train-logloss:0.63593
[1821]	train-logloss:0.63538
[1822]	train-logloss:0.63517
[1823]	train-logloss:0.63479
[1824]	train-logloss:0.63480
[1825]	train-logloss:0.63417
[1826]	train-logloss:0.63417
[1827]	train-logloss:0.63357
[1828]	train-logloss:0.63291
[1829]	train-logloss:0.63237
[1830]	train-logloss:0.63229
[1831]	train-logloss:0.63239
[1832]	train-logloss:0.63236
[1833]	train-logloss:0.63249
[1834]	train-logloss:0.63265
[1835]	train-logloss:0.63263
[1836]	train-logloss:0.63264
[1837]	train-logloss:0.63256
[1838]	train-logloss:0.63256
[1839]	train-logloss:0.63252
[1840]	train-logloss:0.63264
[1841]	train-logloss:0.63257
[1842]	train-logloss:0.63266
[1843]	train-logloss:0.63267
[1844]	train-logloss:0.63223
[1845]	train-logloss:0.63223
[1846]	train-logloss:0.63218
[1847]	train-logloss:0.63234
[1848]	train-logloss:0.63234
[1849]	train-logloss:0.63235
[1850]	train-logloss:0.63175
[1851]	train-logloss:0.63204
[1852]	train-logloss:0.63210
[1853]	train-logloss:0.63177
[1854]	train-logloss:0.63243
[1855]	train-logloss:0.63226
[1856]	train-logloss:0.63271
[1857]	train-logloss:0.63206
[1858]	train-logloss:0.63206
[1859]	train-logloss:0.63191
[1860]	train-logloss:0.63220
[1861]	train-logloss:0.63236
[1862]	train-logloss:0.63214
[1863]	train-logloss:0.63248
[1864]	train-logloss:0.63216
[1865]	train-logloss:0.63245
[1866]	train-logloss:0.63247
[1867]	train-logloss:0.63262
[1868]	train-logloss:0.63261
[1869]	train-logloss:0.63266
[1870]	train-logloss:0.63278
[1871]	train-logloss:0.63256
[1872]	train-logloss:0.63322
[1873]	train-logloss:0.63320
[1874]	train-logloss:0.63290
[1875]	train-logloss:0.63291
[1876]	train-logloss:0.63290
[1877]	train-logloss:0.63275
[1878]	train-logloss:0.63277
[1879]	train-logloss:0.63280
[1880]	train-logloss:0.63254
[1881]	train-logloss:0.63225
[1882]	train-logloss:0.63286
[1883]	train-logloss:0.63271
[1884]	train-logloss:0.63270
[1885]	train-logloss:0.63268
[1886]	train-logloss:0.63268
[1887]	train-logloss:0.63276
[1888]	train-logloss:0.63250
[1889]	train-logloss:0.63276
[1890]	train-logloss:0.63270
[1891]	train-logloss:0.63247
[1892]	train-logloss:0.63222
[1893]	train-logloss:0.63252
[1894]	train-logloss:0.63280
[1895]	train-logloss:0.63284
[1896]	train-logloss:0.63253
[1897]	train-logloss:0.63241
[1898]	train-logloss:0.63218
[1899]	train-logloss:0.63219
[1900]	train-logloss:0.63192
[1901]	train-logloss:0.63223
[1902]	train-logloss:0.63201
[1903]	train-logloss:0.63173
[1904]	train-logloss:0.63202
[1905]	train-logloss:0.63222
[1906]	train-logloss:0.63181
[1907]	train-logloss:0.63178
[1908]	train-logloss:0.63213
[1909]	train-logloss:0.63178
[1910]	train-logloss:0.63225
[1911]	train-logloss:0.63274
[1912]	train-logloss:0.63294
[1913]	train-logloss:0.63338
[1914]	train-logloss:0.63338
[1915]	train-logloss:0.63338
[1916]	train-logloss:0.63341
[1917]	train-logloss:0.63340
[1918]	train-logloss:0.63349
[1919]	train-logloss:0.63310
[1920]	train-logloss:0.63315
[1921]	train-logloss:0.63328
[1922]	train-logloss:0.63319
[1923]	train-logloss:0.63287
[1924]	train-logloss:0.63251
[1925]	train-logloss:0.63272
[1926]	train-logloss:0.63240
[1927]	train-logloss:0.63280
[1928]	train-logloss:0.63241
[1929]	train-logloss:0.63241
[1930]	train-logloss:0.63241
[1931]	train-logloss:0.63229
[1932]	train-logloss:0.63205
[1933]	train-logloss:0.63170
[1934]	train-logloss:0.63269
[1935]	train-logloss:0.63312
[1936]	train-logloss:0.63253
[1937]	train-logloss:0.63222
[1938]	train-logloss:0.63223
[1939]	train-logloss:0.63224
[1940]	train-logloss:0.63252
[1941]	train-logloss:0.63260
[1942]	train-logloss:0.63329
[1943]	train-logloss:0.63331
[1944]	train-logloss:0.63432
[1945]	train-logloss:0.63457
[1946]	train-logloss:0.63454
[1947]	train-logloss:0.63421
[1948]	train-logloss:0.63418
[1949]	train-logloss:0.63412
[1950]	train-logloss:0.63373
[1951]	train-logloss:0.63307
[1952]	train-logloss:0.63306
[1953]	train-logloss:0.63307
[1954]	train-logloss:0.63296
[1955]	train-logloss:0.63289
[1956]	train-logloss:0.63286
[1957]	train-logloss:0.63286
[1958]	train-logloss:0.63286
[1959]	train-logloss:0.63268
[1960]	train-logloss:0.63289
[1961]	train-logloss:0.63299
[1962]	train-logloss:0.63288
[1963]	train-logloss:0.63288
[1964]	train-logloss:0.63280
[1965]	train-logloss:0.63254
[1966]	train-logloss:0.63272
[1967]	train-logloss:0.63287
[1968]	train-logloss:0.63327
[1969]	train-logloss:0.63324
[1970]	train-logloss:0.63324
[1971]	train-logloss:0.63336
[1972]	train-logloss:0.63382
[1973]	train-logloss:0.63386
[1974]	train-logloss:0.63427
[1975]	train-logloss:0.63428
[1976]	train-logloss:0.63462
[1977]	train-logloss:0.63443
[1978]	train-logloss:0.63445
[1979]	train-logloss:0.63453
[1980]	train-logloss:0.63466
[1981]	train-logloss:0.63527
[1982]	train-logloss:0.63546
[1983]	train-logloss:0.63513
[1984]	train-logloss:0.63484
[1985]	train-logloss:0.63482
[1986]	train-logloss:0.63484
[1987]	train-logloss:0.63513
[1988]	train-logloss:0.63536
[1989]	train-logloss:0.63516
[1990]	train-logloss:0.63468
[1991]	train-logloss:0.63452
[1992]	train-logloss:0.63448
[1993]	train-logloss:0.63460
[1994]	train-logloss:0.63451
[1995]	train-logloss:0.63422
[1996]	train-logloss:0.63409
[1997]	train-logloss:0.63412
[1998]	train-logloss:0.63406
[1999]	train-logloss:0.63402

step-04 用模型预测

ytrain=model.predict(dtrain)

注意：

这里model.predict()预测得到的是概率值，而不是0或者1的结果
下面将结果转换为0或者1

ytrain_class = (ytrain>= 0.5)*1

ytest=model.predict(dtest)
y_pred = (ytest >= 0.5)*1

step-05 评价模型效果

print(‘train_roc_auc_score:’,metrics.roc_auc_score(y_train,ytrain))
print(‘test_roc_auc_score:’,metrics.roc_auc_score(y_test, ytest))
print(‘train_accuracy_score:’,metrics.accuracy_score(y_train, ytrain_class))
print(‘test_accuracy_score:’,metrics.accuracy_score(y_test,y_pred ))

step-06 保存模型并调用

joblib.dump(model , r'D:\Ensemble_Learning\xgboostinfo\xgboost01.model')
load_model=joblib.load(r'D:\Ensemble_Learning\xgboostinfo\xgboost01.model')
ytest=load_model.predict(dtest)
ytest[0:5]

array([0.265046  , 0.39359182, 0.82298654, 0.07664716, 0.28468448],
      dtype=float32)

三. 网格搜索最优xgboost参数

1.step-01 配置参数列表

from sklearn.model_selection import GridSearchCV
## 定义参数取值范围
learning_rate = [0.1] #0.15,0.11
subsample = [ 0.65] #0.7,0.8
colsample_bytree = [0.6] #0.7, 0.5
colsample_bylevel=[0.7] #0.8,
colsample_bynode=[0.7] #0.8,
max_depth = [6] #,7
n_estimators=[1000] #,900
gamma=[0,0.1]
reg_alpha=[1,2]
reg_lambda=[2,3]
min_child_weight=[30,50]
max_bin=[12,16]
base_score=[0.4,0.5,0.6]

parameters = { 
              'learning_rate': learning_rate,
              'subsample': subsample,
              'colsample_bytree':colsample_bytree,
              'colsample_bylevel':colsample_bylevel,
              'colsample_bynode':colsample_bynode,
              'max_depth': max_depth,
              'n_estimators':n_estimators,
               'gamma':gamma,
               'reg_alpha':reg_alpha,
               'reg_lambda':reg_lambda,
               'min_child_weight':min_child_weight,
               'max_bin':max_bin,
               'base_score':base_score,
              }

step-02 选择待优化模型

model = XGBClassifier( eval_metric="logloss")

step-03 进行网格搜索拟合模型

clf = GridSearchCV(model, parameters, cv=2, scoring='accuracy',verbose=1,n_jobs=-1)
clf = clf.fit(train.values, y_train.values,eval_set=eval_set)

Fitting 2 folds for each of 96 candidates, totalling 192 fits
[0]	validation_0-logloss:0.68822
[1]	validation_0-logloss:0.68488
[2]	validation_0-logloss:0.67979
[3]	validation_0-logloss:0.67770
[4]	validation_0-logloss:0.67431
[5]	validation_0-logloss:0.67095
[6]	validation_0-logloss:0.66894
[7]	validation_0-logloss:0.66736
[8]	validation_0-logloss:0.66269
[9]	validation_0-logloss:0.65911
[10]	validation_0-logloss:0.65691
[11]	validation_0-logloss:0.65429
[12]	validation_0-logloss:0.64994
[13]	validation_0-logloss:0.64843
[14]	validation_0-logloss:0.64748
[15]	validation_0-logloss:0.64628
[16]	validation_0-logloss:0.64424
[17]	validation_0-logloss:0.64260
[18]	validation_0-logloss:0.64172
[19]	validation_0-logloss:0.64020
[20]	validation_0-logloss:0.63933
[21]	validation_0-logloss:0.63795
[22]	validation_0-logloss:0.63296
[23]	validation_0-logloss:0.63192
[24]	validation_0-logloss:0.63157
[25]	validation_0-logloss:0.63006
[26]	validation_0-logloss:0.62925
[27]	validation_0-logloss:0.62915
[28]	validation_0-logloss:0.62914
[29]	validation_0-logloss:0.62940
[30]	validation_0-logloss:0.62872
[31]	validation_0-logloss:0.62866
[32]	validation_0-logloss:0.62860
[33]	validation_0-logloss:0.62812
[34]	validation_0-logloss:0.62823
[35]	validation_0-logloss:0.62819
[36]	validation_0-logloss:0.62489
[37]	validation_0-logloss:0.62490
[38]	validation_0-logloss:0.62293
[39]	validation_0-logloss:0.62222
[40]	validation_0-logloss:0.62102
[41]	validation_0-logloss:0.61937
[42]	validation_0-logloss:0.61839
[43]	validation_0-logloss:0.61829
[44]	validation_0-logloss:0.61782
[45]	validation_0-logloss:0.61781
[46]	validation_0-logloss:0.61763
[47]	validation_0-logloss:0.61733
[48]	validation_0-logloss:0.61704
[49]	validation_0-logloss:0.61602
[50]	validation_0-logloss:0.61585
[51]	validation_0-logloss:0.61632
[52]	validation_0-logloss:0.61601
[53]	validation_0-logloss:0.61658
[54]	validation_0-logloss:0.61598
[55]	validation_0-logloss:0.61581
[56]	validation_0-logloss:0.61530
[57]	validation_0-logloss:0.61455
[58]	validation_0-logloss:0.61557
[59]	validation_0-logloss:0.61533
[60]	validation_0-logloss:0.61390
[61]	validation_0-logloss:0.61426
[62]	validation_0-logloss:0.61365
[63]	validation_0-logloss:0.61269
[64]	validation_0-logloss:0.61244
[65]	validation_0-logloss:0.61196
[66]	validation_0-logloss:0.61196
[67]	validation_0-logloss:0.61175
[68]	validation_0-logloss:0.61179
[69]	validation_0-logloss:0.61195
[70]	validation_0-logloss:0.61165
[71]	validation_0-logloss:0.61130
[72]	validation_0-logloss:0.61112
[73]	validation_0-logloss:0.61133
[74]	validation_0-logloss:0.61152
[75]	validation_0-logloss:0.61118
[76]	validation_0-logloss:0.61160
[77]	validation_0-logloss:0.61167
[78]	validation_0-logloss:0.61175
[79]	validation_0-logloss:0.61156
[80]	validation_0-logloss:0.61164
[81]	validation_0-logloss:0.61126
[82]	validation_0-logloss:0.61166
[83]	validation_0-logloss:0.61163
[84]	validation_0-logloss:0.61156
[85]	validation_0-logloss:0.61177
[86]	validation_0-logloss:0.61271
[87]	validation_0-logloss:0.61074
[88]	validation_0-logloss:0.61048
[89]	validation_0-logloss:0.60983
[90]	validation_0-logloss:0.60992
[91]	validation_0-logloss:0.60904
[92]	validation_0-logloss:0.60858
[93]	validation_0-logloss:0.60805
[94]	validation_0-logloss:0.60787
[95]	validation_0-logloss:0.60836
[96]	validation_0-logloss:0.60857
[97]	validation_0-logloss:0.60862
[98]	validation_0-logloss:0.60874
[99]	validation_0-logloss:0.60815
[100]	validation_0-logloss:0.60815
[101]	validation_0-logloss:0.60762
[102]	validation_0-logloss:0.60721
[103]	validation_0-logloss:0.60722
[104]	validation_0-logloss:0.60713
[105]	validation_0-logloss:0.60712
[106]	validation_0-logloss:0.60659
[107]	validation_0-logloss:0.60623
[108]	validation_0-logloss:0.60603
[109]	validation_0-logloss:0.60549
[110]	validation_0-logloss:0.60546
[111]	validation_0-logloss:0.60535
[112]	validation_0-logloss:0.60451
[113]	validation_0-logloss:0.60451
[114]	validation_0-logloss:0.60397
[115]	validation_0-logloss:0.60426
[116]	validation_0-logloss:0.60452
[117]	validation_0-logloss:0.60424
[118]	validation_0-logloss:0.60428
[119]	validation_0-logloss:0.60379
[120]	validation_0-logloss:0.60408
[121]	validation_0-logloss:0.60420
[122]	validation_0-logloss:0.60399
[123]	validation_0-logloss:0.60389
[124]	validation_0-logloss:0.60441
[125]	validation_0-logloss:0.60494
[126]	validation_0-logloss:0.60457
[127]	validation_0-logloss:0.60444
[128]	validation_0-logloss:0.60442
[129]	validation_0-logloss:0.60438
[130]	validation_0-logloss:0.60436
[131]	validation_0-logloss:0.60378
[132]	validation_0-logloss:0.60310
[133]	validation_0-logloss:0.60328
[134]	validation_0-logloss:0.60349
[135]	validation_0-logloss:0.60336
[136]	validation_0-logloss:0.60355
[137]	validation_0-logloss:0.60356
[138]	validation_0-logloss:0.60385
[139]	validation_0-logloss:0.60383
[140]	validation_0-logloss:0.60363
[141]	validation_0-logloss:0.60288
[142]	validation_0-logloss:0.60319
[143]	validation_0-logloss:0.60344
[144]	validation_0-logloss:0.60350
[145]	validation_0-logloss:0.60393
[146]	validation_0-logloss:0.60399
[147]	validation_0-logloss:0.60408
[148]	validation_0-logloss:0.60428
[149]	validation_0-logloss:0.60439
[150]	validation_0-logloss:0.60444
[151]	validation_0-logloss:0.60460
[152]	validation_0-logloss:0.60519
[153]	validation_0-logloss:0.60553
[154]	validation_0-logloss:0.60516
[155]	validation_0-logloss:0.60552
[156]	validation_0-logloss:0.60554
[157]	validation_0-logloss:0.60521
[158]	validation_0-logloss:0.60540
[159]	validation_0-logloss:0.60549
[160]	validation_0-logloss:0.60561
[161]	validation_0-logloss:0.60576
[162]	validation_0-logloss:0.60609
[163]	validation_0-logloss:0.60591
[164]	validation_0-logloss:0.60582
[165]	validation_0-logloss:0.60576
[166]	validation_0-logloss:0.60607
[167]	validation_0-logloss:0.60569
[168]	validation_0-logloss:0.60565
[169]	validation_0-logloss:0.60612
[170]	validation_0-logloss:0.60641
[171]	validation_0-logloss:0.60640
[172]	validation_0-logloss:0.60609
[173]	validation_0-logloss:0.60584
[174]	validation_0-logloss:0.60604
[175]	validation_0-logloss:0.60608
[176]	validation_0-logloss:0.60609
[177]	validation_0-logloss:0.60606
[178]	validation_0-logloss:0.60660
[179]	validation_0-logloss:0.60601
[180]	validation_0-logloss:0.60543
[181]	validation_0-logloss:0.60482
[182]	validation_0-logloss:0.60460
[183]	validation_0-logloss:0.60465
[184]	validation_0-logloss:0.60453
[185]	validation_0-logloss:0.60450
[186]	validation_0-logloss:0.60447
[187]	validation_0-logloss:0.60442
[188]	validation_0-logloss:0.60432
[189]	validation_0-logloss:0.60451
[190]	validation_0-logloss:0.60469
[191]	validation_0-logloss:0.60473
[192]	validation_0-logloss:0.60455
[193]	validation_0-logloss:0.60426
[194]	validation_0-logloss:0.60474
[195]	validation_0-logloss:0.60463
[196]	validation_0-logloss:0.60473
[197]	validation_0-logloss:0.60477
[198]	validation_0-logloss:0.60532
[199]	validation_0-logloss:0.60515
[200]	validation_0-logloss:0.60518
[201]	validation_0-logloss:0.60515
[202]	validation_0-logloss:0.60500
[203]	validation_0-logloss:0.60524
[204]	validation_0-logloss:0.60522
[205]	validation_0-logloss:0.60516
[206]	validation_0-logloss:0.60473
[207]	validation_0-logloss:0.60459
[208]	validation_0-logloss:0.60468
[209]	validation_0-logloss:0.60497
[210]	validation_0-logloss:0.60538
[211]	validation_0-logloss:0.60584
[212]	validation_0-logloss:0.60534
[213]	validation_0-logloss:0.60530
[214]	validation_0-logloss:0.60557
[215]	validation_0-logloss:0.60565
[216]	validation_0-logloss:0.60610
[217]	validation_0-logloss:0.60636
[218]	validation_0-logloss:0.60650
[219]	validation_0-logloss:0.60661
[220]	validation_0-logloss:0.60655
[221]	validation_0-logloss:0.60701
[222]	validation_0-logloss:0.60714
[223]	validation_0-logloss:0.60700
[224]	validation_0-logloss:0.60750
[225]	validation_0-logloss:0.60757
[226]	validation_0-logloss:0.60762
[227]	validation_0-logloss:0.60722
[228]	validation_0-logloss:0.60706
[229]	validation_0-logloss:0.60686
[230]	validation_0-logloss:0.60654
[231]	validation_0-logloss:0.60657
[232]	validation_0-logloss:0.60676
[233]	validation_0-logloss:0.60664
[234]	validation_0-logloss:0.60668
[235]	validation_0-logloss:0.60694
[236]	validation_0-logloss:0.60680
[237]	validation_0-logloss:0.60677
[238]	validation_0-logloss:0.60649
[239]	validation_0-logloss:0.60630
[240]	validation_0-logloss:0.60609
[241]	validation_0-logloss:0.60574
[242]	validation_0-logloss:0.60603
[243]	validation_0-logloss:0.60609
[244]	validation_0-logloss:0.60588
[245]	validation_0-logloss:0.60599
[246]	validation_0-logloss:0.60576
[247]	validation_0-logloss:0.60621
[248]	validation_0-logloss:0.60669
[249]	validation_0-logloss:0.60657
[250]	validation_0-logloss:0.60696
[251]	validation_0-logloss:0.60693
[252]	validation_0-logloss:0.60653
[253]	validation_0-logloss:0.60678
[254]	validation_0-logloss:0.60658
[255]	validation_0-logloss:0.60608
[256]	validation_0-logloss:0.60590
[257]	validation_0-logloss:0.60587
[258]	validation_0-logloss:0.60539
[259]	validation_0-logloss:0.60528
[260]	validation_0-logloss:0.60510
[261]	validation_0-logloss:0.60560
[262]	validation_0-logloss:0.60583
[263]	validation_0-logloss:0.60592
[264]	validation_0-logloss:0.60591
[265]	validation_0-logloss:0.60541
[266]	validation_0-logloss:0.60535
[267]	validation_0-logloss:0.60566
[268]	validation_0-logloss:0.60543
[269]	validation_0-logloss:0.60562
[270]	validation_0-logloss:0.60554
[271]	validation_0-logloss:0.60535
[272]	validation_0-logloss:0.60563
[273]	validation_0-logloss:0.60566
[274]	validation_0-logloss:0.60529
[275]	validation_0-logloss:0.60534
[276]	validation_0-logloss:0.60551
[277]	validation_0-logloss:0.60549
[278]	validation_0-logloss:0.60546
[279]	validation_0-logloss:0.60526
[280]	validation_0-logloss:0.60515
[281]	validation_0-logloss:0.60527
[282]	validation_0-logloss:0.60511
[283]	validation_0-logloss:0.60428
[284]	validation_0-logloss:0.60414
[285]	validation_0-logloss:0.60400
[286]	validation_0-logloss:0.60428
[287]	validation_0-logloss:0.60393
[288]	validation_0-logloss:0.60395
[289]	validation_0-logloss:0.60418
[290]	validation_0-logloss:0.60400
[291]	validation_0-logloss:0.60397
[292]	validation_0-logloss:0.60400
[293]	validation_0-logloss:0.60457
[294]	validation_0-logloss:0.60491
[295]	validation_0-logloss:0.60482
[296]	validation_0-logloss:0.60503
[297]	validation_0-logloss:0.60526
[298]	validation_0-logloss:0.60520
[299]	validation_0-logloss:0.60509
[300]	validation_0-logloss:0.60484
[301]	validation_0-logloss:0.60457
[302]	validation_0-logloss:0.60474
[303]	validation_0-logloss:0.60462
[304]	validation_0-logloss:0.60472
[305]	validation_0-logloss:0.60515
[306]	validation_0-logloss:0.60481
[307]	validation_0-logloss:0.60471
[308]	validation_0-logloss:0.60469
[309]	validation_0-logloss:0.60460
[310]	validation_0-logloss:0.60466
[311]	validation_0-logloss:0.60474
[312]	validation_0-logloss:0.60487
[313]	validation_0-logloss:0.60508
[314]	validation_0-logloss:0.60515
[315]	validation_0-logloss:0.60525
[316]	validation_0-logloss:0.60464
[317]	validation_0-logloss:0.60475
[318]	validation_0-logloss:0.60480
[319]	validation_0-logloss:0.60429
[320]	validation_0-logloss:0.60425
[321]	validation_0-logloss:0.60446
[322]	validation_0-logloss:0.60442
[323]	validation_0-logloss:0.60446
[324]	validation_0-logloss:0.60472
[325]	validation_0-logloss:0.60480
[326]	validation_0-logloss:0.60463
[327]	validation_0-logloss:0.60456
[328]	validation_0-logloss:0.60465
[329]	validation_0-logloss:0.60469
[330]	validation_0-logloss:0.60477
[331]	validation_0-logloss:0.60517
[332]	validation_0-logloss:0.60530
[333]	validation_0-logloss:0.60528
[334]	validation_0-logloss:0.60485
[335]	validation_0-logloss:0.60464
[336]	validation_0-logloss:0.60450
[337]	validation_0-logloss:0.60485
[338]	validation_0-logloss:0.60507
[339]	validation_0-logloss:0.60503
[340]	validation_0-logloss:0.60486
[341]	validation_0-logloss:0.60507
[342]	validation_0-logloss:0.60502
[343]	validation_0-logloss:0.60454
[344]	validation_0-logloss:0.60476
[345]	validation_0-logloss:0.60511
[346]	validation_0-logloss:0.60532
[347]	validation_0-logloss:0.60501
[348]	validation_0-logloss:0.60510
[349]	validation_0-logloss:0.60524
[350]	validation_0-logloss:0.60553
[351]	validation_0-logloss:0.60552
[352]	validation_0-logloss:0.60485
[353]	validation_0-logloss:0.60502
[354]	validation_0-logloss:0.60475
[355]	validation_0-logloss:0.60484
[356]	validation_0-logloss:0.60499
[357]	validation_0-logloss:0.60494
[358]	validation_0-logloss:0.60474
[359]	validation_0-logloss:0.60461
[360]	validation_0-logloss:0.60477
[361]	validation_0-logloss:0.60355
[362]	validation_0-logloss:0.60340
[363]	validation_0-logloss:0.60368
[364]	validation_0-logloss:0.60373
[365]	validation_0-logloss:0.60382
[366]	validation_0-logloss:0.60382
[367]	validation_0-logloss:0.60366
[368]	validation_0-logloss:0.60367
[369]	validation_0-logloss:0.60350
[370]	validation_0-logloss:0.60348
[371]	validation_0-logloss:0.60336
[372]	validation_0-logloss:0.60300
[373]	validation_0-logloss:0.60334
[374]	validation_0-logloss:0.60330
[375]	validation_0-logloss:0.60371
[376]	validation_0-logloss:0.60409
[377]	validation_0-logloss:0.60424
[378]	validation_0-logloss:0.60393
[379]	validation_0-logloss:0.60401
[380]	validation_0-logloss:0.60403
[381]	validation_0-logloss:0.60395
[382]	validation_0-logloss:0.60366
[383]	validation_0-logloss:0.60358
[384]	validation_0-logloss:0.60356
[385]	validation_0-logloss:0.60394
[386]	validation_0-logloss:0.60367
[387]	validation_0-logloss:0.60399
[388]	validation_0-logloss:0.60392
[389]	validation_0-logloss:0.60449
[390]	validation_0-logloss:0.60467
[391]	validation_0-logloss:0.60516
[392]	validation_0-logloss:0.60514
[393]	validation_0-logloss:0.60507
[394]	validation_0-logloss:0.60519
[395]	validation_0-logloss:0.60530
[396]	validation_0-logloss:0.60509
[397]	validation_0-logloss:0.60484
[398]	validation_0-logloss:0.60473
[399]	validation_0-logloss:0.60446
[400]	validation_0-logloss:0.60440
[401]	validation_0-logloss:0.60455
[402]	validation_0-logloss:0.60452
[403]	validation_0-logloss:0.60424
[404]	validation_0-logloss:0.60409
[405]	validation_0-logloss:0.60405
[406]	validation_0-logloss:0.60397
[407]	validation_0-logloss:0.60402
[408]	validation_0-logloss:0.60391
[409]	validation_0-logloss:0.60378
[410]	validation_0-logloss:0.60382
[411]	validation_0-logloss:0.60386
[412]	validation_0-logloss:0.60359
[413]	validation_0-logloss:0.60344
[414]	validation_0-logloss:0.60370
[415]	validation_0-logloss:0.60382
[416]	validation_0-logloss:0.60394
[417]	validation_0-logloss:0.60401
[418]	validation_0-logloss:0.60385
[419]	validation_0-logloss:0.60374
[420]	validation_0-logloss:0.60382
[421]	validation_0-logloss:0.60395
[422]	validation_0-logloss:0.60394
[423]	validation_0-logloss:0.60395
[424]	validation_0-logloss:0.60385
[425]	validation_0-logloss:0.60374
[426]	validation_0-logloss:0.60343
[427]	validation_0-logloss:0.60384
[428]	validation_0-logloss:0.60435
[429]	validation_0-logloss:0.60471
[430]	validation_0-logloss:0.60426
[431]	validation_0-logloss:0.60393
[432]	validation_0-logloss:0.60411
[433]	validation_0-logloss:0.60418
[434]	validation_0-logloss:0.60446
[435]	validation_0-logloss:0.60360
[436]	validation_0-logloss:0.60333
[437]	validation_0-logloss:0.60326
[438]	validation_0-logloss:0.60335
[439]	validation_0-logloss:0.60329
[440]	validation_0-logloss:0.60312
[441]	validation_0-logloss:0.60343
[442]	validation_0-logloss:0.60387
[443]	validation_0-logloss:0.60386
[444]	validation_0-logloss:0.60377
[445]	validation_0-logloss:0.60369
[446]	validation_0-logloss:0.60395
[447]	validation_0-logloss:0.60427
[448]	validation_0-logloss:0.60443
[449]	validation_0-logloss:0.60459
[450]	validation_0-logloss:0.60452
[451]	validation_0-logloss:0.60487
[452]	validation_0-logloss:0.60499
[453]	validation_0-logloss:0.60422
[454]	validation_0-logloss:0.60429
[455]	validation_0-logloss:0.60423
[456]	validation_0-logloss:0.60457
[457]	validation_0-logloss:0.60458
[458]	validation_0-logloss:0.60459
[459]	validation_0-logloss:0.60461
[460]	validation_0-logloss:0.60487
[461]	validation_0-logloss:0.60523
[462]	validation_0-logloss:0.60522
[463]	validation_0-logloss:0.60511
[464]	validation_0-logloss:0.60496
[465]	validation_0-logloss:0.60522
[466]	validation_0-logloss:0.60537
[467]	validation_0-logloss:0.60529
[468]	validation_0-logloss:0.60488
[469]	validation_0-logloss:0.60495
[470]	validation_0-logloss:0.60476
[471]	validation_0-logloss:0.60436
[472]	validation_0-logloss:0.60453
[473]	validation_0-logloss:0.60423
[474]	validation_0-logloss:0.60389
[475]	validation_0-logloss:0.60389
[476]	validation_0-logloss:0.60365
[477]	validation_0-logloss:0.60376
[478]	validation_0-logloss:0.60377
[479]	validation_0-logloss:0.60350
[480]	validation_0-logloss:0.60341
[481]	validation_0-logloss:0.60335
[482]	validation_0-logloss:0.60350
[483]	validation_0-logloss:0.60303
[484]	validation_0-logloss:0.60329
[485]	validation_0-logloss:0.60326
[486]	validation_0-logloss:0.60336
[487]	validation_0-logloss:0.60346
[488]	validation_0-logloss:0.60365
[489]	validation_0-logloss:0.60350
[490]	validation_0-logloss:0.60350
[491]	validation_0-logloss:0.60373
[492]	validation_0-logloss:0.60363
[493]	validation_0-logloss:0.60397
[494]	validation_0-logloss:0.60403
[495]	validation_0-logloss:0.60371
[496]	validation_0-logloss:0.60382
[497]	validation_0-logloss:0.60378
[498]	validation_0-logloss:0.60390
[499]	validation_0-logloss:0.60406
[500]	validation_0-logloss:0.60411
[501]	validation_0-logloss:0.60401
[502]	validation_0-logloss:0.60416
[503]	validation_0-logloss:0.60469
[504]	validation_0-logloss:0.60466
[505]	validation_0-logloss:0.60460
[506]	validation_0-logloss:0.60480
[507]	validation_0-logloss:0.60445
[508]	validation_0-logloss:0.60471
[509]	validation_0-logloss:0.60446
[510]	validation_0-logloss:0.60447
[511]	validation_0-logloss:0.60452
[512]	validation_0-logloss:0.60432
[513]	validation_0-logloss:0.60395
[514]	validation_0-logloss:0.60411
[515]	validation_0-logloss:0.60397
[516]	validation_0-logloss:0.60418
[517]	validation_0-logloss:0.60432
[518]	validation_0-logloss:0.60424
[519]	validation_0-logloss:0.60419
[520]	validation_0-logloss:0.60442
[521]	validation_0-logloss:0.60408
[522]	validation_0-logloss:0.60413
[523]	validation_0-logloss:0.60399
[524]	validation_0-logloss:0.60416
[525]	validation_0-logloss:0.60426
[526]	validation_0-logloss:0.60448
[527]	validation_0-logloss:0.60472
[528]	validation_0-logloss:0.60455
[529]	validation_0-logloss:0.60461
[530]	validation_0-logloss:0.60446
[531]	validation_0-logloss:0.60432
[532]	validation_0-logloss:0.60416
[533]	validation_0-logloss:0.60405
[534]	validation_0-logloss:0.60423
[535]	validation_0-logloss:0.60428
[536]	validation_0-logloss:0.60378
[537]	validation_0-logloss:0.60372
[538]	validation_0-logloss:0.60382
[539]	validation_0-logloss:0.60379
[540]	validation_0-logloss:0.60388
[541]	validation_0-logloss:0.60372
[542]	validation_0-logloss:0.60382
[543]	validation_0-logloss:0.60378
[544]	validation_0-logloss:0.60367
[545]	validation_0-logloss:0.60397
[546]	validation_0-logloss:0.60379
[547]	validation_0-logloss:0.60401
[548]	validation_0-logloss:0.60416
[549]	validation_0-logloss:0.60442
[550]	validation_0-logloss:0.60443
[551]	validation_0-logloss:0.60432
[552]	validation_0-logloss:0.60414
[553]	validation_0-logloss:0.60427
[554]	validation_0-logloss:0.60457
[555]	validation_0-logloss:0.60423
[556]	validation_0-logloss:0.60474
[557]	validation_0-logloss:0.60459
[558]	validation_0-logloss:0.60463
[559]	validation_0-logloss:0.60445
[560]	validation_0-logloss:0.60412
[561]	validation_0-logloss:0.60404
[562]	validation_0-logloss:0.60418
[563]	validation_0-logloss:0.60409
[564]	validation_0-logloss:0.60425
[565]	validation_0-logloss:0.60470
[566]	validation_0-logloss:0.60461
[567]	validation_0-logloss:0.60490
[568]	validation_0-logloss:0.60464
[569]	validation_0-logloss:0.60456
[570]	validation_0-logloss:0.60474
[571]	validation_0-logloss:0.60472
[572]	validation_0-logloss:0.60466
[573]	validation_0-logloss:0.60453
[574]	validation_0-logloss:0.60497
[575]	validation_0-logloss:0.60498
[576]	validation_0-logloss:0.60512
[577]	validation_0-logloss:0.60532
[578]	validation_0-logloss:0.60528
[579]	validation_0-logloss:0.60516
[580]	validation_0-logloss:0.60537
[581]	validation_0-logloss:0.60552
[582]	validation_0-logloss:0.60537
[583]	validation_0-logloss:0.60543
[584]	validation_0-logloss:0.60534
[585]	validation_0-logloss:0.60534
[586]	validation_0-logloss:0.60523
[587]	validation_0-logloss:0.60507
[588]	validation_0-logloss:0.60517
[589]	validation_0-logloss:0.60532
[590]	validation_0-logloss:0.60511
[591]	validation_0-logloss:0.60522
[592]	validation_0-logloss:0.60522
[593]	validation_0-logloss:0.60500
[594]	validation_0-logloss:0.60504
[595]	validation_0-logloss:0.60453
[596]	validation_0-logloss:0.60472
[597]	validation_0-logloss:0.60476
[598]	validation_0-logloss:0.60454
[599]	validation_0-logloss:0.60482
[600]	validation_0-logloss:0.60493
[601]	validation_0-logloss:0.60508
[602]	validation_0-logloss:0.60498
[603]	validation_0-logloss:0.60468
[604]	validation_0-logloss:0.60489
[605]	validation_0-logloss:0.60471
[606]	validation_0-logloss:0.60445
[607]	validation_0-logloss:0.60449
[608]	validation_0-logloss:0.60416
[609]	validation_0-logloss:0.60470
[610]	validation_0-logloss:0.60475
[611]	validation_0-logloss:0.60463
[612]	validation_0-logloss:0.60459
[613]	validation_0-logloss:0.60463
[614]	validation_0-logloss:0.60483
[615]	validation_0-logloss:0.60463
[616]	validation_0-logloss:0.60455
[617]	validation_0-logloss:0.60469
[618]	validation_0-logloss:0.60512
[619]	validation_0-logloss:0.60497
[620]	validation_0-logloss:0.60498
[621]	validation_0-logloss:0.60506
[622]	validation_0-logloss:0.60505
[623]	validation_0-logloss:0.60511
[624]	validation_0-logloss:0.60516
[625]	validation_0-logloss:0.60471
[626]	validation_0-logloss:0.60465
[627]	validation_0-logloss:0.60462
[628]	validation_0-logloss:0.60465
[629]	validation_0-logloss:0.60461
[630]	validation_0-logloss:0.60509
[631]	validation_0-logloss:0.60494
[632]	validation_0-logloss:0.60538
[633]	validation_0-logloss:0.60578
[634]	validation_0-logloss:0.60573
[635]	validation_0-logloss:0.60580
[636]	validation_0-logloss:0.60596
[637]	validation_0-logloss:0.60593
[638]	validation_0-logloss:0.60586
[639]	validation_0-logloss:0.60597
[640]	validation_0-logloss:0.60609
[641]	validation_0-logloss:0.60606
[642]	validation_0-logloss:0.60550
[643]	validation_0-logloss:0.60544
[644]	validation_0-logloss:0.60542
[645]	validation_0-logloss:0.60576
[646]	validation_0-logloss:0.60561
[647]	validation_0-logloss:0.60587
[648]	validation_0-logloss:0.60584
[649]	validation_0-logloss:0.60494
[650]	validation_0-logloss:0.60505
[651]	validation_0-logloss:0.60494
[652]	validation_0-logloss:0.60488
[653]	validation_0-logloss:0.60494
[654]	validation_0-logloss:0.60439
[655]	validation_0-logloss:0.60448
[656]	validation_0-logloss:0.60448
[657]	validation_0-logloss:0.60455
[658]	validation_0-logloss:0.60459
[659]	validation_0-logloss:0.60436
[660]	validation_0-logloss:0.60424
[661]	validation_0-logloss:0.60412
[662]	validation_0-logloss:0.60409
[663]	validation_0-logloss:0.60410
[664]	validation_0-logloss:0.60421
[665]	validation_0-logloss:0.60425
[666]	validation_0-logloss:0.60453
[667]	validation_0-logloss:0.60444
[668]	validation_0-logloss:0.60434
[669]	validation_0-logloss:0.60442
[670]	validation_0-logloss:0.60437
[671]	validation_0-logloss:0.60456
[672]	validation_0-logloss:0.60458
[673]	validation_0-logloss:0.60443
[674]	validation_0-logloss:0.60407
[675]	validation_0-logloss:0.60402
[676]	validation_0-logloss:0.60406
[677]	validation_0-logloss:0.60406
[678]	validation_0-logloss:0.60412
[679]	validation_0-logloss:0.60435
[680]	validation_0-logloss:0.60433
[681]	validation_0-logloss:0.60408
[682]	validation_0-logloss:0.60389
[683]	validation_0-logloss:0.60368
[684]	validation_0-logloss:0.60364
[685]	validation_0-logloss:0.60370
[686]	validation_0-logloss:0.60360
[687]	validation_0-logloss:0.60370
[688]	validation_0-logloss:0.60363
[689]	validation_0-logloss:0.60367
[690]	validation_0-logloss:0.60391
[691]	validation_0-logloss:0.60374
[692]	validation_0-logloss:0.60393
[693]	validation_0-logloss:0.60394
[694]	validation_0-logloss:0.60422
[695]	validation_0-logloss:0.60424
[696]	validation_0-logloss:0.60417
[697]	validation_0-logloss:0.60411
[698]	validation_0-logloss:0.60426
[699]	validation_0-logloss:0.60473
[700]	validation_0-logloss:0.60487
[701]	validation_0-logloss:0.60560
[702]	validation_0-logloss:0.60577
[703]	validation_0-logloss:0.60570
[704]	validation_0-logloss:0.60535
[705]	validation_0-logloss:0.60524
[706]	validation_0-logloss:0.60532
[707]	validation_0-logloss:0.60555
[708]	validation_0-logloss:0.60548
[709]	validation_0-logloss:0.60556
[710]	validation_0-logloss:0.60569
[711]	validation_0-logloss:0.60592
[712]	validation_0-logloss:0.60615
[713]	validation_0-logloss:0.60617
[714]	validation_0-logloss:0.60631
[715]	validation_0-logloss:0.60655
[716]	validation_0-logloss:0.60684
[717]	validation_0-logloss:0.60676
[718]	validation_0-logloss:0.60646
[719]	validation_0-logloss:0.60614
[720]	validation_0-logloss:0.60583
[721]	validation_0-logloss:0.60571
[722]	validation_0-logloss:0.60550
[723]	validation_0-logloss:0.60545
[724]	validation_0-logloss:0.60471
[725]	validation_0-logloss:0.60475
[726]	validation_0-logloss:0.60462
[727]	validation_0-logloss:0.60456
[728]	validation_0-logloss:0.60422
[729]	validation_0-logloss:0.60413
[730]	validation_0-logloss:0.60415
[731]	validation_0-logloss:0.60436
[732]	validation_0-logloss:0.60453
[733]	validation_0-logloss:0.60435
[734]	validation_0-logloss:0.60413
[735]	validation_0-logloss:0.60428
[736]	validation_0-logloss:0.60421
[737]	validation_0-logloss:0.60376
[738]	validation_0-logloss:0.60376
[739]	validation_0-logloss:0.60379
[740]	validation_0-logloss:0.60400
[741]	validation_0-logloss:0.60416
[742]	validation_0-logloss:0.60410
[743]	validation_0-logloss:0.60400
[744]	validation_0-logloss:0.60408
[745]	validation_0-logloss:0.60419
[746]	validation_0-logloss:0.60411
[747]	validation_0-logloss:0.60401
[748]	validation_0-logloss:0.60395
[749]	validation_0-logloss:0.60409
[750]	validation_0-logloss:0.60397
[751]	validation_0-logloss:0.60388
[752]	validation_0-logloss:0.60448
[753]	validation_0-logloss:0.60439
[754]	validation_0-logloss:0.60436
[755]	validation_0-logloss:0.60419
[756]	validation_0-logloss:0.60411
[757]	validation_0-logloss:0.60439
[758]	validation_0-logloss:0.60456
[759]	validation_0-logloss:0.60472
[760]	validation_0-logloss:0.60418
[761]	validation_0-logloss:0.60395
[762]	validation_0-logloss:0.60395
[763]	validation_0-logloss:0.60384
[764]	validation_0-logloss:0.60380
[765]	validation_0-logloss:0.60412
[766]	validation_0-logloss:0.60415
[767]	validation_0-logloss:0.60427
[768]	validation_0-logloss:0.60411
[769]	validation_0-logloss:0.60426
[770]	validation_0-logloss:0.60430
[771]	validation_0-logloss:0.60455
[772]	validation_0-logloss:0.60482
[773]	validation_0-logloss:0.60490
[774]	validation_0-logloss:0.60482
[775]	validation_0-logloss:0.60506
[776]	validation_0-logloss:0.60499
[777]	validation_0-logloss:0.60479
[778]	validation_0-logloss:0.60462
[779]	validation_0-logloss:0.60462
[780]	validation_0-logloss:0.60461
[781]	validation_0-logloss:0.60505
[782]	validation_0-logloss:0.60512
[783]	validation_0-logloss:0.60534
[784]	validation_0-logloss:0.60552
[785]	validation_0-logloss:0.60558
[786]	validation_0-logloss:0.60575
[787]	validation_0-logloss:0.60570
[788]	validation_0-logloss:0.60578
[789]	validation_0-logloss:0.60564
[790]	validation_0-logloss:0.60568
[791]	validation_0-logloss:0.60587
[792]	validation_0-logloss:0.60602
[793]	validation_0-logloss:0.60574
[794]	validation_0-logloss:0.60576
[795]	validation_0-logloss:0.60569
[796]	validation_0-logloss:0.60569
[797]	validation_0-logloss:0.60633
[798]	validation_0-logloss:0.60678
[799]	validation_0-logloss:0.60706
[800]	validation_0-logloss:0.60701
[801]	validation_0-logloss:0.60686
[802]	validation_0-logloss:0.60681
[803]	validation_0-logloss:0.60680
[804]	validation_0-logloss:0.60670
[805]	validation_0-logloss:0.60700
[806]	validation_0-logloss:0.60709
[807]	validation_0-logloss:0.60697
[808]	validation_0-logloss:0.60676
[809]	validation_0-logloss:0.60660
[810]	validation_0-logloss:0.60628
[811]	validation_0-logloss:0.60646
[812]	validation_0-logloss:0.60627
[813]	validation_0-logloss:0.60681
[814]	validation_0-logloss:0.60678
[815]	validation_0-logloss:0.60702
[816]	validation_0-logloss:0.60666
[817]	validation_0-logloss:0.60681
[818]	validation_0-logloss:0.60716
[819]	validation_0-logloss:0.60757
[820]	validation_0-logloss:0.60738
[821]	validation_0-logloss:0.60758
[822]	validation_0-logloss:0.60761
[823]	validation_0-logloss:0.60766
[824]	validation_0-logloss:0.60746
[825]	validation_0-logloss:0.60728
[826]	validation_0-logloss:0.60736
[827]	validation_0-logloss:0.60739
[828]	validation_0-logloss:0.60743
[829]	validation_0-logloss:0.60748
[830]	validation_0-logloss:0.60727
[831]	validation_0-logloss:0.60745
[832]	validation_0-logloss:0.60717
[833]	validation_0-logloss:0.60697
[834]	validation_0-logloss:0.60676
[835]	validation_0-logloss:0.60640
[836]	validation_0-logloss:0.60708
[837]	validation_0-logloss:0.60744
[838]	validation_0-logloss:0.60775
[839]	validation_0-logloss:0.60798
[840]	validation_0-logloss:0.60808
[841]	validation_0-logloss:0.60765
[842]	validation_0-logloss:0.60776
[843]	validation_0-logloss:0.60782
[844]	validation_0-logloss:0.60783
[845]	validation_0-logloss:0.60776
[846]	validation_0-logloss:0.60800
[847]	validation_0-logloss:0.60782
[848]	validation_0-logloss:0.60815
[849]	validation_0-logloss:0.60799
[850]	validation_0-logloss:0.60784
[851]	validation_0-logloss:0.60796
[852]	validation_0-logloss:0.60805
[853]	validation_0-logloss:0.60803
[854]	validation_0-logloss:0.60794
[855]	validation_0-logloss:0.60811
[856]	validation_0-logloss:0.60789
[857]	validation_0-logloss:0.60779
[858]	validation_0-logloss:0.60777
[859]	validation_0-logloss:0.60769
[860]	validation_0-logloss:0.60778
[861]	validation_0-logloss:0.60765
[862]	validation_0-logloss:0.60734
[863]	validation_0-logloss:0.60729
[864]	validation_0-logloss:0.60720
[865]	validation_0-logloss:0.60696
[866]	validation_0-logloss:0.60701
[867]	validation_0-logloss:0.60726
[868]	validation_0-logloss:0.60718
[869]	validation_0-logloss:0.60698
[870]	validation_0-logloss:0.60683
[871]	validation_0-logloss:0.60689
[872]	validation_0-logloss:0.60708
[873]	validation_0-logloss:0.60722
[874]	validation_0-logloss:0.60703
[875]	validation_0-logloss:0.60677
[876]	validation_0-logloss:0.60664
[877]	validation_0-logloss:0.60656
[878]	validation_0-logloss:0.60645
[879]	validation_0-logloss:0.60644
[880]	validation_0-logloss:0.60642
[881]	validation_0-logloss:0.60628
[882]	validation_0-logloss:0.60623
[883]	validation_0-logloss:0.60586
[884]	validation_0-logloss:0.60563
[885]	validation_0-logloss:0.60562
[886]	validation_0-logloss:0.60593
[887]	validation_0-logloss:0.60599
[888]	validation_0-logloss:0.60578
[889]	validation_0-logloss:0.60594
[890]	validation_0-logloss:0.60606
[891]	validation_0-logloss:0.60617
[892]	validation_0-logloss:0.60616
[893]	validation_0-logloss:0.60620
[894]	validation_0-logloss:0.60611
[895]	validation_0-logloss:0.60604
[896]	validation_0-logloss:0.60608
[897]	validation_0-logloss:0.60654
[898]	validation_0-logloss:0.60656
[899]	validation_0-logloss:0.60647
[900]	validation_0-logloss:0.60649
[901]	validation_0-logloss:0.60647
[902]	validation_0-logloss:0.60646
[903]	validation_0-logloss:0.60673
[904]	validation_0-logloss:0.60678
[905]	validation_0-logloss:0.60708
[906]	validation_0-logloss:0.60672
[907]	validation_0-logloss:0.60680
[908]	validation_0-logloss:0.60665
[909]	validation_0-logloss:0.60660
[910]	validation_0-logloss:0.60646
[911]	validation_0-logloss:0.60655
[912]	validation_0-logloss:0.60660
[913]	validation_0-logloss:0.60635
[914]	validation_0-logloss:0.60667
[915]	validation_0-logloss:0.60676
[916]	validation_0-logloss:0.60678
[917]	validation_0-logloss:0.60682
[918]	validation_0-logloss:0.60632
[919]	validation_0-logloss:0.60579
[920]	validation_0-logloss:0.60602
[921]	validation_0-logloss:0.60611
[922]	validation_0-logloss:0.60623
[923]	validation_0-logloss:0.60628
[924]	validation_0-logloss:0.60643
[925]	validation_0-logloss:0.60628
[926]	validation_0-logloss:0.60611
[927]	validation_0-logloss:0.60583
[928]	validation_0-logloss:0.60574
[929]	validation_0-logloss:0.60544
[930]	validation_0-logloss:0.60559
[931]	validation_0-logloss:0.60561
[932]	validation_0-logloss:0.60572
[933]	validation_0-logloss:0.60564
[934]	validation_0-logloss:0.60589
[935]	validation_0-logloss:0.60591
[936]	validation_0-logloss:0.60569
[937]	validation_0-logloss:0.60572
[938]	validation_0-logloss:0.60552
[939]	validation_0-logloss:0.60558
[940]	validation_0-logloss:0.60522
[941]	validation_0-logloss:0.60468
[942]	validation_0-logloss:0.60427
[943]	validation_0-logloss:0.60452
[944]	validation_0-logloss:0.60500
[945]	validation_0-logloss:0.60481
[946]	validation_0-logloss:0.60507
[947]	validation_0-logloss:0.60503
[948]	validation_0-logloss:0.60505
[949]	validation_0-logloss:0.60494
[950]	validation_0-logloss:0.60439
[951]	validation_0-logloss:0.60454
[952]	validation_0-logloss:0.60453
[953]	validation_0-logloss:0.60467
[954]	validation_0-logloss:0.60456
[955]	validation_0-logloss:0.60452
[956]	validation_0-logloss:0.60464
[957]	validation_0-logloss:0.60494
[958]	validation_0-logloss:0.60493
[959]	validation_0-logloss:0.60518
[960]	validation_0-logloss:0.60535
[961]	validation_0-logloss:0.60534
[962]	validation_0-logloss:0.60530
[963]	validation_0-logloss:0.60515
[964]	validation_0-logloss:0.60497
[965]	validation_0-logloss:0.60475
[966]	validation_0-logloss:0.60487
[967]	validation_0-logloss:0.60496
[968]	validation_0-logloss:0.60503
[969]	validation_0-logloss:0.60510
[970]	validation_0-logloss:0.60502
[971]	validation_0-logloss:0.60511
[972]	validation_0-logloss:0.60512
[973]	validation_0-logloss:0.60506
[974]	validation_0-logloss:0.60495
[975]	validation_0-logloss:0.60517
[976]	validation_0-logloss:0.60527
[977]	validation_0-logloss:0.60520
[978]	validation_0-logloss:0.60499
[979]	validation_0-logloss:0.60524
[980]	validation_0-logloss:0.60502
[981]	validation_0-logloss:0.60549
[982]	validation_0-logloss:0.60578
[983]	validation_0-logloss:0.60528
[984]	validation_0-logloss:0.60477
[985]	validation_0-logloss:0.60478
[986]	validation_0-logloss:0.60509
[987]	validation_0-logloss:0.60460
[988]	validation_0-logloss:0.60440
[989]	validation_0-logloss:0.60463
[990]	validation_0-logloss:0.60491
[991]	validation_0-logloss:0.60490
[992]	validation_0-logloss:0.60493
[993]	validation_0-logloss:0.60501
[994]	validation_0-logloss:0.60499
[995]	validation_0-logloss:0.60497
[996]	validation_0-logloss:0.60508
[997]	validation_0-logloss:0.60511
[998]	validation_0-logloss:0.60555
[999]	validation_0-logloss:0.60554

step-04 确认最优参数

print(clf.best_params_)

{'base_score': 0.5, 'colsample_bylevel': 0.7, 'colsample_bynode': 0.7, 'colsample_bytree': 0.6, 'gamma': 0, 'learning_rate': 0.1, 'max_bin': 12, 'max_depth': 6, 'min_child_weight': 30, 'n_estimators': 1000, 'reg_alpha': 2, 'reg_lambda': 3, 'subsample': 0.65}

step-05 选取最优模型

best_model=clf.best_estimator_

step-06 评价最优模型

model_eval2(best_model, train.values, test.values)

train_roc_auc_score: 0.8766644056264636
test_roc_auc_score: 0.7278343023255814
train_accuracy_score: 0.8
test_accuracy_score: 0.6833333333333333
train_precision_score: 0.8069963811821471
test__precision_score: 0.7162921348314607
train_recall_score: 0.8479087452471483
test_recall_score: 0.7412790697674418
train_f1_score: 0.8269468479604452
test_f1_score: 0.7285714285714285

step-07 保存并调用模型

joblib.dump(best_model ,  r'D:\Ensemble_Learning\xgboostinfo\xgboostgridbest.model')
best_model=joblib.load( r'D:\Ensemble_Learning\xgboostinfo\xgboostgridbest.model')
model_eval2(best_model, train.values, test.values)

train_roc_auc_score: 0.8766644056264636
test_roc_auc_score: 0.7278343023255814
train_accuracy_score: 0.8
test_accuracy_score: 0.6833333333333333
train_precision_score: 0.8069963811821471
test__precision_score: 0.7162921348314607
train_recall_score: 0.8479087452471483
test_recall_score: 0.7412790697674418
train_f1_score: 0.8269468479604452
test_f1_score: 0.7285714285714285

你可能感兴趣的:(python机器学习,数据挖掘,算法)

机器学习与深度学习间关系与区别 ℒℴѵℯ心·动ꦿ໊ོ꫞ 人工智能学习深度学习 python
一、机器学习概述定义机器学习（MachineLearning,ML）是一种通过数据驱动的方法，利用统计学和计算算法来训练模型，使计算机能够从数据中学习并自动进行预测或决策。机器学习通过分析大量数据样本，识别其中的模式和规律，从而对新的数据进行判断。其核心在于通过训练过程，让模型不断优化和提升其预测准确性。主要类型1.监督学习（SupervisedLearning）监督学习是指在训练数据集中包含输入
Goolge earth studio 进阶4——路径修改与平滑陟彼高冈yu Google earth studio 进阶教程旅游
如果我们希望在大约中途时获得更多的城市鸟瞰视角。可以将相机拖动到这里并创建一个新的关键帧。camera_target_clip_7EarthStudio会自动平滑我们的路径，所以当我们通过这个关键帧时，不是一个生硬的角度，而是一个平滑的曲线。camera_target_clip_8路径上有贝塞尔控制手柄，允许我们调整路径的形状。右键单击，我们可以选择“平滑路径”，这是默认的自动平滑算法，或者我们可
基于社交网络算法优化的二维最大熵图像分割智能算法研学社（Jack旭）智能优化算法应用图像分割算法 php 开发语言
智能优化算法应用：基于社交网络优化的二维最大熵图像阈值分割-附代码文章目录智能优化算法应用：基于社交网络优化的二维最大熵图像阈值分割-附代码1.前言2.二维最大熵阈值分割原理3.基于社交网络优化的多阈值分割4.算法结果：5.参考文献：6.Matlab代码摘要：本文介绍基于最大熵的图像分割，并且应用社交网络算法进行阈值寻优。1.前言阅读此文章前，请阅读《图像分割：直方图区域划分及信息统计介绍》htt
121. 买卖股票的最佳时机薄荷糖的味道_fb40
给定一个数组，它的第i个元素是一支给定股票第i天的价格。如果你最多只允许完成一笔交易（即买入和卖出一支股票），设计一个算法来计算你所能获取的最大利润。注意你不能在买入股票前卖出股票。示例1:输入:[7,1,5,3,6,4]输出:5解释:在第2天（股票价格=1）的时候买入，在第5天（股票价格=6）的时候卖出，最大利润=6-1=5。注意利润不能是7-1=6,因为卖出价格需要大于买入价格。示例2:输入:
每日算法&面试题，大厂特训二十八天——第二十天（树）肥学 ⚡算法题⚡面试题每日精进 java 算法数据结构
目录标题导读算法特训二十八天面试题点击直接资料领取导读肥友们为了更好的去帮助新同学适应算法和面试题，最近我们开始进行专项突击一步一步来。上一期我们完成了动态规划二十一天现在我们进行下一项对各类算法进行二十八天的一个小总结。还在等什么快来一起肥学进行二十八天挑战吧！！特别介绍小白练手专栏，适合刚入手的新人欢迎订阅编程小白进阶python有趣练手项目里面包括了像《机器人尬聊》《恶搞程序》这样的有趣文章
回溯算法-重新安排行程 chirou_ 算法数据结构图论 c++图搜索
leetcode332.重新安排行程这题我还没自己ac过，只能现在凭着刚学完的热乎劲把我对题解的理解记下来。本题我认为对数据结构的考察比较多，用什么数据结构去存数据，去读取数据，都是很重要的。classSolution{private:unordered_map>targets;boolbacktracking(intticketNum,vector&result){//1.确定参数和返回值//2
Faiss：高效相似性搜索与聚类的利器网络·魚大数据 faiss
Faiss是一个针对大规模向量集合的相似性搜索库，由FacebookAIResearch开发。它提供了一系列高效的算法和数据结构，用于加速向量之间的相似性搜索，特别是在大规模数据集上。本文将介绍Faiss的原理、核心功能以及如何在实际项目中使用它。Faiss原理：近似最近邻搜索：Faiss的核心功能之一是近似最近邻搜索，它能够高效地在大规模数据集中找到与给定查询向量最相似的向量。这种搜索是近似的，
nosql数据库技术与应用知识点皆过客，揽星河 NoSQL nosql 数据库大数据数据分析数据结构非关系型数据库
Nosql知识回顾大数据处理流程数据采集(flume、爬虫、传感器)数据存储(本门课程NoSQL所处的阶段)Hdfs、MongoDB、HBase等数据清洗(入仓)Hive等数据处理、分析(Spark、Flink等)数据可视化数据挖掘、机器学习应用(Python、SparkMLlib等)大数据时代存储的挑战(三高)高并发(同一时间很多人访问)高扩展(要求随时根据需求扩展存储)高效率(要求读写速度快)
insert into select 主键自增_mybatis拦截器实现主键自动生成 weixin_39521651 insert into select 主键自增 mybatis delete返回值 mybatis insert返回主键 mybatis insert返回对象 mybatis plus insert返回主键 mybatis plus 插入生成id
前言前阵子和朋友聊天，他说他们项目有个需求，要实现主键自动生成，不想每次新增的时候，都手动设置主键。于是我就问他，那你们数据库表设置主键自动递增不就得了。他的回答是他们项目目前的id都是采用雪花算法来生成，因此为了项目稳定性，不会切换id的生成方式。朋友问我有没有什么实现思路，他们公司的orm框架是mybatis，我就建议他说，不然让你老大把mybatis切换成mybatis-plus。mybat
k均值聚类算法考试例题_k均值算法(k均值聚类算法计算题) 寻找你83497 k均值聚类算法考试例题
?算法：第一步：选K个初始聚类中心，z1(1),z2(1)，…，zK(1)，其中括号内的序号为寻找聚类中心的迭代运算的次序号。聚类中心的向量值可任意设定，例如可选开始的K个.k均值聚类：---------一种硬聚类算法，隶属度只有两个取值0或1，提出的基本根据是“类内误差平方和最小化”准则；模糊的c均值聚类算法：--------一种模糊聚类算法，是.K均值聚类算法是先随机选取K个对象作为初始的聚类
Python实现简单的机器学习算法 master_chenchengg python python 办公效率 python开发 IT
Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl
推荐算法_隐语义-梯度下降 _feivirus_ 算法机器学习和数学推荐算法机器学习隐语义
importnumpyasnp1.模型实现"""inputrate_matrix:M行N列的评分矩阵，值为P*Q.P:初始化用户特征矩阵M*K.Q:初始化物品特征矩阵K*N.latent_feature_cnt:隐特征的向量个数max_iteration:最大迭代次数alpha:步长lamda:正则化系数output分解之后的P和Q"""defLFM_grad_desc(rate_matrix,l
K近邻算法_分类鸢尾花数据集 _feivirus_ 算法机器学习和数学分类机器学习 K近邻
importnumpyasnpimportpandasaspdfromsklearn.datasetsimportload_irisfromsklearn.model_selectionimporttrain_test_splitfromsklearn.metricsimportaccuracy_score1.数据预处理iris=load_iris()df=pd.DataFrame(data=ir
数据结构 | 栈和队列 TT-Kun 数据结构与算法数据结构栈队列 C语言
文章目录栈和队列1.栈：后进先出（LIFO）的数据结构1.1概念与结构1.2栈的实现2.队列：先进先出（FIFO）的数据结构2.1概念与结构2.2队列的实现3.栈和队列算法题3.1有效的括号3.2用队列实现栈3.3用栈实现队列3.4设计循环队列结论栈和队列在计算机科学中，栈和队列是两种基本且重要的数据结构，它们在处理数据存储和访问顺序方面有着独特的规则和应用。本文将详细介绍栈和队列的概念、结构、实
[Python] 数据结构详解及代码 AIAdvocate 算法 python 数据结构链表
今日内容大纲介绍数据结构介绍列表链表1.数据结构和算法简介程序大白话翻译,程序=数据结构+算法数据结构指的是存储,组织数据的方式.算法指的是为了解决实际业务问题而思考思路和方法,就叫:算法.2.算法的5大特性介绍算法具有独立性算法是解决问题的思路和方式,最重要的是思维,而不是语言,其(算法)可以通过多种语言进行演绎.5大特性有输入,需要传入1或者多个参数有输出,需要返回1个或者多个结果有穷性,执行
Python算法L5：贪心算法小熊同学哦 Python算法算法 python 贪心算法
Python贪心算法简介目录Python贪心算法简介贪心算法的基本步骤贪心算法的适用场景经典贪心算法问题1.**零钱兑换问题**2.**区间调度问题**3.**背包问题**贪心算法的优缺点优点：缺点：结语贪心算法（GreedyAlgorithm）是一种在每一步选择中都采取当前最优或最优解的算法。它的核心思想是，在保证每一步局部最优的情况下，希望通过贪心选择达到全局最优解。虽然贪心算法并不总能得到全
【RabbitMQ 项目】服务端：数据管理模块之绑定管理月夜星辉雪 rabbitmq 分布式
文章目录一.编写思路二.代码实践一.编写思路定义绑定信息类交换机名称队列名称绑定关键字：交换机的路由交换算法中会用到没有是否持久化的标志，因为绑定是否持久化取决于交换机和队列是否持久化，只有它们都持久化时绑定才需要持久化。绑定就好像一根绳子，两端连接着交换机和队列，当一方不存在，它就没有存在的必要了定义绑定持久化类构造函数：如果数据库文件不存在则创建，打开数据库，创建binding_table插入
非对称加密算法原理与应用2——RSA私钥加密文件私语茶馆云部署与开发架构及产品灵感记录 RSA2048 私钥加密
作者：私语茶馆1.相关章节（1）非对称加密算法原理与应用1——秘钥的生成-CSDN博客第一章节讲述的是创建秘钥对，并将公钥和私钥导出为文件格式存储。本章节继续讲如何利用私钥加密内容，包括从密钥库或文件中读取私钥，并用RSA算法加密文件和String。2.私钥加密的概述本文主要基于第一章节的RSA2048bit的非对称加密算法讲述如何利用私钥加密文件。这种加密后的文件，只能由该私钥对应的公钥来解密。
粒子群优化 (PSO) 在三维正弦波函数中的应用 subject625Ruben 机器学习人工智能 matlab 算法
在这篇博客中，我们将展示如何使用粒子群优化（PSO）算法求解三维正弦波函数，并通过增加正弦波扰动，使优化过程更加复杂和有趣。本文将介绍目标函数的定义、PSO参数设置以及算法执行的详细过程，并展示搜索空间中的动态过程和收敛曲线。1.目标函数定义我们使用的目标函数是一个三维正弦波函数，定义如下：objectiveFunc=@(x)sin(sqrt(x(1).^2+x(2).^2))+0.5*sin(5
非对称加密算法————RSA理论及详情 hu19930613
转自：https://www.kancloud.cn/kancloud/rsa_algorithm/48484一、一点历史1976年以前，所有的加密方法都是同一种模式：（1）甲方选择某一种加密规则，对信息进行加密；（2）乙方使用同一种规则，对信息进行解密。由于加密和解密使用同样规则（简称"密钥"），这被称为"对称加密算法"（Symmetric-keyalgorithm）。这种加密模式有一个最大弱点
ai绘画工具midjourney怎么下载？附作品管理教程设计师早上好
Midjourney是一款功能强大的AI绘画工具，它使用机器学习技术和深度神经网络等算法，可以生成各种艺术风格的绘画作品。在创意设计、广告宣传等方面有着广泛的应用前景。那么，ai绘画工具midjourney怎么下载？本文将为您介绍Midjourney的下载以及作品的相关管理。一、Midjourney下载Midjourney的下载非常简单，只需打开Midjourney官网（点击“GetMidjour
【加密算法基础——对称加密和非对称加密】 XWWW668899 网络安全服务器笔记
对称加密与非对称加密对称加密和非对称加密是两种基本的加密方法，各自有不同的特点和用途。以下是详细比较：1.对称加密特点密钥:使用相同的密钥进行加密和解密。发送方和接收方必须共享这个密钥。速度:通常速度较快，适合处理大量数据。实现:算法相对简单，计算效率高。常见算法AES(高级加密标准)DES(数据加密标准)3DES(三重数据加密标准)RC4(流密码)应用场景文件加密磁盘加密传输大量数据时的加密2.
【算法练习】IDEA集成leetcode插件实现快速刷 2401_84102892 2024年程序员学习算法 intellij-idea leetcode
============点击右侧边leetcode->设置->配置地址、用户名、密码、存放目录、文件模板用户名要登录后在账号信息里看模板代码1.codefilename!velocityTool.camelC
【加密算法基础——RSA 加密】 XWWW668899 网络服务器笔记 python
RSA加密RSA（Rivest-Shamir-Adleman）加密是非对称加密，一种广泛使用的公钥加密算法，主要用于安全数据传输。公钥用于加密，私钥用于解密。RSA加密算法的名称来源于其三位发明者的姓氏：R:RonRivestS:AdiShamirA:LeonardAdleman这三位计算机科学家在1977年共同提出了这一算法，并发表了相关论文。他们的工作为公钥加密的基础奠定了重要基础，使得安全通
Python实现关联规则推荐这孩子谁懂哈 Python Machine Learning python 关联规则机器学习
1.什么关联规则关联规则（AssociationRules）是反映一个事物与其他事物之间的相互依存性和关联性，如果两个或多个事物之间存在一定的关联关系，那么，其中一个事物就能通过其他事物预测到。关联规则是数据挖掘的一个重要技术，用于从大量数据中挖掘出有价值的数据项之间的相关关系。关联规则挖掘的最经典的例子就是沃尔玛的啤酒与尿布的故事，通过对超市购物篮数据进行分析，即顾客放入购物篮中不同商品之间的关
机器学习-聚类算法不良人龍木木机器学习机器学习算法聚类
机器学习-聚类算法1.AHC2.K-means3.SC4.MCL仅个人笔记，感谢点赞关注！1.AHC2.K-means3.SC传统谱聚类：个人对谱聚类算法的理解以及改进4.MCL目前仅专注于NLP的技术学习和分享感谢大家的关注与支持！
生成式地图制图 Bwywb_3 深度学习机器学习深度学习生成对抗网络
生成式地图制图（GenerativeCartography）是一种利用生成式算法和人工智能技术自动创建地图的技术。它结合了传统的地理信息系统（GIS）技术与现代生成模型（如深度学习、GANs等），能够根据输入的数据自动生成符合需求的地图。这种方法在城市规划、虚拟环境设计、游戏开发等多个领域具有应用前景。主要特点：自动化生成：通过算法和模型，系统能够根据输入的地理或空间数据自动生成地图，而无需人工逐
高性能javascript--算法和流程控制海淀萌狗
-for,while和do-while性能相当-避免使用for-in循环，==除非遍历一个属性量未知的对象==es5:for-in遍历的对象便不局限于数组，还可以遍历对象。原因：for-in每次迭代操作会同时搜索实例或者原型属性，for-in循环的每次迭代都会产生更多开销，因此要比其他循环类型慢，一般速度为其他类型循环的1/7。因此，除非明确需要迭代一个属性数量未知的对象，否则应避免使用for-i
深度 Qlearning：在直播推荐系统中的应用 AGI通用人工智能之禅程序员提升自我硅基计算碳基计算认知计算生物计算深度学习神经网络大数据 AIGC AGI LLM Java Python 架构设计 Agent 程序员实现财富自由
深度Q-learning：在直播推荐系统中的应用关键词：深度Q-learning,强化学习,直播推荐系统,个性化推荐1.背景介绍1.1问题的由来随着互联网技术的飞速发展,直播平台如雨后春笋般涌现。面对海量的直播内容,用户很难快速找到自己感兴趣的内容。因此,个性化推荐系统在直播平台中扮演着越来越重要的角色。1.2研究现状目前,主流的个性化推荐算法包括协同过滤、基于内容的推荐等。这些方法在一定程度上缓
JVM源码分析之堆外内存完全解读 HeapDump性能社区
概述广义的堆外内存说到堆外内存，那大家肯定想到堆内内存，这也是我们大家接触最多的，我们在jvm参数里通常设置-Xmx来指定我们的堆的最大值，不过这还不是我们理解的Java堆，-Xmx的值是新生代和老生代的和的最大值，我们在jvm参数里通常还会加一个参数-XX:MaxPermSize来指定持久代的最大值，那么我们认识的Java堆的最大值其实是-Xmx和-XX:MaxPermSize的总和，在分代算法
jQuery 跨域访问的三种方式 No 'Access-Control-Allow-Origin' header is present on the reque qiaolevip 每天进步一点点学习永无止境跨域众观千象
XMLHttpRequest cannot load http://v.xxx.com. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:63342' is therefore not allowed access. test.html:1
mysql 分区查询优化 annan211 java 分区优化 mysql
分区查询优化引入分区可以给查询带来一定的优势，但同时也会引入一些bug. 分区最大的优点就是优化器可以根据分区函数来过滤掉一些分区，通过分区过滤可以让查询扫描更少的数据。所以，对于访问分区表来说，很重要的一点是要在where 条件中带入分区，让优化器过滤掉无需访问的分区。可以通过查看explain执行计划，是否携带 partitions
MYSQL存储过程中使用游标 chicony Mysql存储过程
DELIMITER $$ DROP PROCEDURE IF EXISTS getUserInfo $$ CREATE PROCEDURE getUserInfo(in date_day datetime)-- -- 实例-- 存储过程名为：getUserInfo-- 参数为：date_day日期格式:2008-03-08-- BEGINdecla
mysql 和 sqlite 区别 Array_06 sqlite
转载： http://www.cnblogs.com/ygm900/p/3460663.html mysql 和 sqlite 区别 SQLITE是单机数据库。功能简约，小型化，追求最大磁盘效率 MYSQL是完善的服务器数据库。功能全面，综合化，追求最大并发效率 MYSQL、Sybase、Oracle等这些都是试用于服务器数据量大功能多需要安装，例如网站访问量比较大的。而sq
pinyin4j使用 oloz pinyin4j
首先需要pinyin4j的jar包支持；jar包已上传至附件内方法一:把汉字转换为拼音；例如：编程转换后则为biancheng /** * 将汉字转换为全拼 * @param src 你的需要转换的汉字 * @param isUPPERCASE 是否转换为大写的拼音； true:转换为大写；fal
微博发送私信随意而生微博
在前面文章中说了如和获取登陆时候所需要的cookie，现在只要拿到最后登陆所需要的cookie，然后抓包分析一下微博私信发送界面 http://weibo.com/message/history?uid=****&name=**** 可以发现其发送提交的Post请求和其中的数据，让后用程序模拟发送POST请求中的数据，带着cookie发送到私信的接入口，就可以实现发私信的功能了。
jsp 香水浓 jsp
JSP初始化容器载入JSP文件后，它会在为请求提供任何服务前调用jspInit()方法。如果您需要执行自定义的JSP初始化任务，复写jspInit()方法就行了 JSP执行这一阶段描述了JSP生命周期中一切与请求相关的交互行为，直到被销毁。当JSP网页完成初始化后
在 Windows 上安装 SVN Subversion 服务端 AdyZhang SVN
在 Windows 上安装 SVN Subversion 服务端2009-09-16高宏伟哈尔滨市道里区通达街291号最佳阅读效果请访问原地址：http://blog.donews.com/dukejoe/archive/2009/09/16/1560917.aspx 现在的Subversion已经足够稳定，而且已经进入了它的黄金时段。我们看到大量的项目都在使
android开发中如何使用 alertDialog从listView中删除数据？ aijuans android
我现在使用listView展示了很多的配置信息，我现在想在点击其中一条的时候填出 alertDialog,点击确认后就删除该条数据，（ ArrayAdapter ，ArrayList，listView 全部删除），我知道在下面的onItemLongClick 方法中参数 arg2 是选中的序号，但是我不知道如何继续处理下去 1 2 3
jdk-6u26-linux-x64.bin 安装 baalwolf linux
1.上传安装文件(jdk-6u26-linux-x64.bin) 2.修改权限 [root@localhost ~]# ls -l /usr/local/jdk-6u26-linux-x64.bin 3.执行安装文件 [root@localhost ~]# cd /usr/local [root@localhost local]# ./jdk-6u26-linux-x64.bin&nbs
MongoDB经典面试题集锦 BigBird2012 mongodb
1.什么是NoSQL数据库？NoSQL和RDBMS有什么区别？在哪些情况下使用和不使用NoSQL数据库？ NoSQL是非关系型数据库，NoSQL = Not Only SQL。关系型数据库采用的结构化的数据，NoSQL采用的是键值对的方式存储数据。在处理非结构化/半结构化的大数据时；在水平方向上进行扩展时；随时应对动态增加的数据项时可以优先考虑使用NoSQL数据库。在考虑数据库的成熟
JavaScript异步编程Promise模式的6个特性 bijian1013 JavaScript Promise
Promise是一个非常有价值的构造器，能够帮助你避免使用镶套匿名方法，而使用更具有可读性的方式组装异步代码。这里我们将介绍6个最简单的特性。在我们开始正式介绍之前，我们想看看Javascript Promise的样子： var p = new Promise(function(r
[Zookeeper学习笔记之八]Zookeeper源代码分析之Zookeeper.ZKWatchManager bit1129 zookeeper
ClientWatchManager接口 //接口的唯一方法materialize用于确定那些Watcher需要被通知 //确定Watcher需要三方面的因素1.事件状态 2.事件类型 3.znode的path public interface ClientWatchManager { /** * Return a set of watchers that should
【Scala十五】Scala核心九：隐式转换之二 bit1129 scala
隐式转换存在的必要性，在Java Swing中，按钮点击事件的处理，转换为Scala的的写法如下： val button = new JButton button.addActionListener( new ActionListener { def actionPerformed(event: ActionEvent) {
Android JSON数据的解析与封装小Demo ronin47
转自：http://www.open-open.com/lib/view/open1420529336406.html package com.example.jsondemo; import org.json.JSONArray; import org.json.JSONException; import org.json.JSONObject; impor
[设计]字体创意设计方法谈 brotherlamp UI ui自学 ui视频 ui教程 ui资料
从古至今，文字在我们的生活中是必不可少的事物，我们不能想象没有文字的世界将会是怎样。在平面设计中，UI设计师在文字上所花的心思和功夫最多，因为文字能直观地表达UI设计师所的意念。在文字上的创造设计，直接反映出平面作品的主题。如设计一幅戴尔笔记本电脑的广告海报，假设海报上没有出现“戴尔”两个文字，即使放上所有戴尔笔记本电脑的图片都不能让人们得知这些电脑是什么品牌。只要写上“戴尔笔
单调队列-用一个长度为k的窗在整数数列上移动，求窗里面所包含的数的最大值 bylijinnan java 算法面试题
import java.util.LinkedList; /* 单调队列滑动窗口单调队列是这样的一个队列：队列里面的元素是有序的，是递增或者递减题目：给定一个长度为N的整数数列a(i),i=0,1,...,N-1和窗长度k. 要求：f(i) = max{a(i-k+1),a(i-k+2),..., a(i)},i = 0,1,...,N-1 问题的另一种描述就
struts2处理一个form多个submit chiangfai struts2
web应用中，为完成不同工作，一个jsp的form标签可能有多个submit。如下代码： <s:form action="submit" method="post" namespace="/my"> <s:textfield name="msg" label="叙述：">
shell查找上个月，陷阱及野路子 chenchao051 shell
date -d "-1 month" +%F 以上这段代码，假如在2012/10/31执行，结果并不会出现你预计的9月份，而是会出现八月份，原因是10月份有31天，9月份30天，所以-1 month在10月份看来要减去31天，所以直接到了8月31日这天，这不靠谱。野路子解决：假设当天日期大于15号
mysql导出数据中文乱码问题 daizj mysql 中文乱码导数据
解决mysql导入导出数据乱码问题方法：１、进入mysql，通过如下命令查看数据库编码方式： mysql> show variables like 'character_set_%'; +--------------------------+----------------------------------------+ | Variable_name&nbs
SAE部署Smarty出现：Uncaught exception 'SmartyException' with message 'unable to write dcj3sjt126com PHP smarty sae
对于SAE出现的问题：Uncaught exception 'SmartyException' with message 'unable to write file...。官方给出了详细的FAQ：http://sae.sina.com.cn/?m=faqs&catId=11#show_213 解决方案为： 01 $path
《教父》系列台词 dcj3sjt126com
Your love is also your weak point. 你的所爱同时也是你的弱点。 If anything in this life is certain, if history has taught us anything, it is that you can kill anyone. 不顾家的人永远不可能成为一个真正的男人。 &
mongodb安装与使用 dyy_gusi mongo
一.MongoDB安装和启动,widndows和linux基本相同 1.下载数据库, linux:mongodb-linux-x86_64-ubuntu1404-3.0.3.tgz 2.解压文件,并且放置到合适的位置 tar -vxf mongodb-linux-x86_64-ubun
Git排除目录 geeksun git
在Git的版本控制中，可能有些文件是不需要加入控制的，那我们在提交代码时就需要忽略这些文件，下面讲讲应该怎么给Git配置一些忽略规则。有三种方法可以忽略掉这些文件，这三种方法都能达到目的，只不过适用情景不一样。 1. 针对单一工程排除文件这种方式会让这个工程的所有修改者在克隆代码的同时，也能克隆到过滤规则，而不用自己再写一份，这就能保证所有修改者应用的都是同一
Ubuntu 创建开机自启动脚本的方法 hongtoushizi ubuntu
转载自： http://rongjih.blog.163.com/blog/static/33574461201111504843245/ Ubuntu 创建开机自启动脚本的步骤如下： 1) 将你的启动脚本复制到 /etc/init.d目录下以下假设你的脚本文件名为 test。 2) 设置脚本文件的权限 $ sudo chmod 755
第八章流量复制/AB测试/协程 jinnianshilongnian nginx lua coroutine
流量复制在实际开发中经常涉及到项目的升级，而该升级不能简单的上线就完事了，需要验证该升级是否兼容老的上线，因此可能需要并行运行两个项目一段时间进行数据比对和校验，待没问题后再进行上线。这其实就需要进行流量复制，把流量复制到其他服务器上，一种方式是使用如tcpcopy引流；另外我们还可以使用nginx的HttpLuaModule模块中的ngx.location.capture_multi进行并发
电商系统商品表设计 lkl
DROP TABLE IF EXISTS `category`; -- 类目表 /*!40101 SET @saved_cs_client = @@character_set_client */; /*!40101 SET character_set_client = utf8 */; CREATE TABLE `category` ( `id` int(11) NOT NUL
修改phpMyAdmin导入SQL文件的大小限制 pda158 sql mysql
　用phpMyAdmin导入mysql数据库时，我的10M的数据库不能导入，提示mysql数据库最大只能导入2M。　　 phpMyAdmin数据库导入出错：　　You probably tried to upload too large file. Please refer to documentation for ways to workaround this limit.
Tomcat性能调优方案 Sobfist apache jvm tomcat 应用服务器
一、操作系统调优对于操作系统优化来说，是尽可能的增大可使用的内存容量、提高CPU的频率，保证文件系统的读写速率等。经过压力测试验证，在并发连接很多的情况下，CPU的处理能力越强，系统运行速度越快。。【适用场景】任何项目。二、Java虚拟机调优应该选择SUN的JVM，在满足项目需要的前提下，尽量选用版本较高的JVM，一般来说高版本产品在速度和效率上比低版本会有改进。 J
SQLServer学习笔记 vipbooks 数据结构 xml
1、create database school 创建数据库school 2、drop database school 删除数据库school 3、use school 连接到school数据库，使其成为当前数据库 4、create table class(classID int primary key identity not null) 创建一个名为class的表，其有一

集成学习01_xgboost参数讲解与实战

一.xgboost 模型参数介绍

二.xgboost 两种方式实现

三. 网格搜索最优xgboost参数

一.XGBoost的参数

1 通用参数

1）booster[默认gbtree]

2）silent[默认0]

3）nthread[默认值为最大可能的线程数]

4）num_feature [set automatically by xgboost, no need to be set by user]

2 booster参数

1）eta[默认0.3]

2）min_child_weight[默认1]

3）max_depth[默认6]

4）max_leaf_nodes

5）gamma[默认0]

6）max_delta_step[默认0]

7）subsample[默认1]

8）colsample_bytree[默认1]

9）colsample_bylevel[默认1]

10）lambda[默认1]

11）alpha[默认1]

12）scale_pos_weight[默认1]

13) Parameter for Linear Booster

lambda_bias

3 学习目标参数

1）objective[默认reg:linear]

2）eval_metric[默认值取决于objective参数的取值]

3）seed(默认0)

4）sklearn 参数对照

4.平台控制参数 Console Parameters

二.xgboost 实现

1.导入相关包

2.数据处理

3.结合sklearn的xgboot模型

step01-拟合模型

step02-评价模型

step03-利用模型预测

step04-保存和调用模型

注意点

4.直接采用xgboost的模型

step-01 构建参数

step-02 处理数据

step-03 拟合模型

step-04 用模型预测

注意：

step-05 评价模型效果

step-06 保存模型并调用

三. 网格搜索最优xgboost参数

1.step-01 配置参数列表

step-02 选择待优化模型

step-03 进行网格搜索 拟合模型

step-04 确认最优参数

step-05 选取最优模型

step-06 评价最优模型

step-07 保存并调用模型

你可能感兴趣的:(python机器学习,数据挖掘,算法)

step-03 进行网格搜索拟合模型