雪龙无敌

集成学习02_catboost参数介绍与实战

一.catboost 模型参数介绍

二.catboost实现

三. 网格搜索最优catboost模型

# 一.catboost 模型参数介绍

catboost参数分为通用参数，性能参数和默认参数三类，由于参数众多，很多参数不重要，只解释部分重要的参数，训练时需要重点考虑的。

1.通用参数

1.loss_function 损失函数
支持的有RMSE,
Logloss,
MAE,
CrossEntropy,
Quantile,
LogLinQuantile,
Multiclass,
MultiClassOneVsAll,
MAPE,
Poisson。
默认RMSE。
2.custom_metric
训练过程中输出的度量值。这些功能未经优化，仅出于信息目的显示。支持以下：
RMSE
Logloss
MAE
CrossEntropy
Recall
Precision
F1
Accuracy
AUC
R2
默认None。
3.eval_metric
用于过拟合检验（设置True）和最佳模型选择（设置True）的loss function，用于优化。
4.iterations
最大树数。默认1000。
5.learning_rate
学习率。默认0.03。
6.random_seed
训练时候的随机种子
7.l2_leaf_reg L2
正则参数。默认3
8.bootstrap_type
定义权重计算逻辑，可选参数：Poisson (supported for GPU only)/Bayesian/Bernoulli/No，默认为Bayesian
9.bagging_temperature
贝叶斯套袋控制强度，区间[0, 1]。默认1。
10.subsample
设置样本率，当bootstrap_type为Poisson或Bernoulli时使用，默认66
11.sampling_frequency
设置创建树时的采样频率，可选值PerTree/PerTreeLevel，默认为PerTreeLevel
12.random_strength
分数标准差乘数。默认1。
13.use_best_model
设置此参数时，需要提供测试数据，树的个数通过训练参数和优化loss function获得。默认False。
14.best_model_min_trees
最佳模型应该具有的树的最小数目。
15.depth
树深，最大16，建议在1到10之间。默认6。
16.ignored_features
忽略数据集中的某些特征。默认None。
17.one_hot_max_size
如果feature包含的不同值的数目超过了指定值，将feature转化为float。默认False
18.has_time
在将categorical features转化为numerical
19.features
和选择树结构时，顺序选择输入数据。默认False（随机）
20.rsm
随机子空间（Random subspace method）。默认1。
21.nan_mode
处理输入数据中缺失值的方法，包括：
Forbidden(禁止存在缺失)，
Min(用最小值补)，
Max(用最大值补)。
默认Min。
22.fold_permutation_block_size
数据集中的对象在随机排列之前按块分组。此参数定义块的大小。值越小，训练越慢。较大的值可能导致质量下降。
23.leaf_estimation_method
计算叶子值的方法，Newton/ Gradient。默认Gradient。
24.leaf_estimation_iterations
计算叶子值时梯度步数。
25.leaf_estimation_backtracking
在梯度下降期间要使用的回溯类型。
26.fold_len_multiplier folds
长度系数。设置大于1的参数，在参数较小时获得最佳结果。默认2。
27.approx_on_full_history
计算近似值，False：使用1／fold_len_multiplier计算；True：使用fold中前面所有行计算。默认False。
28.class_weights
类别的权重。默认None。
29.scale_pos_weight
二进制分类中class 1的权重。该值用作class 1中对象权重的乘数。
30.boosting_type
增压方案
31.allow_const_label
使用它为所有对象训练具有相同标签值的数据集的模型。默认为False

2.性能参数

1.thread_count=-1：
训练时所用的cpu/gpu核数
2.used_ram_limit=None：
CTR问题，计算时的内存限制
3.gpu_ram_part=None：
GPU内存限制

3.默认参数

CatBoost默认参数：

‘iterations’: 1000,
‘learning_rate’:0.03,
‘l2_leaf_reg’:3,
‘bagging_temperature’:1,
‘subsample’:0.66,
‘random_strength’:1,
‘depth’:6,
‘rsm’:1,
‘one_hot_max_size’:2
‘leaf_estimation_method’:’Gradient’,
‘fold_len_multiplier’:2,
‘border_count’:128,

4.CatBoostClassifier方法参数

1.fit

X: 输入数据数据类型可以是，list; pandas.DataFrame; pandas.Series
y=None
cat_features=None: 拿来做处理的类别特征
sample_weight=None: 输入数据的样本权重
logging_level=None: 控制是否输出日志信息，或者何种信息
plot=False: 训练过程中，绘制，度量值，所用时间等
eval_set=None: 验证集合，数据类型list(X, y)tuples
baseline=None
use_best_model=None
verbose=None

2.predict

返回验证样本所属类别，数据类型为np.array

3.predict_proba

返回验证样本所属类别的概率，数据类型为np.array

5.get_feature_importance

6.eval_metrics

7.save_model

8.load_model

9.get_params

10.score

二.catboost实现

1.导入相关包

import pandas as pd, numpy as np
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn import metrics
import catboost as cb
import os
import joblib

2.数据处理

data=pd.read_excel(r'D:\Ensemble_Learning\car_coupon01.xlsx')
data.head(5)

	ID	destination	passanger	toCoupon_GEQ15min	direction_same	direction_opp	gender	age	maritalStatus	...	Bar	CoffeeHouse	CarryAway	RestaurantLessThan20	Restaurant20To50	weather	time	coupon	expiration	Y
0	11263	No Urgent Place	Friend(s)	0	0	1	Male	50plus	Widowed	...	never	never	less1	1~3	less1	Sunny	2PM	Coffee House	1d	1
1	20136	Work	Alone	1	1	0	Female	26	Married partner	...	never	never	1~3	4~8	less1	Sunny	7AM	Bar	1d	0
2	14763	Work	Alone	1	0	1	Female	50plus	Single	...	never	never	less1	4~8	less1	Sunny	7AM	Coffee House	1d	0
3	12612	No Urgent Place	Kid(s)	1	0	1	Female	41	Married partner	...	never	1~3	1~3	1~3	less1	Sunny	10AM	Carry out & Take away	2h	0
4	17850	No Urgent Place	Partner	1	0	1	Female	31	Married partner	...	less1	less1	gt8	4~8	less1	Snowy	2PM	Coffee House	2h	0

5 rows × 23 columns

设置id

data.set_index('ID',inplace=True)

指定分类变量的行索引

cat_features_index = [0, 1, 6,7, 8,9,10,11,12,13,14,15,16,17,18,19,20]

#填充数据
data.fillna(-999,inplace=True)
#划分训练集和测试结合
train, test, y_train, y_test = train_test_split(data.drop(["Y"], axis=1), data["Y"],
                                                random_state=10, test_size=0.3)

3.撰写评价函数

def model_eval(m, train, test):
    print('train_roc_auc_score:',metrics.roc_auc_score(y_train, m.predict_proba(train)[:, 1]))
    print('test_roc_auc_score:',metrics.roc_auc_score(y_test, m.predict_proba(test)[:, 1]))
    print('train_accuracy_score:',metrics.accuracy_score(y_train,  m.predict(train)))
    print('test_accuracy_score:',metrics.accuracy_score(y_test, m.predict(test)))
    print('train_precision_score:',metrics.precision_score(y_train, m.predict(train)))
    print('test__precision_score:',metrics.precision_score(y_test, m.predict(test)))
    print('train_recall_score:',metrics.recall_score(y_train, m.predict(train)))
    print('test_recall_score:',metrics.recall_score(y_test, m.predict(test)))
    print('train_f1_score:',metrics.f1_score(y_train, m.predict(train)))
    print('test_f1_score:',metrics.f1_score(y_test, m.predict(test)))

4.拟合catboost模型

clf = cb.CatBoostClassifier(eval_metric='Accuracy', one_hot_max_size=25, depth=7, 
                             l2_leaf_reg=10,min_data_in_leaf=30,penalties_coefficient=15,
                             subsample=0.55, colsample_bylevel=0.58,iterations=800,
                             ctr_leaf_count_limit=200,
                            learning_rate=0.15,random_seed=11)
clf.fit(train, y_train, cat_features=cat_features_index)

0:	learn: 0.6821429	total: 145ms	remaining: 1m 56s
1:	learn: 0.7114286	total: 147ms	remaining: 58.6s
2:	learn: 0.7078571	total: 148ms	remaining: 39.3s
3:	learn: 0.7164286	total: 149ms	remaining: 29.7s
4:	learn: 0.7485714	total: 150ms	remaining: 23.9s
5:	learn: 0.7521429	total: 152ms	remaining: 20.1s
6:	learn: 0.7585714	total: 153ms	remaining: 17.3s
7:	learn: 0.7564286	total: 154ms	remaining: 15.3s
8:	learn: 0.7707143	total: 155ms	remaining: 13.6s
9:	learn: 0.7707143	total: 157ms	remaining: 12.4s
10:	learn: 0.7764286	total: 158ms	remaining: 11.3s
11:	learn: 0.7950000	total: 159ms	remaining: 10.4s
12:	learn: 0.8007143	total: 160ms	remaining: 9.7s
13:	learn: 0.8107143	total: 162ms	remaining: 9.07s
14:	learn: 0.8114286	total: 163ms	remaining: 8.52s
15:	learn: 0.8114286	total: 164ms	remaining: 8.03s
16:	learn: 0.8142857	total: 165ms	remaining: 7.61s
17:	learn: 0.8185714	total: 166ms	remaining: 7.22s
18:	learn: 0.8221429	total: 168ms	remaining: 6.89s
19:	learn: 0.8257143	total: 168ms	remaining: 6.57s
20:	learn: 0.8257143	total: 170ms	remaining: 6.29s
21:	learn: 0.8285714	total: 171ms	remaining: 6.04s
22:	learn: 0.8285714	total: 172ms	remaining: 5.81s
23:	learn: 0.8271429	total: 173ms	remaining: 5.6s
24:	learn: 0.8307143	total: 175ms	remaining: 5.41s
25:	learn: 0.8335714	total: 176ms	remaining: 5.24s
26:	learn: 0.8378571	total: 177ms	remaining: 5.07s
27:	learn: 0.8392857	total: 178ms	remaining: 4.91s
28:	learn: 0.8414286	total: 179ms	remaining: 4.77s
29:	learn: 0.8428571	total: 180ms	remaining: 4.63s
30:	learn: 0.8492857	total: 182ms	remaining: 4.5s
31:	learn: 0.8500000	total: 183ms	remaining: 4.39s
32:	learn: 0.8492857	total: 184ms	remaining: 4.28s
33:	learn: 0.8528571	total: 186ms	remaining: 4.18s
34:	learn: 0.8500000	total: 187ms	remaining: 4.09s
35:	learn: 0.8542857	total: 188ms	remaining: 4s
36:	learn: 0.8600000	total: 190ms	remaining: 3.91s
37:	learn: 0.8614286	total: 191ms	remaining: 3.83s
38:	learn: 0.8600000	total: 192ms	remaining: 3.75s
39:	learn: 0.8635714	total: 193ms	remaining: 3.67s
40:	learn: 0.8628571	total: 195ms	remaining: 3.6s
41:	learn: 0.8650000	total: 196ms	remaining: 3.53s
42:	learn: 0.8650000	total: 197ms	remaining: 3.47s
43:	learn: 0.8678571	total: 198ms	remaining: 3.41s
44:	learn: 0.8664286	total: 200ms	remaining: 3.35s
45:	learn: 0.8635714	total: 201ms	remaining: 3.29s
46:	learn: 0.8657143	total: 202ms	remaining: 3.23s
47:	learn: 0.8700000	total: 203ms	remaining: 3.18s
48:	learn: 0.8728571	total: 204ms	remaining: 3.13s
49:	learn: 0.8742857	total: 206ms	remaining: 3.08s
50:	learn: 0.8814286	total: 207ms	remaining: 3.04s
51:	learn: 0.8835714	total: 208ms	remaining: 2.99s
52:	learn: 0.8835714	total: 209ms	remaining: 2.95s
53:	learn: 0.8857143	total: 210ms	remaining: 2.9s
54:	learn: 0.8842857	total: 211ms	remaining: 2.86s
55:	learn: 0.8835714	total: 213ms	remaining: 2.83s
56:	learn: 0.8871429	total: 214ms	remaining: 2.79s
57:	learn: 0.8885714	total: 215ms	remaining: 2.75s
58:	learn: 0.8900000	total: 216ms	remaining: 2.71s
59:	learn: 0.8978571	total: 218ms	remaining: 2.68s
60:	learn: 0.9021429	total: 219ms	remaining: 2.65s
61:	learn: 0.9042857	total: 220ms	remaining: 2.62s
62:	learn: 0.9064286	total: 221ms	remaining: 2.59s
63:	learn: 0.9064286	total: 223ms	remaining: 2.56s
64:	learn: 0.9078571	total: 224ms	remaining: 2.53s
65:	learn: 0.9121429	total: 225ms	remaining: 2.5s
66:	learn: 0.9107143	total: 226ms	remaining: 2.48s
67:	learn: 0.9135714	total: 228ms	remaining: 2.45s
68:	learn: 0.9135714	total: 229ms	remaining: 2.42s
69:	learn: 0.9185714	total: 230ms	remaining: 2.4s
70:	learn: 0.9185714	total: 231ms	remaining: 2.37s
71:	learn: 0.9250000	total: 232ms	remaining: 2.35s
72:	learn: 0.9250000	total: 234ms	remaining: 2.33s
73:	learn: 0.9307143	total: 235ms	remaining: 2.3s
74:	learn: 0.9335714	total: 236ms	remaining: 2.28s
75:	learn: 0.9342857	total: 237ms	remaining: 2.26s
76:	learn: 0.9335714	total: 238ms	remaining: 2.24s
77:	learn: 0.9328571	total: 240ms	remaining: 2.22s
78:	learn: 0.9328571	total: 241ms	remaining: 2.2s
79:	learn: 0.9371429	total: 242ms	remaining: 2.18s
80:	learn: 0.9378571	total: 243ms	remaining: 2.16s
81:	learn: 0.9378571	total: 245ms	remaining: 2.14s
82:	learn: 0.9414286	total: 246ms	remaining: 2.13s
83:	learn: 0.9421429	total: 247ms	remaining: 2.11s
84:	learn: 0.9428571	total: 248ms	remaining: 2.09s
85:	learn: 0.9428571	total: 250ms	remaining: 2.07s
86:	learn: 0.9450000	total: 251ms	remaining: 2.06s
87:	learn: 0.9464286	total: 252ms	remaining: 2.04s
88:	learn: 0.9450000	total: 253ms	remaining: 2.02s
89:	learn: 0.9471429	total: 254ms	remaining: 2.01s
90:	learn: 0.9471429	total: 256ms	remaining: 1.99s
91:	learn: 0.9464286	total: 257ms	remaining: 1.98s
92:	learn: 0.9478571	total: 258ms	remaining: 1.96s
93:	learn: 0.9485714	total: 259ms	remaining: 1.95s
94:	learn: 0.9478571	total: 260ms	remaining: 1.93s
95:	learn: 0.9500000	total: 262ms	remaining: 1.92s
96:	learn: 0.9507143	total: 263ms	remaining: 1.9s
97:	learn: 0.9507143	total: 264ms	remaining: 1.89s
98:	learn: 0.9500000	total: 265ms	remaining: 1.88s
99:	learn: 0.9507143	total: 266ms	remaining: 1.86s
100:	learn: 0.9514286	total: 268ms	remaining: 1.85s
101:	learn: 0.9571429	total: 269ms	remaining: 1.84s
102:	learn: 0.9592857	total: 270ms	remaining: 1.83s
103:	learn: 0.9621429	total: 271ms	remaining: 1.81s
104:	learn: 0.9628571	total: 273ms	remaining: 1.8s
105:	learn: 0.9628571	total: 274ms	remaining: 1.79s
106:	learn: 0.9657143	total: 275ms	remaining: 1.78s
107:	learn: 0.9657143	total: 276ms	remaining: 1.77s
108:	learn: 0.9664286	total: 277ms	remaining: 1.76s
109:	learn: 0.9671429	total: 279ms	remaining: 1.75s
110:	learn: 0.9671429	total: 280ms	remaining: 1.74s
111:	learn: 0.9671429	total: 281ms	remaining: 1.73s
112:	learn: 0.9678571	total: 282ms	remaining: 1.72s
113:	learn: 0.9692857	total: 284ms	remaining: 1.71s
114:	learn: 0.9707143	total: 285ms	remaining: 1.7s
115:	learn: 0.9714286	total: 286ms	remaining: 1.69s
116:	learn: 0.9714286	total: 287ms	remaining: 1.68s
117:	learn: 0.9728571	total: 289ms	remaining: 1.67s
118:	learn: 0.9750000	total: 290ms	remaining: 1.66s
119:	learn: 0.9757143	total: 291ms	remaining: 1.65s
120:	learn: 0.9757143	total: 292ms	remaining: 1.64s
121:	learn: 0.9764286	total: 293ms	remaining: 1.63s
122:	learn: 0.9757143	total: 295ms	remaining: 1.62s
123:	learn: 0.9764286	total: 296ms	remaining: 1.61s
124:	learn: 0.9771429	total: 297ms	remaining: 1.6s
125:	learn: 0.9771429	total: 298ms	remaining: 1.6s
126:	learn: 0.9778571	total: 300ms	remaining: 1.59s
127:	learn: 0.9771429	total: 301ms	remaining: 1.58s
128:	learn: 0.9778571	total: 302ms	remaining: 1.57s
129:	learn: 0.9771429	total: 303ms	remaining: 1.56s
130:	learn: 0.9771429	total: 305ms	remaining: 1.56s
131:	learn: 0.9778571	total: 306ms	remaining: 1.55s
132:	learn: 0.9785714	total: 307ms	remaining: 1.54s
133:	learn: 0.9785714	total: 308ms	remaining: 1.53s
134:	learn: 0.9792857	total: 309ms	remaining: 1.52s
135:	learn: 0.9785714	total: 311ms	remaining: 1.52s
136:	learn: 0.9792857	total: 312ms	remaining: 1.51s
137:	learn: 0.9792857	total: 313ms	remaining: 1.5s
138:	learn: 0.9800000	total: 314ms	remaining: 1.49s
139:	learn: 0.9814286	total: 315ms	remaining: 1.49s
140:	learn: 0.9814286	total: 318ms	remaining: 1.49s
141:	learn: 0.9821429	total: 319ms	remaining: 1.48s
142:	learn: 0.9842857	total: 321ms	remaining: 1.47s
143:	learn: 0.9835714	total: 322ms	remaining: 1.47s
144:	learn: 0.9857143	total: 323ms	remaining: 1.46s
145:	learn: 0.9871429	total: 324ms	remaining: 1.45s
146:	learn: 0.9857143	total: 326ms	remaining: 1.45s
147:	learn: 0.9864286	total: 327ms	remaining: 1.44s
148:	learn: 0.9885714	total: 329ms	remaining: 1.44s
149:	learn: 0.9878571	total: 330ms	remaining: 1.43s
150:	learn: 0.9885714	total: 332ms	remaining: 1.43s
151:	learn: 0.9892857	total: 333ms	remaining: 1.42s
152:	learn: 0.9885714	total: 334ms	remaining: 1.41s
153:	learn: 0.9892857	total: 336ms	remaining: 1.41s
154:	learn: 0.9892857	total: 338ms	remaining: 1.41s
155:	learn: 0.9885714	total: 339ms	remaining: 1.4s
156:	learn: 0.9885714	total: 341ms	remaining: 1.39s
157:	learn: 0.9878571	total: 342ms	remaining: 1.39s
158:	learn: 0.9878571	total: 343ms	remaining: 1.38s
159:	learn: 0.9900000	total: 345ms	remaining: 1.38s
160:	learn: 0.9907143	total: 346ms	remaining: 1.37s
161:	learn: 0.9907143	total: 347ms	remaining: 1.37s
162:	learn: 0.9935714	total: 349ms	remaining: 1.36s
163:	learn: 0.9942857	total: 350ms	remaining: 1.36s
164:	learn: 0.9935714	total: 352ms	remaining: 1.35s
165:	learn: 0.9950000	total: 353ms	remaining: 1.35s
166:	learn: 0.9950000	total: 354ms	remaining: 1.34s
167:	learn: 0.9950000	total: 355ms	remaining: 1.34s
168:	learn: 0.9950000	total: 357ms	remaining: 1.33s
169:	learn: 0.9957143	total: 358ms	remaining: 1.33s
170:	learn: 0.9957143	total: 359ms	remaining: 1.32s
171:	learn: 0.9957143	total: 360ms	remaining: 1.32s
172:	learn: 0.9957143	total: 362ms	remaining: 1.31s
173:	learn: 0.9957143	total: 363ms	remaining: 1.3s
174:	learn: 0.9957143	total: 364ms	remaining: 1.3s
175:	learn: 0.9957143	total: 365ms	remaining: 1.29s
176:	learn: 0.9957143	total: 367ms	remaining: 1.29s
177:	learn: 0.9957143	total: 368ms	remaining: 1.28s
178:	learn: 0.9957143	total: 369ms	remaining: 1.28s
179:	learn: 0.9957143	total: 370ms	remaining: 1.27s
180:	learn: 0.9957143	total: 371ms	remaining: 1.27s
181:	learn: 0.9957143	total: 373ms	remaining: 1.26s
182:	learn: 0.9971429	total: 374ms	remaining: 1.26s
183:	learn: 0.9971429	total: 375ms	remaining: 1.25s
184:	learn: 0.9971429	total: 376ms	remaining: 1.25s
185:	learn: 0.9964286	total: 377ms	remaining: 1.25s
186:	learn: 0.9964286	total: 379ms	remaining: 1.24s
187:	learn: 0.9964286	total: 380ms	remaining: 1.24s
188:	learn: 0.9964286	total: 381ms	remaining: 1.23s
189:	learn: 0.9964286	total: 382ms	remaining: 1.23s
190:	learn: 0.9971429	total: 384ms	remaining: 1.22s
191:	learn: 0.9971429	total: 385ms	remaining: 1.22s
192:	learn: 0.9971429	total: 386ms	remaining: 1.21s
193:	learn: 0.9971429	total: 387ms	remaining: 1.21s
194:	learn: 0.9971429	total: 389ms	remaining: 1.21s
195:	learn: 0.9971429	total: 390ms	remaining: 1.2s
196:	learn: 0.9971429	total: 391ms	remaining: 1.2s
197:	learn: 0.9971429	total: 392ms	remaining: 1.19s
198:	learn: 0.9971429	total: 393ms	remaining: 1.19s
199:	learn: 0.9971429	total: 394ms	remaining: 1.18s
200:	learn: 0.9971429	total: 396ms	remaining: 1.18s
201:	learn: 0.9971429	total: 397ms	remaining: 1.18s
202:	learn: 0.9971429	total: 398ms	remaining: 1.17s
203:	learn: 0.9971429	total: 399ms	remaining: 1.17s
204:	learn: 0.9978571	total: 401ms	remaining: 1.16s
205:	learn: 0.9978571	total: 402ms	remaining: 1.16s
206:	learn: 0.9978571	total: 403ms	remaining: 1.15s
207:	learn: 0.9971429	total: 404ms	remaining: 1.15s
208:	learn: 0.9978571	total: 405ms	remaining: 1.15s
209:	learn: 0.9978571	total: 407ms	remaining: 1.14s
210:	learn: 0.9978571	total: 408ms	remaining: 1.14s
211:	learn: 0.9978571	total: 409ms	remaining: 1.13s
212:	learn: 0.9985714	total: 410ms	remaining: 1.13s
213:	learn: 0.9978571	total: 412ms	remaining: 1.13s
214:	learn: 0.9978571	total: 413ms	remaining: 1.12s
215:	learn: 0.9978571	total: 414ms	remaining: 1.12s
216:	learn: 0.9978571	total: 415ms	remaining: 1.12s
217:	learn: 0.9978571	total: 417ms	remaining: 1.11s
218:	learn: 0.9978571	total: 418ms	remaining: 1.11s
219:	learn: 0.9978571	total: 419ms	remaining: 1.1s
220:	learn: 0.9978571	total: 420ms	remaining: 1.1s
221:	learn: 0.9978571	total: 422ms	remaining: 1.1s
222:	learn: 0.9978571	total: 423ms	remaining: 1.09s
223:	learn: 0.9978571	total: 424ms	remaining: 1.09s
224:	learn: 0.9978571	total: 425ms	remaining: 1.09s
225:	learn: 0.9978571	total: 427ms	remaining: 1.08s
226:	learn: 0.9978571	total: 428ms	remaining: 1.08s
227:	learn: 0.9978571	total: 429ms	remaining: 1.08s
228:	learn: 0.9985714	total: 430ms	remaining: 1.07s
229:	learn: 0.9985714	total: 432ms	remaining: 1.07s
230:	learn: 0.9985714	total: 433ms	remaining: 1.06s
231:	learn: 0.9978571	total: 434ms	remaining: 1.06s
232:	learn: 0.9985714	total: 435ms	remaining: 1.06s
233:	learn: 0.9985714	total: 436ms	remaining: 1.05s
234:	learn: 0.9985714	total: 438ms	remaining: 1.05s
235:	learn: 0.9985714	total: 439ms	remaining: 1.05s
236:	learn: 0.9985714	total: 440ms	remaining: 1.04s
237:	learn: 0.9985714	total: 441ms	remaining: 1.04s
238:	learn: 0.9985714	total: 442ms	remaining: 1.04s
239:	learn: 0.9985714	total: 444ms	remaining: 1.03s
240:	learn: 0.9992857	total: 445ms	remaining: 1.03s
241:	learn: 0.9992857	total: 446ms	remaining: 1.03s
242:	learn: 0.9992857	total: 447ms	remaining: 1.02s
243:	learn: 0.9992857	total: 448ms	remaining: 1.02s
244:	learn: 0.9992857	total: 449ms	remaining: 1.02s
245:	learn: 0.9992857	total: 451ms	remaining: 1.01s
246:	learn: 0.9992857	total: 452ms	remaining: 1.01s
247:	learn: 0.9992857	total: 453ms	remaining: 1.01s
248:	learn: 0.9992857	total: 454ms	remaining: 1s
249:	learn: 0.9992857	total: 455ms	remaining: 1s
250:	learn: 0.9992857	total: 457ms	remaining: 999ms
251:	learn: 0.9992857	total: 458ms	remaining: 996ms
252:	learn: 0.9992857	total: 459ms	remaining: 993ms
253:	learn: 0.9992857	total: 460ms	remaining: 990ms
254:	learn: 0.9992857	total: 462ms	remaining: 987ms
255:	learn: 0.9992857	total: 463ms	remaining: 984ms
256:	learn: 0.9992857	total: 464ms	remaining: 981ms
257:	learn: 0.9992857	total: 465ms	remaining: 978ms
258:	learn: 0.9992857	total: 467ms	remaining: 975ms
259:	learn: 0.9992857	total: 468ms	remaining: 971ms
260:	learn: 0.9992857	total: 469ms	remaining: 968ms
261:	learn: 0.9992857	total: 470ms	remaining: 965ms
262:	learn: 0.9992857	total: 471ms	remaining: 963ms
263:	learn: 0.9992857	total: 473ms	remaining: 960ms
264:	learn: 0.9992857	total: 474ms	remaining: 957ms
265:	learn: 0.9992857	total: 475ms	remaining: 954ms
266:	learn: 0.9992857	total: 476ms	remaining: 951ms
267:	learn: 0.9992857	total: 478ms	remaining: 948ms
268:	learn: 0.9992857	total: 479ms	remaining: 945ms
269:	learn: 0.9992857	total: 480ms	remaining: 943ms
270:	learn: 0.9992857	total: 481ms	remaining: 940ms
271:	learn: 0.9992857	total: 482ms	remaining: 937ms
272:	learn: 0.9992857	total: 484ms	remaining: 934ms
273:	learn: 0.9992857	total: 485ms	remaining: 931ms
274:	learn: 0.9992857	total: 486ms	remaining: 928ms
275:	learn: 0.9992857	total: 487ms	remaining: 925ms
276:	learn: 0.9992857	total: 488ms	remaining: 922ms
277:	learn: 0.9992857	total: 489ms	remaining: 919ms
278:	learn: 0.9992857	total: 491ms	remaining: 916ms
279:	learn: 0.9992857	total: 492ms	remaining: 914ms
280:	learn: 0.9992857	total: 493ms	remaining: 911ms
281:	learn: 0.9992857	total: 495ms	remaining: 908ms
282:	learn: 0.9992857	total: 496ms	remaining: 906ms
283:	learn: 0.9992857	total: 497ms	remaining: 903ms
284:	learn: 0.9992857	total: 498ms	remaining: 900ms
285:	learn: 0.9992857	total: 500ms	remaining: 898ms
286:	learn: 0.9992857	total: 501ms	remaining: 895ms
287:	learn: 0.9992857	total: 502ms	remaining: 892ms
288:	learn: 0.9992857	total: 503ms	remaining: 890ms
289:	learn: 0.9992857	total: 505ms	remaining: 887ms
290:	learn: 0.9992857	total: 506ms	remaining: 885ms
291:	learn: 0.9992857	total: 507ms	remaining: 883ms
292:	learn: 0.9992857	total: 509ms	remaining: 880ms
293:	learn: 0.9992857	total: 510ms	remaining: 878ms
294:	learn: 0.9992857	total: 511ms	remaining: 875ms
295:	learn: 0.9992857	total: 513ms	remaining: 873ms
296:	learn: 0.9992857	total: 514ms	remaining: 871ms
297:	learn: 0.9992857	total: 515ms	remaining: 868ms
298:	learn: 0.9992857	total: 517ms	remaining: 866ms
299:	learn: 0.9992857	total: 518ms	remaining: 864ms
300:	learn: 0.9992857	total: 519ms	remaining: 861ms
301:	learn: 0.9992857	total: 521ms	remaining: 858ms
302:	learn: 0.9992857	total: 522ms	remaining: 856ms
303:	learn: 0.9992857	total: 523ms	remaining: 853ms
304:	learn: 0.9992857	total: 524ms	remaining: 851ms
305:	learn: 0.9992857	total: 526ms	remaining: 849ms
306:	learn: 0.9992857	total: 527ms	remaining: 846ms
307:	learn: 0.9992857	total: 528ms	remaining: 843ms
308:	learn: 0.9992857	total: 529ms	remaining: 841ms
309:	learn: 0.9992857	total: 530ms	remaining: 838ms
310:	learn: 0.9992857	total: 532ms	remaining: 836ms
311:	learn: 0.9992857	total: 533ms	remaining: 833ms
312:	learn: 0.9992857	total: 534ms	remaining: 831ms
313:	learn: 0.9992857	total: 535ms	remaining: 829ms
314:	learn: 0.9992857	total: 537ms	remaining: 826ms
315:	learn: 0.9992857	total: 538ms	remaining: 824ms
316:	learn: 0.9992857	total: 539ms	remaining: 822ms
317:	learn: 0.9992857	total: 541ms	remaining: 819ms
318:	learn: 0.9992857	total: 542ms	remaining: 817ms
319:	learn: 0.9992857	total: 543ms	remaining: 814ms
320:	learn: 0.9992857	total: 544ms	remaining: 812ms
321:	learn: 0.9992857	total: 545ms	remaining: 809ms
322:	learn: 0.9992857	total: 547ms	remaining: 807ms
323:	learn: 0.9992857	total: 548ms	remaining: 805ms
324:	learn: 0.9992857	total: 549ms	remaining: 802ms
325:	learn: 0.9992857	total: 550ms	remaining: 800ms
326:	learn: 0.9992857	total: 551ms	remaining: 798ms
327:	learn: 0.9992857	total: 553ms	remaining: 795ms
328:	learn: 0.9992857	total: 554ms	remaining: 793ms
329:	learn: 0.9992857	total: 555ms	remaining: 790ms
330:	learn: 0.9992857	total: 556ms	remaining: 788ms
331:	learn: 0.9992857	total: 557ms	remaining: 786ms
332:	learn: 0.9992857	total: 559ms	remaining: 783ms
333:	learn: 0.9992857	total: 560ms	remaining: 781ms
334:	learn: 0.9992857	total: 561ms	remaining: 779ms
335:	learn: 0.9992857	total: 562ms	remaining: 777ms
336:	learn: 0.9992857	total: 564ms	remaining: 774ms
337:	learn: 0.9992857	total: 565ms	remaining: 772ms
338:	learn: 0.9992857	total: 566ms	remaining: 770ms
339:	learn: 0.9992857	total: 567ms	remaining: 767ms
340:	learn: 0.9992857	total: 568ms	remaining: 765ms
341:	learn: 0.9992857	total: 570ms	remaining: 763ms
342:	learn: 0.9992857	total: 571ms	remaining: 761ms
343:	learn: 0.9992857	total: 572ms	remaining: 759ms
344:	learn: 0.9992857	total: 573ms	remaining: 756ms
345:	learn: 0.9992857	total: 575ms	remaining: 754ms
346:	learn: 0.9992857	total: 576ms	remaining: 752ms
347:	learn: 0.9992857	total: 577ms	remaining: 750ms
348:	learn: 0.9992857	total: 579ms	remaining: 748ms
349:	learn: 0.9992857	total: 580ms	remaining: 745ms
350:	learn: 0.9992857	total: 581ms	remaining: 743ms
351:	learn: 0.9992857	total: 582ms	remaining: 741ms
352:	learn: 0.9992857	total: 583ms	remaining: 739ms
353:	learn: 0.9992857	total: 585ms	remaining: 737ms
354:	learn: 0.9992857	total: 586ms	remaining: 734ms
355:	learn: 0.9992857	total: 587ms	remaining: 732ms
356:	learn: 0.9992857	total: 588ms	remaining: 730ms
357:	learn: 0.9992857	total: 590ms	remaining: 728ms
358:	learn: 0.9992857	total: 591ms	remaining: 726ms
359:	learn: 0.9992857	total: 592ms	remaining: 724ms
360:	learn: 0.9992857	total: 593ms	remaining: 721ms
361:	learn: 0.9992857	total: 594ms	remaining: 719ms
362:	learn: 0.9992857	total: 596ms	remaining: 717ms
363:	learn: 0.9992857	total: 597ms	remaining: 715ms
364:	learn: 0.9992857	total: 598ms	remaining: 713ms
365:	learn: 0.9992857	total: 599ms	remaining: 711ms
366:	learn: 0.9992857	total: 600ms	remaining: 708ms
367:	learn: 0.9992857	total: 602ms	remaining: 706ms
368:	learn: 0.9992857	total: 603ms	remaining: 704ms
369:	learn: 0.9992857	total: 604ms	remaining: 702ms
370:	learn: 0.9992857	total: 605ms	remaining: 700ms
371:	learn: 0.9992857	total: 606ms	remaining: 698ms
372:	learn: 0.9992857	total: 608ms	remaining: 695ms
373:	learn: 0.9992857	total: 609ms	remaining: 693ms
374:	learn: 0.9992857	total: 610ms	remaining: 691ms
375:	learn: 0.9992857	total: 611ms	remaining: 689ms
376:	learn: 0.9992857	total: 612ms	remaining: 687ms
377:	learn: 0.9992857	total: 614ms	remaining: 685ms
378:	learn: 0.9992857	total: 615ms	remaining: 683ms
379:	learn: 0.9992857	total: 616ms	remaining: 681ms
380:	learn: 0.9992857	total: 617ms	remaining: 679ms
381:	learn: 0.9992857	total: 619ms	remaining: 677ms
382:	learn: 0.9992857	total: 620ms	remaining: 675ms
383:	learn: 0.9992857	total: 621ms	remaining: 673ms
384:	learn: 0.9992857	total: 622ms	remaining: 671ms
385:	learn: 0.9992857	total: 623ms	remaining: 669ms
386:	learn: 0.9992857	total: 625ms	remaining: 667ms
387:	learn: 0.9992857	total: 626ms	remaining: 665ms
388:	learn: 0.9992857	total: 627ms	remaining: 662ms
389:	learn: 0.9992857	total: 628ms	remaining: 661ms
390:	learn: 0.9992857	total: 630ms	remaining: 659ms
391:	learn: 0.9992857	total: 631ms	remaining: 656ms
392:	learn: 0.9992857	total: 632ms	remaining: 655ms
393:	learn: 0.9992857	total: 633ms	remaining: 653ms
394:	learn: 0.9992857	total: 634ms	remaining: 651ms
395:	learn: 0.9992857	total: 636ms	remaining: 648ms
396:	learn: 0.9992857	total: 637ms	remaining: 647ms
397:	learn: 0.9992857	total: 638ms	remaining: 645ms
398:	learn: 0.9992857	total: 639ms	remaining: 642ms
399:	learn: 0.9992857	total: 641ms	remaining: 641ms
400:	learn: 0.9992857	total: 642ms	remaining: 638ms
401:	learn: 0.9992857	total: 643ms	remaining: 637ms
402:	learn: 0.9992857	total: 644ms	remaining: 634ms
403:	learn: 0.9992857	total: 645ms	remaining: 633ms
404:	learn: 0.9992857	total: 647ms	remaining: 631ms
405:	learn: 0.9992857	total: 648ms	remaining: 629ms
406:	learn: 0.9992857	total: 649ms	remaining: 627ms
407:	learn: 0.9992857	total: 650ms	remaining: 625ms
408:	learn: 0.9992857	total: 652ms	remaining: 623ms
409:	learn: 0.9992857	total: 653ms	remaining: 621ms
410:	learn: 0.9992857	total: 654ms	remaining: 619ms
411:	learn: 0.9992857	total: 655ms	remaining: 617ms
412:	learn: 0.9992857	total: 657ms	remaining: 615ms
413:	learn: 0.9992857	total: 658ms	remaining: 613ms
414:	learn: 0.9992857	total: 659ms	remaining: 611ms
415:	learn: 0.9992857	total: 660ms	remaining: 609ms
416:	learn: 0.9992857	total: 661ms	remaining: 607ms
417:	learn: 0.9992857	total: 663ms	remaining: 606ms
418:	learn: 0.9992857	total: 664ms	remaining: 604ms
419:	learn: 0.9992857	total: 665ms	remaining: 602ms
420:	learn: 0.9992857	total: 667ms	remaining: 600ms
421:	learn: 0.9992857	total: 668ms	remaining: 598ms
422:	learn: 0.9992857	total: 669ms	remaining: 597ms
423:	learn: 0.9992857	total: 671ms	remaining: 595ms
424:	learn: 0.9992857	total: 672ms	remaining: 593ms
425:	learn: 0.9992857	total: 673ms	remaining: 591ms
426:	learn: 0.9992857	total: 674ms	remaining: 589ms
427:	learn: 0.9992857	total: 676ms	remaining: 587ms
428:	learn: 0.9992857	total: 677ms	remaining: 586ms
429:	learn: 0.9992857	total: 679ms	remaining: 584ms
430:	learn: 0.9992857	total: 680ms	remaining: 582ms
431:	learn: 0.9992857	total: 681ms	remaining: 580ms
432:	learn: 0.9992857	total: 683ms	remaining: 579ms
433:	learn: 0.9992857	total: 684ms	remaining: 577ms
434:	learn: 0.9992857	total: 685ms	remaining: 575ms
435:	learn: 0.9992857	total: 686ms	remaining: 573ms
436:	learn: 0.9992857	total: 688ms	remaining: 571ms
437:	learn: 0.9992857	total: 689ms	remaining: 569ms
438:	learn: 0.9992857	total: 690ms	remaining: 567ms
439:	learn: 0.9992857	total: 691ms	remaining: 566ms
440:	learn: 0.9992857	total: 693ms	remaining: 564ms
441:	learn: 0.9992857	total: 694ms	remaining: 562ms
442:	learn: 0.9992857	total: 695ms	remaining: 560ms
443:	learn: 0.9992857	total: 696ms	remaining: 558ms
444:	learn: 0.9992857	total: 698ms	remaining: 556ms
445:	learn: 0.9992857	total: 699ms	remaining: 555ms
446:	learn: 0.9992857	total: 700ms	remaining: 553ms
447:	learn: 0.9992857	total: 701ms	remaining: 551ms
448:	learn: 0.9992857	total: 702ms	remaining: 549ms
449:	learn: 0.9992857	total: 704ms	remaining: 547ms
450:	learn: 0.9992857	total: 705ms	remaining: 545ms
451:	learn: 0.9992857	total: 706ms	remaining: 544ms
452:	learn: 0.9992857	total: 707ms	remaining: 542ms
453:	learn: 0.9992857	total: 708ms	remaining: 540ms
454:	learn: 0.9992857	total: 710ms	remaining: 538ms
455:	learn: 0.9992857	total: 711ms	remaining: 536ms
456:	learn: 0.9992857	total: 712ms	remaining: 534ms
457:	learn: 0.9992857	total: 713ms	remaining: 533ms
458:	learn: 0.9992857	total: 714ms	remaining: 531ms
459:	learn: 0.9992857	total: 716ms	remaining: 529ms
460:	learn: 0.9992857	total: 717ms	remaining: 527ms
461:	learn: 0.9992857	total: 718ms	remaining: 526ms
462:	learn: 0.9992857	total: 720ms	remaining: 524ms
463:	learn: 0.9992857	total: 721ms	remaining: 522ms
464:	learn: 0.9992857	total: 722ms	remaining: 520ms
465:	learn: 0.9992857	total: 723ms	remaining: 518ms
466:	learn: 0.9992857	total: 724ms	remaining: 516ms
467:	learn: 0.9992857	total: 726ms	remaining: 515ms
468:	learn: 0.9992857	total: 727ms	remaining: 513ms
469:	learn: 0.9992857	total: 728ms	remaining: 511ms
470:	learn: 0.9992857	total: 729ms	remaining: 510ms
471:	learn: 0.9992857	total: 731ms	remaining: 508ms
472:	learn: 0.9992857	total: 732ms	remaining: 506ms
473:	learn: 0.9992857	total: 733ms	remaining: 504ms
474:	learn: 0.9992857	total: 734ms	remaining: 502ms
475:	learn: 0.9992857	total: 736ms	remaining: 501ms
476:	learn: 0.9992857	total: 737ms	remaining: 499ms
477:	learn: 0.9992857	total: 738ms	remaining: 497ms
478:	learn: 0.9992857	total: 739ms	remaining: 495ms
479:	learn: 0.9992857	total: 740ms	remaining: 494ms
480:	learn: 0.9992857	total: 742ms	remaining: 492ms
481:	learn: 0.9992857	total: 743ms	remaining: 490ms
482:	learn: 0.9992857	total: 744ms	remaining: 488ms
483:	learn: 0.9992857	total: 745ms	remaining: 487ms
484:	learn: 0.9992857	total: 747ms	remaining: 485ms
485:	learn: 0.9992857	total: 748ms	remaining: 483ms
486:	learn: 0.9992857	total: 749ms	remaining: 481ms
487:	learn: 0.9992857	total: 750ms	remaining: 480ms
488:	learn: 0.9992857	total: 752ms	remaining: 478ms
489:	learn: 0.9992857	total: 753ms	remaining: 476ms
490:	learn: 0.9992857	total: 754ms	remaining: 475ms
491:	learn: 0.9992857	total: 755ms	remaining: 473ms
492:	learn: 0.9992857	total: 757ms	remaining: 471ms
493:	learn: 0.9992857	total: 758ms	remaining: 469ms
494:	learn: 0.9992857	total: 759ms	remaining: 468ms
495:	learn: 0.9992857	total: 760ms	remaining: 466ms
496:	learn: 0.9992857	total: 761ms	remaining: 464ms
497:	learn: 0.9992857	total: 763ms	remaining: 463ms
498:	learn: 0.9992857	total: 764ms	remaining: 461ms
499:	learn: 0.9992857	total: 765ms	remaining: 459ms
500:	learn: 0.9992857	total: 766ms	remaining: 457ms
501:	learn: 0.9992857	total: 768ms	remaining: 456ms
502:	learn: 0.9992857	total: 769ms	remaining: 454ms
503:	learn: 0.9992857	total: 770ms	remaining: 452ms
504:	learn: 0.9992857	total: 771ms	remaining: 451ms
505:	learn: 0.9992857	total: 773ms	remaining: 449ms
506:	learn: 0.9992857	total: 774ms	remaining: 447ms
507:	learn: 0.9992857	total: 775ms	remaining: 446ms
508:	learn: 0.9992857	total: 776ms	remaining: 444ms
509:	learn: 0.9992857	total: 777ms	remaining: 442ms
510:	learn: 0.9992857	total: 779ms	remaining: 440ms
511:	learn: 0.9992857	total: 780ms	remaining: 439ms
512:	learn: 0.9992857	total: 781ms	remaining: 437ms
513:	learn: 0.9992857	total: 782ms	remaining: 435ms
514:	learn: 0.9992857	total: 783ms	remaining: 434ms
515:	learn: 0.9992857	total: 785ms	remaining: 432ms
516:	learn: 0.9992857	total: 786ms	remaining: 430ms
517:	learn: 0.9992857	total: 787ms	remaining: 428ms
518:	learn: 0.9992857	total: 788ms	remaining: 427ms
519:	learn: 0.9992857	total: 789ms	remaining: 425ms
520:	learn: 0.9992857	total: 791ms	remaining: 423ms
521:	learn: 0.9992857	total: 792ms	remaining: 422ms
522:	learn: 0.9992857	total: 793ms	remaining: 420ms
523:	learn: 0.9992857	total: 794ms	remaining: 418ms
524:	learn: 0.9992857	total: 795ms	remaining: 417ms
525:	learn: 0.9992857	total: 797ms	remaining: 415ms
526:	learn: 0.9992857	total: 798ms	remaining: 413ms
527:	learn: 0.9992857	total: 799ms	remaining: 412ms
528:	learn: 0.9992857	total: 800ms	remaining: 410ms
529:	learn: 0.9992857	total: 802ms	remaining: 408ms
530:	learn: 0.9992857	total: 803ms	remaining: 407ms
531:	learn: 0.9992857	total: 804ms	remaining: 405ms
532:	learn: 0.9992857	total: 805ms	remaining: 403ms
533:	learn: 0.9992857	total: 806ms	remaining: 402ms
534:	learn: 0.9992857	total: 807ms	remaining: 400ms
535:	learn: 0.9992857	total: 809ms	remaining: 398ms
536:	learn: 0.9992857	total: 810ms	remaining: 397ms
537:	learn: 0.9992857	total: 811ms	remaining: 395ms
538:	learn: 0.9992857	total: 812ms	remaining: 393ms
539:	learn: 0.9992857	total: 814ms	remaining: 392ms
540:	learn: 0.9992857	total: 815ms	remaining: 390ms
541:	learn: 0.9992857	total: 816ms	remaining: 388ms
542:	learn: 0.9992857	total: 817ms	remaining: 387ms
543:	learn: 0.9992857	total: 818ms	remaining: 385ms
544:	learn: 0.9992857	total: 820ms	remaining: 383ms
545:	learn: 0.9992857	total: 821ms	remaining: 382ms
546:	learn: 0.9992857	total: 822ms	remaining: 380ms
547:	learn: 0.9992857	total: 823ms	remaining: 379ms
548:	learn: 0.9992857	total: 825ms	remaining: 377ms
549:	learn: 0.9992857	total: 826ms	remaining: 375ms
550:	learn: 0.9992857	total: 827ms	remaining: 374ms
551:	learn: 0.9992857	total: 828ms	remaining: 372ms
552:	learn: 0.9992857	total: 829ms	remaining: 370ms
553:	learn: 0.9992857	total: 831ms	remaining: 369ms
554:	learn: 0.9992857	total: 832ms	remaining: 367ms
555:	learn: 0.9992857	total: 833ms	remaining: 366ms
556:	learn: 0.9992857	total: 834ms	remaining: 364ms
557:	learn: 0.9992857	total: 835ms	remaining: 362ms
558:	learn: 0.9992857	total: 837ms	remaining: 361ms
559:	learn: 0.9992857	total: 838ms	remaining: 359ms
560:	learn: 0.9992857	total: 839ms	remaining: 358ms
561:	learn: 0.9992857	total: 841ms	remaining: 356ms
562:	learn: 0.9992857	total: 842ms	remaining: 354ms
563:	learn: 0.9992857	total: 843ms	remaining: 353ms
564:	learn: 0.9992857	total: 844ms	remaining: 351ms
565:	learn: 0.9992857	total: 846ms	remaining: 350ms
566:	learn: 0.9992857	total: 847ms	remaining: 348ms
567:	learn: 0.9992857	total: 848ms	remaining: 346ms
568:	learn: 0.9992857	total: 849ms	remaining: 345ms
569:	learn: 0.9992857	total: 850ms	remaining: 343ms
570:	learn: 0.9992857	total: 852ms	remaining: 342ms
571:	learn: 0.9992857	total: 853ms	remaining: 340ms
572:	learn: 0.9992857	total: 854ms	remaining: 338ms
573:	learn: 0.9992857	total: 855ms	remaining: 337ms
574:	learn: 0.9992857	total: 857ms	remaining: 335ms
575:	learn: 0.9992857	total: 858ms	remaining: 334ms
576:	learn: 0.9992857	total: 859ms	remaining: 332ms
577:	learn: 0.9992857	total: 861ms	remaining: 331ms
578:	learn: 0.9992857	total: 862ms	remaining: 329ms
579:	learn: 0.9992857	total: 863ms	remaining: 327ms
580:	learn: 0.9992857	total: 864ms	remaining: 326ms
581:	learn: 0.9992857	total: 866ms	remaining: 324ms
582:	learn: 0.9992857	total: 867ms	remaining: 323ms
583:	learn: 0.9992857	total: 868ms	remaining: 321ms
584:	learn: 0.9992857	total: 870ms	remaining: 320ms
585:	learn: 0.9992857	total: 871ms	remaining: 318ms
586:	learn: 0.9992857	total: 872ms	remaining: 316ms
587:	learn: 0.9992857	total: 873ms	remaining: 315ms
588:	learn: 0.9992857	total: 875ms	remaining: 313ms
589:	learn: 0.9992857	total: 876ms	remaining: 312ms
590:	learn: 0.9992857	total: 877ms	remaining: 310ms
591:	learn: 0.9992857	total: 878ms	remaining: 309ms
592:	learn: 0.9992857	total: 880ms	remaining: 307ms
593:	learn: 0.9992857	total: 881ms	remaining: 305ms
594:	learn: 0.9992857	total: 882ms	remaining: 304ms
595:	learn: 0.9992857	total: 883ms	remaining: 302ms
596:	learn: 0.9992857	total: 885ms	remaining: 301ms
597:	learn: 0.9992857	total: 886ms	remaining: 299ms
598:	learn: 0.9992857	total: 887ms	remaining: 298ms
599:	learn: 0.9992857	total: 888ms	remaining: 296ms
600:	learn: 0.9992857	total: 889ms	remaining: 294ms
601:	learn: 0.9992857	total: 891ms	remaining: 293ms
602:	learn: 0.9992857	total: 892ms	remaining: 291ms
603:	learn: 0.9992857	total: 893ms	remaining: 290ms
604:	learn: 0.9992857	total: 894ms	remaining: 288ms
605:	learn: 0.9992857	total: 896ms	remaining: 287ms
606:	learn: 0.9992857	total: 897ms	remaining: 285ms
607:	learn: 0.9992857	total: 898ms	remaining: 284ms
608:	learn: 0.9992857	total: 899ms	remaining: 282ms
609:	learn: 0.9992857	total: 900ms	remaining: 280ms
610:	learn: 0.9992857	total: 902ms	remaining: 279ms
611:	learn: 0.9992857	total: 903ms	remaining: 277ms
612:	learn: 0.9992857	total: 904ms	remaining: 276ms
613:	learn: 0.9992857	total: 905ms	remaining: 274ms
614:	learn: 0.9992857	total: 907ms	remaining: 273ms
615:	learn: 0.9992857	total: 908ms	remaining: 271ms
616:	learn: 0.9992857	total: 909ms	remaining: 270ms
617:	learn: 0.9992857	total: 910ms	remaining: 268ms
618:	learn: 0.9992857	total: 911ms	remaining: 267ms
619:	learn: 0.9992857	total: 913ms	remaining: 265ms
620:	learn: 0.9992857	total: 914ms	remaining: 263ms
621:	learn: 0.9992857	total: 915ms	remaining: 262ms
622:	learn: 0.9992857	total: 916ms	remaining: 260ms
623:	learn: 0.9992857	total: 918ms	remaining: 259ms
624:	learn: 0.9992857	total: 919ms	remaining: 257ms
625:	learn: 0.9992857	total: 920ms	remaining: 256ms
626:	learn: 0.9992857	total: 921ms	remaining: 254ms
627:	learn: 0.9992857	total: 922ms	remaining: 253ms
628:	learn: 0.9992857	total: 924ms	remaining: 251ms
629:	learn: 0.9992857	total: 925ms	remaining: 250ms
630:	learn: 0.9992857	total: 926ms	remaining: 248ms
631:	learn: 0.9992857	total: 928ms	remaining: 247ms
632:	learn: 0.9992857	total: 929ms	remaining: 245ms
633:	learn: 0.9992857	total: 930ms	remaining: 244ms
634:	learn: 0.9992857	total: 932ms	remaining: 242ms
635:	learn: 0.9992857	total: 933ms	remaining: 241ms
636:	learn: 0.9992857	total: 934ms	remaining: 239ms
637:	learn: 0.9992857	total: 936ms	remaining: 238ms
638:	learn: 0.9992857	total: 937ms	remaining: 236ms
639:	learn: 0.9992857	total: 938ms	remaining: 235ms
640:	learn: 0.9992857	total: 939ms	remaining: 233ms
641:	learn: 0.9992857	total: 941ms	remaining: 232ms
642:	learn: 0.9992857	total: 942ms	remaining: 230ms
643:	learn: 0.9992857	total: 944ms	remaining: 229ms
644:	learn: 0.9992857	total: 945ms	remaining: 227ms
645:	learn: 0.9992857	total: 946ms	remaining: 226ms
646:	learn: 0.9992857	total: 948ms	remaining: 224ms
647:	learn: 0.9992857	total: 949ms	remaining: 223ms
648:	learn: 0.9992857	total: 950ms	remaining: 221ms
649:	learn: 0.9992857	total: 952ms	remaining: 220ms
650:	learn: 0.9992857	total: 953ms	remaining: 218ms
651:	learn: 0.9992857	total: 954ms	remaining: 217ms
652:	learn: 0.9992857	total: 955ms	remaining: 215ms
653:	learn: 0.9992857	total: 957ms	remaining: 214ms
654:	learn: 0.9992857	total: 958ms	remaining: 212ms
655:	learn: 0.9992857	total: 959ms	remaining: 211ms
656:	learn: 0.9992857	total: 960ms	remaining: 209ms
657:	learn: 0.9992857	total: 962ms	remaining: 208ms
658:	learn: 0.9992857	total: 963ms	remaining: 206ms
659:	learn: 0.9992857	total: 964ms	remaining: 205ms
660:	learn: 0.9992857	total: 965ms	remaining: 203ms
661:	learn: 0.9992857	total: 966ms	remaining: 201ms
662:	learn: 0.9992857	total: 968ms	remaining: 200ms
663:	learn: 0.9992857	total: 969ms	remaining: 198ms
664:	learn: 0.9992857	total: 970ms	remaining: 197ms
665:	learn: 0.9992857	total: 971ms	remaining: 195ms
666:	learn: 0.9992857	total: 973ms	remaining: 194ms
667:	learn: 0.9992857	total: 974ms	remaining: 192ms
668:	learn: 0.9992857	total: 975ms	remaining: 191ms
669:	learn: 0.9992857	total: 976ms	remaining: 189ms
670:	learn: 0.9992857	total: 978ms	remaining: 188ms
671:	learn: 0.9992857	total: 979ms	remaining: 186ms
672:	learn: 0.9992857	total: 980ms	remaining: 185ms
673:	learn: 0.9992857	total: 981ms	remaining: 183ms
674:	learn: 0.9992857	total: 983ms	remaining: 182ms
675:	learn: 0.9992857	total: 984ms	remaining: 180ms
676:	learn: 0.9992857	total: 985ms	remaining: 179ms
677:	learn: 0.9992857	total: 986ms	remaining: 177ms
678:	learn: 0.9992857	total: 987ms	remaining: 176ms
679:	learn: 0.9992857	total: 989ms	remaining: 174ms
680:	learn: 0.9992857	total: 990ms	remaining: 173ms
681:	learn: 0.9992857	total: 991ms	remaining: 171ms
682:	learn: 0.9992857	total: 992ms	remaining: 170ms
683:	learn: 0.9992857	total: 994ms	remaining: 169ms
684:	learn: 0.9992857	total: 995ms	remaining: 167ms
685:	learn: 0.9992857	total: 996ms	remaining: 166ms
686:	learn: 0.9992857	total: 997ms	remaining: 164ms
687:	learn: 0.9992857	total: 998ms	remaining: 163ms
688:	learn: 0.9992857	total: 1000ms	remaining: 161ms
689:	learn: 0.9992857	total: 1s	remaining: 160ms
690:	learn: 0.9992857	total: 1s	remaining: 158ms
691:	learn: 0.9992857	total: 1s	remaining: 157ms
692:	learn: 0.9992857	total: 1s	remaining: 155ms
693:	learn: 0.9992857	total: 1s	remaining: 154ms
694:	learn: 0.9992857	total: 1.01s	remaining: 152ms
695:	learn: 0.9992857	total: 1.01s	remaining: 151ms
696:	learn: 0.9992857	total: 1.01s	remaining: 149ms
697:	learn: 0.9992857	total: 1.01s	remaining: 148ms
698:	learn: 0.9992857	total: 1.01s	remaining: 146ms
699:	learn: 0.9992857	total: 1.01s	remaining: 145ms
700:	learn: 0.9992857	total: 1.01s	remaining: 143ms
701:	learn: 0.9992857	total: 1.02s	remaining: 142ms
702:	learn: 0.9992857	total: 1.02s	remaining: 140ms
703:	learn: 0.9992857	total: 1.02s	remaining: 139ms
704:	learn: 0.9992857	total: 1.02s	remaining: 137ms
705:	learn: 0.9992857	total: 1.02s	remaining: 136ms
706:	learn: 0.9992857	total: 1.02s	remaining: 135ms
707:	learn: 0.9992857	total: 1.02s	remaining: 133ms
708:	learn: 0.9992857	total: 1.03s	remaining: 132ms
709:	learn: 0.9992857	total: 1.03s	remaining: 130ms
710:	learn: 0.9992857	total: 1.03s	remaining: 129ms
711:	learn: 0.9992857	total: 1.03s	remaining: 127ms
712:	learn: 0.9992857	total: 1.03s	remaining: 126ms
713:	learn: 0.9992857	total: 1.03s	remaining: 124ms
714:	learn: 0.9992857	total: 1.03s	remaining: 123ms
715:	learn: 0.9992857	total: 1.03s	remaining: 122ms
716:	learn: 0.9992857	total: 1.04s	remaining: 120ms
717:	learn: 0.9992857	total: 1.04s	remaining: 119ms
718:	learn: 0.9992857	total: 1.04s	remaining: 117ms
719:	learn: 0.9992857	total: 1.04s	remaining: 116ms
720:	learn: 0.9992857	total: 1.04s	remaining: 114ms
721:	learn: 0.9992857	total: 1.04s	remaining: 113ms
722:	learn: 0.9992857	total: 1.04s	remaining: 111ms
723:	learn: 0.9992857	total: 1.05s	remaining: 110ms
724:	learn: 0.9992857	total: 1.05s	remaining: 108ms
725:	learn: 0.9992857	total: 1.05s	remaining: 107ms
726:	learn: 0.9992857	total: 1.05s	remaining: 105ms
727:	learn: 0.9992857	total: 1.05s	remaining: 104ms
728:	learn: 0.9992857	total: 1.05s	remaining: 103ms
729:	learn: 0.9992857	total: 1.05s	remaining: 101ms
730:	learn: 0.9992857	total: 1.05s	remaining: 99.6ms
731:	learn: 0.9992857	total: 1.06s	remaining: 98.2ms
732:	learn: 0.9992857	total: 1.06s	remaining: 96.7ms
733:	learn: 0.9992857	total: 1.06s	remaining: 95.2ms
734:	learn: 0.9992857	total: 1.06s	remaining: 93.8ms
735:	learn: 0.9992857	total: 1.06s	remaining: 92.3ms
736:	learn: 0.9992857	total: 1.06s	remaining: 90.8ms
737:	learn: 0.9992857	total: 1.06s	remaining: 89.4ms
738:	learn: 0.9992857	total: 1.06s	remaining: 87.9ms
739:	learn: 0.9992857	total: 1.07s	remaining: 86.5ms
740:	learn: 0.9992857	total: 1.07s	remaining: 85ms
741:	learn: 0.9992857	total: 1.07s	remaining: 83.6ms
742:	learn: 0.9992857	total: 1.07s	remaining: 82.1ms
743:	learn: 0.9992857	total: 1.07s	remaining: 80.7ms
744:	learn: 0.9992857	total: 1.07s	remaining: 79.2ms
745:	learn: 0.9992857	total: 1.07s	remaining: 77.7ms
746:	learn: 0.9992857	total: 1.07s	remaining: 76.3ms
747:	learn: 0.9992857	total: 1.08s	remaining: 74.8ms
748:	learn: 0.9992857	total: 1.08s	remaining: 73.4ms
749:	learn: 0.9992857	total: 1.08s	remaining: 71.9ms
750:	learn: 0.9992857	total: 1.08s	remaining: 70.5ms
751:	learn: 0.9992857	total: 1.08s	remaining: 69ms
752:	learn: 0.9992857	total: 1.08s	remaining: 67.6ms
753:	learn: 0.9992857	total: 1.08s	remaining: 66.1ms
754:	learn: 0.9992857	total: 1.08s	remaining: 64.7ms
755:	learn: 0.9992857	total: 1.09s	remaining: 63.3ms
756:	learn: 0.9992857	total: 1.09s	remaining: 61.8ms
757:	learn: 0.9992857	total: 1.09s	remaining: 60.4ms
758:	learn: 0.9992857	total: 1.09s	remaining: 58.9ms
759:	learn: 0.9992857	total: 1.09s	remaining: 57.5ms
760:	learn: 0.9992857	total: 1.09s	remaining: 56ms
761:	learn: 0.9992857	total: 1.09s	remaining: 54.6ms
762:	learn: 0.9992857	total: 1.09s	remaining: 53.1ms
763:	learn: 0.9992857	total: 1.1s	remaining: 51.7ms
764:	learn: 0.9992857	total: 1.1s	remaining: 50.3ms
765:	learn: 0.9992857	total: 1.1s	remaining: 48.8ms
766:	learn: 0.9992857	total: 1.1s	remaining: 47.4ms
767:	learn: 0.9992857	total: 1.1s	remaining: 45.9ms
768:	learn: 0.9992857	total: 1.1s	remaining: 44.5ms
769:	learn: 0.9992857	total: 1.1s	remaining: 43ms
770:	learn: 0.9992857	total: 1.11s	remaining: 41.6ms
771:	learn: 0.9992857	total: 1.11s	remaining: 40.2ms
772:	learn: 0.9992857	total: 1.11s	remaining: 38.7ms
773:	learn: 0.9992857	total: 1.11s	remaining: 37.3ms
774:	learn: 0.9992857	total: 1.11s	remaining: 35.8ms
775:	learn: 0.9992857	total: 1.11s	remaining: 34.4ms
776:	learn: 0.9992857	total: 1.11s	remaining: 33ms
777:	learn: 0.9992857	total: 1.11s	remaining: 31.5ms
778:	learn: 0.9992857	total: 1.12s	remaining: 30.1ms
779:	learn: 0.9992857	total: 1.12s	remaining: 28.7ms
780:	learn: 0.9992857	total: 1.12s	remaining: 27.2ms
781:	learn: 0.9992857	total: 1.12s	remaining: 25.8ms
782:	learn: 0.9992857	total: 1.12s	remaining: 24.4ms
783:	learn: 0.9992857	total: 1.12s	remaining: 22.9ms
784:	learn: 0.9992857	total: 1.12s	remaining: 21.5ms
785:	learn: 0.9992857	total: 1.13s	remaining: 20.1ms
786:	learn: 0.9992857	total: 1.13s	remaining: 18.6ms
787:	learn: 0.9992857	total: 1.13s	remaining: 17.2ms
788:	learn: 0.9992857	total: 1.13s	remaining: 15.8ms
789:	learn: 0.9992857	total: 1.13s	remaining: 14.3ms
790:	learn: 0.9992857	total: 1.13s	remaining: 12.9ms
791:	learn: 0.9992857	total: 1.13s	remaining: 11.5ms
792:	learn: 0.9992857	total: 1.13s	remaining: 10ms
793:	learn: 0.9992857	total: 1.14s	remaining: 8.59ms
794:	learn: 0.9992857	total: 1.14s	remaining: 7.16ms
795:	learn: 0.9992857	total: 1.14s	remaining: 5.72ms
796:	learn: 0.9992857	total: 1.14s	remaining: 4.29ms
797:	learn: 0.9992857	total: 1.14s	remaining: 2.86ms
798:	learn: 0.9992857	total: 1.14s	remaining: 1.43ms
799:	learn: 0.9992857	total: 1.14s	remaining: 0us

5.评价模型

model_eval(clf, train, test)

train_roc_auc_score: 0.999998962825595
test_roc_auc_score: 0.7423918968023258
train_accuracy_score: 0.9992857142857143
test_accuracy_score: 0.7083333333333334
train_precision_score: 1.0
test__precision_score: 0.7393767705382436
train_recall_score: 0.9987325728770595
test_recall_score: 0.7587209302325582
train_f1_score: 0.9993658845909955
test_f1_score: 0.7489239598278336

6.利用模型预测

预测0,1值

注意：

这里面输入test.values与test 都是可以的，即这里支持 numpy.ndarray和pandas的DataFrame

clf.predict( test.values )
clf.predict( test)

array([0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1,
       0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0,
       1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1,
       1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1,
       0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0,
       1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1,
       1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,
       1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1,
       1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0,
       1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0,
       1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0,
       1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
       1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0,
       1, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1,
       1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1,
       0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,
       0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0,
       0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1,
       1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1,
       1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0,
       1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1,
       1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1,
       1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1,
       1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1,
       0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1,
       1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0,
       1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1,
       1, 1, 1, 1, 0, 0], dtype=int64)

预测概率值

clf.predict_proba(test.values )

array([[0.78013413, 0.21986587],
       [0.1147824 , 0.8852176 ],
       [0.41869628, 0.58130372],
       ...,
       [0.00353316, 0.99646684],
       [0.5682076 , 0.4317924 ],
       [0.98157093, 0.01842907]])

7.保存模型并调用

joblib.dump(clf , r'D:\Ensemble_Learning\catboost_info\catboostsingle.model')
load_model=joblib.load(r'D:\Ensemble_Learning\catboost_info\catboostsingle.model')
load_model.predict( test )
load_model.predict_proba(test )

array([[0.78013413, 0.21986587],
       [0.1147824 , 0.8852176 ],
       [0.41869628, 0.58130372],
       ...,
       [0.00353316, 0.99646684],
       [0.5682076 , 0.4317924 ],
       [0.98157093, 0.01842907]])

三. 网格搜索最优catboost模型

from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import KFold

1.step-01 配置参数列表

grid_parameters = { #'depth': [6,7],
                    #'l2_leaf_reg': [9,10,11],
                    #'penalties_coefficient':[15,20],
                    #'ctr_leaf_count_limit':[200,300],
                   #'one_hot_max_size':[8,10]
                   #'learning_rate': [0.15,0.14],
                   #'best_model_min_trees':[10,8],
                   'min_data_in_leaf':[100], #60
                   'border_count':[12], #24
                   #'max_ctr_complexity':[2]
                   # 'subsample':[0.5,0.53],
                  # 'colsample_bylevel':[0.55,0.58]
        }

step-02 选择待优化模型

model_ini = cb.CatBoostClassifier(cat_features=cat_features_index,
                                eval_metric='Accuracy',
                                random_seed=11,
                                random_strength=1,                                
                                depth=7,
                                l2_leaf_reg=10,
                                penalties_coefficient=15,
                                one_hot_max_size=12,
                                iterations=900,
                                learning_rate=0.15,
                                #min_data_in_leaf=30,
                                od_pval=10e-3,
                                early_stopping_rounds=20,
                                #border_count= 64,
                                max_ctr_complexity=2,
                                #grow_policy='Depthwise',
                              subsample=0.53,
                              colsample_bylevel=0.58,
                              )

step-03 进行网格搜索拟合模型

clf = GridSearchCV(model_ini, grid_parameters, cv=2, scoring='accuracy',verbose=1,n_jobs=-1)
clf.fit(train, y_train)

Fitting 2 folds for each of 1 candidates, totalling 2 fits
0:	learn: 0.5957143	total: 10.4ms	remaining: 9.31s
1:	learn: 0.6957143	total: 20.1ms	remaining: 9.01s
2:	learn: 0.7235714	total: 27.1ms	remaining: 8.09s
3:	learn: 0.7350000	total: 35.4ms	remaining: 7.93s
4:	learn: 0.7457143	total: 45.5ms	remaining: 8.14s
5:	learn: 0.7521429	total: 52.1ms	remaining: 7.77s
6:	learn: 0.7600000	total: 55.9ms	remaining: 7.13s
7:	learn: 0.7685714	total: 63.3ms	remaining: 7.05s
8:	learn: 0.7842857	total: 69.8ms	remaining: 6.91s
9:	learn: 0.7800000	total: 75.3ms	remaining: 6.7s
10:	learn: 0.7864286	total: 82.3ms	remaining: 6.65s
11:	learn: 0.7857143	total: 89.6ms	remaining: 6.63s
12:	learn: 0.7957143	total: 96.3ms	remaining: 6.57s
13:	learn: 0.7992857	total: 104ms	remaining: 6.57s
14:	learn: 0.8121429	total: 111ms	remaining: 6.54s
15:	learn: 0.8178571	total: 117ms	remaining: 6.44s
16:	learn: 0.8207143	total: 121ms	remaining: 6.31s
17:	learn: 0.8185714	total: 126ms	remaining: 6.19s
18:	learn: 0.8207143	total: 132ms	remaining: 6.14s
19:	learn: 0.8278571	total: 137ms	remaining: 6.03s
20:	learn: 0.8257143	total: 144ms	remaining: 6.03s
21:	learn: 0.8271429	total: 150ms	remaining: 5.97s
22:	learn: 0.8350000	total: 155ms	remaining: 5.9s
23:	learn: 0.8414286	total: 163ms	remaining: 5.94s
24:	learn: 0.8442857	total: 170ms	remaining: 5.96s
25:	learn: 0.8435714	total: 174ms	remaining: 5.85s
26:	learn: 0.8421429	total: 182ms	remaining: 5.89s
27:	learn: 0.8521429	total: 190ms	remaining: 5.91s
28:	learn: 0.8550000	total: 198ms	remaining: 5.95s
29:	learn: 0.8557143	total: 210ms	remaining: 6.08s
30:	learn: 0.8578571	total: 222ms	remaining: 6.21s
31:	learn: 0.8557143	total: 228ms	remaining: 6.19s
32:	learn: 0.8571429	total: 236ms	remaining: 6.21s
33:	learn: 0.8635714	total: 244ms	remaining: 6.22s
34:	learn: 0.8628571	total: 252ms	remaining: 6.22s
35:	learn: 0.8635714	total: 257ms	remaining: 6.17s
36:	learn: 0.8685714	total: 263ms	remaining: 6.13s
37:	learn: 0.8700000	total: 271ms	remaining: 6.14s
38:	learn: 0.8728571	total: 277ms	remaining: 6.13s
39:	learn: 0.8721429	total: 286ms	remaining: 6.14s
40:	learn: 0.8778571	total: 289ms	remaining: 6.06s
41:	learn: 0.8807143	total: 297ms	remaining: 6.06s
42:	learn: 0.8785714	total: 302ms	remaining: 6.03s
43:	learn: 0.8800000	total: 308ms	remaining: 5.99s
44:	learn: 0.8807143	total: 313ms	remaining: 5.96s
45:	learn: 0.8807143	total: 321ms	remaining: 5.96s
46:	learn: 0.8814286	total: 329ms	remaining: 5.97s
47:	learn: 0.8828571	total: 336ms	remaining: 5.96s
48:	learn: 0.8864286	total: 343ms	remaining: 5.96s
49:	learn: 0.8892857	total: 349ms	remaining: 5.94s
50:	learn: 0.8885714	total: 355ms	remaining: 5.91s
51:	learn: 0.8942857	total: 362ms	remaining: 5.89s
52:	learn: 0.8957143	total: 370ms	remaining: 5.92s
53:	learn: 0.9014286	total: 375ms	remaining: 5.88s
54:	learn: 0.9021429	total: 383ms	remaining: 5.89s
55:	learn: 0.9035714	total: 389ms	remaining: 5.86s
56:	learn: 0.9064286	total: 403ms	remaining: 5.96s
57:	learn: 0.9064286	total: 414ms	remaining: 6.01s
58:	learn: 0.9114286	total: 421ms	remaining: 6s
59:	learn: 0.9114286	total: 428ms	remaining: 5.99s
60:	learn: 0.9135714	total: 436ms	remaining: 6s
61:	learn: 0.9150000	total: 442ms	remaining: 5.97s
62:	learn: 0.9178571	total: 448ms	remaining: 5.95s
63:	learn: 0.9178571	total: 455ms	remaining: 5.94s
64:	learn: 0.9228571	total: 461ms	remaining: 5.93s
65:	learn: 0.9250000	total: 469ms	remaining: 5.92s
66:	learn: 0.9228571	total: 476ms	remaining: 5.92s
67:	learn: 0.9250000	total: 480ms	remaining: 5.88s
68:	learn: 0.9271429	total: 487ms	remaining: 5.86s
69:	learn: 0.9278571	total: 494ms	remaining: 5.86s
70:	learn: 0.9257143	total: 500ms	remaining: 5.84s
71:	learn: 0.9278571	total: 507ms	remaining: 5.83s
72:	learn: 0.9292857	total: 513ms	remaining: 5.81s
73:	learn: 0.9314286	total: 521ms	remaining: 5.81s
74:	learn: 0.9300000	total: 527ms	remaining: 5.8s
75:	learn: 0.9307143	total: 532ms	remaining: 5.77s
76:	learn: 0.9342857	total: 541ms	remaining: 5.78s
77:	learn: 0.9342857	total: 548ms	remaining: 5.78s
78:	learn: 0.9328571	total: 555ms	remaining: 5.77s
79:	learn: 0.9350000	total: 562ms	remaining: 5.76s
80:	learn: 0.9371429	total: 568ms	remaining: 5.74s
81:	learn: 0.9385714	total: 575ms	remaining: 5.74s
82:	learn: 0.9407143	total: 582ms	remaining: 5.73s
83:	learn: 0.9421429	total: 591ms	remaining: 5.75s
84:	learn: 0.9421429	total: 602ms	remaining: 5.77s
85:	learn: 0.9421429	total: 609ms	remaining: 5.76s
86:	learn: 0.9450000	total: 613ms	remaining: 5.73s
87:	learn: 0.9442857	total: 618ms	remaining: 5.7s
88:	learn: 0.9457143	total: 623ms	remaining: 5.68s
89:	learn: 0.9464286	total: 630ms	remaining: 5.67s
90:	learn: 0.9457143	total: 636ms	remaining: 5.66s
91:	learn: 0.9435714	total: 643ms	remaining: 5.64s
92:	learn: 0.9485714	total: 649ms	remaining: 5.63s
93:	learn: 0.9492857	total: 654ms	remaining: 5.61s
94:	learn: 0.9542857	total: 660ms	remaining: 5.59s
95:	learn: 0.9564286	total: 667ms	remaining: 5.59s
96:	learn: 0.9550000	total: 673ms	remaining: 5.57s
97:	learn: 0.9550000	total: 683ms	remaining: 5.58s
98:	learn: 0.9550000	total: 687ms	remaining: 5.56s
99:	learn: 0.9571429	total: 691ms	remaining: 5.53s
100:	learn: 0.9578571	total: 700ms	remaining: 5.54s
101:	learn: 0.9621429	total: 706ms	remaining: 5.52s
102:	learn: 0.9607143	total: 712ms	remaining: 5.5s
103:	learn: 0.9614286	total: 716ms	remaining: 5.48s
104:	learn: 0.9635714	total: 724ms	remaining: 5.48s
105:	learn: 0.9635714	total: 730ms	remaining: 5.47s
106:	learn: 0.9635714	total: 738ms	remaining: 5.47s
107:	learn: 0.9621429	total: 742ms	remaining: 5.44s
108:	learn: 0.9650000	total: 748ms	remaining: 5.43s
109:	learn: 0.9664286	total: 752ms	remaining: 5.4s
110:	learn: 0.9642857	total: 760ms	remaining: 5.4s
111:	learn: 0.9657143	total: 767ms	remaining: 5.4s
112:	learn: 0.9664286	total: 776ms	remaining: 5.4s
113:	learn: 0.9671429	total: 786ms	remaining: 5.42s
114:	learn: 0.9671429	total: 800ms	remaining: 5.46s
115:	learn: 0.9707143	total: 808ms	remaining: 5.46s
116:	learn: 0.9728571	total: 814ms	remaining: 5.45s
117:	learn: 0.9728571	total: 821ms	remaining: 5.44s
118:	learn: 0.9742857	total: 829ms	remaining: 5.44s
119:	learn: 0.9750000	total: 836ms	remaining: 5.43s
120:	learn: 0.9771429	total: 844ms	remaining: 5.43s
121:	learn: 0.9771429	total: 850ms	remaining: 5.42s
122:	learn: 0.9764286	total: 857ms	remaining: 5.42s
123:	learn: 0.9771429	total: 864ms	remaining: 5.4s
124:	learn: 0.9778571	total: 870ms	remaining: 5.39s
125:	learn: 0.9800000	total: 877ms	remaining: 5.38s
126:	learn: 0.9800000	total: 883ms	remaining: 5.37s
127:	learn: 0.9807143	total: 889ms	remaining: 5.36s
128:	learn: 0.9792857	total: 893ms	remaining: 5.34s
129:	learn: 0.9785714	total: 901ms	remaining: 5.34s
130:	learn: 0.9792857	total: 908ms	remaining: 5.33s
131:	learn: 0.9807143	total: 913ms	remaining: 5.31s
132:	learn: 0.9807143	total: 921ms	remaining: 5.31s
133:	learn: 0.9800000	total: 930ms	remaining: 5.32s
134:	learn: 0.9800000	total: 937ms	remaining: 5.31s
135:	learn: 0.9807143	total: 941ms	remaining: 5.29s
136:	learn: 0.9807143	total: 948ms	remaining: 5.28s
137:	learn: 0.9792857	total: 955ms	remaining: 5.27s
138:	learn: 0.9807143	total: 962ms	remaining: 5.27s
139:	learn: 0.9821429	total: 970ms	remaining: 5.26s
140:	learn: 0.9835714	total: 984ms	remaining: 5.3s
141:	learn: 0.9842857	total: 993ms	remaining: 5.3s
142:	learn: 0.9842857	total: 1s	remaining: 5.29s
143:	learn: 0.9850000	total: 1s	remaining: 5.28s
144:	learn: 0.9850000	total: 1.01s	remaining: 5.28s
145:	learn: 0.9850000	total: 1.02s	remaining: 5.28s
146:	learn: 0.9850000	total: 1.03s	remaining: 5.26s
147:	learn: 0.9857143	total: 1.03s	remaining: 5.25s
148:	learn: 0.9864286	total: 1.04s	remaining: 5.25s
149:	learn: 0.9857143	total: 1.05s	remaining: 5.24s
150:	learn: 0.9850000	total: 1.05s	remaining: 5.22s
151:	learn: 0.9857143	total: 1.06s	remaining: 5.22s
152:	learn: 0.9871429	total: 1.06s	remaining: 5.2s
153:	learn: 0.9871429	total: 1.07s	remaining: 5.19s
154:	learn: 0.9878571	total: 1.08s	remaining: 5.17s
155:	learn: 0.9892857	total: 1.08s	remaining: 5.17s
156:	learn: 0.9892857	total: 1.09s	remaining: 5.17s
157:	learn: 0.9900000	total: 1.1s	remaining: 5.16s
158:	learn: 0.9885714	total: 1.11s	remaining: 5.16s
159:	learn: 0.9885714	total: 1.11s	remaining: 5.15s
160:	learn: 0.9892857	total: 1.12s	remaining: 5.14s
161:	learn: 0.9885714	total: 1.13s	remaining: 5.13s
162:	learn: 0.9900000	total: 1.13s	remaining: 5.11s
163:	learn: 0.9885714	total: 1.14s	remaining: 5.11s
164:	learn: 0.9907143	total: 1.14s	remaining: 5.09s
165:	learn: 0.9907143	total: 1.15s	remaining: 5.09s
166:	learn: 0.9907143	total: 1.16s	remaining: 5.09s
167:	learn: 0.9907143	total: 1.16s	remaining: 5.08s
168:	learn: 0.9907143	total: 1.17s	remaining: 5.07s
169:	learn: 0.9907143	total: 1.18s	remaining: 5.07s
170:	learn: 0.9907143	total: 1.19s	remaining: 5.08s
171:	learn: 0.9907143	total: 1.2s	remaining: 5.06s
172:	learn: 0.9907143	total: 1.2s	remaining: 5.06s
173:	learn: 0.9907143	total: 1.21s	remaining: 5.06s
174:	learn: 0.9907143	total: 1.22s	remaining: 5.06s
175:	learn: 0.9907143	total: 1.23s	remaining: 5.06s
176:	learn: 0.9914286	total: 1.24s	remaining: 5.05s
177:	learn: 0.9921429	total: 1.24s	remaining: 5.04s
178:	learn: 0.9921429	total: 1.25s	remaining: 5.03s
179:	learn: 0.9921429	total: 1.25s	remaining: 5.02s
180:	learn: 0.9921429	total: 1.26s	remaining: 5.02s
181:	learn: 0.9928571	total: 1.27s	remaining: 5.01s
182:	learn: 0.9928571	total: 1.28s	remaining: 5.01s
183:	learn: 0.9928571	total: 1.28s	remaining: 5s
184:	learn: 0.9935714	total: 1.29s	remaining: 4.99s
185:	learn: 0.9935714	total: 1.29s	remaining: 4.97s
186:	learn: 0.9935714	total: 1.3s	remaining: 4.97s
187:	learn: 0.9935714	total: 1.31s	remaining: 4.96s
188:	learn: 0.9942857	total: 1.32s	remaining: 4.95s
189:	learn: 0.9950000	total: 1.32s	remaining: 4.95s
190:	learn: 0.9942857	total: 1.33s	remaining: 4.95s
191:	learn: 0.9950000	total: 1.34s	remaining: 4.95s
192:	learn: 0.9950000	total: 1.35s	remaining: 4.94s
193:	learn: 0.9950000	total: 1.36s	remaining: 4.94s
194:	learn: 0.9971429	total: 1.36s	remaining: 4.93s
195:	learn: 0.9971429	total: 1.38s	remaining: 4.94s
196:	learn: 0.9971429	total: 1.39s	remaining: 4.94s
197:	learn: 0.9971429	total: 1.39s	remaining: 4.94s
198:	learn: 0.9964286	total: 1.4s	remaining: 4.93s
199:	learn: 0.9964286	total: 1.41s	remaining: 4.93s
200:	learn: 0.9964286	total: 1.41s	remaining: 4.92s
201:	learn: 0.9964286	total: 1.42s	remaining: 4.92s
202:	learn: 0.9957143	total: 1.43s	remaining: 4.89s
203:	learn: 0.9957143	total: 1.43s	remaining: 4.89s
204:	learn: 0.9957143	total: 1.44s	remaining: 4.87s
205:	learn: 0.9964286	total: 1.44s	remaining: 4.86s
206:	learn: 0.9964286	total: 1.45s	remaining: 4.86s
207:	learn: 0.9964286	total: 1.46s	remaining: 4.85s
208:	learn: 0.9971429	total: 1.47s	remaining: 4.84s
209:	learn: 0.9971429	total: 1.47s	remaining: 4.83s
210:	learn: 0.9971429	total: 1.48s	remaining: 4.82s
211:	learn: 0.9971429	total: 1.48s	remaining: 4.81s
212:	learn: 0.9971429	total: 1.49s	remaining: 4.81s
213:	learn: 0.9971429	total: 1.5s	remaining: 4.8s
214:	learn: 0.9964286	total: 1.5s	remaining: 4.79s
215:	learn: 0.9964286	total: 1.51s	remaining: 4.78s
216:	learn: 0.9964286	total: 1.51s	remaining: 4.77s
217:	learn: 0.9971429	total: 1.52s	remaining: 4.76s
218:	learn: 0.9971429	total: 1.53s	remaining: 4.75s
219:	learn: 0.9971429	total: 1.54s	remaining: 4.76s
220:	learn: 0.9971429	total: 1.55s	remaining: 4.76s
221:	learn: 0.9978571	total: 1.56s	remaining: 4.76s
222:	learn: 0.9978571	total: 1.57s	remaining: 4.75s
223:	learn: 0.9978571	total: 1.57s	remaining: 4.75s
224:	learn: 0.9985714	total: 1.58s	remaining: 4.73s
225:	learn: 0.9985714	total: 1.58s	remaining: 4.73s
226:	learn: 0.9985714	total: 1.59s	remaining: 4.73s
227:	learn: 0.9985714	total: 1.6s	remaining: 4.73s
228:	learn: 0.9985714	total: 1.61s	remaining: 4.73s
229:	learn: 0.9985714	total: 1.62s	remaining: 4.72s
230:	learn: 0.9985714	total: 1.63s	remaining: 4.71s
231:	learn: 0.9985714	total: 1.63s	remaining: 4.71s
232:	learn: 0.9985714	total: 1.64s	remaining: 4.69s
233:	learn: 0.9985714	total: 1.65s	remaining: 4.68s
234:	learn: 0.9985714	total: 1.65s	remaining: 4.67s
235:	learn: 0.9985714	total: 1.66s	remaining: 4.67s
236:	learn: 0.9985714	total: 1.67s	remaining: 4.66s
237:	learn: 0.9978571	total: 1.67s	remaining: 4.65s
238:	learn: 0.9978571	total: 1.68s	remaining: 4.65s
239:	learn: 0.9978571	total: 1.69s	remaining: 4.65s
240:	learn: 0.9985714	total: 1.7s	remaining: 4.64s
241:	learn: 0.9985714	total: 1.7s	remaining: 4.63s
242:	learn: 0.9985714	total: 1.71s	remaining: 4.62s
243:	learn: 0.9985714	total: 1.71s	remaining: 4.61s
244:	learn: 0.9985714	total: 1.72s	remaining: 4.6s
245:	learn: 0.9985714	total: 1.73s	remaining: 4.59s
246:	learn: 0.9978571	total: 1.74s	remaining: 4.59s
247:	learn: 0.9985714	total: 1.74s	remaining: 4.58s
248:	learn: 0.9985714	total: 1.76s	remaining: 4.59s
249:	learn: 0.9985714	total: 1.76s	remaining: 4.59s
250:	learn: 0.9985714	total: 1.77s	remaining: 4.58s
251:	learn: 0.9985714	total: 1.78s	remaining: 4.57s
252:	learn: 0.9985714	total: 1.78s	remaining: 4.56s
253:	learn: 0.9985714	total: 1.79s	remaining: 4.55s
254:	learn: 0.9985714	total: 1.8s	remaining: 4.54s
255:	learn: 0.9985714	total: 1.8s	remaining: 4.54s
256:	learn: 0.9985714	total: 1.81s	remaining: 4.53s
257:	learn: 0.9985714	total: 1.82s	remaining: 4.52s
258:	learn: 0.9985714	total: 1.82s	remaining: 4.52s
259:	learn: 0.9985714	total: 1.83s	remaining: 4.51s
260:	learn: 0.9985714	total: 1.84s	remaining: 4.5s
261:	learn: 0.9985714	total: 1.85s	remaining: 4.5s
262:	learn: 0.9985714	total: 1.85s	remaining: 4.49s
263:	learn: 0.9985714	total: 1.86s	remaining: 4.48s
264:	learn: 0.9992857	total: 1.87s	remaining: 4.47s
265:	learn: 0.9992857	total: 1.87s	remaining: 4.46s
266:	learn: 0.9992857	total: 1.88s	remaining: 4.46s
267:	learn: 0.9985714	total: 1.89s	remaining: 4.45s
268:	learn: 0.9985714	total: 1.89s	remaining: 4.44s
269:	learn: 0.9985714	total: 1.9s	remaining: 4.44s
270:	learn: 0.9985714	total: 1.91s	remaining: 4.43s
271:	learn: 0.9985714	total: 1.91s	remaining: 4.42s
272:	learn: 0.9985714	total: 1.92s	remaining: 4.42s
273:	learn: 0.9992857	total: 1.93s	remaining: 4.41s
274:	learn: 0.9985714	total: 1.94s	remaining: 4.41s
275:	learn: 0.9985714	total: 1.95s	remaining: 4.4s
276:	learn: 0.9985714	total: 1.95s	remaining: 4.4s
277:	learn: 0.9992857	total: 1.96s	remaining: 4.39s
278:	learn: 0.9985714	total: 1.97s	remaining: 4.38s
279:	learn: 0.9985714	total: 1.98s	remaining: 4.38s
280:	learn: 0.9992857	total: 1.98s	remaining: 4.37s
281:	learn: 0.9985714	total: 1.99s	remaining: 4.36s
282:	learn: 0.9992857	total: 2s	remaining: 4.35s
283:	learn: 1.0000000	total: 2s	remaining: 4.34s
284:	learn: 1.0000000	total: 2.01s	remaining: 4.33s
285:	learn: 1.0000000	total: 2.01s	remaining: 4.32s
286:	learn: 1.0000000	total: 2.02s	remaining: 4.31s
287:	learn: 1.0000000	total: 2.03s	remaining: 4.31s
288:	learn: 1.0000000	total: 2.04s	remaining: 4.3s
289:	learn: 1.0000000	total: 2.04s	remaining: 4.29s
290:	learn: 1.0000000	total: 2.04s	remaining: 4.28s
291:	learn: 1.0000000	total: 2.06s	remaining: 4.28s
292:	learn: 1.0000000	total: 2.06s	remaining: 4.27s
293:	learn: 1.0000000	total: 2.07s	remaining: 4.26s
294:	learn: 1.0000000	total: 2.08s	remaining: 4.26s
295:	learn: 1.0000000	total: 2.08s	remaining: 4.25s
296:	learn: 1.0000000	total: 2.09s	remaining: 4.25s
297:	learn: 1.0000000	total: 2.1s	remaining: 4.24s
298:	learn: 1.0000000	total: 2.11s	remaining: 4.24s
299:	learn: 1.0000000	total: 2.12s	remaining: 4.24s
300:	learn: 1.0000000	total: 2.12s	remaining: 4.22s
301:	learn: 1.0000000	total: 2.13s	remaining: 4.22s
302:	learn: 1.0000000	total: 2.14s	remaining: 4.22s
303:	learn: 1.0000000	total: 2.15s	remaining: 4.21s
304:	learn: 1.0000000	total: 2.15s	remaining: 4.2s
305:	learn: 0.9992857	total: 2.16s	remaining: 4.19s
306:	learn: 0.9992857	total: 2.17s	remaining: 4.18s
307:	learn: 0.9992857	total: 2.17s	remaining: 4.18s
308:	learn: 0.9992857	total: 2.18s	remaining: 4.17s
309:	learn: 0.9992857	total: 2.19s	remaining: 4.17s
310:	learn: 0.9992857	total: 2.2s	remaining: 4.16s
311:	learn: 0.9992857	total: 2.21s	remaining: 4.16s
312:	learn: 0.9992857	total: 2.21s	remaining: 4.15s
313:	learn: 0.9992857	total: 2.22s	remaining: 4.14s
314:	learn: 0.9992857	total: 2.22s	remaining: 4.13s
315:	learn: 0.9992857	total: 2.23s	remaining: 4.12s
316:	learn: 0.9992857	total: 2.24s	remaining: 4.12s
317:	learn: 0.9992857	total: 2.25s	remaining: 4.11s
318:	learn: 1.0000000	total: 2.25s	remaining: 4.11s
319:	learn: 1.0000000	total: 2.26s	remaining: 4.1s
320:	learn: 1.0000000	total: 2.27s	remaining: 4.09s
321:	learn: 1.0000000	total: 2.28s	remaining: 4.09s
322:	learn: 1.0000000	total: 2.28s	remaining: 4.08s
323:	learn: 1.0000000	total: 2.29s	remaining: 4.08s
324:	learn: 1.0000000	total: 2.3s	remaining: 4.07s
325:	learn: 1.0000000	total: 2.31s	remaining: 4.06s
326:	learn: 1.0000000	total: 2.32s	remaining: 4.06s
327:	learn: 1.0000000	total: 2.32s	remaining: 4.05s
328:	learn: 1.0000000	total: 2.33s	remaining: 4.04s
329:	learn: 1.0000000	total: 2.34s	remaining: 4.04s
330:	learn: 1.0000000	total: 2.34s	remaining: 4.03s
331:	learn: 1.0000000	total: 2.35s	remaining: 4.02s
332:	learn: 1.0000000	total: 2.36s	remaining: 4.02s
333:	learn: 1.0000000	total: 2.37s	remaining: 4.01s
334:	learn: 1.0000000	total: 2.38s	remaining: 4.01s
335:	learn: 1.0000000	total: 2.38s	remaining: 4s
336:	learn: 1.0000000	total: 2.39s	remaining: 3.99s
337:	learn: 1.0000000	total: 2.39s	remaining: 3.98s
338:	learn: 1.0000000	total: 2.4s	remaining: 3.98s
339:	learn: 1.0000000	total: 2.41s	remaining: 3.96s
340:	learn: 1.0000000	total: 2.41s	remaining: 3.96s
341:	learn: 1.0000000	total: 2.42s	remaining: 3.95s
342:	learn: 1.0000000	total: 2.42s	remaining: 3.93s
343:	learn: 1.0000000	total: 2.43s	remaining: 3.93s
344:	learn: 1.0000000	total: 2.44s	remaining: 3.92s
345:	learn: 1.0000000	total: 2.44s	remaining: 3.92s
346:	learn: 1.0000000	total: 2.45s	remaining: 3.91s
347:	learn: 1.0000000	total: 2.46s	remaining: 3.9s
348:	learn: 1.0000000	total: 2.47s	remaining: 3.9s
349:	learn: 1.0000000	total: 2.48s	remaining: 3.89s
350:	learn: 1.0000000	total: 2.48s	remaining: 3.89s
351:	learn: 1.0000000	total: 2.49s	remaining: 3.88s
352:	learn: 1.0000000	total: 2.5s	remaining: 3.87s
353:	learn: 1.0000000	total: 2.51s	remaining: 3.87s
354:	learn: 1.0000000	total: 2.52s	remaining: 3.87s
355:	learn: 1.0000000	total: 2.52s	remaining: 3.86s
356:	learn: 1.0000000	total: 2.53s	remaining: 3.85s
357:	learn: 1.0000000	total: 2.54s	remaining: 3.84s
358:	learn: 1.0000000	total: 2.54s	remaining: 3.83s
359:	learn: 1.0000000	total: 2.55s	remaining: 3.83s
360:	learn: 1.0000000	total: 2.56s	remaining: 3.82s
361:	learn: 1.0000000	total: 2.56s	remaining: 3.81s
362:	learn: 1.0000000	total: 2.57s	remaining: 3.8s
363:	learn: 1.0000000	total: 2.58s	remaining: 3.8s
364:	learn: 1.0000000	total: 2.59s	remaining: 3.79s
365:	learn: 1.0000000	total: 2.59s	remaining: 3.78s
366:	learn: 1.0000000	total: 2.6s	remaining: 3.78s
367:	learn: 1.0000000	total: 2.61s	remaining: 3.77s
368:	learn: 1.0000000	total: 2.61s	remaining: 3.76s
369:	learn: 1.0000000	total: 2.62s	remaining: 3.75s
370:	learn: 1.0000000	total: 2.63s	remaining: 3.75s
371:	learn: 1.0000000	total: 2.63s	remaining: 3.74s
372:	learn: 1.0000000	total: 2.64s	remaining: 3.73s
373:	learn: 1.0000000	total: 2.65s	remaining: 3.72s
374:	learn: 1.0000000	total: 2.65s	remaining: 3.72s
375:	learn: 1.0000000	total: 2.66s	remaining: 3.71s
376:	learn: 1.0000000	total: 2.67s	remaining: 3.71s
377:	learn: 1.0000000	total: 2.68s	remaining: 3.7s
378:	learn: 1.0000000	total: 2.69s	remaining: 3.69s
379:	learn: 1.0000000	total: 2.69s	remaining: 3.69s
380:	learn: 1.0000000	total: 2.7s	remaining: 3.68s
381:	learn: 1.0000000	total: 2.71s	remaining: 3.67s
382:	learn: 1.0000000	total: 2.72s	remaining: 3.67s
383:	learn: 1.0000000	total: 2.72s	remaining: 3.66s
384:	learn: 1.0000000	total: 2.73s	remaining: 3.65s
385:	learn: 1.0000000	total: 2.74s	remaining: 3.65s
386:	learn: 1.0000000	total: 2.75s	remaining: 3.64s
387:	learn: 1.0000000	total: 2.75s	remaining: 3.63s
388:	learn: 1.0000000	total: 2.76s	remaining: 3.62s
389:	learn: 1.0000000	total: 2.77s	remaining: 3.62s
390:	learn: 1.0000000	total: 2.77s	remaining: 3.61s
391:	learn: 1.0000000	total: 2.78s	remaining: 3.6s
392:	learn: 1.0000000	total: 2.79s	remaining: 3.59s
393:	learn: 1.0000000	total: 2.79s	remaining: 3.59s
394:	learn: 1.0000000	total: 2.8s	remaining: 3.58s
395:	learn: 1.0000000	total: 2.81s	remaining: 3.57s
396:	learn: 1.0000000	total: 2.81s	remaining: 3.56s
397:	learn: 1.0000000	total: 2.82s	remaining: 3.56s
398:	learn: 1.0000000	total: 2.83s	remaining: 3.55s
399:	learn: 1.0000000	total: 2.83s	remaining: 3.54s
400:	learn: 1.0000000	total: 2.85s	remaining: 3.54s
401:	learn: 1.0000000	total: 2.86s	remaining: 3.54s
402:	learn: 1.0000000	total: 2.86s	remaining: 3.53s
403:	learn: 1.0000000	total: 2.87s	remaining: 3.52s
404:	learn: 1.0000000	total: 2.88s	remaining: 3.52s
405:	learn: 1.0000000	total: 2.88s	remaining: 3.51s
406:	learn: 1.0000000	total: 2.89s	remaining: 3.5s
407:	learn: 1.0000000	total: 2.9s	remaining: 3.49s
408:	learn: 1.0000000	total: 2.9s	remaining: 3.48s
409:	learn: 1.0000000	total: 2.91s	remaining: 3.48s
410:	learn: 1.0000000	total: 2.92s	remaining: 3.47s
411:	learn: 1.0000000	total: 2.93s	remaining: 3.47s
412:	learn: 1.0000000	total: 2.93s	remaining: 3.46s
413:	learn: 1.0000000	total: 2.94s	remaining: 3.46s
414:	learn: 1.0000000	total: 2.95s	remaining: 3.45s
415:	learn: 1.0000000	total: 2.96s	remaining: 3.44s
416:	learn: 1.0000000	total: 2.96s	remaining: 3.43s
417:	learn: 1.0000000	total: 2.97s	remaining: 3.43s
418:	learn: 1.0000000	total: 2.98s	remaining: 3.42s
419:	learn: 1.0000000	total: 2.99s	remaining: 3.41s
420:	learn: 1.0000000	total: 2.99s	remaining: 3.4s
421:	learn: 1.0000000	total: 3s	remaining: 3.4s
422:	learn: 1.0000000	total: 3.01s	remaining: 3.39s
423:	learn: 1.0000000	total: 3.01s	remaining: 3.38s
424:	learn: 1.0000000	total: 3.02s	remaining: 3.38s
425:	learn: 1.0000000	total: 3.03s	remaining: 3.37s
426:	learn: 1.0000000	total: 3.04s	remaining: 3.36s
427:	learn: 1.0000000	total: 3.04s	remaining: 3.35s
428:	learn: 1.0000000	total: 3.05s	remaining: 3.35s
429:	learn: 1.0000000	total: 3.06s	remaining: 3.34s
430:	learn: 1.0000000	total: 3.07s	remaining: 3.34s
431:	learn: 1.0000000	total: 3.08s	remaining: 3.33s
432:	learn: 1.0000000	total: 3.08s	remaining: 3.33s
433:	learn: 1.0000000	total: 3.09s	remaining: 3.32s
434:	learn: 1.0000000	total: 3.1s	remaining: 3.31s
435:	learn: 1.0000000	total: 3.1s	remaining: 3.31s
436:	learn: 1.0000000	total: 3.11s	remaining: 3.3s
437:	learn: 1.0000000	total: 3.12s	remaining: 3.29s
438:	learn: 1.0000000	total: 3.13s	remaining: 3.29s
439:	learn: 1.0000000	total: 3.14s	remaining: 3.28s
440:	learn: 1.0000000	total: 3.15s	remaining: 3.27s
441:	learn: 1.0000000	total: 3.15s	remaining: 3.27s
442:	learn: 1.0000000	total: 3.16s	remaining: 3.26s
443:	learn: 1.0000000	total: 3.17s	remaining: 3.25s
444:	learn: 1.0000000	total: 3.18s	remaining: 3.25s
445:	learn: 1.0000000	total: 3.18s	remaining: 3.24s
446:	learn: 1.0000000	total: 3.19s	remaining: 3.23s
447:	learn: 1.0000000	total: 3.2s	remaining: 3.22s
448:	learn: 1.0000000	total: 3.2s	remaining: 3.21s
449:	learn: 1.0000000	total: 3.21s	remaining: 3.21s
450:	learn: 1.0000000	total: 3.22s	remaining: 3.2s
451:	learn: 1.0000000	total: 3.22s	remaining: 3.19s
452:	learn: 1.0000000	total: 3.23s	remaining: 3.19s
453:	learn: 1.0000000	total: 3.25s	remaining: 3.19s
454:	learn: 1.0000000	total: 3.25s	remaining: 3.18s
455:	learn: 1.0000000	total: 3.26s	remaining: 3.17s
456:	learn: 1.0000000	total: 3.26s	remaining: 3.16s
457:	learn: 1.0000000	total: 3.27s	remaining: 3.16s
458:	learn: 1.0000000	total: 3.28s	remaining: 3.15s
459:	learn: 1.0000000	total: 3.29s	remaining: 3.14s
460:	learn: 1.0000000	total: 3.29s	remaining: 3.13s
461:	learn: 1.0000000	total: 3.3s	remaining: 3.13s
462:	learn: 1.0000000	total: 3.31s	remaining: 3.12s
463:	learn: 1.0000000	total: 3.32s	remaining: 3.12s
464:	learn: 1.0000000	total: 3.32s	remaining: 3.11s
465:	learn: 1.0000000	total: 3.33s	remaining: 3.1s
466:	learn: 1.0000000	total: 3.34s	remaining: 3.1s
467:	learn: 1.0000000	total: 3.35s	remaining: 3.09s
468:	learn: 1.0000000	total: 3.35s	remaining: 3.08s
469:	learn: 1.0000000	total: 3.36s	remaining: 3.08s
470:	learn: 1.0000000	total: 3.37s	remaining: 3.07s
471:	learn: 1.0000000	total: 3.38s	remaining: 3.06s
472:	learn: 1.0000000	total: 3.38s	remaining: 3.05s
473:	learn: 1.0000000	total: 3.39s	remaining: 3.05s
474:	learn: 1.0000000	total: 3.4s	remaining: 3.04s
475:	learn: 1.0000000	total: 3.4s	remaining: 3.03s
476:	learn: 1.0000000	total: 3.4s	remaining: 3.02s
477:	learn: 1.0000000	total: 3.41s	remaining: 3.01s
478:	learn: 1.0000000	total: 3.42s	remaining: 3s
479:	learn: 1.0000000	total: 3.42s	remaining: 2.99s
480:	learn: 1.0000000	total: 3.43s	remaining: 2.99s
481:	learn: 1.0000000	total: 3.44s	remaining: 2.98s
482:	learn: 1.0000000	total: 3.45s	remaining: 2.98s
483:	learn: 1.0000000	total: 3.46s	remaining: 2.97s
484:	learn: 1.0000000	total: 3.46s	remaining: 2.96s
485:	learn: 1.0000000	total: 3.47s	remaining: 2.96s
486:	learn: 1.0000000	total: 3.48s	remaining: 2.95s
487:	learn: 1.0000000	total: 3.49s	remaining: 2.94s
488:	learn: 1.0000000	total: 3.49s	remaining: 2.94s
489:	learn: 1.0000000	total: 3.5s	remaining: 2.93s
490:	learn: 1.0000000	total: 3.51s	remaining: 2.92s
491:	learn: 1.0000000	total: 3.51s	remaining: 2.91s
492:	learn: 1.0000000	total: 3.52s	remaining: 2.91s
493:	learn: 1.0000000	total: 3.53s	remaining: 2.9s
494:	learn: 1.0000000	total: 3.53s	remaining: 2.89s
495:	learn: 1.0000000	total: 3.54s	remaining: 2.88s
496:	learn: 1.0000000	total: 3.55s	remaining: 2.88s
497:	learn: 1.0000000	total: 3.55s	remaining: 2.87s
498:	learn: 1.0000000	total: 3.56s	remaining: 2.86s
499:	learn: 1.0000000	total: 3.56s	remaining: 2.85s
500:	learn: 1.0000000	total: 3.57s	remaining: 2.85s
501:	learn: 1.0000000	total: 3.58s	remaining: 2.84s
502:	learn: 1.0000000	total: 3.58s	remaining: 2.83s
503:	learn: 1.0000000	total: 3.59s	remaining: 2.82s
504:	learn: 1.0000000	total: 3.6s	remaining: 2.81s
505:	learn: 1.0000000	total: 3.61s	remaining: 2.81s
506:	learn: 1.0000000	total: 3.61s	remaining: 2.8s
507:	learn: 1.0000000	total: 3.62s	remaining: 2.79s
508:	learn: 1.0000000	total: 3.63s	remaining: 2.79s
509:	learn: 1.0000000	total: 3.63s	remaining: 2.78s
510:	learn: 1.0000000	total: 3.65s	remaining: 2.78s
511:	learn: 1.0000000	total: 3.66s	remaining: 2.77s
512:	learn: 1.0000000	total: 3.66s	remaining: 2.76s
513:	learn: 1.0000000	total: 3.67s	remaining: 2.75s
514:	learn: 1.0000000	total: 3.68s	remaining: 2.75s
515:	learn: 1.0000000	total: 3.68s	remaining: 2.74s
516:	learn: 1.0000000	total: 3.69s	remaining: 2.73s
517:	learn: 1.0000000	total: 3.69s	remaining: 2.73s
518:	learn: 1.0000000	total: 3.7s	remaining: 2.72s
519:	learn: 1.0000000	total: 3.71s	remaining: 2.71s
520:	learn: 1.0000000	total: 3.72s	remaining: 2.7s
521:	learn: 1.0000000	total: 3.72s	remaining: 2.7s
522:	learn: 1.0000000	total: 3.73s	remaining: 2.69s
523:	learn: 1.0000000	total: 3.74s	remaining: 2.68s
524:	learn: 1.0000000	total: 3.75s	remaining: 2.67s
525:	learn: 1.0000000	total: 3.75s	remaining: 2.67s
526:	learn: 1.0000000	total: 3.76s	remaining: 2.66s
527:	learn: 1.0000000	total: 3.77s	remaining: 2.65s
528:	learn: 1.0000000	total: 3.77s	remaining: 2.65s
529:	learn: 1.0000000	total: 3.78s	remaining: 2.64s
530:	learn: 1.0000000	total: 3.79s	remaining: 2.63s
531:	learn: 1.0000000	total: 3.79s	remaining: 2.62s
532:	learn: 1.0000000	total: 3.8s	remaining: 2.62s
533:	learn: 1.0000000	total: 3.81s	remaining: 2.61s
534:	learn: 1.0000000	total: 3.81s	remaining: 2.6s
535:	learn: 1.0000000	total: 3.82s	remaining: 2.59s
536:	learn: 1.0000000	total: 3.83s	remaining: 2.59s
537:	learn: 1.0000000	total: 3.84s	remaining: 2.58s
538:	learn: 1.0000000	total: 3.84s	remaining: 2.57s
539:	learn: 1.0000000	total: 3.85s	remaining: 2.57s
540:	learn: 1.0000000	total: 3.86s	remaining: 2.56s
541:	learn: 1.0000000	total: 3.87s	remaining: 2.55s
542:	learn: 1.0000000	total: 3.87s	remaining: 2.55s
543:	learn: 1.0000000	total: 3.88s	remaining: 2.54s
544:	learn: 1.0000000	total: 3.89s	remaining: 2.53s
545:	learn: 1.0000000	total: 3.9s	remaining: 2.52s
546:	learn: 1.0000000	total: 3.9s	remaining: 2.52s
547:	learn: 1.0000000	total: 3.91s	remaining: 2.51s
548:	learn: 1.0000000	total: 3.92s	remaining: 2.51s
549:	learn: 1.0000000	total: 3.92s	remaining: 2.5s
550:	learn: 1.0000000	total: 3.93s	remaining: 2.49s
551:	learn: 1.0000000	total: 3.94s	remaining: 2.48s
552:	learn: 1.0000000	total: 3.94s	remaining: 2.47s
553:	learn: 1.0000000	total: 3.95s	remaining: 2.47s
554:	learn: 1.0000000	total: 3.96s	remaining: 2.46s
555:	learn: 1.0000000	total: 3.96s	remaining: 2.45s
556:	learn: 1.0000000	total: 3.97s	remaining: 2.44s
557:	learn: 1.0000000	total: 3.98s	remaining: 2.44s
558:	learn: 1.0000000	total: 3.99s	remaining: 2.43s
559:	learn: 1.0000000	total: 3.99s	remaining: 2.42s
560:	learn: 1.0000000	total: 4s	remaining: 2.42s
561:	learn: 1.0000000	total: 4.01s	remaining: 2.41s
562:	learn: 1.0000000	total: 4.01s	remaining: 2.4s
563:	learn: 1.0000000	total: 4.03s	remaining: 2.4s
564:	learn: 1.0000000	total: 4.04s	remaining: 2.39s
565:	learn: 1.0000000	total: 4.04s	remaining: 2.39s
566:	learn: 1.0000000	total: 4.05s	remaining: 2.38s
567:	learn: 1.0000000	total: 4.06s	remaining: 2.37s
568:	learn: 1.0000000	total: 4.07s	remaining: 2.37s
569:	learn: 1.0000000	total: 4.07s	remaining: 2.36s
570:	learn: 1.0000000	total: 4.08s	remaining: 2.35s
571:	learn: 1.0000000	total: 4.09s	remaining: 2.34s
572:	learn: 1.0000000	total: 4.09s	remaining: 2.34s
573:	learn: 1.0000000	total: 4.1s	remaining: 2.33s
574:	learn: 1.0000000	total: 4.11s	remaining: 2.32s
575:	learn: 1.0000000	total: 4.12s	remaining: 2.31s
576:	learn: 1.0000000	total: 4.12s	remaining: 2.31s
577:	learn: 1.0000000	total: 4.13s	remaining: 2.3s
578:	learn: 1.0000000	total: 4.14s	remaining: 2.29s
579:	learn: 1.0000000	total: 4.14s	remaining: 2.29s
580:	learn: 1.0000000	total: 4.15s	remaining: 2.28s
581:	learn: 1.0000000	total: 4.16s	remaining: 2.27s
582:	learn: 1.0000000	total: 4.17s	remaining: 2.27s
583:	learn: 1.0000000	total: 4.17s	remaining: 2.26s
584:	learn: 1.0000000	total: 4.18s	remaining: 2.25s
585:	learn: 1.0000000	total: 4.18s	remaining: 2.24s
586:	learn: 1.0000000	total: 4.19s	remaining: 2.23s
587:	learn: 1.0000000	total: 4.2s	remaining: 2.23s
588:	learn: 1.0000000	total: 4.21s	remaining: 2.22s
589:	learn: 1.0000000	total: 4.21s	remaining: 2.21s
590:	learn: 1.0000000	total: 4.22s	remaining: 2.21s
591:	learn: 1.0000000	total: 4.23s	remaining: 2.2s
592:	learn: 1.0000000	total: 4.24s	remaining: 2.19s
593:	learn: 1.0000000	total: 4.25s	remaining: 2.19s
594:	learn: 1.0000000	total: 4.25s	remaining: 2.18s
595:	learn: 1.0000000	total: 4.26s	remaining: 2.17s
596:	learn: 1.0000000	total: 4.27s	remaining: 2.17s
597:	learn: 1.0000000	total: 4.28s	remaining: 2.16s
598:	learn: 1.0000000	total: 4.28s	remaining: 2.15s
599:	learn: 1.0000000	total: 4.29s	remaining: 2.15s
600:	learn: 1.0000000	total: 4.3s	remaining: 2.14s
601:	learn: 1.0000000	total: 4.31s	remaining: 2.13s
602:	learn: 1.0000000	total: 4.32s	remaining: 2.13s
603:	learn: 1.0000000	total: 4.33s	remaining: 2.12s
604:	learn: 1.0000000	total: 4.33s	remaining: 2.11s
605:	learn: 1.0000000	total: 4.34s	remaining: 2.1s
606:	learn: 1.0000000	total: 4.35s	remaining: 2.1s
607:	learn: 1.0000000	total: 4.36s	remaining: 2.09s
608:	learn: 1.0000000	total: 4.36s	remaining: 2.08s
609:	learn: 1.0000000	total: 4.37s	remaining: 2.08s
610:	learn: 1.0000000	total: 4.38s	remaining: 2.07s
611:	learn: 1.0000000	total: 4.38s	remaining: 2.06s
612:	learn: 1.0000000	total: 4.39s	remaining: 2.06s
613:	learn: 1.0000000	total: 4.4s	remaining: 2.05s
614:	learn: 1.0000000	total: 4.41s	remaining: 2.04s
615:	learn: 1.0000000	total: 4.41s	remaining: 2.03s
616:	learn: 1.0000000	total: 4.42s	remaining: 2.03s
617:	learn: 1.0000000	total: 4.43s	remaining: 2.02s
618:	learn: 1.0000000	total: 4.43s	remaining: 2.01s
619:	learn: 1.0000000	total: 4.44s	remaining: 2s
620:	learn: 1.0000000	total: 4.45s	remaining: 2s
621:	learn: 1.0000000	total: 4.46s	remaining: 1.99s
622:	learn: 1.0000000	total: 4.46s	remaining: 1.98s
623:	learn: 1.0000000	total: 4.47s	remaining: 1.98s
624:	learn: 1.0000000	total: 4.48s	remaining: 1.97s
625:	learn: 1.0000000	total: 4.49s	remaining: 1.96s
626:	learn: 1.0000000	total: 4.49s	remaining: 1.96s
627:	learn: 1.0000000	total: 4.5s	remaining: 1.95s
628:	learn: 1.0000000	total: 4.51s	remaining: 1.94s
629:	learn: 1.0000000	total: 4.51s	remaining: 1.93s
630:	learn: 1.0000000	total: 4.52s	remaining: 1.93s
631:	learn: 1.0000000	total: 4.53s	remaining: 1.92s
632:	learn: 1.0000000	total: 4.54s	remaining: 1.91s
633:	learn: 1.0000000	total: 4.54s	remaining: 1.91s
634:	learn: 1.0000000	total: 4.55s	remaining: 1.9s
635:	learn: 1.0000000	total: 4.56s	remaining: 1.89s
636:	learn: 1.0000000	total: 4.57s	remaining: 1.89s
637:	learn: 1.0000000	total: 4.57s	remaining: 1.88s
638:	learn: 1.0000000	total: 4.58s	remaining: 1.87s
639:	learn: 1.0000000	total: 4.59s	remaining: 1.86s
640:	learn: 1.0000000	total: 4.59s	remaining: 1.86s
641:	learn: 1.0000000	total: 4.61s	remaining: 1.85s
642:	learn: 1.0000000	total: 4.61s	remaining: 1.84s
643:	learn: 1.0000000	total: 4.62s	remaining: 1.84s
644:	learn: 1.0000000	total: 4.63s	remaining: 1.83s
645:	learn: 1.0000000	total: 4.63s	remaining: 1.82s
646:	learn: 1.0000000	total: 4.64s	remaining: 1.81s
647:	learn: 1.0000000	total: 4.65s	remaining: 1.81s
648:	learn: 1.0000000	total: 4.65s	remaining: 1.8s
649:	learn: 1.0000000	total: 4.66s	remaining: 1.79s
650:	learn: 1.0000000	total: 4.67s	remaining: 1.78s
651:	learn: 1.0000000	total: 4.67s	remaining: 1.78s
652:	learn: 1.0000000	total: 4.68s	remaining: 1.77s
653:	learn: 1.0000000	total: 4.69s	remaining: 1.76s
654:	learn: 1.0000000	total: 4.69s	remaining: 1.75s
655:	learn: 1.0000000	total: 4.7s	remaining: 1.75s
656:	learn: 1.0000000	total: 4.71s	remaining: 1.74s
657:	learn: 1.0000000	total: 4.72s	remaining: 1.73s
658:	learn: 1.0000000	total: 4.72s	remaining: 1.73s
659:	learn: 1.0000000	total: 4.73s	remaining: 1.72s
660:	learn: 1.0000000	total: 4.74s	remaining: 1.71s
661:	learn: 1.0000000	total: 4.74s	remaining: 1.7s
662:	learn: 1.0000000	total: 4.75s	remaining: 1.7s
663:	learn: 1.0000000	total: 4.76s	remaining: 1.69s
664:	learn: 1.0000000	total: 4.76s	remaining: 1.68s
665:	learn: 1.0000000	total: 4.77s	remaining: 1.68s
666:	learn: 1.0000000	total: 4.78s	remaining: 1.67s
667:	learn: 1.0000000	total: 4.79s	remaining: 1.66s
668:	learn: 1.0000000	total: 4.8s	remaining: 1.66s
669:	learn: 1.0000000	total: 4.8s	remaining: 1.65s
670:	learn: 1.0000000	total: 4.81s	remaining: 1.64s
671:	learn: 1.0000000	total: 4.82s	remaining: 1.63s
672:	learn: 1.0000000	total: 4.82s	remaining: 1.63s
673:	learn: 1.0000000	total: 4.83s	remaining: 1.62s
674:	learn: 1.0000000	total: 4.84s	remaining: 1.61s
675:	learn: 1.0000000	total: 4.84s	remaining: 1.6s
676:	learn: 1.0000000	total: 4.85s	remaining: 1.6s
677:	learn: 1.0000000	total: 4.86s	remaining: 1.59s
678:	learn: 1.0000000	total: 4.87s	remaining: 1.58s
679:	learn: 1.0000000	total: 4.87s	remaining: 1.57s
680:	learn: 1.0000000	total: 4.88s	remaining: 1.57s
681:	learn: 1.0000000	total: 4.88s	remaining: 1.56s
682:	learn: 1.0000000	total: 4.89s	remaining: 1.55s
683:	learn: 1.0000000	total: 4.9s	remaining: 1.55s
684:	learn: 1.0000000	total: 4.9s	remaining: 1.54s
685:	learn: 1.0000000	total: 4.91s	remaining: 1.53s
686:	learn: 1.0000000	total: 4.92s	remaining: 1.52s
687:	learn: 1.0000000	total: 4.93s	remaining: 1.52s
688:	learn: 1.0000000	total: 4.93s	remaining: 1.51s
689:	learn: 1.0000000	total: 4.94s	remaining: 1.5s
690:	learn: 1.0000000	total: 4.95s	remaining: 1.5s
691:	learn: 1.0000000	total: 4.96s	remaining: 1.49s
692:	learn: 1.0000000	total: 4.96s	remaining: 1.48s
693:	learn: 1.0000000	total: 4.97s	remaining: 1.48s
694:	learn: 1.0000000	total: 4.98s	remaining: 1.47s
695:	learn: 1.0000000	total: 4.99s	remaining: 1.46s
696:	learn: 1.0000000	total: 5s	remaining: 1.46s
697:	learn: 1.0000000	total: 5s	remaining: 1.45s
698:	learn: 1.0000000	total: 5.01s	remaining: 1.44s
699:	learn: 1.0000000	total: 5.02s	remaining: 1.43s
700:	learn: 1.0000000	total: 5.03s	remaining: 1.43s
701:	learn: 1.0000000	total: 5.04s	remaining: 1.42s
702:	learn: 1.0000000	total: 5.04s	remaining: 1.41s
703:	learn: 1.0000000	total: 5.05s	remaining: 1.41s
704:	learn: 1.0000000	total: 5.06s	remaining: 1.4s
705:	learn: 1.0000000	total: 5.07s	remaining: 1.39s
706:	learn: 1.0000000	total: 5.07s	remaining: 1.38s
707:	learn: 1.0000000	total: 5.08s	remaining: 1.38s
708:	learn: 1.0000000	total: 5.09s	remaining: 1.37s
709:	learn: 1.0000000	total: 5.1s	remaining: 1.36s
710:	learn: 1.0000000	total: 5.1s	remaining: 1.36s
711:	learn: 1.0000000	total: 5.11s	remaining: 1.35s
712:	learn: 1.0000000	total: 5.12s	remaining: 1.34s
713:	learn: 1.0000000	total: 5.13s	remaining: 1.33s
714:	learn: 1.0000000	total: 5.13s	remaining: 1.33s
715:	learn: 1.0000000	total: 5.14s	remaining: 1.32s
716:	learn: 1.0000000	total: 5.14s	remaining: 1.31s
717:	learn: 1.0000000	total: 5.15s	remaining: 1.31s
718:	learn: 1.0000000	total: 5.16s	remaining: 1.3s
719:	learn: 1.0000000	total: 5.17s	remaining: 1.29s
720:	learn: 1.0000000	total: 5.17s	remaining: 1.28s
721:	learn: 1.0000000	total: 5.18s	remaining: 1.28s
722:	learn: 1.0000000	total: 5.19s	remaining: 1.27s
723:	learn: 1.0000000	total: 5.2s	remaining: 1.26s
724:	learn: 1.0000000	total: 5.21s	remaining: 1.26s
725:	learn: 1.0000000	total: 5.21s	remaining: 1.25s
726:	learn: 1.0000000	total: 5.22s	remaining: 1.24s
727:	learn: 1.0000000	total: 5.22s	remaining: 1.23s
728:	learn: 1.0000000	total: 5.23s	remaining: 1.23s
729:	learn: 1.0000000	total: 5.24s	remaining: 1.22s
730:	learn: 1.0000000	total: 5.25s	remaining: 1.21s
731:	learn: 1.0000000	total: 5.26s	remaining: 1.21s
732:	learn: 1.0000000	total: 5.26s	remaining: 1.2s
733:	learn: 1.0000000	total: 5.27s	remaining: 1.19s
734:	learn: 1.0000000	total: 5.28s	remaining: 1.18s
735:	learn: 1.0000000	total: 5.28s	remaining: 1.18s
736:	learn: 1.0000000	total: 5.29s	remaining: 1.17s
737:	learn: 1.0000000	total: 5.3s	remaining: 1.16s
738:	learn: 1.0000000	total: 5.3s	remaining: 1.16s
739:	learn: 1.0000000	total: 5.31s	remaining: 1.15s
740:	learn: 1.0000000	total: 5.32s	remaining: 1.14s
741:	learn: 1.0000000	total: 5.33s	remaining: 1.13s
742:	learn: 1.0000000	total: 5.33s	remaining: 1.13s
743:	learn: 1.0000000	total: 5.34s	remaining: 1.12s
744:	learn: 1.0000000	total: 5.34s	remaining: 1.11s
745:	learn: 1.0000000	total: 5.35s	remaining: 1.1s
746:	learn: 1.0000000	total: 5.36s	remaining: 1.1s
747:	learn: 1.0000000	total: 5.37s	remaining: 1.09s
748:	learn: 1.0000000	total: 5.38s	remaining: 1.08s
749:	learn: 1.0000000	total: 5.39s	remaining: 1.08s
750:	learn: 1.0000000	total: 5.39s	remaining: 1.07s
751:	learn: 1.0000000	total: 5.4s	remaining: 1.06s
752:	learn: 1.0000000	total: 5.41s	remaining: 1.05s
753:	learn: 1.0000000	total: 5.41s	remaining: 1.05s
754:	learn: 1.0000000	total: 5.42s	remaining: 1.04s
755:	learn: 1.0000000	total: 5.42s	remaining: 1.03s
756:	learn: 1.0000000	total: 5.43s	remaining: 1.02s
757:	learn: 1.0000000	total: 5.44s	remaining: 1.02s
758:	learn: 1.0000000	total: 5.44s	remaining: 1.01s
759:	learn: 1.0000000	total: 5.45s	remaining: 1s
760:	learn: 1.0000000	total: 5.46s	remaining: 997ms
761:	learn: 1.0000000	total: 5.46s	remaining: 990ms
762:	learn: 1.0000000	total: 5.47s	remaining: 983ms
763:	learn: 1.0000000	total: 5.48s	remaining: 976ms
764:	learn: 1.0000000	total: 5.49s	remaining: 968ms
765:	learn: 1.0000000	total: 5.49s	remaining: 961ms
766:	learn: 1.0000000	total: 5.5s	remaining: 954ms
767:	learn: 1.0000000	total: 5.51s	remaining: 947ms
768:	learn: 1.0000000	total: 5.52s	remaining: 940ms
769:	learn: 1.0000000	total: 5.53s	remaining: 933ms
770:	learn: 1.0000000	total: 5.53s	remaining: 926ms
771:	learn: 1.0000000	total: 5.54s	remaining: 919ms
772:	learn: 1.0000000	total: 5.55s	remaining: 912ms
773:	learn: 1.0000000	total: 5.56s	remaining: 905ms
774:	learn: 1.0000000	total: 5.57s	remaining: 898ms
775:	learn: 1.0000000	total: 5.58s	remaining: 892ms
776:	learn: 1.0000000	total: 5.59s	remaining: 884ms
777:	learn: 1.0000000	total: 5.59s	remaining: 877ms
778:	learn: 1.0000000	total: 5.6s	remaining: 870ms
779:	learn: 1.0000000	total: 5.61s	remaining: 863ms
780:	learn: 1.0000000	total: 5.62s	remaining: 856ms
781:	learn: 1.0000000	total: 5.63s	remaining: 849ms
782:	learn: 1.0000000	total: 5.63s	remaining: 842ms
783:	learn: 1.0000000	total: 5.64s	remaining: 835ms
784:	learn: 1.0000000	total: 5.65s	remaining: 827ms
785:	learn: 1.0000000	total: 5.66s	remaining: 820ms
786:	learn: 1.0000000	total: 5.66s	remaining: 813ms
787:	learn: 1.0000000	total: 5.67s	remaining: 805ms
788:	learn: 1.0000000	total: 5.67s	remaining: 798ms
789:	learn: 1.0000000	total: 5.68s	remaining: 791ms
790:	learn: 1.0000000	total: 5.68s	remaining: 783ms
791:	learn: 1.0000000	total: 5.69s	remaining: 776ms
792:	learn: 1.0000000	total: 5.7s	remaining: 769ms
793:	learn: 1.0000000	total: 5.7s	remaining: 761ms
794:	learn: 1.0000000	total: 5.71s	remaining: 754ms
795:	learn: 1.0000000	total: 5.72s	remaining: 747ms
796:	learn: 1.0000000	total: 5.73s	remaining: 740ms
797:	learn: 1.0000000	total: 5.74s	remaining: 733ms
798:	learn: 1.0000000	total: 5.74s	remaining: 726ms
799:	learn: 1.0000000	total: 5.75s	remaining: 719ms
800:	learn: 1.0000000	total: 5.76s	remaining: 712ms
801:	learn: 1.0000000	total: 5.77s	remaining: 705ms
802:	learn: 1.0000000	total: 5.78s	remaining: 698ms
803:	learn: 1.0000000	total: 5.79s	remaining: 691ms
804:	learn: 1.0000000	total: 5.79s	remaining: 684ms
805:	learn: 1.0000000	total: 5.8s	remaining: 677ms
806:	learn: 1.0000000	total: 5.81s	remaining: 670ms
807:	learn: 1.0000000	total: 5.82s	remaining: 663ms
808:	learn: 1.0000000	total: 5.83s	remaining: 656ms
809:	learn: 1.0000000	total: 5.83s	remaining: 648ms
810:	learn: 1.0000000	total: 5.84s	remaining: 641ms
811:	learn: 1.0000000	total: 5.85s	remaining: 634ms
812:	learn: 1.0000000	total: 5.86s	remaining: 627ms
813:	learn: 1.0000000	total: 5.86s	remaining: 620ms
814:	learn: 1.0000000	total: 5.87s	remaining: 612ms
815:	learn: 1.0000000	total: 5.88s	remaining: 605ms
816:	learn: 1.0000000	total: 5.88s	remaining: 598ms
817:	learn: 1.0000000	total: 5.89s	remaining: 590ms
818:	learn: 1.0000000	total: 5.89s	remaining: 583ms
819:	learn: 1.0000000	total: 5.9s	remaining: 576ms
820:	learn: 1.0000000	total: 5.91s	remaining: 569ms
821:	learn: 1.0000000	total: 5.92s	remaining: 561ms
822:	learn: 1.0000000	total: 5.92s	remaining: 554ms
823:	learn: 1.0000000	total: 5.93s	remaining: 547ms
824:	learn: 1.0000000	total: 5.94s	remaining: 540ms
825:	learn: 1.0000000	total: 5.95s	remaining: 533ms
826:	learn: 1.0000000	total: 5.95s	remaining: 525ms
827:	learn: 1.0000000	total: 5.96s	remaining: 518ms
828:	learn: 1.0000000	total: 5.97s	remaining: 511ms
829:	learn: 1.0000000	total: 5.98s	remaining: 504ms
830:	learn: 1.0000000	total: 5.99s	remaining: 497ms
831:	learn: 1.0000000	total: 6s	remaining: 490ms
832:	learn: 1.0000000	total: 6s	remaining: 483ms
833:	learn: 1.0000000	total: 6.01s	remaining: 476ms
834:	learn: 1.0000000	total: 6.01s	remaining: 468ms
835:	learn: 1.0000000	total: 6.02s	remaining: 461ms
836:	learn: 1.0000000	total: 6.03s	remaining: 454ms
837:	learn: 1.0000000	total: 6.04s	remaining: 447ms
838:	learn: 1.0000000	total: 6.05s	remaining: 440ms
839:	learn: 1.0000000	total: 6.05s	remaining: 432ms
840:	learn: 1.0000000	total: 6.06s	remaining: 425ms
841:	learn: 1.0000000	total: 6.07s	remaining: 418ms
842:	learn: 1.0000000	total: 6.08s	remaining: 411ms
843:	learn: 1.0000000	total: 6.08s	remaining: 404ms
844:	learn: 1.0000000	total: 6.09s	remaining: 397ms
845:	learn: 1.0000000	total: 6.1s	remaining: 389ms
846:	learn: 1.0000000	total: 6.11s	remaining: 382ms
847:	learn: 1.0000000	total: 6.11s	remaining: 375ms
848:	learn: 1.0000000	total: 6.12s	remaining: 368ms
849:	learn: 1.0000000	total: 6.13s	remaining: 361ms
850:	learn: 1.0000000	total: 6.14s	remaining: 353ms
851:	learn: 1.0000000	total: 6.14s	remaining: 346ms
852:	learn: 1.0000000	total: 6.15s	remaining: 339ms
853:	learn: 1.0000000	total: 6.17s	remaining: 332ms
854:	learn: 1.0000000	total: 6.17s	remaining: 325ms
855:	learn: 1.0000000	total: 6.18s	remaining: 318ms
856:	learn: 1.0000000	total: 6.19s	remaining: 310ms
857:	learn: 1.0000000	total: 6.19s	remaining: 303ms
858:	learn: 1.0000000	total: 6.2s	remaining: 296ms
859:	learn: 1.0000000	total: 6.21s	remaining: 289ms
860:	learn: 1.0000000	total: 6.22s	remaining: 282ms
861:	learn: 1.0000000	total: 6.22s	remaining: 274ms
862:	learn: 1.0000000	total: 6.23s	remaining: 267ms
863:	learn: 1.0000000	total: 6.24s	remaining: 260ms
864:	learn: 1.0000000	total: 6.24s	remaining: 253ms
865:	learn: 1.0000000	total: 6.25s	remaining: 245ms
866:	learn: 1.0000000	total: 6.25s	remaining: 238ms
867:	learn: 1.0000000	total: 6.26s	remaining: 231ms
868:	learn: 1.0000000	total: 6.27s	remaining: 224ms
869:	learn: 1.0000000	total: 6.27s	remaining: 216ms
870:	learn: 1.0000000	total: 6.28s	remaining: 209ms
871:	learn: 1.0000000	total: 6.29s	remaining: 202ms
872:	learn: 1.0000000	total: 6.3s	remaining: 195ms
873:	learn: 1.0000000	total: 6.3s	remaining: 188ms
874:	learn: 1.0000000	total: 6.31s	remaining: 180ms
875:	learn: 1.0000000	total: 6.32s	remaining: 173ms
876:	learn: 1.0000000	total: 6.33s	remaining: 166ms
877:	learn: 1.0000000	total: 6.33s	remaining: 159ms
878:	learn: 1.0000000	total: 6.34s	remaining: 151ms
879:	learn: 1.0000000	total: 6.35s	remaining: 144ms
880:	learn: 1.0000000	total: 6.36s	remaining: 137ms
881:	learn: 1.0000000	total: 6.36s	remaining: 130ms
882:	learn: 1.0000000	total: 6.37s	remaining: 123ms
883:	learn: 1.0000000	total: 6.38s	remaining: 116ms
884:	learn: 1.0000000	total: 6.39s	remaining: 108ms
885:	learn: 1.0000000	total: 6.4s	remaining: 101ms
886:	learn: 1.0000000	total: 6.4s	remaining: 93.9ms
887:	learn: 1.0000000	total: 6.41s	remaining: 86.6ms
888:	learn: 1.0000000	total: 6.42s	remaining: 79.4ms
889:	learn: 1.0000000	total: 6.42s	remaining: 72.2ms
890:	learn: 1.0000000	total: 6.43s	remaining: 64.9ms
891:	learn: 1.0000000	total: 6.44s	remaining: 57.7ms
892:	learn: 1.0000000	total: 6.45s	remaining: 50.5ms
893:	learn: 1.0000000	total: 6.45s	remaining: 43.3ms
894:	learn: 1.0000000	total: 6.46s	remaining: 36.1ms
895:	learn: 1.0000000	total: 6.46s	remaining: 28.9ms
896:	learn: 1.0000000	total: 6.47s	remaining: 21.7ms
897:	learn: 1.0000000	total: 6.48s	remaining: 14.4ms
898:	learn: 1.0000000	total: 6.49s	remaining: 7.22ms
899:	learn: 1.0000000	total: 6.5s	remaining: 0us

GridSearchCV(cv=2,
         estimator=<catboost.core.CatBoostClassifier object at 0x00000164A0B7E3D0>,
         n_jobs=-1,
         param_grid={'border_count': [12], 'min_data_in_leaf': [100]},
         scoring='accuracy', verbose=1)
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. 
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

step-04 确认最优参数

print(clf.best_params_)

{'border_count': 12, 'min_data_in_leaf': 100}

step-05 选取最优模型

model=clf.best_estimator_

step-06 评价最优模型

model_eval(model, train, test)

train_roc_auc_score: 0.9999284349660532
test_roc_auc_score: 0.7438226744186047
train_accuracy_score: 0.9985714285714286
test_accuracy_score: 0.7016666666666667
train_precision_score: 0.9974715549936789
test__precision_score: 0.7298050139275766
train_recall_score: 1.0
test_recall_score: 0.7616279069767442
train_f1_score: 0.9987341772151899
test_f1_score: 0.7453769559032717

step-07 用最优模型预测

model.predict(train)

array([1, 1, 1, ..., 1, 1, 1], dtype=int64)

model.predict_proba(train)

array([[0.00437277, 0.99562723],
       [0.00522084, 0.99477916],
       [0.02550793, 0.97449207],
       ...,
       [0.0098453 , 0.9901547 ],
       [0.09563274, 0.90436726],
       [0.00210313, 0.99789687]])

step-08 保存并调用模型

joblib.dump(clf , r'D:\Ensemble_Learning\catboost_info\catboostgrid.model')
load_model=joblib.load(r'D:\Ensemble_Learning\catboost_info\catboostgrid.model')
load_model.predict( test.values )
load_model.predict_proba(test.values )

array([[0.9088261 , 0.0911739 ],
       [0.25607168, 0.74392832],
       [0.0904113 , 0.9095887 ],
       ...,
       [0.00316616, 0.99683384],
       [0.76160727, 0.23839273],
       [0.97218793, 0.02781207]])

你可能感兴趣的:(python机器学习,集成学习)

Python实现简单的机器学习算法 master_chenchengg python python 办公效率 python开发 IT
Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl
Python前沿技术：机器学习与人工智能 4.0啊 Python 人工智能 python 机器学习
Python前沿技术：机器学习与人工智能一、引言随着科技的飞速发展，机器学习和人工智能（AI）已经成为了计算机科学领域的热门话题。Python作为一门易学易用且功能强大的编程语言，已经成为了这两个领域的首选语言之一。本文将深入探讨Python在机器学习和人工智能领域的应用，以及一些前沿技术和工具。二、Python机器学习基础2.1机器学习概述机器学习是人工智能（AI）的一个关键子集，它的核心在于让
梯度提升机 (Gradient Boosting Machines, GBM) ALGORITHM LOL boosting 集成学习机器学习
梯度提升机(GradientBoostingMachines,GBM)通俗易懂算法梯度提升机（GradientBoostingMachines，GBM）是一种集成学习算法，主要用于回归和分类问题。GBM本质上是通过训练一系列简单的模型（通常是决策树），然后将这些模型组合起来，从而提高整体预测性能。基本步骤初始模型：首先，我们用一个简单的模型（如一个常数值）作为预测模型，记为F0(x)F_0(x)F
Python自动化办公2.0 即将发布百里图书自动化人工智能 python
第一节课：数据整理与清洗第二节课：数据筛选、过滤与排序第三节课：高级数据处理技巧第四节课：数据可视化与实践案例第五节课：统计分析与报表第六节：常见的Excel报表与下方的课程形成知识体系：Python自动化办公(面向2020,Python3.7,不断更新ing)_在线视频教程-CSDN程序员研修院https://edu.csdn.net/course/detail/28031Python机器学习教
【Python机器学习】循环神经网络（RNN）——传递数据并训练 zhangbin_237 Python机器学习机器学习 python rnn 人工智能开发语言深度学习神经网络
与其他Keras模型一样，我们需要向.fit()方法传递数据，并告诉它我们希望训练多少个训练周期（epoch）：model.fit(X_train,y_train,batch_size=batch_size,epochs=epochs,validation_data=(X_test,y_test))因为个人小电脑内存不足，所以吧maxlen参数改成了100重新运行。保存模型：model_struc
【Python机器学习】循环神经网络（RNN）——对RNN进行预测 zhangbin_237 Python机器学习机器学习 python rnn 深度学习人工智能自然语言处理
目录有状态性双向RNN编码向量如果有一个经过训练的模型，接下来就可以对其进行预测：sample_1="""Ihatethatthedismalweatherhadmedownforsolong,whenwillitbreak!Ugh,whendoeshappinessreturn?Thesunisblindingandthepuffycloudsaretoothin.Ican'twaitforth
《机器学习》—— XGBoost（xgb.XGBClassifier）分类器张小生180 机器学习人工智能
文章目录一、XGBoost分类器的介绍二、XGBoost（xgb.XGBClassifier）分类器与随机森林分类器（RandomForestClassifier）的区别三、XGBoost（xgb.XGBClassifier）分类器代码使用示例一、XGBoost分类器的介绍XGBoost分类器是一种基于梯度提升决策树（GradientBoostingDecisionTree，GBDT）的集成学习算
Python 机器学习基础之数据表示与特征工程【分箱、离散化、线性模型与树 / 交互特征与多项式特征】的简单说明仙魁XAN Python 机器学习基础+实战案例机器学习 python 分箱离散化线性模型与树交互特征与多项式特征
Python机器学习基础之数据表示与特征工程【分箱、离散化、线性模型与树/交互特征与多项式特征】的简单说明目录Python机器学习基础之数据表示与特征工程【分箱、离散化、线性模型与树/交互特征与多项式特征】的简单说明一、简单介绍二、分箱、离散化、线性模型与树三、交互特征与多项式特征附录一、参考文献一、简单介绍Python是一种跨平台的计算机程序设计语言。是一种面向对象的动态类型语言，最初被设计用于
2024 数学建模国赛 C 题模型及算法（无废话版）不染53 数学建模数学建模算法 python
目录写在开始需要掌握的数学模型/算法评价体系/评价类问题时间序列处理数据降维聚类问题（无监督）分类问题（有监督）集成学习（Bagging/Boosting）回归问题关联分析统计学方法/统计模型智能优化算法需要掌握的Python专业库需要掌握的软件/工具写在开始本人获2023年数学建模国赛C题国家级一等奖，备赛期间专攻C题。本文总结了在备赛期间总结的模型和算法，足以应对90%国赛C题中涉及到的问题。
【Python机器学习】机器学习任务中常见的数据异质问题和模型异构问题是什么？解决策略是什么？惊鸿若梦一书生 Python机器学习 python 深度学习开发语言
文章目录数据异质模型异构数据异质数据异质问题（Heterogeneityindata）通常指数据集内部的不一致性，这些不一致性可能来自多种源。在实际应用中，数据异质性可以表现为多种形式，包括：不同来源的数据：数据可能来自不同的数据源，每个源可能采用不同的数据收集方法和标准。例如，社交媒体数据和传统调查数据就可能有很大的差异。不同类型的数据：数据可以是结构化的（例如，数据库中的表格数据），半结构化的
【Python机器学习】卷积神经网络（CNN）的工具包 zhangbin_237 Python机器学习机器学习 python cnn 神经网络自然语言处理开发语言
Python是神经网络工具包最丰富的语言之一。两个主要的神经网络架构分别是Theano和TensorFlow。这两者的底层计算深度依赖C语言，不过它们都提供了强大的PythonAPI。Torch在Python里面也有一个对应的API是PyTorch。这些框架都是高度抽象的工具集，适用于从头构建模型。Python社区开发了一些第三方库来简化这些底层架构的使用。其中Keras在API的友好性和功能性方
【Python机器学习】卷积神经网络（CNN） zhangbin_237 Python机器学习机器学习 python cnn 开发语言自然语言处理
卷积神经网络（CNN）得名于在数据样本上用滑动窗口（或卷积）的概念。卷积在数学中应用很广泛，通常与时间序列数据相关。它是用一个可视化盒子在一个区域内滑动，如下图所示：构建块卷积神经网络最早出现在图像处理和图像识别领域，它能够捕捉每个样本中数据点之间的空间关系，也就能识别出图像中是猫还是狗。卷积网络，也称为convnet，不像传统的前馈网络那样对每个元素（图中的像素）分配权重，而是定义了一组在图像上
python机器学习算法--贝叶斯算法在下小天n 机器学习 python 机器学习算法
1.贝叶斯定理在20世纪60年代初就引入到文字信息检索中，仍然是文字分类的一种热门（基准）方法。文字分类是以词频为特征判断文件所属类型或其他（如垃圾邮件、合法性、新闻分类等）的问题。原理牵涉到概率论的问题，不在详细说明。sklearn.naive_bayes.GaussianNB(priors=None,var_smoothing=1e-09)#Bayes函数·priors：矩阵，shape=[n
Spark MLlib模型训练—回归算法 Random forest regression 不二人生 Spark ML 实战 spark-ml 回归随机森林
SparkMLlib模型训练—回归算法Randomforestregression随机森林回归(RandomForestRegression)是一种集成学习方法，通过结合多个决策树的预测结果来提升模型的准确性和稳健性。相较于单一的决策树模型，随机森林通过随机采样和多棵树的集成，减少了模型的方差，从而在处理复杂数据集时展现出更好的性能。本文将详细介绍随机森林回归的原理、实现方法、应用场景，并通过Sc
【Rust】——采用发布配置自定义构建 Y小夜 Rust（官方文档重点总结）rust 开发语言后端
博主现有专栏：C51单片机（STC89C516），c语言，c++，离散数学，算法设计与分析，数据结构，Python，Java基础，MySQL，linux，基于HTML5的网页设计及应用，Rust（官方文档重点总结），jQuery，前端vue.js，Javaweb开发，Python机器学习等主页链接：Y小夜-CSDN博客今日学习推荐：在当今这个飞速发展的信息时代，人工智能（AI）已经成为了一个不可或
【Rust】——高级类型 Y小夜 Rust（官方文档重点总结）rust 开发语言后端
博主现有专栏：C51单片机（STC89C516），c语言，c++，离散数学，算法设计与分析，数据结构，Python，Java基础，MySQL，linux，基于HTML5的网页设计及应用，Rust（官方文档重点总结），jQuery，前端vue.js，Javaweb开发，Python机器学习等主页链接：Y小夜-CSDN博客目录为了类型安全和抽象而使用的newtype模式类型别名用来创建类型同义词不返回
基于Python的机器学习系列（18）：梯度提升分类（Gradient Boosting Classification）会飞的Anthony 信息系统机器学习人工智能机器学习 python 分类
简介梯度提升（GradientBoosting）是一种集成学习方法，通过逐步添加新的预测器来改进模型。在回归问题中，我们使用梯度来最小化残差。在分类问题中，我们可以利用梯度提升来进行二分类或多分类任务。与回归不同，分类问题需要使用如softmax这样的概率模型来处理类别标签。梯度提升分类的工作原理梯度提升分类的基本步骤与回归类似，但在分类任务中，我们使用概率模型来处理预测结果：初始化模型：选择一个
基于Python的机器学习系列（17）：梯度提升回归（Gradient Boosting Regression）会飞的Anthony 人工智能信息系统机器学习机器学习 python 回归
简介梯度提升（GradientBoosting）是一种强大的集成学习方法，类似于AdaBoost，但与其不同的是，梯度提升通过在每一步添加新的预测器来减少前一步预测器的残差。这种方法通过逐步改进模型，能够有效提高预测准确性。梯度提升回归的工作原理在梯度提升回归中，我们逐步添加预测器来修正模型的残差。以下是梯度提升的基本步骤：初始化模型：选择一个初始预测器h0(x)，计算该预测器的预测值。计算残差：
基于CNN-BiLSTM-Adaboost风电功率预测研究（Matlab代码实现）创新优化代码学习 cnn matlab 人工智能
欢迎来到本博客❤️❤️博主优势：博客内容尽量做到思维缜密，逻辑清晰，为了方便读者。⛳️座右铭：行百里者，半于九十。本文目录如下：目录⛳️赠与读者1概述一、研究背景与意义二、研究方法1.数据准备与预处理2.CNN特征提取3.BiLSTM序列建模4.Adaboost集成学习5.模型训练与评估三、研究优势四、未来展望2运行结果3参考文献4Matlab代码、数据⛳️赠与读者‍做科研，涉及到一个深在的思想系
【Python机器学习】NLP词频背后的含义——隐性语义分析 zhangbin_237 Python机器学习 python 机器学习自然语言处理人工智能开发语言
隐性语义分析基于最古老和最常用的降维技术——奇异值分解（SVD）。SVD将一个矩阵分解成3个方阵，其中一个是对角矩阵。SVD的一个应用是求逆矩阵。一个矩阵可以分解成3个最简单的方阵，然后对这些方阵求转置后再把它们相乘，就得到了原始矩阵的逆矩阵。它为我们提供了一个对大型复杂矩阵求逆的捷径。SVD适用于桁架结构的应力和应变分析等机械工程问题，它对电气工程中的电路分析也很有用，它甚至在数据科学中被用于基
机器学习基础（四）——决策树与随机森林 Bayesian小孙机器学习基础决策树机器学习随机森林
决策树与随机森林文章目录决策树与随机森林一、知识概要（一）二、决策树使用的算法三、sklearn决策树API四、决策树的案例1.数据清洗2.特征工程3.调用决策树API五、集成学习方法-随机森林1.知识概要（二）2.集成学习API3.随机森林的案例importpandasaspdfromsklearn.feature_extractionimportDictVectorizerfromsklear
机器学习之决策树与随机森林的实现 SEVEN-YEARS 机器学习决策树随机森林
引言随着互联网技术的发展，垃圾邮件过滤已成为一项重要的任务。机器学习技术，尤其是决策树和随机森林，在解决这类问题时表现出色。本文将介绍随机森林的基本概念，并通过一个具体的案例——筛选垃圾电子邮件——来展示随机森林的实际应用。随机森林简介随机森林是一种基于决策树的集成学习方法，它通过构建多个决策树并综合它们的预测结果来提高准确性和防止过拟合。随机森林的工作原理主要包括以下几个步骤：自助采样：从原始数
每天一个数据分析题（五百零五）- 提升方法跟着紫枫学姐学CDA 数据分析题库数据分析
提升方法（Boosting），是一种可以用来减小监督式学习中偏差的机器学习算法。基于Boosting的集成学习，其代表算法不包括？A.AdaboostB.GBDTC.XGBOOSTD.随机森林数据分析认证考试介绍：点击进入题目来源于CDA模拟题库点击此处获取答案数据分析专项练习题库内容涵盖Python，SQL，统计学，数据分析理论，深度学习，可视化，机器学习，Spark八个方向的专项练习题库，数据
【Python机器学习】NLP分词——利用分词器构建词汇表（三）——度量词袋之间的重合度 zhangbin_237 Python机器学习机器学习自然语言处理人工智能 python 开发语言
如果能够度量两个向量词袋之间的重合度，就可以很好地估计他们所用词的相似程度，而这也是它们语义上重合度的一个很好的估计。因此，下面用点积来估计一些新句子和原始的Jefferson句子之间的词袋向量重合度：importpandasaspdsentence="""ThomasJeffersonBeganbulidingMonticelliastheageof26.\n"""sentence=senten
【Python机器学习】NLP概述——深度处理 zhangbin_237 Python机器学习 python 机器学习自然语言处理人工智能机器人
自然语言处理流水线的各个阶段可以看作是层，就像是前馈神经网络中的层一样。深度学习就是通过在传统的两层机器学习模型架构（特征提取+建模）中添加额外的处理层来创建更复杂的模型和行为。上图中，前四层对应于聊天机器人流水线中的前两个阶段（特征提取和特征分析）。例如，词性标注（POS标注）是在聊天机器人流水线的分析阶段生成特征的一种方法。POS标签由默认的SpaCY流水线自动生成，该流水线包括上图中所有的前
【Python机器学习】NLP分词——词干还原的挑战 zhangbin_237 Python机器学习自然语言处理人工智能机器学习 python 开发语言
要想使用自然语言处理的相关应用，第一件事就是需要一个强大的词汇表。我们要把文档或任何字符串拆分为离散的有意义的词条，这里说的词条仅限于词、标点符号和数值，但是这里使用的技术可以很容易推广到字符序列包含的任何其他有意义的单元，比如ASCII表情符号、Unicode表情符号和数学符号。从文档中检索词条需要一些字符串处理方法，这些方法不仅仅是str.split()，处理时需要把标点符号与词分开，还需要将
每天一个数据分析题（二百二十）跟着紫枫学姐学CDA 数据分析题库数据分析数据挖掘
在集成学习的GBDT算法中，每次训练新的决策树的目的是()？A.预测原始数据的标签B.预测上一个模型的残差C.降低模型的偏差D.降低模型的方差题目来源于CDA模拟题库点击此处获取答案
【Python机器学习】NLP概述——自然语言智商 zhangbin_237 Python机器学习机器学习自然语言处理人工智能 python 机器人
就像人类的智能一样，如果不考虑多个智能维度，单凭一个智商分数是无法轻易衡量NLP流水线的能力的。衡量机器人系统能力的一种常见方法是：根据系统行为的复杂性和所需的人类监督成都这两个维度来衡量。但是对自然语言处理流水线而言，其目标是建立一个完全自动化的自然语言处理系统，会消除所有的人工监督（一旦模型被训练和部署）。因此，一对更好的IQ维度应该能捕捉到自然语言流水线复杂的广度和深度。像Alexa或All
【Python机器学习】NLP概述——聊天机器人的自然语言流水线 zhangbin_237 Python机器学习自然语言处理机器人人工智能 python 机器学习
构建对话引擎或者聊天机器人所需的NLP流水线类似于某些问答系统。聊天机器人需要4个处理阶段和一个数据库来维护过去语句和回复的记录。这4个处理阶段中的每个阶段都可以包含一个或多个并行或串行工作的处理算法。如下图所示：1、解析：从自然语言文本中提取特征、结构化数值数；2、分析：通过对文本的情感、语法合法度及语义打分，生成和组合特征；3、生成：使用模板、搜索或语言模型生成可能的回复；4、执行：根据对话历
【深度学习】吴恩达-课后作业-搭建多层神经网络以及应用 —Xi— 深度学习深度学习机器学习人工智能 python 神经网络
Ng的深度学习，其实前几个月就听完了，课后作业也是大懂不懂的都做了一遍，代码也跟着各种各样的参考敲了一遍，但暑假几个月没怎么学习。。。基本也忘得差不多了，这几周回顾了一下深度学习这门课的笔记，看了别的博主的总结，对CNN，RNN,LSTM,注意力机制等网络结构进行了复盘，虽然感觉自己很心浮气躁，一边也在学集成学习那几个算法和推荐系统相关，这里也告诉自己：贪多嚼不烂，心急吃不了热豆腐，慢慢来，还是要
HQL之投影查询归来朝歌 HQL Hibernate 查询语句投影查询
在HQL查询中，常常面临这样一个场景，对于多表查询，是要将一个表的对象查出来还是要只需要每个表中的几个字段，最后放在一起显示？针对上面的场景，如果需要将一个对象查出来： HQL语句写“from 对象”即可 Session session = HibernateUtil.openSession();
Spring整合redis bylijinnan redis
pom.xml <dependencies>  <dependency> <groupId>org.springframework.data</groupId> <artifactId>spring-data-redi
org.hibernate.NonUniqueResultException: query did not return a unique result: 2 0624chenhong Hibernate
参考：http://blog.csdn.net/qingfeilee/article/details/7052736 org.hibernate.NonUniqueResultException: query did not return a unique result: 2 在项目中出现了org.hiber
android动画效果不懂事的小屁孩 android动画
前几天弄alertdialog和popupwindow的时候，用到了android的动画效果，今天专门研究了一下关于android的动画效果，列出来，方便以后使用。 Android 平台提供了两类动画。一类是Tween动画，就是对场景里的对象不断的进行图像变化来产生动画效果（旋转、平移、放缩和渐变）。第二类就是 Frame动画，即顺序的播放事先做好的图像，与gif图片原理类似。
js delete 删除机理以及它的内存泄露问题的解决方案换个号韩国红果果 JavaScript
delete删除属性时只是解除了属性与对象的绑定，故当属性值为一个对象时，删除时会造成内存泄露（其实还未删除）举例： var person={name:{firstname:'bob'}} var p=person.name delete person.name p.firstname -->'bob' // 依然可以访问p.firstname，存在内存泄露
Oracle将零干预分析加入网络即服务计划蓝儿唯美 oracle
由Oracle通信技术部门主导的演示项目并没有在本月较早前法国南斯举行的行业集团TM论坛大会中获得嘉奖。但是，Oracle通信官员解雇致力于打造一个支持零干预分配和编制功能的网络即服务（NaaS）平台，帮助企业以更灵活和更适合云的方式实现通信服务提供商（CSP）的连接产品。这个Oracle主导的项目属于TM Forum Live!活动上展示的Catalyst计划的19个项目之一。Catalyst计
spring学习——springmvc（二） a-john springMVC
Spring MVC提供了非常方便的文件上传功能。 1，配置Spring支持文件上传： DispatcherServlet本身并不知道如何处理multipart的表单数据，需要一个multipart解析器把POST请求的multipart数据中抽取出来，这样DispatcherServlet就能将其传递给我们的控制器了。为了在Spring中注册multipart解析器，需要声明一个实现了Mul
POJ-2828-Buy Tickets aijuans ACM_POJ
POJ-2828-Buy Tickets http://poj.org/problem?id=2828 线段树，逆序插入 #include<iostream>#include<cstdio>#include<cstring>#include<cstdlib>using namespace std;#define N 200010struct
Java Ant build.xml详解 asia007 build.xml
1,什么是antant是构建工具2,什么是构建概念到处可查到，形象来说，你要把代码从某个地方拿来，编译，再拷贝到某个地方去等等操作，当然不仅与此，但是主要用来干这个3,ant的好处跨平台 --因为ant是使用java实现的，所以它跨平台使用简单--与ant的兄弟make比起来语法清晰--同样是和make相比功能强大--ant能做的事情很多，可能你用了很久，你仍然不知道它能有
android按钮监听器的四种技术百合不是茶 android xml配置监听器实现接口
android开发中经常会用到各种各样的监听器,android监听器的写法与java又有不同的地方; 1,activity中使用内部类实现接口 ,创建内部类实例使用add方法与java类似创建监听器的实例 myLis lis = new myLis(); 使用add方法给按钮添加监听器
软件架构师不等同于资深程序员 bijian1013 程序员架构师架构设计
本文的作者Armel Nene是ETAPIX Global公司的首席架构师，他居住在伦敦，他参与过的开源项目包括 Apache Lucene,，Apache Nutch， Liferay 和 Pentaho等。如今很多的公司
TeamForge Wiki Syntax & CollabNet User Information Center sunjing TeamForge How do Attachement Anchor Wiki Syntax
the CollabNet user information center http://help.collab.net/ How do I create a new Wiki page? A CollabNet TeamForge project can have any number of Wiki pages. All Wiki pages are linked, and
【Redis四】Redis数据类型 bit1129 redis
概述 Redis是一个高性能的数据结构服务器，称之为数据结构服务器的原因是，它提供了丰富的数据类型以满足不同的应用场景，本文对Redis的数据类型以及对这些类型可能的操作进行总结。 Redis常用的数据类型包括string、set、list、hash以及sorted set.Redis本身是K/V系统，这里的数据类型指的是value的类型，而不是key的类型，key的类型只有一种即string
SSH2整合-附源码白糖_ eclipse spring tomcat Hibernate Google
今天用eclipse终于整合出了struts2+hibernate+spring框架。我创建的是tomcat项目，需要有tomcat插件。导入项目以后，鼠标右键选择属性，然后再找到“tomcat”项，勾选一下“Is a tomcat project”即可。具体方法见源码里的jsp图片，sql也在源码里。补充1：项目中部分jar包不是最新版的，可能导
[转]开源项目代码的学习方法 braveCS 学习方法
转自： http://blog.sina.com.cn/s/blog_693458530100lk5m.html http://www.cnblogs.com/west-link/archive/2011/06/07/2074466.html 1）阅读features。以此来搞清楚该项目有哪些特性2）思考。想想如果自己来做有这些features的项目该如何构架3）下载并安装d
编程之美-子数组的最大和（二维） bylijinnan 编程之美
package beautyOfCoding; import java.util.Arrays; import java.util.Random; public class MaxSubArraySum2 { /** * 编程之美子数组之和的最大值（二维） */ private static final int ROW = 5; private stat
读书笔记-3 chengxuyuancsdn jquery笔记 resultMap配置 ibatis一对多配置
1、resultMap配置 2、ibatis一对多配置 3、jquery笔记 1、resultMap配置当<select resultMap="topic_data"> <resultMap id="topic_data">必须一一对应。 (1)<resultMap class="tblTopic&q
[物理与天文]物理学新进展 comsci
如果我们必须获得某种地球上没有的矿石,才能够进行某些能量输出装置的设计和建造,而要获得这种矿石,又必须首先进行深空探测,而要进行深空探测,又必须获得这种能量输出装置,这个矛盾的循环,会导致地球联盟在与宇宙文明建立关系的时候,陷入困境怎么办呢?
Oracle 11g新特性:Automatic Diagnostic Repository daizj oracle ADR
Oracle Database 11g的FDI（Fault Diagnosability Infrastructure）是自动化诊断方面的又一增强。 FDI的一个关键组件是自动诊断库（Automatic Diagnostic Repository-ADR）。在oracle 11g中，alert文件的信息是以xml的文件格式存在的，另外提供了普通文本格式的alert文件。这两份log文
简单排序:选择排序 dieslrae 选择排序
public void selectSort(int[] array){ int select; for(int i=0;i<array.length;i++){ select = i; for(int k=i+1;k<array.leng
C语言学习六指针的经典程序，互换两个数字 dcj3sjt126com c
示例程序，swap_1和swap_2都是错误的，推理从1开始推到2，2没完成，推到3就完成了 # include <stdio.h> void swap_1(int, int); void swap_2(int *, int *); void swap_3(int *, int *); int main(void) { int a = 3; int b =
php 5.4中php-fpm 的重启、终止操作命令 dcj3sjt126com PHP
php 5.4中php-fpm 的重启、终止操作命令: 查看php运行目录命令：which php/usr/bin/php 查看php-fpm进程数：ps aux | grep -c php-fpm 查看运行内存/usr/bin/php -i|grep mem 重启php-fpm/etc/init.d/php-fpm restart 在phpinfo()输出内容可以看到php
线程同步工具类 shuizhaosi888 同步工具类
同步工具类包括信号量（Semaphore）、栅栏（barrier）、闭锁（CountDownLatch）闭锁（CountDownLatch） public class RunMain { public long timeTasks(int nThreads, final Runnable task) throws InterruptedException { fin
bleeding edge是什么意思 haojinghua DI
不止一次，看到很多讲技术的文章里面出现过这个词语。今天终于弄懂了——通过朋友给的浏览软件，上了wiki。我再一次感到，没有辞典能像WiKi一样，给出这样体贴人心、一清二楚的解释了。为了表达我对WiKi的喜爱，只好在此一一中英对照，给大家上次课。 In computer science, bleeding edge is a term that
c中实现utf8和gbk的互转 jimmee c iconv utf8&gbk编码
#include <iconv.h> #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <fcntl.h> #include <string.h> #include <sys/stat.h> int code_c
大型分布式网站架构设计与实践 lilin530 应用服务器搜索引擎
1.大型网站软件系统的特点？ a.高并发，大流量。 b.高可用。 c.海量数据。 d.用户分布广泛，网络情况复杂。 e.安全环境恶劣。 f.需求快速变更，发布频繁。 g.渐进式发展。 2.大型网站架构演化发展历程？ a.初始阶段的网站架构。应用程序，数据库，文件等所有的资源都在一台服务器上。 b.应用服务器和数据服务器分离。 c.使用缓存改善网站性能。 d.使用应用
在代码中获取Android theme中的attr属性值 OliveExcel android theme
Android的Theme是由各种attr组合而成, 每个attr对应了这个属性的一个引用, 这个引用又可以是各种东西. 在某些情况下, 我们需要获取非自定义的主题下某个属性的内容 (比如拿到系统默认的配色colorAccent), 操作方式举例一则: int defaultColor = 0xFF000000; int[] attrsArray = { andorid.r.
基于Zookeeper的分布式共享锁 roadrunners zookeeper 分布式共享锁
首先，说说我们的场景，订单服务是做成集群的，当两个以上结点同时收到一个相同订单的创建指令，这时并发就产生了，系统就会重复创建订单。等等......场景。这时，分布式共享锁就闪亮登场了。共享锁在同一个进程中是很容易实现的，但在跨进程或者在不同Server之间就不好实现了。Zookeeper就很容易实现。具体的实现原理官网和其它网站也有翻译，这里就不在赘述了。官
两个容易被忽略的MySQL知识 tomcat_oracle mysql
1、varchar(5)可以存储多少个汉字，多少个字母数字？　　相信有好多人应该跟我一样，对这个已经很熟悉了，根据经验我们能很快的做出决定，比如说用varchar(200)去存储url等等，但是，即使你用了很多次也很熟悉了，也有可能对上面的问题做出错误的回答。　　这个问题我查了好多资料，有的人说是可以存储5个字符，2.5个汉字（每个汉字占用两个字节的话），有的人说这个要区分版本，5.0
zoj 3827 Information Entropy(水题) 阿尔萨斯 format
题目链接：zoj 3827 Information Entropy 题目大意：三种底，计算和。解题思路：调用库函数就可以直接算了，不过要注意Pi = 0的时候，不过它题目里居然也讲了。。。limp→0+plogb(p)=0，因为p是logp的高阶。 #include <cstdio> #include <cstring> #include <cmath&

集成学习02_catboost参数介绍与实战

一.catboost 模型参数介绍

二.catboost实现

三. 网格搜索最优catboost模型

1.通用参数

2.性能参数

3.默认参数

4.CatBoostClassifier方法参数

1.fit

2.predict

3.predict_proba

5.get_feature_importance

6.eval_metrics

7.save_model

8.load_model

9.get_params

10.score

二.catboost实现

1.导入相关包

2.数据处理

3.撰写评价函数

4.拟合catboost模型

5.评价模型

6.利用模型预测

注意：

7.保存模型并调用

三. 网格搜索最优catboost模型

1.step-01 配置参数列表

step-02 选择待优化模型

step-03 进行网格搜索 拟合模型

step-04 确认最优参数

step-05 选取最优模型

step-06 评价最优模型

step-07 用最优模型预测

step-08 保存并调用模型

你可能感兴趣的:(python机器学习,集成学习)

step-03 进行网格搜索拟合模型