旭旭_哥

lightgbm python基本使用 --mac下操作

好几天没有更新博客，最近指标压力大，没去摸索算法，今天写这个博客算是忙里偷闲吧，lightgbm的基本使用，python接口，这个工具微软开源的，号称比xgboost快，具体没怎么对比，先看看如何使用的.

安装编译，其中第一步和第二步以前安装xgboost时候安装过，可以直接跳过

brew install cmake
brew install gcc --without-multilib
git clone --recursive https://github.com/Microsoft/LightGBM ; cd LightGBM
mkdir build ; cd build
cmake ..
make -j

安装lightgbm python包

cd python-packages
sudo python3 setup.py install

代码训练,跟xgboost类似，数据是lightgbm自带的数据:

import json
import lightgbm as lgb
import pandas as pd
from sklearn.metrics import roc_auc_score
path="/Users/shuubiasahi/Documents/githup/LightGBM/examples/regression/"
print("load data")
df_train=pd.read_csv(path+"regression.train",header=None,sep='\t')
df_test=pd.read_csv(path+"regression.train",header=None,sep='\t')
y_train = df_train[0].values
y_test = df_test[0].values
X_train = df_train.drop(0, axis=1).values
X_test = df_test.drop(0, axis=1).values
# create dataset for lightgbm
lgb_train = lgb.Dataset(X_train, y_train)
lgb_eval = lgb.Dataset(X_test, y_test, reference=lgb_train)
# specify your configurations as a dict
params = {
'task': 'train',
'boosting_type': 'gbdt',
'objective': 'binary',
'metric': {'l2', 'auc'},
'num_leaves': 31,
'learning_rate': 0.05,
'feature_fraction': 0.9,
'bagging_fraction': 0.8,
'bagging_freq': 5,
'verbose': 0
}
print('Start training...')
# train
gbm = lgb.train(params,
lgb_train,
num_boost_round=20,
valid_sets=lgb_eval,
early_stopping_rounds=5)
print('Save model...')
# save model to file
gbm.save_model('lightgbm/model.txt')
print('Start predicting...')
# predict
y_pred = gbm.predict(X_test, num_iteration=gbm.best_iteration)
# eval
print(y_pred)
print('The roc of prediction is:', roc_auc_score(y_test, y_pred) )
print('Dump model to JSON...')
# dump model to json (and save to file)
model_json = gbm.dump_model()
with open('lightgbm/model.json', 'w+') as f:
json.dump(model_json, f, indent=4)
print('Feature names:', gbm.feature_name())
print('Calculate feature importances...')
# feature importances
print('Feature importances:', list(gbm.feature_importance()))

结果如下:

/anaconda/bin/python3 /Users/shuubiasahi/Documents/python/lgb.py

load data

Start training...

[1] valid_0's auc: 0.768095 valid_0's l2: 0.24445

Train until valid scores didn't improve in 5 rounds.

[2] valid_0's auc: 0.780194 valid_0's l2: 0.24024

[3] valid_0's auc: 0.793988 valid_0's l2: 0.236395

[4] valid_0's auc: 0.799031 valid_0's l2: 0.231919

[5] valid_0's auc: 0.799646 valid_0's l2: 0.227804

[6] valid_0's auc: 0.805888 valid_0's l2: 0.224428

[7] valid_0's auc: 0.810546 valid_0's l2: 0.221569

[8] valid_0's auc: 0.812721 valid_0's l2: 0.218367

[9] valid_0's auc: 0.814419 valid_0's l2: 0.215412

[10] valid_0's auc: 0.815477 valid_0's l2: 0.212688

[11] valid_0's auc: 0.816437 valid_0's l2: 0.209996

[12] valid_0's auc: 0.819061 valid_0's l2: 0.207497

[13] valid_0's auc: 0.819953 valid_0's l2: 0.205227

[14] valid_0's auc: 0.82138 valid_0's l2: 0.203252

[15] valid_0's auc: 0.822767 valid_0's l2: 0.201168

[16] valid_0's auc: 0.825256 valid_0's l2: 0.199154

[17] valid_0's auc: 0.82648 valid_0's l2: 0.197123

[18] valid_0's auc: 0.829691 valid_0's l2: 0.19538

[19] valid_0's auc: 0.830232 valid_0's l2: 0.193633

[20] valid_0's auc: 0.831021 valid_0's l2: 0.191994

Save model...

Start predicting...

[ 0.63631928 0.731689 0.73191171 ..., 0.26398355 0.43711743

0.45558644]

The roc of prediction is: 0.831021275808

Dump model to JSON...

Feature names: ['Column_0', 'Column_1', 'Column_2', 'Column_3', 'Column_4', 'Column_5', 'Column_6', 'Column_7', 'Column_8', 'Column_9', 'Column_10', 'Column_11', 'Column_12', 'Column_13', 'Column_14', 'Column_15', 'Column_16', 'Column_17', 'Column_18', 'Column_19', 'Column_20', 'Column_21', 'Column_22', 'Column_23', 'Column_24', 'Column_25', 'Column_26', 'Column_27']

Calculate feature importances...

Feature importances: [26, 10, 2, 34, 8, 53, 9, 0, 1, 31, 5, 6, 1, 27, 8, 4, 2, 7, 4, 7, 1, 24, 63, 3, 53, 90, 56, 65]

官方推荐用配置的形式训练模型,编译之后回生成lightgbm可执行文件，模型训练，下面也演示一下

先看看train.conf配置：

# 配置目标是用于训练
task = train
# 训练方式
boosting_type = gbdt
#目标二分类
objective = binary
# 损失函数
metric = binary_logloss,auc
# frequence for metric output
metric_freq = 1
# true if need output metric for training data, alias: tranining_metric, train_metric
is_training_metric = true
# 特征最大分割
max_bin = 255
#训练数据地址
data = /Users/shuubiasahi/Documents/githup/LightGBM/examples/binary_classification/binary.train
#测试数据
#valid_data = binary.test
# 树的棵树
num_trees = 100
# 学习率
learning_rate = 0.1
# number of leaves for one tree, alias: num_leaf
num_leaves = 63
tree_learner = serial
# 最大线程个数
# num_threads = 8
# feature sub-sample, will random select 80% feature to train on each iteration
# alias: sub_feature
feature_fraction = 0.8
# Support bagging (data sub-sample), will perform bagging every 5 iterations
bagging_freq = 5
# Bagging farction, will random select 80% data on bagging
# alias: sub_row
bagging_fraction = 0.8
# minimal number data for one leaf, use this to deal with over-fit
# alias : min_data_per_leaf, min_data
min_data_in_leaf = 50
# minial sum hessians for one leaf, use this to deal with over-fit
min_sum_hessian_in_leaf = 5.0
# save memory and faster speed for sparse feature, alias: is_sparse
is_enable_sparse = true
# when data is bigger than memory size, set this to true. otherwise set false will have faster speed
# alias: two_round_loading, two_round
use_two_round_loading = false
# true if need to save data to binary file and application will auto load data from binary file next time
# alias: is_save_binary, save_binary
is_save_binary_file = false
# 模型输出文件
output_model = /Users/shuubiasahi/Documents/githup/LightGBM/examples/binary_classification/LightGBM_model.txt
machine_list_file = /Users/shuubiasahi/Documents/githup/LightGBM/examples/binary_classification/LightGBM_model.txt/mlist.txt

模型训练:

./lightgbm config=train.conf

模型中间结果演示:

[LightGBM] [Info] Finished loading parameters

[LightGBM] [Info] Loading weights...

[LightGBM] [Info] Finished loading data in 0.036442 seconds

[LightGBM] [Info] Number of positive: 3716, number of negative: 3284

[LightGBM] [Info] Total Bins 6143

[LightGBM] [Info] Number of data: 7000, number of used features: 28

[LightGBM] [Info] Finished initializing training

[LightGBM] [Info] Started training...

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:1, training auc : 0.787798

[LightGBM] [Info] Iteration:1, training binary_logloss : 0.667949

[LightGBM] [Info] 0.013430 seconds elapsed, finished iteration 1

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:2, training auc : 0.805675

[LightGBM] [Info] Iteration:2, training binary_logloss : 0.649776

[LightGBM] [Info] 0.026941 seconds elapsed, finished iteration 2

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:3, training auc : 0.823995

[LightGBM] [Info] Iteration:3, training binary_logloss : 0.634349

[LightGBM] [Info] 0.043322 seconds elapsed, finished iteration 3

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=9

[LightGBM] [Info] Iteration:4, training auc : 0.829869

[LightGBM] [Info] Iteration:4, training binary_logloss : 0.620079

[LightGBM] [Info] 0.057615 seconds elapsed, finished iteration 4

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:5, training auc : 0.841468

[LightGBM] [Info] Iteration:5, training binary_logloss : 0.604578

[LightGBM] [Info] 0.071709 seconds elapsed, finished iteration 5

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=10

[LightGBM] [Info] Iteration:6, training auc : 0.850717

[LightGBM] [Info] Iteration:6, training binary_logloss : 0.591481

[LightGBM] [Info] 0.086316 seconds elapsed, finished iteration 6

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:7, training auc : 0.856189

[LightGBM] [Info] Iteration:7, training binary_logloss : 0.580923

[LightGBM] [Info] 0.099731 seconds elapsed, finished iteration 7

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:8, training auc : 0.860896

[LightGBM] [Info] Iteration:8, training binary_logloss : 0.570068

[LightGBM] [Info] 0.114074 seconds elapsed, finished iteration 8

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:9, training auc : 0.864321

[LightGBM] [Info] Iteration:9, training binary_logloss : 0.559213

[LightGBM] [Info] 0.128178 seconds elapsed, finished iteration 9

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=10

[LightGBM] [Info] Iteration:10, training auc : 0.865847

[LightGBM] [Info] Iteration:10, training binary_logloss : 0.549887

[LightGBM] [Info] 0.142092 seconds elapsed, finished iteration 10

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:11, training auc : 0.869032

[LightGBM] [Info] Iteration:11, training binary_logloss : 0.541422

[LightGBM] [Info] 0.156388 seconds elapsed, finished iteration 11

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:12, training auc : 0.871975

[LightGBM] [Info] Iteration:12, training binary_logloss : 0.532812

[LightGBM] [Info] 0.170416 seconds elapsed, finished iteration 12

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:13, training auc : 0.875643

[LightGBM] [Info] Iteration:13, training binary_logloss : 0.524978

[LightGBM] [Info] 0.185625 seconds elapsed, finished iteration 13

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:14, training auc : 0.879003

[LightGBM] [Info] Iteration:14, training binary_logloss : 0.517616

[LightGBM] [Info] 0.199136 seconds elapsed, finished iteration 14

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=10

[LightGBM] [Info] Iteration:15, training auc : 0.88134

[LightGBM] [Info] Iteration:15, training binary_logloss : 0.510784

[LightGBM] [Info] 0.213376 seconds elapsed, finished iteration 15

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:16, training auc : 0.885158

[LightGBM] [Info] Iteration:16, training binary_logloss : 0.503586

[LightGBM] [Info] 0.227964 seconds elapsed, finished iteration 16

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:17, training auc : 0.887269

[LightGBM] [Info] Iteration:17, training binary_logloss : 0.497071

[LightGBM] [Info] 0.241420 seconds elapsed, finished iteration 17

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:18, training auc : 0.891545

[LightGBM] [Info] Iteration:18, training binary_logloss : 0.491137

[LightGBM] [Info] 0.255633 seconds elapsed, finished iteration 18

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:19, training auc : 0.894663

[LightGBM] [Info] Iteration:19, training binary_logloss : 0.485116

[LightGBM] [Info] 0.270698 seconds elapsed, finished iteration 19

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:20, training auc : 0.897144

[LightGBM] [Info] Iteration:20, training binary_logloss : 0.479071

[LightGBM] [Info] 0.283444 seconds elapsed, finished iteration 20

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:21, training auc : 0.901085

[LightGBM] [Info] Iteration:21, training binary_logloss : 0.47325

[LightGBM] [Info] 0.297565 seconds elapsed, finished iteration 21

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:22, training auc : 0.904153

[LightGBM] [Info] Iteration:22, training binary_logloss : 0.468264

[LightGBM] [Info] 0.311035 seconds elapsed, finished iteration 22

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:23, training auc : 0.907717

[LightGBM] [Info] Iteration:23, training binary_logloss : 0.462459

[LightGBM] [Info] 0.324005 seconds elapsed, finished iteration 23

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=10

[LightGBM] [Info] Iteration:24, training auc : 0.91095

[LightGBM] [Info] Iteration:24, training binary_logloss : 0.457094

[LightGBM] [Info] 0.338215 seconds elapsed, finished iteration 24

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:25, training auc : 0.91396

[LightGBM] [Info] Iteration:25, training binary_logloss : 0.452346

[LightGBM] [Info] 0.351690 seconds elapsed, finished iteration 25

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:26, training auc : 0.916548

[LightGBM] [Info] Iteration:26, training binary_logloss : 0.447526

[LightGBM] [Info] 0.365339 seconds elapsed, finished iteration 26

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:27, training auc : 0.918787

[LightGBM] [Info] Iteration:27, training binary_logloss : 0.442881

[LightGBM] [Info] 0.379938 seconds elapsed, finished iteration 27

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:28, training auc : 0.920954

[LightGBM] [Info] Iteration:28, training binary_logloss : 0.437881

[LightGBM] [Info] 0.394119 seconds elapsed, finished iteration 28

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:29, training auc : 0.92347

[LightGBM] [Info] Iteration:29, training binary_logloss : 0.433232

[LightGBM] [Info] 0.407316 seconds elapsed, finished iteration 29

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:30, training auc : 0.926489

[LightGBM] [Info] Iteration:30, training binary_logloss : 0.428527

[LightGBM] [Info] 0.422244 seconds elapsed, finished iteration 30

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:31, training auc : 0.929286

[LightGBM] [Info] Iteration:31, training binary_logloss : 0.4238

[LightGBM] [Info] 0.436830 seconds elapsed, finished iteration 31

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:32, training auc : 0.931723

[LightGBM] [Info] Iteration:32, training binary_logloss : 0.418874

[LightGBM] [Info] 0.449980 seconds elapsed, finished iteration 32

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:33, training auc : 0.934119

[LightGBM] [Info] Iteration:33, training binary_logloss : 0.414036

[LightGBM] [Info] 0.464594 seconds elapsed, finished iteration 33

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:34, training auc : 0.935877

[LightGBM] [Info] Iteration:34, training binary_logloss : 0.409539

[LightGBM] [Info] 0.478952 seconds elapsed, finished iteration 34

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:35, training auc : 0.938338

[LightGBM] [Info] Iteration:35, training binary_logloss : 0.405378

[LightGBM] [Info] 0.492851 seconds elapsed, finished iteration 35

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:36, training auc : 0.941145

[LightGBM] [Info] Iteration:36, training binary_logloss : 0.40095

[LightGBM] [Info] 0.507314 seconds elapsed, finished iteration 36

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:37, training auc : 0.943501

[LightGBM] [Info] Iteration:37, training binary_logloss : 0.396802

[LightGBM] [Info] 0.520034 seconds elapsed, finished iteration 37

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:38, training auc : 0.946192

[LightGBM] [Info] Iteration:38, training binary_logloss : 0.392725

[LightGBM] [Info] 0.532794 seconds elapsed, finished iteration 38

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:39, training auc : 0.948446

[LightGBM] [Info] Iteration:39, training binary_logloss : 0.388764

[LightGBM] [Info] 0.547331 seconds elapsed, finished iteration 39

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:40, training auc : 0.950573

[LightGBM] [Info] Iteration:40, training binary_logloss : 0.384881

[LightGBM] [Info] 0.561565 seconds elapsed, finished iteration 40

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:41, training auc : 0.952163

[LightGBM] [Info] Iteration:41, training binary_logloss : 0.381236

[LightGBM] [Info] 0.574713 seconds elapsed, finished iteration 41

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:42, training auc : 0.95388

[LightGBM] [Info] Iteration:42, training binary_logloss : 0.377978

[LightGBM] [Info] 0.590710 seconds elapsed, finished iteration 42

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:43, training auc : 0.956048

[LightGBM] [Info] Iteration:43, training binary_logloss : 0.374032

[LightGBM] [Info] 0.603500 seconds elapsed, finished iteration 43

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:44, training auc : 0.957686

[LightGBM] [Info] Iteration:44, training binary_logloss : 0.37035

[LightGBM] [Info] 0.617002 seconds elapsed, finished iteration 44

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:45, training auc : 0.959158

[LightGBM] [Info] Iteration:45, training binary_logloss : 0.36696

[LightGBM] [Info] 0.631168 seconds elapsed, finished iteration 45

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:46, training auc : 0.960496

[LightGBM] [Info] Iteration:46, training binary_logloss : 0.363445

[LightGBM] [Info] 0.644741 seconds elapsed, finished iteration 46

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:47, training auc : 0.96195

[LightGBM] [Info] Iteration:47, training binary_logloss : 0.359715

[LightGBM] [Info] 0.657944 seconds elapsed, finished iteration 47

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=22

[LightGBM] [Info] Iteration:48, training auc : 0.963024

[LightGBM] [Info] Iteration:48, training binary_logloss : 0.356493

[LightGBM] [Info] 0.672357 seconds elapsed, finished iteration 48

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:49, training auc : 0.964386

[LightGBM] [Info] Iteration:49, training binary_logloss : 0.353024

[LightGBM] [Info] 0.685231 seconds elapsed, finished iteration 49

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:50, training auc : 0.965478

[LightGBM] [Info] Iteration:50, training binary_logloss : 0.350048

[LightGBM] [Info] 0.697683 seconds elapsed, finished iteration 50

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:51, training auc : 0.967142

[LightGBM] [Info] Iteration:51, training binary_logloss : 0.346614

[LightGBM] [Info] 0.712489 seconds elapsed, finished iteration 51

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=11

[LightGBM] [Info] Iteration:52, training auc : 0.968305

[LightGBM] [Info] Iteration:52, training binary_logloss : 0.343021

[LightGBM] [Info] 0.725911 seconds elapsed, finished iteration 52

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:53, training auc : 0.969855

[LightGBM] [Info] Iteration:53, training binary_logloss : 0.33965

[LightGBM] [Info] 0.738662 seconds elapsed, finished iteration 53

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:54, training auc : 0.97138

[LightGBM] [Info] Iteration:54, training binary_logloss : 0.336329

[LightGBM] [Info] 0.753535 seconds elapsed, finished iteration 54

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:55, training auc : 0.9726

[LightGBM] [Info] Iteration:55, training binary_logloss : 0.333113

[LightGBM] [Info] 0.767232 seconds elapsed, finished iteration 55

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:56, training auc : 0.973942

[LightGBM] [Info] Iteration:56, training binary_logloss : 0.329822

[LightGBM] [Info] 0.780910 seconds elapsed, finished iteration 56

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:57, training auc : 0.975169

[LightGBM] [Info] Iteration:57, training binary_logloss : 0.326521

[LightGBM] [Info] 0.794841 seconds elapsed, finished iteration 57

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:58, training auc : 0.976082

[LightGBM] [Info] Iteration:58, training binary_logloss : 0.323446

[LightGBM] [Info] 0.809004 seconds elapsed, finished iteration 58

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:59, training auc : 0.977198

[LightGBM] [Info] Iteration:59, training binary_logloss : 0.320572

[LightGBM] [Info] 0.822078 seconds elapsed, finished iteration 59

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:60, training auc : 0.978064

[LightGBM] [Info] Iteration:60, training binary_logloss : 0.317873

[LightGBM] [Info] 0.836867 seconds elapsed, finished iteration 60

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:61, training auc : 0.978937

[LightGBM] [Info] Iteration:61, training binary_logloss : 0.315203

[LightGBM] [Info] 0.850909 seconds elapsed, finished iteration 61

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=20

[LightGBM] [Info] Iteration:62, training auc : 0.979873

[LightGBM] [Info] Iteration:62, training binary_logloss : 0.312697

[LightGBM] [Info] 0.863607 seconds elapsed, finished iteration 62

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:63, training auc : 0.98087

[LightGBM] [Info] Iteration:63, training binary_logloss : 0.309975

[LightGBM] [Info] 0.877317 seconds elapsed, finished iteration 63

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=12

[LightGBM] [Info] Iteration:64, training auc : 0.981946

[LightGBM] [Info] Iteration:64, training binary_logloss : 0.306929

[LightGBM] [Info] 0.890285 seconds elapsed, finished iteration 64

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:65, training auc : 0.982631

[LightGBM] [Info] Iteration:65, training binary_logloss : 0.304005

[LightGBM] [Info] 0.903545 seconds elapsed, finished iteration 65

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:66, training auc : 0.983237

[LightGBM] [Info] Iteration:66, training binary_logloss : 0.301214

[LightGBM] [Info] 0.917765 seconds elapsed, finished iteration 66

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=22

[LightGBM] [Info] Iteration:67, training auc : 0.983893

[LightGBM] [Info] Iteration:67, training binary_logloss : 0.298476

[LightGBM] [Info] 0.931893 seconds elapsed, finished iteration 67

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=19

[LightGBM] [Info] Iteration:68, training auc : 0.984635

[LightGBM] [Info] Iteration:68, training binary_logloss : 0.296003

[LightGBM] [Info] 0.945163 seconds elapsed, finished iteration 68

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=19

[LightGBM] [Info] Iteration:69, training auc : 0.985368

[LightGBM] [Info] Iteration:69, training binary_logloss : 0.293345

[LightGBM] [Info] 0.960573 seconds elapsed, finished iteration 69

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=24

[LightGBM] [Info] Iteration:70, training auc : 0.986053

[LightGBM] [Info] Iteration:70, training binary_logloss : 0.290797

[LightGBM] [Info] 0.974280 seconds elapsed, finished iteration 70

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=22

[LightGBM] [Info] Iteration:71, training auc : 0.986643

[LightGBM] [Info] Iteration:71, training binary_logloss : 0.288291

[LightGBM] [Info] 0.987650 seconds elapsed, finished iteration 71

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:72, training auc : 0.987332

[LightGBM] [Info] Iteration:72, training binary_logloss : 0.28575

[LightGBM] [Info] 1.001018 seconds elapsed, finished iteration 72

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:73, training auc : 0.988057

[LightGBM] [Info] Iteration:73, training binary_logloss : 0.283294

[LightGBM] [Info] 1.014847 seconds elapsed, finished iteration 73

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=26

[LightGBM] [Info] Iteration:74, training auc : 0.988452

[LightGBM] [Info] Iteration:74, training binary_logloss : 0.280918

[LightGBM] [Info] 1.028188 seconds elapsed, finished iteration 74

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:75, training auc : 0.989083

[LightGBM] [Info] Iteration:75, training binary_logloss : 0.27835

[LightGBM] [Info] 1.042464 seconds elapsed, finished iteration 75

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=20

[LightGBM] [Info] Iteration:76, training auc : 0.989593

[LightGBM] [Info] Iteration:76, training binary_logloss : 0.275878

[LightGBM] [Info] 1.056295 seconds elapsed, finished iteration 76

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=20

[LightGBM] [Info] Iteration:77, training auc : 0.990151

[LightGBM] [Info] Iteration:77, training binary_logloss : 0.273444

[LightGBM] [Info] 1.069793 seconds elapsed, finished iteration 77

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:78, training auc : 0.990569

[LightGBM] [Info] Iteration:78, training binary_logloss : 0.271126

[LightGBM] [Info] 1.083776 seconds elapsed, finished iteration 78

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:79, training auc : 0.990993

[LightGBM] [Info] Iteration:79, training binary_logloss : 0.268766

[LightGBM] [Info] 1.097278 seconds elapsed, finished iteration 79

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=21

[LightGBM] [Info] Iteration:80, training auc : 0.99142

[LightGBM] [Info] Iteration:80, training binary_logloss : 0.266402

[LightGBM] [Info] 1.110613 seconds elapsed, finished iteration 80

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:81, training auc : 0.991887

[LightGBM] [Info] Iteration:81, training binary_logloss : 0.264169

[LightGBM] [Info] 1.125377 seconds elapsed, finished iteration 81

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=20

[LightGBM] [Info] Iteration:82, training auc : 0.99234

[LightGBM] [Info] Iteration:82, training binary_logloss : 0.261749

[LightGBM] [Info] 1.138764 seconds elapsed, finished iteration 82

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:83, training auc : 0.992836

[LightGBM] [Info] Iteration:83, training binary_logloss : 0.259547

[LightGBM] [Info] 1.151774 seconds elapsed, finished iteration 83

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:84, training auc : 0.993185

[LightGBM] [Info] Iteration:84, training binary_logloss : 0.257249

[LightGBM] [Info] 1.166518 seconds elapsed, finished iteration 84

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:85, training auc : 0.993482

[LightGBM] [Info] Iteration:85, training binary_logloss : 0.254958

[LightGBM] [Info] 1.180124 seconds elapsed, finished iteration 85

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:86, training auc : 0.993851

[LightGBM] [Info] Iteration:86, training binary_logloss : 0.252758

[LightGBM] [Info] 1.193683 seconds elapsed, finished iteration 86

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:87, training auc : 0.994185

[LightGBM] [Info] Iteration:87, training binary_logloss : 0.25062

[LightGBM] [Info] 1.208783 seconds elapsed, finished iteration 87

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=13

[LightGBM] [Info] Iteration:88, training auc : 0.994543

[LightGBM] [Info] Iteration:88, training binary_logloss : 0.248303

[LightGBM] [Info] 1.222327 seconds elapsed, finished iteration 88

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:89, training auc : 0.994812

[LightGBM] [Info] Iteration:89, training binary_logloss : 0.246079

[LightGBM] [Info] 1.235147 seconds elapsed, finished iteration 89

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=19

[LightGBM] [Info] Iteration:90, training auc : 0.995107

[LightGBM] [Info] Iteration:90, training binary_logloss : 0.243942

[LightGBM] [Info] 1.249424 seconds elapsed, finished iteration 90

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=14

[LightGBM] [Info] Iteration:91, training auc : 0.995351

[LightGBM] [Info] Iteration:91, training binary_logloss : 0.241682

[LightGBM] [Info] 1.262943 seconds elapsed, finished iteration 91

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:92, training auc : 0.995663

[LightGBM] [Info] Iteration:92, training binary_logloss : 0.239551

[LightGBM] [Info] 1.276080 seconds elapsed, finished iteration 92

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:93, training auc : 0.995831

[LightGBM] [Info] Iteration:93, training binary_logloss : 0.237387

[LightGBM] [Info] 1.290925 seconds elapsed, finished iteration 93

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=17

[LightGBM] [Info] Iteration:94, training auc : 0.996133

[LightGBM] [Info] Iteration:94, training binary_logloss : 0.23537

[LightGBM] [Info] 1.303520 seconds elapsed, finished iteration 94

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=18

[LightGBM] [Info] Iteration:95, training auc : 0.996366

[LightGBM] [Info] Iteration:95, training binary_logloss : 0.233472

[LightGBM] [Info] 1.316507 seconds elapsed, finished iteration 95

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=22

[LightGBM] [Info] Iteration:96, training auc : 0.996716

[LightGBM] [Info] Iteration:96, training binary_logloss : 0.23139

[LightGBM] [Info] 1.330392 seconds elapsed, finished iteration 96

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:97, training auc : 0.996983

[LightGBM] [Info] Iteration:97, training binary_logloss : 0.229236

[LightGBM] [Info] 1.344879 seconds elapsed, finished iteration 97

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=15

[LightGBM] [Info] Iteration:98, training auc : 0.997187

[LightGBM] [Info] Iteration:98, training binary_logloss : 0.227213

[LightGBM] [Info] 1.357977 seconds elapsed, finished iteration 98

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:99, training auc : 0.997419

[LightGBM] [Info] Iteration:99, training binary_logloss : 0.225164

[LightGBM] [Info] 1.372758 seconds elapsed, finished iteration 99

[LightGBM] [Info] Trained a tree with leaves=63 and max_depth=16

[LightGBM] [Info] Iteration:100, training auc : 0.997572

[LightGBM] [Info] Iteration:100, training binary_logloss : 0.223258

[LightGBM] [Info] 1.385942 seconds elapsed, finished iteration 100

[LightGBM] [Info] Finished training

你可能感兴趣的:(python编程,机器学习)

机器学习与深度学习间关系与区别 ℒℴѵℯ心·动ꦿ໊ོ꫞ 人工智能学习深度学习 python
一、机器学习概述定义机器学习（MachineLearning,ML）是一种通过数据驱动的方法，利用统计学和计算算法来训练模型，使计算机能够从数据中学习并自动进行预测或决策。机器学习通过分析大量数据样本，识别其中的模式和规律，从而对新的数据进行判断。其核心在于通过训练过程，让模型不断优化和提升其预测准确性。主要类型1.监督学习（SupervisedLearning）监督学习是指在训练数据集中包含输入
数字里的世界17期：2021年全球10大顶级数据中心，中国移动榜首张三叨
你知道吗？2016年，全球的数据中心共计用电4160亿千瓦时，比整个英国的发电量还多40％！前言每天，我们都会创造超过250万TB的数据。并且随着物联网（IOT）的不断普及，这一数据将持续增长。如此庞大的数据被存储在被称为“数据中心”的专用设施中。虽然最早的数据中心建于20世纪40年代，但直到1997-2000年的互联网泡沫期间才逐渐成为主流。当前人类的技术，比如人工智能和机器学习，已经将我们推向
nosql数据库技术与应用知识点皆过客，揽星河 NoSQL nosql 数据库大数据数据分析数据结构非关系型数据库
Nosql知识回顾大数据处理流程数据采集(flume、爬虫、传感器)数据存储(本门课程NoSQL所处的阶段)Hdfs、MongoDB、HBase等数据清洗(入仓)Hive等数据处理、分析(Spark、Flink等)数据可视化数据挖掘、机器学习应用(Python、SparkMLlib等)大数据时代存储的挑战(三高)高并发(同一时间很多人访问)高扩展(要求随时根据需求扩展存储)高效率(要求读写速度快)
Python开发常用的三方模块如下：换个网名有点难 python 开发语言
Python是一门功能强大的编程语言，拥有丰富的第三方库，这些库为开发者提供了极大的便利。以下是100个常用的Python库，涵盖了多个领域：1、NumPy，用于科学计算的基础库。2、Pandas，提供数据结构和数据分析工具。3、Matplotlib，一个绘图库。4、Scikit-learn，机器学习库。5、SciPy，用于数学、科学和工程的库。6、TensorFlow，由Google开发的开源机
Python实现简单的机器学习算法 master_chenchengg python python 办公效率 python开发 IT
Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl
Python入门之Lesson2:Python基础语法小熊同学哦 Python入门课程 python 开发语言算法数据结构青少年编程
目录前言一.介绍1.变量和数据类型2.常见运算符3.输入输出4.条件语句5.循环结构二.练习三.总结前言欢迎来到《Python入门》系列博客的第二课。在上一课中，我们了解了Python的安装及运行环境的配置。在这一课中，我们将深入学习Python的基础语法，这是编写Python代码的根基。通过本节内容的学习，你将掌握变量、数据类型、运算符、输入输出、条件语句等Python编程的基础知识。一.介绍1
遥感影像的切片处理 sand&wich 计算机视觉 python 图像处理
在遥感影像分析中，经常需要将大尺寸的影像切分成小片段，以便于进行详细的分析和处理。这种方法特别适用于机器学习和图像处理任务，如对象检测、图像分类等。以下是如何使用Python和OpenCV库来实现这一过程，同时确保每个影像片段保留正确的地理信息。准备环境首先，确保安装了必要的Python库，包括numpy、opencv-python和xml.etree.ElementTree。这些库将用于图像处理
ai绘画工具midjourney怎么下载？附作品管理教程设计师早上好
Midjourney是一款功能强大的AI绘画工具，它使用机器学习技术和深度神经网络等算法，可以生成各种艺术风格的绘画作品。在创意设计、广告宣传等方面有着广泛的应用前景。那么，ai绘画工具midjourney怎么下载？本文将为您介绍Midjourney的下载以及作品的相关管理。一、Midjourney下载Midjourney的下载非常简单，只需打开Midjourney官网（点击“GetMidjour
Python编程 - 函数进阶易辰君 Python核心编程 python 开发语言
目录前言一、函数参数的高级用法（一）缺省参数（二）命名参数（三）不定长参数二、拆包（一）函数返回值拆包（二）通过星号拆包（三）总结三、匿名函数（一）函数定义（二）使用匿名函数四、递归函数（一）简介（二）基本结构（三）简单示例（四）优缺点总结前言上篇文章主要了解了函数基础，如何定义函数，函数种类以及局部变量和全局变量的差异等，接下来就讲解python函数较为进阶的知识点，若有任何想法欢迎一起沟通讨论
Python编程 - 初识面向对象易辰君 Python核心编程 python 开发语言
目录前言一、面向对象二、类和对象（一）类简介定义类（二）对象简介创建对象（三）总结三、实例属性和实例方法（一）实例属性创建的基本语法使用示例（二）实例方法定义实例方法的基本语法调用示例方法的示例（三）总结四、类中的self（一）基本概念（二）作用访问实例属性调用其他实例方法在构造函数中初始化对象（三）总结五、__init__方法（一）__init__方法的特点（二）基本语法（三）示例（四）总结前言
[实践应用] 深度学习之模型性能评估指标 YuanDaima2048 深度学习工具使用深度学习人工智能损失函数性能评估 pytorch python 机器学习
文章总览：YuanDaiMa2048博客文章总览深度学习之模型性能评估指标分类任务回归任务排序任务聚类任务生成任务其他介绍在机器学习和深度学习领域，评估模型性能是一项至关重要的任务。不同的学习任务需要不同的性能指标来衡量模型的有效性。以下是对一些常见任务及其相应的性能评估指标的详细解释和总结。分类任务分类任务是指模型需要将输入数据分配到预定义的类别或标签中。以下是分类任务中常用的性能指标：准确率(
机器学习-聚类算法不良人龍木木机器学习机器学习算法聚类
机器学习-聚类算法1.AHC2.K-means3.SC4.MCL仅个人笔记，感谢点赞关注！1.AHC2.K-means3.SC传统谱聚类：个人对谱聚类算法的理解以及改进4.MCL目前仅专注于NLP的技术学习和分享感谢大家的关注与支持！
未来软件市场是怎么样的？做开发的生存空间如何？ cesske 软件需求
目录前言一、未来软件市场的发展趋势二、软件开发人员的生存空间前言未来软件市场是怎么样的？做开发的生存空间如何？一、未来软件市场的发展趋势技术趋势：人工智能与机器学习：随着技术的不断成熟，人工智能将在更多领域得到应用，如智能客服、自动驾驶、智能制造等，这将极大地推动软件市场的增长。云计算与大数据：云计算服务将继续普及，大数据技术的应用也将更加广泛。企业将更加依赖云计算和大数据来优化运营、提升效率，并
深入理解Python中的“_,”：一个实用的语法特性小桥流水---人工智能 Python程序代码 Python常见bug 深度学习 python 开发语言
在Python编程中，你可能经常会看到一个特殊的标识符“_”。这个符号在Python中有多种用途，其具体含义依上下文而定。本文将探讨其中一种常见用法——作为一个临时性的占位符——并解释它在实际编程中的实用性和应用场景。1.“_”作为占位符在Python中，下划线（_）经常被用作一个临时或不重要的变量。当你在解包（unpacking）一个表达式但又不想使用其中某些值时，下划线可用作占位符。这样做的好
python中zeros用法_Python中的numpy.zeros()用法江平舟 python中zeros用法
numpy.zeros()函数是最重要的函数之一,广泛用于机器学习程序中。此函数用于生成包含零的数组。numpy.zeros()函数提供给定形状和类型的新数组,并用零填充。句法numpy.zeros(shape,dtype=float,order='C'参数形状：整数或整数元组此参数用于定义数组的尺寸。此参数用于我们要在其中创建数组的形状,例如(3,2)或2。dtype：数据类型(可选)此参数用于
【Python基础】Python迭代器与生成器（两种强大工具）姑苏老陈 Python编程入门 python 开发语言 python迭代器与生成器
本文收录于《Python编程入门》专栏，从零基础开始，分享一些Python编程基础知识，欢迎关注，谢谢！文章目录一、前言二、迭代器2.1创建迭代器2.2自定义迭代器2.3处理大型文件三、生成器四、生成器表达式五、实际应用案例5.1数据库查询5.2网络数据流处理六、总结一、前言在Python中，迭代器与生成器是两种非常强大的工具，它们可以帮助我们有效地处理大量数据，特别是在需要逐个访问元素的情况下。
【NumPy】深入解析numpy.zeros()函数二七830 numpy
欢迎莅临我的个人主页这里是我深耕Python编程、机器学习和自然语言处理（NLP）领域，并乐于分享知识与经验的小天地！博主简介：我是二七830，一名对技术充满热情的探索者。多年的Python编程和机器学习实践，使我深入理解了这些技术的核心原理，并能够在实际项目中灵活应用。尤其是在NLP领域，我积累了丰富的经验，能够处理各种复杂的自然语言任务。技术专长：我熟练掌握Python编程语言，并深入研究了机
Python字符串比较：如何判断两个字符串是否相等？程序员杨弋 Python全栈工程师学习指南 python 开发语言
在Python编程中，字符串是最常用的数据类型之一。当我们需要比较两个字符串时，通常需要判断它们是否相等。这篇文章将详细介绍Python中比较两个字符串的方法。1、使用“==”运算符比较字符串在Python中，可以使用“==”运算符比较两个字符串是否相等。这个运算符返回True，表示两个字符串相等，返回False，表示两个字符串不相等。例如：str1="hello"str2="hello"ifst
【中国国际航空-注册_登录安全分析报告】风控牛验证码接口安全评测系列安全行为验证极验网易易盾智能手机
前言由于网站注册入口容易被黑客攻击，存在如下安全问题：1.暴力破解密码，造成用户信息泄露2.短信盗刷的安全问题，影响业务及导致用户投诉3.带来经济损失，尤其是后付费客户，风险巨大，造成亏损无底洞所以大部分网站及App都采取图形验证码或滑动验证码等交互解决方案，但在机器学习能力提高的当下，连百度这样的大厂都遭受攻击导致点名批评，图形验证及交互验证方式的安全性到底如何？请看具体分析一、中国国际航空PC
机器学习流形数据降维：UMAP 降维算法小嗷犬 Python 机器学习 #数据分析及可视化机器学习算法人工智能
✅作者简介：人工智能专业本科在读，喜欢计算机与编程，写博客记录自己的学习历程。个人主页：小嗷犬的个人主页个人网站：小嗷犬的技术小站个人信条：为天地立心，为生民立命，为往圣继绝学，为万世开太平。本文目录UMAP简介理论基础特点与优势应用场景在Python中使用UMAP安装umap-learn库使用UMAP可视化手写数字数据集UMAP简介UMAP（UniformManifoldApproximatio
七.正则化愿风去了
吴恩达机器学习之正则化（Regularization）http://www.cnblogs.com/jianxinzhou/p/4083921.html从数学公式上理解L1和L2https://blog.csdn.net/b876144622/article/details/81276818虽然在线性回归中加入基函数会使模型更加灵活，但是很容易引起数据的过拟合。例如将数据投影到30维的基函数上，模
机器学习-------数据标准化罔闻_spider 数据分析算法机器学习人工智能
什么是归一化，它与标准化的区别是什么？一作用在做训练时，需要先将特征值与标签标准化，可以防止梯度防炸和过拟合；将标签标准化后，网络预测出的数据是符合标准正态分布的—StandarScaler()，与真实值有很大差别。因为StandarScaler()对数据的处理是（真实值-平均值）/标准差。同时在做预测时需要将输出数据逆标准化提升模型精度：标准化/归一化使不同维度的特征在数值上更具比较性，提高分类
分享一个基于python的电子书数据采集与可视化分析 hadoop电子书数据分析与推荐系统 spark大数据毕设项目（源码、调试、LW、开题、PPT) 计算机源码社 Python项目大数据大数据 python hadoop 计算机毕业设计选题计算机毕业设计源码数据分析 spark毕设
作者：计算机源码社个人简介：本人八年开发经验，擅长Java、Python、PHP、.NET、Node.js、Android、微信小程序、爬虫、大数据、机器学习等，大家有这一块的问题可以一起交流！学习资料、程序开发、技术解答、文档报告如需要源码，可以扫取文章下方二维码联系咨询Java项目微信小程序项目Android项目Python项目PHP项目ASP.NET项目Node.js项目选题推荐项目实战|p
python转换为可执行文件的方法有哪些嗷呜大嘴狼 python 开发语言 git
转换为可执行文件的方法有哪些在Python编程中，我们通常会遇到需要将代码转换为可执行文件的场景，以便在其他计算机上运行代码。那么，Python中转换为可执行文件的方法有哪些呢？本文将从三个方面介绍。转换为可执行文件的方法有哪些一、使用pyinstallerPyinstaller是一个流行的Python打包工具，可以将Python代码打包为可执行文件。使用pyinstaller的步骤如下：1、安装
软件测试/测试开发/全日制 |利用Django REST framework构建微服务霍格沃兹-慕漓 django 微服务 sqlite
霍格沃兹测试开发学社推出了《Python全栈开发与自动化测试班》。本课程面向开发人员、测试人员与运维人员，课程内容涵盖Python编程语言、人工智能应用、数据分析、自动化办公、平台开发、UI自动化测试、接口测试、性能测试等方向。为大家提供更全面、更深入、更系统化的学习体验，课程还增加了名企私教服务内容，不仅有名企经理为你1v1辅导，还有行业专家进行技术指导，针对性地解决学习、工作中遇到的难题。让找
两种方法判断Python的位数是32位还是64位 sanqima Python编程电脑 python 开发语言
Python从1991年发布以来，凭借其简洁、清晰、易读的语法、丰富的标准库和第三方工具，在Web开发、自动化测试、人工智能、图形识别、机器学习等领域发展迅猛。 Python是一种胶水语言，通过Cython库与C/C++语言进行链接，通过Jython库与Java语言进行链接。 Python是跨平台的，可运行在多种操作系统上，包括但不限于Windows、Linux和macOS。这意味着用Py
Python round函数详解寒秋丶 Python 自动化测试性能测试 python 开发语言测试开发软件开发软件测试自动化测试性能测试
大家好，在Python编程中，经常需要对数字进行舍入操作。无论是在金融领域的货币计算，还是科学计算中的数据处理，都可能需要使用到四舍五入功能。为了满足这一需求，Python提供了一个内置函数round()，它能够方便地对数字进行舍入操作。在本文中，将深入探讨Python中round()函数的用法和特性。将从基本语法开始，逐步深入，讨论该函数在不同情况下的行为，以及如何在实际编程中灵活运用。无论您是
使用最大边际相关性(MMR)选择示例：提高AI模型的多样性和相关性 aehrutktrjk 人工智能 easyui 前端 python
使用最大边际相关性(MMR)选择示例：提高AI模型的多样性和相关性引言在机器学习和自然语言处理领域，选择合适的训练示例对模型性能至关重要。最大边际相关性(MaximalMarginalRelevance,MMR)是一种优秀的示例选择方法，它不仅考虑了示例与输入的相关性，还注重保持所选示例之间的多样性。本文将深入探讨如何使用MMR来选择示例，以提高AI模型的性能和泛化能力。什么是最大边际相关性(MM
LangChain集成指南:如何利用多样化的AI提供商 aehrutktrjk 人工智能 langchain python
LangChain集成指南:如何利用多样化的AI提供商引言在人工智能和机器学习领域,LangChain已成为一个强大而灵活的框架,允许开发者轻松集成各种AI服务提供商。本文将深入探讨LangChain的集成能力,介绍如何利用不同的AI提供商来增强你的应用程序,并提供实用的代码示例。LangChain集成概览LangChain支持多种AI提供商的集成,这些集成可以分为两类:独立包集成:这些提供商有独
机器学习VS深度学习 nfgo 机器学习
机器学习（MachineLearning,ML）和深度学习（DeepLearning,DL）是人工智能（AI）的两个子领域，它们有许多相似之处，但在技术实现和应用范围上也有显著区别。下面从几个方面对两者进行区分：1.概念层面机器学习：是让计算机通过算法从数据中自动学习和改进的技术。它依赖于手动设计的特征和数学模型来进行学习，常用的模型有决策树、支持向量机、线性回归等。深度学习：是机器学习的一个子领
JAVA基础灵静志远位运算加载 Date 字符串池覆盖
一、类的初始化顺序 1 （静态变量，静态代码块）-->（变量，初始化块）--> 构造器同一括号里的，根据它们在程序中的顺序来决定。上面所述是同一类中。如果是继承的情况，那就在父类到子类交替初始化。二、String 1 String a = "abc"; JAVA虚拟机首先在字符串池中查找是否已经存在了值为"abc"的对象，根
keepalived实现redis主从高可用 bylijinnan redis
方案说明两台机器（称为A和B），以统一的VIP对外提供服务 1.正常情况下，A和B都启动，B会把A的数据同步过来（B is slave of A） 2.当A挂了后，VIP漂移到B；B的keepalived 通知redis 执行：slaveof no one，由B提供服务 3.当A起来后，VIP不切换，仍在B上面；而A的keepalived 通知redis 执行slaveof B，开始
java文件操作大全 0624chenhong java
最近在博客园看到一篇比较全面的文件操作文章，转过来留着。 http://www.cnblogs.com/zhuocheng/archive/2011/12/12/2285290.html 转自http://blog.sina.com.cn/s/blog_4a9f789a0100ik3p.html 一.获得控制台用户输入的信息 &nbs
android学习任务不懂事的小屁孩工作
任务完成情况搞清楚带箭头的pupupwindows和不带的使用已完成熟练使用pupupwindows和alertdialog，并搞清楚两者的区别已完成熟练使用android的线程handler,并敲示例代码进行中了解游戏2048的流程，并完成其代码工作进行中-差几个actionbar 研究一下android的动画效果，写一个实例已完成复习fragem
zoom.js 换个号韩国红果果 oom
它的基于bootstrap 的 https://raw.github.com/twbs/bootstrap/master/js/transition.js transition.js模块引用顺序 <link rel="stylesheet" href="style/zoom.css"> <script src=&q
详解Oracle云操作系统Solaris 11.2 蓝儿唯美 Solaris
当Oracle发布Solaris 11时，它将自己的操作系统称为第一个面向云的操作系统。Oracle在发布Solaris 11.2时继续它以云为中心的基调。但是，这些说法没有告诉我们为什么Solaris是配得上云的。幸好，我们不需要等太久。Solaris11.2有4个重要的技术可以在一个有效的云实现中发挥重要作用：OpenStack、内核域、统一存档（UA）和弹性虚拟交换（EVS）。
spring学习——springmvc（一） a-john springMVC
Spring MVC基于模型-视图-控制器（Model-View-Controller，MVC）实现，能够帮助我们构建像Spring框架那样灵活和松耦合的Web应用程序。 1，跟踪Spring MVC的请求请求的第一站是Spring的DispatcherServlet。与大多数基于Java的Web框架一样，Spring MVC所有的请求都会通过一个前端控制器Servlet。前
hdu4342 History repeat itself-------多校联合五 aijuans 数论
水题就不多说什么了。 #include<iostream>#include<cstdlib>#include<stdio.h>#define ll __int64using namespace std;int main(){ int t; ll n; scanf("%d",&t); while(t--)
EJB和javabean的区别 asia007 bean ejb
EJB不是一般的JavaBean,EJB是企业级JavaBean,EJB一共分为3种,实体Bean,消息Bean,会话Bean,书写EJB是需要遵循一定的规范的,具体规范你可以参考相关的资料.另外,要运行EJB,你需要相应的EJB容器,比如Weblogic,Jboss等,而JavaBean不需要,只需要安装Tomcat就可以了 1.EJB用于服务端应用开发, 而JavaBeans
Struts的action和Result总结百合不是茶 struts Action配置 Result配置
一:Action的配置详解: 下面是一个Struts中一个空的Struts.xml的配置文件 <?xml version="1.0" encoding="UTF-8" ?> <!DOCTYPE struts PUBLIC &quo
如何带好自已的团队 bijian1013 项目管理团队管理团队
在网上看到博客" 怎么才能让团队成员好好干活"的评论，觉得写的比较好。原文如下：我做团队管理有几年了吧，我和你分享一下我认为带好团队的几点： 1.诚信对团队内成员，无论是技术研究、交流、问题探讨，要尽可能的保持一种诚信的态度，用心去做好，你的团队会感觉得到。 2.努力提
Java代码混淆工具 sunjing ProGuard
Open Source Obfuscators ProGuard http://java-source.net/open-source/obfuscators/proguardProGuard is a free Java class file shrinker and obfuscator. It can detect and remove unused classes, fields, m
【Redis三】基于Redis sentinel的自动failover主从复制 bit1129 redis
在第二篇中使用2.8.17搭建了主从复制，但是它存在Master单点问题，为了解决这个问题，Redis从2.6开始引入sentinel，用于监控和管理Redis的主从复制环境，进行自动failover，即Master挂了后，sentinel自动从从服务器选出一个Master使主从复制集群仍然可以工作，如果Master醒来再次加入集群，只能以从服务器的形式工作。什么是Sentine
使用代理实现Hibernate Dao层自动事务白糖_ DAO spring AOP 框架 Hibernate
都说spring利用AOP实现自动事务处理机制非常好，但在只有hibernate这个框架情况下，我们开启session、管理事务就往往很麻烦。 public void save(Object obj){ Session session = this.getSession(); Transaction tran = session.beginTransaction(); try
maven3实战读书笔记 braveCS maven3
Maven简介是什么？ Is a software project management and comprehension tool.项目管理工具是基于POM概念(工程对象模型) [设计重复、编码重复、文档重复、构建重复，maven最大化消除了构建的重复] [与XP：简单、交流与反馈；测试驱动开发、十分钟构建、持续集成、富有信息的工作区] 功能：
编程之美-子数组的最大乘积 bylijinnan 编程之美
public class MaxProduct { /** * 编程之美子数组的最大乘积 * 题目: 给定一个长度为N的整数数组，只允许使用乘法，不能用除法，计算任意N-1个数的组合中乘积中最大的一组，并写出算法的时间复杂度。 * 以下程序对应书上两种方法，求得“乘积中最大的一组”的乘积——都是有溢出的可能的。 * 但按题目的意思，是要求得这个子数组，而不
读书笔记-2 chengxuyuancsdn 读书笔记
1、反射 2、oracle年-月-日时-分-秒 3、oracle创建有参、无参函数 4、oracle行转列 5、Struts2拦截器 6、Filter过滤器(web.xml) 1、反射 (1)检查类的结构在java.lang.reflect包里有3个类Field,Method,Constructor分别用于描述类的域、方法和构造器。 2、oracle年月日时分秒 s
[求学与房地产]慎重选择IT培训学校 comsci it
关于培训学校的教学和教师的问题,我们就不讨论了,我主要关心的是这个问题培训学校的教学楼和宿舍的环境和稳定性问题我们大家都知道，房子是一个比较昂贵的东西，特别是那种能够当教室的房子... &nb
RMAN配置中通道(CHANNEL)相关参数 PARALLELISM 、FILESPERSET的关系 daizj oracle rman filesperset PARALLELISM
RMAN配置中通道(CHANNEL)相关参数 PARALLELISM 、FILESPERSET的关系转 PARALLELISM --- 我们还可以通过parallelism参数来指定同时"自动"创建多少个通道： RMAN > configure device type disk parallelism 3 ; 表示启动三个通道，可以加快备份恢复的速度。
简单排序:冒泡排序 dieslrae 冒泡排序
public void bubbleSort(int[] array){ for(int i=1;i<array.length;i++){ for(int k=0;k<array.length-i;k++){ if(array[k] > array[k+1]){
初二上学期难记单词三 dcj3sjt126com sciet
concert 音乐会 tonight 今晚 famous 有名的；著名的 song 歌曲 thousand 千 accident 事故；灾难 careless 粗心的，大意的 break 折断；断裂；破碎 heart 心（脏） happen 偶尔发生，碰巧 tourist 旅游者；观光者 science （自然）科学 marry 结婚 subject 题目；
I.安装Memcahce 1. 安装依赖包libevent Memcache需要安装libevent,所以安装前可能需要执行 Shell代码收藏代码 dcj3sjt126com redis
wget http://download.redis.io/redis-stable.tar.gz tar xvzf redis-stable.tar.gz cd redis-stable make 前面3步应该没有问题，主要的问题是执行make的时候，出现了异常。异常一： make[2]: cc: Command not found 异常原因：没有安装g
并发容器 shuizhaosi888 并发容器
通过并发容器来改善同步容器的性能，同步容器将所有对容器状态的访问都串行化，来实现线程安全，这种方式严重降低并发性，当多个线程访问时，吞吐量严重降低。并发容器ConcurrentHashMap 替代同步基于散列的Map，通过Lock控制。 &nb
Spring Security（12）——Remember-Me功能 234390216 Spring Security Remember Me 记住我
Remember-Me功能目录 1.1 概述 1.2 基于简单加密token的方法 1.3 基于持久化token的方法 1.4 Remember-Me相关接口和实现
位运算焦志广位运算
一、位运算符Ｃ语言提供了六种位运算符： & 按位与 | 按位或 ^ 按位异或 ~ 取反 << 左移 >> 右移 1. 按位与运算按位与运算符"&"是双目运算符。其功能是参与运算的两数各对应的二进位相与。只有对应的两个二进位均为1时，结果位才为1 ，否则为0。参与运算的数以补码方式出现。例如：9&am
nodejs 数据库连接 mongodb mysql liguangsong mongodb mysql node 数据库连接
1.mysql 连接 package.json中dependencies加入 "mysql":"~2.7.0" 执行 npm install 在config 下创建文件 database.js
java动态编译 olive6615 java HotSpot jvm 动态编译
在HotSpot虚拟机中，有两个技术是至关重要的，即动态编译(Dynamic compilation)和Profiling。 HotSpot是如何动态编译Javad的bytecode呢？Java bytecode是以解释方式被load到虚拟机的。HotSpot里有一个运行监视器，即Profile Monitor,专门监视
Storm0.9.5的集群部署配置优化 roadrunners 优化 storm.yaml
nimbus结点配置（storm.yaml）信息： # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional inf
101个MySQL 的调节和优化的提示 tomcat_oracle mysql
　1. 拥有足够的物理内存来把整个InnoDB文件加载到内存中——在内存中访问文件时的速度要比在硬盘中访问时快的多。　　2. 不惜一切代价避免使用Swap交换分区 – 交换时是从硬盘读取的，它的速度很慢。　　3. 使用电池供电的RAM（注：RAM即随机存储器）。　　4. 使用高级的RAID（注：Redundant Arrays of Inexpensive Disks，即磁盘阵列
zoj 3829 Known Notation(贪心) 阿尔萨斯 ZOJ
题目链接：zoj 3829 Known Notation 题目大意：给定一个不完整的后缀表达式，要求有2种不同操作，用尽量少的操作使得表达式完整。解题思路：贪心，数字的个数要要保证比∗的个数多1，不够的话优先补在开头是最优的。然后遍历一遍字符串，碰到数字+1，碰到∗-1,保证数字的个数大于等1，如果不够减的话，可以和最后面的一个数字交换位置（用栈维护十分方便），因为添加和交换代价都是1