【NLP数据竞赛】“达观杯”文本智能处理挑战赛（六）模型调优

一、网格搜索

• 网格搜索（Grid Search）用简答的话来说就是手动的给出一个模型中你想要改动的所用的参数，程序自动的帮你使用穷举法来将所用的参数都运行一遍。决策树中我们常常将最大树深作为需要调节的参数；
• K次验证：
  

二、模型调优与参数融合

选择均匀融合，调参结果为：
模型    最优参数    F1评分
LR    C=10, max_iter=20    0.713
SVM    C=1, max_iter=20    0.722
LightGBM    learning_rate=0.1, n_estimate=50, num_leaves=10    0.647
最优结果    LR+SVM    0.724
模型融合时，应该要注意：

(1) 应选择性能较好的模型进行融合。

(2)应选择具有差异性的模型进行融合，取长补短。

模型融合时，常用函数predict_proba()来输出预测的概率，然后通过几个模型的平均或权值线性组合得到最终的结果。再利用np.argmax()来获得每一行数据对于所有class的最大概率的index，作为我们的预测类别。
 

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn import svm
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
from sklearn.linear_model import LogisticRegression
import lightgbm as lgb
from sklearn.model_selection import GridSearchCV
import gensim
import time
import pickle
import csv,sys

# read data
df = pd.read_csv('data/train_set.csv', nrows=5000)
df.drop(columns='article', inplace=True)

# observe data
# print(df['class'].value_counts(normalize=True, ascending=False))

# TF-IDF
vectorizer = TfidfVectorizer(ngram_range=(1, 2), min_df=3, max_df=0.9, sublinear_tf=True)
vectorizer.fit(df['word_seg'])
x_train = vectorizer.transform(df['word_seg'])

# split training set and validation set
predictor = ['word_seg']
x_train, x_validation, y_train, y_validation = train_test_split(x_train, df['class'], test_size=0.2)


clf = LogisticRegression(C=10, max_iter=20)
clf = svm.LinearSVC(C=1, max_iter=20)
clf = lgb.sklearn.LGBMClassifier(learning_rate=0.1, n_estimators=50, num_leaves=10)

algorithms=[
LogisticRegression(C=10, max_iter=20),
svm.LinearSVC(C=1, max_iter=20),
]
full_predictions = []
for alg in algorithms:
    # Fit the algorithm using the full training data.
    alg.fit(x_train, y_train)
    # Predict using the test dataset.  We have to convert all the columns to floats to avoid an error.
    predictions = alg.decision_function(x_validation.astype(float))
    full_predictions.append(predictions)


y_prediction = (full_predictions[0] + full_predictions[1]) / 2

# adjust labels from 1 to 19
y_prediction = np.argmax(y_prediction, axis=1)+1


# # grid search for model
# param_grid = {
#     'num_leaves': [10, 20, 30],
#     'learning_rate': [0.01, 0.05, 0.1],
#     'n_estimators': [10, 20, 50]
# }
# gbm = GridSearchCV(clf, param_grid, cv=5, scoring='f1_micro', n_jobs=4, verbose=1)
# gbm.fit(x_train, y_train)
# print('网格搜索得到的最优参数是:', gbm.best_params_)

# test model
label = []
for i in range(1, 20):
    label.append(i)
f1 = f1_score(y_validation, y_prediction, labels=label, average='micro')
print('The F1 Score: ' + str("%.4f" % f1))

参考原文：
https://blog.csdn.net/weixin_43314778/article/details/89321726
https://blog.csdn.net/weixin_41151521/article/details/89321671