论文趋势分析-多标签文本分类

github地址：链接
多标签文本分类简介：链接

1数据预处理

1.1数据读取

• 为方便处理，随机抽样10%的数据进行分析

import seaborn as sns #用于画图
from bs4 import BeautifulSoup #用于爬取arxiv的数据
import re #用于正则表达式，匹配字符串的模式
import requests #用于网络连接，发送网络请求，使用域名获取对应信息
import json #读取数据，我们的数据为json格式的
import pandas as pd #数据处理，数据分析
import matplotlib.pyplot as plt #画图工具

def readArxivFile(path, columns=['id', 'submitter', 'authors', 'title', 'comments', 'journal-ref', 'doi',
       'report-no', 'categories', 'license', 'abstract', 'versions',
       'update_date', 'authors_parsed'], count=None):
    '''
    定义读取文件的函数
        path: 文件路径
        columns: 需要选择的列
        count: 读取行数
    '''
    
    data  = []
    with open(path, 'r') as f: 
        for idx, line in enumerate(f): 
            if idx == count:
                break
                
            d = json.loads(line)
            d = {
     col : d[col] for col in columns}
            data.append(d)

    data = pd.DataFrame(data)
    return data

data = readArxivFile('D:\code\Github\data\AcademicTrendsAnalysis/arxiv-metadata-oai-snapshot.json', 
                     ['id', 'title', 'categories', 'abstract'])

data = data.sample(frac = 0.1)
data.shape

(179691, 5)

1.2文本提取

# 合并title和abtraact
data['text'] = data['title'] + data['abstract']
#将换行符替换位空格
data['text'] = data['text'].str.replace('\n',' ')
# 将大写全部转换成小写
data['text'] = data['text'].str.lower()
# 删除多余列
data = data.drop(['abstract','title'],axis = 1) 

1.3类别转换

data['categories'] = data.categories.str.split(' ')
data['categories_big'] = data.categories.apply(lambda x : [xx.split('.')[0] for xx in x])
data.head(3)

	id	title	categories	abstract	categories_big
0	0704.0001	Calculation of prompt diphoton production cros...	[hep-ph]	A fully differential calculation in perturba...	[hep-ph]
1	0704.0002	Sparsity-certifying Graph Decompositions	[math.CO, cs.CG]	We describe a new algorithm, the $(k,\ell)$-...	[math, cs]
2	0704.0003	The evolution of the Earth-Moon system based o...	[physics.gen-ph]	The evolution of Earth-Moon system is descri...	[physics]

1.4将目标数据（论文大类）二值化

• MultiLabelBinarizer的使用方法链接

• 类似于One-Hot编码，只不过自变量可以是元组或列表

from sklearn.preprocessing import MultiLabelBinarizer
mlb = MultiLabelBinarizer()
data_label = mlb.fit_transform(data.categories_big)

data_label.shape

(179691, 38)

2TF-IDF+机器学习分类器

2.1分词

• TfidfVectorizer简介：链接
• 将原始句子转换成词频向量

from sklearn.feature_extraction.text import TfidfVectorizer 
vecter = TfidfVectorizer(max_features=4000)
data_tfidf = vecter.fit_transform(data.text)

2.2数据集划分

## 训练集划分  
from sklearn.model_selection import train_test_split 

X_train,X_test,y_train,y_test = train_test_split(data_tfidf,data_label)

2.3多分类贝叶斯模型

• MultiOutputClassifier将单类预测模型转换为多类模型，类似于一个模型只预测一个类

# 构建多标签分类模型
from sklearn.multioutput import MultiOutputClassifier
from sklearn.naive_bayes import MultinomialNB
clf = MultiOutputClassifier(MultinomialNB()).fit(X_train, y_train)

训练结果 ：

• 平均预测精度只有0.53

from sklearn.metrics import accuracy_score
accuracy_score(y_test,clf.predict(X_test))

0.5262782984217439

2.4XGBoost模型

• 训练太过耗时，未得出结果我就终止了训练
• 因为每个y_train都是一个37维的向量，因此需要训练37个模型，加之训练集太大，导致训练时间非常长，而且如果要调参的话，时间就更长了

from sklearn.pipeline import make_pipeline
from sklearn.model_selection import GridSearchCV,KFold
import xgboost as xgb 

model = MultiOutputClassifier( xgb.XGBClassifier(n_jobs = -1))
model.fit(X_train, y_train)
accuracy_score(y_test,model.predict(X_test))

3深度学习模型

from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(data['text'].iloc[:100000], data_label[:100000],test_size = 0.15,random_state = 1)

3.1分词与embedding

• Tokenizer使用方法：链接
  链接
• 本文使用的tensorflow2.3.0版本

# parameter
max_features= 500#最大分词数
max_len= 150#最大截取截取长度
embed_size=100#
batch_size = 128
epochs = 5

from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing import sequence
tokens = Tokenizer(num_words = max_features)
tokens.fit_on_texts(list(x_train))

#y_train = data_label[:100000]
x_sub_train = tokens.texts_to_sequences(x_train)
x_sub_train = sequence.pad_sequences(x_sub_train, maxlen=max_len)

• 将每个text都转换为了一个150维的向量

x_sub_train.shape,x_train.shape

((85000, 150), (85000,))

3.2定义模型并训练

• 由于训练时间太长，因此训练完一个epoch就暂停了kernel

from tensorflow.keras.layers import Dense,Input,LSTM,Bidirectional,Activation,Conv1D,GRU
from tensorflow.keras.layers import Dropout,Embedding,GlobalMaxPooling1D, MaxPooling1D, Add, Flatten
from tensorflow.keras.layers import GlobalAveragePooling1D, GlobalMaxPooling1D, concatenate, SpatialDropout1D# Keras Callback Functions:
from tensorflow.keras.callbacks import Callback
from tensorflow.keras.callbacks import EarlyStopping,ModelCheckpoint
from tensorflow.keras import initializers, regularizers, constraints, optimizers, layers, callbacks
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam

sequence_input = Input(shape=(max_len, ))
x = Embedding(max_features, embed_size, trainable=True)(sequence_input)
x = SpatialDropout1D(0.2)(x)
x = Bidirectional(GRU(128, return_sequences=True,dropout=0.1,recurrent_dropout=0.1))(x)
x = Conv1D(64, kernel_size = 3, padding = "valid", kernel_initializer = "glorot_uniform")(x)
avg_pool = GlobalAveragePooling1D()(x)
max_pool = GlobalMaxPooling1D()(x)
x = concatenate([avg_pool, max_pool]) 
preds = Dense(38, activation="sigmoid")(x)

model = Model(sequence_input, preds)
model.compile(loss='binary_crossentropy',optimizer=Adam(lr=1e-3),metrics=['accuracy'])
model.fit(x_sub_train, y_train, 
          batch_size=batch_size, 
          validation_split=0.2,
          epochs=epochs)
          

Epoch 1/5
532/532 [==============================] - 1139s 2s/step - loss: 0.1034 - accuracy: 0.4690 - val_loss: 0.0705 - val_accuracy: 0.6457
Epoch 2/5
  7/532 [..............................] - ETA: 16:59 - loss: 0.0724 - accuracy: 0.6127