文献学习03_GloVe: Global Vectors for Word Representation_20221124

论文信息
Subjects:《2014年自然语言处理经验方法会议论文集》（EMNLP），第1532–1543页，2014年10月25日至29日，

（1）题目：GloVe: Global Vectors for Word Representation （GloVe:单词表示的全局向量）

（2）文章下载地址：https://aclanthology.org/D14-1162
PDF：https://aclanthology.org/D14-1162.pdf

（3）相关代码：stanfordnlp/GloVe + additional community code

（4）作者信息：Jeffrey Pennington

————————————————

Abstract

Recent methods for learning vector space representations of words have succeeded in capturing fine-grained semantic and syntactic regularities using vector arithmetic, but the origin of these regularities has remained opaque. We analyze and make explicit the model properties needed for such regularities to emerge in word vectors. The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods. Our model efficiently leverages statistical information by training only on the nonzero elements in a word-word cooccurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus. The model produces a vector space with meaningful substructure, as evidenced by its performance of 75% on a recent word analogy task. It also outperforms related models on similarity tasks and named entity recognition.

最近用于学习单词的向量空间表示的方法已经成功地使用向量算法捕获了细粒度的语义和句法规则，但是这些规则的来源仍然是不透明的。我们分析并明确了这些规则在单词向量中出现所需的模型属性。结果是一个新的全局对数双线性回归模型，它结合了文献中两个主要模型族的优点：全局矩阵分解和局部上下文窗口方法。 我们的模型通过仅对单词-单词共现矩阵中的非零元素进行训练 ，而不是对整个稀疏矩阵或大型语料库中的单个上下文窗口进行训练，有效地利用了统计信息。该模型产生了一个具有有意义子结构的向量空间，最近一项单词类比任务中75%的表现证明了这一点。它在相似性任务和命名实体识别方面也优于相关模型。

Introduction