ACL2022赶会必备，拿来即用之Abstract和Related Work

每天给你送来NLP技术干货！

---

© 作者｜刘沛羽

机构｜中国人民大学高瓴人工智能学院

研究方向｜自然语言处理，模型压缩

来自 | RUC AI Box

导读：

ACL 是计算语言学和自然语言处理领域顶级国际会议，本文从ACL 2021年发表的主会长文中整理出来了其中涉及的27个子领域的100篇论文。本次整理主要有以下几个特点：

1. 一句话概括研究背景和要解决的问题。摘要中通常会用一句话概括研究背景和问题，对于新手来说，借此机会方便掌握自然语言处理领域前沿的研究方向，同时也是积累成熟的写作表达方式的机会。

2. 相关工作整理。相关工作是作者用完整逻辑串起来的文献调研。最近正是很多科研工作者投稿ACL2022的窗口期，对于正在赶会的忙于写作的同学，本文提供了较全的相关工作整理，便于查询。 

3. 涉及范围广。有一些冷门领域，由于做的人比较少，大家平时看到的论文推荐中很难发现它们的身影，但是这些论文提供了非常全面的相关背景的介绍，有助于我们扩展视野和学科交叉。

本文主要先以几个主题为主要章节，每个章节中整理论文按照标题、关键词、背景/问题以及相关工作的顺序整理。其中，“背景/问题”来源于作者在摘要中的开头几句引文，是作者经过千锤百炼修改后的表达，有助于我们对相关领域有个准确的认识。另外，“相关工作”部分，如果原文中“Related Work”没有明确的子标题，笔者会根据内容用中文进行归纳，英文则是原文中作者分类采用的子标题，读者若对具体某个问题感兴趣可以查看原文细节。

预训练语言模型

[1] How is BERT surprised? Layerwise detection of linguistic anomalies

关键词：语言模型，Transformer

背景和问题：Transformer language models have shown remarkable ability in detecting when a word is anomalous in context, but likelihood scores offer no information about the cause of the anomaly.

• Probing LMs for linguistic knowledge

• Neural grammaticality judgments

• Tests of selectional restriction

[2] Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models

关键词：Transformer

背景和问题：In this paper, we detail the relationship between convolutions and self-attention in natural language tasks. We show that relative position embeddings in self-attention layers are equivalent to recently-proposed dynamic lightweight convolutions, and we consider multiple new ways of integrating convolutions into Transformer self-attention.

[3] Consistency Regularization for Cross-Lingual Fine-Tuning

关键词：微调

背景和问题：Fine-tuning pre-trained cross-lingual language models can transfer task-specific supervision from one language to the others.

• Cross-lingual transfer

• Cross-Lingual Data Augmentation

• Consistency Regularization

[4] Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning

关键词：微调

背景和问题：Although pretrained language models can be fine-tuned to produce state-of-the-art results for a very wide range of language understanding tasks, the dynamics of this process are not well understood, especially in the low data regime.

[5] Prefix-Tuning: Optimizing Continuous Prompts for Generation

关键词：微调

背景和问题：Fine-tuning is the de facto way of leveraging large pre-trained language models for downstream tasks. However, fine-tuning modifies all the language model parameters and therefore necessitates storing a full copy for each task.

• Fine-tuning for natural language generation

• Lightweight fine-tuning

• Prompting

• Controllable generation

• P*-tuning

[6] How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

关键词：多语言模型

背景和问题：In this work, we provide a systematic and comprehensive empirical comparison of pre-trained multilingual language models versus their monolingual counterparts with regard to their monolingual task performance.

• Multilingual LMs

• Monolingual versus Multilingual LMs

[7] Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

关键词：跨语种语言模型

背景：The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences. In this paper, we introduce denoising word alignment as a new cross-lingual pre-training task.

• Cross-lingual LM pre-training

• Word alignment

[8] GhostBERT: Generate More Features with Cheap Operations for BERT

关键词：模型压缩

背景和问题：Transformer-based pre-trained language mod- els like BERT, though powerful in many tasks, are expensive in both memory and computa- tion, due to their large number of parameters. Previous works show that some parameters in these models can be pruned away without se- vere accuracy drop.

• Network Pruning in Transformer

• Enhanced Representation in Transformer-based Models

[9] Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

关键词：模型压缩，彩票理论

背景和问题：The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of “lottery tickets”, and training a certain collection of them (i.e.,a subnetwork) can match the performance of the full model.

• Transformer Architecture

• Structured and Unstructured LTHs

[10] Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

关键词：模型压缩，知识蒸馏

背景和问题：Pre-trained language models have been applied to various NLP tasks with considerable performance gains. However, the large model sizes, together with the long inference time, limit the deployment of such models in realtime applications.

• Knowledge Distillation (KD)

• PLM Compression

• Transfer Learning and Meta-learning

[11] UnNatural Language Inference

关键词：自然语言理解任务

背景和问题：Recent investigations into the inner-workings of state-of-the-art large-scale pre-trained Transformer-based Natural Language Understanding (NLU) models indicate that they appear to know humanlike syntax, at least to some extent.

• 模型可以获得语法信息——Models appear to have acquired syntax

• 模型到底可以获得多少的语法信息似乎又是不明确的——Models appear to struggle with syntax

• 模型对于乱序的不敏感性——Insensitivity to Perturbation

• 语言推断模型过度关注特定单词来进行预测——NLI Models are very sensitive to words

[12] Semi-Supervised Text Classification with Balanced Deep Representation Distributions

关键词：半监督的文本分类问题

背景：Semi-Supervised Text Classification (SSTC) mainly works under the spirit of self-training. They initialize the deep classifier by training over labeled texts; and then alternatively predict unlabeled texts as their pseudo-labels and train the deep classifier over the mixture of labeled and pseudo-labeled texts.

• 在无标签数据上使用VAE预训练生成样本，再分类——Variational Methods for Pretraining In Resourcelimited Environments

• 深度自训练——Deep self-training

[13] DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations

关键词：句子嵌入表示

背景：Sentence embeddings are an important component of many natural language processing (NLP) systems.

• Supervised or semi-supervised

• Unsupervised

[14] LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding

关键词：多模态预训练

背景：Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents.

[15] Automated Concatenation of Embeddings for Structured Prediction

关键词：上下文信息预训练

背景：Pretrained contextualized embeddings are powerful word representations for structured prediction tasks. Recent work found that better word representations can be obtained by concatenating different types of embeddings

• Embeddings

• Neural Architecture Search

[16] IrEne: Interpretable Energy Prediction for Transformers

关键词：预训练模型能量消耗预测

背景和问题：Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution

[17] Syntax-Enhanced Pre-trained Model

关键词：预训练语言模型

背景和问题：We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages.

• Probing Pre-trained Models

• Integrating Syntax into Pre-trained Models

[18] PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction

关键词：拼写修正

背景和问题：Chinese spelling correction (CSC) is a task to detect and correct spelling errors in texts. CSC is essentially a linguistic problem, thus the ability of language understanding is crucial to this task.

• 中文拼写修正

[19] EnsLM: Ensemble Language Model for Data Diversity by Semantic Clustering

关键词：集成

背景：Natural language processing often faces the problem of data diversity such as different domains, themes, styles and so on.

• 主题模型和语言模型（Topic model and language model）

[20] StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling

关键词：语法信息

背景：There are two major classes of natural language grammars — the dependency grammar that models one-to-one correspondences between words and the constituency grammar that models the assembly of one or several corresponded words.  问题：While previous unsupervised parsing methods mostly focus on only inducing one class of grammars, we introduce a novel model, StructFormer, that can simultaneously induce dependency and constituency structure.

• 基于依赖关系（dependency grammar）

• 基于选区关系（constituency grammar）

检测

[21] Towards Propagation Uncertainty: Edge-enhanced Bayesian Graph Convolutional Networks for Rumor Detection

关键词：谣言检测

背景：Detecting rumors on social media is a very critical task with significant implications to the economy, public health, etc. Previous works generally capture effective features from texts and the propagation structure.

• Rumor Detection

• GNN

[22] PROTAUGMENT: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning

关键词：意图检测

背景：Recent research considers few-shot intent detection as a meta-learning problem: the model is learning to learn from a consecutive set of small tasks named episodes.

• 意图检测——Intent detection

[23] Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training

关键词：意图检测

背景：Out-of-scope intent detection is of practical importance in task-oriented dialogue systems.

• Out-of-Distribution Detection

• Out-of-Scope Intent Detection

[24] Supporting Cognitive and Emotional Empathic Writing of Students

关键词：情绪检测

背景：We present an annotation approach to capturing emotional and cognitive empathy in student-written peer reviews on business models in German.

• The Construct of Empathy

• Emotion and Empathy Detection

• Empathy Annotated Corpora and Annotation Schemes

[25] Position Bias Mitigation: A Knowledge-Aware Graph Model for Emotion Cause Extraction

关键词：情绪因果分析

背景：The Emotion Cause Extraction (ECE) task aims to identify clauses which contain emotion-evoking information for a particular emotion expressed in text.

• Position-insensitive Models.

• Position-aware Models.

[26] Psycholinguistic Tripartite Graph Network for Personality Detection

关键词：个性检测

背景和问题：Most of the recent work on personality detection from online posts adopts multifarious deep neural networks to represent the posts and builds predictive models in a data-driven manner, without the exploitation of psycholinguistic knowledge that may unveil the connections between one’s language usage and his psychological traits.

• Personality Detection

• Graph Neural Networks

[27] LearnDA: Learnable Knowledge-Guided Data Augmentation for Event Causality Identification

关键词：事件因果检测

背景：Modern models for event causality identification (ECI) are mainly based on supervised learning, which are prone to the data lacking problem.

[28] Unified Dual-view Cognitive Model for Interpretable Claim Verification

关键词：虚假消息验证

背景：Recent studies constructing direct interactions between the claim and each single user response to capture evidence have shown remarkable success in interpretable claim verification.

• 自动的验证方法——Automatic verification approaches

• 可解释的验证方法——interpretable claim verification

机器翻译

[29] Vocabulary Learning via Optimal Transport for Neural Machine Translation

关键词：机器翻译，词典选择

背景和问题：The choice of token vocabulary affects the performance of machine translation. This paper aims to figure out what is a good vocabulary and whether one can find the optimal vocabulary without trial training.

[30] Rewriter-Evaluator Architecture for Neural Machine Translation

关键词：机器翻译

背景和问题：A few approaches have been developed to improve neural machine translation (NMT) models with multiple passes of decoding. However, their performance gains are limited because of lacking proper policies to terminate the multi-pass process.

• Multi-pass decoding

[31] Neural Machine Translation with Monolingual Translation Memory

关键词：Translation memory

背景：Prior work has proved that Translation memory (TM) can boost the performance of Neural Machine Translation (NMT).

• TM-augmented NMT

• Retrieval for Text Generation

• NMT using Monolingual Data

[32] Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers

关键词：评估

背景：This paper presents the first large-scale meta evaluation of machine translation (MT).

• A Survey on MT Evaluation

[33] Crafting Adversarial Examples for Neural Machine Translation

关键词：对抗生成

背景：Effective adversary generation for neural machine translation (NMT) is a crucial prerequisite for building robust machine translation systems.

[34] SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation

关键词：预训练，机器翻译

背景：While pre-training techniques are working very well in natural language processing, how to pre-train a decoder and effectively leverage it for neural machine translation (NMT) still remains a tricky issue.

• 预训练（Pre-training）

• 机器翻译中的预训练：

• 预训练做特征抽取（feature extractor）

• 预训练整个的seq2seq模型【MASS，BART，CSP，mRASP】

[35] Improving Zero-Shot Translation by Disentangling Positional Information

关键词：小样本，机器翻译

背景：Multilingual neural machine translation has shown the capability of directly translating between language pairs unseen in training, i.e. zero-shot translation. 问题：Despite being conceptually attractive, it often suffers from low output quality

• 全参数共享（models with full parameter sharing have gained popularity）

• 单语种预训练（monolingual pre-training）

[36] Fast and Accurate Neural Machine Translation with Translation Memory

关键词：快速机器翻译

背景和问题：It is generally believed that a translation memory (TM) should be beneficial for machine translation tasks. Unfortunately, existing wisdom demonstrates the superiority of TMbased neural machine translation (NMT) only on the TM-specialized translation tasks rather than general tasks, with a non-negligible computational overhead.

• 统计机器翻译（statistical machine translation, SMT）

• TM-based NMT

对话系统

[37] Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking

关键词：对话状态跟踪

背景：Existing dialog state tracking (DST) models are trained with dialog data in a random order, neglecting rich structural information in a dataset.

• Curriculum Learning

[38] TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systems

关键词：对话系统

背景：Building a dialog system that handles human conversational behavior is challenging because it must respond sensibly and relevantly to a wide variety of context-sensitive user input over multiple conversation turns.

• Datasets

• Modular vs. end-to-end architectures

[39] SocAoG: Incremental Graph Parsing for Social Relation Inference in Dialogues

关键词：社交关系，对话系统

背景：Inferring social relations from dialogues is vital for building emotionally intelligent robots to interpret human language better and act accordingly.

• Relation Inference from Documents

• Relation Inference from Dialogues

[40] Comprehensive Study: How the Context Information of Different Granularity Affects Dialogue State Tracking?

关键词：对话系统

背景：Dialogue state tracking (DST) plays a key role in task-oriented dialogue systems to monitor the user’s goal.

[41] A Joint Model for Dropped Pronoun Recovery and Conversational Discourse Parsing in Chinese Conversational Speech

关键词：删除代词恢复

背景：Pronouns are often dropped in Chinese conversations as the identity of the pronoun can be inferred from the context without causing the sentence to be incomprehensible.

[42] Discovering Dialog Structure Graph for Coherent Dialog Generation

关键词：对话结构

背景：Learning discrete dialog structure graph from human-human dialogs yields basic insights into the structure of conversation, and also provides background knowledge to facilitate dialog generation.   问题：However, this problem is less studied in open-domain dialogue

• Dialog structure learning for task-oriented dialogs

• Knowledge aware conversation generation

• Latent variable models for chitchat

[43] Dialogue Response Selection with Hierarchical Curriculum Learning

关键词：对话回复选择

背景：Motivated by the recent finding that models trained with random negative samples are not ideal in real-world scenarios, we propose a hierarchical curriculum learning framework that trains the matching model in an “easy-to-difficult” scheme.

• Dialogue Response Selection

• Curriculum Learning

[44] Directed Acyclic Graph Network for Conversational Emotion Recognition

关键词：情感识别

背景：The modeling of conversational context plays a vital role in emotion recognition from conversation (ERC).

• Emotion Recognition in Conversation

• Directed Acyclic Graph Neural Network

[45] Diversifying Dialog Generation via Adaptive Label Smoothing

关键词：对话生成

背景：Neural dialogue generation models trained with the one-hot target distribution suffer from the over-confidence issue, which leads to poor generation diversity as widely reported in the literature.

• Diversity Promotion

• Confidence Calibration

[46] BoB: BERT Over BERT for Training Persona-based Dialogue Models from Limited Personalized Data

关键词：对话系统，Persona-based Dialogue

背景和问题：Maintaining consistent personas is essential for dialogue agents.Although tremendous advancements have been brought, the limitedscale of annotated persona-dense data are still barriers towards training robust and consistent persona-based dialogue models.

• Persona-based Dialogues

• Disentangled Representation

• Unlikelihood Training

文本生成

[47] Generalising Multilingual Concept-to-Text NLG with Language Agnostic Delexicalisation

关键词：概念到文本的生成

背景：Concept-to-text Natural Language Generation is the task of expressing an input meaning representation in natural language. 问题：Previous approaches in this task have been able to generalise to rare or unseen instances by relying on a delexicalisation of the input.

• 多语言生成——Multilingual Generation Techniques

[48] Competence-based Multimodal Curriculum Learning for Medical Report Generation (Short)

关键词：医疗报告

背景：Medical report generation task, which targets to produce long and coherent descriptions of medical images, has attracted growing research interests recently.Different from the general image captioning tasks, medical report generation is more challenging for data-driven neural models  问题：This is mainly due to 1) the serious data bias and 2) the limited medical data.

[49] Control Image Captioning Spatially and Temporally

关键词：图像描述生成

背景：Generating image captions with user intention is an emerging need. The recently published Localized Narratives dataset takes mouse traces as another input to the image captioning task, which is an intuitive and efficient way for a user to control what to describe in the image.  问题：However, how to effectively employ traces to improve generation quality and controllability is still under exploration.

• Controllable Image Captioning

• Contrastive Learning

[50] Hierarchical Context-aware Network for Dense Video Event Captioning

关键词：视频字幕生成

背景：Dense video event captioning aims to generate a sequence of descriptive captions for each event in a long untrimmed video. Video-level context provides important information and facilities the model to generate consistent and less redundant captions between events.

• Video Captioning

• Multi-modal Video Captioning

• Context-aware Language Generation

[51] BACO: A Background Knowledge- and Content-Based Framework for Citing Sentence Generation

关键词：引用描述生成

背景：In this paper, we focus on the problem of citing sentence generation, which entails generating a short text to capture the salient information in a cited paper and the connection between the citing and cited paper.

• 基于关键词的摘要方法（keyword-based summarization methods）

• 抽象的摘要方法（abstractive summarization approaches）

[52] Mention Flags (MF): Constraining Transformer-based Text Generators

关键词：文本生成，Transformer

背景和问题：This paper focuses on Seq2Seq (S2S) constrained text generation where the text generator is constrained to mention specific words, which are inputs to the encoder, in the generated outputs.

• Training S2S Models

• Constrained Decoding

[53] Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting

关键词：难度可控

背景：This paper explores the task of DifficultyControllable Question Generation (DCQG), which aims at generating questions with required difficulty levels.  问题：Previous research on this task mainly defines the difficulty of a question as whether it can be correctly answered by a Question Answering (QA) system, lacking interpretability and controllability.

• Deep Question Generation

• Difficulty-Controllable Question Generation

• Question Rewriting

[54] OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics

关键词：自动指标

背景：Automatic metrics are essential for developing natural language generation (NLG) models, particularly for open-ended language generation tasks such as story generation.  问题：However, existing automatic metrics are observed to correlate poorly with human evaluation. The lack of standardized benchmark datasets makes it difficult to fully evaluate the capabilities of a metric and fairly compare different metrics

[55] Improving Encoder by Auxiliary Supervision Tasks for Table-to-Text Generation

关键词：表格转文本

背景：Table-to-text generation aims at automatically generating natural text to help people conveniently obtain salient information in tables.  问题：Previous methods cannot deduce the factual results from the entity’s (player or team) performance and the relations between entities.

• 引入外部知识的生成（introducing external knowledge）

• 显式建模表格结构（explicitly modeling the table’s structure）

[56] Improving Factual Consistency of Abstractive Summarization via Question Answering

关键词：事实一致性

背景：A commonly observed problem with the stateof-the art abstractive summarization models is that the generated summaries can be factually inconsistent with the input documents.  问题：The fact that automatic summarization may produce plausible-sounding yet inaccurate summaries is a major concern that limits its wide application.

[57] Long-Span Summarization via Local Attention and Content Selection

关键词：摘要生成

背景：Transformer-based models have achieved state-of-the-art results in a wide range of natural language processing (NLP) tasks including document summarization.Typically these systems are trained by fine-tuning a large pretrained model to the target task.  问题：One issue with these transformer-based models is that they do not scale well in terms of memory and compute requirements as the input length grows. Thus, for long document summarization, it can be challenging to train or fine-tune these models

[58] Cross-Lingual Abstractive Summarization with Limited Parallel Resources

关键词：跨语言

背景和问题：Parallel cross-lingual summarization data is scarce, requiring models to better use the limited available cross-lingual resources. Existing methods to do so often adopt sequenceto-sequence networks with multi-task frameworks. Such approaches apply multiple decoders, each of which is utilized for a specific task.

• Cross-Lingual Summarization

• Low-Resource Natural Language Generation

[59] Writing by Memorizing: Hierarchical Retrieval-based Medical Report Generation

关键词：医疗报告

背景：Medical report generation is one of the most challenging tasks in medical image analysis.  问题：Although existing approaches have achieved promising results, they either require a predefined template database in order to retrieve sentences or ignore the hierarchical nature of medical report generation

• Generation-based report generation

• Retrieval-based report generation

[60] Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech

关键词：counter narrative generation

背景和问题：Undermining the impact of hateful content with informed and non-aggressive responses, called counter narratives, has emerged as a possible solution for having healthier online communities.Thus, some NLP studies have started addressing the task of counter narrative generation.

• Hate detection datasets

• Hate countering datasets

• Hybrid models for data collection

[61] Factorising Meaning and Form for Intent-Preserving Paraphrasing

关键词：文本生成（生成语义相同但是形式不同的提问）

背景和问题：We propose a method for generating paraphrases of English questions that retain the original intent but use a different surface form.

• Paraphrasing

• Syntactic Templates

事件类

[62] OntoED: Low-resource Event Detection with Ontology Embedding

关键词：事件探测

背景：Event Detection (ED) aims to identify event trigger words from a given text and classify it into an event type.  问题：Most of current methods to ED rely heavily on training instances, and almost ignore the correlation of event types. Hence, they tend to suffer from data scarcity and fail to handle new unseen event types

• 基于监督学习

• 基于元学习

• 基于知识增强和迁移学习

[63] Conditional Generation of Temporally-ordered Event Sequences

关键词：事件排序

背景和问题：Models of narrative schema knowledge have proven useful for a range of event-related tasks, but they typically do not capture the temporal relationships between events.

• Temporal Event Ordering

• Schema Induction

[64] Document-level Event Extraction via Heterogeneous Graph-based Interaction Model with a Tracker

关键词：事件抽取

背景：Document-level event extraction aims to recognize event information from a whole piece of article.  问题：Existing methods are not effective due to two challenges of this task: a) the target event arguments are scattered across sentences; b) the correlation among events in a document is non-trivial to model

• Sentence-level Event Extraction

• Document-level Event Extraction

语句解析

[65] Adversarial Learning for Discourse Rhetorical Structure Parsing

关键词：修辞结构

背景：Text-level discourse rhetorical structure (DRS) parsing is known to be challenging due to the notorious lack of training data.   问题：Although recent top-down DRS parsers can better leverage global document context and have achieved certain success, the performance is still far from perfect.

因果关系

[66] Knowledge-Enriched Event Causality Identification via Latent Structure Induction Networks

关键词：事件因果关系识别

背景：Identifying causal relations of events is an important task in natural language processing area.  问题：However, the task is very challenging, because event causality is usually expressed in diverse forms that often lack explicit causal clues.  

• Event causality identification

事实检测

[67] Claim Matching Beyond English to Scale Global Fact-Checking

关键词：事实检测，多语种

背景：Manual fact-checking does not scale well to serve the needs of the internet. This issue is further compounded in non-English contexts

• Semantic Textual Similarity

• Multilingual Embedding Models

• Claim Matching

[68] Joint Verification and Reranking for Open Fact Checking Over Tables

关键词：表格数据

背景：Structured information is an important knowledge source for automatic verification of factual claims.  问题：Nevertheless, the majority of existing research into this task has focused on textual data, and the few recent inquiries into structured data have been for the closed-domain setting where appropriate evidence for each claim is assumed to have already been retrieved.

• 基于大规模表格数据的问答（Semantic querying against large collections of tables）

• 基于表格数据的封闭域的语义提取（Closed-domain semantic parsing over tables）

• （Open-domain fact verification and question answering over unstructured, textual data）

[69] Evidence-based Factual Error Correction

关键词：错误纠正

背景：This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence. This extends the well-studied task of fact verification by providing a mechanism to correct written texts that are refuted or only partially supported by evidence.  问题：We demonstrate that it is feasible to train factual error correction systems from existing fact checking datasets which only contain labeled claims accompanied by evidence, but not the correction.

• 基于句子的修改（Make corrections to sentences）

• 语法错误修改（Grammatical Error Correction）

• 借助PLM中的知识进行修改

情感分析

[70] Dual Graph Convolutional Networks for Aspect-based Sentiment Analysis

关键词：情感分析，图卷积网络

背景：Aspect-based sentiment analysis is a finegrained sentiment classification task. Recently, graph neural networks over dependency trees have been explored to explicitly model connections between aspects and opinion words.

• 基于注意力机制的模型——attention-based neural networks

• 显式的引入句法知识——syntactic knowledge

• 句法树——use GCN and GAN by means of a syntactical dependency tree

[71] Multi-Label Few-Shot Learning for Aspect Category Detection

关键词：情感分析

背景：Aspect category detection (ACD) in sentiment analysis aims to identify the aspect categories mentioned in a sentence.

• Aspect Category Detection

• Few-Shot Learning

• Multi-Label Few-Shot Learning

[72] DynaSent: A Dynamic Benchmark for Sentiment Analysis

关键词：情感分析

背景：We introduce DynaSent (‘Dynamic Sentiment’), a new English-language benchmark task for ternary (positive/negative/neutral) sentiment analysis.

• Sentiment Benchmarks

• Challenge and Adversarial Datasets

• Crowdsourcing Methods

手语

[73] Including Signed Languages in Natural Language Processing

关键词：手语

背景：Signed languages are the primary means of communication for many deaf and hard of hearing individuals.

数据样本

[74] Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering

关键词：异常点，在线学习

背景：Active learning promises to alleviate the massive data needs of supervised machine learning: it has successfully improved sample efficiency by an order of magnitude on traditional tasks like topic classification and object recognition.

• Active Learning

• Visual Question Answering

• Interpreting and Analyzing Datasets

[75] Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models

关键词：反事实样本

背景：While counterfactual examples are useful for analysis and training of NLP models, current generation methods either rely on manual labor to create very few counterfactuals, or only instantiate limited types of perturbations such as paraphrases or word substitutions.

• 基于人工和专家的方法

• 自动生成反事实样本

对抗攻击

[76] Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble

关键词：对抗攻击

背景：Although deep neural networks have achieved prominent performance on many NLP tasks, they are vulnerable to adversarial examples.

• 文本领域使用对抗训练来防御——adversarial training

• 认证防御——certified defenses

• 图像领域使用随机来防御——In the image domain, randomization has been shown to overcome many of these obstacles in the IBP-based defense.

[77] A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger’s Adversarial Attacks

关键词：对抗攻击

背景：The Universal Trigger (UniTrigger) is a recently-proposed powerful adversarial textual attack method. Utilizing a learning-based mechanism, UniTrigger generates a fixed phrase that, when added to any benign inputs, can drop the prediction accuracy of a textual neural network (NN) model to near zero on a target class.

• Adversarial Text Detection

• Honeypot-based Adversarial Detection

文本可解释性

[78] KACE: Generating Knowledge-Aware Contrastive Explanations for Natural Language Inference

关键词：模型可解释性

背景：In order to better understand the reason behind model behaviors (i.e., making predictions), most recent work has exploited generative models to provide complementary explanations.

• Counterfactual Example Generation

• Post-hoc Explanation Generation

• Natural Language Inference

[79] Improving the Faithfulness of Attention-based Explanations with Task-specific Information for Text Classification

关键词：可解释性

背景：Neural network architectures in natural language processing often use attention mechanisms to produce probability distributions over input token representations. Attention has empirically been demonstrated to improve performance in various tasks, while its weights have been extensively used as explanations for model predictions.

• Model Interpretability

• Attention as Explanation

语义消歧

[80] Word Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives

关键词：语义消歧

背景：Lately proposed Word Sense Disambiguation (WSD) systems have approached the estimated upper bound of the task on standard evaluation benchmarks.

• Supervised Method

• Context Exploitation

[81] R2D2: Recursive Transformer based on Differentiable Tree for Interpretable Hierarchical Language Modeling

关键词：Transformer模型，语义消歧

背景：Human language understanding operates at multiple levels of granularity (e.g., words, phrases, and sentences) with increasing levels of abstraction that can be hierarchically combined.   

问题：However, existing deep models with stacked layers do not explicitly model any sort of hierarchical process.

相关工作：

• Pre-trained models

• Representation with structures

自动问答

[82] Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering

关键词：开放域问答

背景：The current state-of-the-art generative models for open-domain question answering (ODQA) have focused on generating direct answers from unstructured textual information  问题：However, a large amount of world’s knowledge is stored in structured databases, and need to be accessed using query languages such as SQL.

• Open Domain Question Answering

• Table Parsing

• Hybrid QA

[83] Explanations for CommonsenseQA: New Dataset and Models

关键词：常识问答

背景：CommonsenseQA (CQA) (Talmor et al., 2019) dataset was recently released to advance the research on common-sense question answering (QA) task.

[84] Answering Ambiguous Questions through Generative Evidence Fusion and Round-Trip Prediction

关键词：开放域问答，歧义

背景：In open-domain question answering, questions are highly likely to be ambiguous because users may not know the scope of relevant topics when formulating them.  问题：Therefore, a system needs to find possible interpretations of the question, and predict one or multiple plausible answers.

• 提问包含歧义（question ambiguity problem）

[85] CoSQA: 20,000+ Web Queries for Code Search and Question Answering

关键词：代码查询

背景：Finding codes given natural language query is beneficial to the productivity of software developers. Future progress towards better semantic matching between query and code requires richer supervised training resources

多模态

[86] PhotoChat: A Human-Human Dialogue Dataset With Photo Sharing Behavior For Joint Image-Text Modeling

关键词：对话数据集

背景：We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in online messaging.

• Image-text Dataset

• Image-text Modeling

[87] Text-Free Image-to-Speech Synthesis Using Learned Segmental Units

关键词：图片转语音

背景：In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision.

• Image-to-Text and Image-to-Speech Captioning.

• Voice Conversion without Text

• Speech Pre-Training and Its Applications

[88] Maria: A Visual Experience Powered Conversational Agent

关键词：会话代理

背景：Arguably, the visual perception of conversational agents to the physical world is a key way for them to exhibit the human-like intelligence.Image-grounded conversation is thus proposed to address this challenge.  问题：Existing works focus on exploring the multimodal dialog models that ground the conversation on a given image.

• Vision and Language

• Dialog Generation

命名实体识别

[89] Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data

关键词：命名实体识别

背景：Weak supervision has shown promising results in many natural language processing tasks, such as Named Entity Recognition (NER).  问题：Existing work mainly focuses on learning deep NER models only with weak supervision, i.e., without any human annotation, and shows that by merely using weakly labeled data, one can achieve good performance, though still underperforms fully supervised NER with manually/strongly labeled data

[90] Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition

关键词：嵌套命名实体（Nested Named Entity）

背景：Named entity recognition (NER) is a wellstudied task in natural language processing  问题：Traditional NER research only deals with flat entities and ignores nested entities.

• Nested Named Entity Recognition

• Object Detection

[91] Subsequence Based Deep Active Learning for Named Entity Recognition

关键词：在线学习

背景：Active Learning (AL) has been successfully applied to Deep Learning in order to drastically reduce the amount of data required to achieve high performance. Previous works have shown that lightweight architectures for Named Entity Recognition (NER) can achieve optimal performance with only 25% of the original training data. 问题：However, these methods do not exploit the sequential nature of language and the heterogeneity of uncertainty within each instance, requiring the labelling of whole sentences.

[92] BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition

关键词：弱监督

背景：We study the problem of learning a named entity recognition (NER) tagger using noisy labels from multiple weak supervision sources.  问题：Though cheap to obtain, the labels from weak supervision sources are often incomplete, inaccurate, and contradictory, making it difficult to learn an accurate NER model.

• Weakly Supervised NER

• Neuralizing the Hidden Markov Model

[93] A Large-Scale Chinese Multimodal NER Dataset with Speech Clues

关键词：多模态，命名实体识别

背景：In this paper, we aim to explore an uncharted territory, which is Chinese multimodal named entity recognition (NER) with both textual and acoustic contents.

• Multimodal NER

• Chinese NER

自动解数学题

[94] Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning

关键词：自动解题

背景：Geometry problem solving has attracted much attention in the NLP community recently. The task is challenging as it requires abstract problem understanding and symbolic reasoning with axiomatic knowledge.

• Datasets for Geometry Problem Solving

• Approaches for Geometry Problem Solving

• Interpretable Math Problem Solving

序数分类

[95] Evaluating Evaluation Measures for Ordinal Classification and Ordinal Quantification

关键词：序数分类

背景：Ordinal Classification (OC) is an important classification task where the classes are ordinal.

• Evaluating Ordinal Classification

• Evaluating Ordinal Quantification

协同过滤

[96] Unsupervised Extractive Summarization-Based Representations for Accurate and Explainable Collaborative Filtering

关键词：协同过滤

背景和问题：We pioneer the first extractive summarizationbased collaborative filtering model called ESCOFILT. Our proposed model specifically produces extractive summaries for each item and user. Unlike other types of explanations, summary-level explanations closely resemble real-life explanations

• DeepCoNN（first deep learning-based model representing users and items from reviews in a coordinated manner）

• 基于注意力机制的模型

信息检索

[97] Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval

关键词：伪查询嵌入

背景：Recently, the retrieval models based on dense representations have been gradually applied in the first stage of the document retrieval tasks, showing better performance than traditional sparse vector space models. To obtain high efficiency, the basic structure of these models is Bi-encoder in most cases.  问题：However, this simple structure may cause serious information loss during the encoding of documents since the queries are agnostic.

文本分类

[98] Concept-Based Label Embedding via Dynamic Routing for Hierarchical Text Classification

关键词：层次化文本分类

背景：Hierarchical Text Classification (HTC) is a challenging task that categorizes a textual description within a taxonomic hierarchy.  问题：Most of the existing methods focus on modeling the text. Recently, researchers attempt to model the class representations with some resources (e.g., external dictionaries). However, the concept shared among classes which is a kind of domain-specific and fine-grained information has been ignored in previous work.

• Hierarchical text classification with label embeddings

• Hierarchical text classification besides label embeddings

主题一致性

[99] Evaluation of Thematic Coherence in Microblogs

背景：Collecting together microblogs representing opinions about the same topics within the same timeframe is useful to a number of different tasks and practitioners.  问题：A major question is how to evaluate the quality of such thematic clusters.

• Measures of topic model coherence

• Text Generation Metrics

阅读理解

[100] Coreference Reasoning in Machine Reading Comprehension

关键词：Coreference resolution

背景：Coreference resolution is essential for natural language understanding and has been long studied in NLP. In recent years, as the format of Question Answering (QA) became a standard for machine reading comprehension (MRC), there have been data collection efforts, e.g., Dasigi et al. (2019), that attempt to evaluate the ability of MRC models to reason about coreference.  问题：However, as we show, coreference reasoning in MRC is a greater challenge than earlier thought; MRC datasets do not reflect the natural distribution and, consequently, the challenges of coreference reasoning.

• Artifacts in NLP datasets

• Joint QA and Coreference Reasoning

---

投稿或交流学习，备注：昵称-学校（公司）-方向，进入DL&NLP交流群。

方向有很多：机器学习、深度学习，python，情感分析、意见挖掘、句法分析、机器翻译、人机对话、知识图谱、语音识别等。

记得备注呦

整理不易，还望给个在看！