2021 kdd Controllable generation from pre-trained language models via inverse prompting

Zou, X., Yin, D., Zhong, Q., Yang, H., Yang, Z., & Tang, J. (2021, August). Controllable generation from pre-trained language models via inverse prompting. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (pp. 2450-2460).

这篇文章，通过inverse prompt来对控制生成的文本朝着相关联的方向生成。

Motivation

Large-scale pre-trained language models have demonstrated strong capabilities of generating realistic texts. However, it remains challenging to control the generation results. Previous approaches such as prompting are far from sufficient, and lack of controllability limits the usage of language models.

Core of this paper

The core idea of inverse prompting is to use generated text to inversely predict the prompt during beam search, which enhances the relevance between the prompt and the generated text and thus improves controllability.

inverse prompting的核心是使用生成的文本，来反向地预测prompt（在beam search阶段），这样的话就能够提高prompt与生成文本之间的相关性，进而提高controllability。

主要做法

一个样例：

这个是具体的，如何对生成的诗句进行评分的方法。

这篇文章，对我有什么启发呢？

这篇文章中采用的是一个inverse prompt的方法，来增强prompt的控制的能力。在之后的prompt的方法中（尤其是用prompt来做一些生成的任务的时候），我是不是也可以采用这篇文章中所提到的方法来加强prompt与生成结果之间的关联性。