KBQA中用到强化学习的相关论文
最近对用到强化学习的论文进行了简单整理，方便之后查看。我主要是分为两类统计：基于语义解析的方法和基于信息检索的方法。具体的如下：
基于语义解析的KBQA方法
1. 原论文：GraphParser (2014 ACL)
Large-scale Semantic Parsing without Question-Answer Pairs
提到的有关强化学习的论文：
Reinforcement Learning for Mapping Instructions to Actions (发表年份:2009 发表会议：AFNLP)
本文提出了一个强化学习的方法，将natural language instructions 映射到 executable actions 序列。我们假设可以使用奖励函数来定义执行操作的质量。在训练过程中，learner重复地为一组文件构建操作序列，执行这些操作，并观察得到的奖励。我们使用一个policy gradient (策略梯度算法)来估计一个对数线性模型的参数来进行操作选择。
本文在两个领域(Windows故障排除指南和游戏教程)应用此方法解释说明 。当需要少量或者没有标注的训练样例时，此方法能比得上监督学习。
2. 原论文：TextRay(2019 CIKM)
Large-scale Semantic Parsing without Question-Answer Pairs
提到的有关强化学习的论文：
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (2017 EMNLP)
3. 原论文：SSRP(2019 IJCAI)
Neural Program Induction for KBQA Without Gold Programs or Query Annotations
提到的有关强化学习的论文：
Leveraging grammar and reinforcement learning for neural program synthesis (2018 CoRR)
Robust distant supervision relation extraction via deep reinforcement learning(2018 ACL)
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
4. 原论文：FullModel (2019 IJCAI)
Knowledge Base Question Answering with Topic Units
提到的有关强化学习的论文：
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
5. 原论文： UHop(2019 NAACL)
UHop: An Unrestricted-Hop Relation Extraction Framework for Knowledge-Based Question Answering
提到的有关强化学习的论文：
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (2017 EMNLP)
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
6. 原论文： QGG(2020 ACL)
Query Graph Generation for Answering Multi-hop Complex Questions from Knowledge Bases
提到的有关强化学习的论文：
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
7. 原论文：DAC(2020 CIKM)
Hierarchical Query Graph Generation for Complex Question Answering over Knowledge Graph
提到的有关强化学习的论文：
Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation (2016)
Human-level control through deep reinforcement learning(2015 Nature)
Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning (2017 EMNLP)
Intrinsically motivated reinforcement learning (2005)
Reinforcement learning: An introduction (2018)
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning(1999 Artificial intelligence)
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
8. 原论文：BART-large (2021 EMNLP)
Unseen Entity Handling in Complex Question Answering over KnowledgeBase via Language Generation
提到的有关强化学习的论文：
Neural symbolic machines: Learning semantic parsers on freebase with weak supervision (2017 ACL)
Learning to decompose compound questions with reinforcement learning (2019)
9. 原论文：CWQ(2018 NAACL)
The Web as a Knowledge-base for Answering Complex Questions
提到的有关强化学习的论文：
Ask the right questions: Active question reformulation with reinforcement learning
(2017 arXiv)
From language to programs: Bridging reinforcement learning and maximum marginal likelihood (2017 ACL)
Seq2sql: Generating structured queries from natural language using reinforcement learning
(2017 arXiv)
10. 原论文：MULTIQUE(2020)
Answering Complex Questions by Combining Information from Curated and Extracted Knowledge Bases
提到的有关强化学习的论文：
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (2017 EMNLP)
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
11. 原论文：AMR解析(2020)
Question Answering over Knowledge Bases by Leveraging Semantic Parsing and Neuro-Symbolic Reasoning
提到的有关强化学习的论文：
Rewarding Smatch: Transition-Based AMR Parsing with Reinforcement Learning.(2019 arXiv)
12. 原论文：Computer-Programmer(2017 ACL)
Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision
提到的有关强化学习的论文：
Deep reinforcement learning for dialogue generation(2016 arXiv)
Reinforcement Learning: An Introduction.(1998 MA)
Reinforcement learning neural turing machines(2015 arXiv)
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
13. 原论文：CBR(2021)
Case-Based Reasoning for Natural Language Queries over Knowledge Bases
提到的有关强化学习的论文：
Seq2sql: Generating structured queries from natural language using reinforcement learning (2017 arXiv)
基于信息检索的KBQA方法
14. 原论文：ReifKB(2020 arXiv)
SCALABLE NEURAL METHODS FOR REASONING WITH A SYMBOLIC KNOWLEDGE BASE
提到的有关强化学习的论文：
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (2017 EMNLP)
Seq2sql: Generating structured queries from natural language using reinforcement learning (2017 arXiv)
15. 原论文：Rigel-E2E(2021 EMNLP)
Less is more: Data-efficient complex question answering over knowledge bases.(2020)
Learning to generalize from sparse and underspecified rewards (2019 PMLR)
16. 原论文：VRN(2017 arXiv)
Variational Reasoning for Question Answering with Knowledge Graph
提到的有关强化学习的论文：
Simple statistical gradient-following algorithms for connectionist reinforcement learning(1992)
17. 原论文： IRN(2018 COLING)
An Interpretable Reasoning Network for Multi-Relation Question Answering
提到的有关强化学习的论文：
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
提到的有关强化学习的论文：
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning (2017 arXiv)
R3: Reinforced reader-ranker for open-domain question answering (2018 AAAI)
具体问题：当遇到有多个关系的问题时，现存的基于嵌入的方法将考虑以主题实体为中心的整个子图，但这会导致高的时间复杂度。与此同时，由于数据标注的高成本，对于一个复杂的问题，要一步一步地给出答案是不现实的，只有最后的答案才会被标记 用作弱监督。
SRN将multi-relation 问答转化为序列决策问题。模型在知识图谱上搜索有效路径获取答案，并利用束搜索减少候选路径的数量。与此同时，基于注意力机制和神经网络，策略梯度可以在三元组选择上增强给定问题不同部分的独特影响。此外，为了缓解由弱监督带来的延迟和稀疏奖励问题，提出来一个potential-based reward shping strategy，从而加快了训练算法的收敛速度，提高了模型的性能。
DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning (2017 EMNLP)
多跳问答算法只能收到最终答案的反馈，缺乏中间步骤的监督信号。本文提出了teacher-student 方法 。Student network目的是找到query的正确答案，teacher network目的是学习中间的监督信号(主要体现在实体消歧)以提升student network的推理能力。重点创新是在teacher network的双向推理(从主题实体到答案实体 和 从答案实体到主题实体)。
Student network 是基于Neural State Machine(NSM)实现的，NSM最先在2019年的视觉问答中使用过。Teacher network是通过加入双向推理机制修改了NSM的结构。