Curriculum Learning

从一篇论文学习“课程学习”，着重关注Curriculum Learning，在此简单记录相关内容。

 

论文题目：Curriculum Learning for Natural Answer Generation（中科院自动化所）

论文地址：https://www.ijcai.org/proceedings/2018/0587.pdf

 

一、简介参考链接：https://www.sohu.com/a/236007345_100118081

1.我们的工作主要针对目前训练数据质量参差不齐的现象而设计的模型。

2.课程学习，它是机器学习中的一个训练策略，简单说就是先学习简单训练数据，再学习复杂训练数据。

3.现在的训练数据下，面临两个挑战：一，难以选出高质量的训练数据。二，选出高质量的数据，数据的规模会明显下降。

4.首先从低质量的、简短的训练数据中学出一个基本模型，接下来从高质量的、长的复杂的训练数据中学习出一个更好的模型。

5.两个问题，1确定数据的复杂度，需要和数据质量联系在一起。2确定学习进度，什么时候学习简单的，什么时候学习复杂的。

5.1.确定数据的复杂度：一个是根据词语频率选择，第二个是根据语法选择。

5.2.确定学习进度：课程学习对普通的实例是递减的，对目标实例是递增的。（这里具体要怎么做？？）

 

二、论文中章节3.2 Instance Scheduler是这样写的：

Similar to Sachan and Xing [2016], we formalize the idea as follows. Let w ∈ [0, 1]|Q| represent the probability of sampling in each QA-pair, where |Q| is the size of QA-pairs, and w is related to the complexity of QA-pairs and progress (such as the number of current training epoch). Common instances and target instances are marked as Qc and Qt, respectively. At first, the model tends to select common instances, so wQc ≫ wQt. Subsequently, wQc decreases and wQt increases monotonically. Eventually, wQc ≪ wQt , which means that the model is favor of target instances. The probabilities on the target and common instances are as follows. wQt = ( epocht / |epoch| )2 (1) wQc = 1 − wQt (2) where, epocht and |epoch| are the number of current epoch and entire epochs in training, respectively. Moreover, the probability of sampling is normalized by the accumulated probabilities on all samples.

该论文中只有这一段简单的描述，并引用论文Sachan and Xing [2016]

于是我们找到该论文

 

三、论文题目：Easy Questions First? A Case Study on Curriculum Learning for Question Answering

论文链接：http://www.cs.cmu.edu/~epxing/papers/2016/Sachan_Xing_ACL16a.pdf