【论文阅读】Self-Paced Curriculum Learning

论文下载
代码
Supplementary Materials
bib:

@INPROCEEDINGS{,
	 title		= {Self-Paced Curriculum Learning},
	 author	    = {Lu Jiang and Deyu Meng and Qian Zhao and Shiguang Shan and Alexander Hauptmann},
	 booktitle	= {AAAI},
	 year		= {2015},
	 pages      = {2694--2700}
}

1. 摘要

Curriculum learning (CL) or self-paced learning (SPL) represents a recently proposed learning regime inspired by the learning process of humans and animals that gradually proceeds from easy to more complex samples in training.

The two methods share a similar conceptual learning paradigm, but differ in specific learning schemes.

In CL, the curriculum is predetermined by prior knowledge, and remain fixed thereafter.

Therefore, this type of method heavily relies on the quality of prior knowledge while ignoring feedback about the learner.

In SPL, the curriculum is dynamically determined to adjust to the learning pace of the leaner.

However, SPL is unable to deal with prior knowledge, rendering it prone to overfitting.

In this paper, we discover the missing link between CL and SPL, and propose a unified framework named self-paced curriculum leaning (SPCL).

SPCL is formulated as a concise optimization problem that takes into account both prior knowledge known before training and the learning progress during training.

In comparison to human education, SPCL is analogous to “instructor-student-collaborative” learning mode, as opposed to “instructor-driven” in CL or “student-driven” in SPL.

Empirically, we show that the advantage of SPCL on two tasks.

课程学习（CL）或自定进度学习（SPL）代表了最近提出的一种学习制度，其灵感来自人类和动物的学习过程，在训练中逐渐从简单到更复杂的样本进行。 这两种方法具有相似的概念学习范式，但具体的学习方案有所不同。 在 CL 中，课程是由先验知识预先确定的，并且此后保持固定。 因此，这种方法严重依赖先验知识的质量，而忽略了学习者的反馈。 在 SPL 中，课程是动态确定的，以适应学习者的学习节奏。 然而，SPL 无法处理先验知识，因此容易出现过度拟合。 在本文中，我们发现了 CL 和 SPL 之间缺失的联系，并提出了一个名为自定进度课程学习（SPCL）的统一框架。 SPCL 被表述为一个简洁的优化问题，它考虑了训练前已知的先验知识和训练期间的学习进度。 与人类教育相比，SPCL类似于“师生协作”的学习模式，而不是CL中的“教师驱动”或SPL中的“学生驱动”。 根据经验，我们展示了 SPCL 在两项任务上的优势。

Note：

1. 课程学习依赖于课程先验，在许多场景中，课程先验一般都是缺失的。这种方法严重依赖于先验知识的质量，而忽略了学习者的反馈，相当于是老师主导。
2. 自步学习中课程是动态确定的，以适应学习者节奏。其中，动态确定只是按照loss的高低当作是样本的难易，无法处理额外加入的知识先验。

2. 算法描述

2.1. 自步学习

$$\underset{\mathbf{w},\mathbf{v}\in [0,1{]}^n}{\min }\mathbb{E}(\mathbf{w},\mathbf{v};\lambda )=\sum _{i=1}^n{v}_{i}L(y_i,f(x_i,\mathbf{w}))-\lambda \Vert \mathbf{v}{\Vert }_1\text{\hspace{1em}(1)}$$

等式1应该就是自步学习中最经典的形式了。其中，对于自步正则$-\Vert \mathbf{v}{\Vert }_1$是可以替换的，有很多的类型，这个是最经典的hard型（非0即1）。对于等式1的求解可以采用ACS (Alternative Convex Search)。

1. 固定$\mathbf{w}$，求解$\mathbf{v}$。存在闭式解${\mathbf{v}}^{\ast }=[v_1^{\ast },\dots ,v_n^{\ast }]$,
   $$v_i^{\ast }=\{\begin{array}{ll}1,& L(y_i,f(x_i,\mathbf{w}))<\lambda ;\\ 0,& \text{otherwise.}\end{array}$$
2. 固定$\mathbf{v}$，用梯度下降法（也可以用其他优化方法）求解$\mathbf{w}$。

存在的弊端:

However, since the learning is completely dominated by the training loss, the learning may be prone to overfitting. Moreover, it provides no way to incorporate prior guidance in learning. To the best of our knowledge, there has been no studies to incorporate prior knowledge into SPL, nor to analyze the relation between CL and SPL.

2.1. 自步课程学习

自步课程学习想要打造一种师生协同的学习范式，其中，同时考虑训练前已知的先验知识和训练期间学到的知识。

Self-paced Curriculum Learning:

3. 实验

from scipy.optimize import minimize
import numpy as np

# samples
id = np.array(['a', 'b', 'c', 'd', 'e', 'f'])
myloss = np.array([0.1, 0.2, 0.4, 0.6, 0.5, 0.3]).reshape([-1, 1])
print("####### input loss ######")
print(myloss)

# 4) calculate curriculum constraints
# A = matrix(0, nrow=length(id), ncol=1)
# curriculum constraints matrix
A = np.zeros([len(id), 1])
A[:, 0] = np.array([0.1, 0.0, 0.4, 0.3, 0.5, 1.0])
# A[:, 0] = np.array([2.3, 2.2, 2.1, 2.0, 1.7, 1.5])
print("####### A matrix ######")
print("A: ", A)
c = 1.0
# c = 6.0

# 5) optimize v with modality constraint (A)
# v0 = replicate(length(id),0)
v0 = np.repeat(0, len(id))
print("v0: ", v0)
# a small constant for optmization accuracy
tolerance = 1e-7
print("tolerance: ", tolerance)

# tolerance = 0
# parameter in self-paced learning
lambda_var = 0.8333

# paramaters
u1 = -1 * A  # -Av >= -c	i.e.	Av <= c

# c1 = -1 * c - tolerance  # -Av >= -c	i.e.	Av <= c
c1 = c + tolerance

# 三个约束
# v 大于等于 0
# v 小于等于 1
# A^T \times V <= c
# inequality means that it is to be non-negative.
cons = ({'type': 'ineq', 'fun': lambda v: v @ u1 + c1})


# 定义目标函数
def objective_function(v):
    obj = v @ myloss - lambda_var * np.sum(v)
    return obj


# 定义目标函数的梯度
def objective_gradient(v):
    grads = myloss - lambda_var
    return grads


# 设置bounds
# bounds = tuple([(-tolerance, 1 + tolerance) for i in range(len(v0))])
bounds = tuple([(0, 1) for i in range(len(v0))])
# x0 = np.zeros([len(id)])
# res_SLSQP = minimize(objective_function, v0, method='SLSQP', jac=objective_gradient, constraints=cons, bounds=bounds)


# res = minimize(objective_function, v0, method='SLSQP', constraints=cons, bounds=bounds)
print("###res_SLSQP###")
res_SLSQP = minimize(objective_function, v0, method='SLSQP', jac=objective_gradient, constraints=cons, bounds=bounds,
                     options={"maxiter": 100, "disp": True})
print('最小值：', res_SLSQP.fun)
print('最优解：', res_SLSQP.x)
print('迭代终止是否成功：', res_SLSQP.success)
print('迭代终止原因：', res_SLSQP.message)
print("最终解是否满足先验课程知识： ", res_SLSQP.x @ u1 + c1 >= 0)

print("###COBYLA###")
res_COBYLA = minimize(objective_function, v0, method='COBYLA', jac=objective_gradient, constraints=cons, bounds=bounds,
                      options={"maxiter": 100, "disp": True})
print('最小值：', res_COBYLA.fun)
print('最优解：', res_COBYLA.x)
print('迭代终止是否成功：', res_COBYLA.success)
print('迭代终止原因：', res_COBYLA.message)
print("最终解是否满足先验课程知识： ", res_COBYLA.x @ u1 + c1 >= 0)

# v_2 = np.array([1, 1, 1, 0.88, 0.47, 0])
v_2 = np.array([1, 0.91, 0.10, 0.00, 0.00, 1.00])
print('论文最优解：', v_2)
print("论文最小值：", objective_function(v_2))
print("论文最终解是否满足先验课程知识： ", v_2 @ u1 + c1 >= 0)