论文下载
bib:
@INPROCEEDINGS{PiLi2016SPBL,
title = {Self-Paced Boost Learning for Classification},
author = {Te Pi and Xi Li and Zhongfei Zhang and Deyu Meng and Fei Wu and Jun Xiao and Yueting Zhuang},
booktitle = {IJCAI},
year = {2016},
pages = {1932--1938}
}
GitHub
Effectiveness
androbustness
are two essential aspects of supervised learning studies.
For effective learning, ensemble methods are developed to build a strong effective model from ensemble of weak models.
For robust learning, self-paced learning (
SPL
) is proposed to learn in a self-controlled pace from easy samples to complex ones.
Motivated by simultaneously enhancing the learning effectiveness and robustness, we propose a unified framework, Self-Paced Boost Learning (SPBL).
With an adaptive from-easy-to-hard pace in boosting process, SPBL asymptotically guides the model to focus more on the insufficiently learned samples with higher reliability.
Via a max-margin boosting optimization with self-paced sample selection, SPBL is capable of capturing the intrinsic inter-class discriminative patterns while ensuring the reliability of the samples involved in learning.
We formulate SPBL as a fully-corrective optimization for classification.
The experiments on several real-world datasets show the superiority of SPBL in terms of both effectiveness and robustness.
Note:
Self-paced learning
(自步学习,从容易到难的学习)和Boost
(集成学习)融合在一起,同时保证有效性与鲁棒性。问题:
多分类问题
y ~ ( x ) = arg max r ∈ { 1 , … , C } F r ( x ; Θ ) (1) \widetilde{y}(x) = \argmax_{r \in \{1, \dots, C\} }F_r(x; \Theta) \tag{1} y (x)=r∈{1,…,C}argmaxFr(x;Θ)(1)
值得注意的是
, 这里相当于将多分类问题转化为了 C C C个二分类问题,对应于OvA策略。优点是只用训练类别数目 C C C个分类器,缺点是,会出现类别不平衡的问题(A对应类别样本多)。boost:
boost是一种集成学习中的一个方法,目的是集成多个弱学习器成为一个强学习器。
F r ( x ; W ) = ∑ j = 1 k w r j h j ( x ) , r ∈ { 1 , … , C } (2) F_r(x;W) = \sum_{j=1}^k w_{rj}h_j(x), r \in \{1, \dots, C\} \tag{2} Fr(x;W)=j=1∑kwrjhj(x),r∈{1,…,C}(2)
general objective of SPBL
:
min W , v ∑ i = 1 n v i ∑ r = 1 C L ( ρ i r ) + ∑ i = 1 n g ( v i ; λ ) + υ R ( W ) s . t . ∀ i , r , ρ i , r = H i : w y i − H i : w r ; W ≥ 0 ; v ∈ [ 0 , 1 ] n (3) \min_{W, v}\sum^{n}_{i=1}v_i\sum^{C}_{r=1}L(\rho_{ir}) + \sum^{n}_{i=1}g(v_i;\lambda) + \upsilon R(W) s.t. \forall i,r, \rho_{i,r} = H_{i:}w_{y_i} - H_{i:}w_{r}; W \geq 0; v \in [0, 1]^n \tag{3} W,vmini=1∑nvir=1∑CL(ρir)+i=1∑ng(vi;λ)+υR(W)s.t.∀i,r,ρi,r=Hi:wyi−Hi:wr;W≥0;v∈[0,1]n(3).
specific formulation
:
min W , v ∑ i , r v i ln ( 1 + exp ( − ρ i r ) ) + ∑ i = 1 n g ( v i ; λ ) + υ ∥ W ∥ 2 , 1 \min_{W, v}\sum_{i, r}v_i \ln(1+ \exp(-\rho_{ir})) + \sum^{n}_{i=1}g(v_i;\lambda) + \upsilon \|W\|_{2, 1} W,vmini,r∑viln(1+exp(−ρir))+i=1∑ng(vi;λ)+υ∥W∥2,1
s.t. ∀ i , r , ρ i , r = H i : w y i − H i : w r ; W ≥ 0 ; v ∈ [ 0 , 1 ] n (3) \text{s.t.} \forall i,r, \rho_{i,r} = H_{i:}w_{y_i} - H_{i:}w_{r}; W \geq 0; v \in [0, 1]^n \tag{3} s.t.∀i,r,ρi,r=Hi:wyi−Hi:wr;W≥0;v∈[0,1]n(3)
关于优化目标的求解,涉及到了对偶问题(dual problem),实在是懂不了了。