A Probabilistic Framework for Ensemble Learning
Abstract
This paper considers the problems of learning concepts from large-scale data sets. The way we take is completely classification algorithm independent. Firstly, the original problem is decomposed into a series of smaller two-class sub-problems which are easier to be solved. Secondly we present two principles, namely the shrink and expansion principles, to restore the global solution from the intermediate results learned from the sub-problems. In the theoretical analysis, this procedure of integration is described as a statistical inference of a posterior probability and is degraded as the min-max principles in the special case considering 0-1 outputs. We also propose a revised approach which reduces the computational complexity of the training and testing stage to a linear level . Finally, experiments on both the synthetic and text-classification data are demonstrated. The results indicate that our methods are effective to large scale problems.
Publications
Learning Concepts from Large-Scale Data Sets by Pairwise Coupling with Probabilistic Outputs
F. Zhou and B. Lu
International Joint Conference on Neural Networks (IJCNN), 2007
[Paper 1M] [Slides 1M]
Research on Ensemble Learning
F. Zhou and B. Lu
Master Thesis, 2007
[Paper 2MB (in Chinese)] [Slides 3MB (in English)]
来源： << Home
Ensemble learning algorithms:
Ensemble learning techniques have been demonstrated to be an effective way to reduce the error of a base learner across a wide variety of tasks. The basic idea is to vote together the predictions of a set of classifiers that have been trained slightly differently for the same task. This work proposed a novel ensemble learning algorithm called Triskel, which learns an ensemble of classifiers that are biased to have high precision for one particular class. Triskel outperforms boosting on a variety of real-world tasks, in terms of both accuracy and training time. It represents a middle ground between covering
algorithms and ensemble techniques such as boosting. See recent publications for more details.
来源：http://khusainr.myweb.port.ac.uk/research.html
Active learning differs from passive "learning from examples" in that the learning algorithm itself attempts to select the most informative data for training. Since supervised labeling of data is expensive, active learning attempts to reduce the human effort needed to learn an accurate result by selecting only the most informative examples for labeling. Our work has focused on diverse ensembles for active learning and applications of active learning to problems in natural-language processing and semi-supervised learning. We have also addressed the problem of actively acquiring the most useful
features values of examples as well as supervised class labels.
Ensemble Learning combines multiple learned models under the assumption that "two (or more) heads are better than one." The decisions of multiple hypotheses are combined in ensemble learning to produce more accurate results. Boosting and bagging are two popular approaches. Our work focuses on building diverse committees that are more effective than those built by existing methods, and, in particular, are useful for active learning.
For a general, popular book on the utility of combining diverse, independent opinions in human decision-making, see The Wisdom of Crowds.
Traditional machine learning algorithms operate under the assumption that learning for each new task starts from scratch, thus disregarding any knowledge they may have gained while learning in previous domains. Naturally, if the domains encountered during learning are related, this
tabula rasa approach would waste both data and computer time to develop hypotheses that could have been recovered by simply examining and possibly slightly modifying previously acquired knowledge. Moreover, the knowledge learned in earlier domains could capture generally valid rules that are not easily recoverable from small amounts of data, thus allowing the algorithm to achieve even higher levels of accuracy than it would if it starts from scratch.
The field of transfer learning, which has witnessed a great increase in popularity in recent years, addresses the problem of how to leverage previously acquired knowledge in order to improve the efficiency and accuracy of learning in a new domain that is in some way related to the original one. In particular, our current research is focused on developing transfer learning techniques for Markov Logic Networks (MLNs), a recently developed approach to statistical relational learning.
Our research in the area is currently sponsored by the Defense Advanced Research Projects Agency (DARPA) and managed by the Air Force Research Laboratory (AFRL) under contract FA8750-05-2-0283.
来源：http://www.cs.utexas.edu/~ai-lab/people-view.php?PID=362
