#Paper Reading# Lifelong Machine Learning Systems: Beyond Learning Algorithms

论文题目：Lifelong Machine Learning Systems: Beyond Learning Algorithms
论文地址：http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.696.7800&rep=rep1&type=pdf

论文大体内容：
本文对比、分析了以前的LML（Lifelong Machine Learning）式的研究，提出更系统化的LML Framework，并阐述了LML的一些challenges和benefits。作者主要强调对于AI组织，现在是合适的时期去大力推广LML的计算方式。

1、Supervised Learning的LML式研究
①Constructive Inductive Learning[1]：两种解决方案，(i)给hypothesis找最好的表达空间；(ii)在当前空间找最好的hypothesis；
②Incremental Learning[2]；
③Explanation-Based Neural Networks[3]：EBNN is able to transfers knowledge across multiple learning tasks；
④Sequential Learning and Consolidation Systems[4]；
⑤Knowledge-Based Cascade-correlation Neural Networks[5]；

2、Unsupervised Learning的LML式研究
①ART (Adaptive Resonance Theory) neural networks[14]：Unsupervised ART networks learn a mapping between “bottom-up” input sensory
nodes and “top-down” expectation nodes (or cluster nodes)；
②Cluster Ensemble Framework[6]：reuse previous partitionings of a set objects without accessing the original features；
③Self-taught Learning method[7]：build high-level features using unlabeled data for a set of tasks；
④Never-Ending Language Learner[8]：each day must (1) extract, or read, information from the web to populate a growing structured knowledge base, and (2) learn to perform this task better than on the previous day；
⑤Deep Architectures of Neural Networks[9];
⑥Deep Learning method[15];

3、Reinforcement Learning的LML式研究
①Continual Learning[10]：The system can efficiently solve reinforcement-learning tasks and can then transfer its skills to related but more complicated tasks；
②A lifelong reinforcement learning method for autonomous-robots by treating multiple environments as multiple-tasks[11]；
③Used prior knowledge to reduce the hypothesis space for reinforcement learning[12]；
④Learning should continue during an agent’s operations since the environment may change making prior learning insufficient[13]：an agent is proposed to adapt to different local environments when encountering different parts of its world over an extended period of time；

4、为什么需要LML
①Inductive Bias is Essential to Learning；
②LML在AI问题上有理论上的进展；
③Practical Agents/Robots Require LML；
④计算能力在不断增加，使得LML成为可能；

5、反对者的观点
①Machine Learning应该关注基础算法的研究，而非分心于系统的应用；
②LML太泛，并且花费时间代价巨大；

6、LML定义
Lifelong Machine Learning, or LML, considers systems that can learn many tasks over a lifetime from one or more domains. They efficiently and effectively retain the knowledge they have learned and use that knowledge to more efficiently and effectively learn new tasks.

7、LML基本要素
①知识的存储；
②对知识选择性的应用到新的任务中；
③系统的方法对于知识进行高效且有效地获取和使用；

8、LML Framework

9、Challenges
①机器学习方法：单独使用哪一种方法会使得效果更好，还是将一些方法整合一起使用，效果更好呢？这是open question；（个人观点：尝试+具体问题具体分析）
②输出与输入类型：基本型还是复杂结构型；（个人观点：具体问题具体分析）
③先验知识权重的选择；（个人观点：通过实验分析）
④高效且有效的知识存储（知识获取）；
⑤高效且有效的知识迁移（知识使用）；
⑥高拓展性：因为是Lifelong的过程，所以必须要适应大数据；
⑦异源数据集：如何使得当前的knowledge不对异源数据集的学习产生负面的影响；

10、Benefits
LML对于software agents和robots都有比较好的应用，能解决冷启动问题等；

11、Next Steps
建立一个类似Weka的开源项目或社区，然后让研究人员在里面分享LML系统获得到的knowledge，并进行transfer到各个LML系统，促进共同进步。

参考资料：
[1]、http://link.springer.com/chapter/10.1007/978-0-585-27366-2_1#page-1
[2]、http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.69.7405&rep=rep1&type=pdf
[3]、http://link.springer.com/chapter/10.1007/978-1-4613-1381-6_2#page-1
[4]、http://link.springer.com/chapter/10.1007/978-3-540-24840-8_16#page-1
[5]、http://china.tandfonline.com/doi/abs/10.1080/09540090110047767
[6]、http://www.jmlr.org/papers/v3/strehl02a.html
[7]、http://dl.acm.org/citation.cfm?id=1273592
[8]、https://www.cs.cmu.edu/afs/cs.cmu.edu/Web/People/acarlson/papers/carlson-aaai10.pdf
[9]、http://dl.acm.org/citation.cfm?id=1658424
[10]、http://link.springer.com/chapter/10.1007/978-1-4615-5529-2_11#page-1
[11]、http://www.isi.imi.i.u-tokyo.ac.jp/~f-tanaka/paper/Tanaka_EWLR-97.pdf
[12]、https://people.eecs.berkeley.edu/~russell/classes/cs294/s11/readings/Parr%20Russell:1998.pdf
[13]、http://dl.acm.org/citation.cfm?id=1273606
[14]、http://onlinelibrary.wiley.com/doi/10.1111/j.1551-6708.1987.tb00862.x/full
[15]、http://ieeexplore.ieee.org/document/6639343/?arnumber=6639343&tag=1

以上均为个人见解，因本人水平有限，如发现有所错漏，敬请指出，谢谢！