(1997)Machine Learning [CMU+T.M. Mitchell] 读书笔记 - [第一章]

此博客只是在拜读图书Machine Learning过程中，所做的读书笔记。由于个人理论水平和实践经验都比较有限，难免会有理解不透、甚至贻笑大方之处，还敬请赐教！此博客的开始时间为: 2015-11-24，第一次修改时间为: 2016-01-16，第二次修改时间为: 2016-01-23，最终定稿时间为: 2016-02-06。

卡内基梅隆大学 [CMU] CS教授T.M. Mitchell在1997年所著的Machine Learning，是介绍机器学习领域核心概念和基本算法的经典书籍之一 [“present the key algorithms and theory that form the core of machine learning”]。

第一章 · 基本概念的介绍
[1]. 机器学习概念

• “A computer program is said to learn form experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

对机器学习这一概念，最著名的探讨——恐怕要归属于图灵在1950年所发表的那篇划时代论文。

此书对 [机器学习] 概念进行了通俗化、直观化、简单化的定义，因而被广泛地引用和借鉴。

机器学习是一个跨学科的研究领域，主要涉及到 [artificial intelligence、computational complexity theory、control theory、information theory、philosophy、psychology、neurobiology、statistics] 等领域的专业知识。究竟机器学习如何与如此众多的领域联系、交叉在一起？这里暂且按下不表。对我个人而言，目前最感兴趣的话题是: 机器学习如何与philosophy联系、交叉在一起?

机器学习领域最常涉及到的philosophy概念恐怕就是: 奥卡姆剃刀原理 [Occam razor, “suggesting that the simplest hypothesis is the best”]。

[2]. 机器学习应用

• “ranging from data-mining programs that learn to detect fraudulent credit transactions, to information-filtering systems that learn user’s reading preferences, to autonomous vehicles that learn to drive on public highways” ……
• “learning to recognize spoken words, learning to classify new astronomical structures, learning to play world-class backgammon” ……

[3]. Learning

• “learning from direct training feedback is typically easier than learning from indirect feedback. In general, learning is most reliable when the training example follow a distribution similar to that of future test examples.”
• “We shall see that most current theory of ML rests on the crucial assumption that the distribution of training examples is identical to the distribution of test examples. Despite our need to make this assumption in order to obtain theoretical results, it is important to keep in mind that this assumption must often be violated in practice.”

[4]. 机器学习与优化 [Optimization]

• “One useful perspective on machine learning is that it involves searching a very large space of possible hypotheses to determine one that best fits the observed data and any prior knowledge held by the learner.”

当前阶段，许多 [Learning] 问题都被巧妙地转化为 [Optimization] 问题。想要优雅地解决机器学习领域所涉及到的、具有不同性质的函数优化问题，往往需要发挥 [The Beauty of Mathematics]。

[5]. 重要研究刊物

• “Relevant journals include Machine Learning, Neural Computation, Neural Networks, Journal of the American Statistical Association, and the IEEE Transactions on Pattern Analysis and Machine Intelligence.”

补充：Journal of Machine Learning Research [简称JMLR，其高尚的分享精神值得大书特书！]。

• “The Journal of Machine Learning Research (JMLR) provides an international forum for the electronic and paper publication of high-quality scholarly articles in all areas of machine learning. All published papers are freely available online.”