吴恩达机器学习笔记（week6——）

http://ai-start.com/ml2014/html/week6.html

Week6

十、应用机器学习的建议(Advice for Applying Machine Learning)

10.1 决定下一步做什么

10.2 评估一个假设

10.3 模型选择和交叉验证集

10.4 诊断偏差和方差

10.5 正则化和偏差/方差

10.6 学习曲线

10.7 决定下一步做什么

十一、机器学习系统的设计(Machine Learning System Design)

11.1 首先要做什么

11.2 误差分析

11.3 类偏斜的误差度量

11.4 查准率和查全率之间的权衡

11.5 机器学习的数据

十、应用机器学习的建议(Advice for Applying Machine Learning)

10.1 10.2 

The test set error

10.3

笔记里的最后一个公式x和y的下标应为test

10.5

In the figure above, we see that as $\lambda$ increases, our fit becomes more rigid. On the other hand, as $\lambda$ approaches 0, we tend to over overfit the data. So how do we choose our parameter $\lambda$ to get it 'just right' ? In order to choose the model and the regularization term λ, we need to:

1. Create a list of lambdas (i.e. λ∈{0,0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24});
2. Create a set of models with different degrees or any other variants.
3. Iterate through the $\lambda$s and for each $\lambda$ go through all the models to learn some $\Theta$.
4. Compute the cross validation error using the learned Θ (computed with λ) on the J_{CV}(\Theta)JCV​(Θ) without regularization or λ = 0.
5. Select the best combo that produces the lowest error on the cross validation set.
6. Using the best combo Θ and λ, apply it on J_{test}(\Theta)Jtest​(Θ) to see if it has a good generalization of the problem.

10.7

Diagnosing Neural Networks

• A neural network with fewer parameters is prone to underfitting. It is also computationally cheaper.
• A large neural network with more parameters is prone to overfitting. It is also computationally expensive. In this case you can use regularization (increase λ) to address the overfitting.

Using a single hidden layer is a good starting default. You can train your neural network on a number of hidden layers using your cross validation set. You can then select the one that performs best.

Model Complexity Effects:

• Lower-order polynomials (low model complexity) have high bias and low variance. In this case, the model fits poorly consistently.
• Higher-order polynomials (high model complexity) fit the training data extremely well and the test data extremely poorly. These have low bias on the training data, but very high variance.
• In reality, we would want to choose a model somewhere in between, that can generalize well but also fits the data reasonably well.

Week7

http://ai-start.com/ml2014/html/week7.html

十二、支持向量机(Support Vector Machines)

12.1 优化目标

12.2 大边界的直观理解

12.3 数学背后的大边界分类（选修）

12.4 核函数1

12.5 核函数2

12.6 使用支持向量机

Week8

十三、聚类(Clustering)

13.1 无监督学习：简介

13.2 K-均值算法

13.3 优化目标

13.4 随机初始化

13.5 选择聚类数

十四、降维(Dimensionality Reduction)

14.1 动机一：数据压缩

14.2 动机二：数据可视化

14.3 主成分分析问题

14.4 主成分分析算法

14.5 选择主成分的数量

14.6 重建的压缩表示

14.7 主成分分析法的应用建议

Week9

http://ai-start.com/ml2014/html/week9.html#header-n95

十五、异常检测(Anomaly Detection)

15.1 问题的动机

15.2 高斯分布

15.3 算法

15.4 开发和评价一个异常检测系统

15.5 异常检测与监督学习对比

15.6 选择特征

15.7 多元高斯分布（选修）

15.8 使用多元高斯分布进行异常检测（选修）

十六、推荐系统(Recommender Systems)

16.1 问题形式化

16.2 基于内容的推荐系统

16.3 协同过滤

16.4 协同过滤算法

16.5 向量化：低秩矩阵分解

16.6 推行工作上的细节：均值归一化

16.3 协同过滤：

    对代价函数求偏导数的结果如下：