machine learning学习笔记三之回归小笔记

补一个前面回归欠的小笔记： 

1、在做回归的时候，往往用梯度下降算法来最小化损失函数，梯度是一个矢量，梯度方向上的导数最大，所以梯度是增长最快的方向，我们就可以推出，-梯度就是下降最快的方向。

2、在fit模型的时候，未必越复杂越好，可能会引起overfitting。 A more complex model does not always lead to better performance on testing data.

3、损失函数往往可以加入一个正则化条件，比如lamda乘上一个系数平方的和，以此引入系数的波动。系数波动的越大，越不靠谱。一般来说，抖动得很厉害的function，一般都不太对。we believe that smoother function is more likely to be correct。越平滑，感觉越靠谱点。

3.模型error的来源。源于bias和error。 简单的model，bias大（截距b），variance小，function space小，underfitting；复杂的模型一般是bias小，variance大，overfitting。

4.怎么看underfit,overfit。 如果模型对train data 比较fit， 对test data不fit, 说明是overfitting了；另一方面，如果对train data都fit不起来，那就是underfitting了。

5.怎么解决large variance的问题：

（1）增加data（增加N。 E(S^2)=((N-1）/N)sigma^2. N越大ES方和sigma方的差距越小）

（2）正规化 regularization

   (3) cross-validation交叉验证

6.Naive Bayes在条件概率=0的时候怎么办？用拉普拉斯平滑，分子加1，分母加n（attribute的个数）Use Laplacian smoothing to avoid the conditional probability equals to 0. Method: add one to numerator and add # of attributes to the denominator.

7.SVM的是否对outlier敏感性问题：

“effect does not directly mean sensitive”. Outliers may affect the hyperplane does not mean this model would be sensitive to outliers. Outlier’s effect on hyperplane mainly depends on the parameter C. The smaller the C is, the margin would get bigger, the model is more tolerant for outliers. The larger the C is, the model is less tolerant for outliers and end up with a smaller margin. It also means a smaller training error and maybe over fitting.

8.holt-winters和季节arima的异同之处：

For the difference between holt-winters and seasonal arima. Sarima and holt-winters both extracted the development features of the series but sarima is more general. The ETS are the method to find solutions for arima or the special cases of arima. However, arima won’t cover all ets solutions. For example, Holt-winter’s addictive method is very close to sarima given specific parameter. But holt-winter’s multiplicative method has no equivalent sarima counterparts.