Machine Learning No.4: Regularization

1. Underfit = High bias

  Overfit = High varience

2. Addressing overfitting:

  (1) reduce number of features.

    Manually select which features to keep.

    Model selection algorithm

      disadvantage: throw out some useful information

     (2) Regularization

    Keep all the features, but reduce magnitude/values of parameters θj

    works well when we have a lot of features, each of which contributλes a bit to predicting y.

3. Regularization

if λ is extremely large, , then J(θ) will be underfitting

4. Gradient desent

Repeat {

  

         (j = 1, 2 ... n)

}

5. Normal equation

if λ > 0

Machine Learning No.4: Regularization

if m <= n

is non-invertible/singular

but using regularization will avoid this problem

 

你可能感兴趣的:(Machine Learning No.4: Regularization)