Normalization、Standardization and Regularization

1、Normalization

On one hand,normalization rescales the values into a range of [0,1]. This might be useful in some cases where all parameters need to have the same positive scale. However, the outliers from the data set are lost.

Xchanged=X−XminXmax−XminXchanged=X−XminXmax−Xmin

On the other hand,In Linear Algebra, Normalization seems to refer to the dividing of a vector by its length.

2、Standardization

Standardization rescales data to have a mean (μμ) of 0 and standard deviation (σσ) of 1 (unit variance).

Xchanged=X−μσXchanged=X−μσ

For most applications standardization is recommended.

3、Regularization

regularization is a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting.