kaggle——MNIST之一

ml的入门教程，使用svm来做mnist的分类：
细节：直接将image拉成一个向量，然后直接采用svm分类，结果 10%的准确率，和随机猜测ch差不多；

改进版：将grey的image转换为二值化图，即为0或1图，然后相同的的方法训练，准确率飙升至88%；

其他人的版本：将grey归一化到【0，1】，其准确率ch差不多也是88%；

关于此的分析：

So instead of setting values to 0 or 1, I rescaled the features to be within [0, 1] (notice that the numbers still takes continuous values, not only just binary states). The resulting precision is 0.883, very similar to 0.887 here.

I think the true reason has to do with the relative penalty you put on regularization vs. in sample loss in SVM. The objective function you are minimizing is:

regularization term + hinge loss * C

By default, C is set to be 1 in sklearn svm.SVC. When the value of the feature is between 0 and 255, I think the regularization term is effectively so large that SVM does not learn anything from the samples.

By playing with the paramter C, you will see the same effect without rescaling/changing the feature.

Here's an excerpt from sklearn:

"Support Vector Machine algorithms are not scale invariant, so it is highly recommended to scale your data. For example, scale each attribute on the input vector X to [0,1] or [-1,+1], or standardize it to have mean 0 and variance 1. Note that the same scaling must be applied to the test vector to obtain meaningful results. See section Preprocessing data for more details on scaling and normalization."

（所以，出现问题，多看看说明手册，或许会有da'a答案）