固定随机种子是否有必要？

为什么要固定随机种子

固定随机性能解决这些问题：实验结果可复现、有效的对比实验

为什么需要随机性？

In applied machine learning, we run a machine learning “algorithm” on a dataset to get a machine learning “model.” The model can then be evaluated on data not used during training or used to make predictions on new data, also not seen during training.

• Algorithm: Procedure run on data that results in a model (e.g. training or learning).
• Model: Data structure and coefficients used to make predictions on data.

这里的Algorithm具有随机性。They allow the model to escape local optima or deceptive local optima where the learning algorithm might get such, and help find better solutions, even a global optima. Neural networks (deep learning) are a stochastic machine learning algorithm. The random initial weights allow the model to try learning from a different starting point in the search space each algorithm run and allow the learning algorithm to “break symmetry” during learning. The random shuffle of examples during training ensures that each gradient estimate and weight update is slightly different.

What should we do

在我们做对比试验，需要验证不同trick、不同模型、不同超参数的有效性的时候，最好是能固定随机种子使得模型是可复现的。但使用nondeterministic的算法往往能取得更好的performance，带来更快的运算速度。

另外，随机种子不是一个超参数，并不存在一个最好的随机种子。

如果是为了工程中的模型部署，更好地做法是去跑多个模型（指多次训练得到的input到output的映射function），计算其统计特征（比如std、mean），用来评估模型好坏。