GridSearch

不同搜索空间的比较图：

1. 空间搜索
2. 螺旋搜索
3. 线性搜索
4. 网格搜索

可以看到，在超参的搜索过程汇总，网格搜索和螺旋搜索，搜索的效果更好，更细致！

如何应对网格搜索巨大的资源消耗问题？

one limitation of GridSearch is that it can be very computationally expensive when dealing with a large number of different hyperparameters and much bigger datasets. Therefore there are two other techniques that we could explore to validate our hyperparameters

1. RandomizedSearchCV which can sample a given number of candidates from a parameter space with a specified distribution. Which performs surprisingly well! 从搜索空间中根据特定的分布进行采样，事件证明很有效。
2. Or a train / validation / test split, and we can validate our model on the validation set. Often used with much bigger datasets. （有点不太明白和之前学习的有什么区别）