西瓜书-机器学习5.4 全局最小与局部极小

两种“最优”：“局部极小”(local minimum)和"全局最小"(global minimum)

对和，若存在 使得

• 多组不同参数值初始化多个神经网络
• 使用“模拟退火”：以一定的概率接受比当前解更差的结果，有助于“跳出”局部极小
• 使用随机梯度下降
• 遗传算法(genetic algorithms)[Goldberg, 1989]也常用来训练神经网络

以上用于跳出局部极小的技术大多是启发式，理论上商缺乏保障。

 

Goldberg, 1989, Genetic Algorithms in Search, Optimization and Machine Learning

David Goldberg's Genetic Algorithms in Search, Optimization and Machine Learning is by far the bestselling introduction to genetic algorithms. Goldberg is one of the preeminent researchers in the field--he has published over 100 research articles on genetic algorithms and is a student of John Holland, the father of genetic algorithms--and his deep understanding of the material shines through. The book contains a complete listing of a simple genetic algorithm in Pascal, which C programmers can easily understand. The book covers all of the important topics in the field, including crossover, mutation, classifier systems, and fitness scaling, giving a novice with a computer science background enough information to implement a genetic algorithm and describe genetic algorithms to a friend.