强化学习知识点集锦

在强化学习的世界，我们相信如果输入是确定的，那么输出也一定是确定的。强化学习算法要有用，就是相信在强化学习中每一次参数的调整都会对世界造成确定性的影响。

 

1.on-policy和off-policy

下图是Q-learning和Sarsa的流程图，分别对应off-policy和on-policy

2-tradeoff variance and bias 

wiki里的解释：

In statistics and machine learning, the bias–variance tradeoff is the property of a set of predictive models whereby models with a lower bias in parameter estimation have a higher variance of the parameter estimates across samples, and vice versa. The bias–variance dilemma or bias–variance problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learning algorithms from generalizing beyond their training set:[citation needed]

• The bias error is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting).
• The variance is an error from sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (overfitting).

偏差和方差则需要在统计上做对应的定义。

• 偏差（bias）描述的是通过学习拟合出来的结果之期望，与真实规律之间的差距，记作 Bias(X)=E[f^(X)]−f(X)Bias(X)=E[f^(X)]−f(X)。
• 方差（variance）即是统计学中的定义，描述的是通过学习拟合出来的结果自身的不稳定性，记作 Var(X)=E[(f^(X)−E[f^(X)])2]Var(X)=E[(f^(X)−E[f^(X)])2]。