Eng: Several Concepts Explanation

The standard deviation, or SD, measures the amount of variability or dispersion for a subject set of data from the mean, while the standard error of the mean, or SEM, measures how far the sample mean of the data is likely to be from the true population mean.

The standard deviation is a statistic that measures the dispersion of a dataset relative to its mean and is calculated as the square root of the variance. 

SD of sample

SD of population

---

Before talking about SEM, let's talk about SE standard error firstly:

The standard error(SE) is very similar to standard deviation. Both are measures of spread. The higher the number, the more spread out your data is. To put it simply, the two terms are essentially equal — but there is one important difference. While the standard error uses statistics (sample data)only, but standard deviations use parameters (population data) and statistics (if mentioned SD of sample data).

The standard error is same as the standard deviation of various sample statistics such as the mean or median. For example, the "standard error of the mean,SEM" refers to the standard deviation of the distribution of sample means taken from a population. The smaller the standard error, the more representative the sample will be of the overall population.

SD of sample mean: SD of population, divided by root of sample size

σ is the standard deviation of the population. n is the size (number of observations) of the sample.

Since the population standard deviation is seldom known, the standard error of the mean is usually estimated as the sample standard deviation divided by the square root of the sample size (assuming statistical independence of the values in the sample).

s is the sample standard deviation (i.e., the sample-based estimate of the standard deviation of the population), and n is the size (number of observations) of the sample.

SEM: 

例子：

比如总体是1 2 3 4 5 6 7 8 9 10 总体平均数是：5.5

而甲对总体进行抽样，可能得到 5 8 3 2 平均数是4.5

乙进行抽样，得到 3 7 9 2 平均数5.25

丙抽样，得到 4 6 9 2 4 1 平均数为4.3

丁...............................平均数为x

那么，4.5， 5.25 ，4.3.........x 组成一个新分布，这是一个以样本平均数为分布的，那么这个分布的标准差是什么呢？利用公式，我们可以得出这个分布的标准差，而这个标准差就是标准误。（当然这个分布的统计量有平均数，标准差，方差，相关系数等等。而我们这里以平均数为栗子。）

再次辨析：

Standard deviation (of a random variable)

Standard error of the mean/ of the standard deviation/ of the variance ...

Standard error of the regression

SD 是 Var的开方，用来反映一个随机变量的离散程度。

SE 是指通过样本sample估计总体population的时候，样本分布的标准差SD。

SEM 是通过样本均值sample mean估计总体均值population mean的时候，均值的样本分布（sampling distribution of the mean）的标准差。

Standard error of the regression 只特指在OLS估计方法的回归方程中，是误差项error的标准差

---

An error is the difference between the observed value and the true value (very often unobserved, generated by the DGP).

A residual is the difference between the observed value and the predicted value (by the model).

error 误差：测量值与真实值之间的差异。

residual 残差：算法拟合出的值与测量值之间的差异

---

无偏估计（unbiased estimate)：样本统计量的抽样分布均值和要估计的总体参数相等，就认为这个统计量是参数的无偏估计。 

有偏估计（biased estimate）：抽样分布的均值和要顾及的参数不相等，就认为这个统计量是参数的有偏估计。