最大似然估计会低估方差

根据最大似然估计可以得到方差的估计值为:

σ M L 2 = 1 N ∑ n = 1 N ( x n − μ M L ) 2 (1) \sigma_{ML}^{2}=\frac{1}{N}\sum_{n=1}^{N}(x_{n}-\mu_{ML})^{2}\tag {1} σML2=N1n=1N(xnμML)2(1)

其中 μ M L 2 = 1 N ∑ i = 1 N x i \mu_{ML}^{2}=\frac{1}{N}\sum_{i=1}^{N}x_{i} μML2=N1i=1Nxi

σ M L 2 \sigma_{ML}^{2} σML2求期望的结果为:

E ( σ M L 2 ) = N − 1 N σ 2 (2) E(\sigma_{ML}^{2})=\frac{N-1}{N}\sigma^{2}\tag {2} E(σML2)=NN1σ2(2)

从这个结果来看,可以知道最大似然估计对实际的方差 σ 2 \sigma^{2} σ2低估了,主要是因为系数 N − 1 N \frac{N-1}{N} NN1

下面对这个期望结果进行证明:

( 1 ) (1) (1)式代入 ( 2 ) (2) (2)可得,

E ( σ M L 2 ) = E ( 1 N ∑ n = 1 N ( x n − μ M L ) 2 ) = 1 N ∑ n = 1 N E ( ( x n − μ M L ) 2 ) = 1 N ∑ n = 1 N E ( x n 2 − 2 x n μ M L + μ M L 2 ) = 1 N ∑ n = 1 N E ( x n 2 ) − 2 N ∑ n = 1 N E ( ( x n 2 + 1 N ∑ i = 1 , i ≠ n N x n x i ) ) + 1 N ∑ n = 1 N E ( μ M L 2 ) ⏟ ∗ = σ 2 + μ 2 − 2 N μ 2 − 2 N σ 2 − 2 N − 1 N μ 2 + μ 2 + 1 N σ 2 = N − 1 N σ 2 \begin{aligned} E(\sigma_{ML}^{2}) &= E(\frac{1}{N}\sum_{n=1}^{N}(x_{n}-\mu_{ML})^{2}) \\ & = \frac{1}{N}\sum_{n=1}^{N}E((x_{n}-\mu_{ML})^{2}) \\ & = \frac{1}{N}\sum_{n=1}^{N}E(x_{n}^{2}-2x_{n}\mu_{ML}+\mu_{ML}^{2})\\ & = \frac{1}{N}\sum_{n=1}^{N}E(x_{n}^{2})-\frac{2}{N}\sum_{n=1}^{N}E((x_{n}^{2}+\frac{1}{N}\sum_{i=1,i\neq{n}}^{N}x_{n}x_{i}))+\frac{1}{N}\sum_{n=1}^{N}\underbrace{E(\mu_{ML}^{2})}_{*}\\ & = \sigma^{2}+\mu^{2}-\frac{2}{N}\mu^{2}-\frac{2}{N}\sigma^{2}-2\frac{N-1}{N}\mu^{2}+\mu^{2}+\frac{1}{N}\sigma^{2}\\ & = \frac{N-1}{N}\sigma^{2} \end{aligned} E(σML2)=E(N1n=1N(xnμML)2)=N1n=1NE((xnμML)2)=N1n=1NE(xn22xnμML+μML2)=N1n=1NE(xn2)N2n=1NE((xn2+N1i=1,i=nNxnxi))+N1n=1N E(μML2)=σ2+μ2N2μ2N2σ22NN1μ2+μ2+N1σ2=NN1σ2
其中 μ \mu μ是数据的真实均值, σ 2 \sigma^{2} σ2是数据的真实方差,下方花括号括起来的*式的具体计算过程如下:
E ( μ M L 2 ) = E 2 ( μ M L ) + D ( μ M L ) = μ 2 + D ( 1 N ∑ i = 1 N x i ) = μ 2 + 1 N 2 D ( ∑ i = 1 N x i ) = μ 2 + 1 N 2 N σ 2 = μ 2 + 1 N σ 2 \begin{aligned} E(\mu_{ML}^{2}) & = E^{2}(\mu_{ML})+D(\mu_{ML})\\ & = \mu^{2}+D(\frac{1}{N}\sum_{i=1}^{N}x_{i})\\ & = \mu^{2}+\frac{1}{N^{2}}D(\sum_{i=1}^{N}x_{i})\\ & = \mu^{2}+\frac{1}{N^{2}}N\sigma^{2}\\ & = \mu^{2}+\frac{1}{N}\sigma^{2} \end{aligned} E(μML2)=E2(μML)+D(μML)=μ2+D(N1i=1Nxi)=μ2+N21D(i=1Nxi)=μ2+N21Nσ2=μ2+N1σ2

证毕。

你可能感兴趣的:(概率论)