计量笔记(一) | OLS估计量推导

一、基准方程推导

总体回归模型
$$y_i={\beta }_1+{\beta }_2{x}_{2i}+{\beta }_3{x}_{3i}+\cdots +{\beta }_k{x}_{ki}+{\epsilon }_i$$
样本回归模型
$$y_i={\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+{\hat{\beta }}_3{x}_{3i}+\cdots +{\hat{\beta }}_k{x}_{ki}+e_i$$

$${\left(\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_i\end{array}\right)}_{n\times 1}{\left(\begin{array}{ccccc}1& x_{21}& x_{31}& \cdots & x_{k1}\\ 1& x_{22}& x_{32}& \cdots & x_{k2}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ 1& x_{2k}& x_{3k}& \cdots & x_{kk}\end{array}\right)}_{n\times k}{\left(\begin{array}{c}{\beta }_1\\ {\beta }_2\\ ⋮\\ {\beta }_k\end{array}\right)}_{n\times 1}+{\left(\begin{array}{c}{e}_1\\ e_2\\ ⋮\\ e_k\end{array}\right)}_{n\times 1}$$

$$Y=X\hat{\beta }\text{β^}\hat{\beta }+e\text{e}e$$

残差平方和
$$\sum e_i^2=\sum (y_i-{\hat{y}}_i{)}^2=\sum [y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki}){]}^2$$
对残差平方和各参数（${\beta }_1,{\beta }_2\cdots ,{\beta }_k$）求导，并令其等于0
$$\begin{array}{r}\{\begin{array}{llc}\sum 2e_i(-1)& =-2\sum [y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\\ \sum 2e_i(-x_{2i})& =-2\sum x_{2i}[y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\\ ⋮& & \\ \sum 2e_i(-x_{ki})& =-2\sum x_{ki}[y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\end{array}\end{array}$$
得到正规方程组
$$\begin{array}{r}\{\begin{array}{llc}\sum e_i& =\sum [y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\\ \sum e_i{x}_{2i}& =\sum x_{2i}[y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\\ ⋮& & \\ \sum e_i{x}_{ki}& =\sum x_{ki}[y_i-({\hat{\beta }}_1+{\hat{\beta }}_2{x}_{2i}+\cdots +{\hat{\beta }}_k{x}_{ki})]& =0\end{array}\end{array}$$
写成矩阵形式
$$\left(\begin{array}{cccc}1& 1& \cdots & 1\\ x_{21}& x_{22}& \cdots & x_{2i}\\ ⋮& ⋮& ⋮& ⋮\\ x_{k1}& x_{k2}& \cdots & x_{ki}\end{array}\right)\left(\begin{array}{c}{e}_1\\ e_2\\ ⋮\\ e_3\end{array}\right)=X^{\tau }e\text{e}e=0\text{0}0$$
对样本回归模型$Y=X\hat{\beta }\text{β^}\hat{\beta }+e\text{e}e$两边同左乘$X^{\tau }$
$$X^{\tau }Y=X^{\tau }X\hat{\beta }\text{β^}\hat{\beta }+X^{\tau }e\text{e}e$$
由于$X^{\tau }e\text{e}e=0\text{0}0$，所以$X^{\tau }Y=X^{\tau }X\hat{\beta }\text{β^}\hat{\beta }$

得到参数估计量
$$\hat{\beta }\text{β^}\hat{\beta }=(X^{\tau }X{)}^{-1}{X}^{\tau }Y$$
二、矩阵形式推导
$$\begin{array}{rl}Q(\hat{\beta }\text{β^}\hat{\beta })=e^{\tau }\text{eτ}{e}^{\tau }e\text{e}e& =(Y-X\hat{\beta }\text{β^}\hat{\beta }{)}^{\tau }(Y-X\hat{\beta }\text{β^}\hat{\beta })\\ & =(Y^{\tau }-{\hat{\beta }\text{β^}\hat{\beta }}^{\tau }{X}^{\tau })(Y-X\hat{\beta }\text{β^}\hat{\beta })\\ & =Y^{\tau }Y-Y^{\tau }X\hat{\beta }\text{β^}\hat{\beta }-{\hat{\beta }\text{β^}\hat{\beta }}^{\tau }{X}^{\tau }Y+{\hat{\beta }\text{β^}\hat{\beta }}^{\tau }{X}^{\tau }X\hat{\beta }\text{β^}\hat{\beta }\\ & =Y^{\tau }Y-2{\hat{\beta }\text{β^}\hat{\beta }}^{\tau }{X}^{\tau }Y+{\hat{\beta }\text{β^}\hat{\beta }}^{\tau }{X}^{\tau }X\hat{\beta }\text{β^}\hat{\beta }\end{array}$$
对残差平方和各参数（${\beta }_1,{\beta }_2\cdots ,{\beta }_k$）求导，并令其等于0
$$\frac{\partial Q(\hat{\beta }\text{β^}\hat{\beta })}{\partial \hat{\beta }\text{β^}\hat{\beta }}=-2X^{\tau }Y+2X^{\tau }X\hat{\beta }\text{β^}\hat{\beta }=0\text{0}0$$
得到参数估计量
$$\hat{\beta }\text{β^}\hat{\beta }=(X^{\tau }X{)}^{-1}{X}^{\tau }Y$$