机器学习-白板推导 P2_2

多维高斯分布

$$p(x)=\frac{1}{{2\pi }^{\frac{p}{2}}}\exp \left(-\frac{1}{2}(x-\mu {)}^T{\Sigma }^{-1}(x-\mu )\right)$$
$x\in R^p$，是随机变量
$$x=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_p\end{array}\right]\mu =\left[\begin{array}{c}{\mu }_1\\ {\mu }_2\\ ⋮\\ {\mu }_p\end{array}\right]\Sigma ={\left[\begin{array}{cccc}{\sigma }_{11}& {\sigma }_{12}& \cdots & {\sigma }_{1p}\\ {\sigma }_{21}& {\sigma }_{22}& \cdots & {\sigma }_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ {\sigma }_{p1}& {\sigma }_{p2}& \cdots & {\sigma }_{pp}\end{array}\right]}_{p\times p}通常\Sigma 是半正定的$$
$(x-\mu {)}^T{\Sigma }^{-1}(x-\mu )$:马氏距离($x$与$\mu$之间) mahalabonis distance
如果：$\Sigma =I$，马氏距离=欧氏距离

$\Sigma$特征分解：
$$\begin{array}{rl}& \Sigma =U\Lambda U^T\\ & U{U}^T=U^{T}U=I\\ & \Lambda =diag({\lambda }_i)i=1,...p\\ & U=(u_1,u_2...u_p{)}_{p\ast p}\end{array}$$
$$\begin{array}{rl}\Sigma & =\left[\begin{array}{cccc}{u}_1& u_2& \cdots & u_p\end{array}\right]\left[\begin{array}{cccc}{\lambda }_1& 0& \cdots & 0\\ 0& {\lambda }_2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& \cdots & \cdots & {\lambda }_p\end{array}\right]\left[\begin{array}{c}{u}_1^T\\ u_2^T\\ ⋮\\ u_p^T\end{array}\right]\\ & =\sum _{i=1}^p{\mu }_i{\lambda }_i{\mu }_i^T\end{array}$$
$$\begin{array}{rl}& {\Sigma }^{-1}=(U\Lambda U^T{)}^{-1}=(U^T{)}^{-1}{\Lambda }^{-1}{U}^{-1}=U{\Lambda }^{-1}{U}^T=\sum _{i=1}^p{\mu }_i\frac{1}{{\lambda }_i}{\mu }_i^T\\ & {\Lambda }^{-1}=diag(\frac{1}{{\lambda }_i})i=1,...p\end{array}$$

$$定义:y=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_p\end{array}\right]令：y_i=(x-\mu {)}^T{u}_i$$

$$\begin{array}{rl}\Delta =& (x-\mu {)}^T{\Sigma }^{-1}(x-\mu )=(x-\mu {)}^T\sum _{i=1}^p{\mu }_i\frac{1}{{\lambda }_i}{\mu }_i^T(x-\mu )\\ & =\sum _{i=1}^p(x-\mu {)}^T{\mu }_i\frac{1}{{\lambda }_i}{\mu }_i^T(x-\mu )\\ & =\sum _{i=1}^p{y}_i\frac{1}{{\lambda }_i}{y}_i^T\\ & =\sum _{i=1}^p\frac{y_i^2}{{\lambda }_i}\end{array}$$

令$p=2$，$\Delta =\frac{y_1^2}{{\lambda }_1}+\frac{y_2^2}{{\lambda }_2}$
假设$\Delta =\frac{y_1^2}{{\lambda }_1}+\frac{y_2^2}{{\lambda }_2}=1$，图形为椭圆。
$y_i=(x-\mu {)}^T{u}_i$物理解释，$x-\mu$在$u_i$方向上的投影。