Introduction to statistical machine learning复习笔记1概率密度部分

概率密度部分

1. 期望和峰值 当outliers很严重
   

2. skewness and kurtosis
   $$\begin{array}{l}\text{ Skewness: }\frac{E\left[(x-E[x]{)}^3\right]}{(D[x]{)}^3}\\ \text{ Kurtosis: }\frac{E\left[(x-E[x]{)}^4\right]}{(D[x]{)}^4}-3\end{array}$$
   

3. Moment 生成函数
   As the limit, if the moments of all orders are specifed, the probability distribution is uniquely determined.
   $$M_x(t)=E\left[e^{tx}\right]=\{\begin{array}{ll}{\sum }_x{e}^{tx}f(x)& \text{ (Discrete) }\\ \int e^{tx}f(x)\mathrm{d}x& \text{ (Continuous) }\end{array}$$
   $$e^{tx}=1+(tx)+\frac{(tx{)}^2}{2!}+\frac{(tx{)}^3}{3!}+\cdots$$
   $$E\left[e^{tx}\right]=M_x(t)=1+t{\mu }_1+t^2\frac{{\mu }_2}{2!}+t^3\frac{{\mu }_3}{3!}+\cdots$$
   $$\begin{array}{rl}{M}_x^{\prime }(t)& ={\mu }_1+{\mu }_{2}t+\frac{{\mu }_3}{2!}{t}^2+\frac{{\mu }_4}{3!}{t}^3+\cdots \\ M_x^{\prime \prime }(t)& ={\mu }_2+{\mu }_{3}t+\frac{{\mu }_4}{2!}{t}^2+\frac{{\mu }_5}{3!}{t}^3+\cdots \\ & ⋮\\ M_x^{(k)}(t)& ={\mu }_k+{\mu }_{k+1}t+\frac{{\mu }_{k+2}}{2!}{t}^2+\frac{{\mu }_{k+3}}{3!}{t}^3+\cdots \end{array}$$

4. 分布的特征方程
   是概率密度的傅里叶变换
   $${\phi }_x(t)=M_{ix}(t)=M_x(it)$$

5. TRANSFORMATION OF RANDOM VARIABLES
   要乘上雅克比行列式：
   Integration of function $f(x)$ over $\mathcal{X}$ can be expressed by using function $g(r)$ on $\mathcal{R}$ such that
   $$x=g(r)\text{ and }X=g(\mathcal{R})$$
   as
   $${\int }_{\mathcal{X}}f(x)\mathrm{d}x={\int }_{\mathcal{R}}f(g(r))\left|\frac{\mathrm{d}x}{\mathrm{d}r}\right|\mathrm{d}r$$

6. 伽马分布 指数分布
   $$f(x)=\frac{{\lambda }^{\alpha }}{\Gamma (\alpha )}{x}^{\alpha -1}{e}^{-\lambda x}\text{ for }x\ge 0$$
   $$f(x)=\lambda e^{-\lambda x}$$

7. beta分布
   当x和y为正整数时，x + y−1个随机变量中独立服从连续均匀分布的第x个最小值(或者说第y个最大值)U(0,1)服从beta分布。

8. 拉普拉斯分布
   两个指数分布之差服从的分布。
   $$f(x)=\frac{1}{2b}\exp \left(-\frac{|x-a|}{b}\right)$$
   也叫双指数分布

柯西分布和拉普拉斯分布经常用来搞有outliers的数据