Selected Solutions to Kevin Murphy's Machine Learning

《machine learning a probabilistic perspective》部分习题解答，持续更新中

Chapter 3

Ex 3.2 Beta-Bernoulli模型的边缘似然函数

由3.3.4节得到，后验预测分布为
![][1]
则边际分布为
![][2]
![][3]
![][4]

Ex 3.3 Beta-Binomial模型的后验预测分布

![][5]
n = 1时
![][6]

Ex 3.4 Beta updating from censored likelihood

抛掷硬币5次，X为朝上次数。仅仅知道X < 3而不知道X的确切值，求相应后验概率（未归一化）结果为一个混合分布
![][7]

Ex 3.12 非共轭先验的Bernoulli分布参数的MAP估计

a

![][8]
![][9]

b

当N很小时，采用a问题的先验可以得到比较好的估计，当N很大时采用Beta分布作为先验可以得到比较好的估计
[9]:http://latex.codecogs.com/gif.latex?when%20%5C%20%5Cfrac%7BN_1%7D%7BN_0%7D%20%3E%20%5Cfrac%7Blog%280.6/0.5%29%7D%7Blog%280.5/0.4%29%7D%20%5C%5C%20%5Ctheta%20%3D%200.5%20%5C%5C%20else%20%5C%20%5Ctheta%20%3D%200.4
[8]:http://latex.codecogs.com/gif.latex?p%28%5Ctheta%7CD%29%20%5Cpropto%20p%28%5Ctheta%29p%28D%7C%5Ctheta%29%20%5C%5C%20%3D%20p%28%5Ctheta%29%5Ctheta%5E%7BN_1%7D%281-%5Ctheta%29%5E%7BN%20-%20N_1%7D%20%5C%5C%20%3D%200.5%20%5Cdelta%28%5Ctheta%20-%200.5%29%5Ctheta%5E%7BN_1%7D%281-%5Ctheta%29%5E%7BN%20-%20N_1%7D%20+%200.5%20%5Cdelta%28%5Ctheta%20-%200.4%29%5Ctheta%5E%7BN_1%7D%281-%5Ctheta%29%5E%7BN%20-%20N_1%7D
[7]:http://latex.codecogs.com/gif.latex?p%28%5Ctheta%20%7C%20X%20%3C%203%29%20%5Cpropto%20p%28%5Ctheta%2C%20X%20%3C%203%29%20%5C%5C%20%3D%20%5Csum_%7Bi%20%3D%200%7D%5E%7B2%7Dp%28%5Ctheta%2C%20X%20%3D%20i%20%7C%20n%20%3D%205%29%20%5C%5C%20%3D%20%5Csum_%7Bi%20%3D%200%7D%5E%7B2%7Dp%28%5Ctheta%29P%28X%20%3D%20i%7C%5Ctheta%29%5C%5C%20%3D%5Csum_%7Bi%20%3D%200%7D%5E%7B2%7DC_5%5Ei%5Ctheta%5Ei%281%20-%20theta%29%5E%7B5-i%7D
[1]:http://latex.codecogs.com/gif.latex?p%28X%3Dk%7CD_%7B1%3AN%7D%29%20%3D%20%5Cfrac%7BN_k%20+%20%5Calpha_k%7D%7B%5Csum_i%20N_i%20+%20%5Calpha_i%7D%20%3D%20%5Cfrac%7BN_k%20+%20%5Calpha_k%7D%7BN%20+%20%5Calpha%7D
[2]:http://latex.codecogs.com/gif.latex?p%28D%29%20%3D%20%5Cfrac%7B%5Calpha_1%28%5Calpha_1%20+%201%29...%28%5Calpha_1%20+%20N_1%20-1%29%5Calpha_0%28%5Calpha_0%20+%201%29...%28%5Calpha_0%20+%20N_0%20-1%29%7D%7B%5Calpha%28%5Calpha%20+%201%29...%28%5Calpha%20+%20N%20-1%29%7D
[3]:http://latex.codecogs.com/gif.latex?%3D%5Cfrac%7B%28%20%5Calpha_1%20+%20N_1%20-%201%29%21%7D%7B%28%5Calpha_1%20-%201%29%21%7D%5Cfrac%7B%28%5Calpha_0%20+%20N_0%20-1%29%21%7D%7B%28%5Calpha_0%20-%201%29%21%7D%5Cfrac%7B%28%5Calpha%20-%201%29%21%7D%7B%28%5Calpha%20+%20N%20-%201%29%21%7D
[4]:http://latex.codecogs.com/gif.latex?%5Cfrac%7B%5CGamma%28%5Calpha_1%20+%20N_1%29%5CGamma%28%5Calpha_0%20+%20N_0%29%5CGamma%28%5Calpha%29%7D%7B%5CGamma%28%5Calpha_1%29%5CGamma%28%5Calpha_0%29%5CGamma%28%5Calpha%20+%20N%29%7D
[5]:http://latex.codecogs.com/gif.latex?p%28x%7C%20n%2CD%29%20%3D%20Bb%28x%7C%5Calpha_0%27%2C%5Calpha_1%27%2Cn%29%20%5C%5C%20%3D%20C_n%5Ex%5Cfrac%7BB%28x%20+%20%5Calpha_1%27%2Cn-x+%5Calpha_0%27%20%29%7D%7BB%28%5Calpha_0%27%2C%20%5Calpha_1%27%29%7D%20%5C%5C%20%3D%20C_n%5Ex%5Cfrac%7B%5CGamma%28x%20+%20%5Calpha_1%27%29%5CGamma%28n-x+%5Calpha_0%27%29%7D%7B%5CGamma%28%5Calpha_0%27+n+%5Calpha_1%27%29%7D%5Cfrac%7B%5CGamma%28%5Calpha_0%27+%5Calpha_1%27%29%7D%7B%5CGamma%28%5Calpha_0%27%29%5CGamma%28%5Calpha_1%27%29%7D
[6]:http://latex.codecogs.com/gif.latex?x%20%5Cin%20%5C%7B0%2C1%5C%7D%20%5C%5C%20p%28x%20%3D%201%20%7C%201%2CD%29%20%3D%20%5Cfrac%7B%5CGamma%281%20+%20%5Calpha_1%27%29%7D%7B%5CGamma%28%5Calpha_1%27%29%7D%5Cfrac%7B%5CGamma%28%5Calpha_0%27%29%7D%7B%5CGamma%28%5Calpha_0%27%29%7D%5Cfrac%7B%5CGamma%28%5Calpha_0%27%20+%20%5Calpha_1%27%29%7D%7B%5CGamma%28%5Calpha_0%27%20+%20%5Calpha_1%27%20+%201%29%7D%20%5C%5C%20%3D%20%5Cfrac%7B%5Calpha_1%27%20+%201%7D%7B%5Calpha_0%27%20+%20%5Calpha_1%27%20+%201%7D