概率推理(贝叶斯推理)

一、随机事件

在一定条件下,可能发生也可能不发生的实验结果叫做随机事件。

随 机 事 件 ( 事 件 ) { 必 然 事 件 不 可 能 事 件 随机事件(事件)\begin{cases} 必然事件 \\ 不可能事件 \end{cases} {


二、概率的基本性质

1、对任一事件 A,有 0 ≤ P ( A ) ≤ 1 0 ≤ P(A) ≤ 1 0P(A)1 P ( A ˉ ) = 1 − P ( A ) P(\bar{A}) = 1-P(A) P(Aˉ)=1P(A)

2、必然事件 P ( M ) = 1 P(M) = 1 P(M)=1;不可能事件 P ( ∅ ) = 0 P(∅) = 0 P()=0

3、对任意两个事件,有 P ( A ∪ B ) = P ( A ) + P ( B ) − P ( A B ) P(A∪B)=P(A)+P(B)-P(AB) P(AB)=P(A)+P(B)P(AB)

  • 若事件A、B互不相容(互斥),有 P ( A ∪ B ) = P ( A ) + P ( B ) P(A∪B)=P(A)+P(B) P(AB)=P(A)+P(B)
  • 若B ⊂ \subset A(事件B的发生必然导致事件A的发生),则A发生而B不发生
    ( P(A\B) ① )的概率为 ① = P ( A ) − P ( B ) ①=P(A)-P(B) =P(A)P(B)

4、若事件 A 1 , A 2 , ⋅ ⋅ ⋅ , A K A_{1} ,A_{2} ,···,A_{K} A1,A2,,AK两两互不相容的事件,即有 A i ∩ A j = ∅ ( i ≠ j ) A_{i} ∩A_{j} =∅(i≠j) AiAj=i=j,则 P ( ⋃ i = 1 k A i ) = P ( A 1 ) + P ( A 2 ) + ⋅ ⋅ ⋅ + P ( A K ) P(\bigcup_{i=1}^{k} A_{i})= P(A_{1}) +P(A_{2})+···+P(A_{K}) P(i=1kAi)=P(A1)+P(A2)++P(AK)

三、概率的常用计算公式

1、条件概率与乘法公式
P(A|B):在事件 B 已发生的条件下事件 A 发生的概率

P ( B ) > 0 P(B)>0 P(B)>0时, P ( A ∣ B ) = P ( A ∩ B ) P ( B ) P(A|B) = \frac{P(A \cap B)}{P(B)} P(AB)=P(B)P(AB)
P ( B ) = 0 P(B)=0 P(B)=0时,规定 P ( A ∣ B ) = 0 P(A|B) =0 P(AB)=0, 由此得出乘法公式:

  • P ( A ∩ B ) = P ( A ∣ B ) ( B ) = P ( B ∣ A ) P ( A ) P(A\cap B)=P(A|B)(B)=P(B|A)P(A) P(AB)=P(AB)(B)=P(BA)P(A)
  • P ( A 1 A 2 ⋅ ⋅ ⋅ A n ) = P ( A 1 ) P ( A 2 ∣ A 1 ) P ( A 3 ∣ A 1 A 2 ) ⋅ ⋅ ⋅ P ( A n ∣ A 1 A 2 ⋅ ⋅ ⋅ A n − 1 ) , P ( A 1 A 2 ⋅ ⋅ ⋅ A n − 1 ) > 0 P(A_{1}A_{2}···A_{n})=P(A_{1})P(A_{2}|A_{1})P(A_{3}|A_{1}A_{2})···P(A_{n}|A_{1}A_{2}···A_{n-1}),P(A_{1}A_{2}···A_{n-1}) > 0 P(A1A2An)=P(A1)P(A2A1)P(A3A1A2)P(AnA1A2An1)P(A1A2An1)>0

2、独立性公式

  • P ( A ∣ B ) = P ( A ) P(A|B)=P(A) P(AB)=P(A),则称事件A关于事件B是独立的
  • 事件A、B相互独立的充要条件 P ( A ∩ B ) = P ( A ) P ( B ) P(A \cap B)=P(A)P(B) P(AB)=P(A)P(B)

3、全概率公式
若事件 B 1 , B 2 , ⋅ ⋅ ⋅ B_{1} ,B_{2} ,··· B1,B2, 满足 B i ∩ B j = ∅ ( i ≠ j ) 互 斥 B_{i} ∩B_{j} =∅(i≠j) 互斥 BiBj=i=j P ( ⋃ i = 1 k B i ) = 1 , P ( B i > 0 ) , i = 1 , 2 , ⋅ ⋅ ⋅ P(\bigcup_{i=1}^{k} B_{i})=1,P(B_{i}>0),i=1,2,··· P(i=1kBi)=1P(Bi>0)i=12
对任一事件A,有下式成立(若 B i B_{i} Bi 只有 n 个也成立) ∑ i = 1 ∞ P ( A ∣ B i ) P ( B i ) \sum_{i=1}^{\infty } P(A|B_{i})P(B_{i}) i=1P(ABi)P(Bi)推导: P ( A ) = P ( A B 1 ) + P ( A B 2 ) + ⋅ ⋅ ⋅ + P ( A B n ) = P ( A ∣ B 1 ) ( B 1 ) + P ( A ∣ B 2 ) ( B 2 ) + ⋅ ⋅ ⋅ + P ( A ∣ B n ) ( B n ) = ∑ i = 1 n P ( A ∣ B i ) P ( B i ) \begin{aligned} P(A)&=P(AB_{1})+P(AB_{2})+···+P(AB_{n})\\ &=P(A|B_{1})(B_{1}) +P(A|B_{2})(B_{2})+···+P(A|B_{n})(B_{n}) \\ &=\sum_{i=1}^{n } P(A|B_{i})P(B_{i}) \end{aligned} P(A)=P(AB1)+P(AB2)++P(ABn)=P(AB1)(B1)+P(AB2)(B2)++P(ABn)(Bn)=i=1nP(ABi)P(Bi)

4、贝叶斯(Bayes)公式(要求各事件彼此独立
若事件 B 1 , B 2 , ⋅ ⋅ ⋅ B_{1} ,B_{2} ,··· B1,B2, 满足全概率公式条件,则对于任一事件 A (P(A) > 0)有下式成立(若 B i B_{i} Bi 只有 n 个也成立)
P ( B i ∣ A ) = P ( A ∣ B i ) P ( B i ) ∑ i = 1 ∞ P ( A ∣ B i ) P ( B i ) P(B_{i}|A)=\frac{P(A|B_{i})P(B_{i})}{\sum_{i=1}^{\infty } P(A|B_{i})P(B_{i})} P(BiA)=i=1P(ABi)P(Bi)P(ABi)P(Bi)


四、概率推理方法(不精确推理)
概率方法不精确性推理的目的就是求出在证据 E 下结论 H 发生的概率 P(H|E)

概率推理方法具有较强的理论基础和较好的数学描述。当证据和结论彼此独立时,计算并不复杂,但是获取概率数据相当困难。如果证据间存在依赖关系,那么不能直接采用这种方法。

1、设有产生式规则:IF E THEN H

对象 说明
E 证据/前提条件
H 结论
目的 求 P(H|E)
P(H) H 的先验概率

使用贝叶斯方法用于不精确推理的一个原始条件是:P(E)、P(E|H)、P(H)已知,有 P ( H ∣ E ) = P ( E ∣ H ) P ( H ) P ( E ) P(H|E)=\frac{P(E|H)P(H)}{P(E)} P(HE)=P(E)P(EH)P(H)

2、若一个证据 E 支持多个假设 H 1 , H 2 , ⋅ ⋅ ⋅ , H n H_{1},H_{2},···,H_{n} H1H2Hn,即 IF E THEN H i H_{i} Hi,i = 1,2,···,n,则可得贝叶斯公式 P ( H i ∣ E ) = P ( E ∣ H i ) P ( H i ) ∑ j = 1 n P ( E ∣ H j ) P ( H j ) , i = 1 , 2 , ⋅ ⋅ ⋅ , n P(H_{i}|E)=\frac{P(E|H_{i})P(H_{i})}{\sum_{j=1}^{n } P(E|H_{j})P(H_{j})},i=1,2,···,n P(HiE)=j=1nP(EHj)P(Hj)P(EHi)P(Hi),i=1,2,,n

3、若有多个证据 E 1 , E 2 , ⋅ ⋅ ⋅ , E m E_{1},E_{2},···,E_{m} E1E2Em 支持多个假设 H 1 , H 2 , ⋅ ⋅ ⋅ , H n H_{1},H_{2},···,H_{n} H1H2Hnn为结论 H 的个数),并且每个证据都以一定程度支持结论,则可得贝叶斯公式 P ( H i ∣ E 1 E 2 ⋅ ⋅ ⋅ E m ) = P ( E 1 ∣ H i ) P ( E 2 ∣ H i ) ⋅ ⋅ ⋅ P ( E m ∣ H i ) P ( H i ) ∑ j = 1 n P ( E 1 ∣ H j ) P ( E 2 ∣ H j ) ⋅ ⋅ ⋅ P ( E m ∣ H j ) P ( H j ) P(H_{i}|E_{1}E_{2}···E_{m})=\frac{P(E_{1}|H_{i})P(E_{2}|H_{i})···P(E_{m}|H_{i})P(H_{i})}{\sum_{j=1}^{n } P(E_{1}|H_{j})P(E_{2}|H_{j})···P(E_{m}|H_{j})P(H_{j})} P(HiE1E2Em)=j=1nP(E1Hj)P(E2Hj)P(EmHj)P(Hj)P(E1Hi)P(E2Hi)P(EmHi)P(Hi)此时,知道P( H i H_{i} Hi), P ( E 1 ∣ H i ) 、 P ( E 2 ∣ H i ) 、 ⋅ ⋅ ⋅ 、 P ( E m ∣ H i ) P(E_{1}|H_{i})、P(E_{2}|H_{i})、···、P(E_{m}|H_{i}) P(E1Hi)P(E2Hi)P(EmHi),就可求得 P ( H i ∣ E 1 E 2 ⋅ ⋅ ⋅ E m ) P(H_{i}|E_{1}E_{2}···E_{m}) P(HiE1E2Em)

五、例题

1、设 H 1 , H 2 , H 3 H_{1},H_{2},H_{3} H1H2H3 为三个结论,E 是支持这些结论的证据,且已知:

P ( H 1 ) = 0.3 P(H_{1}) = 0.3 P(H1)=0.3 P ( H 2 ) = 0.4 P(H_{2}) = 0.4 P(H2)=0.4 P ( H 3 ) = 0.5 P(H_{3}) = 0.5 P(H3)=0.5
P ( E ∣ H 1 ) = 0.5 P(E|H_{1}) = 0.5 P(EH1)=0.5 P ( E ∣ H 2 ) = 0.3 P(E|H_{2}) = 0.3 P(EH2)=0.3 P ( E ∣ H 3 ) = 0.4 P(E|H_{3}) = 0.4 P(EH3)=0.4

P ( H 1 ∣ E ) , P ( H 2 ∣ E ) , P ( H 3 ∣ E ) P(H_{1}|E),P(H_{2}|E),P(H_{3}|E) P(H1E)P(H2E)P(H3E) 的值。

:根据题意 n = 3,分别将 i = 1,2,3 代入下列公式 P ( H i ∣ E ) = P ( E ∣ H i ) P ( H i ) ∑ j = 1 n P ( E ∣ H j ) P ( H j ) , i = 1 , 2 , ⋅ ⋅ ⋅ , n P(H_{i}|E)=\frac{P(E|H_{i})P(H_{i})}{\sum_{j=1}^{n } P(E|H_{j})P(H_{j})},i=1,2,···,n P(HiE)=j=1nP(EHj)P(Hj)P(EHi)P(Hi),i=1,2,,n
P ( H 1 ∣ E ) = P ( E ∣ H 1 ) P ( H 1 ) P ( E ∣ H 1 ) P ( H 1 ) + P ( E ∣ H 2 ) P ( H 2 ) + P ( E ∣ H 3 ) P ( H 3 ) = 0.15 0.15 + 0.12 + 0.2 ( ≈ 0.32 ) = 0.32 \begin{aligned} P(H_{1}|E)&=\frac{P(E|H_{1})P(H_{1})}{P(E|H_{1})P(H_{1})+P(E|H_{2})P(H_{2})+P(E|H_{3})P(H_{3})}\\ &=\frac{0.15}{0.15+0.12+0.2}(\approx0.32 )\\ &=0.32 \end{aligned} P(H1E)=P(EH1)P(H1)+P(EH2)P(H2)+P(EH3)P(H3)P(EH1)P(H1)=0.15+0.12+0.20.15(0.32)=0.32
P ( H 2 ∣ E ) = 0.26 , P ( H 3 ∣ E ) = 0.43 P(H_{2}|E)=0.26,P(H_{3}|E)=0.43 P(H2E)=0.26P(H3E)=0.43
计算结果表明:由于证据 E 的出现, H 1 H_{1} H1 成立的可能性略有增加,而 H 2 H_{2} H2 H 3 H_{3} H3 成立的可能性却有不同程度的下降。

2、已知:

P ( H 1 ) = 0.4 P(H_{1}) = 0.4 P(H1)=0.4 P ( H 2 ) = 0.3 P(H_{2}) = 0.3 P(H2)=0.3 P ( H 3 ) = 0.3 P(H_{3}) = 0.3 P(H3)=0.3
P ( E 1 ∣ H 1 ) = 0.5 P(E_{1}|H_{1}) = 0.5 P(E1H1)=0.5 P ( E 1 ∣ H 2 ) = 0.6 P(E_{1}|H_{2}) = 0.6 P(E1H2)=0.6 P ( E 1 ∣ H 3 ) = 0.3 P(E_{1}|H_{3}) = 0.3 P(E1H3)=0.3
P ( E 2 ∣ H 1 ) = 0.7 P(E_{2}|H_{1}) = 0.7 P(E2H1)=0.7 P ( E 2 ∣ H 2 ) = 0.9 P(E_{2}|H_{2}) = 0.9 P(E2H2)=0.9 P ( E 2 ∣ H 3 ) = 0.1 P(E_{2}|H_{3}) = 0.1 P(E2H3)=0.1

P ( H 1 ∣ E 1 E 2 ) , P ( H 2 ∣ E 1 E 2 ) , P ( H 3 ∣ E 1 E 2 ) P(H_{1}|E_{1}E_{2}),P(H_{2}|E_{1}E_{2}),P(H_{3}|E_{1}E_{2}) P(H1E1E2)P(H2E1E2)P(H3E1E2) 的值。
:根据题意 n = 3,m = 2 ,分别将 i = 1,2,3 代入下列公式
P ( H i ∣ E 1 E 2 ⋅ ⋅ ⋅ E m ) = P ( E 1 ∣ H i ) P ( E 2 ∣ H i ) ⋅ ⋅ ⋅ P ( E m ∣ H i ) P ( H i ) ∑ j = 1 n P ( E 1 ∣ H j ) P ( E 2 ∣ H j ) ⋅ ⋅ ⋅ P ( E m ∣ H j ) P ( H j ) P(H_{i}|E_{1}E_{2}···E_{m})=\frac{P(E_{1}|H_{i})P(E_{2}|H_{i})···P(E_{m}|H_{i})P(H_{i})}{\sum_{j=1}^{n } P(E_{1}|H_{j})P(E_{2}|H_{j})···P(E_{m}|H_{j})P(H_{j})} P(HiE1E2Em)=j=1nP(E1Hj)P(E2Hj)P(EmHj)P(Hj)P(E1Hi)P(E2Hi)P(EmHi)P(Hi) P ( H 1 ∣ E 1 E 2 ) = P ( E 1 ∣ H 1 ) P ( E 2 ∣ H 1 ) P ( H 1 ) P ( E 1 ∣ H 1 ) P ( E 2 ∣ H 1 ) P ( H 1 ) + P ( E 1 ∣ H 2 ) P ( E 2 ∣ H 2 ) P ( H 2 ) + P ( E 1 ∣ H 3 ) P ( E 2 ∣ H 3 ) P ( H 3 ) = 0.45 \begin{aligned}P(H_{1}|E_{1}E_{2})&=\frac{P(E_{1}|H_{1})P(E_{2}|H_{1})P(H_{1})}{ P(E_{1}|H_{1})P(E_{2}|H_{1})P(H_{1}) + P(E_{1}|H_{2})P(E_{2}|H_{2})P(H_{2})+ P(E_{1}|H_{3})P(E_{2}|H_{3})P(H_{3})}\\ &=0.45\end{aligned} P(H1E1E2)=P(E1H1)P(E2H1)P(H1)+P(E1H2)P(E2H2)P(H2)+P(E1H3)P(E2H3)P(H3)P(E1H1)P(E2H1)P(H1)=0.45
P ( H 2 ∣ E 1 E 2 ) = 0.52 , P ( H 2 ∣ E 1 E 2 ) = 0.03 P(H_{2}|E_{1}E_{2}) = 0.52,P(H_{2}|E_{1}E_{2})=0.03 P(H2E1E2)=0.52P(H2E1E2)=0.03
计算结果表明:由于证据 E 1 E_{1} E1 E 2 E_{2} E2 的出现,使 H 1 H_{1} H1 H 2 H_{2} H2 成立的有不同程度的增加,而 H 3 H_{3} H3 成立的可能性下降了。

你可能感兴趣的:(人工智能,概率论,机器学习)