贝叶斯决策理论

Bayesian

Bayes's Theorem

prior:
likelihood:
posterior:

Optimal Bayes Decision Rule: minimize the probability of error.
    if then True state of nature ;
    if then True state of nature .

Prove: For a particular ,
         if we decide ;
         if we decide .
Bayes Decision Rule:Decide if ;otherwise decide .
Therefore: .
The unconditional error obtained by integration over all .

Bayesian Decision Theory

c state of nature:
a possible actions:
the loss for taking action when the true state of nature is :

Select the action for which the conditional risk is minimum.
Bayes Risk: .

• Example 1:
  action : deciding
  action : deciding
  
  
  
  if , action is taken: deciding .
• Example 2:
  Suppose
  Conditional risk
  
  Minimizing the risk Maximizing the posterior .
  So we have the discriminant function(max. discriminant corresponds to min. risk):
  
  
  
  
  
  Set of discriminant functions:
  Classifier assigns a feature vector to class if:

Binary classification Multi‐class classfication

• One vs. One
  class, design classifiers, denote for result.
• One vs. Rest
  design classifiers, choose the one which prediction is positive.
• ECOC (Error‐Correcting Output Codes)
  The code consisting of the labels predicted by these classifiers is compared with each line, and the one with the smallest distance between codes is the result.

	f1	f2	f3
c1	-1	1	-1
c2	1	-1	-1
c3	-1	1	1