Chapter 3 (General Random Variables): The Continuous Bayes‘ Rule (连续贝叶斯准则)

本文为 $Introduction$ $to$ $Probability$ 的读书笔记

The Continuous Bayes’ Rule

$$f_{X|Y}(x|y)=\frac{f_X(x)f_{Y|X}(y|x)}{f_Y(y)}=\frac{f_X(x)f_{Y|X}(y|x)}{{\int }_{-\infty }^{\infty }{f}_X(t)f_{Y|X}(y|t)dt}$$

---

Example 3.19.
A light bulb is known to have an exponentially distributed lifetime $Y$. On any given day, the parameter $\lambda$ of the PDF of $Y$ is actually a random variable, uniformly distributed in the interval $[1,3/2]$. We test a light bulb and record its lifetime. What can we say about the underlying parameter $\lambda$?

SOLUTION

• We model the parameter $\lambda$ in terms of a uniform random variable $\Lambda$ with PDF
  $$\begin{gathered} f_{\Lambda }(\lambda )=2,for1\le \lambda \le \frac{3}{2} \\ {f}_{\Lambda |Y}(\lambda |y)=\frac{f_{\Lambda }(\lambda )f_{Y|\Lambda }(y|\lambda )}{{\int }_{-\infty }^{\infty }{f}_{\Lambda }(t)f_{Y|\Lambda }(y|t)dt}=\frac{2\lambda e^{-\lambda y}}{{\int }_1^{3/2}2t{e}^{-ty}dt},for1\le \lambda \le \frac{3}{2} \end{gathered}$$

Inference about a Discrete Random Variable

• We first consider the case where the unobserved phenomenon is described in terms of an event $A$ whose occurrence is unknown. Let $P(A)$ be the probability of event $A$. Let $Y$ be a continuous random variable, and assume that the conditional PDFs $f_{Y|A}(Y$) and $f_{Y|A^C}(y)$ are known. We are interested in the conditional probability $P(A|Y=y)$ of the event $A$, given the value $y$ of $Y$.
• Instead of working with the conditioning event $\{Y=y\}$, which has zero probability, let us instead condition on the event $\{y\le Y\le y+\delta \}$, where $\delta$ is a small positive number, and then take the limit as $\delta$ tends to zero. We have, using Bayes’rule, and assuming that $f_Y(y)>0$,
  $$\begin{array}{rl}P(A|Y=y)& \approx P(A|y\le Y\le y+\delta )\\ & =\frac{P(A)P(y\le Y\le y+\delta |A)}{P(y\le Y\le y+\delta )}\\ & \approx \frac{P(A)f_{Y|A}(y)\delta }{f_Y(y)\delta }\\ & =\frac{P(A)f_{Y|A}(y)}{f_Y(y)}\\ & =\frac{P(A)f_{Y|A}(y)}{P(A)f_{Y|A}(y)+P(A^C)f_{Y|A^C}(y)}\end{array}$$
• In a variant of this formula, we consider an event $A$ of the form $\{N=n\}$, where $N$ is a discrete random variable that represents the different discrete possibilities for the unobserved phenomenon of interest. Let $p_N$ be the PMF of $N$. Let also $Y$ be a continuous random variable which, for any given value $n$ of $N$, is described by a conditional PDF $f_{Y|N}(y|n)$. The above formula becomes
  $$P(N=n|Y=y)=\frac{p_N(n)f_{Y|N}(y|n)}{f_Y(y)}=\frac{p_N(n)f_{Y|N}(y|n)}{{\sum }_i{p}_N(i)f_{Y|N}(y|i)}$$

---

Example 3.20. Signal Detection.
A binary signal $S$ is transmitted, and we are given that $P(S=1)=p$ and $P(S=-1)=1-p$. The received signal is $Y=N+S$, where $N$ is normal noise, with zero mean and unit variance, independent of $S$. What is the probability that $S=1$, as a function of the observed value $y$ of $Y$?

SOLUTION

• Conditioned on $S=s$, the random variable $Y$ has a normal distribution with mean $s$ and unit variance. Applying the formulas given above, we obtain
  $$P(S=1|Y=y)=\frac{p_S(1)f_{Y|S}(y|1)}{f_Y(y)}=\frac{\frac{p}{\sqrt{2\pi }}{e}^{-(y-1{)}^2/2}}{\frac{p}{\sqrt{2\pi }}{e}^{-(y-1{)}^2/2}+\frac{1-p}{\sqrt{2\pi }}{e}^{-(y+1{)}^2/2}}=\frac{p{e}^y}{p{e}^y+(1-p)e^{-y}}$$

Inference Based on Discrete Observations

• Our earlier forrnula expressing $P(A|Y=y)$ in terms of $f_{Y|A}(y)$ can be turned around to yield
  $$\begin{array}{rl}{f}_{Y|A}(y)& =\frac{f_Y(y)P(A|Y=y)}{P(A)}\\ & =\frac{f_Y(y)P(A|Y=y)}{{\int }_{-\infty }^{\infty }{f}_Y(t)P(A|Y=t)dt}\end{array}$$This formula can be used to make an inference about a random variable $Y$ when an event $A$ is observed.
• There is a similar formula for the case where the event $A$ is of the form $\{N=n\}$, where $N$ is an observed discrete random variable that depends on $Y$ in a manner described by a conditional PMF $p_{N|Y}(n|y)$

---

Problem 35.
Let $X$ and $Y$ be independent continuous random variables with PDFs $f_X$ and $f_Y$, respectively, and let $Z=X+Y$.

• (a) Show that $f_{Z|X}(z|x)=f_Y(z-x)$. [Hint: Write an expression for the conditional CDF of $Z$ given $X$, and differentiate.]
• (b) Assume that $X$ and $Y$ are exponentially distributed with parameter $\lambda$. Find the conditional PDF of $X$, given that $Z=z$.

SOLUTION

• $(a)$ We have
  $$\begin{array}{rl}P(Z\le z|X=x)& =P(X+Y\le z|X=x)\\ & =P(x+Y\le z|X=x)\\ & =P(x+Y\le z)\\ & =P(Y\le z-x)\end{array}$$where the third equality follows from the independence of $X$ and $Y$. By differentiating both sides with respect to $z$, the result follows.
• $(b)$ We have, for $0\le x\le z$,
  $$f_{X|Z}(x|z)=\frac{f_{Z|X}(z|x)f_X(x)}{f_Z(z)}=\frac{f_Y(z-x)f_X(x)}{f_Z(z)}=\frac{{\lambda }^2{e}^{-\lambda z}}{f_Z(z)}$$Since this is the same for all $x$, it follows that the conditional distribution of $X$ is uniform on the interval $[0,z]$, with PDF $f_{X|Z}(x|z)=1/z$.