概率论与随机过程笔记（2）：离散随机变量（1）

概率论与随机过程笔记（2）：离散随机变量（1）

2019-11-25

---

*这部分的笔记依据Dimitri P. Bertsekas和John N. Tsitsiklis的《概率导论》第2章前半部分的内容。

2.1 关于离散随机变量的几个概念

Given an experiment and the corresponding set of possible outcornes (the sample space), a random variable associates a particular number with each outcome: see Fig. 2.1. We refer to this number as the numerical value or simply the value of the random variable. mathematically, a random variable is a real-valued function of the experimental outcome.

Main Concepts Related to Random Variables:

Starting with a probabilistic model of an experiment:

• A random variable is a real-valued function of the outcome of the experiment.
• A function of a random variable defines another random variable.
• We can associate with each random variable certain “averages” of interest, such as the mean and the variance.
• A random variable can be conditioned on an event or on another random variable.
• There is a notion of independence of a random variable from an event or from another random variable.

A random variable is called discrete if its range (the set of values that it can take) is either finite or countably infinite. A random variable that can take an uncountably infinite number of values is not discrete.

Concepts Related to Discrete Random Variables

Starting with a probabilistic model of an experiment:

• A discrete random variable is a real-valued function of the outcome of the experiment that can take a finite or countably infinite number of values.
• A discrete random variable has an associated probability mass funcion (PMF), which gives the probability of each numeric the random variable can take.
• A function of a discrete random variable defines another discrete random variable, whose PMF can be obtained from the PMF of the original random variable.

2.2 概率质量函数（probability mass function）

The most important way to characterize a random variable is through the probabilities of the values that it can take. For a discrete random variable $X$, these are captured by the probability mass function (PMF for short) of $X$, denoted $p_X$. In particular. if x is any possible value of $X$. the probability mass of $x$. lenoted $p_X(x)$. is the probability of the event $\{X=x\}$ consisting of: that give rise to a value of $X$ equal to $x$: $$p_X(x)=P(\{X=x\})$$

We will use upper case characters ($X$) to denote random variables, and lower case characters ($x$) to denote real numbers such as the numerical values of a random variable. Note that $$\sum _x{p}_X(x)=1$$

Calculation of the PMF of a Random Variable $X$

For each possible value $x$ of $X$:

• collect all the posible outcomes that give rise to the event $\{X=x\}$
• add their probabilities to obtain $p_X(x)$

By a similar argument, for any set $S$ of possible values of $X$,we have $$P(x\in S)=\sum _{x\in S}{p}_X(x)$$

【伯努利随机变量（The Bernoulli Random Variable）】Consider the toss of a coin, which comes up a head with probability $p$, and a tail with probability $1-p$. The Bernoulli random variable takes the two values 1 and 0, depending on whether the outcome is a head or a tail:
$$f(x)=\{\begin{array}{ll}1& \text{if a head}\\ 0& \text{if a tail}\end{array}$$ Its PMF is
$$f(x)=\{\begin{array}{ll}p& \text{if k=1}\\ 1-p& \text{if k=0}\end{array}$$
For all its simplicity. the Bernoulli random variable is very important. In practice. it is used to model generic probabilistic situations with just two outcomes.

【二项随机变量（The Binomial Random Variable）】A coin is tossed n times. At each toss, the coin comes up a head with probability $p$, and a tail with probability $1-p$, independent of prior tosses. Let $X$ be the number of heads in the $n$-toss sequence. We refer to $X$ as a binomial random variable with parameters $n$ and $p$. PMF of $X$ consists of the binomial probabilities that were calculated
$$p_X(k)=P(x=K)=\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}$$The normalization property, specialized to the binomial random variable, is written as $$\sum _{k=0}^n\left(\genfrac{}{}{0ex}{}{n}{k}\right)p^k(1-p{)}^{n-k}=1$$
Some special cases of the binomial PMF are sketched in Fig. 2.3.

【几何随机变量（The Geometric Random Variable）】Suppose that we repeatedly and independently toss a coin with probability of a head equal to $p$, where $0<p<1$. The geometric random variable is the number $X$ of tosses needed for a head to come up for the first time. Its PMF is
given by $$p_X(k)=(1-p{)}^{k-1}p,k=1,2,\cdots ,$$since $(1-p{)}^{k-1}p$ is the probability of the sequence consisting of $k-1$ successive tails followed ba a head; see Fig. 2.4. This is a legitimate PMF because
$$\sum _{k=1}^{\infty }{p}_X(k)=\sum _{k=1}^{\infty }(1-p{)}^{k-1}p$$ $$=p\sum _{k=0}^{\infty }(1-p{)}^k=p\cdot \frac{1}{1-(1-p)}=1$$Naturally, the use of coin tosses here is just to provide insight. More generally, we can interpret the geometric random variable in terms of repeated independent trials until the first “success.” Each trial has probability of success $p$ and the number of trials until (and including) the first success is modeled by the geometric random variable. The meaning of “success” is context-dependent.
【泊松随机变量（The Poisson Random Variable）】A Poisson random variable has a PMF given by $$p_X(k)=e^{-\lambda }\frac{{\lambda }^k}{k!}$$where $\lambda$ is a positive parameter characterizing the PMF, see Fig. 2.5.

This is a legitimate PMF because $$\sum _{k=0}^{\infty }{e}^{-\lambda }\frac{{\lambda }^k}{k!}=e^{-\lambda }(1+\lambda +\frac{{\lambda }^2}{2!}+\frac{{\lambda }^3}{3!}+\cdots )=e^{-\lambda }{e}^{\lambda }=1$$To get a feel for the Poisson random variable. think of a binomial random variable with very small p and very large $n$. For example. let $X$ be the number of typos in a book with a total of $n$ words. Then $X$ is binomial, but since the probability p that any one word is misspelled is very small. $X$ can also be well-modeled with a Poisson PMF (let p be the probability of heads in tossing a coin. and associate misspelled words with coin tosses that result in heads).
More precisely. the Poisson PMF with parameter $\lambda$ is a good approximation for a binomial PMF with parameters $n$ and $p$. i.e … $$e^{-\lambda }\frac{{\lambda }^k}{k!}\approx \frac{n!}{k!(n-k)!}{p}^k(1-p{)}^{n-k}k=0,1,\cdots ,$$provided $\lambda =np$. $n$ is very large. and $p$ is very smalll. In this case. using the
Poisson PMF may result in simpler models and calculations.

2.3 随机变量的函数（Function of Random Variables）

If $Y=g(X)$ is a function of a random variable $X$, then $Y$ is also a random variable, since it provides a numerical value for each possible outcome. This is because every outcome in the sample space defines a numerical value $x$ for $X$ and hence also the numerical value $y=g(x)$ for $Y$. If $X$ is discrete with PMF $p_X$. then $Y$ is also discrete, and its PMF $p_Y$ can be calculated using the PMF of $X$. In particular, to obtain $p_Y(y)$ for any $y$, we add the probabilities of all values of $x$ such that $g(x)=y$: $$p_Y(y)=\sum _{\{x|g(x)=y\}}{p}_X(x)$$

2.4 期望、均值和方差（Expectation, Mean and Variance）

【期望】The PMF of a random variable $X$ provides us with several numbers. the probabilities of all the possible values of $X$. It is often desirable. however, to summarize this information in a single representative number, This is accomplished by the expectation of $X$. which is a weighted (in proportion to probabilities) average of the possible values of $X$.
We define the expected value (also called the expectation or the mean) of a random variable $X$, with PMF $p_X$, by $$E[X]=\sum _{x}x{p}_X(x)$$It is useful to view the mean of $X$ as a ‘‘representative’’ value of $X$, which lies somewhere in the middle of its range. We can make this statement more precise, by viewing the mean as the center of gravity of the PMF. In particular, if the PMF is symmetric around a certain point, that point must be equal to the mean.
【矩（moment）和方差（variance）】We define the 2nd moment of the random variable $X$ as the expected value of the random variable $X^2$. More generally, we define the $n$th moment as $E[X^n]$, the expected value of the random variable $X^n$. With this terminology, the 1st moment of $X$ is just the mean.
Variance is denoted by $\text{var}(X)$ and is defined as the expected value of the random variable $(X-E[X]{)}^2$, i.e., $$\text{var}(X)=E[(X-E[X]{)}^2]$$ Since $(X-E[X]{)}^2$ can only ake nonnegative values, the variance is always nonnegative.
The variance provides a measure of dispersion of $X$ around its mean. Another measure of dispersion is the standard deviation of $X$, which is defined as the square root of the variance and is denoted by ${\sigma }_x$: $$\sigma =\sqrt{\text{var}(X)}$$The standard deviation is often easier to interpret because it has the same units as $X$.
One way to calculate $\text{var}(X)$, is to use the definition of expected value, after calculating the PMF of the random variable $(X-E[X]{)}^2$. It turns out that there is an easier method to calculate $\text{var}(X)$. which uses the PMF of $X$ but does not require the PMF of $(X-E[X]{)}^2$. This method is based on the following rule:

Expected Value Rule for Functions of Radom Variables
Let $X$ be a random variable with PMF $p_X$, and let $g(X)$ be a function of $X$. Then, the expected value of the random variable $g(X)$ is given by $$E[g(X)]=\sum _{x}g(x)p_X(x)$$

To verify this rule, we let $Y=g(X)$ and use the formula $$p_Y(y)=\sum _{\{x|g(x)=y\}}{p}_X(x)$$ derived in the preceding section. We have $$E[g(X)]=EY$$ $$=\sum _{y}y\cdot p_Y(y)$$ $$=\sum _{y}y\sum _{\{x|g(x)=y\}}{p}_X(x)$$ $$=\sum _y\sum _{\{x|g(x)=y\}}y\cdot p_X(x)$$ $$=\sum _y\sum _{\{x|g(x)=y\}}g(x)p_X(x)$$ $$=\sum _{x}g(x)p_X(x)$$ Using the expected value rule, we can write the variance of $X$ as Using the expected value rule, we can write the variance of $X$ as $$\text{var}(X)=E[(X-E(X){)}^2]$$ $$=\sum _x(x-E[X]{)}^2{p}_X(x)$$ Similarly, the $n$th moment is given by $$E[X^n]=\sum _x(x-E[X]{)}^n{p}_X(x)$$
As we have already noted, the variance is always nonnegative, but could it be zero? Since every term in the formula ${\sum }_x(x-E[X]{)}^2{p}_X(x)$ is nonnegative, the sum is zero if and only if $(x-E[X]{)}^n{p}_X(x)=0$ for every $x$. This condition implies that for any $x$ with $px(x)>0$, we must have $x=E[X]$ and the random variable $X$ is not really ‘‘random’’: its value is equal to the mean $E[X]$, with probability 1