Example 3 for Markov Chain -《双态机-Two-state machine》

Consider a machine that has two state, i.e., up and down. The probability of it is down on day n+1 when it is up on day n is a,  The probability of it is up on day n+1 when it is down on day n is b, where both a and b range in [0,1], and both are independent often past. Let Xn be the state of the machine(0 if it is up, and 1 if it is down).

Then we can plot the state transition diagram as follows:

The one-step and n-step state transition Matrix is 

.

Then the N-step transition Matrix is 

• If |1-a-b|=1, P=P^n=I, for a+b=0; 
• If |1-a-b|<1,

The stationary distribution of Xn can be calculated by two methods as follows:

• If |1-a-b|=1, 

• If |1-a-b|<1,