python：马尔科夫链
参考学习网址：MCMC(二)马尔科夫链 - 刘建平Pinard - 博客园 (cnblogs.com)
目录
一、什么是马尔科夫链
1.介绍
2.基本概念
3.马尔科夫链模型分析
二、马尔科夫链模型状态转移矩阵的性质（代码实例）
sample1:
sample2：
sample3
一、什么是马尔科夫链
1.介绍
马尔科夫链（Markov chains）是一种非常常见且相对简单的统计随机过程，从文本生成到金融建模，它们在许多不同领域都得到了应用。马尔科夫链在概念上非常直观且易于实现，因为它们不需要使用任何高级的数学概念，是一种概率建模和数据分析的经典方法、是一类重要的随机过程，它的状态空间是有限的或可数无限的。经过一段时间系统从一个状态转到另一个状态这种进程只依赖于当前出发时的状态而与以前的历史无关。马尔可夫链有着广泛的应用，也是研究排队系统的重要工具。
2.基本概念
离散时间参数的马尔可夫链
定义1：
定义2：
定义3：
连续时间参数的马尔可夫链
定义1：
定义2：
3.马尔科夫链模型分析
简单来说，随机过程就是使用统计模型一些事物的过程进行预测和处理 ，比如天气预报通过今天是否下雨，预测明天后天是否下雨。这些过程都是可以通过数学公式进行量化计算的。通过下雨、晴天、下雪的概率，用公式就可以推导出来 N 天后的状况。
这个马尔科夫链是表示股市模型的，共有三种状态：牛市（Bull market）, 熊市（Bear market）和横盘（Stagnant market）。每一个状态都以一定的概率转化到下一个状态。比如，牛市以0.025的概率转化到横盘的状态。这个状态概率转化图可以以矩阵的形式表示。如果我们定义矩阵阵P某一位置P(i,j)的值为P(j|i)即从状态i转化到状态j的概率，并定义牛市为状态0， 熊市为状态1, 横盘为状态2. 这样我们得到了马尔科夫链模型的状态转移矩阵为：
二、马尔科夫链模型状态转移矩阵的性质（代码实例）
仍然以上面的这个状态转移矩阵为例。假设我们当前股市的概率分布为：[0.3,0.4,0.3],即30%概率的牛市，40%概率的熊盘与30%的横盘。然后这个状态作为序列概率分布的初始状态t0，将其带入这个状态转移矩阵计算t1,t2,t3...的状态。代码如下：
sample1:
import numpy as np
matrix = np.matrix([[0.9, 0.075, 0.025], [0.15, 0.8, 0.05], [0.25, 0.25, 0.5]], dtype=float)
vector1 = np.matrix([[0.3, 0.4, 0.3]], dtype=float)
for i in range(100):
vector1 = vector1 * matrix
print("Current round:", i + 1)
print(vector1)
结果截图：
具体运行结果：
Current round: 1
[[0.405 0.4175 0.1775]]
Current round: 2
[[0.4715 0.40875 0.11975]]
Current round: 3
[[0.5156 0.3923 0.0921]]
Current round: 4
[[0.54591 0.375535 0.078555]]
Current round: 5
[[0.567288 0.36101 0.071702]]
Current round: 6
[[0.5826362 0.3492801 0.0680837]]
Current round: 7
[[0.59378552 0.34014272 0.06607176]]
Current round: 8
[[0.60194632 0.33316603 0.06488765]]
Current round: 9
[[0.6079485 0.32790071 0.06415079]]
Current round: 10
[[0.61237646 0.3239544 0.06366914]]
Current round: 11
[[0.61564926 0.32100904 0.0633417 ]]
Current round: 12
[[0.61807111 0.31881635 0.06311253]]
Current round: 13
[[0.61986459 0.31718655 0.06294886]]
Current round: 14
[[0.62119333 0.3159763 0.06283037]]
Current round: 15
[[0.62217803 0.31507813 0.06274383]]
Current round: 16
[[0.62290791 0.31441182 0.06268027]]
Current round: 17
[[0.62344896 0.31391762 0.06263343]]
Current round: 18
[[0.62385006 0.31355112 0.06259882]]
Current round: 19
[[0.62414743 0.31327936 0.06257322]]
Current round: 20
[[0.62436789 0.31307785 0.06255426]]
Current round: 21
[[0.62453135 0.31292843 0.06254022]]
Current round: 22
[[0.62465253 0.31281765 0.06252982]]
Current round: 23
[[0.62474238 0.31273552 0.0625221 ]]
Current round: 24
[[0.624809 0.31267462 0.06251639]]
Current round: 25
[[0.62485839 0.31262947 0.06251215]]
Current round: 26
[[0.624895 0.31259599 0.06250901]]
Current round: 27
[[0.62492215 0.31257117 0.06250668]]
Current round: 28
[[0.62494228 0.31255277 0.06250495]]
Current round: 29
[[0.62495721 0.31253912 0.06250367]]
Current round: 30
[[0.62496827 0.31252901 0.06250272]]
Current round: 31
[[0.62497648 0.31252151 0.06250202]]
Current round: 32
[[0.62498256 0.31251594 0.0625015 ]]
Current round: 33
[[0.62498707 0.31251182 0.06250111]]
Current round: 34
[[0.62499041 0.31250876 0.06250082]]
Current round: 35
[[0.62499289 0.3125065 0.06250061]]
Current round: 36
[[0.62499473 0.31250482 0.06250045]]
Current round: 37
[[0.62499609 0.31250357 0.06250034]]
Current round: 38
[[0.6249971 0.31250265 0.06250025]]
Current round: 39
[[0.62499785 0.31250196 0.06250018]]
Current round: 40
[[0.62499841 0.31250146 0.06250014]]
Current round: 41
[[0.62499882 0.31250108 0.0625001 ]]
Current round: 42
[[0.62499912 0.3125008 0.06250008]]
Current round: 43
[[0.62499935 0.31250059 0.06250006]]
Current round: 44
[[0.62499952 0.31250044 0.06250004]]
Current round: 45
[[0.62499964 0.31250033 0.06250003]]
Current round: 46
[[0.62499974 0.31250024 0.06250002]]
Current round: 47
[[0.6249998 0.31250018 0.06250002]]
Current round: 48
[[0.62499985 0.31250013 0.06250001]]
Current round: 49
[[0.62499989 0.3125001 0.06250001]]
Current round: 50
[[0.62499992 0.31250007 0.06250001]]
Current round: 51
[[0.62499994 0.31250005 0.06250001]]
Current round: 52
[[0.62499996 0.31250004 0.0625 ]]
Current round: 53
[[0.62499997 0.31250003 0.0625 ]]
Current round: 54
[[0.62499998 0.31250002 0.0625 ]]
Current round: 55
[[0.62499998 0.31250002 0.0625 ]]
Current round: 56
[[0.62499999 0.31250001 0.0625 ]]
Current round: 57
[[0.62499999 0.31250001 0.0625 ]]
Current round: 58
[[0.62499999 0.31250001 0.0625 ]]
Current round: 59
[[0.62499999 0.3125 0.0625 ]]
Current round: 60
[[0.625 0.3125 0.0625]]
Current round: 61
[[0.625 0.3125 0.0625]]
Current round: 62
[[0.625 0.3125 0.0625]]
Current round: 63
[[0.625 0.3125 0.0625]]
Current round: 64
[[0.625 0.3125 0.0625]]
Current round: 65
[[0.625 0.3125 0.0625]]
Current round: 66
[[0.625 0.3125 0.0625]]
Current round: 67
[[0.625 0.3125 0.0625]]
Current round: 68
[[0.625 0.3125 0.0625]]
Current round: 69
[[0.625 0.3125 0.0625]]
Current round: 70
[[0.625 0.3125 0.0625]]
Current round: 71
[[0.625 0.3125 0.0625]]
Current round: 72
[[0.625 0.3125 0.0625]]
Current round: 73
[[0.625 0.3125 0.0625]]
Current round: 74
[[0.625 0.3125 0.0625]]
Current round: 75
[[0.625 0.3125 0.0625]]
Current round: 76
[[0.625 0.3125 0.0625]]
Current round: 77
[[0.625 0.3125 0.0625]]
Current round: 78
[[0.625 0.3125 0.0625]]
Current round: 79
[[0.625 0.3125 0.0625]]
Current round: 80
[[0.625 0.3125 0.0625]]
Current round: 81
[[0.625 0.3125 0.0625]]
Current round: 82
[[0.625 0.3125 0.0625]]
Current round: 83
[[0.625 0.3125 0.0625]]
Current round: 84
[[0.625 0.3125 0.0625]]
Current round: 85
[[0.625 0.3125 0.0625]]
Current round: 86
[[0.625 0.3125 0.0625]]
Current round: 87
[[0.625 0.3125 0.0625]]
Current round: 88
[[0.625 0.3125 0.0625]]
Current round: 89
[[0.625 0.3125 0.0625]]
Current round: 90
[[0.625 0.3125 0.0625]]
Current round: 91
[[0.625 0.3125 0.0625]]
Current round: 92
[[0.625 0.3125 0.0625]]
Current round: 93
[[0.625 0.3125 0.0625]]
Current round: 94
[[0.625 0.3125 0.0625]]
Current round: 95
[[0.625 0.3125 0.0625]]
Current round: 96
[[0.625 0.3125 0.0625]]
Current round: 97
[[0.625 0.3125 0.0625]]
Current round: 98
[[0.625 0.3125 0.0625]]
Current round: 99
[[0.625 0.3125 0.0625]]
Current round: 100
[[0.625 0.3125 0.0625]]
可以发现，从第60轮开始，我们的状态概率分布就不变了，一直保持在[0.625 0.3125 0.0625]，即62.5%的牛市，31.25%的熊市与6.25%的横盘。
我们现在换一个初始概率分布试一试，现在我们用[0.7,0.1,0.2]作为初始概率分布，然后这个状态作为序列概率分布的初始状态t0，将其带入这个状态转移矩阵计算t1,t2,t3...的状态。代码如下：
sample2：
import numpy as np
matrix = np.matrix([[0.9, 0.075, 0.025], [0.15, 0.8, 0.05], [0.25, 0.25, 0.5]], dtype=float)
vector1 = np.matrix([[0.7, 0.1, 0.2]], dtype=float)
for i in range(100):
vector1 = vector1 * matrix
print("Current round:", i + 1)
print(vector1)结果截图：
具体运行结果：
Current round: 1
[[0.695 0.1825 0.1225]]
Current round: 2
[[0.6835 0.22875 0.08775]]
Current round: 3
[[0.6714 0.2562 0.0724]]
Current round: 4
[[0.66079 0.273415 0.065795]]
Current round: 5
[[0.652172 0.28474 0.063088]]
Current round: 6
[[0.6454378 0.2924769 0.0620853]]
Current round: 7
[[0.64028688 0.29791068 0.06180244]]
Current round: 8
[[0.6363954 0.30180067 0.06180393]]
Current round: 9
[[0.63347695 0.30462117 0.06190188]]
Current round: 10
[[0.6312979 0.30668318 0.06201892]]
Current round: 11
[[0.62967532 0.30819862 0.06212607]]
Current round: 12
[[0.62846909 0.30931606 0.06221485]]
Current round: 13
[[0.6275733 0.31014174 0.06228495]]
Current round: 14
[[0.62690847 0.31075263 0.0623389 ]]
Current round: 15
[[0.62641525 0.31120496 0.06237979]]
Current round: 16
[[0.62604941 0.31154006 0.06241053]]
Current round: 17
[[0.62577811 0.31178839 0.0624335 ]]
Current round: 18
[[0.62557693 0.31197244 0.06245062]]
Current round: 19
[[0.62542776 0.31210888 0.06246336]]
Current round: 20
[[0.62531716 0.31221003 0.06247282]]
Current round: 21
[[0.62523515 0.31228501 0.06247984]]
Current round: 22
[[0.62517435 0.31234061 0.06248505]]
Current round: 23
[[0.62512926 0.31238182 0.06248891]]
Current round: 24
[[0.62509584 0.31241238 0.06249178]]
Current round: 25
[[0.62507106 0.31243504 0.0624939 ]]
Current round: 26
[[0.62505268 0.31245184 0.06249548]]
Current round: 27
[[0.62503906 0.31246429 0.06249665]]
Current round: 28
[[0.62502896 0.31247352 0.06249752]]
Current round: 29
[[0.62502147 0.31248037 0.06249816]]
Current round: 30
[[0.62501592 0.31248545 0.06249863]]
Current round: 31
[[0.6250118 0.31248921 0.06249899]]
Current round: 32
[[0.62500875 0.312492 0.06249925]]
Current round: 33
[[0.62500649 0.31249407 0.06249944]]
Current round: 34
[[0.62500481 0.3124956 0.06249959]]
Current round: 35
[[0.62500357 0.31249674 0.06249969]]
Current round: 36
[[0.62500264 0.31249758 0.06249977]]
Current round: 37
[[0.62500196 0.31249821 0.06249983]]
Current round: 38
[[0.62500145 0.31249867 0.06249988]]
Current round: 39
[[0.62500108 0.31249901 0.06249991]]
Current round: 40
[[0.6250008 0.31249927 0.06249993]]
Current round: 41
[[0.62500059 0.31249946 0.06249995]]
Current round: 42
[[0.62500044 0.3124996 0.06249996]]
Current round: 43
[[0.62500033 0.3124997 0.06249997]]
Current round: 44
[[0.62500024 0.31249978 0.06249998]]
Current round: 45
[[0.62500018 0.31249984 0.06249998]]
Current round: 46
[[0.62500013 0.31249988 0.06249999]]
Current round: 47
[[0.6250001 0.31249991 0.06249999]]
Current round: 48
[[0.62500007 0.31249993 0.06249999]]
Current round: 49
[[0.62500005 0.31249995 0.0625 ]]
Current round: 50
[[0.62500004 0.31249996 0.0625 ]]
Current round: 51
[[0.62500003 0.31249997 0.0625 ]]
Current round: 52
[[0.62500002 0.31249998 0.0625 ]]
Current round: 53
[[0.62500002 0.31249999 0.0625 ]]
Current round: 54
[[0.62500001 0.31249999 0.0625 ]]
Current round: 55
[[0.62500001 0.31249999 0.0625 ]]
Current round: 56
[[0.62500001 0.31249999 0.0625 ]]
Current round: 57
[[0.625 0.3125 0.0625]]
Current round: 58
[[0.625 0.3125 0.0625]]
Current round: 59
[[0.625 0.3125 0.0625]]
Current round: 60
[[0.625 0.3125 0.0625]]
Current round: 61
[[0.625 0.3125 0.0625]]
Current round: 62
[[0.625 0.3125 0.0625]]
Current round: 63
[[0.625 0.3125 0.0625]]
Current round: 64
[[0.625 0.3125 0.0625]]
Current round: 65
[[0.625 0.3125 0.0625]]
Current round: 66
[[0.625 0.3125 0.0625]]
Current round: 67
[[0.625 0.3125 0.0625]]
Current round: 68
[[0.625 0.3125 0.0625]]
Current round: 69
[[0.625 0.3125 0.0625]]
Current round: 70
[[0.625 0.3125 0.0625]]
Current round: 71
[[0.625 0.3125 0.0625]]
Current round: 72
[[0.625 0.3125 0.0625]]
Current round: 73
[[0.625 0.3125 0.0625]]
Current round: 74
[[0.625 0.3125 0.0625]]
Current round: 75
[[0.625 0.3125 0.0625]]
Current round: 76
[[0.625 0.3125 0.0625]]
Current round: 77
[[0.625 0.3125 0.0625]]
Current round: 78
[[0.625 0.3125 0.0625]]
Current round: 79
[[0.625 0.3125 0.0625]]
Current round: 80
[[0.625 0.3125 0.0625]]
Current round: 81
[[0.625 0.3125 0.0625]]
Current round: 82
[[0.625 0.3125 0.0625]]
Current round: 83
[[0.625 0.3125 0.0625]]
Current round: 84
[[0.625 0.3125 0.0625]]
Current round: 85
[[0.625 0.3125 0.0625]]
Current round: 86
[[0.625 0.3125 0.0625]]
Current round: 87
[[0.625 0.3125 0.0625]]
Current round: 88
[[0.625 0.3125 0.0625]]
Current round: 89
[[0.625 0.3125 0.0625]]
Current round: 90
[[0.625 0.3125 0.0625]]
Current round: 91
[[0.625 0.3125 0.0625]]
Current round: 92
[[0.625 0.3125 0.0625]]
Current round: 93
[[0.625 0.3125 0.0625]]
Current round: 94
[[0.625 0.3125 0.0625]]
Current round: 95
[[0.625 0.3125 0.0625]]
Current round: 96
[[0.625 0.3125 0.0625]]
Current round: 97
[[0.625 0.3125 0.0625]]
Current round: 98
[[0.625 0.3125 0.0625]]
Current round: 99
[[0.625 0.3125 0.0625]]
Current round: 100
[[0.625 0.3125 0.0625]]
可以看出，尽管这次我们采用了不同初始概率分布，最终状态的概率分布趋于同一个稳定的概率分布[0.625 0.3125 0.0625]， 也就是说我们的马尔科夫链模型的状态转移矩阵收敛到的稳定概率分布与我们的初始状态概率分布无关。这是一个非常好的性质，也就是说，如果我们得到了这个稳定概率分布对应的马尔科夫链模型的状态转移矩阵，则我们可以用任意的概率分布样本开始，带入马尔科夫链模型的状态转移矩阵，这样经过一些序列的转换，最终就可以得到符合对应稳定概率分布的样本。
这个性质不光对我们上面的状态转移矩阵有效，对于绝大多数的其他的马尔科夫链模型的状态转移矩阵也有效。同时不光是离散状态，连续状态时也成立。
sample3
同时，对于一个确定的状态转移矩阵P，它的n次幂Pn在当n大于一定的值的时候也可以发现是确定的，我们还是以上面的例子为例，计算代码如下：
import numpy as np
matrix = np.matrix([[0.9, 0.075, 0.025], [0.15, 0.8, 0.05], [0.25, 0.25, 0.5]], dtype=float)
for i in range(10):
matrix = matrix * matrix
print("Current round:", i + 1)
print(matrix)
结果截图：
具体运行结果：
Current round: 1
[[0.8275 0.13375 0.03875]
[0.2675 0.66375 0.06875]
[0.3875 0.34375 0.26875]]
Current round: 2
[[0.73555 0.212775 0.051675]
[0.42555 0.499975 0.074475]
[0.51675 0.372375 0.110875]]
Current round: 3
[[0.65828326 0.28213131 0.05958543]
[0.56426262 0.36825403 0.06748335]
[0.5958543 0.33741675 0.06672895]]
Current round: 4
[[0.62803724 0.30972343 0.06223933]
[0.61944687 0.3175772 0.06297594]
[0.6223933 0.3148797 0.062727 ]]
Current round: 5
[[0.62502532 0.31247685 0.06249783]
[0.6249537 0.31254233 0.06250397]
[0.62497828 0.31251986 0.06250186]]
Current round: 6
[[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]]
Current round: 7
[[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]]
Current round: 8
[[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]]
Current round: 9
[[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]]
Current round: 10
[[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]
[0.625 0.3125 0.0625]]
我们可以发现，在n≥6以后，Pn的值稳定不再变化，而且每一行都为[0.625 0.3125 0.0625]，这和我们前面的稳定分布是一致的。这个性质同样不光是离散状态，连续状态时也成立。