markov

随机漫步
$$P(X_{t+1}|...,X_{t-2},X_{t-1},X_t)=P(X_{t+1}|X_t)$$
收敛性
收敛到均衡

• 可能的状态数是有限的
• 转移概率固定不变
• 从任意状态能够转移到任意其他状态
• 不是简单循环

import numpy as np


def markov():
    init_array = np.array([.3, .2, .4, .1])
    transfer_matrix = np.array([
        [0, .5, 0, 0],
        [0.333, 0, 0, .5],
        [0.333, 0, 1, .5],
        [0.333, .5, 0, 0]
    ])

    restmp = init_array
    for i in range(100):
        tmp = np.dot(restmp, transfer_matrix)
        print(i, '===', restmp)
        restmp = tmp


markov()

0 === [0.3 0.2 0.4 0.1]
1 === [0.2331 0.2    0.4    0.3   ]
2 === [0.2997  0.26655 0.4     0.3    ]
3 === [0.32186115 0.29985    0.4        0.333275  ]
4 === [0.34403063 0.32756808 0.4        0.349925  ]
5 === [0.35880519 0.34697781 0.4        0.36378404]
6 === [0.3698837  0.36129462 0.4        0.37348891]
7 === [0.37788291 0.3716863  0.4        0.38064731]
8 === [0.38372709 0.37926511 0.4        0.38584315]
9 === [0.38798105 0.38478512 0.4        0.38963256]
10 === [0.39108109 0.3888068  0.4        0.39239256]
11 === [0.39333939 0.39173682 0.4        0.3944034 ]
12 === [0.39498469 0.39387139 0.4        0.39586841]
13 === [0.39618336 0.39542655 0.4        0.3969357 ]
14 === [0.39705663 0.39655953 0.4        0.39771328]
15 === [0.39769284 0.39738495 0.4        0.39827976]
16 === [0.39815635 0.3979863  0.4        0.39869248]
17 === [0.39849403 0.39842441 0.4        0.39899315]
18 === [0.39874005 0.39874359 0.4        0.39921221]
19 === [0.39891928 0.39897613 0.4        0.3993718 ]
20 === [0.39904986 0.39914554 0.4        0.39948806]
21 === [0.39914499 0.39926896 0.4        0.39957277]
22 === [0.3992143  0.39935888 0.4        0.39963448]
23 === [0.39926479 0.39942439 0.4        0.39967944]
24 === [0.39930157 0.39947211 0.4        0.39971219]
25 === [0.39932837 0.39950688 0.4        0.39973606]
26 === [0.3993479  0.39953222 0.4        0.39975344]
27 === [0.39936212 0.39955067 0.4        0.39976611]
28 === [0.39937249 0.39956412 0.4        0.39977534]
29 === [0.39938004 0.39957391 0.4        0.39978206]
30 === [0.39938554 0.39958105 0.4        0.39978696]
31 === [0.39938955 0.39958625 0.4        0.39979052]
32 === [0.39939246 0.39959003 0.4        0.39979312]
33 === [0.39939459 0.39959279 0.4        0.39979502]
34 === [0.39939614 0.3995948  0.4        0.3997964 ]
35 === [0.39939727 0.39959627 0.4        0.3997974 ]
36 === [0.39939809 0.39959734 0.4        0.39979813]
37 === [0.39939869 0.39959811 0.4        0.39979867]
38 === [0.39939913 0.39959868 0.4        0.39979906]
39 === [0.39939945 0.39959909 0.4        0.39979934]
40 === [0.39939968 0.39959939 0.4        0.39979955]
41 === [0.39939985 0.39959961 0.4        0.3997997 ]
42 === [0.39939997 0.39959977 0.4        0.39979981]
43 === [0.39940006 0.39959989 0.4        0.39979989]
44 === [0.39940012 0.39959997 0.4        0.39979994]
45 === [0.39940017 0.39960003 0.4        0.39979999]
46 === [0.39940021 0.39960008 0.4        0.39980002]
47 === [0.39940023 0.39960011 0.4        0.39980004]
48 === [0.39940025 0.39960014 0.4        0.39980006]
49 === [0.39940026 0.39960015 0.4        0.39980007]
50 === [0.39940027 0.39960017 0.4        0.39980008]
51 === [0.39940028 0.39960018 0.4        0.39980008]
52 === [0.39940029 0.39960018 0.4        0.39980009]
53 === [0.39940029 0.39960019 0.4        0.39980009]
54 === [0.39940029 0.39960019 0.4        0.39980009]
55 === [0.39940029 0.39960019 0.4        0.3998001 ]
56 === [0.3994003  0.39960019 0.4        0.3998001 ]
57 === [0.3994003 0.3996002 0.4       0.3998001]
58 === [0.3994003 0.3996002 0.4       0.3998001]
59 === [0.3994003 0.3996002 0.4       0.3998001]
60 === [0.3994003 0.3996002 0.4       0.3998001]
61 === [0.3994003 0.3996002 0.4       0.3998001]
62 === [0.3994003 0.3996002 0.4       0.3998001]
63 === [0.3994003 0.3996002 0.4       0.3998001]
64 === [0.3994003 0.3996002 0.4       0.3998001]
65 === [0.3994003 0.3996002 0.4       0.3998001]
66 === [0.3994003 0.3996002 0.4       0.3998001]
67 === [0.3994003 0.3996002 0.4       0.3998001]
68 === [0.3994003 0.3996002 0.4       0.3998001]
69 === [0.3994003 0.3996002 0.4       0.3998001]
70 === [0.3994003 0.3996002 0.4       0.3998001]
71 === [0.3994003 0.3996002 0.4       0.3998001]
72 === [0.3994003 0.3996002 0.4       0.3998001]
73 === [0.3994003 0.3996002 0.4       0.3998001]
74 === [0.3994003 0.3996002 0.4       0.3998001]
75 === [0.3994003 0.3996002 0.4       0.3998001]
76 === [0.3994003 0.3996002 0.4       0.3998001]
77 === [0.3994003 0.3996002 0.4       0.3998001]
78 === [0.3994003 0.3996002 0.4       0.3998001]
79 === [0.3994003 0.3996002 0.4       0.3998001]
80 === [0.3994003 0.3996002 0.4       0.3998001]
81 === [0.3994003 0.3996002 0.4       0.3998001]
82 === [0.3994003 0.3996002 0.4       0.3998001]
83 === [0.3994003 0.3996002 0.4       0.3998001]
84 === [0.3994003 0.3996002 0.4       0.3998001]
85 === [0.3994003 0.3996002 0.4       0.3998001]
86 === [0.3994003 0.3996002 0.4       0.3998001]
87 === [0.3994003 0.3996002 0.4       0.3998001]
88 === [0.3994003 0.3996002 0.4       0.3998001]
89 === [0.3994003 0.3996002 0.4       0.3998001]
90 === [0.3994003 0.3996002 0.4       0.3998001]
91 === [0.3994003 0.3996002 0.4       0.3998001]
92 === [0.3994003 0.3996002 0.4       0.3998001]
93 === [0.3994003 0.3996002 0.4       0.3998001]
94 === [0.3994003 0.3996002 0.4       0.3998001]
95 === [0.3994003 0.3996002 0.4       0.3998001]
96 === [0.3994003 0.3996002 0.4       0.3998001]
97 === [0.3994003 0.3996002 0.4       0.3998001]
98 === [0.3994003 0.3996002 0.4       0.3998001]
99 === [0.3994003 0.3996002 0.4       0.3998001]