编程与数学

e=limn→∞(1+1n)n 的收敛性

>>>for n in (2, 4, 10, 50, 100, 1000, 100000):
>>>     print(n, (1./n)**n)
2 2.25
4 2.44140625
10 2.5937424601000023
50 2.691588029073608
100 2.7048138294215285
1000 2.7169239322355936
100000 2.7182682371922975

随机性的实现——各种分布的抽样

二项分布生成掩码，实现对原始数据随机的选择（屏蔽）。

# denoising autoencoder
def get_corrupted_input(self, input, corruption_level):
    mask = self.theano_rng.binomial(n=1,
            p=1-corruption_level,
            size=input.shape, 
            dtype=theano.config.floatX)
    return mask*input
# dropout
def dropout_layer(layer, p_dropout):
    mask = theano_rng.binomial(n=1,
            p=1-p_dropout,
            size=layer.shape,
            dtype=theano.config.floatX)
    return layer*mask

正太分布或者均匀分布（参数由相关定理保证）实现对权值的初始化：

if not W:
    init_W = numpy.asarray(
            numpy_rng.uniform(
                low= -4*numpy.sqrt(6./(n_in+n_out),
                high=4*numpy.sqrt(6./(n_in+n_out)),
                size=(n_in, n_out))
            ),
            dtype=theano.config.floatX
        )
    W = theano.shared(value=init_W, name='W', borrow=True)