DyNet 基础知识(持续更新中...)

1. ParameterCollection()

DyNet最基础的概念之一是参数集合,其定义为如下:
class dynet.ParameterCollection(parent=None)
模型所需要参数的创建、加载和保存都通过此类对象实现,是普通参数和查找表参数的容器。dy.Trainer对象认为ParameterCollection对象中的参数是需要通过训练学习得到的。

add_lookup_parameters(dim=(vocab_szie,emb_size), init=None, name='')
ParameterCollection对象添加查找表参数,默认初始化方式为GlorotInitializer。下面的函数功能与之类似,函数返回值类型都是dynet.Parameters对象

add_parameters(dim, init=None, name='')

1.1 paras.as_array()

通过以上两种方式创建的参数类型均为dynet.Parameters对象,具有函数as_array(),返回值为参数对应的ndarray类型

2. Expressions

Expressions are the building block of a Dynet computation graph.

Expressions are the main data types being manipulated in a DyNet program. Each expression represents a sub-computation in a computation graph.

Expressions are used as an interface to the various functions that can be used to build DyNet computation graphs.

2.1 dynet.parameter()

dynet.parameter(*args)

Add parameters to the computation graph.

Get the expression objects corresponding to parameters. Gradients for parameters will be computed and used by Optimizers to update.

因为通过ParameterCollection的对象并没有加入到计算图中,因此需要上述函数将参数添加进computation graph。函数返回值类型是expression

2.2 dynet.concatenate()

参数xs为需要串接的向量或矩阵元素列表,在函数执行过程中会将需要使用的向量元素加入到计算图中。在nlp应用中因此就不需要专门将dynet.LookupParameters添加进计算图中,而相较之下普通的模型参数dynet.Parameters就需要专门函数语句来将其加入计算图中。当需要串接矩阵时(通常是在minibatch情况下),串接操作作用在矩阵的dim=0维度上,即将矩阵按列进行串接起来,输入矩阵的维度应为 [# of vectors to be concatenated, #batches]。

3. Operations

Operations are used to build expressions.

3.1 dynet.affine_transform()

计算仿射变换,输入参数为列表形式,且包含奇数个元素

  • Parameters:exprs (list) – A list containing an odd number of expressions
  • Returns:An expression equal to: x s [ 0 ] + x s [ 1 ] ∗ x s [ 2 ] + . . . xs[0] + xs[1]*xs[2] + ... xs[0]+xs[1]xs[2]+...
  • Return typedynet.Expression

3.2 dynet.pickneglogsoftmax()

Negative softmax log likelihood

This function takes in a vector of scores x x x, and performs a log softmax, takes the negative, and selects the likelihood corresponding to the element v v v. This is perhaps the most standard loss function for training neural networks to predict one out of a set of elements.

  • Parameters
    • x x xdynet.Expression)- input scores
    • v v vint)- true class
  • Returns − log ⁡ e x v ∑ j e x j -\log\frac{e^{x_v}}{\sum_{j}e^{x_j}} logjexjexv
  • Return typedynet.Expression

3.3 dynet.log_softmax()

函数功能为输入向量的 subset 计算softmax;具体来讲就是没有包含在参数 r e s t r i c t restrict restrict(形式为list)中的元素将会被置为负无穷,包含在参数内的输入向量的子向量会被计算softmax,默认为输入向量的所有元素计算softmax。

  • Parameters:
    • x x x (dynet.Expression) – Input expression
    • r e s t r i c t restrict restrict (list) – List of log softmax to compute (default: (None))
  • Returns: A vector with the log softmax over the specified elements
  • Return type: dynet.Expression

4. Optimizers

优化方法包括梯度下降系列的SGD、Adam、RMSProp等等,也包括dropout这样的tricks。

4.1 dynet.dropout()

Dropout

With a fixed probability, drop out (set to zero) nodes in the input expression, and scale the remaining nodes by 1 p \frac{1}{p} p1.

  • Parameters
    • x x x (dynet.Expression) – Input expression
    • p p p (number) – The dropout probability
  • Returns:The dropped out expression
  • Return typedynet.Expression

你可能感兴趣的:(Python)