13句对匹配实战-(3)Decomposable

image.png

两个句子：

image.png

1. 先对a,b进行一次前馈神经网络+relu函数,提取了一次特征向量

image.png

2. 然后a中的每个词对b中每个词的attention做一次权重加权，当然a中的每个词对b中每个词的attention做一次权重加权

就是a对应句子b中的词，b对应句子a中的词

image.png

3. 比较：其 [a¯i,βi][a¯i,βi] 就是把两部分concat起来
   

   （句子中第 i 个词, 和另一个句子里所有词与它比较的加权词向量）
   image.png

4. 总计aggregate：
   就是分别du把对v1,v2v1,v2 中的每个元素求和，得到两个值之后做分类。

   
   
   image.png

 def _compare(self, sentence, attention, sentence_length):
    # sent_and_alignment has shape [batch, time_steps, num_units]
    inputs = [sentence, attention, sentence - attention, sentence * attention]
    features = tf.concat(axis=2, values=inputs)
    return self._feedforward(features, self._m_config['compare']['dim'])

def _aggregate(self, v1, v2):
    # sum over time steps; resulting shape is [batch, num_units]
    v1 = self._mask3d(v1, self._m_ph_sent1_size, 0, 1)
    v2 = self._mask3d(v2, self._m_ph_sent2_size, 0, 1)
    v1_sum = tf.reduce_sum(v1, 1)
    v2_sum = tf.reduce_sum(v2, 1)
    v1_max = tf.reduce_max(v1, 1)
    v2_max = tf.reduce_max(v2, 1)
    aggregate_features = tf.concat([v1_sum, v2_sum, v1_max, v2_max], axis=1)
    return self._feedforward(aggregate_features, self._m_config['aggregate']['dim'])

def _feedforward(self, inputs, hidden_units):
    if isinstance(hidden_units, int):
        hidden_units = [hidden_units]

    last_inputs = inputs
    for i, dim in enumerate(hidden_units):
        with tf.variable_scope('layer_%d' % (i)):
            inputs = tf.nn.dropout(last_inputs, self._m_ph_dropout_rate)
            last_inputs = tf.layers.dense(inputs, dim, tf.nn.relu)
    return last_inputs

def _mask3d(self, values, sentence_sizes, mask_value, axis=2):
    if axis != 1 and axis != 2:
        raise ValueError("'axis' must be 1 or 2")

    if axis == 1:
        values = tf.transpose(values, [0, 2, 1])

    time_steps1 = tf.shape(values)[1]
    time_steps2 = tf.shape(values)[2]

    # [batch_size, time_step1, time_step2]
    ones = tf.ones_like(values, dtype=tf.float32)
    pad_values = mask_value * ones
    # [batch_size, time_step2]
    mask = tf.sequence_mask(sentence_sizes, time_steps2)

    # [batch_size, 1, time_step2]
    mask3d = tf.expand_dims(mask, 1)
    # [batch_size, time_step1, time_step2]
    mask3d = tf.tile(mask3d, (1, time_steps1, 1))

    masked = tf.where(mask3d, values, pad_values)
    if axis == 1:
        masked = tf.transpose(masked, [0, 2, 1])
    return masked