Tensorflow复用计算图 reuse=tf.AUTO_REUSE

假设有一个场景, 一个结算的函数用于账单的计算, 需要对物品收取5%的税额和10元的运费. 在计算图中我们可以定义以下:

def add_tax_and_shipping(price):
    """Add tax and shipping
    """
    with tf.variable_scope("other_charge") as scope:
        tax = tf.get_variable("tax", (), dtype=tf.float32,
                              initializer=tf.constant_initializer(0.05))  # 0.05%
        shipping = tf.get_variable("shipping", (), dtype=tf.float32,
                                   initializer=tf.constant_initializer(10.0))  # $10
    total_price = price * (1.0 + tax) + shipping
    return total_price

因此, 当我们输入单价到这个图中, 会得到结果

def main():
    book_price = add_tax_and_shipping(10.0)
    init_op = tf.global_variables_initializer()  # Set up operator to assign all init values to variables
    with tf.Session() as s:
        s.run(init_op)  # Actually assign initial value to variables
        price = s.run(book_price)
        print("Price of the book: %f" % (price))
if __name__ == "__main__":
    main()

Price of the book: 20.500000

接下来, 又有一个场景, 我们需要计算另一个物品(sketchpad)的总价, 因此按照原来的做法, 同样运行

book_price = add_tax_and_shipping(10.0)
sketchpad_price = add_tax_and_shipping(20.0)

但是会返回错误:
ValueError: Variable other_charge/tax already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope?
因为此时计算图的节点已经存在了, 新建同名节点会导致错误, 而且也不是我们的期望, 正确的操作是对计算图进行复用. 那么, 我们是不是可以直使用 reuse=True ?

with tf.variable_scope("other_charge", reuse=True) as scope:  # This line causes an exception to be thrown on the next line!
        tax = tf.get_variable("tax", (), dtype=tf.float32,
                              initializer=tf.constant_initializer(0.05))  # 0.05%
        shipping = tf.get_variable("shipping", (), dtype=tf.float32,
                                   initializer=tf.constant_initializer(10.0))  # $10

结果:
ValueError: Variable other_charge/tax does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?
因为我们在首次调用计算图reuse时, 计算图并不存在, 所以产生错误.

解决:

设置为 AUTO_REUSE

with tf.variable_scope("other_charge", reuse=tf.AUTO_REUSE) as scope:
使用AUTO_REUSE是一种方便的做法, 但是要避免产生意外的结果, 这样会导致权值共享问题.
比如定义一个卷积层, 但是要避免对卷积的权值参数进行复用.

def single_conv_block(tens, block_name, input_filters, output_filters, kernel_size=3, padding="SAME", strides=1,
                      is_training=True):
    scope_name = block_name + "_cv"
    with tf.variable_scope(scope_name) as scope:
        weights = tf.get_variable("weights", [kernel_size, kernel_size, input_filters, output_filters],
                                  dtype=tf.float32,
                                  initializer=tf.contrib.layers.xavier_initializer(uniform=False, seed=0))
        bias = tf.get_variable("bias", [output_filters], dtype=tf.float32)
        z = tf.add(tf.nn.conv2d(tens, weights, strides=[1, strides, strides, 1], padding=padding), bias)
        bn = tf.layers.batch_normalization(z, training=is_training)
        activation = tf.nn.relu(bn, name=scope.name)
        return activation

在第二次调用时设置reuse=True

def add_tax_and_shipping(price, reuse=False):
    """Add tax and shipping
    """
    with tf.variable_scope("other_charge", reuse=reuse) as scope:
        tax = tf.get_variable("tax", (), dtype=tf.float32,
                              initializer=tf.constant_initializer(0.05))  # 0.05%
        shipping = tf.get_variable("shipping", (), dtype=tf.float32,
                                   initializer=tf.constant_initializer(10.0))  # $10
def main():
    book_price = add_tax_and_shipping(10.0)
    sketchpad_price = add_tax_and_shipping(20.0, reuse=True)