Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization

• 目标：实时任意风格转移

• 方法：adaptive instance normalization

• 原理：图像的风格就是特征图各个feature channel跨空间的统计信息，比如mean和variance。迁移各个channel的mean和variance就可以实现风格迁移。

• 效果：可实时实现任意风格图片转移，并且可以控制content-style trade-off,style interpolation,color,spatial

• code片段：

def adaptive_instance_normalization(content_feat, style_feat):
    assert (content_feat.data.size()[:2] == style_feat.data.size()[:2])
    size = content_feat.data.size()
    style_mean, style_std = calc_mean_std(style_feat)
    content_mean, content_std = calc_mean_std(content_feat)

    normalized_feat = (content_feat - content_mean.expand(
        size)) / content_std.expand(size)
    return normalized_feat * style_std.expand(size) + style_mean.expand(size)

def style_transfer(vgg, decoder, content, style, alpha=1.0):
    assert (0.0 <= alpha <= 1.0)
    content_f = vgg(content)
    style_f = vgg(style)
    feat = adaptive_instance_normalization(content_f, style_f)
    feat = feat * alpha + content_f * (1 - alpha)
    return decoder(feat)

network.png

demo.png

• paper
• 官方github
• 非官方github