[深度学习论文笔记][Neural Arts] A Neural Algorithm of Artistic Style

Gatys, Leon A., Alexander S. Ecker, and Matthias Bethge. “A neural algorithm of artistic style.” arXiv preprint arXiv:1508.06576 (2015). (Citations: 99).

1 Motivation

Given a content image and a style image, we want to obtain a synthesised image which matches the content representation of the content image and the style representation of the style image.

2 Method

We compute the correlations between the different filter responses as the style statistics, where the expectation is taken over the spatial extend of the input image. These feature correlations are given by the Gram matrix G ∈ R^{D×D}.

Start from an random noise image, we joint minimize the content representation and style representation.

3 Experiments
We matched the content representation on layer conv4-2 and the style representations on layers conv1-1, conv2-1, conv3-1, conv4-1, and conv5-1 of VGGNet. See result in Fig.

It turns out that when matching the style representations up to higher layers in the network, local images structures are matched on an increasingly large scale, leading to a smoother and more continuous visual experience. This can be explained by the increasing receptive field sizes and feature complexity along the networks processing hierarchy.

A strong emphasis on style will result in images that match the appearance of the style image, effectively giving a texturised version of it, but hardly show any of appearance of
the content image. When placing strong emphasis on content, one can clearly identify the photograph, but the style is not as well-matched. For a specific pair of source images one can adjust the trade-off between content and style to create visually appealing images.

4 References
[1]. Two Minute Papers - Deep Neural Network Learns Van Gogh’s Art. https://www.youtube.com/watch?v=-R9bJGNHltQ.