【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer

论文:https://arxiv.org/pdf/1903.00834.pdf

Project Page:http://web.eecs.utk.edu/~zzhang61/project_page/SRNTT/SRNTT.html

Adobe、田纳西大学

https://www.cnblogs.com/alan-blog-TsingHua/p/10564535.html

https://blog.csdn.net/wangchy29/article/details/88566724

 

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第1张图片

基于跟LR及其不相似的图片恢复细节时会抑制不相似的位置,同时也避免制造出虚假的细节。

1、论文贡献(共三点):

  • 提出了一种新的网络super-resolution feedback network (SRFBN)
  • 提出了一种新的结构feedback block (FB)
  • 提出了一种新的训练策略curriculum-based training strategy

2、算法大概流程

SRNTT structure

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第2张图片

 

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第3张图片

processes

  • Feature Swapping

First,apply bicubic up-sampling on ILR to get an upscaled LR image ILR↑

Then,sequentially apply bicubic downsampling and up-sampling with the same factor on IRef to obtain a blurry Ref image IRef↓↑ that matches the frequency band of ILR↑

借鉴CrossNet [41]优点  Instead of estimating a global transformation or optical flow, we match the local patches in ILR↑ and IRef↓↑ so that there is no constraint on the global structure of the Ref image

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第4张图片

Pi(·) denotes sampling the i-th patch from neural feature map, and si,j is the similarity between the i-th LR patch and the j-th Ref patch.

Sj is the similarity map for the j-th Ref patch, and ∗ denotes the correlation operation

  • Neural Texture Transfer

Losses

  • texture loss

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第5张图片

  •  Reconstruction loss

  • Perceptual loss

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第6张图片

where V and C indicate the volume and channel number of the feature maps, respectively, and φi denotes the ith channel of the feature maps extracted from the hidden layer(relu5_1) of VGG19 model. || · ||F denotes the Frobenius norm

  • Adversarial loss

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第7张图片

 

3、实验

Settings

特征提取层来自于不同的VGG层,relu1_1,relu2_1,relu3_1。

的权值分别为1,1e-4,1e-6,1e-4

学习率设置为1e-4.优化器为adam.

网络先只用重构损失训练2epochs,再用全部的损失训练20epochs。

论文还对IRef做数据增强,获取其对应的放缩和旋转

Results

【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer_第8张图片

SRNTT-L2是用MSE

你可能感兴趣的:(【Super Resolution】【论文阅读】【CVPR2019】【SRNTT】Image Super-Resolution by Neural Texture Transfer)