SRCNN通过三个卷积层来完成对图像的超分,也就是特征提取、特征映射和图像重建。
但是其输入需要预先插值为目标尺寸,在速度上有一定的缺陷,难以应用到实时的研究中,因此Dong等人(SRCNN的第一作者)又改进了SRCNN,于是有了FSRCNN,其中的F代表Fast。
As a successful deep model applied in image super-resolution (SR), the Super-Resolution Convolutional Neural Network (SRCNN) has demonstrated superior performance to the previous hand- crafted models either in speed and restoration quality. However, the high computational cost still hinders it from practical usage that demands real-time performance (24 fps). In this paper, we aim at accelerating the current SRCNN, and propose a compact hourglass-shape CNN struc- ture for faster and better SR. We re-design the SRCNN structure mainly in three aspects. First, we introduce a deconvolution layer at the end of the network, then the mapping is learned directly from the original low-resolution image (without interpolation) to the high-resolution one. Second, we reformulate the mapping layer by shrinking the input feature dimension before mapping and expanding back afterwards. Third, we adopt smaller filter sizes but more mapping layers. The proposed model achieves a speed up of more than 40 times with even superior restora- tion quality. Further, we present the parameter settings that can achieve real-time performance on a generic CPU while still maintaining good performance. A corresponding transfer strategy is also proposed for fast training and testing across different upscaling factors.
其新的网络架构为:
FSRCNN对于原来的SRCNN,其改进主要有:
之前讲过,用Pytorch写代码就是搭积木,看着网络结构一点点写就好,下面这行图更清晰点(来自参考链接【2】):
其结构就像是一个沙漏,整体上是对称的,两端比较粗,中间比较细。
Interestingly, the new structure looks like an hourglass, which is symmetri- cal on the whole, thick at the ends, and thin in the middle.
class FSRCNN(nn.Module):
def __init__(self, inchannels):
super(FSRCNN, self).__init__()
self.features = nn.Sequential(
nn.Conv2d(in_channels=inchannels, out_channels=64, kernel_size=5, stride=1, padding=2, padding_mode='replicate'),
nn.PReLU()
)
self.shrinking = nn.Sequential(
nn.Conv2d(in_channels=64,out_channels=32,kernel_size=1, stride=1, padding=0, padding_mode='replicate'),
nn.PReLU()
)
self.mapping = nn.Sequential(
nn.Conv2d(in_channels=32, out_channels=32, kernel_size=3, stride =1, padding=1, padding_mode='replicate'),
nn.Conv2d(in_channels=32, out_channels=32, kernel_size=3, stride =1, padding=1, padding_mode='replicate'),
nn.Conv2d(in_channels=32, out_channels=32, kernel_size=3, stride =1, padding=1, padding_mode='replicate'),
nn.Conv2d(in_channels=32, out_channels=32, kernel_size=3, stride =1, padding=1, padding_mode='replicate'),
nn.PReLU()
)
self.expanding = nn.Sequential(
nn.Conv2d(in_channels=32, out_channels=64, kernel_size=1, stride=1, padding=0),
nn.PReLU()
)
self.deconv = nn.Sequential(
nn.ConvTranspose2d(in_channels=64, out_channels=inchannels, kernel_size=9, stride=3, padding=4, padding_mode='replicate')
)
def forward(self, x):
x = self.features(x)
x = self.shrinking(x)
x = self.mapping(x)
x = self.expanding(x)
x = self.deconv(x)
return x
整体上来说,FSRCNN就是分为上面5个部分:
2. 最后一个反卷积层的参数要自己根据输出的倍数仔细算一下。
【1】DONG C, LOY C C, TANG X. Accelerating the Super-Resolution Convolutional Neural Network[C]//Computer Vision – ECCV 2016.Springer International Publishing,2016:391-407. 10.1007/978-3-319-46475-6_25.
【2】https://towardsdatascience.com/review-fsrcnn-super-resolution-80ca2ee14da4
【3】https://github.com/Lornatang/FSRCNN-PyTorch
本文由 mdnice 多平台发布