Tensorflow2.0 十行代码实现SRCNN
超分辨率技术（Super-Resolution, SR）是指从观测到的低分辨率图像重建出相应的高分辨率图像，在监控设备、卫星图像和医学影像等领域都有重要的应用价值。
SRCNN是深度学习用在超分辨率重建上的开山之作(Image Super-Resolution Using Deep Convolutional Networks)，SRCNN的网络结构非常简单，仅仅用了三个卷积层，网络结构如下图所示。
SRCNN首先使用双三次(bicubic)插值将低分辨率图像放大成目标尺寸，接着通过三层卷积网络拟合非线性映射，最后输出高分辨率图像结果。本文中，作者将三层卷积的结构解释成三个步骤：图像块的提取和特征表示，特征非线性映射和最终的重建。
三个卷积层使用的卷积核的大小分为为9x9,，1x1和5x5，前两个的输出特征个数分别为64和32。用Timofte数据集（包含91幅图像）和ImageNet大数据集进行训练。使用均方误差(Mean Squared Error, MSE)作为损失函数，有利于获得较高的PSNR。
以上引自（有改动）：从SRCNN到EDSR，总结深度学习端到端超分辨率方法发展历程
SRCNN代码使用Cifar10数据集，由32*32*3无损放大为128*128*3，以下全部代码见Github。
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
inputs = keras.Input(shape=(128, 128, 3), name='img')
x = layers.Conv2D(
filters=64, # 卷积层神经元（卷积核）数目
kernel_size=9, # 感受野大小
padding='same', # padding策略（vaild 或 same）
activation=tf.nn.relu # 激活函数
)(inputs)
x = layers.Conv2D(
filters=32,
kernel_size=1,
padding='same',
activation=tf.nn.relu
)(x)
outputs = layers.Conv2D(
filters=3,
kernel_size=5,
padding='same' # 不设置激活函数
)(x)
model = keras.Model(inputs=inputs, outputs=outputs, name='SRCNN_model')# 通过在图层图中指定其输入和输出来创建一个model
model.summary() # 查看模型摘要，需要模型built（实例化）后调用Model: "SRCNN_model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= img (InputLayer) [(None, 128, 128, 3)] 0 _________________________________________________________________ conv2d (Conv2D) (None, 128, 128, 64) 15616 _________________________________________________________________ conv2d_1 (Conv2D) (None, 128, 128, 32) 2080 _________________________________________________________________ conv2d_2 (Conv2D) (None, 128, 128, 3) 2403 ================================================================= Total params: 20,099 Trainable params: 20,099 Non-trainable params: 0 _________________________________________________________________
加载数据
import cv2 as cv
import numpy as np
'''
CIFAR10 had't 128*128*3 images use bicubic alternatived,and bicubic use nearest alternatived.
X_: image applied bicubic interpolation (low-resolution),(50000, 128, 128, 3)
y_: image with original resolution (high-resolution),(10000, 128, 128, 3)
'''
ishape = 128
# load data
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.cifar10.load_data()
# 缩小数据集,控制内存占用(以下win10，8G内存可用)
train_image = train_images[0:10000]
test_image = test_images[0:1000]
X_train = np.array([cv.resize(i,(ishape,ishape), interpolation=cv.INTER_NEAREST) for i in train_image]) / 255.
X_test = np.array([cv.resize(i,(ishape,ishape), interpolation=cv.INTER_NEAREST) for i in test_image]) / 255.
y_train = np.array([cv.resize(i,(ishape,ishape), interpolation=cv.INTER_CUBIC) for i in train_image]) / 255.
y_test = np.array([cv.resize(i,(ishape,ishape), interpolation=cv.INTER_CUBIC) for i in test_image]) / 255.开始训练
model.compile(optimizer=tf.keras.optimizers.Adam(0.01),
loss='mse',
metrics=['mae']) # 编译
history = model.fit(X_train, y_train,
batch_size=64,
epochs=3,
validation_split=0.2)# 训练
test_scores = model.evaluate(X_test, y_test, verbose=2) # 评估
print('Test loss:', test_scores[0])
print('Test mae:', test_scores[1])
# Save entire model to a HDF5 file
model.save('SRCNN.h5')开始应用
from matplotlib import pyplot as plt
ishape = 128
#加载放大图像并显示
img = cv.imread('automobile.png')
img = cv.cvtColor(img, cv.COLOR_BGR2RGB)
img = cv.resize(img,(ishape,ishape), interpolation=cv.INTER_NEAREST)# (36,36,3)->(128,128,3)
plt.imshow(img)
plt.xticks([]), plt.yticks([])
plt.show()原图像：
img = np.reshape(img,(1,ishape,ishape,3)) / 255.
# 处理图像超分辨率
img_SR = model.predict(img)
plt.imshow(img_SR[0])
plt.xticks([]), plt.yticks([])
plt.show()SRCNN处理后图像：
