MMEditing中超分模型训练与测试

OpenMMLab 项目中开源的 MMEditing，是一个图像和视频编辑的工具箱，它目前包含了常见的编辑任务，比如图像修复，图像抠图，超分辨率和生成模型。组合使用以上任务的一个统一的框架下。
参考：https://zhuanlan.zhihu.com/p/178867385
https://github.com/open-mmlab/mmediting
https://mmediting.readthedocs.io/zh_CN/latest/
https://wenku.baidu.com/view/7ccdd97dbd23482fb4daa58da0116c175f0e1e6d.html

以第一个BasicVSR为例，介绍训练和测试过程。
下载mmediting代码

这里有每个模型的文件，打开basicvsr文件夹。

针对两个数据集有三种训练方式，我这里用的是reds数据集。

mmediting-master$ ./tools/dist_train.sh configs/restorers/basicvsr/basicvsr_reds4.py 4

把basicvsr_reds4.py里边的数据集路径换成自己的。
运行时有个奇怪的报错，len(dataset)是 266，后边就报找不到对应的图片。可我的数据集明明是240个文件夹。这里为了快速训练，就在代码中写死了len(dataset)=236.目前看训练结果正常。

2022-06-10 06:20:39,602 - mmedit - INFO - Iter [100/300000]	lr_generator: 2.500e-05, eta: 9 days, 9:08:48, time: 2.703, data_time: 0.884, memory: 10216, loss_pix: 0.0365, loss: 0.0365
2022-06-10 06:23:29,353 - mmedit - INFO - Iter [200/300000]	lr_generator: 2.500e-05, eta: 7 days, 15:12:45, time: 1.697, data_time: 0.006, memory: 10216, loss_pix: 0.0316, loss: 0.0316
2022-06-10 06:26:27,526 - mmedit - INFO - Iter [300/300000]	lr_generator: 2.500e-05, eta: 7 days, 3:32:29, time: 1.782, data_time: 0.005, memory: 10216, loss_pix: 0.0283, loss: 0.0283
2022-06-10 06:29:31,442 - mmedit - INFO - Iter [400/300000]	lr_generator: 2.500e-05, eta: 6 days, 22:52:37, time: 1.839, data_time: 0.005, memory: 10216, loss_pix: 0.0266, loss: 0.0266
2022-06-10 06:32:30,556 - mmedit - INFO - Iter [500/300000]	lr_generator: 2.500e-05, eta: 6 days, 19:15:32, time: 1.791, data_time: 0.005, memory: 10216, loss_pix: 0.0262, loss: 0.0262
2022-06-10 06:35:26,686 - mmedit - INFO - Iter [600/300000]	lr_generator: 2.500e-05, eta: 6 days, 16:25:08, time: 1.761, data_time: 0.005, memory: 10216, loss_pix: 0.0270, loss: 0.0270
2022-06-10 06:38:26,863 - mmedit - INFO - Iter [700/300000]	lr_generator: 2.500e-05, eta: 6 days, 14:51:14, time: 1.802, data_time: 0.005, memory: 10216, loss_pix: 0.0271, loss: 0.0271
2022-06-10 06:41:26,890 - mmedit - INFO - Iter [800/300000]	lr_generator: 2.500e-05, eta: 6 days, 13:39:17, time: 1.800, data_time: 0.005, memory: 10216, loss_pix: 0.0257, loss: 0.0257
2022-06-10 06:44:22,396 - mmedit - INFO - Iter [900/300000]	lr_generator: 2.500e-05, eta: 6 days, 12:17:31, time: 1.755, data_time: 0.005, memory: 10216, loss_pix: 0.0259, loss: 0.0259

训练时，pin_memory报错。
解决：修改pin_memory=False

创建Dataloader时，pin_memory=True表示将load进的数据拷贝进锁页内存区，将内存中的Tensor转移至GPU cuda区会很快；pin_memory=False表示将load进数据放至非锁页内存区，速度会较慢。

当计算机的内存充足的时候，设置pin_memory=True。当系统卡住，或者交换内存使用过多的时候，设置pin_memory=False。因为pin_memory与电脑硬件性能有关，pin_memory默认为False。

realvsr默认参数使用4个gpu训练，到5000iter时又报显存不够。换成6个gpu，又报数据太少不能训练。
realvsr++在resd数据集上训练6w个iters，测试自己的数据，4倍scale。
realvsr++有三個訓練文件，先在resd数据集上訓練60w個epochs，再在video90k上訓練30w個epochs。

下载处理Video90K数据集
https://blog.csdn.net/change_things/article/details/107287770
https://github.com/xinntao/EDVR/blob/master/scripts/matlab_scripts/generate_LR_Vimeo90K.m
自己照着写了python的处理脚本。

处理reds和video90k数据集的脚本
generate_bicubic_img.m

function generate_bicubic_img()
%% matlab code to genetate mod images, bicubic-downsampled images and
%% bicubic_upsampled images

%% set configurations
% comment the unnecessary lines
% input_folder = '../../datasets/Set5/original';
% save_mod_folder = '../../datasets/Set5/GTmod12';
% save_lr_folder = '../../datasets/Set5/LRbicx2';
input_folder = 'E:/tmp_0617/tmp_0617/images';
save_mod_folder = 'E:/tmp_0617/tmp_0617/save1';
save_lr_folder = 'E:/tmp_0617/tmp_0617/save2';
% save_bic_folder = '';

mod_scale = 12;
up_scale = 4;

if exist('save_mod_folder', 'var')
    if exist(save_mod_folder, 'dir')
        disp(['It will cover ', save_mod_folder]);
    else
        mkdir(save_mod_folder);
    end
end
if exist('save_lr_folder', 'var')
    if exist(save_lr_folder, 'dir')
        disp(['It will cover ', save_lr_folder]);
    else
        mkdir(save_lr_folder);
    end
end
if exist('save_bic_folder', 'var')
    if exist(save_bic_folder, 'dir')
        disp(['It will cover ', save_bic_folder]);
    else
        mkdir(save_bic_folder);
    end
end

idx = 0;
filepaths = dir(fullfile(input_folder,'*.*'));
for i = 1 : length(filepaths)
    [paths, img_name, ext] = fileparts(filepaths(i).name);
    if isempty(img_name)
        disp('Ignore . folder.');
    elseif strcmp(img_name, '.')
        disp('Ignore .. folder.');
    else
        idx = idx + 1;
        str_result = sprintf('%d\t%s.\n', idx, img_name);
        fprintf(str_result);

        % read image
        img = imread(fullfile(input_folder, [img_name, ext]));
        img = im2double(img);

        % modcrop
%         img = modcrop(img, mod_scale);
%         if exist('save_mod_folder', 'var')
%             imwrite(img, fullfile(save_mod_folder, [img_name, '.png']));
%         end

        % LR
        im_lr = imresize(img, 1/up_scale, 'bicubic');
        if exist('save_lr_folder', 'var')
            imwrite(im_lr, fullfile(save_lr_folder, [img_name, '.png']));
        end

        % Bicubic
        if exist('save_bic_folder', 'var')
            im_bicubic = imresize(im_lr, up_scale, 'bicubic');
            imwrite(im_bicubic, fullfile(save_bic_folder, [img_name, '.png']));
        end
    end
end
end
%% modcrop
function img = modcrop(img, modulo)
if size(img,3) == 1
    sz = size(img);
    sz = sz - mod(sz, modulo);
    img = img(1:sz(1), 1:sz(2));
else
    tmpsz = size(img);
    sz = tmpsz(1:2);
    sz = sz - mod(sz, modulo);
    img = img(1:sz(1), 1:sz(2),:);
end
end

generate_LR_Vimeo90K.m

function generate_LR_Vimeo90K()
%% matlab code to genetate bicubic-downsampled for Vimeo90K dataset

up_scale = 2;
mod_scale = 2;
idx = 0;
% filepaths = dir('/home/xtwang/datasets/vimeo90k/vimeo_septuplet/sequences/*/*/*.png');
filepaths = dir('E:/tmp_0617/tmp_0617/images/*.png');
for i = 1 : length(filepaths)
    [~,imname,ext] = fileparts(filepaths(i).name);
    folder_path = filepaths(i).folder;
%     save_LR_folder = strrep(folder_path,'vimeo_septuplet','vimeo_septuplet_matlabLRx4');
    save_LR_folder = strrep(folder_path,'tmp_0617','tmp_0617_save3');
    if ~exist(save_LR_folder, 'dir')
        mkdir(save_LR_folder);
    end
    if isempty(imname)
        disp('Ignore . folder.');
    elseif strcmp(imname, '.')
        disp('Ignore .. folder.');
    else
        idx = idx + 1;
        str_result = sprintf('%d\t%s.\n', idx, imname);
        fprintf(str_result);
        % read image
        img = imread(fullfile(folder_path, [imname, ext]));
        img = im2double(img);
        % modcrop
%         img = modcrop(img, mod_scale);
        % LR
        im_LR = imresize(img, 1/up_scale, 'bicubic');
        if exist('save_LR_folder', 'var')
            imwrite(im_LR, fullfile(save_LR_folder, [imname, '.png']));
        end
    end
end
end

%% modcrop
function img = modcrop(img, modulo)
if size(img,3) == 1
    sz = size(img);
    sz = sz - mod(sz, modulo);
    img = img(1:sz(1), 1:sz(2));
else
    tmpsz = size(img);
    sz = tmpsz(1:2);
    sz = sz - mod(sz, modulo);
    img = img(1:sz(1), 1:sz(2),:);
end
end

https://blog.csdn.net/u011994454/article/details/107179796

http://events.jianshu.io/p/05abb917ae57