NVlabs/noise2noise代码（一）概览与运行

背景：github上的noise2noise的官方代码是NVlabs的代码，我们希望在此基础上进行修改，所以需要初步看懂与运行。

目的：运行与跑通noise2noise的代码，训练与测试。

论文地址：https://arxiv.org/abs/1803.04189

源码地址：https://github.com/NVlabs/noise2noise

论文详解：https://blog.csdn.net/weixin_36474809/article/details/86535639

目录

一、环境要求与配置

1.1 用Anaconda配置python3.6 tf pip

1.2 安装依赖项

二、准备训练集与验证集

2.1 将ImageNet生成为训练集

2.2 用BSD300生成训练文件

2.3 Kodak验证集

三、训练与验证

3.1 训练

3.2 验证

3.3 预训练的网络

四、MRI图像去噪

Preparing the MRI training dataset

Training

---

---

一、环境要求与配置

作者github上源码需要安装python 3.6版本，运用Anaconda 5.2来配置相应的Python环境。创建相应的环境与配置。指令汇总如下，注意我们用的anaconda3的激活环境为 source acitvate n2n，去激活用source deactivate

conda create -n n2n python=3.6
conda activate n2n
conda install tensorflow-gpu
python -m pip install --upgrade pip
pip install -r requirements.txt

1.1 用Anaconda配置python3.6 tf pip

我们安装2018.12的anaconda版本，然后实现相应的配置。

Ubuntu14.04安装Anaconda3-2018.12-x86_64 https://blog.csdn.net/weixin_36474809/article/details/87804903

运用Anaconda对python 3.6与tensorflow-gpu与pip环境配置 https://blog.csdn.net/weixin_36474809/article/details/87714182

按照上两篇博文配置好环境

1.2 安装依赖项

pip install -r reauirements.txt

(n2n) jcx@smart-dsp:~/Desktop/xxr2019/NVlabs_noise2noise$ pip install -r requirements.txt

安装完成后，会出现下面信息

1.1.0->-r requirements.txt (line 4)) (1.15.4)
Building wheels for collected packages: nibabel
  Building wheel for nibabel (setup.py) ... done
  Stored in directory: /home/jcx/.cache/pip/wheels/7e/97/9b/7cf05225afac39de2dacef758fc06c0c0bdf3e6efb40fabb9c
Successfully built nibabel
Installing collected packages: Pillow, typeguard, nibabel, scipy
  Found existing installation: scipy 1.2.0
    Uninstalling scipy-1.2.0:
      Successfully uninstalled scipy-1.2.0
Successfully installed Pillow-5.2.0 nibabel-2.3.0 scipy-1.1.0 typeguard-2.2.2

 

二、准备训练集与验证集

程序中，dataset_tool_tf.py用于将相应的文件生成可用于训练n2n网络的tfrecords文件。

2.1 将ImageNet生成为训练集

# This should run through roughly 50K images and output a file called `datasets/imagenet_val_raw.tfrecords`.
python dataset_tool_tf.py 
--input-dir "/ILSVRC2012_img_val" 
--out=datasets/imagenet_val_raw.tfrecords

命令行中，--input-dir为输入的文件，--out为输出的文件

如果正确运行，类似于下面这样的输出：

<...long omitted...>
49997 ./ImageNet/ILSVRC2012_img_val/ILSVRC2012_val_00002873.JPEG
49998 ./ImageNet/ILSVRC2012_img_val/ILSVRC2012_val_00031550.JPEG
49999 ./ImageNet/ILSVRC2012_img_val/ILSVRC2012_val_00009765.JPEG
Dataset statistics:
  Formats:
    RGB: 49100 images
    L: 899 images
    CMYK: 1 images
  width,height buckets:
    >= 256x256: 48627 images
    < 256x256: 1373 images

2.2 用BSD300生成训练文件

BSD300训练出的网络性能比ImageNet稍差，下载BSD300 dataset

然后解压到相应文件夹。然后进行转换：

python dataset_tool_tf.py 
--input-dir datasets/BSDS300-images/BSDS300/images/train 
--out=datasets/bsd300.tfrecords

2.3 Kodak验证集

Training tests validation loss against the Kodak Lossless True Color Image Suite dataset

# Download the kodak validation set from http://r0k.us/graphics/kodak/
python download_kodak.py --output-dir=datasets/kodak

三、训练与验证

3.1 训练

在ImageNet上运用N2N的autoencoder

# try python config.py train --help for available options
python config.py 
--desc='-test' train 
--train-tfrecords=datasets/imagenet_val_raw.tfrecords 
--long-train=true 
--noise=gaussian

然后可以用tensorboard查看相应的训练过程

cd results
tensorboard --logdir .

训练过程相当耗时，例如高斯噪声的去噪网络，作者用ImageNet训练，在NVIDIA Titan V GPU上花费了将近7.5小时

训练结束标志：results/* 文件夹之下生成一个network_final.pickle的文件。

3.2 验证

假定运行后结果存在 results/00001-autoencoder-1gpu-L-n2n 之中

命令行这样输入：

python config.py validate 
--dataset-dir=datasets/kodak 
--network-snapshot=results/00001-autoencoder-1gpu-L-n2n/network_final.pickle

3.3 预训练的网络

泊松噪声和高斯噪声的预训练的网络如下：https://drive.google.com/drive/folders/1-84ORv4wB8W3M6WngFTtccuW7SlPku0V

论文实验的结果：

训练

Noise	Noise2Noise	Command line
Gaussian	Yes	python config.py train --noise=gaussian --noise2noise=true --long-train=true --train-tfrecords=datasets/imagenet_val_raw.tfrecords
Gaussian	No	python config.py train --noise=gaussian --noise2noise=false --long-train=true --train-tfrecords=datasets/imagenet_val_raw.tfrecords
Poisson	Yes	python config.py train --noise=poisson --noise2noise=true --long-train=true --train-tfrecords=datasets/imagenet_val_raw.tfrecords
Poisson	No	python config.py train --noise=poisson --noise2noise=false --long-train=true --train-tfrecords=datasets/imagenet_val_raw.tfrecords

验证：

Noise	Dataset	Command line	Expected PSNR (dB)
Gaussian	kodak	python config.py validate --dataset-dir=datasets/kodak --noise=gaussian --network-snapshot=<.../network_final.pickle>	32.38 (n2c) / 32.39 (n2n)
Gaussian	bsd300	python config.py validate --dataset-dir=datasets/bsd300 --noise=gaussian --network-snapshot=<.../network_final.pickle>	31.01 (n2c) / 31.02 (n2n)
Poisson	kodak	python config.py validate --dataset-dir=datasets/kodak --noise=poisson --network-snapshot=<.../network_final.pickle>	31.66 (n2c) / 31.66 (n2n)
Poisson	bsd300	python config.py validate --dataset-dir=datasets/bsd300 --noise=poisson --network-snapshot=<.../network_final.pickle>	30.27 (n2c) / 30.26 (n2n)

四、MRI图像去噪

Preparing the MRI training dataset

Use the dataset_tool_mri.py script to generate training and validation datasets for the N2N MRI case.

Step #1: Download the IXI-T1 dataset from: https://brain-development.org/ixi-dataset/. Unpack to some location.

Step #2: Convert the IXI-T1 dataset into a set of PNG files:

# Assumes you have downloaded and untarred IXI-T1 under ~/Downloads/IXI-T1.

python dataset_tool_mri.py genpng --ixi-dir=~/Downloads/IXI-T1 --outdir=datasets/ixi-png

Step #3: Convert a subset of the IXI-T1 dataset into training and validation sets:

python dataset_tool_mri.py genpkl --png-dir=datasets/ixi-png --pkl-dir=datasets

Training

python config_mri.py

A successful invocation should output the following:

dnnlib: Running train_mri.train() on localhost...
Loading training set.
Loading dataset from datasets\ixi_train.pkl
<...long log omitted...>
Epoch 297/300: time=107.981, train_loss=0.0126064, test_db_clamped=31.72174, lr=0.000002
Epoch 298/300: time=107.477, train_loss=0.0125972, test_db_clamped=31.73622, lr=0.000001
Epoch 299/300: time=106.927, train_loss=0.0126012, test_db_clamped=31.74232, lr=0.000001
Saving final network weights.
Resetting random seed and saving a bunch of example images.
dnnlib: Finished train_mri.train() in 8h 59m 19s.

The expected average PSNR on the validation set (named test_db_clamped in code) is roughly 31.74 dB.

Noise-to-noise training is enabled by default for the MRI case. To use noise-to-clean training, edit config_mri.py and change corrupt_targets=True to corrupt_targets=False.

Training for 300 epochs takes roughly 9 hours on an NVIDIA Titan V GPU.