论文阅读——《Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning》

对于同一张图片的不同区域，需要的denoise的网络是一样的吗？有些区域可能很简单的网络就可以实现很好的效果，但有些区域需要比较复杂的网络才可以得到不错的效果。

对于不同的图片，也是如此，有些图片需要复杂的网络，有些图片不需要复杂的网络。

如何的自适应地去应对不同的condition？

existing dynamic networks treat all image regions as an integration

different paths for different regions.

如何选patch可以看作为如何选图片类似吗？

 

In contrast to existing methods that concentrate on a single type of distortion, we intend to handle a mix of multiple distortions

To address mixed distortions, we propose to restore the corrupted image step by step with a sequence of restoration tools.因此。如何去选择工具箱，将会十分具备challenge

the choice of the restoration type, level and the processing order all influence the final performance.

The results indicate that minor changes in a toolchain can severely impact the restoration performance. 

除此以外，using improper tools may lead to unnatural outputs（使用不合适的工具会导致结果变化）

故此，合适的工具顺序，合适的工具，对结果影响比较大。那么如何来确定工具的顺序，以及所选的工具是否合适呢（所估计出来的level是否合适）

When the tools are trained on specific tasks, we encounter another problem that none of the tools can perfectly handle the ‘middle state’, which refers to the intermediate result after several steps of processing.并且没有一个工具可以完美的handle中间状态

 

As most distortions are irreversible （不可逆）, the restoration of their mixture is not a simple composition of the corresponding restorers.

New artifacts could be introduced in the middle states. For example, the deblurring operation will also enhance the noises, causing the following denoisers fail in removing the newly introduced artifacts.

• treat the sequential tool selection problem as a Markov Decision Process (MDP) and solve it in a deep reinforcement learning manner.
•  propose a training scheme to refine the agent and tools jointly so that the tools are more well-informed with the middle states observable by the agent.

所以关键就是强化学习选择工具的过程，以及这个训练的方案可以fine tune

 

关于TOOLBOX 

The toolbox contains a set of tools that might be applied to the corrupted image.

each tool is trained only on a narrow range of distortions

用小网络来准确handel一个小范围的退化

After training the agent, all tools are jointly fine-tuned on the basis of the well-trained toolchains.

 

强化学习部分

Given an input image, the agent first selects a tool from the toolbox and uses it to restore the image, then the agent chooses another tool according to the previous result and repeats the restoration process until it decides to stop.

 

关于动态网络