曝光修正相关工作：Related Work on the Exposure Correction

本博客对曝光校准的相关工作进行简单总结，内容选自 2021 CVPR 文章：Learning Multi-Scale Photo Exposure Correction. 

博客：https://blog.csdn.net/u014546828/article/details/122552236 

Related Work on the Exposure Correction

原文：Learning Multi-Scale Photo Exposure Correction

The focus of our paper is on correcting exposure errors in camera-rendered 8-bit sRGB images. We refer the reader to [9, 24, 25, 38] for representative examples for rendering linear raw-RGB images captured with low-light or exposure errors.

本文的重点是校正相机渲染的 8 位 sRGB 图像的曝光误差。请读者参考 [9,24,25,38]，以获得具有低光或曝光错误的线性 raw-RGB 图像。

[9] Learning to see in the dark. In CVPR, 2018.

 [24] Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics (TOG), 35(6):1–12, 2016.

[25] Exposure: A white-box photo postprocessing framework. ACM Transactions on Graphics (TOG), 37(2):26:1–26:17, 2018.

[38] Handheld mobile photography in very low light. ACM Transactions on Graphics (TOG), 38(6):1–16, 2019.

Exposure Correction

Traditional methods for exposure correction and contrast enhancement rely on image histograms to adjust image intensity values [8, 19, 36, 50, 69]. Alternatively, tone curve adjustment is used to correct images with exposure errors. This process is performed by relying either solely on input image information [63] or trained deep learning models [21, 46, 48, 62]. The majority of prior work adopts the Retinex theory [34] by assuming that improperly exposed images can be formulated as a pixel-wise multiplication of target images, captured with correct exposure settings, by illumination maps. Thus, the goal of these methods is to predict illumination maps to recover the well-exposed target images. Representative Retinex-based methods include [23, 29, 34, 44, 57, 64, 65] and the most recent deep learning ones [56, 58, 66]. Most of these methods, however, are restricted to correcting underexposure errors [23,56,58–60,65,66,68]. In contrast to the majority of prior work, our work is the first deep learning method to explicitly correct both overexposed and underexposed photographs with a single model.

传统的曝光校正和对比度增强方法依赖于图像直方图来调整图像强度值。或者，色调曲线调整被用来校正曝光误差的图像。这一过程要么完全依赖输入的图像信息，要么依赖训练好的深度学习模型。之前的大部分工作都采用了 Retinex 理论，假设不恰当曝光的图像可以被表述为通过正确曝光设置捕获的目标图像的像素级乘法。因此，这些方法的目标是预测光照图，以恢复良好曝光的目标图像。代表性的基于 Retinex 的方法包括和最新的深度学习方法。然而，这些方法大多局限于校正曝光不足误差。与之前的大部分工作相比，我们的工作是第一种深度学习方法，通过单个模型明确地校正过度曝光和曝光不足的照片。

[21] Zero-reference deep curve estimation for low-light image enhancement. In CVPR, 2020.

[46] DeepLPF: Deep local parametric filters for image enhancement. In CVPR, 2020. 

[48] Distort-and-recover: Color enhancement using deep reinforcement learning. In CVPR, 2018.

[62] DeepExposure: Learning to expose photos with asynchronously reinforced adversarial learning. In NeurIPS, 2018.

[64] Dual illumination estimation for robust exposure correction. In Computer Graphics Forum, 2019. 

[65] High-quality exposure correction of underexposed photos. In ACM MM, 2018.

[56] Underexposed photo enhancement using deep illumination estimation. In CVPR, 2019.

[66] Kindling the darkness: A practical low-light image enhancer. In ACM MM, 2019.

HDR Restoration and Image Enhancement

HDR restoration is the process of reconstructing scene radiance HDR values from one or more low dynamic range (LDR) input images. Prior work either require access to multiple LDR images [16, 30, 43] or use a single LDR input image, which is converted to an HDR image by hallucinating missing information [15, 47]. Ultimately, these reconstructed HDR images are mapped back to LDR for perceptual visualization. This mapping can be directly performed from the input multi-LDR images [7,13], the reconstructed HDR image [61], or directly from the single input LDR image without the need for radiance HDR reconstruction [11, 18]. There are also methods that focus on general image enhancement that can be applied to enhancing images with poor exposure. In particular, work by [26, 27] was developed primarily to enhance images captured on smartphone cameras by mapping captured images to appear as highquality images captured by a DSLR. Our work does not seek to reconstruct HDR images or general enhancement, but instead is trained to explicitly address exposure errors.

HDR 图像恢复是从一个或多个低动态范围 (LDR) 输入图像中重建场景亮度 HDR 值的过程。之前的工作要么需要访问多个 LDR 图像，要么使用单个 LDR 输入图像，通过幻想丢失信息而转换为HDR 图像 [15,47]。最终，这些重建的 HDR 图像被映射回 LDR 进行感知可视化。这种映射可以直接从输入的多 LDR 图像，重建的 HDR 图像进行，或直接从单输入的 LDR 图像进行，而不需要亮度 HDR 重建。还有一些专注于一般图像增强的方法，可以应用于增强曝光不良的图像。特别是，[26,27] 的工作主要是为了增强智能手机相机捕捉到的图像，将捕捉到的图像映射为数码单反相机捕捉到的高质量图像。本文的工作并不寻求重建 HDR 图像或一般增强，而是训练解决曝光错误。

[15] HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics (TOG), 36(6):178:1–178:15, 2017.

[7] Learning a deep single image contrast enhancer from multi-exposure images. IEEE Transactions on Image Processing, 27(4):2049–2062, 2018.

[61] Image correction via deep reciprocating HDR transformation. In CVPR, 2018.

[11] Deep photo enhancer: Unpaired learning for image enhancement from photographs with GANs. In CVPR, 2018.

[18] Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics (TOG), 36(4):118:1–118:12, 2017.

[26] DSLR-quality photos on mobile devices with deep convolutional networks. In ICCV, 2017.

Paired Dataset

Paired datasets are crucial for supervised learning for image enhancement tasks. Existing paired datasets for exposure correction focus only on low-light underexposed images. Representative examples include Wang et al.’s dataset [56] and the low-light (LOL) paired dataset [58]. Unlike existing datasets for exposure correction, we introduce a large image dataset rendered with a wide range of exposure errors. Fig. 2 shows a comparison between our dataset and the LOL dataset in terms of the number of images and the variety of exposure errors in each dataset. The LOL dataset covers a relatively small fraction of the possible exposure levels, as compared to our introduced dataset. Our dataset is based on the MIT-Adobe FiveK dataset [6] and is accurately rendered by adjusting the high tonal values provided in camera sensor raw-RGB images to realistically emulate camera exposure errors. An alternative worth noting is to use a large HDR dataset to produce training data—for example, the Google HDR+ dataset [24]. One drawback, however, is that this dataset is a composite of a varying number of smartphone captured raw-RGB images that were first aligned to a composite raw-RGB image. The target ground truth image is based on an HDR-to-LDR algorithm applied to this composite raw-RGB image [18,24]. We opt instead to use the FiveK dataset as it starts with a single high-quality raw-RGB image and the ground truth result is generated by an expert photographer.

成对数据集对于图像增强任务的监督学习至关重要。现有的曝光校正成对数据集只对弱光欠曝光图像进行聚焦。典型的例子包括 Wang et al. 的数据集和弱光 (LOL) 配对数据集。与现有的曝光校正数据集不同，本文引入了一个具有广泛曝光误差的大型图像数据集。图 2 显示了该数据集和 LOL 数据集在每个数据集的图像数量和曝光误差的变化情况。

与本文引入的数据集相比，LOL 数据集覆盖了可能暴露水平的一个相对较小的部分。本文的数据集基于 MIT-Adobe FiveK 数据集 [6]，通过调整相机传感器 raw-RGB 图像中提供的高色调值来精确渲染，以真实地模拟相机曝光误差。另一种值得注意的方法是使用大型 HDR 数据集生成训练数据——例如，谷歌 HDR+数据集 [24]。然而，该数据集的一个缺点是，该数据集是由不同数量的智能手机捕获的原始 rgb 图像合成的，这些图像首先对齐到一个复合的原始 rgb 图像。目标 ground truth 图像是基于 HDR-to-LDR 算法应用于复合 raw-RGB 图像。本文选择使用 FiveK 数据集，因为它从一张高质量的 raw-RGB 图像开始，ground truth 结果是由专业摄影师生成的。

[56] Underexposed photo enhancement using deep illumination estimation. In CVPR, 2019.

[58] Deep Retinex decomposition for low-light enhancement. In BMVC, 2018.

[6] Learning photographic global tonal adjustment with a database of input / output image pairs. In CVPR, 2011.

[24] Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics (TOG), 35(6):1–12, 2016.

其它相关论文：

[1] Adobe. Color and camera raw. https://helpx. adobe.com/ca/photoshop- elements/using/ color-camera-raw.html. Accessed: 2020-11-12.

[2] When color constancy goes wrong: Correcting improperly white-balanced images. In CVPR, 2019.

[10] Bilateral guided upsampling. ACM Transactions on Graphics (TOG), 35(6):1–8, 2016.

 [15] Gabriel Eilertsen, Joel Kronander, Gyorgy Denes, Rafa Mantiuk, and Jonas Unger. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics (TOG), 36(6):178:1–178:15, 2017.

[16] Yuki Endo, Yoshihiro Kanamori, and Jun Mitani. Deep reverse tone mapping. ACM Transactions on Graphics (TOG), 36(6):177:1–177:10, 2017.

[17] Xueyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. A weighted variational model for simultaneous reflectance and illumination estimation. In CVPR, 2016.

[22] Xiaojie Guo. LIME: A method for low-light image enhancement. In ACM MM, 2016. 6, 7, 8 [23] Xiaojie Guo, Yu Li, and Haibin Ling. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing, 26(2):982–993, 2017.

[27] Andrey Ignatov, Nikolay Kobyshev, Radu Timofte, Kenneth Vanhoey, and Luc Van Gool. WESPE: Weakly supervised photo enhancer for digital cameras. In CVPR Workshops, 2018. 2

[28] Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang, Xiaohui Shen, Jianchao Yang, Pan Zhou, and Zhangyang Wang. EnlightenGAN: Deep light enhancement without paired supervision. arXiv preprint arXiv:1906.06972, 2019. 8

[29] Daniel J Jobson, Ziaur Rahman, and Glenn A Woodell. A multiscale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing, 6(7):965–976, 1997. 2

[30] Nima Khademi Kalantari and Ravi Ramamoorthi. Deep high dynamic range imaging of dynamic scenes. ACM Transactions on Graphics (TOG), 36(4):144–1, 2017.

[34] Edwin H Land. The Retinex theory of color vision. Scientific American, 237(6):108–129, 1977. 2

[36] Chulwoo Lee, Chul Lee, and Chang-Su Kim. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Transactions on Image Processing, 22(12):5372–5384, 2013.

[40] Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuytelaars, and Luc Van Gool. Pose guided person image generation. In NeurIPS, 2017.

[41] Ruijun Ma, Haifeng Hu, Songlong Xing, and Zhengming Li. Efficient and fast real-world noisy image denoising by combining pyramid neural network and two-pathway unscented Kalman filter. IEEE Transactions on Image Processing, 29(1):3927–3940, 2020.

[42] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579–2605, 2008.

[43] Tom Mertens, Jan Kautz, and Frank Van Reeth. Exposure fusion: A simple and practical alternative to high dynamic range photography. In Computer Graphics Forum, 2009. 2, 5

[44] Laurence Meylan and Sabine Susstrunk. High dynamic range image rendering with a Retinex-based adaptive filter. IEEE Transactions on Image Processing, 15(9):2820–2830, 2006.

[46] Sean Moran, Pierre Marza, Steven McDonagh, Sarah Parisot, and Gregory Slabaugh. DeepLPF: Deep local parametric filters for image enhancement. In CVPR, 2020.

[47] Kenta Moriwaki, Ryota Yoshihashi, Rei Kawakami, Shaodi You, and Takeshi Naemura. Hybrid loss for learning single-image-based HDR reconstruction. arXiv preprint arXiv:1812.07134, 2018.

[48] Jongchan Park, Joon-Young Lee, Donggeun Yoo, and In So Kweon. Distort-and-recover: Color enhancement using deep reinforcement learning. In CVPR, 2018.

[49] Bryan Peterson. Understanding exposure: How to shoot great photographs with any camera. AmPhoto Books, 2016. 1

[51] Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Fredo Durand, and Saman Amarasinghe. ´ Halide: A language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. In ACM PLDI, 2013.

[53] Jeff Schewe and Bruce Fraser. Real World Camera Raw with Adobe Photoshop CS5. Pearson Education, 2010.

[54] Tamar Rott Shaham, Tali Dekel, and Tomer Michaeli. SinGAN: Learning a generative model from a single natural image. In ICCV, 2019. 

[57] Shuhang Wang, Jin Zheng, Hai-Miao Hu, and Bo Li. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Transactions on Image Processing, 22(9):3538–3548, 2013.

[59] Ke Xu, Xin Yang, Baocai Yin, and Rynson WH Lau. Learning to restore low-light images via decomposition-andenhancement. In CVPR, 2020. 2

[60] Wenhan Yang, Shiqi Wang, Yuming Fang, Yue Wang, and Jiaying Liu. From fidelity to perceptual quality: A semisupervised approach for low-light image enhancement. In CVPR, 2020.

[62] Runsheng Yu, Wenyu Liu, Yasen Zhang, Zhi Qu, Deli Zhao, and Bo Zhang. DeepExposure: Learning to expose photos with asynchronously reinforced adversarial learning. In NeurIPS, 2018.

[63] Lu Yuan and Jian Sun. Automatic exposure correction of consumer photographs. In ECCV, 2012.

[67] Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.

[68] Minfeng Zhu, Pingbo Pan, Wei Chen, and Yi Yang. EEMEFN: Low-light image enhancement via edgeenhanced multi-exposure fusion network. In AAAI, 2020.