臺灣博碩士論文加值系統

近年來液晶顯示器蓬勃發展，現今已可以將高動態範圍影像顯示在液晶顯示器上。本論文提出一種深度殘留神經網路模型（Deep residual neural network），該模型可以將色彩深度8bits低動態範圍影像重建到色彩深度12bits高動態範圍影像，先將高動態範圍影像利用色調映射（Tone mapping）獲得低動態範圍影像，再使低動態範圍影像經由深度殘留神經網路模型重建到高動態範圍影像。本論文所提出的方法與單純的線性放大相比，可以很明顯得看出本論文方法可以將亮區域亮度提升並維持暗區域亮度，相對單純的線性放大方法並沒辦法維持暗區域亮度。本論文使用HDR-VDP-2.2 (Visual difference predictor-2.2)來評估效果，所提出的方法效果比單純的線性放大高達20%。

In recent years, LCDs(Liquid crystal display) have been developed rapidly, and high dynamic range images can now be displayed on high dynamic range LCDs. This thesis proposes a deep residual neural network model, which can reconstruct a low-dynamic range image with a color depth of 8bits to a high-dynamic range image with a color depth of 12bits. First, high dynamic range images were toned mapped to obtain low dynamic range images, and then these low dynamic range images were reconstructed to high dynamic range images by using deep residual neural network models. The proposed method is compared with simple linear scaling. It can be clearly seen that the proposed method can enhance the brightness of the bright area and maintain the brightness of the dark area. Relatively, simple linear scaling does not maintain the dark region brightness very well. In this thesis, HDR-VDP-2.2 (Visual difference predictor-2.2) is used to evaluate the effect. The performance of the proposed method is up to 20% higher than the linear scaling.

摘要 i
ABSTRACT ii
誌謝 iii
目錄 iv
表目錄 vi
圖目錄 vii
第一章 緒論 1
1.1研究背景與動機 1
1.2研究方向 2
1.3論文架構 3
第二章 相關演算法研究 4
2.1文獻探討 4
2.1.1單次拍攝高動態範圍成像基於深度卷積網路(Single-shot high dynamic range imaging via deep convolutional neural network) 4
2.1.2位元深度的擴展與消除假輪廓(Bit-depth Expansion and Decontouring) 5
2.1.3反色調映射(Inverse tone mapping) 8
2.1.4亮度飽和區的影像回復(Recovering image in saturated shadow, highlight, and light source regions) 13
第三章 低動態範圍影像轉換高動態範圍殘留神經網路架構 15
3.1 卷積層區 15
3.1.1卷積神經網路（Convolutional Neural Network, CNN） 15
3.1.2殘留神經網路（Residual Neural Network, ResNet） 17
3.2 轉置卷積層區 18
3.2.1轉置卷積神經網路（Transposed Convolutional Neural Network） 18
3.4 Log轉換 19
3.5 激勵函數(Activate function) 19
3.6訓練方法與數據集 20
第四章 實驗結果 21
4.1使用色彩映射獲得的LDR 21
4.1.1線性縮放與本論文結果圖主觀比較 21
4.1.2線性縮放與本論文結果圖客觀比較 27
4.2使用多重曝光合成HDR影像取得其中間曝光值影像做為LDR 28
4.2.1線性縮放與本論文結果圖主觀比較 28
4.2.2線性縮放與本論文結果圖客觀比較 37
4.3 HDR-VDP-2.2 37
4.4 峰值信噪比(Peak signal to noise ratio, PSNR) 38
4.5 均方根誤差(Root-mean-square error, RMSE) 38
第五章 結論與未來工作 39
參考文獻 40

[1]E.Corporation. The Ins and Outs of HDR – What is HDR? [Online]. Available: http://www.eizoglobal.com/library/management/ins-and-outs-of-hdr/index1.html.
[2]V. G. An and C. Lee, "Single-shot high dynamic range imaging via deep convolutional neural network," in 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), 2017: IEEE, pp. 1768-1772.
[3]S. K. Nayar and T. Mitsunaga, "High dynamic range imaging: Spatially varying pixel exposures," in Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), 2000, vol. 1: IEEE, pp. 472-479.
[4]J. Gu, Y. Hitomi, T. Mitsunaga, and S. Nayar, "Coded rolling shutter photography: Flexible space-time sampling," in 2010 IEEE International Conference on Computational Photography (ICCP), 2010: IEEE, pp. 1-8.
[5]H. Cho, S. J. Kim, and S. Lee, "Single‐shot High Dynamic Range Imaging Using Coded Electronic Shutter," in Computer Graphics Forum, 2014, vol. 33, no. 7: Wiley Online Library, pp. 329-338.
[6]R. M. Karol Myszkowski, and Grzegorz Krawczyk. High dynamic range video [Online]. Available: https://www.cl.cam.ac.uk/~rkm38/pdfs/myszkowski08hdrvideo.pdf.
[7]C. R. Carlson, E. H. Adelson, and C. H. Anderson, "System for coring an image-representing signal," ed: Google Patents, 1985.
[8]S. J. Daly and X. Feng, "Decontouring: Prevention and removal of false contour artifacts," in Human Vision and Electronic Imaging IX, 2004, vol. 5292: International Society for Optics and Photonics, pp. 130-150.
[9]S. Bhagavathy, J. Llach, and J. fu Zhai, "Multi-scale probabilistic dithering for suppressing banding artifacts in digital images," in 2007 IEEE International Conference on Image Processing, 2007, vol. 4: IEEE, pp. IV-397-IV-400.
[10]A. O. Akyüz, R. Fleming, B. E. Riecke, E. Reinhard, and H. H. Bülthoff, "Do HDR displays support LDR content?: a psychophysical evaluation," in ACM Transactions on Graphics (TOG), 2007, vol. 26, no. 3: ACM, p. 38.
[11]E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, "Photographic tone reproduction for digital images," in ACM transactions on graphics (TOG), 2002, vol. 21, no. 3: ACM, pp. 267-276.
[12]L. Meylan, S. Daly, and S. Süsstrunk, "The reproduction of specular highlights on high dynamic range displays," in Color and Imaging Conference, 2006, vol. 2006, no. 1: Society for Imaging Science and Technology, pp. 333-338.
[13]L. Meylan, "Tone mapping for high dynamic range images," EPFL, 2006.
[14]L. Meylan, S. Daly, and S. Süsstrunk, "Tone mapping for high dynamic range displays," in Human Vision and Electronic Imaging XII, 2007, vol. 6492: International Society for Optics and Photonics, p. 649210.
[15]F. Banterle, P. Ledda, K. Debattista, and A. Chalmers, "Inverse tone mapping," in Proceedings of the 4th international conference on Computer graphics and interactive techniques in Australasia and Southeast Asia, 2006: ACM, pp. 349-356.
[16]L. Wang, L.-Y. Wei, K. Zhou, B. Guo, and H.-Y. Shum, "High dynamic range image hallucination," in Proceedings of the 18th Eurographics conference on Rendering Techniques, 2007: Eurographics Association, pp. 321-326.
[17]F. Durand and J. Dorsey, "Fast bilateral filtering for the display of high-dynamic-range images," in ACM transactions on graphics (TOG), 2002, vol. 21, no. 3: ACM, pp. 257-266.
[18]I. Drori, D. Cohen-Or, and H. Yeshurun, "Fragment-based image completion," in ACM Transactions on graphics (TOG), 2003, vol. 22, no. 3: ACM, pp. 303-312.
[19]O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015: Springer, pp. 234-241.
[20]S. Albawi, T. A. Mohammed, and S. Al-Zawi, "Understanding of a convolutional neural network," in 2017 International Conference on Engineering and Technology (ICET), 2017: IEEE, pp. 1-6.
[21]C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE transactions on pattern analysis and machine intelligence, vol. 38, no. 2, pp. 295-307, 2015.
[22]D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, "Context encoders: Feature learning by inpainting," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2536-2544.
[23]K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
[24]H. Noh, S. Hong, and B. Han, "Learning deconvolution network for semantic segmentation," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1520-1528.
[25]X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks," in Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011, pp. 315-323.
[26]G. Zaal. HDRI Haven [Online]. Available: https://hdrihaven.com/.
[27]C. Bloch. sIBL Archive - HDRLabs [Online]. Available: http://www.hdrlabs.com/sibl/archive.html.
[28]M. Narwaria, R. Mantiuk, M. P. Da Silva, and P. Le Callet, "HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images," Journal of Electronic Imaging, vol. 24, no. 1, p. 010501, 2015.
[29] 蕭宇宏, "高效能LDR轉換成HDR之硬體模型設計," 科技部專題研究計畫申請書, 2017