臺灣博碩士論文加值系統

現有基於多視角的反射分離(reflection separation)方案，因其並非針對窄基線(narrow baseline)而設計，並不適用於光場影像，又目前僅有的少數光場影像的反射分離方案，皆須先對中心視角估測視差圖(disparity map)。因此，本論文採用基於不含反射之光場影像視差估測所設計之EPINET，對弱反射之混合光場影像進行反射分離，透過訓練以多視角影像堆疊(image stack)為輸入的全卷積網路，以端對端(end-to-end)的方式學習主要方向的多視角背景層之主要卷積特徵，合併後再以全卷積網路，對中央視角直接進行以像素為單位的背景層影像估測。此外，本論文分析光場相機拍攝真實世界中的混合光場影像，其背景層與反射層於各視角間會有不同的位移量，基於此現象建構出滿足真實條件之混合光場影像資料集。實驗結果顯示，採用EPINET輔以本論文提出之混合光場影像資料集，能有效地對合成之混合光場影像重建背景層。

Existing reflection separation schemes designed for multi-view images cannot be applied to light filed images due to the dense light fields with narrow baselines. In order to improve accuracy of the reconstructed background (i.e., the transmitted layer), most light field data based reflection separation schemes estimate a disparity map before reflection separation. Different from previous work, this thesis uses the existing EPINET based on disparity estimation of light field image without reflection, and separates the mixed light field image of weak reflection. At the training stage, the network takes multi-view images stacks along principle directions of light field data as inputs, and significant convolution features of the background layer are learned in an end-to-end manner. Then the FCN learns to predict pixel-wise gray-scale values of the background layer of the central view. Our experimental results show that the background layer can be reconstructed effectively by using EPINET and the mixed light field image dataset proposed in this thesis.

摘要 I
Abstract II
目錄 III
圖目錄 V
表目錄 VI
第一章 緒論 1
1.1 前言 1
1.2 研究動機 1
1.3 研究方法 2
1.4 論文架構 2
第二章 非光場影像之反射分離方案技術介紹 3
2.1基於非深度學習之反射分離方案 3
2.1.1 基於單影像之反射分離方案 3
2.2基於深度學習之反射分離方案 4
2.2.1 現有反射分離方案採用的網路架構 4
2.2.2 基於深度學習之反射分離技術現況 7
2.3 總結 9
第三章 光場影像之反射分離方案技術現況 10
3.1基於非深度學習之光場反射分離方案 10
3.2採用深度學習估測視差值之光場反射分離方案 10
3.3 總結 11
第四章 本論文所提之光場影像反射分離方案 12
4.1 現有EPINET之網路架構 12
4.2合成混合光場資料集 18
4.3 總結 20


第五章 實驗結果與討論 21
5.1 實驗參數與測試資料集規格 21
5.1.1 測試資料集 21
5.1.2參數設定 27
5.2 反射分離實驗結果 30
5.3 總結 36
第六章 結論與未來展望 37
參考文獻 38

[1] A. Levin and Y. Weiss, “User assisted separation of reflections from a single image using a sparsity prior,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 29, pp.1647-1654, Sept. 2007.
[2] S.-K. Yeung, T.-P. Wu, and C.-K. Tang, “Extracting smooth and transparent layers from a single image,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1-7, June 2008.
[3] X. Guo, X. Cao, and Y. Ma, “Robust separation of reflection from multiple images,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2187-2194, June 2014.
[4] Y. Li and M. S. Brown, “Exploiting reflection change for automatic reflection removal, ” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2432-2439, March 2014.
[5] Y. Ni, J. Chen, and L.-P. Chau, “Reflection removal based on single light field capture,” in Proc. IEEE International Symposium on Circuits and Systems (ISSCS), pp. 1-4, May 2017.
[6] T. Li and D.P.-K. Lun, “Robust Reflection Removal Based on Light Field Imaging,” IEEE Transactions on Image Processing, pp.1798-1812, vol. 28, April 2019.
[7] P. Chandramouli, M. Noroozi, and P. Favaro, “ConvNet-Based Depth Estimation, Reflection Separation and Deblurring of Plenoptic Images,” in Proc. Asian Conference on Computer Vision (ACCV), pp. 129-144, 2016.
[8] C. Shin, H.-G. Jeon, Y. Yoon, I. S. Kweon, S. J. Kim, “EPINET: A fully-convolutional neural network using epipolar geometry for depth from light field images,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4748-4757, June 2018.
[9] H. Farid and E. Adelson, “Separating reflections and lighting using independent components analysis,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 262-267, June 1999.
[10] A. Agrawal, R. Raskar, S. Nayar, and Y. Li, “Removing photography artifacts using gradient projection and flashexposure sampling,” in ACM Transactions on Graphics, vol. 24, pp. 828-835, July 2005.
[11] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition, ” in Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[12] J. Long, E. Shelhamer, T. Darrell, “Fully convolutional networks for semantic segmentation, ” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, June 2015.
[13] G. E. Hinton and R. Salakhutdinov, “Reducing the dimensionality of data with neural networks,” Science, vol. 313, no. 5786, pp. 504-507, July 2006.
[14] Q. Fan, J. Yang, G. Hua, B. Chen, and D. Wipf. “A generic deep architecture for single image reflection removal and image smoothing,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 22-29, Oct. 2017.
[15] R. Wan, B. Shi, L.Y. Duan, A.H. Tan, and A.C. Kot, “CRNN: Multi-scale guided concurrent reflection removal network,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4777-4785, June 2018.
[16] P. Wieschollek, O. Gallo, J. Gu, and J. Kautz, “Separating reflection and transmission images in the wild,” in Proc. European Conference on Computer Vision (ECCV), pp. 89-104, Sept. 2018.
[17] Y. Ni, J. Chen, and L. Chau, “Reflection removal on single light field capture using focus manipulation,” IEEE Transactions on Computational Imaging, vol. 4, pp. 562-572, July 2018.
[18] S. Wanner and B. Goldluecke, “Reconstructing reflective and transparent surfaces from epipolar plane images,” in Proc. German Conference on Pattern Recognition (GCPR), vol. 8142, pp. 1-10, Sept. 2013.
[19] Q. Wang, H. Lin, Y. Ma, S. Kang, and J. Yu, “Automatic layer separation using light field imaging,” arXiv preprint arXiv:1506.04721 [cs.CV], June 2015.
[20] O. Johannsen, A. Sulc, and B. Goldluecked, “what sparse light field coding reveals about scene structure,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3262-3270, June 2016.
[21] H.-G. Jeon, J. Park, G. Choe, J. Park, Y. Bok, Y.-W. Tai, and I. S. Kweon, “Depth from a light field image with learning based matching costs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, pp. 297-310, Jan. 2018.
[22] K. Honauer, O. Johannsen, D. Kondermann, and B. Goldluecke, “A dataset and evaluation methodology for depth estimation on 4D light fields,” Asian Conference on Computer Vision. Lecture Notes in Computer Science, vol. 10113, pp. 19-34, March 2016.
[23] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, April 2004.