臺灣博碩士論文加值系統

由於使用時差測距(Time-of-Flight)攝影機取得的深度圖(depth map)有許多雜訊點，若是將此深度圖用於3D相關應用如3D重建，會產生低品質的3D模型，因此去除深度圖上的雜訊就相當重要。本論文提出一種新的雜訊去除組合方法，結合時域型(temporal)去雜訊法和雙邊濾波器(bilateral filter)來增進深度圖品質。時域型雜訊去除，會先在所有從不同角度拍攝的深度圖中選定中央圖像和其左右鄰近的多張圖像，藉由同時參考多張圖像相對應的點並更新深度值來達到去雜訊的效果。雜訊去除的程序主要分為座標系統的對齊和去雜訊兩步驟，首先會將中央圖像鄰近的多張圖像位移到相同的座標系統且擁有共用旋轉軸，接著將多張鄰近圖像旋轉至和中央圖像同方向以達近似重合。找到鄰近圖像在中央圖像上每個點的對應點後，同時參考多張圖像相對應的深度值來取平均或中值作為新的深度值，如此產生了初步的時域型雜訊去除深度圖。最後再利用雙邊濾波器來進一步去雜訊並保留邊緣。實驗結果顯示本論文提出的去雜訊法和其他方法比起來，錯誤率最低、增進率最高並保留較清楚的邊緣。

As a result of noises on the depth maps captured by a Time-of-Flight (ToF) camera, 3D applications such as 3D reconstruction will generate low quality models. Hence, the process of denoisng depth maps is important. This study proposes a novel combination of denoising methods with the idea of the temporal denoising plus a bilateral filter. The temporal denoising represents the way that this study firstly selects a middle frame with multiple adjacent frames from all depth maps captured from different angles and then renew each depth value in the middle frame by referencing these frames. There are two main steps in our denoising process which contains an alignment step and a denoising step. The first step is to move all points to the same coordinate system with shared rotation axis and rotate neighbor frames to the same direction as the middle frame. After finding the corresponding points in each frame, we then evaluates the mean or median depth values of the corresponding points among these frames to generate a preliminary denoised depth map. Finally, the bilateral filter is subsequently applied to smooth the remaining noises while preserving edges. The experiment results support that the proposed method has the highest improvement in denoising while preserving clear edges when comparing to other methods.

論文摘要 I
Abstract II
Contents III
List of Figures IV
List of Tables VII
Chapter 1. Introduction 1
Chapter 2. ToF Data Denoising with Bilateral Filter 5
2.1 Bilateral Filter 5
2.2 Padding for Spatial Denoising 6
2.3 ToF Depth Map Denoising Processes 8
2.4 ToF Depth Maps Alignment 9
2.5 Temporal Denoising with Bilateral Filter 14
Chapter 3. Experiments 15
3.1 ToF Dataset 15
3.2 Discussion on Window Size of Spatial Denoising 17
3.3 Evaluation of ToF Denoised Depth Maps 19
3.4 ToF Denoised Depth Maps and Performance 21
3.4.1 Denoised Results of Angels Dataset 21
3.4.2 Denoised Results of Head Dataset 25
3.4.3 Denoised Results of Buddha Dataset 29
3.4.4 Performance of Angels Dataset 33
3.4.5 Performance of Head Dataset 37
3.4.6 Performance of Buddha Dataset 41
3.4.7 Discussions on RMSE 45
3.4.8 Discussions on Edges 49
Chapter 4. Conclusions and Future Work 52
References 53

[1] K. R. Lee, R. Khoshabeh and T. Nguyen, “Sampling-Based Robust Multi-Lateral Filter for Depth Enhancement,” European Signal Processing Conference (EUSIPCO), 2012.
[2] T. Matsuo, N. Fukushima and Y. Ishibashi, “Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement,” Computer Vision Theory and Applications (VISAPP), 2013.
[3] T. Matsuo, N. Kodera, N. Fukushima and Y. Ishibashi, “Depth Map Refinement Using Reliability Based Joint Trilateral Filter,” Computer and Information Technology, November 2013.
[4] J. Digne, “Similarity Based Filtering of Point Clouds,” Computer Vision and Pattern Recognition Workshops (CVPRW), 2012.
[5] T. R. dos Santos, A. Seitel, H.P. Meinzer and L. Maier-Hein, “Time-of-Flight Surface De-noising through Spectral Decomposition,” Bildverarbeitung fur die Medizin 2011, 2011.
[6] S. Schwarz, M. Sjostrom and R. Olsson, “Depth Map Upscaling Through Edge Weighted Optimization,” Three-Dimensional Image Processing (3DIP) and Applications II, 2012.
[7] S. Schwarz, M. Sjostrom and R. Olsson, “Incremental Depth Upscaling Using an Edge Weighted Optimization Concept,” 3DTV Conference: The True Vision ‐ Capture, Transmission and Display of 3D Video (3DTV-CON), 2012.
[8] M. Georgiev, A. Gotchev and M. Hannuksela, “De-Noising of Distance Maps Sensed by Time-of-Flight Devices in Poor Sensing Environment,” International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013.
[9] E. Cappelletto, P. Zanuttigh and G. M. Cortelazzo, “Handheld Scanning with ToF Sensors and Cameras,” ST-Day 2012, October 2012.
[10] A. Seitel, T. R. dos Santos, S. Mersmann, J. Penne, A. Groch, K. Yung, R. Tetzlaff, H. P. Meinzer and L. Maier-Hein, “Adaptive Bilateral Filter for Image Denoising and Its Application to In-Vitro Time-of-Flight Data,” Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, 2011.
[11] A. Lejeune, M.Van Droogenbroeck and J. Verly, “Adaptive Bilateral Filtering for Range Images,” URSI Benelux Forum, 2012.
[12] F. Lenzen, K. I. Kim, H. Schafer, R. Nair, S. Meister, F. Becker, C. S. Garbe and C. Theobalt, “Denoising Strategies for Time-of-Flight Data,” Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications, Springer, vol. 8200, pp. 25-45, 2013.
[13] Y. Cui, S. Schuon, S. Thrun, D. Stricker and C. Theobalt, “Algorithms for 3D Shape Scanning with a Depth Camera,” Pattern Analysis and Machine Intelligence, May 2013.
[14] H. Schoner, F. Bauer, A. Dorrington, B. Heise, V. Wieser, A. Payne, M. J. Cree and B. Moser, “Image Processing for Three-Dimensional Scans Generated by Time-of-Flight Range Cameras,” Journal of Electronic Imaging, April 2012.
[15] F. Y. Lin, “A Study of Time-of-Flight Depth Maps Denoising and Quality Improvement,” M.S. thesis, Department of Computer Science and Information Engineering, National Taiwan University of Science and Technology, 2013.
[16] M. Hansard, S. Lee, O. Choi and R. Horaud, Time-of-Flight Cameras: Principles, Methods and Applications, Springer, November 2012.
[17] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” International Conference on Computer Vision (ICCV), 1998.
[18] A. K. Tripathi, and S. Mukhopadhyay, “Single Image Fog Removal Using Bilateral Filter,” International Conference on Signal Processing, Computing and Control (ISPCC), 2012.
[19] Y. Cui, S. Schuon, D. Chan, S. Thrun and C. Theobalt, “3D Shape Scanning with a Time-of-Flight Camera,” Computer Vision and Pattern Recognition (CVPR), 2010.
[20] MESA Swissranger SR4000 Standard Time-of-Flight Camera [Online]. Available: http://www.mesa-imaging.ch/products/sr4000/, referenced on August 1st., 2014.
[21] Y. Cui, S. Schuon, D. Chan, S. Thrun and C. Theobalt, ToF & Laser Datasets [Online]. Available: http://www.mpi-inf.mpg.de/~theobalt/tof/, referenced on September 1st., 2013.