臺灣博碩士論文加值系統

中文摘要
近年來戶外監視系統已經非常廣泛地被人們所使用，畫面的品質也越來越受到重視，其中最常遇到影響畫面的因素，首推雨滴。雨是日常生活中常常遇到的天候狀況，也是干擾戶外監視系統畫面較為嚴重的因素之一，破壞原本清晰的畫面並導致後續對監視影像內容的判斷及分析，同時造成監視系統裡頭的自動影像分析次系統的誤判，例如特徵檢測、分割及物件辨識等。由於雨滴是在空間上是任意且快速的移動，因此如何將他們從畫面上移除並將具雜訊的畫面還原回復清晰影像，將是一個極具挑戰性的問題。針對此問題，已有許多理論在文獻中提出，然而它們大多只適用在某些限制條件下，而且許多重要參數都還不能自動調整，使得它們無法在實際的監視系統加以應用。
由於雨滴在畫面中會造成複雜的空間及時間變異訊號，因此本篇論文中，我們把雨滴效應移除以畫面去雜訊方式來處理。我們首先提出使用空間-時間雙邊濾波器(spatial-temporal bilateral filter)以作為雨滴濾波器，同時我們也提供軟硬體共用的方式，並以自我偵測參數的方法來達到具備實際應用上必須能簡易安裝的即插即用(plug and play)功能。實驗結果顯示本演算法在効能上超過其它演算法，同時本論文也詳細描述如何以最精簡及有效率的硬體架構來達到所要求的規格，以及即時實現去除雨干擾的效果。我們把此硬體經由製程技術設計流程完成晶片，雛形晶片係利用台積電90nm技術製造，面積為2.352 x 2.354mm²，可在運算速度為125 MHz的情況下，達到即時處理每秒30張1920 x 1080像素的視訊資料能力。

Abstract
As outdoor surveillance systems have been widely deployed everywhere, the emphasis on the quality of their frames also rises simultaneously. Rain is a weather phenomenon we frequently encounter in our daily lives, and it is also one the most severe interferences to the frames captured in outdoor surveillance systems, which usually leads to unclear frames and makes users hard to judge and analyze the image information. It also causes the failures of several automatic video analysis subsystems in video surveillance, such as feature detection, segmentation, and object recognition. Due to the random and swift movement of raindrops, it becomes a challenge to remove them from the images and to restore the noisy images back to the clear one. Several previous works have been proposed in literatures; however, most of them can only deal with limited conditions, and several important parameters are required to be set manually, which makes these approaches not feasible for surveillance systems.
In this thesis, since the rain drops result in complex space-and-time-varying signals in the images, we treat the rain removal problem as am image de-noising problem. We first propose to employ spatial-temporal bilateral filter as a raindrop filter. Moreover, we also propose software and hardware cooperation methods along with the function of adaptive parameters to achieve the “plug and play” capability required for easy deployment in real applications. The experimental results show that the efficiency of this algorithm surpasses those of other existing algorithms.
Detailed hardware architecture analysis is also presented in this thesis to derive an efficient hardware architecture for the proposed algorithm. We also implement the hardware as a chip by using the ASIC design flow with TSMC 90nm technology, and the die size is 2.352 x 2.354mm². Under the working frequency of 125 MHz, the chip can support real-time 30 1920x1080 fps video processing capability.

Contents
Abstract viii
Chapter 1 Introduction 1
1.1 Overview 1
1.1.1 Patch-Based Rain Detection 2
1.1.2 Pixel-Based Rain Detection 3
1.1.3 Frequency-Based Rain Detection 4
1.1.4 Filter-Based Rain Detection 4
1.1.5 Motivations of Spatial-Temporal Bilateral Filter Rain Removal 5
1.2 Thesis Organization 6
Chapter 2 Proposed Spatial-Temporal Bilateral Filter 7
2.1 Properties of Rain 7
2.1.1 Intensity Property of Rain 7
2.1.2 Temporal Property of Rain 7
2.2 Image Smoothing with Gaussian Convolution 10
2.3 Edge-preserving Filtering with the Bilateral Filter 11
2.3.1 Analysis of Parameters 13
2.3.2 Gaussian versus others weighting functions 15
2.4 Proposed Spatial-temporal Bilateral Filter 16
2.4.1 Kernel Selection of Spatial-Temporal Bilateral Filter 17
2.4.2 Parameter Selection of Spatial-Temporal Bilateral Filter 18
2.5 Algorithm Experiment Result and Analysis 18
Chapter 3 Proposed Algorithm and System Architecture 20
3.1 Software Algorithm Overview 20
3.1.1 Adaptive Intensity Weight Parameter 20
3.2 ASIC Hardware Algorithm Overview 29
3.2.1 Target 29
3.2.2 Hardware Data Flow Analysis 29
3.2.3 Hardware Cost Analysis 31
3.2.4 Reduced Hardware Cost Analysis 36
Chapter 4 Proposed Hardware Architecture 44
4.1 Design Target 44
4.2 Design Challenge 44
4.3 Test Environment 44
4.3.1 Architecture of Arbiter module 45
4.3.2 Architecture of Decoder module 46
4.4 Proposed Design Techniques 46
4.4.1 Function of AHB master module 47
4.4.2 Function of AHB slave module 48
4.4.3 Function of Input Ping-Pong Buffer module 49
4.4.4 Spatial-temporal Bilateral Filter 50
Chapter 5 Hardware Implementation Result 59
5.1 ASIC Design Flow 59
5.1.1 Specification definition and Algorithm Analysis 59
5.1.2 C simulation 60
5.1.3 Architecture / System Design 60
5.1.4 Architecture Design / Verilog RTL Design and Simulation 60
5.1.5 Logic Synthesis and Optimization 60
5.1.6 Gate Level Simulation 61
5.1.7 Floorplanning, Automatic Placement and Rounting 61
5.1.8 Layout Verification (DRC/LVS) 61
5.1.9 Post-Layout Gate Level Simulation 62
5.2 Chip Layout and Specification 62
Chapter 6 Conclusions 63
REFERENCES 64

REFERENCES
[1]Middlen, W.E.K. 1952. Vision Through the Atmosphere. University Of Toronto Press.
[2]McCartney, E.J. 1975. Optics of the Atmosphere: Scattering by molecules and particles. John Wiley and Sons.
[3]Narasimhan, S.G. and Nayar, S.K. 2002. Vision and the Atmosphere. IJCV, 48(3):233–254.
[4]Narasimhan, S.G. and Nayar, S.K. 2003. Contrast restoration of weather degraded images. IEEE Trans. on PAMI, 25(6).
[5]Schechner, Y.Y., Narasimhan, S.G., and Nayar, S.K. 2001. Instant dehazing of images using polarization. CVPR.
[6]Cozman, F. and Krotkov, E. 1997. Depth from scattering. CVPR, 31:801–806.
[7]Oakley, J.P. and Satherley, B.L. 1998. Improving image quality in poor visibility conditions using a physical model for degradation. IEEE Trans. on Image Processing, 7.
[8]Tan, K. and Oakley, J.P. 2000. Enhancement of color images in poor visibility conditions. ICIP, 2.
[9]Hase, H., Miyake, K., Yoneda, M.: Real-time snowfall noise elimination. In: Proceedings of 1999 International Conference on Image Processing ICIP 1999, vol. 2, pp. 406-409 (1999)
[10]Starik, S., Werman, M.: Simulation of rain in videos. In: Texture Workshop, ICCV, vol. 2, pp. 406-409 (2003)
[11]K. Garg and S. K. Nayar, “Detection and removal of rain from videos,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., June 2004, vol. 1, pp. 528-535.
[12]K. Garg and S. K. Nayar, “When does a camera see rain?” in Proc. of IEEE Int. Conf. Comput. Vis., Oct. 2005, vol. 2, pp. 1067-1074.
[13]K. Garg and S. K. Nayar, “Vision and rain. Internatl. Journal of Computer Vision, 75(1):3-27,2007
[14]X. Zhang H. Li, Y. Qi, W. K Leow, and T. K. Ng, “Rain removal in video by combing temporal and chromatic properties,” in Proc. IEEE Int. Conf. Multimedia Expo, Toronto, Ont. Canada, July 2006, pp. 461-464.
[15]P. Barnum, T. Kanade and S.G. Narasimhan, “Spatio-temporal frequency analysis for removing rain and snow from videos,” in Proc. of ICCV, 2007
[16]W. J. Park, and K. H. Lee, “Rain removal using Kalman filter in video,” International Conference on Smart Manufacturing Application, pp. 494-497, 2008.
[17]L. P. Yaroslavsky, Digital Picture Processing. An Introduction. New York: Springer-Verlag, 1985
[18]J. S. Lee, “Digital image smoothing and the sigma filter,” in Computer Vision, Graphics, and Image Processing, vol. 24, 1983, pp. 255-269
[19]P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Machine Intell, vol. 12, no. 7, pp. 629-639, Jul. 1990.
[20]J. Van de Weijer and R. van den Boomgaard, “Local mode filtering,” in Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 428-433, 2001
[21]C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. 6th Int. Conf. Computer Vision, New Delhi, India, 1998, pp. 839-846.
[22]S. Paris, P. Kornprobst, J. Tumblin, F. Durand. Bilateral Filtering: Theory and Application, Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1-73,2009
[23]V. Aurich and J. Weule, “Non-linear Gaussian filters performing edge preserving diffusion,” in Proceedings of the DAGM Symposium, pp. 538-545, 1995.
[24]S. M. Smith and J. M. Brady, “SUSAN – A new approach to low level image processing,” International Journal of Computer Vision, vol. 23, no. 1, pp. 45-78, May 1997.
[25]F. Durand and J. Dorsey, “Fast bilateral filtering for the display of high-dynamic-range images,” ACM Siggraph, vol. 21, no. 3, pp. 257-266, 2002.
[26]C. Liu, W. T. Freeman, R. Szeliski, and S. Kang, “Noise estimate from a single image,” in Proceedings of the Conference on IEEE Computer Vision and Pattern Recognition , vol. 1, pp. 901-908, 2006.
[27]F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel, Robust Statistics: The Approach Based on Influence Functions. New York: Wiley, 1986.
[28]P. J. Huber, Robust Statistics. John Wiley and Sons, New York
[29]M. J. Black, G. Sapiro, D. H. Marimont, and D. Heeger, “Robust anisotropic diffusion,” IEEE Transaction on Image Processing, vol. 7, no. 3, pp. 421-432, March 1998.
[30]J. J. Francis and G. De Jager, “The bilateral median filter,” Transactions of the South African Institute of Electrical Engineers, vol. 96, no. 2, pp. 106-111, 2005
[31]P. Perona and J. Malik. “Scale-space filtering and edge detection using anisotropic diffusion”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629-639, July 1990
[32]Charles V. Stewart. “Robust parameter estimation in computer vision”. SIAM Review, 41(3):513-537, 1999
[33]AMBA Specification (Rev 2.0), 1999
[34]http://www.cadence.com.
[35]http://www.springsoft.com.
[36]http://www.synopsys.com.
[37]http://www.mentor.com.