臺灣博碩士論文加值系統

視訊物體切割是一種基於視訊內容為基礎的應用性關鍵技術，諸如，視訊內容的搜尋與取回(index-retrieval)、壓縮(compression)與表示(representation)等。在使用上視訊物體切割常被置於系統的前處理階段以將輸入的視訊畫面分割成數個視訊物體(video object)。因為自然界的環境條件變化難料，故許多針對視訊物體切割所提出的論文，大多針對較簡單的室內環境或室外白天的環境，以致於這些方法在夜間環境下將失去效用。在夜間條件下，一般道路上的車輛都會開車燈來看清楚前方的環境，車燈照射到地面所產生之亮光，即是造成夜間切割失敗的主因；加上路燈照射下車輛的陰影，同樣降低了切割的準確率。因此，本論文提出一利用顏色變動特性加上車燈資訊並結合改變偵測技術之視訊車輛切割與計數演算法，並將之應用於夜間環境下之單向(來向)道路。目的即是要降低車燈所造成的地面反射光對整個車輛切割準確率的影響，並計算道路上的車流量，以提供後續處理所需的資訊。其中，改變偵測用來得到一初始移動物的資訊，其中包括了地面反射光與陰影區域；針對初始移動物區域，利用顏色變動的特性可大致取得地面反射光的區域並從初始移動物區域中去除。之後利用亮度資訊與車燈判別方法取得車輛的車燈資訊，將之用來實現物體區域補償、陰影區預測與車輛計數，然後使用類似於地面反射光偵測的方法來取得陰影區並去除。最後，我們將以主觀與客觀的方式來評估所提出的切割演算法之優劣。
本論文實驗使用Dong-men bridge 03、Dong-men bridge 06 和Chang-rong bridge 02 等影像序列來評估切割結果。初始移動物切割的平均準確率依序為33.44%、30.78%、18.16%，經過地面反射光、物體區域補償與陰影消除後，平均準確率依序為46.06%、56.39%、47.99%；而平均錯誤改進率依序為47.22%、63.36%、72.87%。車輛計數部份，則使用Dong-men bridge 01、Dong-men bridge 04與Dong-men bridge 08等影像序列。實驗結果顯示車輛計數在正常情況下有80%以上的準確率。雖然夜間環境下影響變數太多以致於切割準確率不是很高，但整體錯誤改進率顯示誤切割區域已大幅的減少。

Video object segmentation is the key technology (e.q. index-retrieval, compression and representation) for content-based video processing. It is usually used to do the pre-processing for content-based video system in order to separate the input video frame into several video objects. Because of the varied environment conditions of real world, most segmentation algorithms focus on the simple circumstance of indoor environment or outdoor environment in daytime. However, these methods do not provide acceptable results in nighttime environment. Normally in nighttime, vehicles moving on the road have turned-on headlights for clear vision. The illumination produced by headlights on the ground deeply reduces the accuracy of segmentation. Moreover, shadow effect is also a problem in segmentation.
In this thesis, we propose a vehicle segmentation and counting algorithm by combining the property of color variation and headlight information and change detection for nighttime traffic environment (one-way road with coming direction). The goal is to reduce the effect of ground-illumination that decreases the accuracy of vehicle segmentation. Besides, the amount of traffic flow is calculated and it can be used in other post-application, like traffic flow reporting back or controlling. The change detection is employed to generate the initial object mask which includes ground-illumination and shadow region, afterwards we utilize the property of color variation to detect the ground-illumination pixels and remove them from the initial object mask. Next, the intensity information and a vehicle headlight classification method are used to obtain the headlight information for object region compensation, shadow region prediction and vehicle counting. Finally, we utilize the method similar to ground-illumination detection to detect the shadow pixels and remove them. In the end, subjective and objective evaluations are showed.
The experiments for segmentation in this thesis are based on the Dong-men bridge 03、Dong-men bridge 06 and Chang-rong bridge 02 sequence. The average accuracy of initial object segmentation is 33.44%、30.78% and 18.16%, respectively. The average accuracy of final object segmentation is 46.06%、56.39% and 47.99%. The average value of improvement ratio is 47.22%、63.36% and 72.87%. For vehicle counting, we use Dong-men bridge 01, Dong-men bridge 04 and Dong-men bridge 08 sequence. The accuracy of vehicle counting is above 80% in normal situation. Although there are other influence factors decreasing the accuracy so that the value is not very high, but the improvement ratio shows that the error detection region has been reduced substantially.

摘要 i
Abstract iii
Contents v
Figure Captions viii
Table Captions xi
Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Introduction to Video Object Segmentation 2
1.2.1 Content-Based Video Processing 2
1.2.2 Video Object Segmentation 3
1.3 Organization of this thesis 5
Chapter 2. Overview of Video Object Segmentation Algorithms 6
2.1 Preliminary of Video Object Segmentation Algorithms 6
2.2 Category of Video Object Segmentation Algorithms 6
2.2.1 Automatic Video Object Segmentation Algorithms 7
2.2.1.1 Edge Information Based Video Segmentation 7
2.2.1.2 Combined Temporal and Spatial Information Based Video
Segmentation 9
2.2.1.3 Change Detection Based Video Segmentation 15
2.2.2 Semi-Automatic Video Object Segmentation Algorithm 17
2.3 Remarks about the Previous Methods 17
2.4 Various Environments of Video Sequences 18
2.5 The Proposed Video Object Segmentation Algorithm 20
Chapter 3. Initial Object Segmentation Based on Change Detection 25
3.1 HSI Color Space Transformation 25
3.2 Gaussian Smoothing Filter 27
3.3 Difference Frame 28
3.4 Histogram-Based Change Detection 30
3.5 Object Region Detection 34
3.6 Background Updating 36
Chapter 4. Ground-illumination and Shadow Region Excluding and Vehicle Counting 40
4.1 Ground-illumination Detection and Excluding 41
4.1.1 Different Colors of Streetlamps and Headlights 41
4.1.2 Features of Ground-illumination 42
4.1.3 Ground-illumination Detection 43
4.1.3.1 Yellow streetlamp and yellow headlight ( ) situation 43
4.1.3.2 Yellow streetlamp with white headlight ( ) situation 44
4.1.3.3 White streetlamp with yellow headlight ( )situation 45
4.1.3.4 White streetlamp with white headlight ( )situation 46
4.1.4 Ground-illumination Removal 47
4.2 Object Region Compensation 48
4.2.1 Vehicle Headlight Detection 48
4.2.2 Classification of Car and Bike Headlight 49
4.2.2.1 Headlight pair of car 50
4.2.2.2 Headlight of bike 52
4.2.3 Compensation Region 53
4.3 Shadow Detection and Removal 55
4.3.1 Prediction of Shadow Region 55
4.3.2 Shadow Detection 56
4.4 Vehicle Counting 58
4.5 Post-Processing Filter 59
Chapter 5. Experimental Results 60
5.1 Simulation Environment 61
5.2 Results of Initial and Final Segmentation 61
5.3 Vehicle counting 71
5.4 Subjective Evaluation 71
5.5 Objective Evaluation 72
Chapter 6. Conclusions 78
6.1 Remarks for Proposed Algorithm 78
6.2 Future Work 79
References 80

[1] Information Technology－Coding of Audio-Visual Objects－Part 2: Visual. ISO/IEC JTC1/SC29/WG 11 N4350, International Standard, July 2001.
[2] J. F. Canny, “A Computational Approach to Edge Detection,” IEEE Transaction on Pattern Analysis. Machine Intelligence, vol. PAMI-6, pp. 679-689, Nov. 1986.
[3] Shao-Yi Chien, Yu-Wen Hwang, and Liang-Gee Chen, “Predictive Watershed: A Fast Watershed Algorithm for Video Segmentation,” IEEE Transaction on Circuits and System for Video Technology, vol. 13, no. 5, pp. 453-461, May 2003.
[4] Thou-Ho (Chao-Ho) Chen, Yung-Chuen Chiou, Ming-Kun Wu and Yi-Fan Li, “An Efficient Video Object Segmentation Algorithm Using Change Detection and Background Updating Technique,” International Conference on Systems and Signals (ICSS), 2005.
[5] Jose M. Cobo, Luis Salgado, and Julian Cabrera, “Adaptive Segmentation for Gymnastic Exercises Based on Change Detection over Multiresolution Combined Differences,” IEEE International Conference on Image Precessing, vol. 1, pp. 337-340, Oct. 2004.
[6] Shao-Yi Chen, Yu-Wen Hwang, Bing-Yu Hsieh, Shyh-Yih Ma, and Liang-Gee Chen, “Fast Video Segmentation Algorithm with Shadow Cancellation, Global Motion Compensation, and Adaptive Threshold Techniques,” IEEE Transaction on Circuits and System foe Video Technology, vol. 6, no. 5, pp. 732-748, Oct. 2004.
[7] Rita Cucchiara, Massimo Piccardi and Paola Mello, “Image Analysis and Rule-Based Reasoning for a Traffic Monitoring System,” IEEE Transactions on Intelligent Transportation Systems, vol. 1, no. 2, June 2000.
[8] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detecting Object, Shadows and Ghosts in Video Streams by Exploiting Color and Motion Information,” in Proceedings of 11th international Conference on Image Analysis and Processing, pp. 360-365, September 2001.
[9] Shao-Yi Chien, Yu-Wen Huang, Bing-Yu Hsieh, Shyh-Yih Ma, and Liang-Gee Chen, “Fast Video Segmentation Algorithm with Shadow Cancellation, Global Motion Compensation, and Adaptive Threshold Techniques,” IEEE Transactions on Multimedia, vol. 6, no. 5, pp. 732-748, Oct. 2004.
[10] A. Cavallaro, E. Salvador and T. Ebrahimi, “Shadow-aware Object-based Video Processing,” IEE Vision, Image and Signal Processing, vol. 152, no. 4, pp. 14-22, August 2005.
[11] Shahbe Mat Desa and Qussay A. Salih, “Image Subtraction for Real Time Moving Object Extraction,” IEEE International Conference on Computer Graphics, Image and Visualization (CGIV’04), pp. 41-45, July 2004.
[12] Rafael C. Gonzalez and Richard E. Wood, Digital Image Processing, 2Ed. New Jersey: Prentice Hall, 2002.
[13] Changick Kim and Jenq-Neng Hwang, “Object-Based Video Abstraction for Video Surveillance Systems,”IEEE Transaction on Circuits and System for Video Technology, vol. 12, no. 12, pp.1128-1138, Dec. 2002.
[14] C. Kim and J. N. Hwang, “Fast and Automatic Video Object Segmentation and Tracking for Content-Based Application,” IEEE Transaction on Circuits and System for Video Technology, vol. 12, pp. 122-129, 2002.
[15] Munchurl Kim, Jae Gark Choi, Daehee Kim, Hyung Lee, Myoung Ho Lee, Chieteuk Ahn and Y0-Sung Ho, “A VOP Generation Tool: Automatic Segmentation of Moving Object in Image Sequences Based on Spatio-Temporal Information,” IEEE Transaction on Circuits and System for Video Technology, vol. 9, no. 8, pp. 1216-1260, Dec. 1999.
[16] Munchurl Kim, J. G. Jeon, J. S. Kwak, M. H. Lee and C. Ahn, “Moving Object Segmentation in Video Sequence by User Interaction and Automatic Object Tracking,” Image and Vision Computing, vol. 19, pp. 245-260, 2001.
[17] Weiping Li, Jens-Rainer Ohm,Mihaela van der Schaar, Hong Jiang, Shipeng Li, “The MPEG-4 Video Standard Verification Model version 18.0,”ISO/IEC JTC1/SC29/WG 11 N3908, Jan. 2001.
[18] Jing Li, Stan Z. Li, Quan Pan and Tao Yang, “Illumination and Motion-Based Video Enhancement for Night Surveillance,” Proceedings 2nd Joint IEEE International Workshop on VS-PETS, Beijing, Oct. 2005.
[19] Jose M. Martinez, “MPEG-7 Overview,”ISO/IEC JTC1/SC29/WG11 N5525, Mar. 2003.
[20] Thomas Meier, King N. Ngan, “Video Segmentation for Content-Based Coding,” IEEE Transaction on Circuits and System for Video Technology, vol. 9, no. 8, pp. 1190-1203, Dec. 1999.
[21] Thomas Meier, King N. Ngan, “Automatic Segmentation of Moving Object for Video Object Plane Generation,” IEEE Transaction on Circuits and System for Video Technology, vol. 8, no. 5, pp. 525-538, Sep. 1998.
[22] Roland Mech and Michael Wollborn, “A noise robust method for 2D shape estimation of moving object in video sequence considering a moving camera,” Signal Processing, vol. 66, pp. 203-217, 1997.

[23] S.-Y. Ma, S.-Y. Chien, and L.-G. Chen, “An efficient moving object segmentation algorithm for MPEG-4 encoding systems,” in Proc. Int. Symp. Intelligent Signal Processing and Communication Systems 2000, 2000.
[24] Urban Meis, Wemer Ritter and Heiko Neumann, “Detection and Classification of Obstacles in Night Vision Traffic Scenes based on Infrared Imagery,” IEEE International Conference on Intelligent Transportation Systems, Shanghai, pp. 1140-1145, Oct. 2003.
[25] King N. Ngan, Thomas Meier, and Douglas Chai, Advanced Video Coding: Principles and Techniques, Elsevier Science B. V., 1999.
[26] Sohail Nadimi and Bir Bhanu, “Moving Shadow Detection Using a Physics-based Approach,” in Proceedings of the 16th IEEE International Conference on Pattern Recognition, pp. 701-704, August 2002.
[27] Thomas Sikora, “The MPEG-4 video standard verification model,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 1,pp.19-31, Feb. 1997.
[28] Philippe Salembier and Montse Pardas, “Hierarchical Morphological Segmentation for Image Sequence Coding,” vol. 3, no. 5, pp. 639-651, Sep. 1994.
[29] Jurgen Stauder, Roland Mech, and Jorn Ostermann, “Detection of Moving Cast Shadows for Object Segmentation,” IEEE Transactions on Multimedia, vol. 1, no. 1, March 1999.
[30] A. M. Tekalp, Digital Video Processing, Prentice Hall PTR, 1995.
[31] Candemir Toklu, A. Murat Tekalp, and A. Tanju Erdem, “Semi-Automatic Video Object Segmentation in the Presence of Occlusion,” IEEE Transaction on Circuits and System for Video Technology, vol. 10, no. 4, pp. 624-629, June 2000.
[32] Luc Vincent and Pierre Soille, “Watershed in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations,” IEEE Transaction on Pattern Analysis. Machine Intelligence, vol. 13, no. 6, pp. 583-598. June 1991.
[33] Wei Wei and King N. Ngan, “Integration of Motion and Image Features for Automatic Video Object Segmentation,” IEEE International Conference on Image Processing, pp. 361-364, Oct. 2004.
[34] Demin Wang, “Unsupervised Video Segmentation Based on Watersheds and Temporal Tracking,” IEEE Transaction on Circuits and System for Video Technology, vol. 8, no. 5, pp. 539-546, 1998.
[35] J. M. Wang, Y. C. Chung, C. L. Chang, and S. W. Chen, “Shadow Detection and Removal for Traffic Images,” in Proceedings of the 2004 IEEE International Conference on Networking, Sensing and Control, Taipei, Taiwan, March 21-23, 2004.