臺灣博碩士論文加值系統

近年來，國際間恐怖事件頻傳，使得公共安全產生莫大的疑慮，因此各國政府開始強化公共安全監控。而在我國雖然未曾發生恐怖攻擊的事件，但是為了遏止犯罪、改善治安。我國政府致力於建構全國治安監控防護網，用以保障人民居家優質生活安全。然而，傳統的視訊監控系統在複雜、動態場景很容易發生誤判，效用相當有限。由於，其屬於被動反應方式，完全仰賴保全管理人員的即時監控，導致實用性降低。除此之外，傳統的視訊監控系統計算量較大且耗費記憶體、硬碟儲存空間；所以成本高不適合於嵌入式平台使用。有鑑於此，發展適合於嵌入式平台上使用的智慧型影像監視系統為一個重要研究議題。
本論文提出兩個適用於前景物件偵測的背景模型，分別為HPB (hybrid pixel-based)背景模型與block-based major color背景模型。由於監控場景必然會有靜態區域、動態區域同時存在的現象；HPB背景模型即是採用靜態、動態背景分離的方式，建立多層背景模型。並且使用homogeneous background subtraction將前景物件擷取出來。而在背景更新方面，靜態、動態背景亦採用不同的背景更新策略，以免因過時的背景模型造成誤判；藉由模擬結果顯示，HPB背景模型在渾沌(chaotic)背景中，其平均錯誤率(average error rate)低於1%。另外，因為場景中相鄰pixel的色彩資訊往往重覆出現，造成背景模型中儲存相同的背景資訊，浪費記憶體空間與計算量。針對此一缺點，本論文亦提出block-based major color背景模型。我們使用major color來表示block內的所有影像資訊，如此即可避免重覆記錄背景資訊。經過與其它演算法比較後，block-based major color背景模型可以降低至少37%記憶體使用量，並且將similarity value提升3%~55%。我們利用演算法、程式最佳化之技術，將本演算法於TI TMS320DM6446 DaVinci開發板上面執行，在影像size為768×576解析度下可達到17 frame per second。
而在物件追蹤部份本論文擷取前景物件的色彩紋理(color texture)特徵進行物件追蹤。並使用MSE快速比對，即可追蹤每一個前景物件。最後我們將物件偵測、物件追蹤整合，並實現於嵌入式平台上。經過測試後，本論文提出之演算法可以克服複雜背景，例如：光線變化、動態背景…等。此外，我們亦完成2項智慧情境分析：(1)入侵區域警告 (2)物件追蹤與計數，達到智慧型影像監控系統之需求。

Recently, many international terroristic attacks caused great disturbance in public safety. Therefore, the governments around the world began to construct and strengthen image surveillance system for public safety. Although Taiwan does not encounter any terroristic attack, the surveillance system is used to diminish crime rate and improve public security. Up to now, Taiwan has strived to establish a nationwide security surveillance network to ensure homeland security such that our people can live in a safe environment. In general, the traditional video surveillance system with complex scene can easily produce false-positive detections. Furthermore, it has lower processing speed and larger more memory consumption. In other words, the traditional video surveillance system is unsuitable for implementing in embedded system. As a result, developing an efficient intelligent video surveillance system for embedded system has become an important research topic.
For foreground object detection, this dissertation has proposed two background models which are HPB (hybrid pixel-based) and block-based major color background model. As the input sequence appears static and dynamic scenes simultaneously, HPB background model adopts single-layer approach for constructing the static background model and use multi-layer method to establish dynamic background model. Subsequently, homogeneous background subtraction is presented to extract foreground objects. For background update period, static and dynamic backgrounds adopt different running-average update strategies to reduce false-positive rate. From the experimental results, it is revealed that HPB background model has average error rate less than 1% in chaotic or complex scenes.
From the detail observation to the background pixels, the adjacent pixels have the similar color information. For each image block, this dissertation proposes the block-based major color background model. Some major colors are adopted to represent the feature of the block. The identical color pixels are not recorded repeatedly. After comparing with other algorithms, the block-based major color background model can decrease at least 37% of memory consumption and improve 3%~55% of similarity value.
For object tracking, we extract color textures from foreground objects, and then use MSE method to perform quick matching and track each individual object correctly. Finally, object detection and tracking are integrated and implemented in the embedded platform. From the experiment, two scenario analyses are completed: (1) warning of the intrusive objects, and (2) object tracking and counting to achieve the requirement of the smart surveillance image system. By adopting optimization techniques, the algorithm is realized in TI TMS320DM6446 DaVinci development kit. The performance can achieve 17 frames per second based on display resolution at 768×576.

中文摘要…………………………………………………………………………………….i
Abstract...……………………………………………………………………………………ii
Contents……………………………………………………………………………….…….v
List of Tables……………………………………………………………………………...viii
List of Figures………………………………………………………………………………ix
1. Introduction……………………………………………………………………………….1
1.1 Background and Motivation……………………………………………………….….1
1.2 Design Objective……………………………………………………………………...2
1.3 Dissertation Organization……………………………………………………………..3
2. Related Researches Introduction and Classification……………………………………....4
2.1 Single Layer Background Model Introduction………………………………………..5
2.1.1 Background Registration Method……………………………………………….5
2.1.2 Probability-based Background Model and Extraction Algorithm………………7
2.1.3 DCT Coefficients Background Model…………………………………………..8
2.1.4 ALW Method……………………………………………………………………9
2.2 Multi-layer Background Model Introduction………………………………………..12
2.2.1 Codebook Model……………...……………………………………………….12
2.2.2 SACON Model………………………………………………………………...14
2.2.3 Texture-based Background Model……………………………………………..17
2.2.4 Hierarchical Method Codebook Model………………………………………..19
2.2.5 ViBe Algorithm..………...……………………………………………………...22
3. Low Error Rate HPB Model and Homogenous Background Subtraction……………….24
3.1 HPB Model Construction in Learning Phase……………………………….………..24
3.1.1 Creating a Stable Background Record……………….………………………...25
3.1.2 Creating a Astable Background Record………………………………………..27
3.2 Foreground object detection with Homogenous Background Subtraction....……….29
3.3 HPB Model Updating....……………………………………………………………..30
3.3.1 Updating the Stable Background Record....…………………………………....30
3.3.2 Updating the Astable Background Records…………………………………....30
3.4 Experimental Results and Comparison……………………………………………....32
3.4.1 ABR Layers Analysis…………………………………………………………..32
3.4.2 Setting the Threshold Value……………………………………………………37
3.4.3 Evaluation of Foreground Object Detection Algorithms.……………………...41
3.4.4 Evaluation of Background Updating Performance.……………………………50
3.5 Embedded Platform Implementation………………………………………………...51
4. Block-Based Major Color Method for Foreground Object Detection…….…………..…53
4.1 Concept of Block-based Major Color………………………………………………..53
4.2 Major Color Spectrum Histogram...............................................................................56
4.3 Major color Construction in Background Training…………………………………..58
4.4 Foreground Object Detection………………………………………………………...60
4.5 Object Boundary Smoothing………………………………………………………....62
4.6 Experimental Results and Comparison……………………………………………....63
4.6.1 Evaluation of Foreground Object Detection Algorithms……………………....63
4.6.2 Similarity Analysis in Different Block Size…………………………………....67
4.7 Embedded Platform Implementation………………………………………………...69
5. Fast Texture-Based Object Tracking Algorithm on Embedded Platform………………..73
5.1 Modeling the Foreground Object by Texture…………………….…....……………..73
5.2 Texture-Based Tracking……………………………………………………………...75
5.3 Algorithm Simulation………………………………………………………………...76
5.4 Experimental results and comparison………………………………………………..77
5.4.1 Execution time comparison on PC...………...……………………..….………..77
5.4.2 Execution Speed Analysis.……………………………………………………..78
5.4.3 Experiment Result………………………………………………………...….....79
5.5. Surveillance System Integration…………………………………………………….80
6. Conclusion……………………………………………………………………………….83
Reference…………………………………………………………………………………...85
Publications…………………………………………………………………………………93
Patents……………………………………………………………..……………96
Contest Awards……..………………………………………………………………………97
Project Award……………………………………………………………………………...101
Exhibitions……………………………………………………………….………………...102
National IC Design Contest Organization…………………………………………………104

[1] C. Stauffer, and W.E. Grimson, “Adaptive background mixture models for real-time tracking,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp.246-252, 1999.
[2] A. Cavallaro, O. Steiger, and T. Ebrahimi, “Tracking video objects in cluttered background,” IEEE Tran. on Circuits and Systems for Video Technology, vol. 15, Issue 4, April, 2005.
[3] L. Zhu, J. Zhou, and J. Song, “Tracking multiple objects through occlusion with online sampling and position estimation,” Pattern Recognition, pp.1091-1105, 2006.

[4] J.-W. Hsieh, Y.-T. Hsu, H.-Y. Mark Liao, and C.-C. Chen, “Video-based human movement analysis and its application to surveillance systems,” IEEE Trans. on Multimedia, vol. 10, pp.372-384 ,April, 2008.

[5] I. Haritaoglu, D. Harwood, and L. S. Davis, “W4: Real-Time surveillance of people and their activities,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.22, pp. 809-830, 2000.

[6] W. Lao, J. Han, and Peter H.N. de With, “Automatic video-based human motion analyzer for consumer surveillance system,” IEEE Trans. on Consumer Electronics, vol. 55, Issue 22, pp. 591-598.2009.

[7] C.-H. Chuang, J.-W. Hsieh, L.-W. Tsai, and S.-Y. Chen, “Carried object detection using ration histogram and its application to suspicious event analysis,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 19, pp. 911-916, 2009.

[8] S. Gupte et al., “Detection and classification of vehicles,” IEEE Trans. on Intelligent Transportation Systems, vol. 3, pp. 37-47, 2002.

[9] S.-C. Chen, M.-L. Shyu, S. Peeta, and C. Zhang, “Learning-Based spatio temporal vehicle tracking and indexing for transportation multimedia database systems,” IEEE Trans. on Intelligent Transportation Systems, vol. 4, pp. 154-167, 2003.

[10] J.-W. Hsieh et al., “Automatic traffic surveillance system for vehicle tracking and classification,” IEEE Trans. on Intelligent Transportation Systems, vol. 2, pp. 175-187, 2006.

[11] W. Zhang, and Q.M. Jonathan Wu, “Moving vehicles detection based on adative motion histogram,” Digital Signal Processing, vol. 20, Issue 3, pp. 793-805, 2010.

[12] N. Buch et al, “Detection and classification of vehicles for urban traffic scenes,” International Conference on Visual Information Engineering, pp. 182-187, 2008.

[13] P.-M. Jodoin, M. Mignotte, and J. Konrad, “Statistical background subtraction using spatial cues,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 17, pp. 1758-1763, 2007.

[14] J. Cheng, J. Yang, Y. Zhou, Y. Cui,“ Flexible background mixture models for foreground segmentation,” Image and Vision Computing, vol. 24, pp. 473-482, 2006

[15] L. Li, W. Huang, Irene Y.-H. Gu, and Q. Tian, “Statistical modeling of complex backgrounds for foreground object detecion,” IEEE Trans. on Image Processing, vol. 13, pp. 1459-1472, 2004.

[16] C. Kim, and J.-N. Hwang, “Object-based video abstraction for video surveillance systems,” IEEE Trans. Circuits and Systems for Video Technology, vol. 12, no. 12, pp. 1128-1138, 2002.

[17] S. -Y. Chien, Y. -W. Huang, B. -Y. Hsieh, S. -Y. Ma, and L. -G. Chen, “Efficient moving object segmentation algorithm using background registration technique,” IEEE Trans. Circuits and Systems for Video Technology, vol. 12, no. 7, pp. 577-586, 2002.

[18] S. -Y. Chien, Y. -W. Huang, B. Y. Hsieh, S. Y. Ma, and L. G. Chen, “Fast video segmentation algorithm with shadow cancellation, global motion compensation, and adaptive threshold techniques,” IEEE Trans. Multimedia, vol. 6, no. 5, pp. 732-748, 2004.

[19] C. -C. Chiu, M. -Y. Ku, et.al., “A robust object segmentation system using a probability-based background extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518-528, 2010.
[20] W. Wang, J. Yang, and W. Gao, “ Modeling background and segmenting moving objects form compressed video,” IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 12, pp. 1477-1490, 2006.
[21] Y. Wang, S. Velipasalar, and M. Casares, “Cooperative object tracking and composite event detection with wireless embedded smart cameras,” IEEE Trans. Image Processing, vol. 19, no. 10, pp. 2614-2633, 2010.
[22] C.-F. Juang, C.-M. Chang, J.-R. Wu, D. Lee, “Computer Vision-Based human body segmentation and posture estimation,” IEEE Trans. on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 39, pp. 119-133, 2009.

[23] C. Stauffer; and W.E. Grimson, “Adaptive background mixture models for real-time tracking,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp.246-252, 1999.
[24] C. Stauffer, and E. L. Grimson, “Learning patterns of activity using
real-time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22,no. 8, pp. 747–757, Aug. 2000.

[25] D.-S. Lee, “Effective Gaussian mixture learning for video background subtraction,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 827-832, 2005.

[26] H. H. P. Wu, J. H. Chang, P. K. Weng, and Y. Y. Wu, “Improved moving object segmentation by multi-resolution and variable thresholding,” Optical Engineering. vol. 45, no. 11, 117003, 2006.

[27] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using codebook model,” Real-Time Imaging, pp.172-185, 2005.

[28] H. Wang, and D. Suter, “A consensus-based method for tracking modelling background scenario and foreground appearance,” Pattern Recognition, pp.1091-1105, 2006.

[29] M. Heikkila, and M. Pietikainen, “A texture-based method for modeling the background and detecting moving objects,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 28, pp. 657-662, 2006.

[30] J.-M. Guo, Y.-F. Lin, C.-H. Hsia, M.-H. Shih, and C.-S. Hsu, “ Hierarchical method for foreground detection using codebook model,” IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 6, pp. 804-815, 2011

[31] O. Barnich, and M. Van Droogenbroeck, “ViBe: A universal background subtraction algorithm for video sequences,” IEEE Trans. on Image Processing, vol. 20, no. 6, pp.1709-1724, 2011

[32] D.-M. Tsai, and W.-Y. Chiu, “Motion detection using Fourier image reconstruction,” Pattern Recognition Letters, vol. 29, pp. 2145-2155, 2008.

[33] D.-M. Tsai, and S.-C. Lai, “Independent component analysis-based background subtraction for indoor surveillance,” IEEE Transactions on Image Processing, vol. 18, pp158-167, 2009.

[34] W.-K. Tsai et al, “High accuracy background model for real-time video foreground object detection,” Optical Engineering, vol. 51, no. 2, pp. 027202, 2012.

[35] W.-K. Chan, and S.-Y. Chien, “Real-time memory-efficient video object segmentation in dynamic background with multi-background registration technique,” IEEE Workshop on Multimedia Signal Processing, pp.219-222, 2007.

[36] W.-K. Chan, J.-Y. Chang, T.-W. Chen, Y.-H. Tseng, and S.-Y. Chien, “Efficient content analysis engine for visual surveillance network,” IEEE Trans. on Circuits and Systems for Video Technology , vol. 19, no. 5, pp.693-703, 2009.

[37] W.-K. Tsai, M.-H. Sheu, C.-L. Su, J.-J. Lin and S.-Y. Tseng, “Image object detection and tracking implementation for outdoor scenes on an embedded SoC platform,” 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp.386-389, September, 2009.

[38] W.-K. Tsai, M.-H. Sheu, C.-C. Lin, “Efficient multi-layer background model on complex environment for foreground object detection,” 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, pp.292-295, October, 2010.

[39] H.-L. Eng, J. Wang, A. H. K. S. Wah, W.-Y. Yau, “Robust human detection within a highly dynamic aquatic environment in real time,” IEEE Trans. on Image Processing, vol. 15, no. 6, pp.1583-1600, 2006.
[40] H.-L. Eng, K.-A. Toh, W.-Y. Yau, and J. Wang, “DEWS: A live visual surveillance system for early drowning detection at pool,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 18, no. 2, pp. 196-210, 2008.

[41] Y.-T. Chen, C.-S. Chen, and C.-R. Huang, Y.-P. Hung, “Efficient hierarchical method for background subtraction,” Pattern Recognition, vol. 40, pp. 2706-2715, 2007.

[42] B. Valentine, S. Apweokin, L. Wills, and S. Wills, “ An efficient, chromatic clustering-based background model for embedded vision platforms,” Computer Vision and Image Understanding, vol. 114, pp. 1152-1163, 2010.

[43] C.-H. Moon, D.-Y. Jang, and J.-N. Choi, “An SoC system for real-time moving object detection,” Lecture notes in computer science, vol. 4681, pp. 879-885, 2007.

[44] M. Casares, S. Velipasalar, and A. Pinto, “Light-weight salient foreground detection for embedded smart cameras,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1223-1237, 2010.

[45] C.-Y. Cho, J.-K. Yang, S.-Y. Tseng, and M.-H. Sheu, “Real-time object detection system on PAC Duo platform,” International Conference on Signal Processing Systems, pp. 585-590, 2010.

[46] S. Apewokin, B. Valentine, J. Choi, L. Wills, and S. Wills, “Real-time adaptive background modeling for multicore embedded systems,” Journal of Signal Processing Systems, vol. 62, no. 1, pp. 65-76, 2011.

[47] C.-C Lin, M.-H. Sheu, C. Liaw, and H.-K. Chiang, “Fast first-order polynomials convolution interpolation for real-time digital image reconstruction,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 20, no. 9, pp. 1260-1264, 2010.

[48] E. J. Delp and O. R. Mitchell, “Image compression using block truncation coding,” IEEE Trans. Commun. System., vol. 27, no. 9, pp. 1335–1342, 1979.

[49] L. Maddalena and A. Petrosino, “A self-organizing approach to background subtraction for visual surveillance applications,” IEEE Trans. Image Process., vol. 17, no. 7, pp. 1168-1177, 2008.

[50] R. Munoz-Salinas, E. Aguirre, M. Garciasilvente, and A. Gonzalez, “A multiple object tracking approach that combines colour and depth information using a confidence measure,” Pattern Recognition Letters, vol. 29, no. 10, pp. 1504-1514, 2008.

[51] K. Hariharakrishnan, and D. Schonfeld, “Fast object tracking using adaptive block matching,” IEEE Trans. on Multimedia, vol. 7, no. 5, pp. 853-859, 2005.

[52] S.H. Khatoonabadi, and M. Rahmati, “Automatic soccer players tracking in goal scenes by camera motion elimination,” Image and Vision Computing, vol. 27, pp. 463-479, 2009.

[53] H.-Y. Cheng, and J.-N Hwang, “Adaptive particle sampling and adaptive appearance for multiple video object tracking,” Signal Processing, vol. 89, pp. 1844-1849, 2009.

[54] N. Paragios and R. Deriche, ”Geodesic active contours and level sets for the detection and tracking of moving objects,” IEEE trans.Pattern Anal.Mach. Intell. vol. 22, no. 3, pp. 266-280, 2000.

[55] Y. Cheng, “Mean shift, mode seeking, and clustering,” IEEE Tran. On Pattern Analysis and Machine Intelligence, vol. 17, Issue 8, pp. 790-799, 1995.

[56] D. Comaniciu, V. Ramesh and P. Meer, “Real-Tine Tracking of Non-Rigid Objects Using Mean Shift,” Computer Vision and Pattern Recognition, pp.142-149, 2000.

[57] Z. H. Khan, and Irene Y.-H. Gu, “Joint feature correspondences and appearance similarity for robust visual object tracking,” IEEE Trans. on Information Forensics and Security, vol. 5, no. 3,pp.591-606, 2010.

[58] D.-T. Lin, and K.-Y. Huang,“ Collaborative pedestrian tracking and data fusion with multiple cameras,” IEEE Trans. on Information Forensics and Security, vol. 6, no. 4, pp. 1432-1444, 2010.

[59] J. S. Kim, and D. H. Yeom, and Y. H. Joo,“ Fast and robust algorithm of tracking multiple moving objects for intelligent video surveillance systems,” IEEE Trans. on Consumer Electronics, vol. 57, no. 3, pp.1165-1170, 2011.

[60] Y. Tian, R. Feris, H. Liu, A. Hampapur and M. Sun,“ Robust detection of abandoned and removed objects in complex surveillance videos,” IEEE Trans. Systems, Man, and Cybernetics-Part C: Applications and Reviews, vol. 41, no. 5, pp. 565-576, 2011.