影像監視系統可以得到最直接的視覺監控效果。同時運用多支攝影機拍攝全面性的視覺場景也能用在許多地方，例如：銀行、交通路口。在影像監視系統中，可以分為前端的物件偵測與特徵萃取以及後端的物件分類與追蹤。本論文是針對前端部份進行探討，研究的主題包含多攝影機場景之影像拼接方法以及影像監控系統之前景物件偵測，最後運用影像標籤化技術取得每一獨立物件的特徵資訊。

在影像拼接中，我們從兩支攝影機的重疊區域中，找出色差較小的接縫線。針對靜態場景時，接縫線的路徑會避開場景中的水平邊緣，藉此消除水平對準的誤差。同時，在動態場景中，可變接縫線能繞過移動的物體，獲得鬼影消除的效果。為了檢測接縫線的品質，我們利用接縫相似度進行比較。在相同的測試影像中，我們的接縫相似度可以達到10%~23% 改善。

在前景物件偵測方面，我們以區塊式背景相減技術為基礎，將影像分成不重疊的影像區塊，並且分析每一影像區塊的紋理資訊。利用影像區塊的紋理變化，建立每一影像區塊的動態背景。首先，我們使用離散餘弦轉換係數建立背景模型。因為人眼對於低頻成份較為敏感，所以我們使用低頻的係數進行背景模型建立與前景物件偵測。利用影像區塊的低頻係數建立的動態背景，可以節省39.66% 的記憶體使用量。然而，離散餘弦轉換需要較高的計算量，我們進而提出簡單的區塊紋理轉換。在這個轉換中，我們只需要加法與減法運算，即可獲得亮度、垂直、水平與對角的四種區塊紋理。透過區塊紋理計算的方式，使我們建立動態背景的記憶體節省23.97% 。在嵌入式平台實現，每秒可以執行20張畫面，在執行速度方面獲得17.64% 改善的幅度。

在物件特徵萃取方面，利用影像標籤化技術，取得每一獨立物件的幾何特徵。我們使用單一回合標籤化方法，同時提出標籤再利用的概念，降低標籤使用量。利用這種方法，可以大幅降低記憶體的大小，同時提昇特徵萃取處理的速度。實驗結果顯示，我們的記憶體可以減少37%，而且執行速度可以達到41% 改善。

Image surveillance systems are used to perform direct visual monitoring. Moreover, multiple cameras are used to acquire complete visual coverage of locations such as banks and traffic intersections. Image surveillance systems can be divided into front-end processes for object detection and feature extraction, and back-end processes for object classification and tracking. This dissertation focuses on the front-end processes. The topics of this study are the development of a ghost-free video stitching method for multiple-camera environments, and the foreground object detection for image surveillance systems. An image labeling technique was applied to acquire feature data on foreground objects.

By using an image stitching method, we identified a variable seam line based on minimal color differences in the overlapping region of images captured by two cameras. By stitching a static scene, we located a seam line to bypass the horizontal edge, thus removing alignment artifacts. Furthermore, in dynamic scenes, the variable seam line removes moving objects to obtain a ghost-free panoramic video image. To evaluate the quality of a seam, we present a seam similarity function for evaluating color difference of a neighboring seam. In the same test sequence, our seam similarity method achieved an improvement of 10%–23%.

Regarding the foreground object detection, we used a block-based background subtraction technique. Initially, the input image is partitioned into non-overlapping image blocks. Subsequently, each image block is analyzed to obtain its textural information. According to the textural variation in each image block, we constructed a dynamic background model. First, we used discrete cosine transform (DCT) coefficients to construct the background model. Because human eyes are sensitive to low-frequency components, the low-order DCT coefficients of each block were used to construct a background model and execute foreground block detection. The block-based method can markedly reduce the required memory when constructing the background model in dynamic scenes. The experimental results showed that the proposed method uses at least 39.66% less memory than conventional method. However, calculating DCT coefficients increases the computational complexity. Therefore, we represent each block as a block texture. The block texture transform requires only addition and subtraction operations. The mean intensity and textural information (vertical, horizontal, and diagonal textures) can be obtained easily. When using block texture representation to construct the background model, the memory consumption can be reduced by 23.97%. The proposed approach was implemented using an embedded system platform. The proposed method can be applied to perform real-time operations at processing speeds of at least 20 fps, representing a 17.64% improvement.

Regarding the object feature extraction, we used an image labeling technique to obtain the feature data for each object. A single-pass labeling method was employed and a label reuse approach is presented to minimize the number of labels. This approach can efficiently reduce the memory consumption and improve the processing speed. The experimental results showed that the memory consumption is decreased by 37% and the execution time improved by 41%.

中文摘要….……………………………………………………………………………i

Abstract…………………………………………………………………………….…ii

Contents…………………………………………………………………………..…iv

List of Tables……………………………………………………………………..….vi

List of Figures…………………………………………………………………..……vii

1. Introduction…………………………………………………………………...…….1

1.1 Background and motivation…………………………………………………….1

1.2 Design objective………………………………………………………………...3

1.3 Dissertation organization ………………………………………………………4

2. Related Works……………………………………………………………………….5

2.1 Image stitching for panoramic images…………………………………………5

2.2 Foreground object detection……………………………………………………6

2.2.1 Pixel-based probability background extraction algorithm………………..8

2.2.2 Pixel-based ALW Method……………………….....…….………………10

2.2.3 Block-based background model from compressed video…..……………12

2.2.4 Block-based classifier cascade with probabilistic decision integration…13

2.2.5 Block-based major color background model……………….……………14

2.3 Image labeling and feature extraction ………………………………………16

2.3.1 Multi-pass algorithm……...………………………………………………18

2.3.2 Two-pass algorithm…….…………………………………………………18

2.3.3 Single-pass algorithm……………………………………………….……19

2.3.4 Summary…………………………………………………………….……21

3. Panoramic Image Stitching by Using a Variable Seam…………………………23

3.1 Variable seam concept…………………………………………………………23

3.2 Process flow of the variable seam line method………………………………24

3.3 Experimental results and comparison…………………………………………29

3.4 Summary…………………………….…………………………………………33

4. Block-based Background Modeling and Foreground Object Detection………….34

4.1 Foreground object detection method in the DCT domain…………………… 34

4.1.1 The DCT coefficient difference and block descriptor……………………..36

4.1.2 Block-based dynamic background model construction……………………39

4.1.3 Foreground block classification and background model updating…………39

4.1.4 Object shape refinement………………………………………………...…40

4.1.5 Experimental results and comparison……………………………………40

4.1.6 Summary…………………………………………………………………43

4.2 Foreground object detection method in block-texture representation………44

4.2.1 Block texture transformation………………………………………………45

4.2.2 Block texture background modeling………………………………………46

4.2.3 Foreground block detection and background block updating……………48

4.2.4 Boundary block detection and pixel-level refinement……………………48

4.2.5 Experimental results and comparison……………………………….……50

4.2.6 Threshold value setting and block size selection………………….……55

4.2.7 Summary……………………………….…………………………………57

5. Image Labeling and Feature Extraction…………………………………………..59

5.1 Single-pass labeling by reusing labels…………...……………………………59

5.1.1 The concept of label reuse……………………………………………….59

5.1.2 Table manipulation for label reuse………………………………………60

5.1.3 Processing flow of single-pass labeling and feature extraction…………63

5.1.4 Experimental results and comparison……………………………………65

5.2 Two-pass multi-pixel parallel labeling………………………………………66

5.2.1 Multi-pixel parallel labeling……………………………………………...66

5.2.2 MPL hardware architecture……………………………………………….73

5.2.3 MPL intellectual property design and implementation………………….77

6. Conclusion………………………………………………………………………..87

Reference…………………………………………………………………………….89

[1] C. Stauffer, and W. E. L. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 246–252, June 1999.



[2] A. Cavallaro, O. Steiger, and T. Ebrahimi, “Tracking Video Objects in Cluttered Background,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 15, no. 4, pp. 575–584, 2005.



[3] H. Wang, and D. Suter, “A Consensus-Based Method for Tracking: Modeling Background Scenario and Foreground Appearance,” Pattern Recognition, vol. 40, no. 3, pp. 1091–1105, 2006.



[4] W. Lao, J. Han, and P. H. N. de With, “Automatic Video-Based Human Motion Analyzer for Consumer Surveillance System,” IEEE Transactions on Consumer Electronics, vol. 55, no. 2, pp. 591–598, 2009.



[5] C. F. Juang, C. M. Chang, J. R. Wu, and D. Lee, “Computer Vision-based Human Body Segmentation and Posture Estimation,” IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 39, no. 1, pp. 119–133, 2009.



[6] J. W. Hsieh, Y. T. Hsu, H.-Y. M. Liao, and C. C. Chen, “Video-based Human Movement Analysis and Its Application to Surveillance Systems,” IEEE Transactions on Multimedia, vol. 10, no. 3, pp. 372–384, 2008.



[7] C. H. Chuang, J. W. Hsien, L. W. Tsai, S. Y. Chen, and K. C. Fan, “Carried Object Detection Using Ratio Histogram and Its Application to Suspicious Event Analysis,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 19, no. 6, pp. 911–916, 2009.



[8] S. Gupte, O. Masoud, R. F. K. Martin, and N. P. Papanikolopoulos, “Detection and Classification of Vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 1, pp. 37–47, 2002.



[9] S. C. Chen, M. L. Shyu, S. Peeta, and C. C. Zhang, “Learning-Based Spatio-Temporal Vehicle Tracking and Indexing for Transportation Multimedia Database Systems,” IEEE Transactions on Intelligent Transportation Systems, vol. 4, no. 3, pp. 154–167, 2003.



[10] J. W. Hsieh, S. H. Yu, Y. S. Chen, and W. F. Hu, “Automatic Traffic Surveillance System for Vehicle Tracking and Classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 2, pp. 175–187, 2006.



[11] W. Zhang, Q. M. Jonathan Wu, and H. B. Yin, “Moving Vehicles Detection Based On Adaptive Motion Histogram,” Digital Signal Processing, vol. 20, no. 3, pp. 793–805, 2010.



[12] N. Buch, J. Orwell, and S. A. Velastin, “Detection and Classification of Vehicles for Urban Traffic Scenes,” In Proceedings of 5th International Conference on Visual Information Engineering, pp. 182–187, 29 July - 01August 2008.



[13] S. Y. Chien, S. Y. Ma, and L. G. Chen, “Efficient Moving Object Segmentation Algorithm using Background Registration Technique,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 7, pp. 577–586. 2002.



[14] C. C. Chiu, M. Y. Ku, and L. W. Liang, “A Robust Object Segmentation System Using A Probability-Based Background Extraction Algorithm,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 4, pp. 518-528, 2010.



[15] W. K. Tsai, M. H. Sheu, and C. C. Lin, “Block-based Major Color method for foreground object detection on Embedded SoC platforms,” IEEE Embedded Systems Letters, vol. 4, no. 2, pp. 49-52, 2012.



[16] W. Wang, J. Yang, and W. Gao, “Modeling background and segmenting moving objects form compressed video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 18, no. 5, pp. 670-681, 2008.



[17] V. Reddy, C. Sanderson, and B. C. Lovell, “Improved Foreground Detection via Block-based Classifier Cascade with Probabilistic Decision Integration, IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 1, pp. 83-93, 2013.



[18] C. Kim, and J. N. Hwang, “Object-Based Video Abstraction for Video Surveillance Systems,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 12, pp. 1128-1138, 2002. doi:10.1109/TCSVT.2002.806813



[19] C. H. Moon, D. Y. Jang, and J. N. Choi. Huang, L., M. Loog, “An SoC System for Real-Time Moving Object Detection,” In Advanced Intelligent Computing Theories and Applications-With Aspects of Theoretical and Methodological Issues. Lecture Notes in Computer Science, vol. 4681, pp. 879-888, 2007.



[20] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-Time Foreground-Background Segmentation Using Codebook Model,” Real-time Imaging, vol. 11, no. 3, pp. 172-185, 2005.



[21] C. Stauffer, and W. E. L. Grimson, “Learning Patterns of Activity Using Real-Time Tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747-757, 2000.



[22] J. M. Guo, Y.-F. Liu, C. H. Hsia, M. H. Shih, and C. S. Hsu. 2011. “Hierarchical Method for Foreground Detection Using Codebook Model.” IEEE Transactions on Circuits and Systems for Video Technology, vol. 21, no. 6, pp. 804-815., 2011.



[23] M. Heikkila, and M. Pietikainen, “A Texture-Based Method for Modeling the Background and Detecting Moving Objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 657-662, 2006.



[24] G. Dalley, J. Migdal, and W. E. L. Grimson, “Background Subtraction for Temporally Irregular Dynamic Textures,” In Proceedings of IEEE Workshop on Applications of Computer Vision, pp. 1–7, Copper Mountain, CO, USA, 7-9 January 2008.



[25] A. B. Chan, V. Mahadevan, and N. Vasconcelos, “Generalized Stauffer-Grimson Background Subtraction for Dynamic Scenes,” Machine Vision and Applications, vol. 22, no. 5, pp. 751–766, 2011.



[26] Y. Wang, S. Velipasalar, and M. Casares, “Cooperative Object Tracking and Composite Event Detection With Wireless Embedded Smart Cameras,” IEEE Transactions on Image Processing, vol. 19, no. 10, pp. 2614-2633, 2010.



[27] C. Y. Cho, J. K. Yang, S. Y. Tseng, I. H. Lee, and M. H. Sheu, “Real-Time Object Detection System on PAC Duo Platform,” In Proceedings of International Conference on Signal Processing Systems, pp. 585-590, July 2010.



[28] O. Barnich, and M. Van Droogenbroeck, “ViBe: A Universal Background Subtraction Algorithm for Video Sequences,” IEEE Transactions on Image Processing, vol. 20, no. 6, pp. 1709–1724, 2011.



[29] M. Casares, S. Velipasalar, A. Pinto, “Light-weight salient foreground detection for embedded smart cameras,” Computer vision and Image Understanding, vol. 114, no. 11, pp. 1223-1237, 2010.



[30] B. Valentine, S. Apewokin, L. Wills, and S. Wills, “An Efficient, Chromatic Clustering-Based Background Model for Embedded Vision Platforms,” Computer Vision and Image Understanding, vol. 114, no. 11, pp. 1152–1163, 2010.



[31] H. Bay, A. Ess, T. Tuytelaars, and L.J. Van Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Process., vol. 110, no. 3, pp. 346-359, Jun. 2008.



[32] M. Brown and D.G. Lowe, “Automatic panoramic image stitching using invariant features,” Int. J. Comput. Vis., vol. 74, no. 1, pp. 59-73, Aug. 2007.



[33] D. G. Lowe, “Distinctive image features from scale-invariant key-points,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.



[34] M. Bartoli, N. Dalal, B. Bose, and R. Horaud, “From video sequence to motion panoramas,” in Proc. Workshop Motion Video Comput., pp.201-207, 2002.



[35] H. S. Sawhny and S. Ayer, “Compact representations of videos through dominant and multiple motion estimation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 8, pp. 814-830, Aug. 1996.



[36] R.A. Alex, Y. Pritch, D. Lischinski, and S. Peleg, “Dynamosaicing: Mosaicing of dynamic scenes,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1789-1801, Oct. 2007.



[37] S. Ha, H. Koo, S. Lee, N. Cho, and S. Kim, “Panorama mosaic optimization for mobile camera systems,” IEEE Trans. on Consumer Electronics, vol. 53, no. 4, pp. 1217-1225, Nov. 2007.



[38] S. Ha, S. Lee, N. Cho, S. Kim, and B. Son, “Embedded panorama mosaic system using auto-shot interface,” IEEE Trans. on Consumer Electronics, vol. 54, no. 1, pp. 16-24, Feb. 2008.



[39] Y. Xiong and K. Pulli, “Fast panorama stitching for high-quality panoramic images on mobile phones,” IEEE Trans. on Consumer Electronics, vol. 56, no. 2, pp. 298-306, May 2010.



[40] R.M. Haralick, “Some neighborhood operations,” in real time/parallel computing image analysis, Plenum Press, New York, pp.11-35, 1981.



[41] A. Hashizume et al., “An algorithm of automated RBC classification and its evaluatioin,” Bio. Med. Eng., vol. 28, no. 1, pp 25-32, 1990.



[42] A. Rosenfeld and J. L. Pfaltz, “Sequential operation in digital picture processing,” Journal of Association for Computing Machinery, vol. 13, pp. 471-494, 1966.



[43] L. D. Stefano and A. Bulgarelli, “A Simple and Efficient Connected Components Labeling Algorithm,” Int’l conf. Image analysis and processing, 1999.



[44] V. Khanna, P. Gupta, and C. J. Hwang, “Finding Connected components in digital images,” Int’l Conf. Information Technology, pp. 652-656, 2001.



[45] C. T. Johnston and D. G. Bailey: ‘FPGA implementation of a Single Pass Connected Components Algorithm’. IEEE Int. Symp. on Electronic Design, Test and Applications, Hong Kong, pp. 228–231, Jan. 2008.



[46] N. Ma, D. G. Bailey, and C. T. Johnston, “Optimised Single Pass Connected Components Analysis,” Int’l Conf. on Field-Programmable Technology, Taipei, pp. 185–192, Dec. 2008.



[47] Thornberg et al., “Real-time Component Labelling and Feature Extraction on FPGA,” Int’l Symp. on Signals, Circuits and Systems, ISSCS, 1-4, 2009.



[48] M. A. Najjar, S. Karlapudi, and M. A. Bayoumi, “Memory-Efficient Architecture for Hysteresis Thresholding and Object Feature Extraction,” IEEE Trans. on Image Processing, vol. 20, no. 12, pp. 3566-3579, Dec. 2011.