臺灣博碩士論文加值系統

隨著智慧型家電及手機產品的普及，許多前瞻影像處理應用需求因應而生，本論文提出了多項演算法及各種實現技術包含了具錯誤容忍的人機控制介面系統、可重組化之OpenVG二維繪圖引擎、以OpenCL實現之自動對比增強、及多項立體視覺3D處理技術，各項開發之演算法須能應用於實際產品中，故皆要求複雜度為硬體可實現以及能適用於各類可能遇到的應用情境及圖庫。
所提出的人機介面系統具有軟體即時處理及抗干擾等特性，其手勢辨識準確度高達95%。在實現的OpenVG二維繪圖引擎中藉由可重組化概念達到低成本又可支援高解析度的目的，使用到的軟硬體分割技術亦大幅提升整體效能。對比增強演算法將統計圖分類並依其趨勢自動偵測適當的調整強度，此外亦考慮了多項副作用抑制技術，經由大量測試圖庫的分析，其表現可與現代高階繪圖軟體之自動影像增強技術相抗衡，最後在異質運算系統中透過OpenCL實現此演算法。立體三維視覺處理包含了二維轉三維影像處理及立體影像匹配（stereo matching）兩部分，二維轉三維技術考慮了二維圖片場景中的消逝線與移動物件等線索決定景深，消逝線演算法使用機械訓練機制，基於簡單的特徵即可找出在複雜場景的消逝線，移動物件之資訊，藉由具內容感知特性的影像切割演算法配合簡單的動量運算以估測前背景區間；立體影像匹配在配對過程中萃取景深可靠度資訊並將正確的景深擴張至可靠度較低之區間，其演算法以低系統頻寬消耗的掃描線記憶體合併技術實現，硬體實現的面積效率為文獻的11倍多。

With the increasing usage of smart mobiles and home appliances, advanced video processing requirements for new applications have become more critical. This thesis proposed several algorithms and design methodologies including a robust and error tolerable human control interface, scalable OpenVG 2D graphic engine, auto contrast enhancement implement using OpenCL, and multiple 3D stereo processing techniques. The algorithms are developed for physical product, thus the algorithms are hardware realizable and widely adoptable for most application environment and image categories.
The proposed human control interface possess anti-interference property and achieves 95% accuracy rate of posture recognition. The developed OpenVG system supports high resolution applications and consumes limited cost considering scalable technique. Furthermore, the introducing of system HW/SW partitioning also raising the overall performance significantly. As for the contrast enhancement algorithm realized in a heterogeneous platform using OpenCL, an auto enhancement level is extracted from the image histogram. Moreover, a plenty of side effect suppression methods are presented. The 2D to 3D conversion and the 3D stereo matching are the most major applications for 3D stereo processing. The vanishing line and motion of segment cues are considered for depth estimation in 2D scene for 2D to 3D conversion. In the 3D stereo matching system, reliable depth is extracted and propagate into unreliable region. A low bandwidth consumption scan-line merged architecture is proposed to realize the algorithm cost effectively. The area efficiency is 11 times higher than previous studies.

1. INTRODUCTION 1
1.1 MOTIVATION 2
1.2 INTRODUCTION 3
1.2.1 OpenVG 2D graphic 3
1.2.2 3D stereo processing 4
1.2.3 Color enhancement 6
1.2.4 Human control interface (HCI) 8
1.3 ORGANIZATION OF THE DISSERTATION 10
2. VIDEO SOC APPLICATIONS REVIEW 11
2.1 OPENVG GRAPHIC 11
2.2 STEREO 3D PROCESSING 12
2.3 CONTRAST ENHANCEMENT 15
2.4 GESTURE RECOGNITION 18
3. SCALABLE OPENVG 2D GRAPHIC ENGINE 20
3.1 EFFICIENT PIPE-LINE BASE OPENVG ARCHITECTURE 21
3.2 LOW COMPLEXITY TESSELLATION UNIT DESIGN 24
3.3 SCALABLE AND RECONFIGURABLE RASTERIZATION UNIT WITH BONDING BOX 27
3.4 BOUNDARY EDGE TEST ALGORITHM 29
3.5 HIGHLY RESOURCE SHARED POST-PROCESSING PROCESSOR 31
3.6 EMBEDDED SYSTEM REALIZATION 33
4 3D STEREO PROCESSING 35
4.1 DEPTH ESTIMATION FOR 2D TO 3D CONVERSION 36
4.1.1 Machine Learning Based Vanishing Line Cue 36
4.1.2 Motion of Segment Cue 42
4.1.3 Content aware image segmentation 47
4.2 LOW COMPLEXITY STEREO MATCHING 53
4.2.1 Depth Reliability Based Algorithm 53
4.2.2 Area-efficient memory-merging line buffer architecture 65
5 AUTO AND SIDE EFFECT SUPPRESSED IMAGE ENHANCEMENT 77
5.1 AUTO CONTRAST ENHANCEMENT WITH HISTOGRAM CLASSIFIER 78
5.2 SIDE EFFECT REDUCTION 86
5.3 SATURATION ENHANCEMENT 91
5.4 EXPERIMENTAL RESULTS 96
6 ROBUST HUMAN CONTROL INTERFACE 103
6.1 FACE RECOGNITION BASED ON CASCADE CLASSIFIER 104
6.2 GESTURE RECOGNITION SYSTEM 110
6.2.1 Candidate Node Detection Algorithm with Reliability Information 110
6.2.2 Hierarchical Posture Recognition 116
6.2.3 Application Specific Error Tolerable Multiple Hand Tracking 122
7 CONCLUSIONS AND FUTURE WORK 129
7.2 CONCLUSIONS 129
7.3 FUTURE WORK 131
BIBLIOGRAPHY 132

[1]OpenVG 1.1 specification, Khronos Group Inc., Dec. 3, 2008.
[2]T. H. Cormen, C. E. Leiserson, and R. L. Rivest, “Introduction to Algorithms, New York: McGraw-Hill, 1990.
[3]Y. Y. Boykov and M. Jolly, “Interactive graph cuts for optimal boundary ＆ region segmentation of objects in N-D images, in Proc. Int. Conf. Computer Vision, July 2001, pp. 105-112.
[4]J. Sun, N. N. Zheng, and H. Y. Shum, “Stereo matching using belief propagation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, pp. 787-800, July 2003.
[5]D. Scharstein, R. Szeliski, and R. Zabih, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, Int. J. Computer Vision, vol. 47, no. 1, pp. 7-42, Apr. 2002.
[6]O. Veksler, “Stereo correspondence with compact windows via minimum ratio cycle, IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 12, pp. 1654-1660, Dec. 2002.
[7]R. K. Gupta and S. Y. Cho, “Real-time stereo matching using adaptive binary window, in Proc. Int. Symposium 3D Data Processing Visualization and Transmission, vol. 1, 2010, pp. 1-8.
[8]N. C. Chang , T. H. Tsai , B. H. Hsu , Y. C. Chen and T. S. Chang, Algorithm and architecture of disparity estimation with mini-census adaptive support weight, IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 792-805, June 2010.
[9]K. J. Yoon and I. S. Kweon, “Adaptive support-weight approach for correspondence search, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, pp. 650-656, Apr. 2006.
[10][Y. Qu, J. Jiang, X. Deng, and Y. Zheng, Robust local stereo matching under varying radiometric conditions, IET Computer Vision, vol. 8, no. 4, pp. 263-276, Aug. 2014.
[11]L. De-Maeztu, A. Villanueva, and R. Cabeza, Near Real-Time Stereo Matching Using Geodesic Diffusion, IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 2, pp. 410-416, Feb. 2012.
[12]M. Baydoun, and M.A. Al-Alaoui, Enhancing Stereo Matching With Classification, IEEE Access, vol. 2, pp. 485-499, 2014.
[13]J. Joglekar, S.S. Gedam, and B.K. Mohan,Image Matching Using SIFT Features and Relaxation Labeling Technique-A Constraint Initializing Method for Dense Stereo Matching, IEEE Trans. on Geoscience and Remote Sensing, vol. 52, no. 9, pp. 5643-5652, Sept. 2014.
[14]M. Hariyama et al., “1000 frame-sec stereo matching VLSI processor with adaptive window-size control, in Proc. Conf. Asian Solid-State Circuits, Nov. 2006, pp. 123-126.
[15]M. Hariyama, “VLSI processor for reliable stereo matching based on window-parallel logic-in-memory architecture, in Proc. IEEE Conf. Symp. on VLSI Circuit Digest of Technical Paper, Jun. 2004, pp. 166-169.
[16]D. J. Jobson, Z.-U. Rahman, and G. A. Woodell, “Properties and performance of a center/surround retinex, IEEE Trans. Image Process., vol. 6, no. 3, pp. 451-462, Mar. 1997.
[17]D. J. Jobson, Z.-U. Rahman, and G. A. Woodell, “A multiscale retinex for bridging the gap between color images and the human observation of scenes, IEEE Trans. Image Process., vol. 6, no. 7, pp. 965-976, July 1997.
[18]A. P. Karen., J. W. Eric, and S. Agaian, “Human visual system-based image enhancement and logarithmic contrast measure, IEEE Trans. Syst., Man, and Cybernetics, vol. 38, no. 1, pp. 174-188, Feb. 2008.
[19]J. Mukherjee and S. K. Mitra, “Enhancement of color images by scaling the DCT coefficients, IEEE Trans. Image Process., vol. 17, no. 10, pp. 1783-1794, Oct. 2008.
[20]T. C. Jen and S. J. Wang, “Bayesian structure-preserving image contrast enhancement and its simplification, IEEE Trans. Image Process., vol. 22, 831-843, June 2012.
[21]A. Choudhury and G. Medioni, Perceptually motivated automatic color contrast enhancement in Proc. IEEE Int. Conf. Comp. Vis. Workshop. (ICCV Workshops) Sept. 2009, pp.1893-1900.
[22]T. Arici, S. Dikbas, and Y. Altunbasak, “A histogram modification framework and its application for image contrast enhancement, IEEE Trans. Image Process., vol. 18, no. 9, pp. 1921–1935, Sept. 2009.
[23]Y.T. Kim, “Contrast Enhancement Using Brightness Preserving Bi-Histogram Equalization, IEEE Trans. Consum. Electron., vol. 43, no. 1, pp. 1-8, Feb. 1997.
[24]Soong-Der Chen, and Ramli, A.R., “Minimum Mean Brightness Error Bi-Histogram Equalization in Contrast Enhancement, IEEE Trans. Consum. Electron., vol. 49, pp. 1310-1319, Nov. 2003.
[25]Yu Wan, Qian Chen and Bao-Min Zhang., “Image Enhancement Based On Equal Area Dualistic Sub-Image Histogram Equalization Method, IEEE Trans. Consum. Electron., vol. 45, no. 1, pp. 68-75, Feb. 1999.
[26]J. Suarez and R. R. Murphy, “Hand gesture recognition with depth images: a review, in Proc. IEEE Int. Symp. Robot and Human Interactive Commun., Sept. 2012, pp. 411-417.
[27]Z. Ren, J. Yuan, J. Meng, and Z. Zhang, “Robust part-based hand gesture recognition using Kinect sensor, IEEE Trans. Multimedia, vol. 15, no. 5 , pp. 1110-1120, Aug. 2013.
[28]M. Tang, Recognizing hand gestures with Microsoft's Kinect, Dept. Elect. Eng., Stanford Uni., Mar. 2011.
[29]O. Rashid, A. Al-Hamadi and B. Michaelis, “A framework for the integration of gesture and posture recognition using HMM and SVM, in Proc. IEEE Int. Conf. Intelligent Computing and Intelligent Syst., Nov. 2009, pp. 572-577.
[30]Y. Li, “Hand gesture recognition using Kinect, in Proc. IEEE Int. Conf. Software Engineering and Service Science, June 2012, pp. 196-199.
[31]V. Fati and D. Prattichizzo, “Using Kinect for hand tracking and rendering in wearable haptics, in Proc. IEEE World Haptics Conf., June 2011, pp. 317-321.
[32]D. Kim, K. Cha, S. Chae, “A high-performance OpenVG Accelerator with dual-scanline filling rendering, IEEE Trans. Consum. Electron., vol. 54, no. 3, pp. 1303-1311, Aug. 2008.
[33]R. Huang and S. Chae, “Implementation of an OpenVG rasterizaer with configurable anti-aliasing and multi-window scissoring, in Proc. IEEE int. Conf. Comput. Inf. Tech., Sept. 2006, pp. 179-184.
[34]Y. L. Shen, S. W. Seo, Y. Zhang, and H. C. Oh, “A low-hardware cost 2D vector graphic rendering algorithm for supersampling antialiasing, in Proc. IEEE Int. Workshop Edu. Tech. Comput. Sci., 2010, pp. 141-144.
[35]S.H. Chen, H.M. Lin, C.C. Hsieh, C.T. Huang, J.J. Liou, and Y.C. Chung, “TurboVG: A HW/SW co-designed multi-core OpenVG accelerator for vector graphics applications with embedded power profiler, Proc. ACM Asia and South Pacific Design Automation Conf., Mar. 2011, pp.97-98.
[36]H. Lee, N. Baek, I. Lee, J. Yoon, O. Pothier, “Accelerating OpenVG and SVG Tiny with Multimedia Hardware, in Proc. IEEE Conf. Consum. Electron., Jan. 2011, pp. 917-918.
[37]J. Foley, A. vanDam, S. Feiner, and J. Hughes, Computer Graphics: pricinple and practice, Addision-Wesley, 1990.
[38]K. Kallio, “Scanline edge-flag algorithm for antialiasing, in Proc. Conf. Theory Practice Comput. Graphics, June 2007, pp. 81-88.
[39]Y. Wan, Q. Chen, and B.M. Zhang., “Image Enhancement Based On Equal Area Dualistic Sub-Image Histogram Equalization Method, IEEE Trans. Consum. Electron., vol. 45, no. 1, pp. 68-75, Feb. 1999.
[40]X. Li, “Gesture recognition based on fuzzy c-means clustering algorithm, Dept. Comput. Science, The Uni. of Tennessee, Knoxville, 2003.
[41]J. P. Wachs , H. Stern and Y. Edan, “Cluster labeling and parameter estimation for the automated setup of a hand-gesture recognition system, IEEE Trans. Syst., Man, and Cybernetics, Part A: Syst. and Humans, vol. 35, no. 6, pp.932 -944, Nov. 2005.
[42]L. Gupta and S. Ma, Gesture-based interaction and communication: automated classification of hand gesture contours, IEEE Trans. Syst., Man, and Cybernetics, Part C: Applications and Reviews, vol. 31, no. 1, Feb. 2001.
[43]S. Y. Tsai, “A real-time hand gesture recognition system and its application in manipulating household appliances, M. S. Thesis, Comput. Science and Inf. Eng., National Central Uni., Taoyuan, Taiwan, June 2010.
[44]A. Yilmaz, O. Javed and M. Shah, Object tracking: a survey, ACM Computing Survveys, vol. 38, no. 4, pp. 1-45, Dec. 2006.
[45]S. Battiato, A. Capra, S. Curti, and L. M. Cascia, “3D stereoscopic image pairs by depth-map generation, in IEEE Int. Symp. on 3D Data Processing, Visualization and Transmission, Sept. 2004, pp. 124-131.
[46]T. Yi-Min, Y. L. Chang, and L. G. Chen, “Block-based vanishing line and vanishing point detection for 3D scene reconstruction, in IEEE Int. Symp. on Intelligent Signal Processing and Commun., Dec. 2006, pp. 586-589.
[47]F. Yu, J. Liu, Y. Ren, J. Sun, Y. Gao, and W. Liu, Depth generation method for 2D to 3D conversion, in IEEE Int. 3DTV Conference, May 2011, pp. 1-4.
[48]C. Ttofis, “Towards accurate hardware stereo correspondence: areal-time FPGA implementation of a segmentation-based adaptive support weight algorithm, in Proc. Design Automation and Test in Europe Conf. and Exhibition, Mar. 2012, pp. 703-708.
[49]N. Jacobson, “An online learning approach to occlusion boundary detection, IEEE Trans. Image Processing, vol. 21, no.1, pp. 252-261, Jan. 2012.
[50]L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, “High-quality real-time stereo using adaptive cost aggregation and dynamic programming, in Proc. Int. Symposium 3D Data Processing Visualization and Transmission, pp. 798-805, Jun. 2006.
[51]H. Hubert, B. Stabernack, and F. Zilly, Architecture of a low latency image rectification engine for stereoscopic 3-D HDTV processing, IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 5, pp. 813-822, May 2013.
[52]Middlebury, “Middlebury Stereo Datasets, http://vision.middlebury.edu/stereo/data/, May 2014.
[53]A. Darabiha, W. MacLean, and J. Rose, “Reconfigurable hardware implementation of a phase-correlation stereo algorithm, Machine Vision and Applications, vol. 17, no.2, 116-132, Apr. 2006.
[54]M. Werner, B. Stabernack, and C. Riechert,Hardware implementation of a full HD real-time disparity estimation algorithm, IEEE Trans. Consum. Electron., vol. 60, no. 1, pp. 66-73, Feb. 2014.
[55]G. Song and X. L. Qiao, “Adaptive color image enhancement based on human visual properties, in Proc. IEEE Conf. Ind. Electron. Appl., June 2008, pp. 1892-1895.
[56]C. Gao and K. Panetta, “A new color contrast enhancement algorithm for robotic applications, in Proc. IEEE Int. Conf. Tech. for Practical Robot Appl., Apr. 2012, pp. 42-47.
[57]Wikipedia, “HSL and HSV, http://en.wikipedia.org/wiki/, Oct. 3 2012.
[58]D. H. Choi , I. H. Jang , M. H. Kim, and N. C. Kim, “Color image enhancement based on single-scale retinex with a JND-based nonlinear filter, in Proc. IEEE Int. Symp. Circuits Syst., May 2007, pp. 3948-3951.
[59]C. H. Lee, J. S. Kim, and K. H. Park, Automatic human face location in a complex background using motion and color information, Pattern Recognition, vol. 29, no. 11, pp. 1877-1889, 1996.
[60]R. Frischholz, “Bao face database at the face detection homepage, 2008. [Online]. Available: http://www.facedetection.com/downloads/- BaoDataBase.zip
[61]C. R. Chen, W. S. Wong, and C. T. Chiu, ‘A 0.64 mm2 real-time cascade face detection design based on reduces two-field extraction, IEEE Tran. VLSI, vol. 19, no. 11, pp. 1937-1948, Nov. 2001.
[62]Y. Hanai, Y. Hori, J. Nishimura, and T. Kuroda, “A versatile recognition processor employing Haar-like feature and cascaded classifier, in Proc. Int. Conf. Solid-State Circuit, Feb. 2009, pp. 148-149.
[63]C. Kyrkou and T. Theocharides, “A flexible parallel hardware architecture for AdaBoost-based real-time object detection, IEEE Tran. VLSI System, vol. 19, no. 6, pp. 1034-1047, May 2011.
[64]K. Imagawa, K. Iwasa, T. Kataoka, T. Nishi, and H. Matsuo, “Realtime face detection with MPEG4 codec LSI for a mobile multimedia terminal, in Proc. IEEE Int. Conf. Consum. Electron., June 2003, pp. 16-17.
[65]Y. Hori, Y. Hanai, and T. Kuroda, “A 0.79-mm2-mW real-time face detection core, in Proc. IP-SOC Conf. Electronic Sysem and Exhibition, Apr. 2007, pp. 790-797.
[66]K. Korekado, T. Morie, O. Nomura, T. Nakano, M. Matsugu, and A. Iwata, “An image filtering processor for face/object recognition using merged/mixed analog digital architecture, in Proc. IEEE Int. Symp. VLSI Circuits Digest of Technical Papers, June 2005, pp. 220-223.
[67]T. Theocharides, G. Link, N. Vijaykrishnan, M. J. Irwin, and W.Wolf, “Embedded hardware face detection, in Proc. IEEE Int. Conf. VLSI Design, 2004, pp. 133-138.
[68]Z. Ren, J.S. Yuan, and Z.Y. Zhang, Hand Gesture Recognition, http://eeeweba.ntu.edu.sg/computervision/people/home/renzhou/HandGesture.htm, May 2014.