臺灣博碩士論文加值系統

　　本論文主要目的在研製一個嵌入式影像處理卡，未來能將其應用在機器人上進行特徵擷取及各種的影像處理演算法。本影像處理卡之電路設計可以分成數位訊號處理器(DSP)及可程式邏輯閘陣列(FPGA)兩個模組。影像擷取採用低功耗及小尺寸的CMOS影像感測器，其輸出格式設定為YUV4:2:2，擷取的影像尺寸則是512x480像素。DSP採用德州儀器公司的TMS320C6414TGLZ7，主要用於實現影像處理演算法；FPGA採用 Altera公司的EP2C35F672C6 ，用於比較低階的影像處理及CMOS影像感測器控制。本論文成功地於所研製的影像處理卡中實現加速強健特徵(SURF)演算法，透過USB介面將所擷取的影像畫面及SURF描述子傳送到個人電腦顯示及進行特徵匹配。實驗結果顯示，即時影像擷取週期大約33.5ms；當SURF描述子數量等於64個，計算時間大約417.5ms；當SURF描述子數量為199個，計算時間大約是985ms；依樣本模型在特徵擷取時設定的門檻值不同，特徵匹配率則由70.2%變化到20%。

This thesis aims to develop an embedded image processing board, which can be used to realize various image feature extraction and processing algorithms for robotic applications. The circuitry of this image board can be divided into two parts, one is for a digital signal processor (DSP) and the other for a field programmable gate array (FPGA). A low power, small size CMOS image sensor with YUV4:2:2 image format and 512x480 pixel size is adopted for image acquisition. The DSP module is mainly used for implementing image processing algorithms, while the FPGA module for lower level image processing and sensor control functions. In this work, the TMS320C6414TGLZ7 DSP from Texas Instruments Corporation and the EP2C35F672C6 FPGA from Altera Corporation were selected for the image board. In this work, speeded-up robust features (SURF) algorithm has been successfully realized on the developed image board. An USB interface with average throughput of 22MB/sec works to transfer acquired image frames and extracted SURF descriptors to a personal computer, where feature matching is executed. Experimental results show that the refresh period of frame grabbing is about 33.5ms. When the number of SURF descriptor is equal to 64, the computing time is about 417.5ms; when the number of SURF descriptor is equal to 199, the computing time is about 985ms. The typical matching rate varies from 70.2% to 20%, depending on the threshold of the patterns used in feature extraction.

中文摘要 ...................................................................................................... i
ABSTRACT................................................................................................. ii
誌　謝......................................................................................................... iii
目　錄 ........................................................................................................ iv
圖目錄 ....................................................................................................... vii
表目錄 ......................................................................................................... x

第一章 緒論.................................................................................... 1
1.1 研究動機................................................................................................ 2
1.2 文獻回顧................................................................................................ 3
1.2.1 低價的嵌入式彩色視覺系統[2]........................................................ 3
1.2.2 以FPGA實作適用於嵌入式視覺系統的影像處理架構[3]................... 3
1.2.3 用於移動式機器人的即時手勢辨識系統[4]....................................... 4
1.2.4 使用FPGA與CMOS影像感測器的影像系統[5]............................... 6
1.2.5 用於微型自主移動式機器人的即時影像辨識系統[6]......................... 7
1.2.6 基於DSP的嵌入式CMOS影像處理系統[7]..................................... 7
1.3 問題描述與論文架構.............................................................................. 8

第二章 嵌入式影像處理系統硬體電路設計....................................... 9
2.1 系統方塊圖............................................................................................ 9
2.2 FPGA模組............................................................................................ 11
2.2.1 FPGA配置方式............................................................................. 11
2.2.2 CMOS影像感測器介面.................................................................. 12
2.2.3 FPGA之記憶體介面...................................................................... 14
2.2.4 晶體震盪器(Oscillator)................................................................. 15
2.2.5 I/O擴充用介面............................................................................. 16
2.3 數位訊號處理器(DSP)模組 .................................................................. 16
2.3.1 外部記憶體介面(External Memory Interface, EMIF)....................... 16
2.3.2 記憶體映射圖(Memory Map)及介面訊號........................................ 17
2.3.3 內部周邊配置............................................................................... 19
2.3.4 開機模式..................................................................................... 19
a) 無開機模式.................................................................................. 19
b) ROM開機模式.............................................................................. 20
2.3.5 外部裝置的通訊介面.................................................................... 21
a) 高速USB 介面.............................................................................. 21
b) RS232介面.................................................................................. 22
2.3.6 GPIO與中斷訊號.......................................................................... 23
2.4 電源模組............................................................................................. 23
2.5 設計總結............................................................................................. 24

第三章 SURF演算法簡介............................................................ 25
3.1 積分影像.............................................................................................. 25
3.2 Fast-Hessian偵測器.............................................................................. 26
3.2.1 近似的Hessian行列式.................................................................. 26
3.2.2 尺度空間表示............................................................................... 28
3.2.3 興趣點定位.................................................................................. 29
3.3 興趣點描述子....................................................................................... 30
3.3.1 可重現的主方位........................................................................... 30
3.3.2 基於Haar小波響應和的描述子..................................................... 31
3.4 快速匹配用的旗標................................................................................ 31
3.5 本章總結............................................................................................. 32

第四章 嵌入式影像處理系統之應用.............................................. 33
4.1 SURF演算法實作系統簡介..................................................................... 33
4.1.1 FPGA內部架構.............................................................................. 33
4.1.2 DSP程式架構................................................................................ 35
4.1.3 人機介面軟體................................................................................ 36
4.2 SURF演算法相關參數定義.................................................................... 36
4.2.1 以Octave為基準的相關參數值....................................................... 37
4.2.2 以箱形濾波器大小為基準的相關參數值........................................... 37
4.3 Avalon介面.......................................................................................... 39
4.4 不同時脈領域之間的資料傳遞................................................................ 41
4.5 積分影像功能實作.................................................................................. 43
4.6 如何建立DSP的Boot ROM................................................................... 45
4.6.1 C6414T的ROM 開機程序.............................................................. 45
4.6.2 產生燒錄用的Hex檔...................................................................... 45
4.6.3 ROM的燒錄程式........................................................................... 47
a) Hex檔解析.................................................................................... 47
b) 燒錄函式....................................................................................... 48
4.7 PC與DSP的通訊命令說明.................................................................... 49
4.8 PC端人機介面的顯示架構..................................................................... 51

第五章 實驗與討論....................................................................... 53
5.1 即時影像擷取....................................................................................... 54
5.2 動態SURF特徵擷取實驗...................................................................... 55
5.3 動態SURF特徵匹配實驗..................................................................... 62
5.4 OpenSURF與本系統計算之SURF特徵比較........................................... 67

第六章 結論與未來工作................................................................ 71
6.1 結論..................................................................................................... 71
6.2 未來工作.............................................................................................. 72

參考文獻....................................................................................... 74
附錄一、嵌入式影像處理系統電路圖............................................. 76
附錄二、CMOS影像感測器(OV7649)電路圖................................. 92

[1] Herbert Bay, Andreas Ess, Tinne Tuytelaars and Luc Van Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding , Vol.110, No.3, 2008, pp.346-359.
[2] Anthony Rowe, Charles Rosenberg and Illah Nourbakhsh, "A Low Cost Embedded Color Vision System," in Proceedings of IROS 2002, Switzerland, 2002, pp.208-213.
[3] Stephanie McBader and Peter Lee, "An FPGA Implementation of a Flexible, Parallel Image Processing Architecture Suitable for Embedded Vision Systems," in Proceedings of the International Parallel and Distributed Processing Symposium, Nice, France, 2003, pp.228-232.
[4] Vanderlei Bonato, Adriano K. Sanches, M.M. Fernandes, João M. P. Cardoso, E.D.V. Simoes and Eduardo Marques, "A Real Time Gesture Recognition System for Mobile Robots," in Proceedings of International Conference on Informatics in Control, Automation, and Robotics, Setúbal, Portugal, 2004, pp.207-214.
[5] P. Chalimbaud and F. Berry, "Design of an Imaging System based on FPGA Technology and CMOS Imager," in Proceedings of IEEE International Conference on Field-Programmable Technology, Brisbane-Australia, 2004, pp407-411.
[6] Stefan Mahlknecht, Roland Oberhammer and Gregor Novak, "A Real-time Image Recognition System for Tiny Autonomous Mobile Robots," in Proceedings of the 10th IEEE Real-Time and Embedded Technology and Applications Symposium, Toronto, Canada, 2004, pp.324-330.
[7] Fen Xu, Jian-Jun Zeng and Yun-Long Zhang, "Design of a DSP-based CMOS Imaging System for Embedded Computer Vision," in Proceedings of IEEE Conference on Cybernetics and Intelligent Systems, Chengdu, China, 2008, pp430-433.
[8] P.A. Viola and M.J. Jones, "Rapid Object Detection Using a Boosted Cascade of Simple Features," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, 2001, pp.511–518.
[9] Christopher Evans, "Notes on the OpenSURF Library," http://www.chrisevansdev.com/ computer-vision-opensurf.html, 2009.
[10] Alexander Neubeck and Luc Van Gool, "Efficient Non-Maximum Suppression," in Proceedings of the 18th International Conference on Pattern Recognition , Hong Kong, 2006.
[11] M. Brown and D.G. Lowe, "Invariant features from interest point groups," in Proceedings of British Machine Vision Conference, Cardiff , Wales, 2002, pp.656–665.

[12] Texas Instruments Inc., "TMS320C6000 DSP External Memory Interface (EMIF) Reference Guide," SPRU266E, 2008.
[13] Altera Corporation, "Cyclone II Device Handbook," Volume 1, 2006.
[14] OminVision Technologies, Inc. "OV7649 Color CMOS VGA(640x480) CAMERACHIP Datasheet," 2004.
[15] OminVision Technologies, Inc. "OmniVision Serial Camera Control Bus(SCCB) Functional Specification," Ver. 2.2, 2007.
[16] Texas Instruments Inc., "TMS320C6000 Chip Support Library API Reference Guide," SPRU401J, 2004.
[17] Spansion Inc., "Am29LV320D Datasheet," 2005.
[18] Texas Instruments Inc., "Creating a Second-Level Bootloader for FLASH Bootloading on TMS320C6000 Platform With Code Composer Studio," Application Report, SPRA999A1, 2006.
[19] Texas Instruments Inc., "TMS320C6000 Tools: Vector Table and Boot ROM Creation," Application Report, SPRA544D, 2004.
[20] 盧怡仁, 蔡偉和, "單晶片於數位信號處理的應用-以TMS320C6000的開發平台為例" 文魁資訊, 台北, 民國九十六年。
[21] Cypress Semiconductor Corp., "EZ-USB FX2LP™ USB Microcontroller-High Speed USB Peripheral Controller," Rev. R, 2010.
[22] Cypress Semiconductor Corp., "EZ-USB® Technical Reference Manual," Rev. C, 2010.
[23] Cypress Semiconductor Corp., "Cypress CyAPI Programmer's Reference," 2011.
[24] Texas Instruments Inc., "TMS320C620x/C642x McBSP: UART," Application Report, SPRA633C, 2008.
[25] Altera Corporation, "Avalon Interface Specifications," ver. 11.0, 2011.
[26] Altera Corporation, "Understanding Metastability in FPGAs," ver. 1.2, 2009.
[27] Texas Instruments Inc., "Creating a Second-Level Bootloader for FLASH Bootloading on TMS320C6000 Platform With Code Composer Studio," Application Report, SPRA999A1, 2006.
[28] Texas Instruments Inc., "TMS320C6000 EMIF to External Flash Memory," Application Report, SPRA568A, 2002.
[29] D. G. Lowe, “Distinctive Image Features from Scale-invariant Keypoints,” International Journal of Computer Vision, 2004, pp.91-110.
[30] Štefan Badura and Stanislav Foltán, “Advanced scale-space, invariant, low detailed feature recognition from images - car brand recognition,” in Proceedings of the International Multiconference on Computer Science and Information Technology, Wisla, 2010, pp.19-23.