多重感測器融合與整合的技術已經廣泛應用在自動化工業的生產與組裝、軍事用途、機械人追蹤與太空探險以及醫療工程等方面，運用這類型技術具有加強系統穩定度及強健度、擴充架構可性度及解析度、和增加量測涵蓋範圍及縮短反應時間等優點﹔本篇論文主要將多重感測器融合與整合的技術應用在智慧型輪椅上，發展出直接操作模式、輔助操作模式、和自動操作模式，以因應不同程度殘障病患的需要，減少電動輪椅的操作危險與困擾，並可應用於醫院自動化。
針對智慧型輪椅系統的需求，本篇論文將其分為兩種層次來作相關的研究，分別為低階硬體建立與整合以及高階人機操作界面。在低階模組方面，以嵌入式理念，完成了建構及整合智慧型輪椅硬體結構、伺服控制單元、網路通訊、以及多樣感測器模組，包括距離測量、方位測量及影像視覺系統﹔在高階模組方面，應用了多重感測器融合理論來擷取環境特徵，並根據這些特徵，結合環境資訊評估及使用者操作參數，透過預測型決策演算法及模糊控制理論，建立力回饋人機輔助操作界面﹔最後，利用並行架構的觀念，整合了低階及高階兩大模組，以最佳化多執行緒的方式實現軟體部份，使其可以幾近達到即時處理之能力。我們利用盲人實際架駛此智慧型輪椅系統在普通室內環境做實驗測試，效果非常良好。

Multisensor fusion and integration is widely used for applications such as assembly, military command and control for battlefield management, mobile robot navigation, multitarget tracking, and aircraft navigation, etc. The advantages of using multisensor fusion and integration includes: improved system reliability and robustness, increased confidence and resolution, extended coverage and shorter response time. We have developed three kinds of operating mode: direct control mode, assistant control mode, and autonomous control mode on intelligent wheelchair for helping patient and applying on hospital automation based on multisensor fusion and integration technology.
According to the requirements of intelligent wheelchair, we develop a two-level approach. In the low-level module, there are hardware architectures, servomotor control, network communication, and sensory system based on an embedded concept. In the high-level module, by employing sensory fusion to extract the environment feature, combine environment information and evaluate user intention, we create force feedback operation interface through prediction decision algorithm and fuzzy theory. Finally, we integrate the low-level and high-level module, and implement software architecture by multithread method to achieve the near real-time performance based on multiple-process concept. We have conducted the experiment by using a blind person operate the intelligent wheelchair in indoor office environment, and the experimental results demonstrates the effectiveness for the intelligent wheelchair system we have developed.

摘要i
Abstractii
Table of Contentsiii
List of Figuresv
Chapter 1 Introduction1
1.1 Introduction1
1.2. Objectives3
1.2.1 Background4
1.2.2 Functional Specification6
Chapter 2 A Brief Review of Multisensor Fusion and Integration9
2.1 Classification of Multisensor Fusion and Integration9
2.2 Theories and Algorithms for Multisensor Fusion and Integration13
2.2.1. Low Level Fusion13
2.2.2. Medium Level Fusion15
2.2.3. High Level Fusion17
2.3 Applications of Multisensor Fusion and Integration20
2.3.1 Application of Multisensor-Based Mobile Robot20
2.3.2 Industrial Application22
2.3.3 Military Application22
Chapter 3 Low Level Module and System Architecture24
3.1. Hardware Configuration24
3.1.1 Mobile Base and Exterior28
3.1.2 Sensor Data Acquisition29
3.1.3. Sonar Ranging Sensors31
3.1.4. Infrared Range Sensors34
3.1.5 Oriental Detection Sensors35
3.1.6 Vision System36
3.1.7 User Operation Interface System37
3.1.8. Operation System38
3.2 Software Architecture39
3.2.1 Servo Control Design39
3.2.2 Sensor Design39
3.2.3 Vision Design40
3.2.4 Network Design40
3.2.5 User Interface Design41
3.2.6 Multithread Design42
3.3 Wheelchair Motion Control42
Chapter 4 High Level Operation with Force Feedback44
4.1 Introduction44
4.2 Concept45
4.3 Object Detection and Representation47
4.4 Joystick Intention and Object Weight49
4.5 Force Feedback Command Decision Process50
4.5.1 Force Feedback Effect50
4.5.2 Grey Model Predictor51
4.5.3 Force Command Decision54
4.5.4 Force Amplitude Command Generation54
Chapter 5 System Integration57
5.1 Introduction57
5.2 Object-Oriental Design58
5.3 Multithread Programming63
Chapter 6 Experiment67
6.1 Introduction67
6.2 Low Level Module67
6.2.1 Motion Control System67
6.2.2 Sensory System69
6.3 High Level Module70
6.3.1 Wall Following70
6.3.2 Across the Two Office Rooms74
Chapter 7 Conclusion78
Reference80
Appendix A Basic Control Program Class List89
A.1 Servo Control Class89
A.2 Vision Control Class89
A.3 Data I/O Class92
A.4 Sensors Data Acquisition Class92
A.5 Operation Interface Class93
A.6 Communication Class93
Appendix B Advance Inference Program Class List95
B.1 Fuzzy Logic Implementation Class95
B.2 Grey Theory Implementation Class95

[1] R. C. Luo and M. G. Kay, "Multisensor Integration and Fusion in Intelligent Systems," IEEE Transactions on System, Man, and Cybernetics, Vol. 19, No. 5, pp. 901-931, 1989.
[2] P. K. Varshney, "Multisensor Data Fusion," IEE Electronics and Communication Engineering Journal, pp.245-253, 1997.
[3] F. Russo and G. Ramponi, "Fuzzy Methods for Multisensor Data Fusion," IEEE Transactions on Instrumentation and Measurement, Vol.43, No.2, pp.288-294, 1994.
[4] J. Vandorpe, H. V. Brussel, H. Xu, "LiAS: A Reflexive Navigation Architecture for an Intelligent Mobile Robot System," IEEE Transactions on Industrial Electronics, Vol.43, No.3, pp.432-440, 1996.
[5] H. R. Beom and H. S. Cho, "A Sensor-Based Navigation for Mobile Robot Using Fuzzy Logic and Reinforcement Learning," IEEE Transactions on System, Man, and Cybernetics, Vol.25, No.3, pp.464-477, 1995.
[6] B. V. Dasarathy, "Decision Fusion Strategies in Multisensor Environments," IEEE Transactions on System, Man, and Cybernetics, Vol.21, No.5, pp.1140-1154, 1991.
[7] S. G. Goodridge, M. G. Kay, R. C. Luo, "Multilayered Fuzzy Behavior Fusion for Real-Time Reactive Control of Systems with Multiple Sensors," IEEE Transactions on Industrial Electronics, Vol.43, No.3, pp.387-394, 1996.
[8] J.A. Stover, D.L. Hall, R.E. Gibson, "A Fuzzy-Logic for Autonomous Multisensor Data Fusion," IEEE Transactions on Industrial Electronics, Vol.43, No.3, pp.403-410, 1996.
[9] M. Kam, X. Zhu, P. Kalata, "Sensor Fusion for Mobile Robot Navigation," Proceedings of the IEEE, Vol.85, No.1, 1997.
[10] R. Krzysztofowicz, and D. Long, "Fusion of Detection Probabilities and Comparison of Multisensor Systems," IEEE Transactions on System, Man, and Cybernetics, Vol. 20, No. 3, pp.665-677, 1990.
[11] N. Ansari, J.-G. Chen, Y.-Z. Zhang, "Adaptive Decision Fusion for Unequiprobable Sources," IEE Proceedings - Radar, Sonar and Navigation, 1997.
[12] N. Ansari, E.S.H. Hou, B.O. Zhu, J.G. Chen, "Adaptive Fusion by Reinforcement Learning for Distributed Detection Systems," IEEE Transactions on Aerospace and Electronic Systems, Vol.32, No.2, pp.524-531, 1996.
[13] D. A. Pados, P. Papantoni-Kazakos, D. Kazakos, A. G. Koyiantis, "On-Line Threshold Learning for Neyman-Pearson Distributed Detection," IEEE Transactions on System, Man, and Cybernetics, Vol.24, No. 10, pp.1519-1531, 1994.
[14] N. S. V. Rao, "Distributed Decision Fusion Using Empirical Estimation," Proc. of IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp.697-704, 1996
[15] C.B. Queen et al, "JSTARS Fusion Integration Technology for Advanced Targeting" in 8th Nat'l Sensor Fusion, Vol. 1, 1995.
[16] G. Bourhis & P. Pino, "Mobile Robotics and Mobility Assistance for People with Motor Impairments: Rational Justification for the VAHM Project," IEEE Trans. On Rehabilitation Engineering, Vol. 4, No. 1, pp7-12, 1996
[17] P. Wellman, V. Krovi, V. Kumar, & W. Harwin, "Design of a Wheelchair with Legs for People with Motor Disabilities," IEEE Trans on Rehabilitation Engineering, Vol. 3, No. 4, pp343-353, 1995.
[18] N.I. Katevas, N.M. Sgouros, S.G. Tzafestas, G. Papakonstantinous, P. Beattie, J.M. Bishop, P. Tsanaksa, & D. Koutsouris, "The Autonomous Mobile Robot SENARIO: A Sensor-Aided Intelligent Navigation System for Powered Wheelchairs," IEEE Robotics & Automation Magazine, pp60-70, 1997
[19] Rory A. Cooper, "Intelligent Control of Power Wheelchairs," IEEE Engineering in Medicine and Biology, pp423-431, 1995
[20] Ren C. Luo, M.Y. Chern, K.S. Hwang, A.Liu, T.M. Chen, C.Y. Hu, K.C. Hong, Y.W. Lee, C.C.Chen, "Development of an Intelligent Electrical Wheelchair for Hospital Automation," Proceedings of Int. Conf. On Mechatronic Technology, pp.417-422, 1998.
[21] Ren C. Luo, Arthur M. D. Shr, Chi-Yang Hu, "Multiagent Based Multisensor Resource Management System," Proceedings of the IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS'98), Vol.2, pp.1034-1039, 1998.
[22] Ren C. Luo, Tse Min Chen, and Chi-Yang Hu,, "The Development of Model-Free Intelligent Assistance Control of Electrical Wheelchair," (Ref. No. AB009, Springer-Verlag), IEEE Int. Conf. On Recent Advances in Mechatronics (ICRAM'99), 1999.
[23] Ren C. Luo and Tse Min Chen, "Multiagent System with Event Driven Control for Autonomous Mobile Robot Navigation," Proceedings of the 1998 IEEE Int'l Conference on Robotics and Automation, pp.964-969, Leuven, Belgium, May, 1998.
[24] Tse Min Chen and Ren C. Luo, "Development and Integration of Multiple Behaviors for Autonomous Mobile Robot Navigation," IEEE International Conference on Industrial Electronics, Control, and Instrumentation, Aachen, Germany, 1998.
[25] Steven G. Goodridge, Ren C. Luo, and Michael G. Kay, "Multi-Layered Fuzzy Behavior Fusion for Real-time Control of System with Many Sensors," in Proc. of IEEE Int'l Conf. on Multisensor Fusion and Integration for Intelligent System, pp.272-279, 1994.
[26] Ren C. Luo and Tse Min Chen, "Multiagent and Event Driven Based Dynamic Collision Avoidance for an Autonomous Mobile Robot," in Proc. Of 1998 IEEE/RSJ International Conference On Intelligent Robots and Systems, Victoria, B.C., Canada, Oct. 13-17, 1998.
[27] Tse Min Chen and Ren C. Luo, "Integrated Multi-behavior Mobile Robot Navigation Using Decentralized Control," in Proc. of 1998 IEEE/RSJ International Conference On Intelligent Robots and Systems, Victoria, B.C., Canada, Oct. 13-17, 1998.
[28] D. T. Batarseh, T. N. Burcham, G. M. McFadyen, "An Ultrasonic Ranging System for the Blind," Proceedings of the IEEE Biomedical Engineering Conference, pp.411-413, 1997.
[29] Deng Julong, Grey Forecasting Control, In Grey System, China Ocean Press, 1988.
[30] F. L. Lewis, "Optimal Estimation," A Wiley-Interscience Publication, pp.19-33, 1986.
[31] P. Aigner and B. McCarragher, "Shared Control Framework applied to a Robotic Aid for the Blind," Proceedings of the 1998 IEEE Int'l Conference on Robotics and Automation, pp.717-722, Leuven, Belgium, May, 1998.
[32] S. Shoval, J. Borenstein, Y. Koren, "Mobile Robot Obstacle Avoidance in a Computerized Travel Aid for the Blind," IEEE, pp.2023-2028, 1994.
[33] Linda, "innovations; Seeing-eye Cane Steers the Blind," IEEE Spectrum, pp.17, 1997.
[34] Ren C. Luo, T.M. Chen, C.Y. Hu, Z. H. Hsiao, "Adaptive Intelligent Assistant Control of Electrical Wheelchair by Grey-Fuzzy Decision Making Algorithm," Accepted by Proceedings of 1999 IEEE International Conference on Robotics and Automation, Vol.3, pp.2014-2019, 1999.
[35] Ren C. Luo, Chi-Yang Hu, Tse Min Chen, Meng-Sien Lin, "Force Reflective Feedback Control for Intelligent Wheelchairs," Proceedings of the IEEE/RSJ Int. Conf. On Intelligent Robots and Systems (IROS'99), 1999.
[36] C.M. Krishna and Kang G. Shin, "Real-Time Systems," The McGraw-Hill Companies, Inc. 1997.
[37] J. K. Hackett and M. Shah, "Multi-Sensor Fusion: A Perspective," IEEE Proc. of International Conference on Robotics and Automation, Vol.2, pp.1324-1330, 1990.
[38] P.L. Bogler, "Dempster-Shafer Reasoning with Applications to Multi-sensor Target Identification Systems," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 17, No. 6, pp. 968-977,1987.
[39] M. Russo, "A Genetic Approach to Fuzzy Learning," IEEE International Symposium on on Neuro-Fuzzy Systems, pp.9-16, 1996.
[40] P. J. McKerrow and S. J. Volk, "A Systems Approach to Data Fusion," IEEE Proceeding of Data Fusion Symposium, pp.217-222, 1996.
[41] L. Hong, W.C. Wang, M. Logan, T. Donohue, "Multiplatform Multisensor Fusion with Adaptive-Rate Data Communication," IEEE Trasactions on Aerospace and Electronic System, Vol.33, No.1, 1997.
[42] N. Ramani and P. Patrick, "Fish Detection and Identification Using Neural Networks - Some Laboratory Results," IEEE Journal of Oceanic Engineering, Vol.17, No.4, pp.364-368, 1992.
[43] B. Ayrulu, B. Barshan, I. Erkmem, A. Erkmem, "Evidential Logical Sensing using Multiple Sensors for the Identification of Target Primitives in a Mobile Robot Environment," Proc. of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp.365-371, 1996.
[44] F. Simon, M. Nobre, "Genetic-Neuro-Fuzzy Systems: A Promising Fusion," Proceedings of 1995 IEEE International Joint Conference on Fuzzy Systems, Vol.1, pp.259-266, 1995.
[45] R.M. Rylatt, C.A. Czarnecki, T.W. Routen, "A Partially Recurrent Gating Network Approach to Learning Action Selection by Reinforcement," Proceedings of International Conference on Neural Networks, Vol.3, pp.1689-1692, 1997.
[46] J. A. Castellanos, J. M. Martinez, J. Neira, J. D. Tardos, "Simultaneous Map Building and Localization for Mobile Robots: A Multisensor Fusion Approach," Proc. of IEEE International Conference on Robotics and Automation, Vol.2, pp.1244-1249, 1998.
[47] K. Hirai, M. Hirose, Y. Haikawa, T. Takenaka, "The Development of Honda Humanoid Robot," Proc. of IEEE International Conference on Robotics and Automation, Vol.2, pp.1321-1326, 1998.
[48] S. Hedberg, "Robots Playing Soccer? RoboCup Poses a New Set of AI Research Challenges," IEEE Expert, Vol.12, Iss.5, pp.5-9, 1997.
[49] S. Hedberg, "Robots Playing Soccer? RoboCup Poses a New Set of Challenges in Intelligent Distributed Computing," IEEE Concurrency, Vol.5, Iss.4, pp.13-17, 1997.
[50] Yao-Yong Li, Nan-Ning Zheng, Li-Xing Yuan, "Controlling the chaotic neural network-a way of information integration," Proc. of IEEE/SICE RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp.775-780, 1996.
[51] T. Sawaragi, J. Umemura, O. Katai, S. Iwai, " Integrating multiple knowledge sources using genetic algorithm applied to hierarchically structured sensor data," Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp.395-402, 1994.
[52] M.L.Cone, S. Musick, "Scheduling for sensor management using genetic search", Proceedings of the IEEE National Aerospace and Electronics Conference, Vol.1, pp.150-156, 1996.
[53] Ricardo Gutierrez-Osuna and Ren C. Luo, "LOLA Probabilistic Navigation for Topological Maps," AI Magazine, Spring 1996, pp. 55-62.
[54] Richard LeGrand and Ren C. Luo, "LOLA Object Manipulation in an Unstructured Environment," AI Magazine, Spring 1996, pp. 63-70.
[55] G. Castellano, G. Attolico, E. Stella, A. Distante, "Automatic generation of rules for fuzzy robotic controller," Proceedings of IEEE/RSJ IROS'96, pp.1179-1186, 1996.
[56] Y. ANDO and S. YUTA, "Following a Wall by an Autonomous Mobile Robot with a Sonar-Ring," Proceedings of IEEE International Conference on Robotics and Automation, pp.2599-2606, 1995.
[57] Ning Xi; Tzyh-Jong Tarn; Bejczy, A.K., "Intelligent planning and control for multirobot coordination: An event-based approach", IEEE Transactions on Robotics and Automation, Vol:12, Iss: 3 pp.439-452, June 1996.
[58] R.C. Luo and M.G. Kay, "Multisensor Integration and Fusion for Intelligent Machines and Systems," Ablex Publishing Corp., 1995.
[59] E. Waltz and J. Llinas, "Multisensor Data Fusion," Artech House, Inc., 1990.
[60] M. A. Abidi and R. C. Gonzalez, "Data Fusion in Robotics and Machine Intelligence," Academic Press, 1992.
[61] H. R. Everett, "Sensors for Mobile Robots," A. K. Peters, Ltd., 1995.
[62] R. S. Pressamn, "Software Engineering," McGRAW-Hill, Inc., 1992.
[63] J. J. Hou, "Dissecting MFC Using Visuall C++ 5.0 & MFC 4.2," Unalis Co., 1997.
[64] J. J. Clark and A. L. Yuille, "Data Fusion for Sensory Information Processing Systems," Kluwer Academic Publishers, 1990.
[65] Jill D. Crisman and Jon A. Webb, "The Warp Machine on Navlab," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 5, pp. 451-465, 1991.
[66] J. Crowley, "World Modeling and Position Estimation for a Mobile Robot Using Ultrasonic Ranging," Proc. of IEEE Int. Conf. On Robot and Automation, pp.674-680, 1989.
[67] I. J. Cox, "Blanche - An Experiment in Guidance and Navigation of an Autonomous Robot Vehicle," IEEE Trans. On Robotics and Automation, Vol.7, pp.193-204, 1991.
[68] J. Borenstein and Y. Koren, "Histogramic In-Motion Mapping for Mobile Robot Obstacle Avoidance," IEEE Trans. On Robotics and Automation, Vol.7, No.4, 1991.
[69] B.S. Rao, H.F. Durrant-Whyte, "Fully Decentralised Algorithm for Multisensor Kalman Filtering," IEE Proceedings-D, Vol.138, No.5, pp.413-420, 1991.
[70] S. Thomopoulos, R. Viswanathan, D. Bougooulias, "Optimal Decision Fusion in Mulitple Sensor Systems," IEEE Trans on Aerospace and Electronic System, Vol.24, pp.309-313, 1988.
[71] M. Deffenbaugh, H. Schmidt, J.G. Bellingham, "Acoustic navigation for Arctic under-ice AUV missions," Proceedings of IEEE Engineering in Harmony with Ocean, Vol.1, pp.204-209, 1993.
[72] H. Greiner, A. Shectman, Won Chikyung, R. Elsley, P. Beith, "Autonomous legged underwater vehicles for near land warfare," Proceedings of IEEE Symposium on Autonomous Underwater Vehicle Technology, pp.41-48, 1996.
[73] B. V. Dasarathy, "Sensor Fusion Potential Exploitation - Innovative Architectures and Illustrative Applications," Proceedings of the IEEE, Vol.85, No.1, pp.24-38, 1997.
[74] D.L. Hall, J. Llinas, "An introduction to multisensor data fusion," Proceedings of the IEEE, Vol. 85, pp.6-23, 1997.