臺灣博碩士論文加值系統

多重感測器的融合及整合，是需要控制理論，訊號處理，人工智慧，以及機率及統計等跨領域的整合型研究，是近幾年來快速發展的新興研究領域。其主要研究內容是藉由將不同或多個感測器所得到的資訊藉由整合其中重複，互補，或更及時性的訊號，而為系統監督者提供系統更可靠且精準的判讀。

這篇論文提供一個現今感測器的資訊總覽，感測器融合方式的分類以及層次，以及描述多種融合演算法的基本精神跟應用層面。其中包含機器人、交通設施、軍事、無線遙控、設備監控、生物應用、定位及追蹤、辨識及分類等。

最後這篇論文，勾勒出未來的研究方向，及提供讀者一份多重感測器融合領域的文獻介紹。

Multisensor fusion and integration is a rapidly evolving research area and requires interdisciplinary knowledge in control theory, signal processing, artificial intelligence, probability and statistics, etc. The advantages gained through the use of redundant, complementary, or more timely information in a system can provide more reliable and accurate information.

This thesis provides an overview of current sensor technologies and describes the paradigm of multisensor fusion and integration as well as fusion techniques at different fusion levels. Algorithms and applications of multisensor fusion in robotics, vehicle sensing, military applications, remote sensing, equipment monitoring and diagnostics, biomedical applications, transportation systems, localization, tracking, identification and classification are also discussed.

Finally, the thesis depicts speculations concerning possible research future directions and a guide to survey and review papers in the area of multisensor fusion and integration.

誌 謝 i
中 文 摘 要 ii
Abstract iii
Table of Contents iv
List of Figures vi
List of Tables x
Chapter 1 Introduction 1
1.1 About Multisensor Fusion and Integration 1
1.1.1 Sensor Technology and Applications 2
1.1.2 Paradigm of Multisensor Fusion and Integration 8
1.2 Thesis Organization 13
Chapter 2 The Classification and Introduction of Fusion Algorithms 14
2.1 Introduction 14
2.1.1 Estimation Methods 15
2.1.2 Classification Methods 16
2.1.3 Inference Methods 18
2.1.4 Artificial Intelligence Methods 19
Low Level Fusion 21
Signal Level 21
Pixel Level 21
Algorithm 1: Covariance Intersection and Covariance Union 22
Case Study 1 Using Covariance Intersection and Covariance Union 26
Case Study 2 Using Covariance Intersection and Covariance Union 33
Case Study 3 Using Covariance Intersection and Covariance Union 43
Medium Level Fusion 56
Feature Level 56
Algorithm 2: Support Vector Machines 56
Case Study 1 Using Support Vector Machines 62
Case Study 2 Using Support Vector Machines 75
High Level Fusion 86
Symbol Level 86
Algorithm 3: Particle Filter 86
Case Study 1 Using Particle Filter 87
Case Study 2 Using Particle Filter 99
Case Study 3 Using Particle Filter 105
Case Study 4 Using Particle Filter 112
Case Study 5 Using Particle Filter 123
Algorithm 4: Bayesian Data Fusion 128
Case Study 1 Using Bayesian Data Fusion 129
Algorithm 5: Dempster-Shafer Theory 132
Case Study 1 Using Dempster-Shafer Theory 133
Case Study 2 Using Dempster-Shafer Theory 140
Chapter 3 Applications of Multisensor Fusion and Integration 148
1. Robotics 148
2. Vehicle Sensing 154
3. Military Applications 161
4. Remote Sensing 162
5. Equipment Monitoring and Diagnostics 163
6. Biomedical Applications 164
7. Transportation Systems 165
8. Localization and Tracking 165
9. Identification and Classification 166
10. Other Applications 166
Chapter 4 Conclusion 168
1. Multilevel Sensor Fusion 168
2. Fault Detection 168
3. Microsensors and Smart Sensors 169
4. Adaptive Multisensor Fusion 170
5. Conclusion 170
List of Publications 172
References 173

[1]R. C. Luo and M. G. Kay, “A tutorial on multisensor integration and fusion,” in Proc. 16th Annu. Conf. IEEE Ind. Electron., vol. 1, pp. 707–722, 1990.
[2]D. L. Hall and J. Llinas, “An introduction to multisensor data fusion,” Proc. IEEE, vol. 85, pp. 6–23, Jan. 1997.
[3]P. K. Varshney, “Scanning the special issue on the data fusion,” Proc. IEEE, vol. 85, pp. 6–23, Jan. 1997.
[4]B. V. Dasarathy, Decision Fusion. Los Alamitos, CA: IEEE Comput. Soc. Press, 1994.
[5]B. V. Dasarathy, “Sensor fusion potential exploitation-innovative architectures and illustrative applications,” Proc. IEEE, vol. 85, pp. 24–38, Jan., 1997.
[6]R. C. Luo, “Sensor technologies and microsensor issues for mechatronics systems,” IEEE/ASME Trans. Mechatron., vol. 1, pp. 39–49, Jan. 1996.
[7]R. C. Luo and T. M. Chen, “Target tracking by gray prediction theory and look-ahead fuzzy logic control,” in Proc. 1999 IEEE Int. Conf. Robot. Automat., vol. 2, pp. 1176–1181. 1999
[8]V. R. Lesser, “Cooperative multiagent systems: A personal view ofthe state of the art,” IEEE Trans. Knowledge Data Eng., vol. 11, pp.133–142, Jan. 1999.
[9]B. R. Bracio,W. Horn, and D. P. F. Moller, “Sensor fusion in biomedicalsystems,” in Proc. 19th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 3, pp. 1387–1390. 1997
[10]S. C. A. Thomopoulos, “Sensor integration and data fusion,” J. Robot.Syst., vol. 7, no. 3, pp. 337–372, 1990.
[11]S. G. Goodridge, M. G. Kay, and R. C. Luo, “Multilayered fuzzy behaviorfusion for real-time reactive control of systems with multiple sensors,” IEEE Trans. Ind. Electron., vol. 43, June 1996.
[12]J. A. Stover, D. L. Hall, and R. E. Gibson, “A fuzzy-logic architecturefor autonomous multisensor data fusion,” IEEE Trans. Ind. Electron., vol. 43, June 1996.
[13]R. Joshi and A. C. Sanderson, “Minimal representation multisensor fusion using differential evolution,” IEEE Trans. Syst., Man, Cybern. A: Systems and Humans, vol. 29, Jan. 1999.
[14]B. Siciliano, “Control in robotics: Open problems and future directions,”in Proc. 1998 IEEE Int. Conf. Control Applicat., vol. 1, pp. 81–85. 1998
[15]K. Hirai, M. Hirose, Y. Haikawa, and T. Takenaka, “The development of Honda humanoid robot,” in Proc. IEEE Int. Conf. Robot. Automat., vol. 2, pp. 1321–1326. 1998
[16]B. Solaiman, R. K. Koffi, M.-C. Mouchot, and A. Hillion, “An information fusion method for multispectral image classification postprocessing,” IEEE Trans. Geosci. Remote Sensing, vol. 36, no. 2, pp. 395–406, 1998.
[17]M. Kam, X. Zhu, and P. Kalata, “Sensor fusion for mobile robot navigation,” Proc. IEEE, vol. 85, no. 1, Jan. 1997.
[18]A. Filippidis, L. C. Jain, and N. Martin, “Multisensor data fusion for surface land-mine detection,” IEEE Trans. Syst., Man, Cybern. C: Applications and Reviews, vol. 30, pp. 145–150, Feb. 2000.
[19]R. R. Carson, Jr., M. P. Meyer, and D. J. Peters, “Fusion of IFF and radar data,” in Proc. 16th DASC AIAA/IEEE Digital Avionics Syst. Conf., vol. 1, pp. 5.3-9–15. 1997
[20]P. Valin, J. Couture, and M. Simard, “Position and attribute fusion ofradar, ESM, IFF and datalink for AAW missions of the Canadian patrolfrigate,” in Proc. 1996 IEEE/SICE/RSJ Int. Conf. Multisensor Fusion Integration Intell. Syst., pp. 63–71. 1996
[21]B. Solaiman, L. E. Pierce, and F. T. Ulaby, “Multisensor data fusion using fuzzy concepts: Application to land-cover classification using ERS-1/JERS-1 SAR composites,” IEEE Trans. Geosci. Remote Sensing, vol. 37, pp. 1316–1326, May 1999.
[22]Partington, M.-R. Keller, P. Seymour, and C. Bertoia, “Data fusion for use of passive microwave data in operational sea-ice monitoring,” in Geosci. Remote Sensing Symp., vol. 3, pp. 1579–1581.1999
[23]O. Rud and M. Gade, “Monitoring algae blooms in the Baltic Sea: A multi sensor approach,” in Proc. OCEANS ’99 MTS/IEEE, vol. 3, pp. 1234–1238. 1999
[24]N. Levy, M. C. Zhou, and Y. Quan, “Real-time tool wear identification using sensor integration with neural network,” in Proc. 33rd IEEE Conf. Decision Control, vol. 2, pp. 1050–1051. 1994
[25]R. J. Hansen, D. L. Hall, and S. K. Kurtz, “A new approach to the challenge of machinery prognostics,” Trans. ASME, J. Eng. Gas Turbines Power, pp. 320–325, Apr. 1995.
[26]R.C. Luo, Ogst Chen, L.C. Tu; Nodes localization through data fusion in sensor network Advanced Information Networking and Applications, 2005. AINA 2005. 19th International Conference on Volume 1, Page(s):337 – 342 vol.1, 28-30 March 2005
[27]A. Scheidig; S. Mueller; C. Martin; H.-M. Gross; Generating Persons Movement Trajectories on a Mobile Robot Robot and Human Interactive Communication, 2006. ROMAN 2006. The 15th IEEE International Symposium on Page(s):747 – 752 Sept. 2006
[28]S. Julier, J. Uhlmann, A nondivergent estimation algorithm in the presence of unknown correlations, in: Proc. American Control Conference, Vol. 4, IEEE, pp. 2369–2373. 1997
[29]R. Brueckmann, A. Scheidig, C. Martin, H.-M. Gross, Integration of a sound source detection into a probabilistic-based multimodal approach for person detection and tracking, in: Proc. Autonome Mobile Systeme (AMS 2005), pp. 131–137. Springer, 2005
[30]P. Viola, M. Jones, Robust real-time object detection, in: Proc. of IEEE Workshop on Statistical and Computational Theories of Vision, 2001.
[31]Y. Bar-Shalom, L. Campo, The Effect of the Common Process Noise on the Two-Sensor Fused Track Covariance, IEEE Trans on Aerospace and Electronic Systems, Vol AES-22, pp. 803-805, Nov 1986.
[32]H. Chen, T. Kirubarajan, Y. Bar-Shalom, Performance Limits of Track-to Track Fusion vs. Centralized Estimation: Theory and Application, Fusion 2001, Vol 1, pp. TuB1-25-33, Aug 2001.
[33]G.W. Ng; C.H. Tan; T.P. Ng; Tracking Ground Targets Using State Vector Fusion Information Fusion, 2005 8th International Conference on Volume 1, Page(s):6 pp. 25-28 July 2005
[34]Zhonghui Hu; Yunze Cai; Xing He; Xiaoming Xu; Fusion of multi-class support vector machines for fault diagnosis American Control Conference, 2005. Proceedings of the 2005
[35]C. W. Hsu and C. J. Lin, “A Comparison of Methods for Multi-class Support Vector Machines,” IEEE Trans. Neural Networks, Vol. 13, no. 2, pp. 415-425, 2002.
[36]D. L. Hall and J. Llinas, “An Introduction to Multisensor Data Fusion,” Proceedings of the IEEE, Vol. 85, no. 1, pp. 6-23, 1997.
[37]H. Pan, Z. P. Liang, T. J. Anastasio and T. S. Huang, “A Hybrid NN-Bayesian Architecture for Information Fusion,” Image Processing, ICIP 98, vol. 1, pp. 368–371, 1998.
[38]H. Pan, Z. P. Liang, T. J. Anastasio and T. S. Huang, “A Hybrid NN-Bayesian Architecture for Information Fusion,” Image Processing, ICIP 98, vol. 1, pp. 368–371, 1998.
[39]M. E. Liggins, C. Y. Chong, I. Kadar, M. G. Alford, et al., “Distributed Fusion Architectures and Algorithms for Target Tracking,” Proceedings of the IEEE, Vol. 85, no. 1, pp. 95-107, 1997.
[40]W. Yan, H. Shao and X. Wang, “Parallel Decision Models Based on Support Vector Machines and their Application to Distributed Fault Diagnosis,” Proceedings of the American Control Conference, Vol. 2, pp. 1770-1775, 2003.
[41]F. E. H. Tay and L. Shen, “Fault diagnosis based on Rough Set Theory,” Engineering Application of Artificial Intelligence, Vol. 16, pp.39-43, 2003.
[42]L.Shen, F. E. H. Tay, L. Qu and Y. Shen, “Fault diagnosis using Rough Sets Theory,” Computer in Industry, Vol. 43, pp. 61-72, 2000.
[43]T. Ogura; K. Okada; M. Inaba; H. Inoue; Behavior network acquisition in multisensor space for whole-body humanoid Multisensor Fusion and Integration for Intelligent Systems, MFI2003. Proceedings of IEEE International Conference on Page(s):317 – 322 30 July-1 Aug. 2003
[44]M. Fujita and K. Kageyama. An Open Architecture for Robot Entertainment. Autonomous Agents, pages 435442, 1997.
[45]M. Inaba, E Kanehiro, S. Kagami. and H. Inoue. Two-armed bipedal robot that can walk, roll-over and stand up. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robotss and Systems, volume 3, pages 297-302, 1995.
[46]F. Kanehiro, M. Inaba, , H. Inoue, and S. Hirai. Developmental Realization of Whole-Body Humanoid Behaviors Based on StateNet Architecture Containing Error Recovery Functions. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 3, pages 297-302, 1995.
[47]F. Kanehiro, M. Inaba, and H. Inoue. StateNet:State Transition Graph Description of Action Space that Includes Error Recovery Function. Journal of Robotics SocieQ of Japan
[48]S. Yoshida, I. Mizuuchi, T. Yoshikai, D. Sato, M. Inaba, and H. Inoue. A Behavior Network Structured 20(8):835-843, 2002.
[49]H. Nakai, K. Okada, D. Satou, D. Sakai, Y. Kino, Y. Kuniyoshi, M. Inaba, and H. Inoue. Development of a humanoid system with portable pc brain for research and education. Proceedings Robotics Society of Japan, pages 785-786, 2001,
[50]M.Inaba, S. Kagami, F. Kanehiro, Y. Hoshino, and H. Inoue. A Platform for Robotics Research Based on the Remote-Brained Robot Approach. The International Journal of Robotics Research, 19(10):933-954, 2000.
[51]M. Inaba, F. Kanehiro, S. Kagami, and H. Inoue. Two-armed bipedal robot that can walk, roll-over and stand up. Proc. Of Int. Con$ on Intelligent Robots and Systems, pages 297-302, 1995.
[52]H.H.S. Ip; J.M.C. Ng; Image Processing, 1994. Proceedings. ICIP-94., IEEE International Conference Volume 2, Page(s):292 - 295 vol.2 13-16 Nov. 1994
[53]H. S. Ip, "Visual Evidence Accumulation in Radiograph Inspection", Proceedings of British Machine Vision Conference 1991, pp. 202-207, 1991
[54]H. S. Ip, H. M. Wong, "Evidential Reasoning in Foreign Exchange Rate Forecasting", Proceedings of the 1st Conference on Artificial Intelligence Applications on Wall Street, October 1991
[55]H. S. Ip "A Dempster-Shafer Theory Approach to Knowledge-Based Radiograph Interpretation", Proceedings of the World Congress on Expert Systems, pp. 1196-1205, December 1991
[56]A. I. Hernandez, G. Carrault, F. Mora, L. Thoraval, G. Passariello, and J. M. Schleich, “Multisensor fusion for atrial and ventricular activity detection in coronary care monitoring,” IEEE Trans. Biomed. Eng., vol. 46, pp. 1186–1190, Oct. 1999.
[57]F. Azuaje,W. Dubitzky, N. Black, and K. Adamson, “Improving clinical decision support through case-based data fusion,” IEEE Trans. Biomed. Eng., vol. 46, pp. 1181–1185, Oct. 1999.
[58]B. Solaiman, R. Debon, F. Pipelier, J.-M. Cauvin, and C. Roux, “Information fusion, application to data and model fusion for ultrasound image segmentation,” IEEE Trans. Biomed. Eng., vol. 46, pp. 1171–1175, Oct. 1999.
[59], “Fault detection and isolation in multisensor train navigation systems,” in Proc. UKACC Int. Conf. Control, vol. 2, pp. 969–974.1998
[60]J. Sato, K. Shigeta, and Y. Nagasaka, “Automatic operation of a combined harvester in a rice field,” in Proc. of IEEE/SICE/RSJ Int. Conf. Multisensor Fusion Integration Intell. Syst., pp. 86–92. 1996
[61]C.-Y. Chong and M. Liggins, “Fusion technologies for drug interdiction,” in Proc. IEEE Int. Conf. Multisensor Fusion Integration Intell. Syst., pp. 435–441. 1994
[62]T. M. Chen and R. C. Luo, “Multilevel multiagent based team decision fusion for autonomous tracking system,” Mach. Intell. Robot. Control, vol. 1, no. 2, pp. 63–69, 1999.
[63], “A generalized look-ahead method for adaptive multiple sequential data fusion and decision making,” in MFI99, pp. 199–204. August 1999
[64]M. Fernandez and H. F. Durrant-Whyte, “A failure detection and isolation algorithm for a decentralized multisensor system,” in MFI 94, Las Vegas, NV, pp. 27–33. Oct. 2–5, 1994
[65]A. Aitouche, D. Maguin, and F. Busson, “Multiple sensor fault detection in heat exchanger system,” in Int. Conf. Control Applicat. Trieste, Italy, pp. 741–745. Sept. 1–4, 1998
[66]P. Balle, D. Fussel, and O. Hecker, “Detection and isolation of sensor faults on nonlinear processes based on local linear models,” in Proc. Amer. Control Conf., Albuquerque, NM, pp. 468–472. June 1997
[67]A. Mirabadi, N. Mort, and F. Schmia, “Fault detection and isolation in multisensor train navigation system,” in UKACC Int. Conf. Control ’98, pp. 909–974. Sept. 1–4, 1998
[68]T.W. Long, L. Hanzevack, andW. L. Bynum, “Sensor fusion and failure detection using virtual sensor,” in Proc. Amer. Control Conf. San Diego, CA, pp. 2417–2420. June 1999
[69]C. M. Hajiyev and F. Caliskan, “Fault detection in fight control systems via innovation sequence of Kalman filter,” in UKACC Int. Conf. Control ’98, pp. 1528–1533. Sep. 1–4, 1998
[70]S. I. Roumeliots, G. S. Sukhatme, and G. A. Bekey, “Sensor fault detection and identification in a mobile robot,” in IROS, pp. 1383–1388. Oct. 1998
[71]P. G. O’Reilly, “Local sensor fault detection using Bayesian decision theory,” in UKACC Int. Conf. Control ’98, pp. 247–251. Sept. 1–4, 1998
[72]R. C. Luo, M. Kay, and W. G. Lee, “Future trends in multisensor integration and fusion,” in Proc. IEEE Int. Symp. Ind. Electron. pp. 7–12. 1994
[73]F. Yalcinnkaya, D. P Atherton, H. Calis, and E. T. Powner, “Intelligent sensors: The way forward to decentralised intelligence and control,” in Proc. Int. Conf. Control ’98 UKACC, vol. 1, pp. 358–363. 1998
[74]R. G. Reynolds, “Robust estimation of covariance matrices,” IEEE Trans. Automat. Control, vol. 35, no. 9, pp. 1047–1051, Sept., 1990.
[75]Ishigaki T; Higuchi T; Watanabe K; Sensors Journal, IEEE Volume 7, Issue 5, Page(s):734 – 742 May 2007
[76]Lee-Ling Ong; Upcroft, B.; Ridley, M.; Bailey, T.; Sukkarieh, S.; Durrant-Whyte, H.; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):85 – 91 Sept. 2006
[77]K. Wendlandt; M. Khider; M. Angermann; P. Robertson; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):92 – 97 Sept. 2006
[78]W. Meeussen; J. Rutgeerts; K. Gadeyne; H. Bruyninckx; J. De Schutter; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):518 – 523 Sept. 2006
[79]F. Wallholf; M. Ablabmeier; G. Rigoll; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):13 – 18 Sept. 2006
[80]Sukhan Lee; Seongsoo Lee; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):395 – 401 Sept. 2006
[81]Jia Zhen; A. Balasuriya; S. Challa; Image Processing, 2005. ICIP 2005. IEEE International Conference on Volume 2, Page(s):II - 826-9 11-14 Sept. 2005
[82]V. Kaftandjian; Zhu Yue Min; O. Dupuis; D. Babot; Instrumentation and Measurement, IEEE Transactions on Volume 54, Issue 5, Page(s):1968 – 1977 Oct. 2005
[83]Kuo L. Su; Systems, Man and Cybernetics, 2006. ICSMC ''06. IEEE International Conference on Volume 2, Page(s):966 – 9718-11 Oct. 2006
[84]T. Pfuetzenreuter; T. Rauschenbach; J. Wernstedt; Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on Page(s):151 – 154 Sept. 2006