臺灣博碩士論文加值系統

本篇論文提出一量子類神經模糊網路及其相關演算法來實現分類應用問題。所提出的量子類神經模糊網路(QNFN)是一個四層架構，其中我們利用量子歸屬函數來完成網路中的第二層，它是一個多層級的活化函數，且每一個量子歸屬函數主要藉由多個雙彎曲函數位移組合而成。一自我組織的學習演算法包括了自我分群演算法和倒傳遞學習演算法，其中所提出來的自我分群演算法可以動態評估輸入樣本空間被分成幾群，而倒傳遞學習演算法主要是用來完成參數的學習。一以熵值為基礎的量子類神經模糊網路(EQNFN)被提出，此網路為一個五層架構，主要是為了使我們所提出的架構能夠得到更好的效能和學習正確性，因此我們採用了TSK形式的架構，並且藉由量子模糊熵值來評估輸入樣本分佈狀態，藉此資訊來決定量子層級數，使得學習更為有效。為了使我們所提出的架構能夠加快學習速度，因此我們提出一補償性量子類神經模糊網路(CQNFN)，主要是利用補償性參數對於架構能更有效地完成適應性之模糊運算，來加速收斂時間。最後，我們將透過分類應用問題來驗證所提出的模型在辨識率上優於其他方法。

In this thesis, a quantum neural fuzzy network (QNFN) for classification applications is proposed. The QNFN model is a four-layer structure. Layer 2 of the QNFN model contains quantum membership functions, which are multilevel activation functions. Each quantum membership function is composed of the sum of sigmoid functions shifted by quantum intervals. A self-organizing learning algorithm, which consists of the self-clustering algorithm (SCA) and the backpropagation algorithm, is also proposed. The proposed the SCA method is a fast, one-pass algorithm for a dynamic estimation of the number of clusters in an input data space. The backpropagation algorithm is used to tune the adjustable parameters. An entropy-based quantum neural fuzzy network (EQNFN) is proposed. The EQNFN model is a five-layer structure, which combines the traditional Takagi-Sugeno-Kang (TSK) to improve performance and learning accuracy. Quantum fuzzy entropy is employed to evaluate the information on pattern distribution in the pattern space. With this information, we can determinate the number of quantum levels. A compensatory quantum neural fuzzy network (CQNFN) is proposed. The compensatory-based fuzzy reasoning method is using adaptive fuzzy operations of neural fuzzy network that can make the fuzzy logic systems more adaptive, effective and converge quickly. Finally, we are used to classification application to demonstrate our models learning capability and performance. The simulation results show that the average classification accuracy of our models is better than other methods.

Abstract in Chinese....Ⅰ
Abstract in English....Ⅲ
Acknowledgements in Chinese....Ⅴ
Contents....Ⅵ
List of Tables....Ⅷ
List of Figures....Ⅸ
1 Introduction....1
1.1 Motivation....1
1.2 Literature Survey....4
1.3 Organization of Thesis....5
2 A Self-Organizing Quantum Neural Fuzzy Network....7
2.1 Quantum Neural Network....7
2.2 The Self-Clustering Algorithm....9
2.3 The Structure of the QNFN....10
2.4 A Learning Algorithm for the QNFN Model....15
2.4.1 The Structure Learning Algorithm for QNFN....16
2.4.2 The Parameter Learning Algorithm for QNFN....21
3 A Self-Organizing Entropy-Based Quantum Neural Fuzzy Network....24
3.1 TSK-Type Neural Fuzzy Network....25
3.2 Entropy Measure....26
3.3 The Structure of the EQNFN....27
3.4 A Learning Algorithm for the EQNFN Model....30
3.4.1 The Structure Learning for EQNFN....31
3.4.2 The Parameter Learning for EQNFN....37
4 A Self-Organizing Compensatory Quantum Neural Fuzzy Network....41
4.1 Compensatory Neural Fuzzy Network....42
4.2 The Compensatory Operation....43
4.3 The Structure of the CQNFN....45
4.4 A Learning Algorithm for the CQNFN Model....50
4.4.1 The Structure Learning for CQNFN....51
4.4.2 The Parameter Learning for CQNFN....53
5 Illustrative Examples....58
5.1 Iris Data Classification....58
5.1.1 Simulation Results of QNFN for Iris Data....60
5.1.2 Simulation Results of EQNFN for Iris Data....66
5.1.3 Simulation Results of CQNFN for Iris Data....72
5.1.4 The Comparison of Performance for Iris Data....78
5.2 Wisconsin Breast Cancer Diagnostic Data....80
5.2.1 Simulation Results of QNFN for Breast Cancer Data....81
5.2.2 Simulation Results of EQNFN for Breast Cancer Data....83
5.2.3 Simulation Results of CQNFN for Breast Cancer Data....86
5.2.4 The Comparison of Performance for Breast Cancer Data....88
6 Conclusion and Future Works....90
Bibliography....92
Vita....97
Publication List....98

[1] P. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis, New York: Wiley, 1973.

[2] C. M. Bishop, Neural Networks for Pattern Recognition, Oxford: Oxford University Press, 1995.

[3] R. Setiono and H. Liu, “Neural-Network Feature Selector,” IEEE Trans. Neural Network, Vol. 8, No. 3, 1997, pp. 654-662.

[4] H. M. Lee, C. M. Chen, J. M. Chen, and Y. L. Jou, “An Efficient Fuzzy Classifier with Feature Selection Based on Fuzzy Entropy,” IEEE Trans. Syst., Man, Cybern., pt. B, Vol. 31, No. 3, 2001, pp. 426-432.

[5] S. Halgamuge and M. Glesner, “Neural Networks in Designing Fuzzy Systems for Real World Applications,” Fuzzy Sets and Systems, Vol. 65, 1994, pp. 1-12.

[6] N. Kasabov, “Learning Fuzzy Rules and Approximate Reasoning in Fuzzy Neural Networks and Hybrid Systems,” Fuzzy Sets and Systems, Vol. 82, 1996, pp. 135-149.

[7] D. Nauck and R. Kruse, “A Neuro-Fuzzy Method to Learn Fuzzy Classification Rules From Data,” Fuzzy Sets and Systems, Vol. 89, 1997, pp. 277-288.

[8] S. Paul and S. Kumar, “Subsethood-Product Fuzzy Neural Inference System (SuPFuNIS),” IEEE Trans. Neural Networks, Vol. 13, No. 3, 2002, pp. 578-599.

[9] M. Russo, “FuGeNeSys—A Fuzzy Genetic Neural System for Fuzzy Modeling,” IEEE Trans. Fuzzy Systems, Vol. 6, No. 3, 1998, pp. 373-388.

[10] J. S. Wang and C. S. George Lee, “Self-Adaptive Neuro-Fuzzy Inference Systems for Classification Applications,” IEEE Trans. Fuzzy Systems, Vol. 10, No. 6, 2002, pp. 790-802.

[11] C. T. Lin and C. S. G. Lee, Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent System, NJ: Prentice-Hall, 1996.

[12] J. S. R. Jang, “ANFIS: Adaptive-Network-Based Fuzzy Inference System,” IEEE Trans. Syst., Man, Cybern., Vol. 23, No. 3, 1993, pp.665-685.

[13] C. J. Lin and C. T. Lin, “Reinforcement Learning for an ART-Based Fuzzy Adaptive Learning Control Network,” IEEE Trans. Neural Networks, Vol. 7, No. 3, June, 1996, pp. 709-731.

[14] C. F. Juang and C. T. Lin, “An On-Line Self-Constructing Neural Fuzzy Inference Network and Its Applications,” IEEE Trans. Fuzzy Systems, Vol. 6, No.1, Feb. 1998, pp. 12-31.

[15] F. J. Lin, C. H. Lin, and P. H. Shen, “Self-Constructing Fuzzy Neural Network Speed Controller for Permanent-Magnet Synchronous Motor Drive,” IEEE Trans. Fuzzy Systems, Vol. 9, No. 5, Oct. 2001, pp. 751-759.

[16] C. T. Chao, T. J. Chen, and C. C. Teng, “Simplification of Fuzzy-Neural Systems Using Similarity Analysis,” IEEE Trans. Syst., Man, Cybern., Vol. 26, No. 2, 1996, pp. 344-354.

[17] C. J. Lin and C. T. Lin, “An ART-Based Fuzzy Adaptive Learning Control Network,” IEEE Trans. Fuzzy Systems, Vol. 5, No. 4, 1997, pp. 477-496.

[18] T. Takagi and M. Sugeno, “Fuzzy Identification of Systems and Its Applications to Modeling and Control,” IEEE Trans. Syst., Man, Cybern., Vol. 15, 1985, pp. 116-132.

[19] G. Purushothaman and N. B. Karayiannis, “Quantum Neural Networks (QNNs): Inherently Fuzzy Feedforward Neural Networks,” IEEE Trans. Neural Networks, Vol. 8, No. 3, May 1997, pp. 679-693.

[20] R. Kretzschmar, R. Bueler, N. B. Karayiannis and F. Eggimann, “Quantum Neural Networks Versus Conventional Feedforward Neural Networks: An Experimental Study,” in Proc. IEEE Int. Conf. Signal Processing, Vol. 1, Dec. 2000, pp. 328-337.

[21] L. Fei, Z. Shengmei and Z. Baoyu, “Quantum Neural Network in Speech Recognition,” in Proc. IEEE Int. Conf. Signal Processing, Vol.6, Aug. 2000, pp. 1267-1270.

[22] M. Leshno, V. Y. Lin, A. Pinkus and S. Schocken, “Multilayer Feedforward Networks with a Nonpolynomial Activation Function can Approximate Any Function,” Neural Networks, Vol. 6, No. 6, 1993, pp.861-867.

[23] M. C. Hung and D. L. Yang, “The Efficient Fuzzy c-Means Clustering Technique,” in Proc. IEEE Int. Conf. Data Mining, Dec. 2001, pp. 225-232.

[24] J. He, L. Liu and G. Palm, “Speaker Identification Using Hybrid LVQ-SLP Networks,” in Proc. IEEE Int. Conf. Neural Networks, Vol. 4, Dec. 1995, pp. 2052-2055.

[25] K. K. Ang, C. Quek, and M. Pasquier, “POPFNN-CRI(S): Pseudo Outer Product Based Fuzzy Neural Network Using the Compositional Rule of Inference and Singleton Fuzzifier,” IEEE Trans. Syst., Man, Cybern., Part B, Vol. 33, No. 6, Dec. 2003, pp. 838-849.

[26] W. L. Tung and C. Quek, “GenSoFNN: A Generic Self-Organizing Fuzzy Neural Network,” IEEE Trans. Neural Networks, Vol. 13, No. 5, Sept. 2002, pp. 1075-1086.

[27] N. K. Kasabov, and Q. Song, “DENFIS: Dynamic Evolving Neural-Fuzzy Inference System and Its Application for Time-Series Prediction,” IEEE Trans. Fuzzy Systems, Vol. 10, No. 2, Apr. 2002, pp. 144-154.

[28] L. X. Wang and J. M. Mendel, “Generating Fuzzy Rules by Learning from Examples,” IEEE Trans. Syst., Man, Cybern., Vol. 22, No. 6, Dec. 1992, pp. 1414-1427.

[29] B. Kosko, “Fuzzy Entropy and Conditioning,” Inform. Sci., Vol. 40, Dec. 1986, pp. 165-174.

[30] S. K. Pal and B. Chakraborty, “Fuzzy Set Theoretic Measure for Automatic Feature Evaluation,” IEEE Trans. Syst., Man, Cybern., Vol. 16, May 1986, pp. 754–760.

[31] R. E. Belahut, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 1987.

[32] T. M. Cover and J. A. Thomas, Elements of Information Theory., New York: Wiley, 1992.

[33] J. Y. Ching et al., “Class-Dependent Discretization for Inductive Learning form Continuous and Mixed-Mode Data,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 17, July 1995, pp. 641–651.

[34] H. J. Zimmermann and P. Zysno, “Latent Connective in Human Decision,” Fuzzy Sets and Systems, Vol. 4, 1980, pp. 31-51.

[35] Y. Q. Zhang and A. Kandel, “Compensatory Neurofuzzy Systems with Fast Learning Algorithms,” IEEE Trans. Neural Networks, Vol. 9, No. 1, Jan. 1998, pp. 83-105.

[36] C. S. Ouyang and S. J. Lee, “An Improved Learning Algorithm for Rule Refinement in Neuro-Fuzzy Modeling,” Third International Conference Knowledge-Based Intelligent Information Engineering Systems, 1999, pp. 238-241.

[37] H. Seker, D. E. Evans, N. Aydin, and E. Yazgan, “Compensatory Fuzzy Neural Networks-Based Intelligent Detection of Abnormal Neonatal Cerebral Doppler Ultrasound Waveforms,” IEEE Trans. Information Technology in Biomedicine, Vol. 5, No. 3, 2001, pp. 187-194.

[38] C. J. Lin and W. H. Ho, “A Pseudo-Gaussian-Based Compensatory Neural Fuzzy System,” IEEE Int’l Conf. on Fuzzy Systems, 2003, pp. 214-220.

[39] C. J. Lin and C. H. Chen, “Nonlinear System Control Using Compensatory Neuro-Fuzzy Networks,” IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. E86-A, No. 9, 2003, pp. 2309-2316.

[40] P. K. Simpson, “Fuzzy Min-Max Neural Networks-Part I: Classification,” IEEE Trans. Neural Networks, No. 3, No. 5, Sept. 1992, pp. 776-786.

[41] H. M. Lee, “A Neural Network Classifier with Disjunctive Fuzzy Information,” Neural Networks, Vol. 11, No. 6, 1998, pp. 1113-1125.

[42] T. P. Wu and S. M. Chen, “A New Method for Constructing Membership Functions and Fuzzy Rules from Training Examples,” IEEE Trans. Syst., Man, Cybern., pt. B, Vol. 29, No. 1, Feb. 1999, pp. 25-40.

[43] B. C. Lovel and A. P. Bradley, “The Multiscale Classifier,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 18, No. 2, 1996, pp. 124-137.