臺灣博碩士論文加值系統

本文提出的遞迴式小波類神經模糊網路(wavelet-based recurrent neuro-fuzzy network, WRNFN)，用於解決非線性動態系統的預測、分類與控制的問題。遞迴式小波類神經模糊網路結合了傳統的TSK模糊模組以及小波類神經網路。它採用的是類似於小波類神經網路的非矩形且緊密連結的模型來表示。經由在前授型小波模糊神經網路(WFNN)的第二層架構中加入一些回授連結可以用來嵌入時間上的關聯性，就如同於一個內部記憶體的功能。
在學習演算法方面，所提出的WRNFN模型包含兩種參數學習演算法: 同步干擾(Simultaneous Perturbation)學習演算法及混合式學習演算法，並且利用線上的模式來調整歸屬函數的形狀與小波類神經網路的連結權重值。為了達到高速的運算及即時應用，我們使用硬體描述語言(very high speed integrated circuit hardware description language, VHDL) 來設計具學習能力之WRNFN，並且將之實現在場效可程式化閘極陣列(Field Programmable Gate Array, FPGA)上。根據模擬結果，我們所設計具學習能力之WRNFN實現在FPGA上是可行的。

This study presents a wavelet-based recurrent neuro-fuzzy network (WRNFN) for control, prediction and identification of nonlinear dynamic systems. The proposed WRNFN model combines the traditional Takagi-Sugeno-Kang (TSK) fuzzy model and the wavelet neural networks (WNN). This study adopts the non-orthogonal and compactly supported functions as wavelet neural network bases. Temporal relations embedded in the network are caused by adding some feedback connections representing the memory units into the second layer of the feedforward wavelet-based fuzzy neural networks (WFNN).
The parameter learning algorithm of the proposed WRNFN consists of the simultaneous perturbation algorithm and the hybrid learning algorithm for adjusting the shape of the membership function and the connection weights of WNN in on-line. In order to obtain the high speed operation and the real-time application, we use very high speed integrated circuit hardware description language (VHDL) to design WRNFN with learning ability and implemented on Field Programmable Gate Array (FPGA). We confirmed the viability of this implementation through simulation of the control of water bath temperature system and the identification of dynamic system.

Chinese Abstract I
English Abstract III
Chinese Acknowledgements V
Contents VII
List of Tables IX
List of Figures IX
CHAPTER I Introduction 1
1.1 Motivation 1
1.2 Literature Review 6
1.3 Thesis Organization 8
CHAPTER II Structure and FPGA Implementation of WRNFN 9
2.1 Description of Wavelet Bases and Wavelet Neural Networks 9
2.2 The Structure of the WRNFN 13
2.3 FPGA Implementation 17
2.3.1 An Overview the Hardware Implementation of the WRNFN Structure 17
2.3.2 Data Representation 18
2.3.3 Implementation of Function Unit 20
CHAPTER III The Simultaneous Perturbation Learning Algorithm and FPGA Implementation 27
3.1 Introduction 27
3.2 Simultaneous Perturbation Algorithm 28
3.3 FPGA Implementation 32
3.4 Illustrative Examples 36
Example 1: Exclusive OR Learning 36
Example 2: Identification of Nonlinear Dynamic System 39
CHAPTER IV The Hybrid Learning Algorithm and FPGA Implementation 45
4.1 Introduction 45
4.2 Review of Particle Swarm Optimization (PSO) 46
4.3 The Hybrid Learning Algorithm 49
4.4 FPGA Implementation 51
4.5 Illustrative Examples 53
Example 1: Approximation of a Sugeno’s Nonlinear Function 53
Example 2: Control of Water Bath Temperature System 57
CHAPTER V Conclusion and Future Works 66
Reference 68
Vita 74

[1]K. Hornik, “Multilayer feedforward networks are universal approximators,” Neural Networks, Vol. 2, no. 5, pp. 359 --366, 1989.
[2]C. Alippi and V. Piuri, “Experimental neural networks for prediction and identification,” IEEE Trans. on Instrumentation and Measurement. , Vol. 45, 1996. pp. 670 - 676
[3]C. W. Anderson, “Strategy learning with multilayer connectionist representations,” Proc. Fourth Int. Workshop on Mach. Learn., Irvine, CA, June 1987, pp.103-114.
[4]L. A. Zadeh, “Fuzzy logic,” IEEE Computer, Trans, Vol. 21, No. 4, 1988, pp. 83-91.
[5]H. J. Zimmermann, Fuzzy set theory and its applications, Boston: Kluwer Academic Publisher, 1991.
[6]C. C. Lee, “Fuzzy logic in control systems: fuzzy logic controllers—parts I, II,” IEEE Trans. on Syst., Man, Cybern., Vol. 20, Mar./Apr. 1990, pp.404–435.
[7]C. T. Lin, C. S. G. Lee, Neural Fuzzy Systems: A neuro-fuzzy synergism to intelligent system, NJ:Prentice-Hall, 1996.
[8]C. J. Lin and C. C. Chin, “A wavelet-based neuro-fuzzy system and its applications,” Proc. IEEE Int. Joint Conference on Neural Networks, pp. 1921-1926, Oregon, July 20-24, 2003.
[9]C. J. Lin and Cheng-Chung Chin, “Prediction and Identification Using Wavelet-Based Recurrent Fuzzy Neural Networks,” IEEE Trans. on systems, Man, Cybern, vol. 34, no. 5, pp:2144 - 2154 October 2004.
[10]J. J. Blake, L. P. Maguire, T. M. McGinnity, B. Roche, and L. J. McDaid, “The implementation of fuzzy systems, neural networks and fuzzy neural networks using FPGAs,” Information Sciences: an International Journal, v.112 n.1-4, p.151-168, Dec. 1998.
[11]Synopsys, HDL Compiler for Verilog Reference Manual version 3.1a,march 1994. http://www.synopsys.com/
[12]N. Sherwani, Algorithms for VLSI Physical Design Automation: Second Edition. Norwell, MA: Kluwer, 1995.
[13]Hopkin, V.; Kirk, B, “FPGA migration to ASICs”, WESCON/''95. Conference record. ''Microelectronics Communications Technology Producing Quality Products Mobile and Portable Power Emerging Technologies'' 7-9 Nov. 1995
[14]C. J. Lin and C. T. Lin, “An ART-based fuzzy adaptive learning control network,” IEEE Trans. On Fuzzy systems, vol. 5, no. 4, pp. 477-496, Nov. 1997.
[15]T. Takagi and M. Sugeno, “Fuzzy identification of systems and its applications to modeling and control,” IEEE Trans. on Syst., Man, Cybern., vol. SMC-15, pp. 116-132, 1985.
[16]J.-S. R. Jang, ”ANFIS: Adaptive-network-based fuzzy inference system,” IEEE Trans. on Syst., Man, and Cybern., vol. 23, pp. 665-685, 1993.
[17]H. Takagi, N. Suzuki, T. Koda, and Y. Kojima, “Neural networks designed on approximate reasoning architecture and their application,” IEEE Trans. on Neural Networks, vol. 3, no. 5, pp. 752-759, 1992.
[18]E. Mizutani and J.-S. R. Jang, “Coactive neural fuzzy modeling,” Proc. Int. Conf. on Neural Networks, pp. 760-765, 1995.
[19]K. S. Narendra and K. Parthasarathy, ”Identification and control of dynamical systems using neural networks,” IEEE Trans. on Neural Networks, vol. 1, pp. 4-27, 1990.
[20]J. L. Elman, “Finding structure in time,” Cognit. Sci., Vol. 14, pp. 179–211, 1990.
[21] S. Santini, A. D. Bimbo, and R. Jain, “Block-structured recurrent neural networks,” Neural Networks, vol. 8, no. 1, pp. 135-147, 1995.
[22]C. H. Lee and C. C. Teng, “Identification and control of dynamic systems using recurrent fuzzy neural networks,” IEEE Trans. on Fuzzy Systems, vol. 8, no. 4, pp. 349-366, Aug. 2000.
[23]C. F. Juang, “ A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms,” IEEE Trans. On Fuzzy Systems , vol. 10, no. 2, pp. 155-170, Apr. 2002.
[24]Q. Zhang and A. Benveniste, “Wavelet networks,” IEEE Trans. Neural Networks, Vol. 3, No. 6, Nov. 1992, pp. 889–898.
[25]Y. C. Pati and P. S. Krishnaprasad, “Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations,” IEEE Trans. on Neural Networks, Vol. 4, No. 2, 1993, pp. 73-85.
[26]H. H. Szu and S. Kadambe, “Neural network adaptive wavelets for signal representation and classification,” Optical Engineering, Vol. 31, No. 9, 1992, pp. 1907-1916.
[27]T. Yamakawa, E. Uchino and T. Samatsu, “Wavelet neural networks employing over-complete number of compactly supported non-orthogonal wavelets and their applications,” Proc. of IEEE Conf. on Neural Networks, Vol. 3, 1994, pp. 1391 –1396.
[28]J. Chen and D. D. Bruns, “WaveARX neural network development for system identification using a systematic design synthesis,” Ind. Eng. Chem. Res., Vol. 34, 1995, pp. 4420–4435.
[29]I. Daubechies, ”Orthonormal Bases of Compactly Supported Wavelets,” Comm. Pur. Appl. Math., Vol. 41, pp. 909--996 , 1998.
[30]Ho D.W.C., P. A. Zhang and J. Xu, “Fuzzy Wavelet Networks for Function Learning,” IEEE Trans. on Fuzzy Systems, Vol. 9, No. 1, 2001, pp. 200 –211.
[31]J. Zhang and A. J. Morris, “Recurrent neuro-fuzzy networks for nonlinear process modeling,” IEEE Trans. on Neural Networks, Vol. 10, No. 2, 1999, pp.313-326.
[32]Parhami, B.: ‘Computer arithmetic: Algorithms and hardware designs’, (Oxford University Press, New York, 2000), p. 282
[33]Y. Maeda, “Learning rule of neural networks for inverse systems,” Trans. Inst. Electron., Inform., Commun. Eng., vol. J75-A, pp. 1364–1369, 1992 (in Japanese); English version of this paper is in Electron. Commun. Japan, vol. 76, pp. 17–23, 1993.
[34]J. C. Spall, “A stochastic approximation technique for generating maximum likelihood parameter estimates,” in Proc. 1987 Amer. Contr. Conf., pp. 1161–1167.
[35]Y. Maeda, H. Hirano, and Y. Kanata, “A learning rule of neural networks via simultaneous perturbation and its hardware implementation,” Neural Networks, vol. 8, pp. 251–259, 1995.
[36]J. H. Kim, D. T. College, A. Gun, and G. Do, “Fuzzy model based predictive control,” Proc. IEEE Int. Conf. Fuzzy Systems, Anchorage, AK, Vol. 1, May 1998, pp. 405-409.
[37]M. Sugeno and T. Yasukawa, “A fuzzy-logic-based approach to qualitative modeling,” IEEE Trans. Fuzzy Syst., vol. 1,no. 1, pp. 7–31, Feb. 1993.
[38]J. Tanomaru and S. Omatu, “Process Control by Online Trained Neural Network,” IEEE Transactions on Industrial Electronics, Vol. 39, pp. 511-521, 1992.