臺灣博碩士論文加值系統

本論文提出三種結合自動化建構演算法之類神經網路來處理不同的動態應用問題。首先，針對未知動態系統鑑別及控制，我們提出兩個遞迴類神經網路與其自動化建構演算法。自動化建構演算法將維度估測、混合式參數初始化與參數學習演算法整合為一個有系統的程序，可鑑別出最小架構及最佳化類神經網路參數。利用本論文所提出的網路結合其自動化建構演算法可精確地將未知非線性系統鑑別成一個系統維度最小的模型表示式。接著，根據模型表示式，發展一個包含非線性消除器及線性控制器之混合式控制方法來消除未知系統的全域非線性行為後再成功地控制未知系統。最後我們研發一個自動建構演算法來建構模糊基底函數辨識器，使其可利用單一無線三軸加速度計來辨識人類日常動作。此建構演算法是以動態線性識別分析方法的結果為基礎，此方法能在不儲存所有訓練資料與增加及刪減模式的情況下動態地更新散佈矩陣。經由真實人類動作辨識及未知非線性動態系統鑑別與控制的基準測試例子之電腦模擬，我們已經成功地驗證所提出的自動化類神經網路及建構演算法於建構高效能精簡網路的有效性。

This dissertation presents three types of neural networks coupled with automated construction algorithms for tackling different dynamic problems. First, we presented two recurrent networks and their automated construction algorithms for unknown dynamic system identification and control. The automated construction algorithms unify order determination, hybrid parameter initialization, and parameter learning into a systematic procedure for identifying a minimal structure and optimizing the network parameters, respectively. The proposed networks with their automated construction algorithms are capable of accurately identifying unknown nonlinear systems into minimal model representations. Next, according to the model representations, a hybrid control scheme consisting of a nonlinear eliminator and a linear controller was developed to remove the global nonlinear behavior and then to control the unknown systems successfully. Finally, an automated construction algorithm was developed for constructing a fuzzy basis function classifier that is capable of recognizing human daily activities using a wireless tri-axial accelerometer. The construction algorithm is based on the outcome of a dynamic linear discriminant analysis which can dynamically update the scatter matrices in both incremental and decremental modes without storing all training data in memory. Computer simulations on real-world human activity recognition and benchmark examples of unknown nonlinear dynamic system identification and control have successfully validated the effectiveness of the proposed neural networks and automated construction algorithms in constructing parsimonious networks with satisfactory performance.

中 文 摘 要 I
ABSTRACT II
CONTENTS IV
TABLES VII
FIGURES VIII
Chapter 1. Introduction 1
1.1 Motivation and Literature Survey 1
1.2 Dissertation Contributions 7
1.3 Dissertation Organization 9
Chapter 2. Fully Automated Construction Algorithm for Hammerstein Recurrent Neuro-Fuzzy Networks 11
2.1 Introduction 11
2.2 Structure of Hammerstein Recurrent Neuro-Fuzzy Network 14
2.3 Online Fully Automated Construction Algorithm 19
2.3.1 System Order Determination Algorithm 21
2.3.2 Online System Order Determination Algorithm 25
2.3.3 MCA Clustering Algorithm 29
2.3.4 Kaczmarz’s Algorithm 34
2.3.5 Recursive Recurrent Learning Algorithm 35
2.3.6 Summary of Online Fully Automated Construction Algorithm 42
2.4 Simulation Results 43
2.5 Summary 60
Chapter 3. Identification and Hybrid Control for Unknown Nonlinear Dynamic System Using Fully Automated Recurrent Neural Network 61
3.1 Introduction 61
3.2 Structure of Fully Automated Recurrent Neural Network (FARNN) 65
3.3 Fully Automated Construction Algorithms for FARNN 68
3.4 Hybrid Controller Design Based on FARNN 81
3.4.1 Nonlinearity Eliminator (NLE) Construction 85
3.4.2 Linear Controller Design Based on NLE 89
3.5 Simulation Results 90
3.6 Summary 107
Chapter 4. Human Activity Detection Using Automated Fuzzy Basis Function Classifier 109
4.1 Introduction 109
4.2 Linear Discriminant Analysis for Feature Dimension Reduction 112
4.2.1 Dynamic Linear Discrimiant Analysis 113
4.2.2 Example: Iris Data 117
4.3 Automated Fuzzy Basis Function Classifier Construction Based on Dynamic Linear Discriminant Analysis 119
4.3.1 Data Preprocessing 119
4.3.2 Feature Extraction 120
4.3.3 Automated Fuzzy Basis Function Classifier Construction Algorithm 122
4.3.4 Summary of Construction Algorithm for Human Activity Detection 127
4.4 Experimental Design and Results 128
4.5 Summary 134
Chapter 5. Conclusions 135
5.1 Conclusions 135
5.2 Recommendations for Future Work 137

[1]J. Abonyi, R. Babuska, M. Ayala Botto, F. Szeifert, and L. Nagy, "Identification and control of nonlinear systems using fuzzy Hammerstein models," Industrial and Engineering Chemistry Research, vol. 39, pp. 4302-4314, 2000.
[2]M. Agarwal, "A systematic classification of neural-network-based control," Control Systems Magazine, IEEE, vol. 17, pp. 75-93, 1997.
[3]G. Aggarwal, A. K. R. Chowdhury, and R. Chellappa, "A system identification approach for video-based face recognition," Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on, vol. 4, 2004.
[4]H. Akaike, "Statistical predictor identification," Annals of the Institute of Statistical Mathematics, vol. 22, pp. 203-217, 1970.
[5]H. Akaike, "New look at the statistical model identification," IEEE Transactions on Automatic Control, vol. AC-19, pp. 716-723, 1974.
[6]H. Al-Duwaish and M. N. Karim, "New method for the identification of Hammerstein model," Automatica, vol. 33, pp. 1871-1875, 1997.
[7]A. Atiya and J. Chuanyi, "How initial conditions affect generalization performance in large networks," IEEE Transactions on Neural Networks, vol. 8, pp. 448-451, 1997.
[8]R. Babuska and H. Verbruggen, "Neuro-fuzzy methods for nonlinear system identification," Annual Reviews in Control, vol. 27 I, pp. 73-85, 2003.
[9]E.-W. Bai and D. Li, "Convergence of the iterative Hammerstein system identification algorithm," IEEE Transactions on Automatic Control, vol. 49, pp. 1929-1940, 2004.
[10]L. Bao and S. S. Intille, "Activity recognition from user-annotated acceleration data," in Proceedings of the 2nd International Conference on Pervasive Computing and Communications, 2004, Lecture Notes in Computer Science, 2004, pp. 1-17.
[11]N. E. Barabanov and D. V. Prokhorov, "Stability analysis of discrete-time recurrent neural networks," IEEE Transactions on Neural Networks, vol. 13, pp. 292-303, 2002.
[12]P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, pp. 711-720, 1997.
[13]R. J. Bhansali and D. Y. Downham, "Some properties of the order of an autoregressive model selected by a generalization of Akaike's EPF criterion," Biometrika, vol. 64, pp. 547-551, 1977.
[14]S. P. Bhattacharyya, H. Chapellat, and L. H. Keel, Robust Control: The Parametric Approach. PTR Upper Saddle River, NJ, USA: Prentice Hall, 1995.
[15]S. A. Billings and S. Y. Fakhouri, "Identification of a class of nonlinear systems using correlation analysis," Proceedings: Institution of Electrical Engineers vol. 125, pp. 691-697, 1978.
[16]J. D. Bomberger and D. E. Seborg, "Determination of model order for NARX models directly from input-output data," Journal of Process Control, vol. 8, pp. 459-468, 1998.
[17]P. Campolucci, A. Uncini, F. Piazza, and B. D. Rao, "On-line learning algorithms for locally recurrent neural networks," IEEE Transactions on Neural Networks, vol. 10, pp. 253-271, 1999.
[18]J. S. I. Caros and J. Cmiral, "Very low complexity algorithm for ambulatory activity classification," in Proceedings of European Medical & Biological Engineering Conference and IFMBE European Conference on Biomedical Engineering, 2005.
[19]F. Chang and R. Luus, "A noniterative method for identification using Hammerstein model," IEEE Transactions on Automatic Control, vol. 16, pp. 464-468, 1971.
[20]G. Chen, Y. Chen, and H. Ogmen, "Identifying chaotic systems via a Wiener-type cascade model," IEEE Control Systems Magazine, vol. 17, pp. 29-36, 1997.
[21]S. Chen, S. A. Billings, and P. M. Grant, "Non-linear system identification using neural networks," International Journal of Control, vol. 51, pp. 1191-1214, 1990.
[22]S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Transactions on Neural Networks, vol. 2, pp. 302-309, 1991.
[23]Y.-P. Chen and J.-S. Wang, "A novel recurrent neural network with minimal representation for dynamic system identification," in Proceedings. 2004 IEEE International Joint Conference on Neural Networks, 2004, pp. 849-854 vol.2.
[24]S. L. Chiu, "Fuzzy model identification based on cluster estimation," Journal of Intelligent and Fuzzy Systems, vol. 2, pp. 267-278, 1994.
[25]E.-S. Choi, W.-C. Bang, S.-J. Cho, J. Yang, D.-Y. Kim, and S.-R. Kim, "Beatbox music phone: Gesture-based interactive mobile phone using a tri-axis accelerometer," in 2005 IEEE International Conference on Industrial Technology, ICIT 2005, Hong Kong, Hong Kong, 2005, pp. 97-102.
[26]E. K. P. Chong and S. H. Zak, An Introduction to Optimization. New York, NY: John Wiley and Sons, 2001.
[27]C. T. Chou and J. M. Maciejowski, "System identification using balanced parameterizations," IEEE Transactions on Automatic Control, vol. 42, pp. 956-974, 1997.
[28]T. W. S. Chow and F. Yong, "A recurrent neural-network-based real-time learning control strategy applying to nonlinear systems with unknown dynamics," IEEE Transactions on Industrial Electronics, vol. 45, pp. 151-161, 1998.
[29]Y. Développement. (2007). Inertial MEMS Markets for Consumer Electronics Applications. Available: http://www.yole.fr/pagesAn/products/Report_sample/WISM.p df
[30]B. De Schutter, "Minimal state-space realization in linear system theory: An overview," Journal of Computational and Applied Mathematics, vol. 121, pp. 331-354, 2000.
[31]E. J. Dempsey and D. T. Westwick, "Identification of hammerstein models with cubic spline nonlinearities," IEEE Transactions on Biomedical Engineering, vol. 51, pp. 237-245, 2004.
[32]J. C. Doyle, B. A. Francis, and A. R. Tannenbaum, Feedback Control Theory. New York: Macmillan, 1992.
[33]J. L. Elman, "Finding structure in time," Cognitive Science: A Multidisciplinary Journal, vol. 14, pp. 179 - 211, 1990.
[34]W. Favoreel, B. De Moor, and P. Van Overschee, "Subspace state space system identification for industrial processes," Journal of Process Control, vol. 10, pp. 149-155, 2000.
[35]G. F. Franklin, J. D. Powell, and M. L. Workman, Digital Control of Dynamic Systems. 2 ed. Boston, MA, USA: Addison-Wesley Longman Publishing Co. Inc. , 1990.
[36]G. F. Franklin, M. L. Workman, and D. Powell, Digital Control of Dynamic Systems. 3 ed. Boston, MA, USA: Addison-Wesley Longman Publishing Co. Inc. , 1997.
[37]P. Frasconi, M. Gori, and G. Soda, "Local feedback multilayered networks," Neural Computation, vol. 4, p. 120, 1992.
[38]C. Gan and K. Danai, "Model-based recurrent neural network for modeling nonlinear dynamic systems," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 30, pp. 344-351, 2000.
[39]U. M. Garca-Palomares, T. J. Wu, and A. Sepulveda, "The weighted average information criterion for order selection in time series and regression models," Statistics and Probability Letters, vol. 39, pp. 1-10, 1998.
[40]C. E. Garcia and M. Morari, "Internal model control - 1. A unifying review and some new results," Industrial & Engineering Chemistry Process Design and Development, vol. 21, pp. 308-323, 1982.
[41]J. Geweke and R. Meese, "Estimating regression models of finite but unknown order," International Economic Review, vol. 22, pp. 55-70, 1981.
[42]G. B. Giannakis and E. Serpedin, "A bibliography on nonlinear system identification," Signal Processing, vol. 81, pp. 533-580, 2001.
[43]M. Gori, M. Mozer, A. C. Tsoi, and R. L. Watrous, "Presenting the special issue on recurrent neural networks for sequence processing," Neurocomputing, vol. 15, pp. 181-182, 1997.
[44]V. Gorrini and H. Bersini, "Recurrent fuzzy systems," in Proceedings of the 3rd IEEE International Conference Fuzzy Systems, Orlando, FL, USA, 1994, pp. 193-198.
[45]W. Greblicki and M. Pawlak, "Identification of discrete Hammerstein systems using kernel regression estimates," IEEE Transactions on Automatic Control, vol. AC-31, pp. 74-77, 1986.
[46]S. Guillaume, "Designing fuzzy inference systems from data: An interpretability-oriented review," IEEE Transactions on Fuzzy Systems, vol. 9, pp. 426-443, 2001.
[47]E. J. Hannan and B. G. Quinn, "The determination of the order of an autoregressive," Journal of the Royal Statistical Society, vol. 41, pp. 190-195, 1979.
[48]M. H. Hansen and B. Yu, "Model Selection and the Principle of Minimum Description Length," Journal of the American Statistical Association, vol. 96, pp. 746-774, 1998.
[49]S. Haykin, Neural Networks: A Comprehensive Foundation. PTR Upper Saddle River, NJ, USA: Prentice Hall 1999.
[50]X. He and H. Asada, "New method for identifying orders of input-output models for nonlinear dynamic systems," in Proceedings of the 1993 American Control Conference Part 3 (of 3), San Francisco, CA, USA, 1993, pp. 2520-2523.
[51]J.-Q. Huang and F. L. Lewis, "Neural-network predictive control for nonlinear dynamic systems with time-delay," IEEE Transactions on Neural Networks, vol. 14, pp. 377-389, 2003.
[52]K. J. Hunt and D. Sbarbaro, "Neural networks for nonlinear internal model control," IEE Proceedings, Part D: Control Theory and Applications, vol. 138, pp. 431-438, 1991.
[53]C. M. Hurvich and C. L. Tsai, "Regression and time series model selection in small samples," Biometrika, vol. 76, pp. 297-307, 1989.
[54]A. Janczak, "Neural network approach for identification of Hammerstein systems," International Journal of Control, vol. 76, pp. 1749-1766, 2003.
[55]L. Jia, M.-S. Chiu, and S. S. Ge, "A noniterative neuro-fuzzy based identification method for Hammerstein processes," Journal of Process Control, vol. 15, pp. 749-761, 2005.
[56]C.-F. Juang and C.-T. Lin, "An online self-constructing neural fuzzy inference network and its applications," IEEE Transactions on Fuzzy Systems, vol. 6, pp. 12-32, 1998.
[57]C.-F. Juang and C.-T. Lin, "Recurrent self-organizing neural fuzzy inference network," IEEE Transactions on Neural Networks, vol. 10, pp. 828-845, 1999.
[58]C.-F. Juang, "A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms," IEEE Transactions on Fuzzy Systems, vol. 10, pp. 155-170, 2002.
[59]C.-F. Juang, "A hybrid of genetic algorithm and particle swarm optimization for recurrent network design," IEEE Transactions on Systems, Man, and Cybernetics, Part B, vol. 34, pp. 997-1006, 2004.
[60]C.-F. Juang and K.-C. Ku, "A recurrent fuzzy network for fuzzy temporal sequence processing and gesture recognition," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, pp. 646-658, 2005.
[61]C. F. Juang and C. T. Lin, "Noisy speech processing by recurrently adaptive fuzzy filters," IEEE Transactions on Fuzzy Systems, vol. 9, pp. 139-152, 2001.
[62]J. N. Juang, Applied System Identification. Upper Saddle River, NJ, USA: Prentice-Hall Inc. , 1994.
[63]S. Kallio, J. Kela, and J. Mantyjarvi, "Online gesture recognition system for mobile interaction," in System Security and Assurance, Washington, DC, United States, 2003, pp. 2070-2076.
[64]D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G. Celler, "Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring," IEEE Transactions on Information Technology in Biomedicine, vol. 10, pp. 156-167, 2006.
[65]H. K. Khalil, Nonlinear Systems. Upper Saddle River, NJ: Prentice Hall, 1996.
[66]T.-K. Kim, S.-F. Wong, B. Stenger, J. Kittler, and R. Cipolla, "Incremental linear discriminant analysis using sufficient spanning set approximations," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, United States, 2007, pp. 1-8.
[67]C.-C. Ku and K. Y. Lee, "Diagonal recurrent neural networks for dynamic systems control," IEEE Transactions on Neural Networks, vol. 6, pp. 144-156, 1995.
[68]O. Kuljaca, N. Swamy, F. L. Lewis, and C. M. Kwan, "Design and implementation of industrial neural network controller using backstepping," IEEE Transactions on Industrial Electronics, vol. 50, pp. 193-201, 2003.
[69]C.-H. Lee and C.-C. Teng, "Identification and control of dynamic systems using recurrent fuzzy neural networks," IEEE Transactions on Fuzzy Systems, vol. 8, pp. 349-366, 2000.
[70]A. U. Levin and K. S. Narendra, "Identification using feedforward networks," Neural Computation, vol. 7, pp. 349-357, 1995.
[71]F. L. Lewis and V. L. Syrmos, Optimal Control. 2 ed: Wiley-Interscience, 1995.
[72]F. L. Lewis, A. Yesildirek, and K. Liu, "Multilayer neural-net robot controller with guaranteed tracking performance," IEEE Transactions on Neural Networks, vol. 7, pp. 388-399, 1996.
[73]C. T. Lin, C. M. Yeh, J. F. Chung, S. F. Liang, and H. C. Pu, "Support-vector-based fuzzy neural networks," International Journal of Computational Intelligence Research, vol. 1, pp. 138-150, 2005.
[74]F. J. Lin and R. J. Wai, "Hybrid control using recurrent fuzzy neural network for linear-induction motor servo drive," IEEE Transactions on Fuzzy Systems, vol. 9, pp. 102-115, 2001.
[75]T. Lin, B. G. Horne, C. L. Giles, and S. Y. Kung, "What to remember: How memory order affects the performance of NARX neural networks," in Proceedings of the 1998 IEEE International Joint Conference on Neural Networks. Part 2 (of 3), Anchorage, AK, USA, 1998, pp. 1051-1056.
[76]X.-J. Liu, F. Lara-Rosano, and C. W. Chan, "Model-reference adaptive control based on neurofuzzy networks," IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 34, pp. 302-309, 2004.
[77]L. Ljung, System Identification: Theory for the User. Upper Saddle River, NJ, USA: Prentice-Hall Inc. , 1999.
[78]L. Ljung, "Prediction error estimation methods," Circuits, Systems, and Signal Processing, vol. 21, pp. 11-21, 2002.
[79]J. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, "Regularization studies on LDA for face recognition," in Proceedings of 2004 International Conference on Image Processing, ICIP 2004, Singapore, 2004, pp. 63-66.
[80]C. L. Mallows, "Some comments on C_P," Technometrics, vol. 15, pp. 661-675, 1973.
[81]S. L. Marple, Digital Spectral Analysis with Applications. Englewood Cliffs: Prentice Hall, 1987.
[82]P. A. Mastorocostas and J. B. Theocharis, "A recurrent fuzzy-neural model for dynamic system identification," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 32, pp. 176-190, 2002.
[83]P. A. Mastorocostas and J. B. Theocharis, "An orthogonal least-squares method for recurrent fuzzy-neural modeling," Fuzzy Sets and Systems, vol. 140, pp. 285-300, 2003.
[84]M. J. Mathie, B. G. Celler, N. H. Lovell, and A. C. F. Coster, "Classification of basic daily movements using a triaxial accelerometer," Medical and Biological Engineering and Computing, vol. 42, pp. 679-687, 2004.
[85]U. Maurer, A. Smailagic, D. P. Siewiorek, and M. Deisher, "Activity recognition and monitoring using multiple sensors on different body positions," Cambridge, MA, United States, 2006, pp. 113-116.
[86]T. McKelvey, "Identification of state-space models from time and frequency data," Ph.D. dissertation, Department of Electrical Engineering, Linkoping University, 1995.
[87]A. D. McQuarrie, "A small-sample correction for the Schwarz SIC model selection criterion," Statistics and Probability Letters, vol. 44, pp. 79-86, 1999.
[88]S. Mitra and Y. Hayashi, "Neuro-fuzzy rule generation: survey in soft computing framework," IEEE Transactions on Neural Networks, vol. 11, pp. 748-768, 2000.
[89]B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, C. J. Bula, and P. Robert, "Ambulatory system for human motion analysis using a kinematic sensor: Monitoring of daily physical activity in the elderly," IEEE Transactions on Biomedical Engineering, vol. 50, pp. 711-723, 2003.
[90]K. Narendra and P. Gallman, "An iterative method for the identification of nonlinear systems using a Hammerstein model," IEEE Transactions on Automatic Control, vol. 11, pp. 546-550, 1966.
[91]K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems. Upper Saddle River, NJ, USA: Prentice-Hall Inc. , 1989.
[92]K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Transactions on Neural Networks, vol. 1, pp. 4-27, 1990.
[93]K. S. Narendra and K. Parthasarathy, "Gradient methods for the optimization of dynamical systems containing neural networks," IEEE Transactions on Neural Networks, vol. 2, pp. 252-262, 1991.
[94]K. S. Narendra, "Neural networks for control: Theory and practice," Proceedings of the IEEE, vol. 84, pp. 1385-1406, 1996.
[95]K. S. Narendra and F. L. Lewis, "Introduction to the special issue on neural network feedback control," Automatica, vol. 37, pp. 1147-1148, 2001.
[96]O. Nelles, Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models. Springer, 2001.
[97]K. Ogata, Modern Control Engineering. 2 ed: Prentice-Hall, 1990.
[98]S. Pang, S. Ozawa, and N. Kasabov, "Incremental linear discriminant analysis for classification of data streams," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, pp. 905-914, 2005.
[99]N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman, "Activity recognition from accelerometer data," in Proceedings of 20th National Conference on Artificial Intelligence and the 17th Innovative Applications of Artificial Intelligence Conference, Pittsburgh, PA, United States, 2005, pp. 1541-1546.
[100]J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, pp. 465-471, 1978.
[101]P. S. Sastry, G. Santharam, and K. P. Unnikrishnan, "Memory neuron networks for identification and control of dynamical systems," IEEE Transactions on Neural Networks, vol. 5, pp. 306-319, 1994.
[102]S. Sastry and M. Bodson, Adaptive Control. Stability, Convergence and Robustness. Englewood Cliffs, NJ: Prentice-Hall, 1989.
[103]D. G. Schultz and J. L. Melsa, State Functions and Linear Control Systems. McGraw-Hill Education, 1967.
[104]G. Schwarz, "Estimating the dimension of a model," The Annals of Statistics, vol. 6, pp. 461-464, 1978.
[105]Z. Sheng and T. Sim, "When fisher meets Fukunaga-Koontz: A new look at linear discriminants," in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2006, New York, NY, United States, 2006, pp. 323-329.
[106]D. Sherrill, M. Moy, J. Reilly, and P. Bonato, "Using hierarchical clustering methods to classify motor activities of COPD patients from wearable sensor data," Journal of NeuroEngineering and Rehabilitation, vol. 2, pp. 1-14, 2005.
[107]R. Shibata, "Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear Process," The Annals of Statistics, vol. 8, pp. 147-164, 1980.
[108]S. C. Sivakumar, W. Robertson, and W. J. Phillips, "On-line stabilization of block-diagonal recurrent neural networks," IEEE Transactions on Neural Networks, vol. 10, pp. 167-175, 1999.
[109]K.-T. Song and Y.-Q. Wang, "Remote activity monitoring of the elderly using a two-axis accelerometer," in Proceedings of 2005 CACS Automatic Control Conference Tainan, Taiwan, 2005, pp. 18-23.
[110]J. T. Spooner, M. Maggiore, R. Ord′o˜nez, and K. M. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques. New York, NY: John Wiley & Sons, Inc., 2002.
[111]V. Strejc, State Space Theory of Discrete Linear Control. New York: Wiley 1981.
[112]S. W. Sung, "System identification method for Hammerstein processes," Industrial & Engineering Chemistry Research, vol. 41, pp. 4295-4302, 2002.
[113]H. C. C. Tan, J. Kui, and L. C. De Silva, "Human activities recognition by head movement using partial recurrent neural network," in Proceedings of Visual Communications and Image Processing, Lugano, Switzerland, 2003, pp. 2007-2014.
[114]K. Tanabe, "Projection method for solving a singular system of linear equations and its applications," Numerische Mathematik, vol. 17, pp. 203-214, 1971.
[115]J. Theocharis and G. Vachtsevanos, "Recursive learning algorithms for training fuzzy recurrent models," International Journal of Intelligent Systems, vol. 11, pp. 1059-1098, 1996.
[116]G. Thimm, E. Fiesler, and M. Idiap, "High-order and multilayer perceptron initialization," IEEE Transactions on Neural Networks, vol. 8, pp. 349-359, 1997.
[117]A. C. Tsoi and A. D. Back, "Locally recurrent globally feedforward networks: A critical review of architectures," IEEE Transactions on Neural Networks, vol. 5, pp. 229-239, 1994.
[118]A. C. Tsoi and A. Back, "Discrete time recurrent neural network architectures: A unifying review," Neurocomputing, vol. 15, pp. 183-223, 1997.
[119]A. C. Tsoi and S. Tan, "Recurrent neural networks: A constructive algorithm, and its properties," Neurocomputing, vol. 15, pp. 309-326, 1997.
[120]A. Tsymbal, S. Puuronen, M. Pechenizkiy, M. Baumgarten, and D. Patterson, "Eigenvector-based feature extraction for classification," in Proceedings of the Fifteenth International Florida Artificial Intelligence Research Society Conference, 2002, pp. 354-358.
[121]M. Uray, D. Skocaj, P. M. Roth, H. Bischof, and A. Leonardis, "Incremental LDA learning by combining reconstructive and discriminative approaches," in Proceedings of British Machine Vision Conference, 2007, pp. 272–281.
[122]A. Varga, H. J. M. Steeneken, M. Tomlinson, and D. Jones, "The NOISEX-92 study on the effect of additive noise on automatic speech recognition," DRA Speech Research Unit, Malvern, England, Tech. Rep, 1992.
[123]M. Vidyasagar, Nonlinear Systems Analysis. Upper Saddle River, NJ, USA: Prentice-Hall Inc. , 1993.
[124]J.-S. Wang and C. S. G. Lee, "Self-adaptive neuro-fuzzy inference systems for classification applications," IEEE Transactions on Fuzzy Systems, vol. 10, pp. 790-802, 2002.
[125]J.-S. Wang and C. S. G. Lee, "Self-adaptive recurrent neuro-fuzzy control of an autonomous underwater vehicle," IEEE Transactions on Robotics and Automation, vol. 19, pp. 283-295, 2003.
[126]L.-X. Wang and J. M. Mendel, "Fuzzy basis functions, universal approximation, and orthogonal least-squares learning," IEEE Transactions on Neural Networks, vol. 3, pp. 807-814, 1992.
[127]S. Wang, J. Yang, N. Chen, X. Chen, and Q. Zhang, "Human activity recognition with user-free accelerometers in the sensor networks," in Proceedings of 2005 International Conference on Neural Networks and Brain, ICNNB'05, Beijing, China, 2005, pp. 1212-1217.
[128]J. A. Ward, P. Lukowicz, G. Troster, and T. E. Starner, "Activity recognition of assembly tasks using body-worn microphones and accelerometers," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, pp. 1553-1566, 2006.
[129]P. J. Werbos, "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences," Ph.D. dissertation, Harvard University, 1974.
[130]B. Widrow and S. D. Stearns, Adaptive Signal Processing. Upper Saddle River, NJ, USA: Prentice-Hall Inc. , 1985.
[131]R. J. William and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Computation, vol. 1, pp. 270-280, 1989.
[132]R. J. Williams and D. Zipser, "A learning algorithm for continually running recurrent neural networks," Neural Computation, vol. 1, pp. 270–280, 1989.
[133]J. Y. F. Yam and T. W. S. Chow, "Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficients," IEEE Transactions on Neural Networks, vol. 12, pp. 430-434, 2001.
[134]J. Ye, R. Janardan, C. H. Park, and H. Park, "An optimization criterion for generalized discriminant analysis on undersampled problems," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 982-994, 2004.
[135]J. Ye, Q. Li, H. Xiong, H. Park, R. Janardan, and V. Kumar, "IDR/QR: An incremental dimension reduction algorithm via QR decomposition," IEEE Transactions on Knowledge and Data Engineering, vol. 17, pp. 1208-1221, 2005.
[136]S. H. Zak, Systems and control. Oxford University Press New York, 2003.
[137]J. Zhang and A. J. Morris, "Recurrent neuro-fuzzy networks for nonlinear process modeling," IEEE Transactions on Neural Networks, vol. 10, pp. 313-326, 1999.
[138]Q. Zhu and L. Guo, "Stable adaptive neurocontrol for nonlinear discrete-time systems," IEEE Transactions on Neural Networks, vol. 15, pp. 653-662, 2004.