跳到主要內容

臺灣博碩士論文加值系統

(3.95.131.146) 您好!臺灣時間:2021/07/29 01:38
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:鄭盛倫
研究生(外文):Sheng-Lon Jheng
論文名稱:混合機器學習與其應用
論文名稱(外文):Hybrid Machine Learning and Its Applications
指導教授:鄭錦聰鄭錦聰引用關係
指導教授(外文):Jin-Tsong Jeng
學位類別:碩士
校院名稱:國立虎尾科技大學
系所名稱:光電與材料科技研究所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2009
畢業學年度:97
語文別:英文
論文頁數:153
中文關鍵詞:支撐向量機最小平方支撐向量機高斯程序稀疏高斯程序回歸分析LTS
外文關鍵詞:support vector machineleast squares support vector machineGaussian processessparse approximation Gaussian processesregression analysisLTS
相關次數:
  • 被引用被引用:0
  • 點閱點閱:438
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
近年來,人工智慧發展的非常迅速;其研究領域包含了機器學習、類神經網路、支撐向量機、最小平方支撐向量機、高斯程序等等。一般的機器學習方法都具有分類及回歸分析的功能。分類是經由從訓練集中學習後,經由結果來預測資料的分布類別;回歸分析是一個數值分析方法,目的是希望取得資料集之間的特定關係,但最重要的是希望可以找出函式來表示資料集之間的特定關係,而藉由找出的函式來建模數學模型就可預測其他資料集。本論文中是針對回歸分析來進行分析及模擬,所以將對機器學習方法中的支撐向量機回歸及高斯程序回歸進行混合而形成兩階段的架構來進行應用,也會將數學模型如LTS與高斯程序回歸及LTS與稀疏高斯程序回歸混合而成的架構來進行應用;本篇論文中所使用的應用有混沌系統、生物系統。混沌系統包括時間序列系統、勞倫斯系統;生物系統包括細胞核自動回朔系統及MAPK系統。
一般普通的機器學習方法對通常都是針對訓練集裡沒有雜訊或離群值來進行學習及預測,而這樣產生出的結果都是相當理想的,而當在訓練集之中加入雜訊及離群值來進行學習及預測時就會對系統的結果產生很大的影響。由此可知當訓練集中加入了雜訊或離群值時,以一般的機器學習方法來進行學習及預測的成果並不是很好,而之前許多的研究中,也有許多的學者提出了許多經過改進的機器學習方法,不過如果就以單一架構的機器學習方法來對訓練集之中含有雜訊及離群值來進行學習及預測,效能還是不佳。所以本論文的目的是混合不同的機器學習方法並與其他單一機器學習方法進行效能比較,除了希望混合特定的機器學習方法可以在訓練集含有雜訊或離群值來進行學習及預測也可以得到良好的結果之外,也證明其雙階段架構效能要比單一機器學習架構來得好。
本論文主要的結果是將混合兩階段學習架構對訓練集含有雜訊或離群值來進行學習及預測,並與最小平方支撐向量機回歸及高斯程序回歸進行比較。在提出的兩階段學習架構中,在第一階段中,以支撐向量機回歸或LTS方法來過濾掉訓練集中的雜訊或離群值,而由於大量的雜訊及離群值從訓練集中幾乎都被移除,所以大量雜訊及離群值產生的影響已降低許多。在第二階段,訓練集中剩餘的其他資料就直接由高斯程序回歸或稀疏高斯程序回歸來進行學習及預測。根據模擬結果可得知當訓練集中含有大量的雜訊或離群值時,混合機器學習方法的效能比最小平方支撐向量機回歸、高斯程序回歸、稀疏高斯程序回歸(PPA)及稀疏高斯程序回歸(SoD)好;總結來說,本論文所提出的雙階段機器學習架構的確可得出比單階段架構機器學習架構更好的結果。
在本論文中未提及的機器學習方法還有許多種,能進行混合的機器學習方法還是相當多的,每一種機器學習方法都有其優點及缺點,但是要如何進行混合建模來對系統取得平衡來是一個相當重要的課題,不過由於現在機器學習發展的相當快,隨時可能有新的機器學習方法出現,如何善用新的及舊的機器學習方法來進行混合架構建模是未來可以發展的方向之一。
In recently decade years, the development of artificial intelligence was very fast. And there are many approaches in this research fields such as machine learning, artificial neural network (ANN), support vector machines (SVM), least squares support vector machines (LS-SVM), Gaussian processes (GPs), and so forth. In general, the machine learning methods have two major capability are classification and regression analysis. Statistical classifier, via learning from training data sets to predicting the distribution of class through the results. Regression analysis is a numerical analysis method, It expect to obtain the special relation between data sets, but the most important thing is want to find the function that can present the specific relation between data sets, and by using this function that we can establish math model to predict other data sets. In this thesis, regression analysis is to be aimed at analysis and simulate the applications. Hence, we proposed that hybrid support vector machines for regression (SVR) and Gaussian processes for regression (GPR) to form the two-stage structure and apply to applications. Besides, we also hybrid the math models; least trim squares (LTS) hybrid with GPR, and LTS with hybrid sparse approximation Gaussian processes for regression (Sparse GPR) to generate the two-stage structure and apply to applications too. The applications using in this thesis are chaotic systems, bioinformatics systems. Chaotic systems include Mackey-Glass time-series systems and Lorenz chaotic systems. Bioinformatics systems include the molecular autoregulatory feedback loop systems (MAFLS) and the mitogen activated protein kinases (MAPK) systems.

The general machine learning method usually trains and predicts under the training data sets without noise or outliers; therefore, the results are very ideal. But if we trains and predicts when training data sets involve the noise and outliers, unfortunately the system performances are affected very huge under this condition. That is, when training data sets have noise or outliers, the common machine learning method performs not well. Hence, there are some researchers proposed the new machine learning methods, however if we use one-stage structure machine learning methods to training and predicting when training data sets involve the noise and outliers, the system performance still not good enough. So the purposes of this thesis are hybrid different machine learning method then compare system performance with other single structure machine learning methods, in addition to we expect to our proposed method can obtain good results, also prove that two-stage learning structure performance is better than single machine learning structure.

In this thesis, the main results are to proposed hybrid two-stage learning structure for training and predicting when training data sets have noise or outliers, and compare the results with LS-SVM for regression and GPR. In the proposed approach, there are two-stage strategies. In the stage 1, the SVR or LTS approach is used to filter out the noise or outliers in the training data set. Because of the large noise points and outliers in the training data set are almost removed, so the large noise effect is to reduce. In the stage 2, the rest of the training data sets is directly training and predicting by the GPR or Sparse GPR. According to the simulation results, the performance of the proposed approach is better than the LS-SVR, GPR, Sparse GPR (PPA) and Sparse GPR (SoD) when a large number of noise or outliers are existed in the training data sets. In summary, the proposed two-stage strategies in this thesis actually perform better than single machine learning strategy.

In this thesis, there are many other machine learning methods that did not mention it, and many of it can mix together to obtain the good performance. That is, each machine learning methods have advantages and disadvantages of itself, but one of the important issues is that how to hybrid ML approaches to ensure that system modeling obtains good balance. However, nowadays the development of artificial intelligence was very fast, we may see new machine learning methods that appears anytime. One of the future developments is how to make a best use of new and old machine learning methods to modeling of hybrid structure.
摘要......................................................i
ABSTRACT................................................iii
ACKNOWLEGMENTS............................................v
LIST OF TABLES...........................................ix
LIST OF FIGURES.........................................xix

CHAPTER 1 INTRODUCTION.....................................1
1.1 Review of Machine Learning.......................1
1.2 Review of Support Vector Machines and Least Squares Support Vector Machines............................2
1.3 Review of Least Trimmed Squares..................5
1.4 Review of Gaussian Processes and Sparse Gaussian Processes .................................................5
1.5 Review of Chaotic Systems........................7
1.6 Review of Bioinformatics Systems.................8
1.7 Motivations and Contributions...................10
1.7.1 Motivations........................................10
1.7.2 Contributions......................................10
1.8 The organization of this thesis.................11
CHAPTER 2 RESEARCH APPROACHS AND ARCHITECTURE............12
2.1 Support Vector Machines Regression..............12
2.2 Least Squares Support Vector Machines Regression...............................................15
2.3 Gaussian Processes Regression...................17
2.4 The Hybrid Support Vector Machines and Gaussian Processes Architecture for Modeling of System with Noise
.........................................................20
2.5 The Hybrid Support Vector Machines and Gaussian Processes Architecture for Modeling of System with Noise and Outliers.............................................22
2.6 Why Use Support Vector Machines and Gaussian Processes Architecture...................................23
CHAPTER 3 HYBRID SVM-GPR FOR MODELING OF CHAOTIC TIME-SERIES SYSTEMS WITH NOISE AND OUTLIERS...................26
3.1 Introduction....................................26
3.2 The Hybrid Support Vector Machines and Gaussian Processes for Chaotic Time-Series Systems with Noise.....27
3.3 The Hybrid Support Vector Machines and Gaussian Processes for Chaotic Time-Series Systems with noise and outliers.................................................30
3.4 The Simulation of Time-Series Systems with Noise by Using SVR-GPR Structure...............................31
3.4.1 Example 1:Mackey-Glass time series equation.......31
3.5 The Simulation of Time-Series systems with Noise and Outliers by Using SVR-GPR Structure..................46
3.5.1 Example 1: Mackey-Glass time series equation.......46
3.5.2 Example 2: Lorenz chaotic systems equation.........48
3.6 Concluding Remarks ..............................59
CHAPTER 4 HYBRID SVR-GPR FOR MODELING OF MAFLS AND MAPK SYSTEMS WITH NOISE AND OUTLIERS..........................60
4.1 Introduction....................................60
4.2 The Hybrid Support Vector Machines and Gaussian Processes for The MAFLS and The MAPK Systems with Noise
.........................................................61
4.3 The Hybrid Support Vector Machines and Gaussian Processes for The MAFLS and The MAPK Systems with Noise and Outliers.................................................65
4.4 The Simulation of The MAFLS and The MAPK Systems with Noise by Using SVR-GPR Structure....................67
4.4.1 Example 1: MAFLS...................................67
4.4.2 Example 2: MAPK systems............................70
4.5 The Simulation of The MAFLS and The MAPK Systems with Noise and Outliers by Using SVR-GPR Structure.......81
4.5.1 Example 1: MAFLS...................................81
4.5.2 Example 2: MAPK systems............................83
4.6 Concluding remarks .............................95
CHAPTER 5 HYBRID LTS-GPR AND LTS-SPARSE GPR FOR MODELING OF CHAOTIC TIME-SERIES SYSTEMS WITH NOISE AND OUTLIERS ..96
5.1 Introduction...................................97
5.2 Research Approaches and Architecture 97
5.2.1 Sparse approximation Gaussian processes regression
........................................................97
5.2.2 Least Trimmed Squares ............................101
5.3 The Hybrid Least Trimmed Squares Method and Gaussian Processes for Chaotic Time-Series Systems with Noise..................................................103
5.4 The Hybrid Least Trimmed Squares Method and Gaussian Processes for Chaotic Time-Series Systems with Outliers ..............................................105
5.5 Why Use LTS and GPR Architecture..............106
5.6 The Hybrid Least Trimmed Squares Method and Sparse Approximation Gaussian Processes for Chaotic Time-Series Systems with Noise..............................107
5.7 The Hybrid Least Trimmed Squares Method and Sparse Approximation Gaussian Processes for Chaotic Time-Series Systems with Noise and Outliers.................109
5.8 Why Use LTS and Sparse GPR Architecture.......110
5.9 The Simulation of Time-Series Systems with Noise by Using LTS-GPR Structure ............................110
5.9.1 Example 1: Mackey-Glass time series equation.....110
5.9.2 Example 2: sinc function.........................114
5.10 The Simulation of Time-Series Systems with Noise and Outliers by Using LTS-GPR Structure................122
5.10.1 Example 1: Mackey-Glass time series equation....122
5.10.2 Example 2: sinc function........................124
5.11 The Simulation of Time-Series Systems with Noise by Using LTS-Sparse GPR Structure......................131
5.11.1 Example 1: Mackey-Glass time series equation 131
5.11.2 Example 2: sinc function........................133
5.12 The Simulation of Time-Series Systems with Noise and Outliers by Using LTS-Sparse GPR Structure.........139
5.12.1 Example 1: Mackey-Glass time series equation....139
5.12.2 Example 2: sinc function........................141
5.13 Concluding Remarks ...........................147
CHAPTER 6 CONCLUSIONS..................................148
REFERENCES.............................................150
[1]H. Drucker , C.J. Burges, L. Kaufman, A. Smola and V. Vapnik, “Support vector regression machines,” Neural Information Processing Systems, Vol. 9, pp. 155-161, 1997.
[2]J. A. K Suykens and J. Vandewelle, “Least squares support vector machine classifiers,” Neural Processing Letters, Vol. 9, No. 3, pp. 293-300, 1999.
[3]M. Espinoza, J. A. K Suykens and B. De Moor, “Short term chaotic time series prediction using symmetric LS-SVM regression,” Proc. of the 2005 International Symposium on Nonlinear Theory and Applications (NOLTA), pp. 606-609, Bruges, Belgium, Oct. 2005.
[4]J. A. K. Suykens, L. Lukas, P. Van Dooren, B. De Moor, and J. Vandewalle, “Least squares support vector machine classifiers: a large scale algorithm,” Proc. of the European Conference on Circuit Theory and Design (ECCTD''99), Stresa, Italy, pp. 839-842, Sep. 1999.
[5]J. A. K Suykens, T. Van Gestel, J. De Brabanter, B. De Moor and J. Vandewalle, Least Squares Support Vector Machines, World Scientific, Singapore, 2002.
[6]J. A. K. Suykens, L. Lukas, J. Vandewalle, “Sparse approximation using least squares support vector machines ,” IEEE International Symposium on Circuits and Systems (ISCAS 2000), Geneva, Switzerland, Vol. 2, pp. 757-760, May. 2000.
[7]H. F. Wang, D. j. Hu, “Comparison of SVM and LS-SVM for regression,” International Conference on Neural Networks and Brain (ICNN&B ''05), Vol. 1, No. 13-15, pp. 279 - 283, Oct. 2005.
[8] J. A. K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle, “Weighted least squares support vector machines: robustness and sparse approximation,” Neurocomputing, Special issue on fundamental and information processing aspects of neurocomputing, Vol. 48, No. 1-4, pp. 85-105, Oct. 2002.
[9]C. C. Chuang, S. F. Su, J. T. Jeng, and C. C. Hsiao, “Robust support vector regression networks for function approximation with outliers,” IEEE Trans. On Neural Networks, Vol. 13, No. 6, pp.1322-1330, 2002.
[10]P. J. Rousseeuw, “Least median of squares regression,” Journal of the American Statistical Association, Vol. 69, pp. 871-880, 1984.
[11] C. Edward Rasmussen and K. I. Williams, Gaussian Processes for Machine Learning, MIT Press, 2006.
[12]D. J. C. MacKey, “Introduction to Gaussian processes,” Technical report, University of Cambridge, http://www.inference.phy.cam.uk/mackey/gpB.pdf, 1997.
[13]C. E. Rasmussen, “Advanced lectures on machine learning,” Gaussian Processes in Machine Learning, pp. 63-71, 2003.
[14]A. J. Smola and P. L. Bartlett, “Sparse greedy Gaussian process regression,” Advances in Neural Information Processing Systems, Vol. 13, pp. 619-625, MIT Press, 2001.
[15]E. Snelson and Z. Ghahramani, “Sparse Gaussian processes using pseudo-inputs,” Advances in Neural Information Processing Systems, Vol. 18, pp. 1259-1266, MIT Press, 2006.
[16]M. Seeger, C. K. I. Williams, and N. Lawrence, “Fast forward selection to speed up sparse Gaussian process regression,” Proceeding of the 9th International Workshop on Artificial Intelligence and Statistics. Society for Artificial Intelligence and Statistics, 2003.
[17]J. Quinonero, C. E. Rasmussen, and C. K. I. Williams, “Approximation methods for Gaussian process regression,” Large-Scale Kernel Machines, pp. 203-224, MIT Press, 2007.
[18]J. Quinonero, and C. E. Rasmussen, “A unifying view of sparse approximate Gaussian process regression,” Journal of Machine Learning Research, Vol. 6, pp. 1935-1959, 2005.
[19] A. S. Weigend, and N. A. Gershenfeld, Time Series Prediction: Forecasting the Future and Understanding the Past, Addison-Wesley, Reading, MA, 1994.
[20] H. Leung, and T. Lo, “Chaotic radar signal processing over the sea,” IEEE Journal of Ocean Engineering, Vol. 18, pp. 287-295, 1993.
[21] Q. H. Zhang, and A. Benvensite, “Wavelet network,” IEEE Transactions on Neural Network, Vol. 37, No. 5, pp. 433-448, 2000.
[22] Zeng Zhaocui, Duan Yurong, Shao Guang, “Chaotic time series analysis based on radial basis function network,” Chinese Journal of application, Chinese University, Vol. 22, No. 6, pp. 113-120, 1999.
[23] Ye Meiying, Wang Xiaodong, “Chaotic time series using least squares support vector machine,” Chinese Physics, Vol. 13, No. 4, pp. 454-458, 2004.
[24] Ding Tao, Zhou Huicheng, “Local prediction for chaotic time series and its application,” Dalian Ligong Daxue Xubao, Vol. 44, No. 3, pp. 445-448, 2004.
[25]J.S. Roger Jang, “ANFIS: adaptive-network-based fuzzy inference system,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. 23, No. 3, pp. 665-685, 1993.
[26] L.P. Maguire, B. Roche, T.M. McGinnity, L.J. McDaid, “Prediction a chaotic time series using a fuzzy neural network,” Information Sciences, Vol. 112, pp. 125-136, 1998.
[27] H. Gu, H. W. Wang, “Fuzzy prediction of chaotic time series based on singular value decomposition,” Applied Mathematics and Computation, Vol. 185, No. 2, pp.1171-1185, 2007.
[28] E. N. Lorenz, “Deterministic non-periodic flow,” Journal of Atmospheric Sciences, Vol. 20, pp. 130-41, 1963.
[29] Z. Galias, P. Zgliczy′nski, “Computer assisted proof of chaos in the Lorenz equations,” Physics D, Vol. 115, pp. 165–188, 1998.
[30]H. H. McAdams and A. Arkin. “Stochastic mechanisms in gene expression,” Proc Natl. Acad. Sci. USA, Vol. 94, pp. 814-819, 1997.
[31]H. Schmidt and E. W. Jacobsen, “Identifying feedback mechanisms behind complex cell behavior,” IEEE Control Systems Magazine, Vol. 24, No. 4, pp. 91-102, 2004.
[32]Z. Szallasi, “Genetic network analysis in light of massively parallel biological data acquisition,” Pacific Symp. Biocomp. 99, Vol. 4, pp. 5-16, 1999.
[33]A. Wagner, “Evolution of gene networks by gene duplications: a mathematical model and its implications on genome organization,” Proc. Natl. Acad. Sci. USA, Vol. 91, pp. 4387-4391, 1994.
[34] D. Thieffry and R. Thomas, “Qualitative analysis of gene networks,” Pacific Symp. Biocomp. 98, Vol. 3, pp. 77-88, 1998.
[35]M. Shea and G. Ackers, “The control system of bacteriophage lambda: a physical-chemical model for generegulation,” J. Molec. Biol., Vol. 181, pp. 211-230, 1985.
[36]M. Wahde and J. Hertz, “Coarse-grained reversed engineering of genetic regulatory networks,” Biosystems (IPCAT99), Vol. 55, pp. 129-136, 1999.
[37] E. Andrianantoandro, S. Basu, D. K. Karig and R. Weiss, “Synthetic biology: new engineering rules for an emerging discipline,” Molecular Systems Biology, Vol. 2, pp. 2006-2006, 2006.
[38]S. Hooshangi, S. Thiberge, and R. Weiss, “Ultrasensitivity and noise propagation in a synthetic transcriptional cascade,” Proc. Natl. Acad Sci. USA, Vol. 102, pp. 3581-3586, 2005.
[39]J. T. Jeng, “Annealing robust fuzzy basis function for modeling with noise and outliers,” Int. J. Computer Applications in Technology, Vol. 27, Nos. 2/3, pp. 166-173, 2006.
[40] M. G. Wilkinson and J. B. A. Millar, “Control of the eukaryotic cell cycle by map kinase signaling pathways,” The FASEB Journal, Vol. 14, No. 14, pp. 2147–2157, Nov. 2000.
[41]C. Y. F. Huang and J. E. Ferrell Jr., “Ultrasensitivity in the mitogen-activated protein kinase cascade,” Proc. Natl. Acad. Sci. USA, Vol. 93, pp. 10078–10083, 1996.
[42]C. Widmann, G. Spencer, M. B. Jarpe, and G. L. Johnson, “Mitogen-activated protein kinase: conservation of a three-kinase module from yeast to human,” Physiol. Rev. Vol. 79, pp. 143–180, 1999.
[43]E. Klipp, R. Herwig, A. Kowald, C. Wierling and H. Lehrach, Systems Biology in Practice: Concepts, Implementation and Application, Wiley, 2005.
[44] D. M. Hawkins, Identification of Outliers, Chapman and Hall, 1980.
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top