臺灣博碩士論文加值系統

基於統計學習理論的支持向量機，在機器學習領域具有最小模型複雜度，高泛化性能等優勢，無論在分類或迴歸的應用都深具潛力。但鑑於其模型是一二次最佳規劃模型，計算複雜度O(n2)，將隨訓練樣本增加而導致維度災難。此一被稱為序列最小優化(Sequential minimal optimization, SMO)的演算法旨在降低其計算複雜度，把整個二次規劃問題分解成為很多易於處理的最小規模問題，一次只優化兩個樣本，並且用解析的方法迭代進行，以最小的計算複雜度最終達到系統的最佳解。本研究基於現有的支向機分類SMO演算法，援引Maximal Gain的工作集選擇方法，拓展成為一個支向機迴歸SMO演算法，以饗眾多迴歸分析使用者之期望。

Based on the statistical learning theory, the support vector machine is excellent with its features in low model complexity and high generalization ability, and is highly potential for the applications in both pattern recognition and function approximation. The quadratic expression in its original model intrinsically corresponds to a high computational complexity in O(n2), and leads it to a curse of dimensionality with the increasing training instances. By employing the sequential minimal optimization (SMO) algorithm which subdivides the big integrated optimization into a series of small two-instance optimization, the computation of the quadratic programming can be effectively reduced, and reach rapidly the optimal solution. With some improved findings, the study extends the SMO for SVM classifications to that for SVM regression. The development would be advantageous to the applications of function approximation.

第一章　緒　論 1
1.1　前言 1
1.2　研究動機 1
1.3　研究目的 1
1.4　序列最小優化演算法的重要性 1
1.5　序列最小優化演算法特點和優勢 4
第二章　支持向量機迴歸 6
2.1　支持向量迴歸概述 6
2.2　損失函數與ε-tube 6
2.3　核函數 9
第三章　支持向量機迴歸之SMO演算法建模 11
3.1　 SMO演算法概述 11
3.1.1　兩個拉格蘭日乘子的優化問題 13
3.1.2　拉格蘭日乘子工作及選擇 16
3.1.3　計算閥值 18
3.2　支持向量機迴歸之序列最小優化演算法 19
3.3　最大增益在工作集的使用上 20
第四章　SMO Regression實現結果 28
第五章　結論與建議 29
參考文獻 30

[1] V. N. Vapnik, in: The nature of statistical learning theory. springer-
verlag, New York, 1995.
[2] C. Corts, V. N. Vapnik, Support vecter networks, Machine Learning,
Vol.20, PP.273-297(1995).
[3] C. J. C. Burges, “A tutorial on support vector machines for pattern
recognition,” Data mining and knowledge discovery, 2(2):121–167,
1998.
[4] Platt, John. "Sequential minimal optimization: A fast algorithm
for training support vector machines." (1998).
[5] Fan, Rong-En, Pai-Hsuen Chen, and Chih-Jen Lin. "Working
set selection using second order information for training
support vector machines." The Journal of Machine Learning
Research 6 (2005): 1889-1918.
[6] Bottou, Léon, and Chih-Jen Lin. "Support vector machine
solvers." Large scale kernel machines (2007): 301-320.
[7] B. Schölkopf, A. J. Smola, Learning with kernels, MIT press
Cambridge, MA, 2002.
[8] Smola, Alex J., and Bernhard Schölkopf. "A tutorial on support
vector regression." Statistics and computing 14.3 (2004):
199-222.
[9] Shevade, Shirish Krishnaj, et al. "Improvements to the SMO
algorithm for SVM regression." Neural Networks, IEEE
Transactions on 11.5 (2000): 1188-1193.
[10] Barbero, Álvaro, and José R. Dorronsoro. "A simple maximum
gain algorithm for support vector regression." Bio-Inspired
Systems: Computational and Ambient Intelligence. Springer
Berlin Heidelberg, 2009. 73-80.