臺灣博碩士論文加值系統

隨著科技的不斷進步，人們的生活正日益受惠。近年來，人工智慧（AI）相關技術發展迅速，機器學習（Machine Learning）和深度學習（Deep Learning）已廣泛應用在不同的領域。常見的應用包括圖像辨識(Image Recognition)、語音辨識(Voice Recognition)、疾病偵測(Disease Detection)或是今日常見的自駕車系統之中。這些應用利用即時收集的大數據進行預測和分析。其中高效能晶片扮演著關鍵角色，提供了必要的運算能力。
隨著製程的不斷演進和第五代移動通訊技術（5G）的日益成熟，物聯網（Internet of Things, IoT）應用正迅速擴展。從穿戴式載具到智慧居家系統，再到衛生醫療和車聯網，各種相關應用正在改變我們的生活方式和工作模式。這些應用需要即時性的數據處理和分析，這就使得邊緣運算（Edge Computing）日益重視。因此，裝置上的機器學習晶片變得更加必要，以滿足對即時訓練和高速分類功能的需求。從而更好地支持終端設備的使用。這種整合機器學習模型到終端設備的趨勢，將更符合經濟效益。本論文提出高效能三模式線性迴歸晶片之通用型硬體架構設計，達到完整的硬體訓練暨分類解決方式。
本論文將提出三項技術，在保持模型準確率的狀況下同時提升線性迴歸演算法機器學習的訓練效率，(1)透過特徵與標籤之間的相關性，來判斷訓練資料的重要性，利用重要性差異，來達成訓練資料的篩選，減少訓練資料的總量，最終降低機器學習訓練的難度。(2)透過彈性選擇脊迴歸模型的參數，針對不同的訓練資料集進行訓練過程的調整，並透過硬體的設計，在不增加訓練難度的狀況下，得到最高準確度的機器學習模型。(3)透過降低特徵數量及群外點移除的功能，提出高效率的訓練策略，針對不一樣的訓練資料集進行調整，降低整體機器學習訓練難度，加速機器學習訓練。

Technological advancements have been continuously improving people''s lives. In recent years, AI-related technologies have developed rapidly, and machine learning and deep learning have been widely applied in various fields. Common applications include image recognition, voice recognition, disease detection, and self-driving car systems. These applications use real-time collected big data for prediction and analysis. High-performance chips play a key role in providing the necessary computing power.
With the advancement of technology and 5G, IoT applications are expanding rapidly. From wearables to smart homes to healthcare and connected cars, these applications are changing our lives and work. They require real-time data processing and analysis, which makes edge computing important. On-device machine learning chips are becoming necessary to meet the demand for real-time training and classification. This trend will be more cost-effective. This paper proposes a general-purpose hardware architecture design for a high-performance three-mode linear regression chip, which achieves a complete hardware training and classification solution.
This paper proposes three techniques to improve machine learning training efficiency. (1) Filter training data by judging its importance. (2) Adjust the training process for different datasets with parameter selection and hardware design. (3) Reduce the number of features and remove outliers to further reduce training difficulty. These techniques achieve a highly efficient machine learning training method.

論文審查書 i
誌謝 ii
摘要 iii
Abstract iv
圖次 vii
表次 ix
第1章 緒論 1
1.1背景 1
1.2動機與設計目標 2
1.3論文架構 4
第2章 線性迴歸介紹 5
2.1線性迴歸理論 5
2.2普通最小平方法線性迴歸演算法 6
2.3最小絕對值收斂和選擇算子線性迴歸演算法 7
2.4脊迴歸演算法 7
第3章 高效能三模式線性迴歸晶片之通用型硬體架構設計 9
3.1系統介紹 9
3.1.1軟硬整合式訓練及分類系統 9
3.1.2控制流程及外部參數介紹 14
3.1.3資料前處理及特徵資料集量化方式 15
3.2技術一:基於斯皮爾曼相關係數的機器學習資料前處理 20
3.2.1機器學習資料前處理降低樣本的重要性與原理 20
3.2.2 斯皮爾曼相關係數演算法介紹 21
3.2.3基於相關係數之機器學習資料前處理 23
3.2.4對多類機器學習模型準確度之分析 27
3.3技術二:自適應脊迴歸正則化參數選擇之設計 28
3.3.1 脊迴歸正則化參數重要性與原理 28
3.3.2正則化參數選擇之影響 30
3.3.3自適應正則化參數驗證設計 33
3.3.4準確度之分析 42
3.4技術三:高效率訓練加速策略 44
3.4.1群外點及低相關特徵移除之重要性 44
3.4.2基於資料統計之群外點移除器設計 45
3.4.3基於皮爾森積動差之低相關特徵排除器設計 49
3.4.4延遲與準確度分析 52
第4章 晶片實現 54
4.1標準元件晶片設計流程 54
4.2代碼覆蓋率 55
4.3合成結果分析 56
4.4可測試性晶片設計與分析 60
4.5晶片佈局成果 61
4.6效能比較 62
第5章 結語與未來展望 65
5.1結語 65
5.2未來展望 65
參考文獻 67

[1]S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[2]S. Drummond, A. Joshi, and K. A. Sudduth, "Application of neural networks: precision farming," in 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No. 98CH36227), 1998, vol. 1: IEEE, pp. 211-215.
[3]J. T. Kilmer and R. L. Rodríguez, "Ordinary least squares regression is indicated for studies of allometry," Journal of evolutionary biology, vol. 30, no. 1, pp. 4-12, 2017.
[4]R. G. D. Steel and J. H. Torrie, "Principles and procedures of statistics," Principles and procedures of statistics., 1960.
[5]S. Wang, B. Ji, J. Zhao, W. Liu, and T. Xu, "Predicting ship fuel consumption based on LASSO regression," Transportation Research Part D: Transport and Environment, vol. 65, pp. 817-824, 2018.
[6]R. Tibshirani, "Regression shrinkage and selection via the lasso," Journal of the Royal Statistical Society Series B: Statistical Methodology, vol. 58, no. 1, pp. 267-288, 1996.
[7]T. C. Carneiro, P. A. Rocha, P. C. Carvalho, and L. M. Fernández-Ramírez, "Ridge regression ensemble of machine learning models applied to solar and wind forecasting in Brazil and Spain," Applied Energy, vol. 314, p. 118936, 2022.
[8]A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, pp. 80-86, 2000.
[9]M. Arashi, M. Roozbeh, N. A. Hamzah, and M. Gasparini, "Ridge regression and its applications in genetic studies," Plos one, vol. 16, no. 4, p. e0245376, 2021.
[10][online]https://archive.ics.uci.edu/ml/index.php
[11][online]https://www.kaggle.com/datasets
[12][online]https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
[13]J. A. Hartigan and M. A. Wong, "Algorithm AS 136: A k-means clustering algorithm," Journal of the royal statistical society. series c (applied statistics), vol. 28, no. 1, pp. 100-108, 1979.
[14]J. Neyman, "On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection," in Breakthroughs in Statistics: Methodology and Distribution: Springer, 1992, pp. 123-150.
[15]X. Y. Shih and Y. Y. Lu, "Systematic and Flexible Genetic-Algorithm-Based Feature Reduction for Decision Tree ML-Validation," 2022 IET International Conference on Engineering Technologies and Applications (IET-ICETA), 14-16 Oct. 2022, pp. 1-2.
[16]F. Song, D. Mei, and H. Li, "Feature selection based on linear discriminant analysis," in 2010 international conference on intelligent system design and engineering application, 2010, vol. 1: IEEE, pp. 746-749.
[17]Y. Saeys, T. Abeel, and Y. Van de Peer, "Robust feature selection using ensemble feature selection techniques," in Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2008, Antwerp, Belgium, September 15-19, 2008, Proceedings, Part II 19, 2008: Springer, pp. 313-325.
[18]I. M. Nasir et al., "Pearson correlation-based feature selection for document classification using balanced training," Sensors, vol. 20, no. 23, p. 6793, 2020.
[19]Joshi, Arpita, and Nurit Haspel. "A novel data instance reduction technique using linear feature reduction," Journal of Artificial Intelligence and Systems, 2.1 (2020), pp. 191-206, 2020
[20]C.-F. Tsai, W.-C. Lin, Y.-H. Hu, and G.-T. Yao, "Under-sampling class imbalanced datasets by combining clustering analysis and instance selection," Information Sciences, vol. 477, pp. 47-54, 2019.
[21]M. O. Arowolo, M. O. Adebiyi, A. A. Adebiyi, and O. Olugbara, "Optimized hybrid investigative based dimensionality reduction methods for malaria vector using KNN classifier," Journal of Big Data, vol. 8, no. 1, pp. 1-14, 2021.
[22]J. L. Myers, A. D. Well, and R. F. Lorch Jr, Research design and statistical analysis. Routledge, 2013.
[23]W. Du, Y. S. Han, and S. Chen, "Privacy-preserving multivariate statistical analysis: Linear regression and classification," in Proceedings of the 2004 SIAM international conference on data mining, 2004: SIAM, pp. 222-233.
[24]M. A. Friedl and C. E. Brodley, "Decision tree classification of land cover from remotely sensed data," Remote sensing of environment, vol. 61, no. 3, pp. 399-409, 1997.
[25]G. M. Foody and A. Mathur, "Toward intelligent training of supervised image classifications: directing training data acquisition for SVM classification," Remote Sensing of Environment, vol. 93, no. 1-2, pp. 107-117, 2004.
[26]L. Wang, J. Zhao, and Y. Zhang, "Ridge-regression Echo State Network for Effluent Ammonia Nitrogen Prediction," in 2022 China Automation Congress (CAC), 2022: IEEE, pp. 6509-6512.
[27]M. Byrtek, F. O''Sullivan, M. Muzi, and A. M. Spence, "An adaptation of ridge regression for improved estimation of kinetic model parameters from PET studies," IEEE transactions on nuclear science, vol. 52, no. 1, pp. 63-68, 2005.
[28]J. Stewart, D. K. Clegg, and S. Watson, Multivariable calculus. Cengage Learning, 2020.
[29]M. N. Daikpor, "Arithemetic Operations on Elliptic Curve Defined over Radix-7 Synmetrical Quaternary Signed-Digit Finite Fields and their Application in Secure Communication Systems," University of Lagos (Nigeria), 2010.
[30]J. Yang, S. Rahardja, and P. Fränti, "Outlier detection: how to threshold outlier scores?," in Proceedings of the international conference on artificial intelligence, information processing and cloud computing, 2019, pp. 1-6.
[31]K. M. Ting, B.-C. Xu, T. Washio, and Z.-H. Zhou, "Isolation distributional kernel: A new tool for kernel based anomaly detection," in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 198-206.
[32]M. Idros, S. Ali, and M. S. Islam, "Condition based engine oil degradation monitoring system, synthesis and realization on ASIC," in 2014 IEEE International Conference on Semiconductor Electronics (ICSE2014), 2014: IEEE, pp. 84-87.
[33]M. Idros, A. Razak, M. Azmi, S. Suliman, A. Halim, and N. Khairudin, "Logic Design for Linear Regression Model Using ASIC in Engine Oil Degradation Monitoring System," Journal of Telecommunication, Electronic and Computer Engineering (JTEC), vol. 9, no. 2-8, pp. 67-71, 2017.