臺灣博碩士論文加值系統

虛實融合系統(Cyber-Physical System, CPS)為工業4.0(Industry 4.0)智慧製造核心系統，整合大數據(Big Data)、工業物聯網(Industrial Internet of Things, IIoT)、人工智慧(Artificial Intelligence, AI)和雲端運算(Cloud Computing)等技術使製造智慧化，透過生產參數及生產狀態資料的即時擷取、過濾、前處理、分析與分析結果的反饋，能夠提升生產效能，降低生產成本、增進生產品質及提高生產產量。機器學習(Machine Learning)為CPS中常用的資料分析技術，它是人工智慧的一個分支，旨在透過以往的大量資料找到隱藏在資料中的資料模式或知識。但一般而言，若不使用特殊工具，機器學習上必須具有些許程式設計背景才能實現。因此，本論文嘗試降低實現機器學習資料分析的門檻，以提升實作效率。
本論文以基於全球資訊網(Web)的 CPS建構輔助系統“PINE”為基礎，聚焦於設計及實作該系統之資料分析機器學習建模功能，整合PINE的網站開發框架及機器學習套件，完成機器學習建模及模型共享架構，並製作測試案例以確保所實作的功能與設計規格相符。支援的機器學習模型包括支援向量機(Support Vector Machine, SVM)、支援向量迴歸(Support Vector Regression, SVR)、深度神經網路(Deep Neural Network, DNN)及卷積神經網路(Convolutional Neural Network, CNN)等。機器學習建模功能讓使用者透過簡單的網頁操作介面即可針對資料建立機器學習分析模型，並產生分析模型架構之程式原始碼。這可降低機器學習建模相關技術之學習成本，簡化建模過程並提升建模效率；另外，分析模型共享功能可讓使用者透過瀏覽器界面上傳及下載建模之超參數、建模結果及模型之使用情境，並可透過搜尋功能讓使用者找出最適當的共享資料分析模型，節省從頭開始建模的成本。

The Cyber-Physical System (CPS) is critical for smart manufacturing of the Industry 4.0 vision. It integrates advanced technologies, such as Industrial Internet of Things (IIoT), Big Data, Cloud Computing, and Artificial Intelligence (AI), to improve production efficiency, reduce production costs and increase production quality. The CPS collects, preprocesses, and analyzes data of manufacturing parameters and states, and feedbacks analyzed results, which contain deep-insight information of the data, to the manufacturing system for realizing online smart control of the whole system. Machine learning plays an important role in data analysis of the CPS. Its objective is to find specific patterns or hidden knowledge in the data through analyzing a large amount of historical data. However, it takes much effort to realize data analysis using machine leaning mechanisms. This motivates us to design and implement machine learning modeling modules for a web-based CPS construction assistant “PINE”. The modules integrate PINE’s web-based CPS development framework and common machine learning packages. The supported machine learning models include Support Vector Machine (SVM), Support Vector Regression (SVR), Deep Neural Network (DNN), and Convolutional Neural Network (CNN). The modules allow users to set and select hyper-parameters for machine learning modeling through simple web interfaces. The source code related to the modeling is generated, so that users can modify it at will. Moreover, the hyper-parameter tuning process for training the model, the best model ever trained, and the model usage information (i.e., the input, output and function of the model) are kept for model reusing and sharing. By establishing metadata for models, users can even search the most appropriate model with certain functions to be applied to specific applications. Many test cases are made and tested to make sure that the implementation meets the design specifications. We believe the design and implementation of machine learning modeling can facilitate users to construct a CPS.

中文摘要 I
Abstract II
致謝 III
目錄 IV
圖目錄 VI
表目錄 VIII
一、 緒論 1
1.1 研究背景與動機 1
1.2 研究目的與貢獻 3
1.3 相關技術研究 3
1.4 論文結構 4
二、 背景知識及相關研究 5
2.1 PINE 5
2.1.1 PINE簡介 5
2.1.2 PINE系統架構 5
2.2 支援向量機(Support Vector Machine) 7
2.2.1 支援向量機簡介 7
2.2.2 支援向量分類(Support Vector Classification) 7
2.2.2.1 Soft-Margin 9
2.2.2.2 多分類 10
2.2.2.3 核心函數 10
2.2.2.4 SVC超參數說明 11
2.2.3 支援向量迴歸(Support Vector Regression) 12
2.2.3.1 SVR超參數說明 13
2.3 深度學習(Deep Learning) 14
2.3.1 深度學習介紹 14
2.3.1.1 人工神經網路(Artificial Neural Network, ANN) 14
2.3.1.2 激勵函數(Activation Function) 16
2.3.1.3 反向傳播演算法(Back-Propagation Algorithm) 17
2.3.2 深度神經網路(Deep Neural Network, DNN) 18
2.3.3 卷積神經網路(Convolutional Neural Network, CNN) 19
2.4 自動機器學習(AutoML) 25
2.4.1 神經網路架構搜索(Neural Architecture Search, NAS) 25
2.4.2 Google Cloud AutoML 28
三、 研究方法 31
3.1 功能分析 31
3.1.1 功能需求 31
3.1.2 使用案例 32
3.2 功能架構與設計 34
3.2.1 機器學習建模 34
3.2.2 分析模型共享 36
3.3 功能實作 38
3.3.1 開發環境 38
3.3.2 實作成果 39
3.3.2.1 機器學習建模 39
3.3.2.1.1資料選取 39
3.3.2.1.2前處理 41
3.3.2.1.3 分析 43
3.3.2.1.4 結果呈現 45
3.3.2.2 分析模型共享 45
3.3.2.2.1 模型儲存 46
3.3.2.2.2 模型擷取 46
四、 系統測試與情境範例 47
4.1 系統測試 47
4.1.1 測試流程 47
4.1.2 測試案例步驟 49
4.1.3 測試結果 52
4.2 範例情境 54
4.2.1 情境一：焊點好壞分析模型訓練 54
4.2.2 情境二：焊點分析模型管理應用 60
五、 結論與未來展望 63
六、 參考文獻 64

[1] Saurabh Vaidya, Prashant Ambad, and Santosh Bhosle. "Industry 4.0–a glimpse." Procedia Manufacturing 20 (2018): 233-238.
[2] Andrew McAfee, et al. "Big data: the management revolution." Harvard Business Review 90.10 (2012): 60-68.
[3] Jayavardhana Gubbi, et al. "Internet of Things (IoT): A vision, architectural elements, and future directions." Future Generation Computer Systems 29.7 (2013): 1645-1660.
[4] Emiliano Sisinni, et al. "Industrial internet of things: Challenges, opportunities, and directions." IEEE Transactions on Industrial Informatics 14.11 (2018): 4724-4734.
[5] Stuart J. Russell, and Peter Norvig. Artificial intelligence: a modern approach. Malaysia; Pearson Education Limited, (2016).
[6] Jay Lee, Behrad Bagheri, and Hung-An Kao. "A cyber-physical systems architecture for industry 4.0-based manufacturing systems." Manufacturing Letters 3 (2015): 18-23.
[7] Jehn-Ruey Jiang. "An improved cyber-physical systems architecture for Industry 4.0 smart factories." Advances in Mechanical Engineering 10.6 (2018): 1687814018784192.
[8] Chih-Wei Hsu, Chih-Chung Chang, and Chih-Jen Lin. "A practical guide to support vector classification." (2003).
[9] Geoffrey E. Hinton, and Ruslan R. Salakhutdinov. "Reducing the dimensionality of data with neural networks." Science 313.5786 (2006): 504-507.
[10] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in Neural Information Processing Systems. (2012).
[11] Karen Simonyan, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv Preprint arXiv:1409.1556 (2014).
[12] SVC hyperparameters description: https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
[13] Smola, Alex J., and Bernhard Schölkopf. "A tutorial on support vector regression." Statistics and Computing 14.3 (2004): 199-222.
[14] SVR hyperparameters description: https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html
[15] Support vector machines: https://www.jeremyjordan.me/support-vector-machines/
[16] An Introduction to Deep Learning: https://blog.algorithmia.com/introduction-to-deep-learning/
[17] Neurons introduction: http://www.hkpe.net/hkdsepe/human_body/neuron.htm
[18] Convolutional Neural Network with many layers: https://www.mathworks.com/solutions/deep-learning/convolutional-neural-network.html
[19] DNN hyperparameters description: https://keras.io/layers/core/
[20] CNN hyperparameters description: https://keras.io/layers/convolutional/
[21] Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. "Automated Machine Learning-Methods, Systems, Challenges." Automated Machine Learning (2019).
[22] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. "Neural architecture search: A survey." arXiv Preprint arXiv:1808.05377 (2018).
[23] Google Cloud AutoML: https://cloud.google.com/automl/?hl=Zh-TW