臺灣博碩士論文加值系統

在現今這個資訊爆炸的時代，人工智慧中的深度學習，因為其高效的表現，被廣泛應用於生活的很多方面，特別是在製造業和高科技半導體製造業中，使用了很多監控傳感器，每天產生大量的原始信息，通過利用產生的信息，將深度學習應用到自動監管中，似乎非常有效，但是深度學習對數據很敏感。實際上，不平衡的數據集是製造業中的典型案例，這種數據集會使深度學習方法傾向於多數類，甚至完全忽略少數類，而導致預測錯誤。此外，製造業數據集中通常會出現噪音。很多研究都發現，數據集中的噪音會極大地導致分類準確率降低和預測結果不佳，這是傳統機器學習和深度學習分類算法的兩個困難的情況。
在本論文中，提供了一個深度學習的概述，並將深度學習應用於半導體製造故障預測中，使用了一個不平衡數據集和一個名為SECOM和SETFI的模擬數據集，而這兩個數據集都有噪音。此外本論文還開發了一個製造故障預測的工作流程，在這兩組半導體製造數據集上進行了實驗，針對這兩組數據集，我提出了不同的深度學習方法，分別是深度自動編碼器和微調人工神經網絡。對於處理目標數據集中的噪聲問題，部署了兩種技術，分別命名為特徵選擇和處理缺失數據，實驗採用隨機森林機器學習和互信息統計技術作為特徵選擇方法。在處理缺失數據時，採用了推算法。在研究結果中，為了證明所提出的方法的效果很好，首先將提出的方法與其他廣泛使用的機器學習和深度學習方法進行了比較，發現本研論文提出的方法比其他的方法要好；再來將本研究提出的方法與其他研究進行了比較，實驗結果顯示，提出的方法在一些性能指標上有相似的結果，甚至更好。詳細來說，使用深度自動編碼器的不平衡數據集SECOM的準確率為0.869 （recall 0.75，AUC 0.812，MCC 0.369），使用微調人工神經網絡的模擬數據集SETFI的準確率為0.596 （recall 0.731，AUC 0.596 ，MCC 0.201）。
綜上所述，本論文了解到深度自動編碼器是一個對於非常不平衡的數據集的前景模型，而深度神經網絡在使用調整技術時效果更好。此外，特徵選擇也是提高深度學習模型性能的重要方法，尤其是在有噪音的數據集上效果更好，通過降低特徵維度，在這些方法增強了所提出深度學習模型的學習能力。因此，它們在訓練模型時最大限度地減少了運行時間和硬件資源消耗。

Nowadays, Artificial intelligence, especially deep learning, is widely applied in many aspects of life owing to its highly effective performance in this era of information explosion. Particularly, in manufacturing and high-tech manufacturing such as semiconductors, many monitoring sensors are used and generate a lot of raw data daily. By using the generated data, the applications of deep learning into auto supervision seem to be very effective. However, deep learning is sensitive to data, because it learns from data. The fact that the imbalanced dataset in manufacturing is very typical. This kind of dataset will make the deep learning methods tend to have a bias towards the majority class, even ignore the minority class altogether lead to the wrong predictions. Besides, the noises usually appear in the manufacturing datasets. Lots of studies have figured out that noise in the dataset dramatically led to decreased classification accuracy and poor prediction results. These are the two hard cases for traditional machine learning and deep learning classification algorithms.
In this study, I provided an overview of deep learning and applied Deep learning into semiconductor manufacturing failure prediction with an imbalanced dataset and the simulation datasets named SECOM and SETFI; both of them had noise. Besides, I developed a work-flow on manufacturing failure prediction and conducted my experiments on these two types of semiconductor manufacturing datasets. I proposed different deep learning approaches methods for each data type named deep Auto-encoder and fine-adjustment deep Artificial neural network. For dealing with noise problem in the target datasets, I deployed two techniques named feature selection and handling missing data. My experiments used the Random Forest machine learning and Mutual Information Statistical Technique as feature selection methods. I used the impute method for handling the missing data. At the end of this study, in order to demonstrate my proposed method's results are good, firstly, I compared my proposed methods with other widely used machine learning and deep learning method. My proposed methods are outperforming the rest. Secondly, I compared my proposed methods with other studies. The output showed that my proposed methods have similar results and even better in some performance metrics. In detail, the imbalanced dataset SECOM using deep Auto-encoder has accuracy 0.869 (recall 0.75, AUC 0.812, MCC 0.369), the simulation dataset SETFI using fine-adjustment deep Artificial neural network has accuracy 0.596 (recall 0.731, AUC 0.596, MCC 0.201).
In conclusion, my experiments figure out that the deep Auto-encoder is a prospecting model for a very imbalanced dataset, and the deep neural network works better when using adjustment techniques. Besides, Feature selection is also an important method that improves the performance of a deep learning model, especially effective on the dataset with noise. By reducing the feature dimensions, these methods can enhance the learning ability of proposed deep learning models. As a result, they minimize the running time and hardware resource consumption while training model.

Title page i
Letter of Approval ii
Abstract in Chinese iii
Abstract in English v
Acknowledgments vii
Table of Contents viii
List of Tables x
List of Figures xi
List of Acronyms xiii
CHAPTER 1. INTRODUCTION 1
1.1 Background 1
1.2 Research questions 2
1.3 Contribution 3
1.4 Study organization 3
CHAPTER 2. METHODOLOGY AND LITERATURE REVIEW 5
2.1 Methodology 5
2.1.1 Approaches based on data 5
2.1.2 Approaches based on algorithms 9
2.2 Literature review 16
2.2.1 Relevant experiments 16
2.2.2 Summary of literature review 19
CHAPTER 3. PROPOSED METHODS 20
3.1 Handling missing data 21
3.2 Feature selection methods 22
3.2.1 Mutual Information for feature selection 22
3.2.2 Random forest for feature selection 23
3.3 Deep learning model building 24
3.3.1 Processing data 24
3.3.2 Modeling 28
3.3.3 Post-Modeling 33
3.4 Evaluation methods and metrics 36
3.4.1 K-fold cross-validation 36
3.4.2 Proposed performance metrics 38
CHAPTER 4. RESULTS 41
4.1 Optimal Number of selected features 41
4.1.1 Optimal Number of the feature on SECOM dataset 41
4.1.2 Optimal Number of the feature on SETFI dataset 43
4.2 Performance results 45
4.2.1 Performance on SECOM dataset 45
4.2.2 Performance on SETFI dataset 47
CHAPTER 5. DISCUSSION 50
5.1 Comparison with other methods 50
5.1.1 Comparison of SECOM dataset 50
5.1.2 Comparison of SETFI dataset 51
5.2 Comparison with other studies published lately 52
5.2.1 Comparison of SECOM dataset 52
5.2.2 Comparison of SETFI dataset 54
CHAPTER 6. CONCLUSION 55
Reference 57

[1]Hubel, D.H. and T.N. Wiesel, Receptive fields of single neurones in the cat's striate cortex. The Journal of physiology, 1959. 148(3): p. 574-591.
[2]Hopfield, J.J. and D.W. Tank, Computing with neural circuits: A model. Science, 1986. 233(4764): p. 625-633.
[3]LeCun, Y., et al., Object recognition with gradient-based learning, in Shape, contour and grouping in computer vision. 1999, Springer. p. 319-345.
[4]Schmidhuber, J., Deep learning in neural networks: An overview. Neural Networks, 2015. 61: p. 85-117.
[5]McCann, M. and A. Johnston, SECOM Data Set.
[6]Intel, A.Y., Manufacturing data: Semiconductor Tool Failure Isolation. 2008.
[7]Gilchrist, A., Industry 4.0: the industrial internet of things. 2016: Springer.
[8]Dang, T.K., et al., Future Data and Security Engineering: 4th International Conference, FDSE 2017, Ho Chi Minh City, Vietnam, November 29–December 1, 2017, Proceedings. Vol. 10646. 2017: Springer.
[9]Abu-Samah, A., et al., Failure Prediction Methodology for Improved Proactive Maintenance using Bayesian Approach. IFAC-PapersOnLine, 2015. 48(21): p. 844-851.
[10]He, H. and E.A. Garcia, Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, 2009. 21(9): p. 1263-1284.
[11]Johnson, J.M. and T.M. Khoshgoftaar, Survey on deep learning with class imbalance. Journal of Big Data, 2019. 6(1): p. 27.
[12]Zhu, X., X. Wu, and Q. Chen. Eliminating class noise in large datasets. in Proceedings of the 20th International Conference on Machine Learning (ICML-03). 2003.
[13]Gupta, S. and A. Gupta, Dealing with Noise Problem in Machine Learning Data-sets: A Systematic Review. Procedia Computer Science, 2019. 161: p. 466-474.
[14]Sukhbaatar, S. and R. Fergus, Learning from noisy labels with deep neural networks. arXiv preprint arXiv:1406.2080, 2014. 2(3): p. 4.
[15]Kerdprasop, K. and N. Kerdprasop, Tool sequence analysis and performance prediction in the wafer fabrication process. International Journal of Systems Applications, Engineering & Development, 2014. 8: p. 268-276.
[16]Alpaydin, E., Introduction to Machine Learning. 2010: The MIT Press.
[17]Marsland, S., Machine Learning: An Algorithmic Perspective, Second Edition. 2014: Chapman & Hall/CRC.
[18]Zhou, Z.-H., Y. Yu, and C. Qian, Evolutionary Learning: Advances in Theories and Algorithms. 2019.
[19]Basheer, I.A. and M. Hajmeer, Artificial neural networks: fundamentals, computing, design, and application. Journal of Microbiological Methods, 2000. 43(1): p. 3-31.
[20]Lau, M.M. and K.H. Lim. Investigation of activation functions in deep belief network. in 2017 2nd International Conference on Control and Robotics Engineering (ICCRE). 2017.
[21]Bolón-Canedo, V., N. Sánchez-Maroño, and A. Alonso-Betanzos, A review of feature selection methods on synthetic data. Knowledge and Information Systems, 2013. 34(3): p. 483-519.
[22]Guyon, I. and A. Elisseeff, An introduction to variable and feature selection. Journal of machine learning research, 2003. 3(Mar): p. 1157-1182.
[23]Madley-Dowd, P., et al., The proportion of missing data should not be used to guide decisions on multiple imputation. Journal of Clinical Epidemiology, 2019. 110: p. 63-73.
[24]Soley-Bori, M., Dealing with missing data: Key assumptions and methods for applied analysis. Boston University, 2013. 4: p. 1-19.
[25]Mawson, V.J. and B.R. Hughes, Deep learning techniques for energy forecasting and condition monitoring in the manufacturing sector. Energy and Buildings, 2020: p. 109966.
[26]Fernández-Fdz, D., J. López-Puente, and R. Zaera, Prediction of the behaviour of CFRPs against high-velocity impact of solids employing an artificial neural network methodology. Composites Part A: applied science and manufacturing, 2008. 39(6): p. 989-996.
[27]Pierre, B., Autoencoders, Unsupervised Learning, and Deep Architectures. 2012, PMLR. p. 37-49.
[28]Shen, C., et al. Imbalanced Data Classification Based on Extreme Learning Machine Autoencoder. in 2018 International Conference on Machine Learning and Cybernetics (ICMLC). 2018.
[29]Tao, S., et al. Bearing failure diagnosis method based on stacked autoencoder and softmax regression. in 2015 34th Chinese Control Conference (CCC). 2015.
[30]Yeh, Y.-C. and C.-Y. Hsu, Application of Auto-Encoder for Time Series Classification with Class Imbalance. 2019, EasyChair.
[31]Kauschke, S., M. Mühlhäuser, and J. Fürnkranz. Towards Semi-Supervised Classification of Event Streams via Denoising Autoencoders. in 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA). 2018.
[32]Aygun, R.C. and A.G. Yavuz. Network Anomaly Detection with Stochastically Improved Autoencoder Based Models. in 2017 IEEE 4th International Conference on Cyber Security and Cloud Computing (CSCloud). 2017.
[33]Kim, J.-K., Y. Han, and J. Lee, Data Imbalance Problem solving for SMOTE Based Oversampling: Study on Failure Detection Prediction Model in Semiconductor Manufacturing Process. Advanced Science and Technology Letters, 2016. 133: p. 79-84.
[34]Moldovan, D., et al. Chicken Swarm Optimization and Deep Learning for Manufacturing Processes. in 2018 17th RoEduNet Conference: Networking in Education and Research (RoEduNet). 2018.
[35]Kerdprasop, K. and N. Kerdprasop, Feature Selection Technique to Improve Performance Prediction in a Wafer Fabrication Process. Latest Trends in Engineering Mechanics, Structures, Engineering Geology, 2014: p. 128-133.
[36]Tseng, J.P.V.S., L.C.H. Motoda, and G. Xu, Advances in knowledge discovery and data mining. Lecture Notes in Artificial Intelligence), vol. LNAI, 2003. 8444.
[37]Abadi, M., et al., TensorFlow: a system for large-scale machine learning, in Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation. 2016, USENIX Association: Savannah, GA, USA. p. 265–283.
[38]Chollet, F.e.a. Keras. 2015; Available from: https://keras.io/.
[39]Pedregosa, F., et al., Scikit-learn: Machine Learning in Python. J. Mach. Learn. Res., 2011. 12(null): p. 2825–2830.
[40]Granitto, P.M., et al., Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products. Chemometrics and Intelligent Laboratory Systems, 2006. 83(2): p. 83-90.
[41]Cortes, C. and M. Mohri. AUC optimization vs. error rate minimization. in Advances in neural information processing systems. 2004.
[42]Ferri, C., J. Hernández-Orallo, and R. Modroiu, An experimental comparison of performance measures for classification. Pattern Recognition Letters, 2009. 30(1): p. 27-38.