臺灣博碩士論文加值系統

卷積類神經網路在圖像辨識方面有強大的效能和進步。卷積類神經網路
可以從資料中萃取出特徵，因此在許多資料辨識中能有很好的表現。但卷積類神經網路在時間序列資料上的應用還存有許多挑戰，其中一個問題是圖像辨識和時間序列資料分類資料特徵就有所不同。我們提出一個資料轉變得版模，將時間序列資料轉換成三維資料，另外還提出一個多變量卷積類神經網路用於時間序列資料分類問題。我們此神經網路和其他優異的演算法做比較，在PHM 2015 challenge 資料集上，多變量卷積類神經網路在分數和預測的精準率上表現亮眼。另外我們還將此架構應用在occupancy detection 資料集上，其準確度也優於Random forest。

In recent years, the convolutional neural network (CNN) has achieved great success
on image classification, since it learns feature representation from a collection
of images. Recent studies have shown that machine learning algorithms could benefit
from good feature representation, explaining why CNN, which considers feature
learning and classification in the same network, has achieved breakthrough performance
in many pattern recognition tasks. However, it is still a challenge to apply
CNN to time series data. One of the problems is that the time series data and image
data possess different characteristics. This thesis proposes a tensor scheme and a
novel deep learning architecture called multivariate convolutional neural network for time series classification. We evaluate our proposed method with PHM 2015 challenge data, and compare with several state-of-the-art algorithms, including gradient boost tree, random forest and logistic regression. The experimental results
indicate that the proposed method outperforms the other alternatives on the scores and false positives.
dditionally, the proposed method achieves better accuracy than random forest on occupancy detection data.

中文論文口試委員會審定書. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Thesis Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
論文著作權授權書. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Aknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1 PHM 2015 data challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Research of PHM 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Machine Learning for Fault Detection . . . . . . . . . . . . . . . . . . . . . . 4
2.4 Sampling Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.5 Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6 Convolutional Neural Networks for Time Series Classification . . . . . . . . . 9
Chapter 3 Multivariate Convolutional Neural Network . . . . . . . . . . . . . . . . . . 11
3.1 Data Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Multivariate Convolutional Neural Network . . . . . . . . . . . . . . . . . . . 12
3.3 Input Tensor Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4 Univariate convolution stage . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.5 Multivariate Convolution Stage . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.6 Fully Connected Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Chapter 4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 Experimental Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2.1 Experimental Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.2.2 Baseline Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4.1 PHM 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4.2 Occupancy Detection Data . . . . . . . . . . . . . . . . . . . . . . . . 28
Chapter 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

[1] Francis R Bach and Michael I Jordan. Learning spectral clustering. In Advances in neural
information processing systems, pages 305–312, 2004.
[2] Mustafa Gokce Baydogan, George Runger, and Eugene Tuv. A bag-of-features framework
to classify time series. IEEE transactions on pattern analysis and machine intelligence,
35(11):2796–2802, 2013.
[3] Luis M Candanedo and Véronique Feldheim. Accurate occupancy detection of an office
room from light, temperature, humidity and co 2 measurements using statistical learning
models. Energy and Buildings, 112:28–39, 2016.
[4] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote:
synthetic minority over-sampling technique. Journal of artificial intelligence research,
16:321–357, 2002.
[5] Leo H Chiang, Mark E Kotanchek, and Arthur K Kordon. Fault diagnosis based on fisher
discriminant analysis and support vector machines. Computers & chemical engineering,
28(8):1389–1401, 2004.
[6] Leo H Chiang, Evan L Russell, and Richard D Braatz. Fault diagnosis in chemical processes
using fisher discriminant analysis, discriminant partial least squares, and principal
component analysis. Chemometrics and intelligent laboratory systems, 50(2):243–252,
2000.
[7] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning,
20(3):273–297, 1995.
[8] Zhicheng Cui, Wenlin Chen, and Yixin Chen. Multi-scale convolutional neural networks
for time series classification. CoRR, abs/1603.06995, 2016.
[9] Tom Fawcett. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874,
2006.
[10] Zhen-yu GUO, Bo CHENG, Min YE, et al. Advances in natural computation, 2006.
[11] Peter Hart. The condensed nearest neighbor rule (corresp.). IEEE transactions on information
theory, 14(3):515–516, 1968.
[12] Q Peter He and Jin Wang. Fault detection using the k-nearest neighbor rule for semiconductor
manufacturing processes. IEEE Transactions on Semiconductor Manufacturing,
20(4):345–354, 2007.
[13] Eamonn Keogh and Chotirat Ann Ratanamahatana. Exact indexing of dynamic time warping.
Knowledge and information systems, 7(3):358–386, 2005.
[14] Hyunjae Kim, Jong Moon Ha, Jungho Park, Sunuwe Kim, Keunsu Kim, Beom Chan Jang,
Hyunseok Oh, and Byeng D Youn. Fault log recovery using an incomplete-data-trained
fda classifier for failure diagnosis of engineered systems.
[15] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv
preprint arXiv:1412.6980, 2014.
[16] J Korbicz, JM Koscielny, Z Kowalczuk, and W Cholewa. Fault diagnosis. models, artificial
intelligence, applications, 2004.
[17] Sanghyuk Lee, Wookje Park, and Sikhang Jung. Fault detection of aircraft system with
random forest algorithm and similarity measure. The Scientific World Journal, 2014, 2014.
[18] Jessica Lin, Eamonn Keogh, Li Wei, and Stefano Lonardi. Experiencing sax: a novel
symbolic representation of time series. Data Mining and knowledge discovery, 15(2):107–
144, 2007.
[19] Nicola Lunardon, Giovanna Menardi, and Nicola Torelli. Rose: A package for binary
imbalanced learning. R Journal, 6(1), 2014.
[20] B Samanta. Gear fault detection using artificial neural networks and support vector machines
with genetic algorithms. Mechanical Systems and Signal Processing, 18(3):625–
644, 2004.
[21] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir
Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper
with convolutions. In Proceedings of the IEEE conference on computer vision and pattern
recognition, pages 1–9, 2015.
[22] Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395–
416, 2007.
[23] Sun-Chong Wang. Artificial neural network. In Interdisciplinary computing in java programming,
pages 81–100. Springer, 2003.
[24] Wei Xiao. A probabilistic machine learning approach to detect industrial plant faults. arXiv
preprint arXiv:1603.05770, 2016.
[25] Wen-Hoar Hsaio Chien-Liang Liu Yao-Chung Tu, Ching-Hsien Lee. Data sampling to
imbalanced data of industrial plant fault prediction. 2016.
[26] Subin Yi, Janghoon Ju, Man-Ki Yoon, and Jaesik Choi. Grouped convolutional neural
networks for multivariate time series. arXiv preprint arXiv:1703.09938, 2017.
[27] Shen Yin, Steven X Ding, Adel Haghani, Haiyang Hao, and Ping Zhang. A comparison
study of basic data-driven fault diagnosis and process monitoring methods on the benchmark
tennessee eastman process. Journal of Process Control, 22(9):1567–1581, 2012.