臺灣博碩士論文加值系統

DL（深度學習）演算法架構的情感分析，由於近年來的準確性提升和擴展應用而變得越來越實用和重要。隨著不斷增長的大數據和高性能計算硬體的易得性，快速成長的AI（人工智慧）工具和方法得到了增強，並將其應用擴展到了關鍵的產業應用中。本研究報告通過建立一個LSTM model, 並使用兩個不同的數據集: (1) IMDb電影影評和(2)Twitter用戶評論, 來訓練該模型。經與先前模型的結果進行比較，展現非常突出的成果表現。此DL模型，經過訓練後，能夠在極短時間內，根據文章的語意極性，完成巨量的文件分析，例如，數以萬計的產品評論分類，同時達到近乎真人的準確結果。

Sentiment analysis with the DL (Deep Learning) model have been becoming more practical and important due to their enhanced accuracy and extended applications in the recent years. The fast developing AI tools and techniques based on the DL, which powered by the growing big data and high performance computing hardware available, have extended their applications into key industrial practices. In this paper, a DL model, LSTM, is built and trained with two different datasets, (1) IMDb movie and (2) Twitter user reviews, to demonstrate its state-of-art performance and compare with the results from the previous models. The trained DL model presented here is capable of classifying large number of documents, e.g. dozen thousands of product reviews, according to their sentiment polarities, while achieving the results of near-human accuracy in minutes.

TABLE OF CONTENTS

論文摘要........ II
Abstract........III
1. Introduction ........1
2. Related Works........4
3. Description of Model........ 6
3.1 Embedding Word Vectors with GloVe........ 7
3.2 Long Short Term Memory (LSTM) Model ........9
3.3 Loss function Used for Training: Cross Entropy ........13
4. Experiment Dataset, Results, and Discussion........ 14
4.1 Dataset ........ 14
4.1.1 IMDb (Internet Movie Database) movie reviews ........ 14
4.1.2 U.S. Airlines Users’ Twitter Comments........ 16
4.1.3 Key Hyperparameter Setting For Training........ 19
4.1.4 Training and Testing Sets Allocated for Validation........20
4.2 Specification of Hardware and Software Running the Experiment........22
4.3 Results and Discussion........23
5. Conclusion........28
6. Reference........29

[1] Y. Goldberg, O. Levy, "Word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method,” arXiv:1402.3722v1[cs.CL], 2013
[2] A. Graves, "Supervised sequence labelling with recurrent neural networks”. Vol. 385. Springer, 2012
[3] S. Hochreiter, J. Schmidhuber, “Long Short-Term Memory,” Journal Neural Computation, Vol. 9 Issue 8, pp. 1735-1780, MIT, USA, 1997
[4] O. Irsoy and C. Cardie, "Opinion Mining with Deep Recurrent Neural Networks," EMNLP, pp. 720-728,2014.
[5] V. A. Kharde, S.S. Sonawane, “Sentiment Analysis of Twitter Data: A Survey of Techniques,” International Journal of Computer Applications (0975 – 8887), Volume 139 –No.11, 2016
[6] Y. Kim, "Convolutional Neural Networks for Sentence Classification," arXiv preprint arXiv: 1408.5882, 2014.
[7] D. P. Kroese, R. Y. Rubinstein, and P. W. Glynn, “The Cross-Entropy Method for Estimation,” Handbook of Statistics, Vol. 31, Chennai: Elsevier B.V., 2013, p. 19-34
[8] Q. Le and T. Mikolov, "Distributed Representations of Sentences and Documents," ICML, vol. 14, pp. 1188-1196, June 2014.
[9] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015.
[10] B. Liu, "Sentiment Analysis and Opinion Mining," Synthesis lectures on human language technologies, vol. 5(1), pp. 1-167, May 2012.
[11] A. L. Maas, R. E. Daly, P. T. Ph am, D. Huang, A. Y. Ng, and C. Potts, "Learning Word Vectors for Sentiment Analysis," Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. I, pp. 142-150, June 2011, Association for Computational Linguistics.
[12] T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Distributed Representations of Words and Phrases and their Compositionality," Advances in neural information processing systems, pp. 3111-3119, 2013.
[13] T. Mikolov, K. Chen, G. Corrado, J. Dean, “Efficient Estimation of Word Representations in Vector Space,” arXiv:1301.3781 [cs.CL], 2013
[14] B. Pang and L. Lee, "Opinion Mining and Sentiment Analysis," Foundations and trends in information retrieval. vol. 2(1-2), pp. 1-135, January 2008.
[15] B. Pang, 1. Lee, and S. Vaithyanathan, "Thumbs up?: sentiment classification using machine learning techniques," Proceedings of the ACL-02 conference on Empirical methods in natural language processing, vol. 10, pp. 79-86, July 2002.
[16] C. Park, D. Lee, K. Sim, “Emotion Recognition of Speech Based on RNN,” Proceedings of the First International Conference on Machine Learning and Cybernetics, Beijing, 4-5 November 2002.
[17] R. Pascanu, T. Mikolov, Y. Bengio, “On the difficulty of training Recurrent Neural Networks,” arXiv:1211.5063v2 [cs.LG], 2013
[18] J. Pennington, R. Socher, C. D. Manning, “GloVe: Global Vectors for Word Representation,” Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Qatar, 2014
[19] R. Socher, 1. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning, "Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions," Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 151-161, July 2011, Association for Computational Linguistics.
[20] D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin, "Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification," Acl, pp. 1555-1565, June 2014.
[21] S. Wang and C. D. Manning, "Baselines and Bigrams : Simple, Good Sentiment and Topic Classification," Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers, vol. 2, pp. 90-94, July 2012, Association for Computational Linguistics .
[22] P. Wang, Y. Qian, F. K. Soong, 1. He, and H. Zhao, "A Unified Tagging Solution: Bidirectional LSTM Recurrent Neural Network with Word Embedding," arXiv preprint arXiv:1511.00215, 2015.
[23] W. Zaremba, I. Sutskever, and O. Vinyals, "Recurrent neural network regularization," arXiv preprint arXiv: 1409.2329,2014.