跳到主要內容

臺灣博碩士論文加值系統

(44.211.24.175) 您好!臺灣時間:2024/11/11 04:51
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:莊長融
研究生(外文):CHUANG,CHANG-JUNG
論文名稱:深度學習之遷移學習模式應用於情緒分類
論文名稱(外文):Transfer Learning with Deep Learning for Sentiment Classification
指導教授:黃承龍黃承龍引用關係
指導教授(外文):HUANG,CHENG-LUNG
口試委員:張育仁黃承龍黃文楨張弘毅
口試委員(外文):CHANG,YU-JENHUANG,CHENG-LUNGHUANG,WEN-CHENCHANG,HUNG-YI
口試日期:2019-06-20
學位類別:碩士
校院名稱:國立高雄科技大學
系所名稱:資訊管理系
學門:電算機學門
學類:電算機一般學類
論文種類:學術論文
論文出版年:2019
畢業學年度:107
語文別:中文
論文頁數:59
中文關鍵詞:情緒分類詞嵌入遷移學習預訓練詞向量深度學習
外文關鍵詞:Sentiment ClassificationWord EmbeddingTransfer LearningPre-trained Word VectorsDeep Learning
相關次數:
  • 被引用被引用:0
  • 點閱點閱:1028
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
現今深度學習已有許多模型應用於分類領域,例如:影像辨識、手寫字辨識、物件識別、人臉辨識…等,皆在各領域有著良好的效果。近年來陸續有新的演算法得以運用於深度學習領域,模型的選擇成為研究時的考量之一。雖然有眾多的選擇,但模型的訓練仍需耗費大量時間,且不一定得到良好的準確率。
遷移學習模式就是把已經訓練好的模型權重,應用於類似領域的新資料集,只要做局部的增量訓練,就可以得到一個新模型,其優點是加快並優化模型的學習,不用從零開始建構;遷移學習也可以適用在資料集不充足的情況下,對於模型不需太多資料集的要求,就可以進行訓練和測試。
本研究透過三種模型:(1)CNN、(2)LSTM、(3)FastText,搭配五種預訓練模型:(1)Google News vectors、(2)Wiki English vectors Skip Gram、(3)Wiki English vectors CBOW、(4)Amazon review polarity、(5)Yelp review polarity,來建立多種模型應用在情緒分類。
本論文針對情緒分類將留言分為(1)正向情緒、(2)負向情緒。以三種資料集:(1)Bauhaus飯店評論、(2)TripAdvisor飯店評論、(3)Yelp餐館評論,來探討多種模型中,何者達到較高的準確率及花費最低的訓練成本。
本研究的實驗結果顯示,資料集達到兩百萬筆而預訓練模型過大造成記憶體不足時,若使用FastText不會碰到此問題,且其準確率與其他模型只有稍降1%至2%,或是在部分實驗比其他模型為佳1%,但在訓練上節省4倍至40倍的時間,大量降低了訓練成本。

Nowadays, Deep learning has many applications on classification tasks. e.g., image recognition, handwriting recognition, object detection, face recognition. They have achieved high accuracy in various tasks. In recent years, new algorithms have been applied to the deep learning, choice of models has become one of the considerations in research. Although there are many options, the training of the model still takes a lot of time and does not necessarily lead to well accuracy.
Transfer learning is to train new model with small amount of new data based on pre-trained weights. The advantages are to speed up and optimize the model learning without constructing from scratch. That is, transfer learning can be applied to insufficient data. In the case, the model can be trained and tested without requiring large amount of data.
This study used Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and FastText combining five pre-trained word vector models, to build several sentiment classification models.
This study divided user reviews into positive and negative polarity for sentiment classification, Three datasets including Bauhaus hotel review, TripAdvisor hotel review and Yelp restaurant review were explored to find the best models with higher accuracy and lowest training costs.
The experimental results show that the data set reaches 2 million and the pre-trained model was too large, the memory is insufficient, using FastText will not encounter this problem. And accuracy was 1% to 2% lower than other models, or 1% better than other models in some experiments, but it saves four to forty times in training and also significantly reduces training costs.

摘要 I
Abstract II
誌 謝 IV
目錄 V
表目錄 VIII
圖目錄 IX
壹、緒論 1
1.1研究背景與動機 1
1.2研究目的 2
1.3研究流程 2
1.4論文架構 5
貳、文獻探討 6
2.1深度學習(Deep learning) 6
2.2卷積神經網路(Convolutional Neural Networks) 6
2.3長短期記憶神經網路(Long Short-Term Memory) 7
2.4詞嵌入(Word embedding) 8
2.5 FastText 9
2.6情緒分類(Sentiment classification) 11
2.7監督式學習(Supervised learning) 11
2.8非監督式學習(Unsupervised learning) 12
2.9遷移學習(Transfer learning) 12
2.9.1歸納式遷移學習(Inductive transfer learning) 14
2.9.2直推式遷移學習(Transductive transfer learning) 15
2.9.3非監督式遷移學習(Unsupervised Transfer Learning) 16
參、系統架構 17
3.1資料前處理 18
3.2建立模型 19
3.3結果分析 19
肆、實驗結果與分析 20
4.1實驗資料來源 20
4.1.1 Bauhaus使用者飯店評論 20
4.1.2 TripAdvisor使用者飯店評論 20
4.1.3 Yelp使用者餐館評論 21
4.2實驗模型來源 21
4.2.1 Google News vectors 21
4.2.2 Wiki English vectors Skip Gram 21
4.2.3 Wiki English vectors CBOW 22
4.2.4 Amazon review polarity 22
4.2.5 Yelp review polarity 22
4.3實驗工具及實驗設備 22
4.4實驗(一): Bauhaus使用者飯店評論 23
4.4.1實驗參數 25
4.4.2實驗結果分析 26
4.5實驗(二): TripAdvisor使用者飯店評論 30
4.5.1 實驗參數 32
4.5.2 實驗結果分析 33
4.6實驗(三): Yelp使用者餐館評論 37
4.6.1 實驗參數 39
4.6.2 實驗結果分析 40
伍、結論 44
5.1結論 44
5.2未來研究方向 45
陸、參考文獻 46


1.Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146.
2.Conneau, A., Schwenk, H., Barrault, L., & Lecun, Y. (2016). Very deep convolutional networks for text classification. arXiv preprint arXiv:1606.01781.
3.Felbo, B., Mislove, A., Søgaard, A., Rahwan, I., & Lehmann, S. (2017). Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. arXiv preprint arXiv:1708.00524.
4.Go, A., Bhayani, R., & Huang, L. (2009). Twitter sentiment classification using distant supervision, CS224N Project Report, Stanford, 1(12).
5.Grave, E., Bojanowski, P., Gupta, P., Joulin, A., & Mikolov, T. (2018). Learning word vectors for 157 languages. arXiv preprint arXiv:1802.06893.
6.Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2016). Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759.
7.Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., & Mikolov, T. (2016). Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651.
8.Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097-1105).
9.Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.
10.Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning (ICML-14) (pp. 1188-1196).
11.LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.
12.Li, Y., Wang, X., & Xu, P. (2018). Chinese text classification model based on deep learning. Future Internet, 10(11), 113.
13.Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
14.Mishkin, D., & Matas, J. (2015). “All you need is a good init,” arXiv preprint arXiv:1511.06422.
15.Majumder, N., Poria, S., Peng, H., Chhaya, N., Cambria, E., & Gelbukh, A. (2019). Sentiment and Sarcasm Classification with Multitask Learning. arXiv preprint arXiv:1901.08014.
16.Nigam, K., McCallum, A. K., Thrun, S., & Mitchell, T. (2000). “Text classification from labeled and unlabeled documents using EM,” Machine Learning, 39(2-3), pp. 103-134.
17.Ozsoy, M. G. (2016). From word embeddings to item recommendation. arXiv preprint arXiv:1601.01356.
18.Pang, B., Lee, L., & Vaithyanathan, S. (2002, July). “Thumbs up?: sentiment classification using machine learning techniques,” In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10 (pp. 79-86). Association for Computational Linguistics.
19.Pan, S. J., & Yang, Q. (2009). “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering, Vol. 22, pp. 1345-1359.
20.Reis, J. C., Correia, A., Murai, F., Veloso, A., Benevenuto, F., & Cambria, E. (2019). Supervised Learning for Fake News Detection. IEEE Intelligent Systems, 34(2), 76-81.
21.Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.
22.Xie, Q., Dai, Z., Hovy, E., Luong, M. T., & Le, Q. V. (2019). Unsupervised data augmentation. arXiv preprint arXiv:1904.12848.
23.Zhang, L., Wang, S., & Liu, B. (2018). “Deep learning for sentiment analysis: A survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, e1253.
24.莊長融&黃承龍,2018,『深度學習之遷移學習模式應用於留言情緒分類』,國際資訊管理暨實務研討會。

QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊