跳到主要內容

臺灣博碩士論文加值系統

(44.192.44.30) 您好!臺灣時間:2024/07/25 08:52
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:尤瑞暘
研究生(外文):YU, JUI-YANG
論文名稱:運用BERT預訓練模型進行中英文表情符號之情感分析
論文名稱(外文):Using BERT Pre-trained Model for Sentiment Analysis of Chinese and English Emoji
指導教授:黃仁鵬黃仁鵬引用關係
指導教授(外文):HUANG, JEN PENG
口試委員:蔡玉娟李昇暾
口試委員(外文):TSAY, YUH-JIUANLI, SHENG-TUN
口試日期:2022-06-24
學位類別:碩士
校院名稱:南臺科技大學
系所名稱:資訊管理系
學門:電算機學門
學類:電算機一般學類
論文種類:學術論文
論文出版年:2022
畢業學年度:110
語文別:中文
論文頁數:65
中文關鍵詞:網路爬蟲自然語言處理中文斷詞深度學習
外文關鍵詞:Web crawlerNatural Language ProcessingChinese word segmentingDeep Learning
相關次數:
  • 被引用被引用:1
  • 點閱點閱:446
  • 評分評分:
  • 下載下載:71
  • 收藏至我的研究室書目清單書目收藏:0
隨著網際網路的發展與訊息爆炸性的增長,現今社群媒體已融入大部分人類的生活中。每個人早上起床的第一件事情也許是直接打開社群軟體查看最新動態,目的就是為了透過社群通訊軟體即時接收與傳遞資訊。這些平台有著非常大量且多樣化的資訊,若能透過相關技術從這些大量的資訊中進行分析,從而獲取文句細微的情緒,即可協助我們判斷句子的主要情緒並提供相關人士利用此資訊去進行策略分析等,也減少相關研究所耗費的時間資源。
本研究將利用深度學習等技術進行建立模型,以這些模型來評估中英文文本中的表情符號是否可以被機器捕捉情緒。選擇演算法為LSTM模型與GOOGLE開源的BERT。BERT為預訓練模型,希望透過BERT預訓練的模型當作embedding,以微調的方法進行訓練BERT,並給予表情符號來進行分類任務。而LSTM為具有時間序列相關性的模型,本研究將探討LSTM是否可以捕捉文本之間相關的訊息,並評估是否使用預訓練模型會有更高的準確度。
本研究透過分析發現在中文與英文的文本上,利用預訓練的模型以準確度為探討。預訓練模型BERT在分類20個表情符號的任務中,中文文本準確度達到28%,英文則是來到32%;以特徵擷取方法探討準確度與效能,利用特徵擷取方法時,中文與英文文本準確度達到37%,並大幅縮短訓練模型的時間。此研究將透過特徵擷取方法,於預訓練模型時降低記憶體負擔、提升準確度與縮短訓練時間,希望透過此模型來捕捉中文、英文語句裡的情緒與之相對應的表情符號,並提供模型與相關建議來降低其他研究人員進入此領域門檻。

As the internet becomes popular, social media have been integrated into the lives of most human beings. When people wake up in the morning, the first thing is open the social media to check the latest news and receive and transmit information. These platforms have a very large and diverse amount of information. If we can analyze this large amount of information through relevant technologies, we can obtain the subtle emotions of the sentences. It can help us determine the main emotions of the sentences, and help relevant people use this information to carry out strategic analysis.
This research will use deep learning and other technologies to build models, and use these models to evaluate whether emojis in Chinese and English texts can be captured by machines. We use pre-training model BERT and LSTM, hope to use the BERT pre-trained model as an embedding, train BERT in a fine-tuning method, and give emojis for classification tasks. This search will explore whether BERT and LSTM can capture relevant information between texts and evaluate whether using pre-trained models will lead to higher accuracy.
Through analysis, this study found in Chinese and English texts, the pre-trained model had a good classification effect on both Chinese and English texts. Using the feature extraction method can improve the accuracy of the BERT pre-training model, and can greatly shorten the training time of the model.


摘要 i
Abstract ii
致謝 iii
目 次 iv
表目錄 vi
圖目錄 vii
第一章 緒論 1
1.1研究背景 1
1.2研究動機 1
1.3研究目的 2
1.4研究流程與說明 3
1.5論文架構 5
第二章 文獻探討 6
2.1 表情符號(Emoji) 6
2.2 網路爬蟲程式(Web crawler) 7
2.2.1. Cloudscraper套件 7
2.2.2. Requests套件 8
2.2.3. BeautifulSoup套件 8
2.2.4. 應用程式介面(API) 9
2.3 資料前處理(Data Preprocessing) 10
2.3.1. 監督式學習(Supervised learning) 10
2.3.2. 非監督式學習(Unsupervised learning) 11
2.3.3. 半監督式學習(Semi-Supervised learning) 11
2.3.4. Jieba斷詞 11
2.4 One-Hot-Encoding(獨熱編碼) 13
2.5 Word-Embedding詞嵌入 14
2.6 Word2Vec 16
2.7 RNN循環神經網路 17
2.8 LSTM長短期記憶網路 20
2.9 Seq2Seq模型 21
2.10 Transformers 22
2.11 BERT 24
2.12 TF-IDF特徵提取 28
第三章 研究方法 30
3.1 研究架構 30
3.2 資料蒐集 31
3.3 資料前置處理模組 32
3.4 建立模型模組 37
3.5 分析模組 44
3.6 實驗方法 44
3.7 實驗設計 45
第四章 研究結果與分析 46
4.1 實驗一 參數對於BERT的影響 46
4.2 實驗二 特徵擷取對於BERT的影響 48
4.3 實驗三 中英文之BERT微調任務 50
4.4 實驗四 比較BERT、LSTM模型於表情符號之情形 54
4.4.1 使用Word2Vec做為特徵,進行LSTM模型建立 54
4.4.2 特徵擷取對於LSTM的影響 55
4.4.3 LSTM與BERT比較 57
第五章 結論與未來發展 59
5.1 結論 59
5.2 研究建議 61
5.3 未來發展 62
參考文獻 64

1. Barbieri, F., Camacho-Collados, J., Ronzano, F., Espinosa Anke, L., Ballesteros, M., Basile, V., Patti, V., and Saggion, H. "Semeval 2018 task 2: Multilingual emoji prediction," 12th International Workshop on Semantic Evaluation (SemEval 2018), Association for Computational Linguistics, 2018, pp. 24-33.
2. Cui, Y., Che, W., Liu, T., Qin, B., and Yang, Z. "Pre-training with whole word masking for chinese bert," IEEE/ACM Transactions on Audio, Speech, and Language Processing Vol. 29, 2021, pp 3504-3514.
3. Daniel, J. "The Most Frequently Used Emoji of 2021," 2021.
4. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. "Bert: Pre-training of deep bidirectional transformers for language understanding," in: arXiv preprint arXiv:1810.04805, 2018.
5. fxsjy "結巴(jieba)斷詞Github."
6. Hochreiter, S., and Schmidhuber, J. "Long short-term memory," Neural computation Vol. 9, No. 8, 1997, pp 1735-1780.
7. Inc., T. "Twitter Investor Relations," 2018.
8. Jing, L.-P., Huang, H.-K., and Shi, H.-B. "Improved feature selection approach TFIDF in text mining," Proceedings. International Conference on Machine Learning and Cybernetics, IEEE, 2002, pp. 944-946.
9. LeeMeng "進擊的 BERT:NLP 界的巨人之力與遷移學習," 2019.
10. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. "Roberta: A robustly optimized bert pretraining approach," arXiv preprint arXiv:1907.11692 Vol. 8, 2019.
11. Lu, R. "Preprocessing Data : 類別型特徵_OneHotEncoder & LabelEncoder 介紹與實作," 2018.
12. Lucas, G. The story of emoji Prestel Verlag, 2016.
13. Mikolov, T., Chen, K., Corrado, G., and Dean, J. "Efficient estimation of word representations in vector space," in: arXiv preprint arXiv:1301.3781, 2013.
14. MingLun "NLP深度學習不能不用的套件 - Transformers," 2020.
15. Pennington, J., Socher, R., and Manning, C.D. "Glove: Global vectors for word representation," Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532-1543.
16. Sun, Y.-T., Chen, C.-L., Liu, C.-C., Liu, C.-L., and Soo, V.-W. "中文短句之情緒分類 (Sentiment Classification of Short Chinese Sentences)[In Chinese]," Proceedings of the 22nd Conference on Computational Linguistics and Speech Processing (ROCLING 2010), 2010, pp. 184-198.
17. Sutskever, I., Vinyals, O., and Le, Q.V. "Sequence to sequence learning with neural networks," Advances in neural information processing systems Vol. 27, 2014.
18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. "Attention is all you need," Advances in neural information processing systems Vol. 30, 2017.
19. Veon, L.B. "Word Embedding 編碼矩陣技術與方法," 2019.
20. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. "Transformers: State-of-the-art natural language processing," Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38-45.
21. Zaremba, W., Sutskever, I., and Vinyals, O. "Recurrent neural network regularization," arXiv preprint arXiv:1409.2329 Vol. 37, 2014.
22. 林家瑋 "深度學習 RNN、LSTM 與 GRU 預測模型之比較:以台灣與美國期貨市場為例," 東吳大學數學系, 台北市, 2022.
23. 陳子揚 "使用生成式對抗網路進行半監督式學習," 國立成功大學工程科學系, 台南市, 2018.
24. 蘇郁涵, 李宜珍, 汪姿伶, and 蔡依娗 "廣告貼文使用 Emoji 對消費者情感及購買意願調查之研究," 圖文傳播藝術學報 . 2020, pp 161-167.

QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊