臺灣博碩士論文加值系統

Content
摘要i
ABSTRACTii
Contentiii
List of Figuresv
List of Tablesvi
1Introduction1
2Related Work5
2.1Multimodal Recommendation5
2.2GCN-based Recommender Systems6
2.3Contrastive Learning7
2.4Prompt Engineering8
3Proposed Approach10
3.1Overview10
3.2Multimodal Data Preprocessing and Feature Extraction11
3.2.1Item Network Data Preprocessing11
3.2.2User Network Data Preprocessing with GAI – Prompt Engineering13
3.2.3Textual Data Embedding Backbone – text-embedding-ada-00216
3.3Uncovering Latent Structures with Modality Awareness18
3.3.1Formation of Initial Modality-aware Graphs19
3.3.2Discovery and Refinement of Latent Modality-aware Graphs20
3.4Graph Convolution-based Learning of User and Item Relationships21
3.5Multimodal Fusion Feature Module23
3.6Multimodal Fusion Aggregation with Collaborative Filtering Models25
3.6.1Collaborative-filtered User-Item Interaction Graph25
3.6.2Modality-aware Feature and Collaborative Embedding Aggregation26
3.7User-Item Preference Score Prediction27
3.8Parallel-stage Contrastive Learning Framework27
3.8.1Stage A: Single and Multimodal Features Contrastive Learning29
3.8.2Stage B: Collaborative Filtered embedding and Multimodal Features Contrastive Learning31
3.8.3Parallel-Stage Contrastive Learning Concatenation32
3.9Optimization33
4Experiment and Evaluation33
4.1Dataset34
4.1.1Data Filtering34
4.1.2User-based Feature Data34
4.1.3Item-based Feature Data34
4.1.4Training and Testing Dataset35
4.2Experimental Setting36
4.2.1Experimental Settings36
4.2.2Evaluation Metrics36
4.3Comparison with Baseline Models36
4.4Evaluation38
4.4.1Evaluation of the effectiveness of Generative AI38
4.4.2Evaluation of Neighbor Graph Mechanism: KNN39
4.4.3Evaluation of the Multimodal Latent Semantic Graph39
4.4.4Evaluation of Different Stage Training of Contrastive Learning40
4.4.4.1Effectiveness of Contrastive Learning – Warm-Start41
4.4.4.2Effectiveness of Contrastive Learning – Cold-Start42
4.4.5Sensitivity Analysis of Model Parameters44
4.4.5.1Impact of the parallel-stage contrastive auxiliary coefficients β and γ44
4.4.6Comparisons with the State-of-the-art Models47
5Conclusion and Future Work50
References50

References
[1] S.M. Anwar, T. Shahzad, Z. Sattar, R. Khan, M. Majid, A game recommender system using collaborative filtering (GAMBIT), 2017 14th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan, 2017, pp. 328-332.
[2] R. Aperdannier, M. Koeppel, T. Unger, S. Schacht, S.K. Barkur, Systematic Evaluation of Different Approaches on Embedding Search, Future of Information and Communication Conference, 2024, pp. 526-536.
[3] C. BharathiPriya, A. Sreenivasu, S. Kumar, Online Video Game Recommendation System Using Content And Collaborative Filtering Techniques, 2021 International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA), Coimbatore, India, 2021, pp. 1-7.
[4] T. Brown, B. Mann, N. Ryder, M. Subbiah, J.D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, Language models are few-shot learners, Advances in neural information processing systems, 33, 2020, pp. 1877-1901.
[5] R. Bunga, F. Batista, R. Ribeiro, From implicit preferences to ratings: video games recommendation based on collaborative filtering, From implicit preferences to ratings: Video games recommendation based on collaborative filtering, 2021, pp. 209-216.
[6] J. Chen, H. Zhang, X. He, L. Nie, W. Liu, T.-S. Chua, Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention, Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, 2017, pp. 335-344.
[7] T. Chen, S. Kornblith, M. Norouzi, G. Hinton, A simple framework for contrastive learning of visual representations, International conference on machine learning, 2020, pp. 1597-1607.
[8] X. Chen, H. Chen, H. Xu, Y. Zhang, Y. Cao, Z. Qin, H. Zha, Personalized fashion recommendation with visual explanations based on multimodal attention network: Towards visually explainable recommendation, Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, 2019, pp. 765-774.
[9] G. Cheuque, J. Guzmán, D. Parra, Recommender systems for online video game platforms: The case of steam, Companion Proceedings of The 2019 World Wide Web Conference, San Francisco, 2019, pp. 763-771.
[10] A. Chow, M.-H.N. Foo, G. Manai, HybridRank: A Hybrid Content-Based Approach To Mobile Game Recommendations, CBRecSys@ RecSys, 2014, pp. 10-13.
[11] Z. Cui, F. Yu, S. Wu, Q. Liu, L. Wang, Disentangled item representation for recommender systems, ACM Transactions on Intelligent Systems and Technology (TIST), 12 (2), 2021, pp. 1-20.
[12] H. Gao, S. Wang, B. Yang, H. Yang, User Preference-oriented Collaborative Recommendation Algorithm in e-commerce, Journal of Software, 9 (7), 2014, pp. 1886-1893.
[13] X. Gao, F. Feng, X. He, H. Huang, X. Guan, C. Feng, Z. Ming, T.-S. Chua, Hierarchical attention network for visually-aware food recommendation, IEEE Transactions on Multimedia, 22 (6), 2019, pp. 1647-1659.
[14] H. Guo, R. Tang, Y. Ye, Z. Li, X. He. DeepFM: a factorization-machine based neural network for CTR prediction, https://doi.org/10.48550/arXiv.1703.04247.
[15] R. Hadsell, S. Chopra, Y. LeCun, Dimensionality reduction by learning an invariant mapping, 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR'06), New York City, New York, 2006, pp. 1735-1742.
[16] K. He, H. Fan, Y. Wu, S. Xie, R. Girshick, Momentum contrast for unsupervised visual representation learning, Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Seattle, WA, USA., 2020, pp. 9729-9738.
[17] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, Nevada, USA., 2016, pp. 770-778.
[18] R. He, J. McAuley. VBPR: visual bayesian personalized ranking from implicit feedback, https://doi.org/10.48550/arXiv.1510.01784.
[19] X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, M. Wang, Lightgcn: Simplifying and powering graph convolution network for recommendation, Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, 2020, pp. 639-648.
[20] G. Hinton, O. Vinyals, J. Dean. Distilling the knowledge in a neural network, https://doi.org/10.48550/arXiv.1503.02531.
[21] R.D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, Y. Bengio. Learning deep representations by mutual information estimation and maximization, https://doi.org/10.48550/arXiv.1808.06670.
[22] A. Holtzman, J. Buys, L. Du, M. Forbes, Y. Choi. The curious case of neural text degeneration, https://doi.org/10.48550/arXiv.1904.09751.
[23] F. Ikram, H. Farooq. Multimedia recommendation system for video game based on high-level visual semantic features, 2022, https://doi.org/10.1155/2022/6084363.
[24] W.-C. Kang, C. Fang, Z. Wang, J. McAuley, Visually-aware fashion recommendation and design with generative image models, 2017 IEEE international conference on data mining (ICDM), New Orleans, LA, USA., 2017, pp. 207-216.
[25] Y. Koren, Collaborative filtering with temporal dynamics, Communications of the ACM, 53 (4), 2010, pp. 89-97.
[26] B. Li, B. Jin, J. Song, Y. Yu, Y. Zheng, W. Zhou, Improving micro-video recommendation via contrastive multiple interests, Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain., 2022, pp. 2377-2381.
[27] J. Li, M. Jing, K. Lu, L. Zhu, Y. Yang, Z. Huang, From zero-shot learning to cold-start recommendation, Proceedings of the AAAI conference on artificial intelligence, Honolulu, Hawaii, USA., 2019, pp. 4189-4196.
[28] M. Liu, L. Nie, M. Wang, B. Chen, Towards micro-video understanding by joint sequential-sparse modeling, Proceedings of the 25th ACM international conference on Multimedia, Mountain View, California, US 2017, pp. 970-978.
[29] M. Liu, L. Nie, X. Wang, Q. Tian, B. Chen, Online data organizer: micro-video categorization by structure-guided multimodal dictionary learning, IEEE Transactions on Image Processing, 28 (3), 2018, pp. 1235-1247.
[30] L. Logeswaran, H. Lee. An efficient framework for learning sentence representations, https://doi.org/10.48550/arXiv.1803.02893.
[31] W.E. Lomanto, V. Andrian, S. Achmad, R. Sutoyo, Collaborative Filtering for Steam Games Recommendation, 2023 5th International Conference on Cybernetics and Intelligent System (ICORIS), ISB Atma Luhur,Pangkal Pinang - Bangka Belitung Islands, Indonesia, 2023, pp. 1-6.
[32] K. Mao, J. Zhu, X. Xiao, B. Lu, Z. Wang, X. He, UltraGCN: ultra simplification of graph convolutional networks for recommendation, Proceedings of the 30th ACM international conference on information & knowledge management, Gold Coast, Queensland, Australia, 2021, pp. 1253-1262.
[33] J. McAuley, C. Targett, Q. Shi, A. Van Den Hengel, Image-based recommendations on styles and substitutes, Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, Santiago, Chile, 2015, pp. 43-52.
[34] A.v.d. Oord, Y. Li, O. Vinyals. Representation learning with contrastive predictive coding, https://doi.org/10.48550/arXiv.1807.03748.
[35] M. Pazzani, D. Billsus, Content-Based Recommendation Systems, The Adaptive Web, Springer Berlin / Heidelber LNCS 4321, 2007, pp. 325-341.
[36] D. Qi, J. Wang. CleanAgent: Automating Data Standardization with LLM-based Agents, https://doi.org/10.48550/arXiv.2403.08291.
[37] S. Rendle, C. Freudenthaler, Z. Gantner, L. Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback, https://doi.org/10.48550/arXiv.1205.2618.
[38] P. Sahoo, A.K. Singh, S. Saha, V. Jain, S. Mondal, A. Chadha. A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications, https://doi.org/10.48550/arXiv.2402.07927.
[39] J. Shuai, K. Zhang, L. Wu, P. Sun, R. Hong, M. Wang, Y. Li, A review-aware graph contrastive learning framework for recommendation, Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, 2022, pp. 1283-1293.
[40] G. Song, W. Chai. Collaborative learning for deep neural networks, 31, https://doi.org/10.48550/arXiv.1805.11761.
[41] X. Su, T.M. Khoshgoftaar. A survey of collaborative filtering techniques, 2009, https://doi.org/10.1155/2009/421425.
[42] Z. Tao, X. Liu, Y. Xia, X. Wang, L. Yang, X. Huang, T.-S. Chua. Self-supervised learning for multimedia recommendation, https://ieeexplore.ieee.org/document/9811387.
[43] Y. Tian, D. Krishnan, P. Isola, Contrastive multiview coding, Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, 2020, pp. 776-794.
[44] T. van der Zant, M. Kouw, L. Schomaker, Generative artificial intelligence, Springer, 2013.
[45] C. Wang, D.M. Blei, Collaborative topic modeling for recommending scientific articles, Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, 2011, pp. 448-456.
[46] D. Wang, M. Moh, T.-S. Moh, Using deep learning and steam user data for better video game recommendations, Proceedings of the 2020 ACM Southeast Conference, 2020, pp. 154-159.
[47] D. Wang, Q. Wang, Y. An, X. Gao, Y. Tian, Online collective matrix factorization hashing for large-scale cross-media retrieval, Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, Virtual Event China, 2020, pp. 1409-1418.
[48] X. Wang, X. He, M. Wang, F. Feng, T.-S. Chua, Neural graph collaborative filtering, Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, Paris, 2019, pp. 165-174.
[49] Y. Wang, Z. Liu, Z. Fan, L. Sun, P.S. Yu, Dskreg: Differentiable sampling on knowledge graph for recommendation with relational gnn, Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Virtual Event Queensland Australia, 2021, pp. 3513-3517.
[50] Y. Wang, K. Zhou, R. Miao, N. Liu, X. Wang, Adagcl: Adaptive subgraph contrastive learning to generalize large-scale graph training, Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, GA, USA, 2022, pp. 2046-2055.
[51] Z. Wang, W. Wei, G. Cong, X.-L. Li, X.-L. Mao, M. Qiu, Global context enhanced graph neural networks for session-based recommendation, Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, Xi'an, China., 2020, pp. 169-178.
[52] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q.V. Le, D. Zhou, Chain-of-thought prompting elicits reasoning in large language models, Advances in neural information processing systems, 35, 2022, pp. 24824-24837.
[53] W. Wei, C. Huang, L. Xia, C. Zhang, Multi-modal self-supervised learning for recommendation, Proceedings of the ACM Web Conference 2023, Austin, Texas, USA, 2023, pp. 790-800.
[54] Y. Wei, X. Wang, Q. Li, L. Nie, Y. Li, X. Li, T.-S. Chua, Contrastive learning for cold-start recommendation, Proceedings of the 29th ACM International Conference on Multimedia, Chengdu, China., 2021, pp. 5382-5390.
[55] Y. Wei, X. Wang, L. Nie, X. He, T.-S. Chua, Graph-refined convolutional network for multimedia recommendation with implicit feedback, Proceedings of the 28th ACM international conference on multimedia, New York,NY,United States, 2020, pp. 3541-3549.
[56] Y. Wei, X. Wang, L. Nie, X. He, R. Hong, T.-S. Chua, MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video, Proceedings of the 27th ACM international conference on multimedia, Nice, France, 2019, pp. 1437-1445.
[57] J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, X. Xie, Self-supervised graph learning for recommendation, Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, Montréal, Quebec, 2021, pp. 726-735.
[58] Z. Wu, Y. Xiong, S.X. Yu, D. Lin, Unsupervised feature learning via non-parametric instance discrimination, Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 2018, pp. 3733-3742.
[59] B. Xia, Y. Bai, J. Yin, Q. Li, L. Xu, MTPR: a multi-task learning based poi recommendation considering temporal check-ins and geographical locations, Applied Sciences, 10 (19), 2020, pp. 6664.
[60] L. Xia, C. Huang, Y. Xu, J. Zhao, D. Yin, J. Huang, Hypergraph contrastive collaborative filtering, Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval, Madrid Spain, 2022, pp. 70-79.
[61] Y. Xiao, C. Li, V. Liu, DFM-GCN: a multi-task learning recommendation based on a deep graph neural network, Mathematics, 10 (5), 2022, pp. 721.
[62] G. Xv, C. Lin, W. Guan, J. Gou, X. Li, H. Deng, J. Xu, B. Zheng, E-commerce Search via Content Collaborative Graph Neural Network, Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach CA USA 2023, pp. 2885-2897.
[63] L. Yang, Z. Liu, Y. Wang, C. Wang, Z. Fan, P.S. Yu, Large-scale personalized video game recommendation via social-aware contextualized graph neural network, Proceedings of the ACM Web Conference 2022, Lyon, France, 2022, pp. 3376-3386.
[64] S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, K. Narasimhan, Tree of thoughts: Deliberate problem solving with large language models, Advances in Neural Information Processing Systems, 2024, pp. 11809--11822.
[65] T. Yao, X. Yi, D.Z. Cheng, F. Yu, T. Chen, A. Menon, L. Hong, E.H. Chi, S. Tjoa, J. Kang, Self-supervised learning for large-scale item recommendations, Proceedings of the 30th ACM International Conference on Information & Knowledge Management, Gold Coast, Queensland, Australia,, 2021, pp. 4321-4330.
[66] Z. Yi, X. Wang, I. Ounis, C. Macdonald, Multi-modal graph contrastive learning for micro-video recommendation, Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain., 2022, pp. 1807-1811.
[67] P. Yu, Z. Tan, G. Lu, B.-K. Bao, Multi-view graph convolutional network for multimedia recommendation, Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, Canada, 2023, pp. 6576-6585.
[68] H. Zhang, Y. Dong, C. Xiao, M. Oyamada. Jellyfish: A Large Language Model for Data Preprocessing, https://doi.org/10.48550/arXiv.2312.01678.
[69] H. Zhang, Y. Dong, C. Xiao, M. Oyamada. Large language models as data preprocessors, https://doi.org/10.48550/arXiv.2308.16361.
[70] J. Zhang, Y. Zhu, Q. Liu, S. Wu, S. Wang, L. Wang, Mining latent structures for multimedia recommendation, Proceedings of the 29th ACM international conference on multimedia, Virtual Event China, 2021, pp. 3872-3880.
[71] M. Zhang, J. Tang, X. Zhang, X. Xue, Addressing cold start in recommender systems: A semi-supervised co-training algorithm, Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, Gold Coast Queensland Australia, 2014, pp. 73-82.
[72] M. Zhang, S. Wu, X. Yu, Q. Liu, L. Wang, Dynamic graph neural networks for sequential recommendation, IEEE Transactions on Knowledge and Data Engineering, 35 (5), 2022, pp. 4741-4753.
[73] Y. Zhou, A.I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, J. Ba. Large language models are human-level prompt engineers, https://doi.org/10.48550/arXiv.2211.01910.
[74] Y. Zhu, J. Lin, S. He, B. Wang, Z. Guan, H. Liu, D. Cai. Addressing the Item Cold-start Problem by Attribute-driven Active Learning, https://doi.org/10.48550/arXiv.1805.09023.
[75] Z. Zhu, S. Sefati, P. Saadatpanah, J. Caverlee, Recommendation for new users and new items via randomized training and mixture-of-experts transformation, Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, Xi'an, China, 2020, pp. 1121-1130.