臺灣博碩士論文加值系統

近幾年來，由於網路事業的崛起，造就了許多網路平台的誕生，改變了我們商業
以及溝通的模式。隨著越來越多的用戶使這些網路平台，網路廣告產業成為眾多企業用來拓展其事業版圖的商業模式，其特色為能夠較快速且精準的尋找新的客戶，使收入模式多樣化。即時競價(RTB)是一種程序化的拍賣模式，促成各個廣告版位被購買以及銷售的系統。為了讓廣告商在即時競價系統中能夠獲得其最大利益，制定一個良好的競價策略是重要且必須的。此篇論文針對廣告點擊率進行研究，廣告點擊率是在制定競價策略時，最重要的特徵元素之一。隨著網路資訊迅速的變遷，網路廣告業的資料集時常遭遇到超出詞彙(OOV)的問題，即為特徵資料出現在測試資料集(testing data)中，卻無法在訓練資料集中(training data)找到相對應的特徵資料。在這篇論文中，我們提出一個全新的方法來處理超出詞彙問題，透過使用字元型式的卷積以及廣度與深度的架構來預測廣告的點擊率。實驗結果呈現我們的模型與經過許多特徵組合的模型比較，有著更好的成果。我們同時也發現所提出的模型能夠解決在字串特徵上的超出詞彙問題。

The Internet consolidated itself as a very powerful platform that has changed the way we do business, and the way we communicate. As more users are used to the Internet, online advertising is one of the most effective ways for businesses of all sizes to expand their reach, find new customers, and diversify their revenue streams. Real-time bidding (RTB)
is a programmatic auction that allows the advertising inventory to be bought and sold on a per-impression basis. To achieve the highest profits for advertisers in RTB, it is an
important issue to formulate a good bidding strategy. This thesis focuses on click through rate (CTR), which is one of the most significant factors in setting bidding strategy. As
online information change rapidly, the data of online advertisement always suffer from out of vocabulary (OOV) problem, namely the data is presented in training set, but absent from the test set. This thesis proposes a novel way to deal with the OOV problem by using character-level convolution and uses a deep-and-wide network architecture to build a deep learning model. The experiment results indicate that the proposed method can achieve better performance than those with multiple feature combinations. Moreover,
the investigation also shows that the proposed method could benefit from the proposed scheme, which resolves the OOV problem presented in string features.

摘要 i
Abstract i
Acknowledgement i
List of Figures v
List of Tables vi
1 Introduction 1
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Paper Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2 Related Work 5
2.1 Neural networks architecture and Convolutional neural networks . . . . . . 5
2.1.1 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Convolutional Neural Networks . . . . . . . . . . . . . . . . . . . . 10
2.2 Recommender Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Collaborative Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Content-based Filtering . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.3 Wide and Deep learning for recommender systems . . . . . . . . . . 14
2.3 Out of vocabulary problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Character-level convolution dealing with out of vocabulary(OOV) 16
3.1 Character-level pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . 16
iii
3.2 Deep model - convolutional neural networks . . . . . . . . . . . . . . . . . 18
3.3 Wide model - logistic regression . . . . . . . . . . . . . . . . . . . . . . . . 19
3.4 Wide and Deep model - joint training . . . . . . . . . . . . . . . . . . . . . 20
4 Experiments 22
4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1.1 Out of vocabulary (OOV) problem . . . . . . . . . . . . . . . . . . 23
4.2 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2.1 Deep model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.2.2 Wide model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2.3 Wide and Deep model . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 Evaluate Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 Experiment results and discussion . . . . . . . . . . . . . . . . . . . . . . . 27
5 Conclusion 30

[1] Jun Wang, Weinan Zhang, and Shuai Yuan. “Display advertising with real-time
bidding (RTB) and behavioural targeting”. In: arXiv preprint arXiv:1610.03013
(2016).
[2] Matthew Richardson, Ewa Dominowska, and Robert Ragno. “Predicting clicks: estimating
the click-through rate for new ads”. In: Proceedings of the 16th international
conference on World Wide Web. ACM. 2007, pp. 521–530.
[3] Steffen Rendle. “Factorization machines with libfm”. In: ACM Transactions on Intelligent
Systems and Technology (TIST) 3.3 (2012), p. 57.
[4] Convolutional Neural Networks for Visual Recognition. http://cs231n.github.
io/convolutional-networks/.
[5] Richard P Lippmann. “Neural network classifiers for speech recognition”. In: Lincoln
Laboratory Journal 1 (1988), pp. 107–124.
[6] Henry A Rowley, Shumeet Baluja, and Takeo Kanade. “Neural network-based face
detection”. In: IEEE Transactions on pattern analysis and machine intelligence 20.1
(1998), pp. 23–38.
[7] Arun D Kulkarni. Computer vision and fuzzy-neural systems. Prentice Hall PTR,
2001.
[8] Ronan Collobert and Jason Weston. “A unified architecture for natural language
processing: Deep neural networks with multitask learning”. In: Proceedings of the
25th international conference on Machine learning. ACM. 2008, pp. 160–167.
[9] Moshe Leshno et al. “Multilayer feedforward networks with a nonpolynomial activation
function can approximate any function”. In: Neural networks 6.6 (1993),
pp. 861–867.
[10] Badrul Sarwar et al. “Item-based collaborative filtering recommendation algorithms”.
In: Proceedings of the 10th international conference on World Wide Web. ACM.
2001, pp. 285–295.
[11] Michael J Pazzani and Daniel Billsus. “Content-based recommendation systems”.
In: The adaptive web. Springer, 2007, pp. 325–341.
[12] Heng-Tze Cheng et al. “Wide & deep learning for recommender systems”. In: Proceedings
of the 1st Workshop on Deep Learning for Recommender Systems. ACM.
2016, pp. 7–10.
[13] Issam Bazzi. “Modelling out-of-vocabulary words for robust speech recognition”.
PhD thesis. Massachusetts Institute of Technology, 2002.
[14] Qiang Liu et al. “A convolutional click prediction model”. In: Proceedings of the 24th
ACM International on Conference on Information and Knowledge Management.
ACM. 2015, pp. 1743–1746.
[15] Enno Shioji and Masayuki Arai. “Neural Feature Embedding for User Response Prediction
in Real-Time Bidding (RTB)”. In: arXiv preprint arXiv:1702.00855 (2017).
[16] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. “Imagenet classification
with deep convolutional neural networks”. In: Advances in neural information processing
systems. 2012, pp. 1097–1105.
[17] Chenyi Lei et al. “Comparative Deep Learning of Hybrid Representations for Image
Recommendations”. In: Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition. 2016, pp. 2545–2553.