臺灣博碩士論文加值系統

由於線上遊戲直播平台的興起，如何有效率地存取這些遊戲影片成為重要的議題。熱門的串流遊戲影片開創了前所未有的市場商機，並造就了許多技術的挑戰。在本篇論文中，我們由兩種層面實現有效率的影片瀏覽：遊戲事件偵測以及遊戲精彩片段偵測。在遊戲事件偵測中，藉由偵測遊戲中重要畫面所出現的關鍵文字並連結時間標記，搭配使用者介面方便遊戲影片的導覽。在遊戲精彩片段偵測中，我們融合了影像的視覺資訊、遊戲事件資訊、以及串流平台中觀眾行為的資訊，建立了兩種不同模式的模組，分別為基於生理情緒模組(arousal model)以及基於數據分析模組(SVM model)。我們並以基因演算法(genetic algorithm)選擇最佳的影片長度，最後將這些精彩片段串聯，便可有效呈現影片中重要之片段。在促進串流平台頻寬調整研究上，我們提出遊戲影片精彩片段預測方法，藉此在即將發生重要片段時，系統能夠規劃給予更多的頻寬，綜合實驗分析結果，我們從各角度展現出以上這些方法的效果及優點。

Efficient access of broadcasted game videos is urgently demanded due to the emergence of live streaming platforms. The popularity of game video streaming builds a big market showing commercial potentials and arising many technical challenges. In this thesis, we facilitate efficient access from two aspects: event detection and game highlight detection. By recognizing designated text displayed on screen when important events occur, we associate game events with time stamps, and accordingly develop an interface to facilitate direct access. For highlight detection, we jointly consider visual features, event features, and viewer’s behaviors to construct two highlight models, i.e., the psychophysiological approach based on the arousal model and the data-driven approach based on support vector machine (SVM). In addition, we propose an optimal subset selection based on the genetic algorithm for video sub-segment selection. The concatenated highlight segments enable compact game video presentation. To facilitate adaptive bitrate streaming, a highlight forecast model is built so that the streaming system can allocate higher bitrate for more important segments on the fly. Comprehensive experiments demonstrate effectiveness of the proposed methods from various perspectives.

Chapter 1 INTRODUCTION 1
1.1 Motivation 1
1.2 System Overview 3
1.3 Contributions 5
1.4 Thesis Organization 6
Chapter 2 RELAED WORKS 7
2.1 Live Streaming Systems 7
2.2 Game Video Analysis 8
2.3 Video Event Detection and Summarization 8
2.4 Summary 10
Chapter 3 AUTOMATIC TEXT BROADCAST 11
3.1 Event Text Detection 11
3.2 Game Event Recognition 13
3.3 Text Broadcast Generation 15
Chapter 4 HIGHLIGHT DETECTION AND FORECAST 16
4.1 Feature Design and Extraction 16
4.1.1 Visual Feature Extraction 16
4.1.2 Event Feature Extraction 19
4.1.3 Chat Feature Extraction 19
4.2 Highlight Detection Models 22
4.2.1 Arousal Model 22
4.2.2 SVM Model 24
4.2.3 Optimal Subset Selection 25
4.3 Highlight Forecast Model 30
4.3.1 Exponential Smoothing Forecast Model 31
Chapter 5 EXPERIMENTAL RESULTS 33
5.1 Dataset and Experimental Setup 33
5.2 Evaluation Results of Text Broadcast 34
5.3 Evaluation Results of Highlight Detection 35
5.4 Evaluation Results of Highlight Forecast 42
Chapter 6 CONCLUSION AND FUTURE WORKS 43
6.1 Conclusion 43
6.2 Future Works 44
REFERENCES 45
APPENDIX 49
PUBLICATION LIST 49

[1] M. Kaytoue, A. Silva, L. Cerf, W. Meira Jr., and C. Raissi. Watch Me Playing, I am a Professional: A First Study on Video Game Live Streaming. In Proceedings of International Conference Companion on World Wide Web, pages 1181–1188, 2012.
[2] A. Hanjalic and L.-Q. Xu. Affective Video Content Representation and Modeling. IEEE Transactions on Multimedia, 7(1), pages 143–154, 2005.
[3] K. Pires and G. Simon. Youtube Live and Twitch: A Tour of User-Generated Live Streaming Systems. In Proceedings of ACM Multimedia Systems Conference, pages 225–230, 2015.
[4] K. Pires and G. Simon. Dash in Twitch: Adaptive Bitrate Streaming in Live Game Streaming Platforms. In Proceedings of Workshop on Design, Quality and Deployment of Adaptive Video Streaming, pages 13–18, 2014.
[5] W. A. Hamilton, O. Garretson, and A. Kerne. Streaming on Twitch: Fostering Participatory Communities of Play within Live Mixed Media. In Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems, pages 1315–1324, 2014.
[6] J. Douglass. Computer Visions of Computer Games: Analysis and Visualization of Play Recordings. In Proceedings of Workshop on Media Arts, Science, and Technology: The Future of Interactive Media, 2009.
[7] J. M. Lewis, P. Trinh, and D. Kirsh. A Corpus Analysis of Strategy Video Game Play in Starcraft: Brood War. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society, pages 687–692, 2011.
[8] F. Rioult, J.-P. Metivier, B. Helleu, N. Scelles, and C. Durand. Mining Tracks of Competitive Video Games. AASRI Procedia, 8, pages 82–87, 2014.
[9] M. Riegler, M. Lux, V. Charvillat, A. Carlier, R. Vliegendhart, and M. Larson. Videojot: A Multifunctional Video Annotation Tool. In Proceedings of ACM International Conference on Multimedia Retrieval, pages 534–537, 2014.
[10] H. K. Stensland, V. R. Gaddam, M. Tennoe, E. Helgedagsrud, M. Naess, H. K. Alstad, A. Mortensen, R. Langseth, S. Ljodal, O. Landsverk, C. Griwodz, P. Halvorsen, M. Stenhaug, and D. Johansen. Bagadus: An Integrated Real-Time System for Soccer Analytics. ACM Transactions on Multimedia Computing, Communications, and Applications, 10(1s), Article No. 14, 2014.
[11] F. Sulser, I. Giangreco, and H. Schuldt. Crowd-Based Semantic Event Detection and Video Annotation for Sports Videos. In Proceedings of ACM Workshop on Crowdsourcing for Multimedia, pages 63–68, 2014.
[12] M. Tavassolipour, M. Karimian, and S. Kasaei. Event Detection and Summarization in Soccer Videos Using Bayesian Network and Copula. IEEE Transactions on Circuits and Systems for Video Technology, 24(2), pages 291–304, 2014.
[13] N. Nguyen and A. Yoshitaka. Soccer Video Summarization based on Cinematography and Motion Analysis. In Proceedings of IEEE International Workshop on Multimedia Signal Processing, 2014.
[14] C.-Y. Chiu, P.-C. Lin, S.-Y. Li, T.-H. Tsai, and Y.-L. Tsai. Tagging Webcast Text in Baseball Videos by Video Segmentation and Text Alignment. IEEE Transactions on Circuits and Systems for Video Technology, 22(7), pages 999–1013, 2012.
[15] M.-C. Hu, M.-H. Chang, J.-L. Wu, and L. Chi. Robust Camera Calibration and Player Tracking in Broadcast Basketball Video. IEEE Transactions on Multimedia, 13(2), pages 266–279, 2011.
[16] C.-M. Chen and L.-H. Chen. Novel Framework for Sports Video Analysis: A Basketball Case Study. In Proceedings of IEEE International Conference on Image Processing, pages 961–965, 2014.
[17] G. Evangelopoulos, A. Zlatintsi, A. Potamianos, P. Maragos, K. Rapantzikos, G. Skoumas, and Y. Avrithis. Multimodal Saliency and Fusion for Movie Summarization based on Aural, Visual, and Textual Attention. IEEE Transactions on Multimedia, 15(7), pages 1553–1568, 2013.
[18] C.-M. Tsai, L.-W. Kang, C.-W. Lin, and W. Lin. Scene-Based Movie Summarization Via Role-Community Networks. IEEE Transactions on Circuits and Systems for Video Technology, 23(11), pages 1927–1940, 2013.
[19] T. Lu, Y. Weng, and G. Wang. Audiotory Movie Summarization by Detecting Scene Changes and Sound Events. In Proceedings of International Conference on Pattern Recognition, pages 756–760, 2014.
[20] L. Duan, D. Xu, and S.-F. Chang. Exploiting Web Images for Event Recognition in Consumer Videos: A Multiple Source Domain Adaptation Approach. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pages 1338–1345, 2012.
[21] C. T. Dang and H. Radha. Heterogeneity Image Patch Index and Its Application to Consumer Video Summarization. IEEE Transactions on Image Processing, 23(6), pages 2704–2718, 2014.
[22] Y. Cong, J. Yuan, and J. Luo. Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection. IEEE Transactions on Multimedia, 14(1), pages 66–75, 2012.
[23] B. Zhao and E. P. Xing. Quasi Real-Time Summarization for Consumer Videos. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pages 2513–2520, 2014.
[24] R. Smith. An Overview of the Tesseract OCR Engine. In Proceedings of International Conference on Document Analysis and Recognition, pages 629-633, 2007.
[25] C. Patel, A. Patel, and D. Patel. Optical Character Recognition by Open Source OCR tool Tesseract: A Case Study. International Journal of Computer Application, 55(10), pages 50-56, 2012.
[26] R. Lienhart. Comparison of Automatic Shot Boundary Detection Algorithms. In Proceedings of SPIE Storage and Retrieval for Still Image and Video Databases VII, volume 3656, pages 290–301, 1999.
[27] C.-C. Chang and C.-J. Lin. Libsvm: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2(27), pages 1–27, 2011.
[28] W.-T. Chu, W.-H. Cheng, J. Y.-J. Hsu, and J.-L. Wu. Towards Semantic Indexing and Retrieval Using Hierarchical Audio Models. Multimedia Systems, 10(6), pages 570–583, 2005.
[29] R. G. Brown. Exponential Smoothing for Predicting Demand. Cambridge, Massachusetts: Arthur D. Little Inc., 1956.
[30] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-Scale Video Classification with Convolutional Neural Networks. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, pages 1725–1732, 2014.
[31] M. Sun, A. Farhadi, and S. Seitz. Ranking Domain-Specific Highlights by Analyzing Edited Videos. In Proceedings of European Conference on Computer Vision, pages 787–802, 2014.