臺灣博碩士論文加值系統

隨著人們對於網路內容的需求日益上升，負責傳輸內容的內容傳遞網路也隨之變得更加重要。而去了解這樣的系統是如何運作的，也成為了一項重要的課題。在本研究中，我們設計出一個有辦法探索直播平台Twitch的影片傳遞架構的爬蟲系統，並藉此來分析Twitch的伺服器分布情況。透過分析封包，我們發現了Twitch用來導向用戶的負載平衡器Usher，並利用這個平衡機制作為我們爬蟲系統的依據。我們同時設計了一個演算法來取得Twitch的服務，且結合了平台數據後，成功設計出了一個能夠大規模探索Twitch內容傳遞網路的系統。和過往的研究比起來，我們不使用Planetlab作為部屬平台，相反的，我們利用了雲端服務平台和虛擬私人網路結合容器技術來達到大規模部屬的目的。這兩種爬蟲方式讓我們觀察到Twitch的伺服器多數集中在北美洲、歐洲、亞洲和大洋洲上的幾個集中的地理位置附近。我們也發現了Twitch傾向將用戶導向到地理位置上較為接近的伺服器做存取。最後，我們探討兩種不同的部屬平台中哪一個較為適合類似我們這種爬蟲目的選擇。

Content Delivery Networks (CDNs) deliver much of the world's content on the Internet today, and as demand for content grows increasingly the role of CDNs remains as crucial as ever, making it imperative to understand the internal workings of such systems. In this work, we implemented a crawler capable of exploring the distribution of Twitch's video delivery infrastructure. By analyzing traffic during a Twitch session, we identified a load-balancer named Usher responsible for redirecting clients to appropriate locations to fetch video content, which serves as the foundation for our data collection system. We designed an algorithm to programmatically access Usher's service, and by combining it with platform statistics provided by Twitch's API, we successfully developed a system that can discover Twitch's content delivery servers at a large scale. Unlike previous measurement work, we deployed our crawler onto cloud computing services as well as used virtual private networks to achieve broad network coverage. By running our crawler using two methodologies, we obtained data showing that Twitch serves its users in relatively geographical concentrated server clusters primarily located in North America, Europe, Asia, and Oceania. We also compared our two crawling methodologies and gave a conclusion which would be more suitable for crawling purposes like ours.

1 Introduction 1
2 Related Work 4
2.1 Live Streaming Services. . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 CDN Infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3 Active Network Exploration . . . . . . . . . . . . . . . . . . . . . . . 6
3 A First Look Into Twitch 8
3.1 Observations on Twitch Network Traces . . . . . . . . . . . . . . . . 9
3.1.1 HTTP Live Streaming . . . . . . . . . . . . . . . . . . . . . . 10
3.1.2 Usher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Twitch’s Server Selection Behavior . . . . . . . . . . . . . . . . . . . 12
3.2.1 Client Centric Redirection . . . . . . . . . . . . . . . . . . . . 12
3.2.2 Multi-Server Stream Hosting . . . . . . . . . . . . . . . . . . . 13
4 Preliminary Experiment 15
4.1 Experiment Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1.1 Channel to address mapping . . . . . . . . . . . . . . . . . . . 16
4.1.2 Channel Selection . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5 Large-Scale Experiment 21
5.1 System Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
5.1.1 Implementation Details . . . . . . . . . . . . . . . . . . . . . . 22
5.2 Deployment Platforms . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.2.1 Azure Virtual Machine . . . . . . . . . . . . . . . . . . . . . . 25
5.2.2 Docker Container + VPN . . . . . . . . . . . . . . . . . . . . 26
6 Evaluation 29
6.1 Data Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.1.1 How many servers does Twitch have? . . . . . . . . . . . . . . 30
6.1.2 Where are Twitch’s content servers located? . . . . . . . . . . 33
6.1.3 Which clients access which content servers? . . . . . . . . . . 34
6.2 System Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7 Conclusions and Future Work 45
7.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Bibliography 48

[1] Adhikari, V. K., Guo, Y., Hao, F., Hilt, V., and Zhang, Z. A tale of three cdns: An active measurement study of hulu and its cdns. In 2012 Proceedings IEEE INFOCOM Workshops (2012), pp. 7-12.
[2] Adhikari, V. K., Jain, S., Chen, Y., and Zhang, Z. Vivisecting youtube: An active measurement study. In 2012 Proceedings IEEE INFOCOM (2012), pp. 2521-2525.
[3] Adhikari, V. K., Yang Guo, Fang Hao, Varvello, M., Hilt, V., Steiner, M., and Zhang, Z. Unreeling netflix: Understanding and improving multi-cdn movie delivery. In 2012 Proceedings IEEE INFOCOM (2012), pp. 1620-1628.
[4] Apple. Http live streaming, 2021.
[5] Bottger, T., Cuadrado, F., Tyson, G., Castro, I., and Uhlig, S. Open connect everywhere: A glimpse at the internet ecosystem through the lens of the netflix cdn. SIGCOMM Comput. Commun. Rev. 48, 1 (Apr. 2018), 28-34.
[6] Calder, M., Fan, X., Hu, Z., Katz-Bassett, E., Heidemann, J., and Govindan, R. Mapping the expansion of google's serving infrastructure. In Proceedings of the 2013 Conference on Internet Measurement Conference (New York, NY, USA, 2013), IMC '13, Association for Computing Machinery, p. 313-326.
[7] et al., D. Internet scale user-generated live video streaming: The twitch case. In Proceedings of PAM (New York, NY, 02 2017), Springer, pp. 60-71.
[8] et al., J. Zooming in on wide-area latencies to a global cloud provider. In Proceedings of the ACM Special Interest Group on Data Communication (New York, NY, USA, 2019), SIGCOMM '19, ACM Press, pp. 104-116.
[9] Giordano, D., Traverso, S., Grimaudo, L., Mellia, M., Baralis, E., Tongaonkar, A., and Saha, S. Youlighter: A cognitive approach to unveil youtube cdn and changes. IEEE Transactions on Cognitive Communications and Networking 1, 2 (2015), 161-174.
[10] Google. Chrome dev tools, 2021.
[11] Jimenez, L. R., Solera, M., and Toril, M. A network-layer qoe model for youtube live in wireless networks. IEEE Access 7 (2019), 70237-70252.
[12] Pires, K., and Simon, G. Youtube live and twitch: A tour of user-generated live streaming systems. In Proceedings of the 6th ACM Multimedia Systems Conference (New York, NY, USA, 2015), MMSys '15, Association for Computing Machinery, p. 225-230.
[13] Quan, L., Heidemann, J., and Pradkin, Y. Trinocular: Understanding internet reliability through adaptive probing. In Proceedings of the ACM SIGCOMM Conference (Hong Kong, China, Aug. 2013), ACM, pp. 255-266.
[14] Raman, A., Tyson, G., and Sastry, N. Facebook (a)live? are live social broadcasts really broadcasts? In Proceedings of the 2018 World Wide Web Conference (Republic and Canton of Geneva, CHE, 2018), WWW '18, International World Wide Web Conferences Steering Committee, p. 1491-1500.
[15] similarweb. similarweb, 2021.
[16] Torres, R., Finamore, A., Kim, J. R., Mellia, M., Munafo, M. M., and Rao, S. Dissecting video server selection strategies in the youtube cdn. In 2011 31st International Conference on Distributed Computing Systems (2011), pp. 248-257.
[17] Twitch. Twitch engineering blog, 2017.
[18] TwitchTracker. Twitchtracker, 2021.
[19] Wang, B., Zhang, X., Wang, G., Zheng, H., and Zhao, B. Y. Anatomy of a personalized livestreaming system. In Proceedings of the 2016 Internet Measurement Conference (New York, NY, USA, 2016), IMC '16, Association for Computing Machinery, p. 485-498.
[20] Warrior, M. A., Xiao, Y., Varvello, M., and Kuzmanovic, A. Dekodi: Understanding the kodi ecosystem. In Proceedings of The Web Conference 2020 (New York, NY, USA, 2020), WWW '20, Association for Computing Machinery, p. 1171-1181.
[21] Yu, C., Xu, Y., Liu, B., and Liu, Y. \can you see me now?" a measurement study of mobile video calls. In IEEE INFOCOM 2014 - IEEE Conference on Computer Communications (2014), pp. 1456-1464.
[22] Zhang, C., and Liu, J. On crowdsourced interactive live streaming: A twitch.tv-based measurement study. Proceedings of the 25th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, NOSS- DAV 2015 (02 2015).