跳到主要內容

臺灣博碩士論文加值系統

(44.222.189.51) 您好!臺灣時間:2024/05/20 15:19
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:王世偉
研究生(外文):Shih-Wei Wang
論文名稱:基於深度強化學習技術實現低軌道衛星軟體定義網路之路由決策
論文名稱(外文):Realization of Routing Decisions for Low-Orbit Satellite Software-Defined Networks Based on the Deep Reinforcement Learning Technique
指導教授:蔡智強蔡智強引用關係
指導教授(外文):Ji-Chiang Tsai
口試委員:袁世一陳立勝
口試委員(外文):Shih-Yi YuanLi-Sheng Chen
口試日期:2021-07-27
學位類別:碩士
校院名稱:國立中興大學
系所名稱:電機工程學系所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2021
畢業學年度:109
語文別:中文
論文頁數:34
中文關鍵詞:低軌道衛星網路動態拓樸軟體定義網路深度強化學習
外文關鍵詞:Low-Earth Orbit (LEO) Satellite NetworksDynamic TopologySoftware-Defined Networking (SDN)Deep Reinforcement Learning (DRL)
相關次數:
  • 被引用被引用:0
  • 點閱點閱:264
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
低軌道衛星網路可輔助現行低覆蓋率的5G網路,然而由於衛星網路具有動態拓樸的性質,衛星間的通訊聯結時時在改變,造成網路路由決策的不確定性,進而影響資料轉送的效能,此時便可以藉由軟體定義網路全域考量的優勢,即時發送符合當前網路拓樸所需的路由決策來解決前述的問題。而近年來發展迅速的深度強化學習技術,很適合提供複雜動態情境下的動作決策,藉由結合前述兩種新穎的技術,便可以提昇低軌道衛星網路的可行性與可用性。

本論文參考美國Iridium銥衛星架構建置深度強化學習所需的訓練模擬環境,此衛星星座共使用六十六顆衛星來涵蓋全球通訊範圍,而由於地面基地站可知所有衛星的運行軌道以及兩通訊點的位置座標,我們於基地站上建置軟體定義網路發送封包轉送規則的控制器(Controller),並使用訓練好的深度強化學習模型進行路由決策。最後,我們也進行模擬實驗來分析深度強化學習模型應用於低軌道衛星路由決策的效能。
The Low-Earth Orbit (LEO) satellite network can assist the current low-coverage 5G network, However, due to the dynamic nature of the satellite network topology, the communication connections between satellites are constantly varying, causing uncertainty in network routing decisions, which in turn affects the efficiency of data transmission. Here, the advantage from the global consideration of Software-Defined Networking (SDN) can be used to instantly send routing decisions that meet the current network topology so as to solve the above issue. On the other hand, the rapid development of Deep Reinforcement Learning (DRL) technology in recent years is very suitable for providing action decisions in complex dynamic situations. By combining the above two novel technologies, the feasibility and availability of LEO satellite networks can be improved.

In this thesis, we construct the training simulation environment required for the DRL model according to the U.S. Iridium satellite architecture. This satellite constellation uses a total of 66 satellites to cover the global communication range. Since the ground base station can know the orbits of all satellites and the position coordinates of the two communication points, we install an SDN controller on the base station to send packet forwarding rules by exploiting the trained DRL model to make routing decisions. Finally, we also conduct simulation experiments to analyze the performance of the DRL model applied to the routing decisions for LEO satellites.
摘要 i
Abstract ii
目錄 iii
表目錄 v
圖目錄 vi
第一章 緒論 1
1.1 前言 1
1.2 研究動機與目的 2
1.3 論文架構 2
第二章 文獻探討 3
2.1 低軌道衛星 3
2.1.1 低軌道衛星特性 3
2.1.2 低軌道衛星星座 4
2.1.3 低軌道衛星間的通訊 4
2.1.4 銥衛星運行架構 5
2.2 軟體定義網路 6
2.2.1 SDN的基本概念 7
2.2.2 SDN的主要構建 7
2.2.3 SDN的控制平面 8
2.2.4 頻內控制與頻外控制 9
2.3 深度強化學習 10
2.3.1 強化學習 10
2.3.2 深度強化學習 10
2.3.3 深度強化學習常用符號 11
2.3.4 策略梯度法 11
2.3.5 決定性策略梯度法 12
2.4 卷積神經網路 12
第三章 系統架構與研究方法 13
3.1 低軌道衛星模擬器 13
3.1.1 模擬器開發動機 13
3.1.2 模擬器架構 14
3.1.3 模擬器拓樸輸出 15
3.2 深度強化學習的實作 16
3.2.1 動作(Action) 16
3.2.2 回饋值(Reward) 18
3.2.3 狀態(State) 19
3.2.4 代理人(Agent) 20
3.2.5 神經網路模型(Model) 20
3.2.6 損失函數(Loss Function) 22
3.3 模型訓練與測試 22
3.3.1 訓練階段 22
3.3.2 測試階段 22
第四章 實驗結果與討論 24
4.1 模擬器操作 24
4.2 訓練過程 25
4.2.1 硬體與軟體環境 25
4.2.2 DPG超參數的設定 26
4.2.3 Loss的收斂 26
4.3 訓練結果 27
4.3.1 正確率 27
4.3.2 預測實例 27
4.3.3 結果探討 29
第五章 結論與未來工作 31
參考文獻 32
[1]Geostationary orbit – Wikipedia, https://en.wikipedia.org/wiki/Geostationary _orbit, June 2021
[2]Low Earth orbit - Wikipedia, https://en.wikipedia.org/wiki/Low_Earth_orbit, 23 July 2021
[3]File:Comparison satellite navigation orbits.svg - Wikipedia, https://en.wikipedia.org/wiki/File:Comparison_satellite_navigation_orbits.svg, 5 August 2020
[4]SpaceX - Wikipedia, https://www.spacex.com/, July 2021
[5]Iridium Satellite Communications Truly Global Communications, https://www.iridium.com/, July 2021
[6]Software-defined networking - Wikipedia ,https://en.wikipedia.org/wiki/ Software-defined_networking, July 2021
[7]Silver, D., Schrittwieser, J., Simonyan, K. et al. Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017). https://doi.org/ 10.1038/ nature24270
[8]V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level controlthrough deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529, 2015.
[9]低軌道衛星:低軌道衛星主要用於軍事目標探測, 利用低軌道衛星容易獲得目標 華人百科,https://www.itsfun.com.tw/%E4%BD%8E%E8%BB %8C%E9%81 %93%E8%A1%9B%E6%98%9F/wiki-8155546, July 2021
[10]E. Ekici, I. F. Akyildiz and M. D. Bender, "A distributed routing algorithm for datagram traffic in LEO satellite networks," in IEEE/ACM Transactions on Networking, vol. 9, no. 2, pp. 137-147, April 2001, doi: 10.1109/90.917071.
[11] Y. Zhu, L. Qian, L. Ding, F. Yang, C. Zhi and T. Song, "Software defined routing algorithm in LEO satellite networks," 2017 International Conference on Electrical Engineering and Informatics (ICELTICs), 2017, pp. 257-262, doi: 10.1109/ICELTICS.2017.8253282.
[12]CelesTrak Orbit Visualization, https://celestrak.com/cesium/orbit- viz.php?tle=/NORAD/elements/iridium-NEXT.txt&satcat=/pub/satcat.txt&orbits=0&pixelSize=3&samplesPerPeriod=90&referenceFrame=1, July 2021
[13]Open Networking Foundation, https://opennetworking.org/, July 2021
[14] What is Software Defined Networking (SDN)? - Sagar Nangare, https://sagarnangare.com/what-is-software-defined-networking-sdn/, December 2017
[15] P18-21 關鍵看法(3).pdf, https://www.taifex.com.tw/file/taifex/CHINESE/ 10/moth/P18-21%20%E9%97%9C%E9%8D%B5%E7%9C%8B%E6%B3 %95(3).pdf, July 2021
[16] A high-level overview of the software-defined networking architecture – Wikipedia, https://en.wikipedia.org/wiki/Software-defined_networking#/ media/File:SDN-architecture-overview-transparent.png, December 2013
[17]OpenFlow network: a) out-of-band; b) in-band control. Download Scientific Diagram, https://www.researchgate.net/figure/OpenFlow-network-a-out-of-band-b-in-band-control_fig1_291952929, January 2016
[18]技術文章-強化學習(Reinforcement Learning):入門指南--MATLAB, Simulink, Rational 專業技術服務盡在鈦思科技, https://www.terasoft.com.tw/support/tech_articles/reinforcement_learning_a_brief_guide.asp, July 2021
[19]Mnih, Volodymyr & Kavukcuoglu, Koray & Silver, David & Graves, Alex & Antonoglou, Ioannis & Wierstra, Daan & Riedmiller, Martin. (2013). Playing Atari with Deep Reinforcement Learning.
[20]Lillicrap, Timothy & Hunt, Jonathan & Pritzel, Alexander & Heess, Nicolas & Erez, Tom & Tassa, Yuval & Silver, David & Wierstra, Daan. (2015). Continuous control with deep reinforcement learning. CoRR.
[21]Silver, David & Lever, Guy & Heess, Nicolas & Degris, Thomas & Wierstra, Daan & Riedmiller, Martin. (2014). Deterministic Policy Gradient Algorithms. 31st International Conference on Machine Learning, ICML 2014. 1.
[22]Sutton R S , Mcallester D , Singh S , et al. Policy Gradient Methods for Reinforcement Learning with Function Approximation[J]. Submitted to Advances in Neural Information Processing Systems, 1999, 12.
[23]Convolutional neural network – Wikipedia, https://en.wikipedia.org/wiki/ Convolutional_neural_network, July 2021
[24]Understanding of Convolutional Neural Network (CNN) — Deep Learning by Prabhu Medium, https://medium.com/@RaghavPrabhu/understanding-of-convolutional-neural-network-cnn-deep-learning-99760835f148, Mar 2018
[25]Systems Tool Kit (STK) – AGI, http://www.agi.com/products/stk, July 2021
[26]Simon Kassing, Debopam Bhattacherjee, André Baptista Águas, Jens Eirik Saethre, and Ankit Singla. 2020. Exploring the "Internet from space" with Hypatia. In Proceedings of the ACM Internet Measurement Conference (IMC '20). Association for Computing Machinery, New York, NY, USA, 214–229. DOI:https://doi.org/10.1145/3419394.3423635
[27]Introduction — cartopy 0.19.0rc2.dev8+gd251b2f documentation https://scitools. org.uk/cartopy/docs/latest/, 2011
[28]入門 從Q學習到DDPG,一文簡述多種強化學習演算法 - ITW01, https://itw01.com/NRTZERA.html, January 2018
[29]Dijkstra's algorithm – Wikipedia, https://en.wikipedia.org/wiki/Dijkstra %27s_ algorithm, July 2011
[30]Gym, https://gym.openai.com/, July 2021
[31]PyEphem Home Page — PyEphem home page, https://rhodesmill.org/ pyephem/, July 2021
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top