(3.236.214.19) 您好!臺灣時間:2021/05/09 21:04
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果

詳目顯示:::

: 
twitterline
研究生:徐媺晴
研究生(外文):Mei-Ching HSU
論文名稱:感知無線電網路之多通道交會問題:強化學習應用
論文名稱(外文):Multi-channel Rendezvous in Cognitive Radio Network: a Reinforcement Learning Approach
指導教授:林嘉慶林嘉慶引用關係
指導教授(外文):Jia-Chin Lin
學位類別:碩士
校院名稱:國立中央大學
系所名稱:通訊工程學系
學門:工程學門
學類:電資工程學類
論文出版年:2020
畢業學年度:108
語文別:中文
論文頁數:54
中文關鍵詞:強化學習多通道交會無線感知系統
相關次數:
  • 被引用被引用:0
  • 點閱點閱:76
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:16
  • 收藏至我的研究室書目清單書目收藏:0
本論文為探討如何使用強化學習解決在感知無線電網路中多通道交會問題。感知無線電網路中多通道交會問題是指兩個二級用戶如何在有限的時間內選擇跳到同個通道並成功交換彼此的訊息。本篇論文是在一個對稱性、同步、同質性以及全局共同標號的假設下進行研究,同時為了更貼近實際情況,我們假設每個通道有著不同的通道狀態,並且無法讓用戶得知該通道的狀態情況,某些狀態可能會導致雖然兩個用戶在同個通道卻無法成功通訊。此種無法得知通道狀態的情況下的交會問題稱作盲交會。我們在上述假設下提出了一個快速強化學習演算法,讓兩個用戶學習在不同的狀態下成功交會的交會策略。我們提出的快速強化學習演算法能大幅增加通道選擇策略學習的效率,並且在收斂後學到一組能與已有最佳解的特定通道狀態相比擬的平均交會時間(ETTR),與在沒有最佳解的狀態下有著最佳的效能(最低的ETTR)。
In this thesis, we consider the multichannel rendezvous problem in cognitive radio networks (CRNs) where the probability that two users hopping on the same channel have a successful rendezvous is a function of channel states. The channel states are modelled by stochastic processes with joint distributions known to users. However, the exact state of a channel at any time is not observable. We derived that the lower bound of the ETTR of the general channel model is the ETTR of the fast time-varying channel model and the upper bound is the ETTR of the slow time-varying channel model. By formulating such a multichannel rendezvous problem as an adversarial bandit problem, we propose using a reinforcement learning approach to learn the channel selection probabilities pi(t), i = 1; 2; : : : ;N. Our experimental results show that the reinforcement learning approach is very effective and yields comparable ETTRs when comparing to various approximation policies in the literature.
目    錄
中文提要 ……………………………………………………………… i
英文摘要 ……………………………………………………………… ii
誌謝 ……………………………………………………………… iii
目錄 ……………………………………………………………… iv
圖目錄 ……………………………………………………………… v
表目錄 ……………………………………………………………… vi
符號說明 ……………………………………………………………… vii
一、 緒論………………………………………………………… 1
二、 系統模型…………………………………………………… 11
三、 相關論文回顧……………………………………………… 13
3-1 廣義時變通道模型………………………………………… 13
3-2 兩種狀態的馬可夫通道模型……………………………… 15
3-3 正相關馬可夫鍊中的通道交會時間期望值下界………… 16
3-4 將多通道交會問題視為強化學習問題…………………… 18
四、 用於解決感知無線電網路多通道交會問題之快速收斂強化學習演算法……………………………………………… 20
五、 實驗與綜合比較…………………………………………… 25
六、 結論………………………………………………………… 36
參考文獻 ……………………………………………………………… 38


圖目錄 List of Figures
圖一 在ω=0.5時,快速強化學習演算法根據不同ρ所學到的通道選擇策略……………………………………………… 27
圖二 在ω=0.5時,〔36〕演算法根據不同ρ所學到的通道選擇策略(直接採用〔36〕的圖一)……………………… 29
圖三 快速強化學習演算法的學習曲線………………………… 34


表目錄 List of Tables
表一 在ρ=0.1時,不同ω情況下的ETTR…………………… 31
表二 在ρ=0.5時,不同ω情況下的ETTR…………………… 32
表三 在ρ=0.9時,不同ω情況下的ETTR…………………… 32
[1]Kolodzy, Paul, and Interference Avoidance. "Spectrum policy task force." Federal Commun. Comm., Washington, DC, Rep. ET Docket 40.4 (2002): 147-158.
[2]Mitola, Joseph, and Gerald Q. Maguire. "Cognitive radio: making software radios more personal." IEEE personal communications 6.4 (1999): 13-18.
[3]Chen, Tao, et al. "CogMesh: A cluster-based cognitive radio network." 2007 2nd IEEE international symposium on new frontiers in dynamic spectrum access networks. IEEE, 2007.
[4]Le, Long, and Ekram Hossain. "A MAC protocol for opportunistic spectrum access in cognitive radio networks." 2008 IEEE Wireless Communications and Networking Conference. IEEE, 2008.
[5]Zhang, Xi, and Hang Su. "CREAM-MAC: Cognitive radio-enabled multi-channel MAC protocol over dynamic spectrum access networks." IEEE Journal of Selected Topics in Signal Processing 5.1 (2010): 110-123.
[6]Zhao, Jun, Haitao Zheng, and Guang-Hua Yang. "Distributed coordination in dynamic spectrum allocation networks." First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005.. IEEE, 2005.
[7]Chang, Guey-Yun, and Jen-Feng Huang. "A fast rendezvous channel-hopping algorithm for cognitive radio networks." IEEE Communications Letters 17.7 (2013): 1475-1478.
[8]Shih, C-F., Tsung Ying Wu, and Wanjiun Liao. "DH-MAC: A dynamic channel hopping MAC protocol for cognitive radio networks." 2010 IEEE International Conference on Communications. IEEE, 2010.
[9]Bahl, Paramvir, Ranveer Chandra, and John Dunagan. "SSCH: slotted seeded channel hopping for capacity improvement in IEEE 802.11 ad-hoc wireless networks." Proceedings of the 10th annual international conference on Mobile computing and networking. 2004.
[10]Bian, Kaigui. "Maximizing rendezvous diversity in rendezvous protocols for decentralized cognitive radio networks." IEEE transactions on Mobile Computing 12.7 (2012): 1294-1307.
[11]Bian, Kaigui, Jung-Min Park, and Ruiliang Chen. "A quorum-based framework for establishing control channels in dynamic spectrum access networks." Proceedings of the 15th annual international conference on Mobile computing and networking. 2009.
[12]Chang, Guey-Yun, et al. "Novel channel-hopping schemes for cognitive radio networks." IEEE Transactions on Mobile Computing 13.2 (2012): 407-421.
[13]DaSilva, Luiz A., and Igor Guerreiro. "Sequence-based rendezvous for dynamic spectrum access." 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks. IEEE, 2008.
[14]Hou, Fen, et al. "Asynchronous multichannel MAC design with difference-set-based hopping sequences." IEEE Transactions on Vehicular Technology 60.4 (2011): 1728-1739.
[15]Kondareddy, Yogesh Reddy, and Prathima Agrawal. "Synchronized MAC protocol for multi-hop cognitive radio networks." 2008 IEEE International Conference on Communications. IEEE, 2008.
[16]Lin, Zhiyong, et al. "Jump-stay based channel-hopping algorithm with guaranteed rendezvous for cognitive radio networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
[17]Mo, Jeonghoon, Hoi-Sheung Wilson So, and Jean Walrand. "Comparison of multichannel MAC protocols." IEEE Transactions on mobile computing 7.1 (2007): 50-65.
[18]Shin, Jongmin, Dongmin Yang, and Cheeha Kim. "A channel rendezvous scheme for cognitive radio networks." IEEE Communications Letters 14.10 (2010): 954-956.
[19]Wang, Jen-Hung, et al. "A Reinforcement Learning Approach for the Multichannel Rendezvous Problem." 2019 IEEE Globecom Workshops (GC Wkshps). IEEE, 2019.
[20]Yang, D., J. Shin, and C. Kim. "Deterministic rendezvous scheme in multichannel access networks." Electronics Letters 46.20 (2010): 1402-1404.
[21]Zhang, Yifan, et al. "ETCH: Efficient channel hopping for communication rendezvous in dynamic spectrum access networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
[22]Bian, Kaigui, Jung-Min Park, and Ruiliang Chen. "A quorum-based framework for establishing control channels in dynamic spectrum access networks." Proceedings of the 15th annual international conference on Mobile computing and networking. 2009.
[23]Yang, D., J. Shin, and C. Kim. "Deterministic rendezvous scheme in multichannel access networks." Electronics Letters 46.20 (2010): 1402-1404.
[24]Theis, Nick C., Ryan W. Thomas, and Luiz A. DaSilva. "Rendezvous for cognitive radios." IEEE transactions on mobile computing 10.2 (2010): 216-227.
[25]Lin, Zhiyong, et al. "Jump-stay based channel-hopping algorithm with guaranteed rendezvous for cognitive radio networks." 2011 Proceedings IEEE INFOCOM. IEEE, 2011.
[26]Gu, Zhaoquan, et al. "Nearly optimal asynchronous blind rendezvous algorithm for cognitive radio networks." 2013 IEEE international conference on sensing, communications and networking (SECON). IEEE, 2013.
[27]Chang, Guey-Yun, and Jen-Feng Huang. "A fast rendezvous channel-hopping algorithm for cognitive radio networks." IEEE Communications Letters 17.7 (2013): 1475-1478.
[28]Chang, Guey-Yun, et al. "Novel channel-hopping schemes for cognitive radio networks." IEEE Transactions on Mobile Computing 13.2 (2012): 407-421.
[29]Gu, Zhaoquan, Qiang-Sheng Hua, and Weiguo Dai. "Fully distributed algorithms for blind rendezvous in cognitive radio networks." Proceedings of the 15th ACM international symposium on Mobile ad hoc networking and computing. 2014.
[30]Chang, Cheng-Shang, et al. "Efficient encoding of user IDs for nearly optimal expected time-to-rendezvous in heterogeneous cognitive radio networks." IEEE/ACM Transactions on Networking 25.6 (2017): 3323-3337.
[31]Li, Guyue, et al. "Deterministic distributed rendezvous algorithms for multi-radio cognitive radio networks." Proceedings of the 17th ACM international conference on Modeling, analysis and simulation of wireless and mobile systems. 2014.
[32]Yu, Lu, et al. "Multiple radios for fast rendezvous in cognitive radio networks." IEEE Transactions on Mobile Computing 14.9 (2014): 1917-1931.
[33] Yu, Lu, et al. "Adjustable rendezvous in multi-radio cognitive radio networks." 2015 IEEE global communications conference (GLOBECOM). IEEE, 2015.
[34]Chang, Yeh-Cheng, Cheng-Shang Chang, and Jang-Ping Sheu. "An enhanced fast multi-radio rendezvous algorithm in heterogeneous cognitive radio networks." IEEE Transactions on Cognitive Communications and Networking 4.4 (2018): 847-859.
[35]Chang, Cheng-Shang, et al. "ETTR Bounds and Approximation Solutions of Blind Rendezvous Policies in Cognitive Radio Networks with Random Channel States." arXiv preprint arXiv:1906.10424 (2019).
[36]Wang, Jen-Hung, et al. "A Reinforcement Learning Approach for the Multichannel Rendezvous Problem." 2019 IEEE Globecom Workshops (GC Wkshps). IEEE, 2019.
[37]Ross, Sheldon M. "Stochastic Processes. John Wiley & Sons." New York (1996).
[38]Chang, Cheng-Shang, XiuLi Chao, and Michael Pinedo. "Integration of discrete-time correlated Markov processes in a TDM system." Probability in the Engineering and Informational Sciences 4.1 (1990): 29-56.
[39]Marshall, Albert W., Ingram Olkin, and Barry C. Arnold. Inequalities: theory of majorization and its applications. Vol. 143. New York: Academic press, 1979.
[40]Tchen, André H. "Inequalities for distributions with given marginals." The Annals of Probability 8.4 (1980): 814-827.
[41]Rolski, Tomasz. "Upper bounds for single server queues with doubly stochastic Poisson arrivals." Mathematics of Operations Research 11.3 (1986): 442-450.
[42]Auer, Peter, et al. "The nonstochastic multiarmed bandit problem." SIAM journal on computing 32.1 (2002): 48-77.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔