近年來由於無線通訊技術的發展，無線網路的用戶也越來越多，頻譜的不足已成為了一個重要的問題。現今除了ISM等頻帶為非授權就能使用的頻帶外，大部分頻帶都需授權才能使用。研究發現，傳統的頻譜分配方式導致了頻譜利用率不高。為了解決頻譜利用率低的問題，感知無線電網路的發展是一項重要技術。
感知無線電是能有效率的提高頻譜利用率的智慧型通訊技術，它是以不干擾主要授權用戶傳輸為原則的跨系統資源分享網路，而快速且準確的頻譜偵測對於建立可靠的感知無線電是很重要的。因此，本篇論文旨在研究主要授權系統的頻譜利用率大小對系統效益的影響，而閒置頻譜利用率會受到頻譜偵測時的誤判率所影響。當次要使用者偵測通道時的誤判率越大，閒置頻譜利用率相對越小，反之，當誤判率越小，閒置頻譜利用率相對越大。本研究使用增強學習機制中的Q-Learning演算法之特點，藉著增強學習機制能不斷學習過去經驗的特性，來有效降低誤判率的發生，並提出修改之方法，稱為DQ-Learning。研究結果顯示，採用本論文所提出的方法可以有效的降低頻譜偵測時的誤判率並提高產出。

As the vigorous development of wireless communications technology in recent years, there are more and more users accessing the wireless network, causing the spectrum scarcity becoming a severe problem, especially when the high data throughout is required. Nowadays, only the ISM band is unlicensed, most other spectrum bands need licensed to use. The Federal Communications Commission(FCC) reports indicated that the traditional static spectrum allocation resulted in low spectrum utilization. To solve this problem, the development of cognitive radio is an important technology.
Cognitive radio is an intelligent communication technology which can enhance the efficiency of spectrum utilization by exploiting the unused resources of primary system, while not interfering with the transmission of primary users. Therefore fast and accurate spectrum sensing is very important in realizing a reliable cognitive radio. However, the error sensing of spectrum will result in low spectrum utilization. In the paper we proposed an reinforcement learning mechanism called DQ-Learning to reduce the false spectrum sensing probability. The simulation results showed that the proposed scheme can effectively reduce the false spectrum sensing probability, therefore increasing the successful channel access rate and system throughput.

摘要 ------------------------------------------------------------- II
英文摘要 --------------------------------------------------------- III
致謝 ------------------------------------------------------------- IV
目次 -------------------------------------------------------------- V
表目錄 ------------------------------------------------------------ VI
圖目錄 ----------------------------------------------------------- VII
第一章 緒論 -------------------------------------------------------- 3
1.1 前言 ----------------------------------------------------------- 3
1.2 研究動機與目的 -------------------------------------------------- 3
1.3 論文架構 ------------------------------------------------------- 5
第二章 文獻探討 ---------------------------------------------------- 6
2.1 感知無線電網路 -------------------------------------------------- 6
2.1.1 感知無線電標準 ------------------------------------------------ 8
2.1.2 系統描述及應用 ----------------------------------------------- 11
2.2 增強學習機制 --------------------------------------------------- 15
2.2.1 Q-Learning演算法 -------------------------------------------- 17
2.3 相關研究 ------------------------------------------------------ 19
第三章 研究方法 --------------------------------------------------- 21
3.1 問題概述 ------------------------------------------------------ 21
3.2使用增強學習機制降低通道誤判率 ------------------------------------ 22
第四章 模擬與效能分析 ---------------------------------------------- 28
第五章 結論與未來研究 ---------------------------------------------- 36
參考文獻 --------------------------------------------------------- 37

[1]Q. Zhao and B. M. Sadler, “A survey of dynamic spectrum access,” IEEE Signal Processing Magazines, vol. 24, no. 3, pp.79-89, May 2007.
[2]A. Sahai, R. Tandra and N. Hoven, “Opportunistic spectrum use for sensor networks: the need for local cooperation,” in Proc. IPSN, 2006.
[3]P. Kolodzy, “Spectrum Policy Task Force Report”, ET Docket, No.02-135, November 2002.
[4]J. Mitola and G. Q. Maquire, “Cognitive Radios: Making Software Radios more Personal,” IEEE Personal Communications, vol. 6, no. 4, pp. 13-18, August 1999.
[5]J. Mitola, “Software radios: wireless architecture for the 21st century,” New York: Wiley, 2000.
[6]J. Mitola, “Software radios：Survey, Critical evaluation and future directions,” IEEE Aerospace and Electronic Systems Magazine, vol. 8, no. 4, pp. 25–36, April 1993.
[7]J. Mitola , “Cognitive radio for flexible mobile multimedia communication,” Mobile Networks and Applications, vol. 6, no. 5, pp. 435-441, September 2001.
[8]P. Houz, D. Ruiz, S. B. Jemaa, and P. Cordier, “Dynamic Spectrum Allocation Algorithm for Cognitive Networks,” in Proc. the third International Conference on Wireless and Mobile Communications, 2007.
[9]D. Cabric, S. M. Mishra, R. Brodersen, and A. Wolisz, “Implementation Issues in Spectrum Sensing for Cognitive Radios”, in Proc. Asilomar Conference on Signals, Systems, and Computers, PACIFIC GROVE, CA, October 29 - November, 2006.
[10]N. Maziar, “Dynamic Spectrum Access with Cognitive Radios: Future Architectures and Research Challenges,” in Proc. Cognitive Radio Oriented Wireless Networks and Communications 2006. 1st International Conference, Mykonos, Greece, pp. 1-5, June 2006.
[11]S. Haykin, “Cognitive radio: brain-empowered wireless communications,” IEEE Journal on selected areas in communications, vol. 23, pp. 201-220, Feb. 2005.
[12]J. Mitola, “Cognitive Radio: An Integrated Agent Architecture for Software Defined Radio,” PhD Dissertation, Royal Inst. Technol. (KTH), Stockholm, Sweden, 2000.
[13]DARPA XG WG, The XG Architectural Framework V1.0, 2003.
[14]DARPA XG WG, The XG Vision RFC V1.0, 2003.
[15]IEEE 802.22 Working Group on Wireless Regional Area Networks, http://www.ieee802.org/22/.
[16]C. Stevenson, G. Chouinard, L. Zhongding, H. Wendong, S. Shellhammer, and W. Caldwell, “IEEE 802.22: The First Cognitive Radio Wireless Regional Area Network Standard,” IEEE Communications Magazines, vol. 47, no. 1, pp. 130-138, January 2009.
[17]IEEE P1900 Working Group, http://www.ieeep1900.org/.
[18]M. Sherman, A. N. Mody, R. Martinez, C. Rodriguez, and R. Reddy, “IEEE Standards Supporting Cognitive Radio and Networks, Dynamic Spectrum Access, and Coexistence,” IEEE Communications Magazines, vol. 46, no. 7, pp. 72–79, July 2008.
[19]F. Granelli et. al., “Standardization and research in cognitive and dynamic spectrum access networks: IEEE SCC41 efforts and other activities,” IEEE Communications Magazine, vol. 48, no.1 pp.71-79, January 2010
[20]S. Buljore, V. Merat, H. Harada, S. Filin, P. Houze, K. Tsagkaris, V. Ivanov, K. Nolte, T. Farnham, and O. Holland, “IEEE P1900.4 System Overview on Architecture and Enablers for Optimised Radio and Spectrum Resource Usage,” in Proc. IEEE DySPAN, Chicago, Illinois, pp. 1-8, October 2008.
[21]IEEE Std 802.16.2-2001 Coexistence of Fixed Broadband Wireless Access Systems, http://standards.ieee.org/reading/ieee/std_public/description/lanma n/802.16.2-2001_desc.html
[22]Z. Tabakovic, “Mobile Broadband Wireless Access Systems Coexistence,” ELMAR, Zadar, Croatia, vol. 1, pp. 219-222, September, 2008.
[23]The WiMAX Forum, http://www.wimaxforum.org/home
[24]Z. Abichar, Y. Peng, and J. M. Chang, “WiMax: The Emergence of Wireless Broadband,” IT Professional, vol. 8, no. 4, pp. 44-48, July-August 2006.
[25]R. W. Thomas, L. A. DaSilva, and A. B. MacKenzie, “Cognitive Networks,” in Proc. IEEE DySPAN, Baltimore, Maryland, pp. 352-360, November 2005.
[26]Z. Hu, N. Guo and R. Qiu, “Wideband waveform optimization with energy detector receiver in cognitive radio,” in Proc. IEEE SoutheastCon 2010, Concord, NC, pp. 198-203, March, 2010.
[27]L. C. Wang, A. Chen, and D. S. L Wei, “A Cognitive MAC Protocol for QoS Provisioning in Overlaying Ad Hoc Networks,” in Proc. 4th IEEE CCNC, Las Vegas, Nevada, pp. 1139 -1143, January 2007.
[28]Q. Shi, D. Taubenheim, S. Kyperountas, P. Gorday, and N. Correal, “Link Maintenance Protocol for Cognitive Radio System with OFDM PHY,” in Proc. IEEE DySPAN, Dublin, Ireland, pp. 440-443, April 2007.
[29]J. Zhao, H. Zheng, and G. Yang, “Distributed Coordination in Dynamic
Spectrum Allocation Networks,” in Proc. IEEE DySPAN, Baltimore, Maryland, pp. 259-268, November 2005.
[30]L. Busoniu, R. Babuska, and B. De Schutter, “Multi-Agent Reinforcement Learning: A Survey,” in Proc. ICARCV 2006, Singapore, pp. 1-6, Dec, 2006.
[31]R. S. Sutton and A. G. Barto, “Reinforcement Learning An Introduction,” MIT Press, Cambridge, Massachusetts London, England, 2001.
[32]I. F. Akyildiz, W.-Y. Lee, M. C. Vuran, and S. Mohanty, “Next generation/dynamic spectrum access/cognitive radio wireless networks: A survey, ” in Proc. Computer Networks, pp. 2127-2159, 2006.
[33]C. J. C. H. Watkins and P. Dayan, “Q-Learning,” in Proc. Machine Learning, pp.279-292, August, 2002.
[34]P. Y. Glorennec, “Fuzzy Q-learning and Dynamical Fuzzy Q-Learning,” in Proc. 3rd IEEE International Conference on Fuzzy Systems, USA, pp.474-479, 1994.
[35]T. Jiang, D. Grace and Y. Liu , “Cognitive Radio Spectrum Sharing Schemes with Reduced Spectrum Sensing Requirements,” in Proc. IET Seminar on IEEE, pp. 1 - 5, Sept. 2008.
[36]L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement Learning: A Survey, ” Journal of artificial intelligence Research, vol. 4, pp. 237-285, May, 1996.
[37]M. Bublin, J. Pan, I. Kambourov, and P. Slanina, “Distributed spectrum sharing by reinforcement and game theory, ” in Proc. presented at 5th Karlsruhe workshop on software radio, Karlsruhe, Germany, March, 2008.
[38]T. Horiuchi, A. Fujino, 0. Katai, and T. Sawaragi, “Fuzzy Interpolation-Based Q-Learning with Continuous States and Actions,” in Proc. 5th IEEE International Conference on Fuzzy Systems, USA, pp.594-600, 1996.
[39]G. Ganesan and Y. G. Li, “Agility Improvement Through Cooperative Diversity in Cognitive Radio,” IEEE GLOBECOM 2005, St. Louis, Missouri, vol. 5, pp. 2505- 2509, December 2005.
[40]T. Jiang, D. Grace and Y. Liu , “Cognitive Radio Spectrum Sharing Schemes with Reduced Spectrum Sensing Requirements,” in Proc. IET Seminar on IEEE , pp. 1 - 5, Sept. 2008.
[41]T. Jiang, D. Grace, and Y. Liu, “Performance of Cognitive Radio Reinforcement Spectrum Sharing Using Different Weighting Factors,” in Proc. ChinaCom 2008, Hangzhou, China, pp. 1195 - 1199, August 2008.
[42]M. H. Wahab , Y. Yang and M. Sooriyabandara , “ Reinforcement Learning Based Spectrum-aware Routing in Multi-hop Cognitive Radio Networks ” in Proc. CROWNCOM , Hannover, Germany , pp. 1 - 5, June 2009.
[43]W. Y. Lee and I. F. Akyildiz, “Joint Spectrum and Power Allocation for Inter-Cell Spectrum Sharing in Cognitive Radio Networks,” in Proc. IEEE DySPAN, Chicago, Illinois, pp. 1-12, October 2008.
[44]Y. B. Reddy , “Detecting Primary Signals for Efficient Utilization of Spectrum Using Q-Learning,” in Proc. Fifth International Conference on ITNG , pp. 360 - 365 , April 2008.