臺灣博碩士論文加值系統

現今室內移動式機器人運用大量包含 SLAM(Simultaneous Localization and Mapping)的自動導航技術，在未知環境移動過程中，透過傳感器獲得環境訊息建構出地圖，同時不斷更新自身位置訊息和使用粒子濾波器的蒙特卡羅定位法來實現自動導航。本研究嘗試用ROS在模擬環境中使用距離感測器在強化學習架構下訓練機器人。距離感測器可偵測出機器人在走廊上的相對位置，再運用強化學習訓練機器人遵守靠右走的社交行動準則。我們將環境和機器人在多種狀態下定義、分類，機器人旋轉方向定義為行動，以訓練後得出的數據 Q-表來執行機器人的自主導航。實驗結果顯示出訓練後機器人可以沿著寬度不一的直線走道行走、避免碰撞牆壁、並依循右側虛擬目標線前進，在某些特定變化的走道環境下亦可順利巡航。

Nowadays indoor mobile robots use a lot of automatic navigation technology including SLAM(Simultaneous Localization and Mapping). During the movement in an unknown environment, a map is constructed by acquiring environmental information through sensors that continuously update own position information and then using Monte Carlo positioning method by particle filter to achieve automatic navigation. This paper attempts to use distance sensors in a ROS simulation environment to train the robot under a reinforcement learning framework. Distance sensors of the robot can detect the relative position of the robot in the corridor and then use reinforcement learning to train the robot to abide by the social guidelines of keeping to the right. We define and classify states from the robot and the environment and then define the rotation direction of the robot as the actions and use the Q-table obtained after training to perform autonomous navigation of the robot. The results show that after training, the robot can move along linear corridors of different widths, avoid collision with walls, and follow the virtual target line on the right, and it can also cruise smoothly in certain variable corridor environments.

摘 要.................................................................i
ABSTRACT.............................................................ii
誌 謝...............................................................iii
目 錄................................................................iv
表目錄...............................................................vi
圖目錄..............................................................vii
第一章 緒論............................................................1
1.1 前言..............................................................1
1.2 文獻回顧...........................................................2
1.3 研究目的...........................................................4
第二章 模擬平台架構....................................................5
2.1 ROS Gazebo.模擬器架構..............................................5
2.2 模擬機器人硬體架構.................................................7
2.3 模擬實驗場景.......................................................9
2.4 系統架構...........................................................9
2.4.1 驅動裝置與速度設定...............................................9
2.4.2 加速方法........................................................10
2.4.3 方向角度計算....................................................11
第三章 強化學習訓練系統................................................12
3.1 Q-Learning系統架構圖與介紹 .......................................12
3.2 學習方法與架構....................................................14
3.2.1 Q-表與參數設定..................................................16
3.2.2 訓練次數定義....................................................19
3.2.3 輪次終結機制....................................................20
3.3 多種訓練情境......................................................20
3.3.1 走廊寬度變化....................................................20
3.3.2 速度變化........................................................22
第四章 不同訓練次數與速度的實驗結果....................................23
4.1 Q-表分析.........................................................23
4.1.1 訓練次數........................................................23
4.1.2 移動速度........................................................25
4.2 測試結果..........................................................29
4.3 情境比較分析......................................................30
第五章 多種走廊寬度的實驗結果..........................................33
5.1 Q-表分析.........................................................33
5.2 測試結果..........................................................34
第六章 寬度變化的走廊實驗結果..........................................35
6.1 判斷Q-表動態使用架構.............................................36
6.2 測試結果..........................................................36
第七章 結論與未來展望.................................................38
參考文獻..............................................................39
附錄
附錄表一 寬度5m速度0.125m/s訓練5000次Q-表..............................41
附錄表二 寬度5m速度0.125m/s訓練10000次Q-表.............................42
附錄表三 寬度5m速度0.125m/s訓練15000次Q-表.............................44
附錄表四 寬度5m速度0.125m/s訓練20000次Q-表.............................45
附錄表五 寬度5m速度0.125m/s訓練25000次Q-表.............................47
附錄表六 寬度2.5m速度0.125m/s Q-表....................................48
附錄表七 寬度2.5m速度0.0625m/s Q-表...................................50
附錄表八 寬度2.5m速度0.03125m/s Q-表..................................51
附錄表九 寬度5m速度0.25m/s Q-表.......................................53
附錄表十 寬度5m速度0.0625m/s Q-表.....................................54
附錄表十一 寬度8m速度0.25m/s Q-表.....................................56
附錄表十二 寬度8m速度0.125m/s Q-表....................................57
附錄表十三 寬度8m速度0.0625m/s Q-表...................................59

1.　李界木，「人工智慧發展現狀」，民報文化事業股份有限公司，2020，專欄。
2.　J. J. Leonard and H. F. Durrant-whyte, “Simultaneous map building and localization for an autonomous mobile robot”, Intelligent Robots and Systems' 91.'Intelligence for Mechanical Systems, Proceedings IROS'91. IEEE/RSJ International Workshop, 1991, pp. 1442-1447.
3.　P. O .N. Saian, Suyoto and Pranowo, “Optimized A-Star algorithm in hexagon-based environment using parallel bidirectional search”, 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE), 2016, Yogyakarta, pp. 1-5.
4.　Y. -Q. Qin, D. -B. Sun, N. Li and Y. -G. Cen, “Path planning for mobile robot using the particle swarm optimization with mutation operator”, Proceedings of 2004 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.04EX826), 2004, Shanghai, China, pp. 2473-2478.
5.　H. Liu, N. Stoll, S. Junginger and K. Thurow, “A Floyd-Dijkstra hybrid application for mobile robot path planning in life science automation”, 2012 IEEE International Conference on Automation Science and Engineering (CASE), 2012, Seoul, pp. 279-284.
6.　簡宜興、彭君豪、許陳鑑、王偉彥，「具定位校正機制之群組機器人室內導航與控制系統之研究」，科技與工程教育學刊，2017，pp. 1-24。
7.　P. Beeson, J. Modayil and B. Kuipers, “Factoring the mapping problem: Mobile robot map-building in the hybrid spatial semantic hierarchy”, The International Journal of Robotics Research, 2010, pp. 428-459.
8. V. Savov, “The OpenAI Dota 2 bots just defeated a team of former pros”, The Verge, 7 August 2018.
9. “Artificial intelligence: Google's AlphaGo beats Go master Lee Se-dol”, BBC News, 17 March 2016.
10. D. Garisto, “Google AI beats top human players at strategy game StarCraft II”, Nature, 30 October 2019.
11.　M. Asada, S. Noda and S. Tawaratsumida, et al., “Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning”, Machine Learning, 1996, pp. 279-303.
12. 莫煩，「什麼是DQN (強化學習)」，知乎，21 February 2017，https://zhuanlan.zhihu.com/p/25290858
13. A. Bera, T. Randhavane, R. Prinja and D. Manocha, “SocioSense: Robot navigation amongst pedestrians with social and psychological constraints”, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, Vancouver, BC, pp. 7018-7025.
14. Y. F. Chen, M. Everett, M. Liu and J.P. How, “Socially aware motion planning with deep reinforcement learning”, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, Vancouver, BC, pp. 1343-1350.
15. M. Luber, L. Spinello, J. Silva and K. O. Arras, “Socially-aware robot navigation: A learning approach”, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, pp. 902-907.