臺灣博碩士論文加值系統

本研究利用三種不同控制方法設計在未知環境下的載具跟隨系統，分別為模糊控制、Q-學習法以及Q-學習輔助模糊控制。在研究中，無人差速輪載具經由實驗空間上方的網路攝影機作為感測器，取得前方引導載具位置資訊及自身的位姿資訊，並使用上述三種演算法進行追蹤引導載具，保持安全距離與貼合引導載具路徑之任務。在使用模糊控制的跟隨系統時，須預先藉由專家經驗得出完整的模糊規則，但當複雜環境改變時，所採用之規則庫可能必須調整，然傳統的模糊控制並未提供調整策略，使其缺乏自適應性；在另一方面，Q-學習法能透過不斷與環境互動進行學習，具有自適應的能力，但因須先進行行為探索，使其應用效率低，且因離散化而產生震盪問題；為解決前兩種控制方法的不足，整合的Q-學習輔助模糊控制則，不但在動態環境下具有學習與適應環境的能力，並可透過模糊規則提高Q-學習法的學習速度。依據模擬和實驗結果，本文所發展之三種控制方法皆能實現任務目標，而經由結果比較可得，Q-學習輔助模糊控制確實能結合兩者優點，在實際導航上具有較高的應用價值。

Three different methods, including fuzzy control, Q-learning, and fuzzy Q-learning, are used to design the vehicle-following system in an unknown environment in this research. A webcam above the experimental space is used as the sensor to obtain the position information of the leader car and the posture information of the follower car. It is desired to accomplish the tasks of tracking the leader car, keeping safe distance, and fitting the path of the leader car. It is seen that the fuzzy control method lacks adaptability, since the fuzzy rules require experts’ knowledge which may not be available in the unknown complex environment. The Q-learning method can improve the performance of the controller by learning through interaction with the environment, so it has the ability of self-tuning. However, the pre-learning process for behavior exploration makes the learning efficiency of this system low. The Q-learning-assisted fuzzy control method can solve the deficiencies of the first two control methods. Not only, the ability to learn and adapt to the dynamic environment, the learning speed can be improved through rule-based adjustments. According to the simulation and experimental results, the three control methods used in this research can all achieve the goal of the task. Through the comparison of the results, it is shown that the Q-learning assisted fuzzy controller on the design of a vehicle-following system can take the advantages of both the fuzzy control method and the Q-learning method, so that it has better application values in navigation and control.

口試委員會審定書 i
誌謝 ii
中文摘要 iii
ABSTRACT iv
目錄 v
圖目錄 viii
表目錄 xi
第一章 緒論 1
1.1 前言與研究動機 1
1.2 文獻回顧 1
1.3 研究內容與成果 2
1.4 論文架構 3
第二章 無人載具跟隨系統架構及運動學模型 5
2.1 系統概述 5
2.2 導航實驗載具本體 6
2.3 控制器 7
2.4 感測器 8
2.5 系統整合 9
2.6 無人載具之運動學模型 10
2.7 控制器輸入變數設定 12
第三章 無人載具跟隨控制系統設計 16
3.1 模糊控制 16
3.1.1 模糊控制的基本理論 16
3.1.2 隸屬函數 17
3.1.3 Sugeno模糊模型 19
3.1.4 模糊控制之載具跟隨系統參數設計 20
3.2 Q-學習法 26
3.2.1 強化學習介紹 26
3.2.2 Q-學習法演算法 28
3.2.3 Q-學習法範例 33
3.2.4 Q-學習法之載具跟隨系統參數設計 35
3.3 Q-學習輔助模糊控制系統 39
3.3.1 Q-學習輔助模糊控制演算法 39
3.3.2 Q-學習輔助模糊控制之載具跟隨系統參數設計 40
第四章 載具位姿判定 44
4.1 數位影像介紹 44
4.2 影像畸變校正 46
4.3 影像特徵提取與追蹤 48
4.3.1 循環移位 48
4.3.2 目標檢測模型 49
4.3.3 實現KCF目標跟蹤方法之結果 50
4.4 載具姿態測定 51
4.4.1 邊緣檢測 51
4.4.2 直線檢測 54
4.4.3 實現載具姿態測定之結果 55
第五章 模擬與實驗結果 56
5.1 模擬結果 56
5.1.1 模糊控制之載具跟隨系統模擬結果 56
5.1.2 Q-學習法之載具跟隨系統模擬結果 58
5.1.3 Q-學習輔助模糊控制之載具跟隨系統模擬結果 61
5.1.4 模擬結果討論 63
5.2 實驗結果 65
5.2.1 模糊控制之載具跟隨系統實驗結果 65
5.2.2 Q-學習法之載具跟隨系統實驗結果 66
5.2.3 Q-學習輔助模糊控制之載具跟隨系統實驗結果 68
5.2.4 實驗結果討論 69
第六章 結論與未來方向 71
參考文獻 72

[1] T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms (Third Edition), MIT Press, 2009.
[2] 王進德, 類神經網路與模糊控制理論入門與應用, 全華圖書股份有限公司, 2007.
[3] L. A. Zadeh, "Fuzzy sets," Information and Control, pp. 338-353, 1965.
[4] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Second Edition), MIT Press, 2014, 2015.
[5] P. Y. Glorennec and L. Jouffe, "Fuzzy Q-learning," in Proceedings of 6th International Fuzzy Systems Conference, Barcelona, Spain, 1997.
[6] R. W. Deng, Y. H. Wang, C. J. Lin, and T. H. S. Li, "Implementation of human following mission by using fuzzy head motion control and Q-learning wheel motion control for home service robot," in 2013 International Conference on Fuzzy Theory and Its Applications (iFUZZY), Taipei, Taiwan, 2013.
[7] P. Chen and B. Wu, “A Q-Learning Based Human Following Fuzzy Controller and New Criterion of Ride Comfort for Intelligent Wheelchair Robots”, Master's Thesis, Institute of Electrical Control Engineering, National Chiao Tung University, Hsinchu, Taiwan, 2015.
[8] R. Dhaouadi and A. A. Hatab, "Dynamic Modelling of Differential-Drive Mobile Robots using Lagrange and Newton-Euler Methodologies: A Unified Framework," Adv Robot Autom 2: 107, 2013.
[9] "微星科技網路商店," [Online]. Available: https://twstore.msi.com/MSI/moreinfo_142086.htm
[10] "AXIS M3006-V Network Camera," 2016. [Online]. Available: https://www.axis.com/files/datasheet/ds_m3006v_1485632_en_1607.pdf
[11] Y. Kanayama, Y. Kimura, F. Miyazaki, and T. Noguchi, "A stable tracking control method for an autonomous mobile robot," in Proceedings., IEEE International Conference on Robotics and Automation, Cincinnati, OH, USA, 1990.
[12] S. K. Alonso, C. T. Blanc, and M. G. S. Torrubia, "eMathTeacher: Mamdani's Fuzzy Inference Metho," [Online]. Available: http://www.dma.fi.upm.es/recursos/aplicaciones/logica_borrosa/web/fuzzy_inferencia/funpert_en.htm
[13] T. M. Mitchell, Machine Learning, 1997.
[14] R. C. Gonzalez and R. E. Woods, Digital Image Processing (Third Edition), Pearson Education, 2008.
[15] L. H. Gao, “Research on the Methodology of Vision System-Assisted Path-following and Collision Avoidance for Autonomous Vehicle”, Master's Thesis, Institute of Applied Mechanics, National Taiwan University, Taipei, Taiwan, 2021.
[16] J. Weng, P. Cohen, and M. Herniou, "Camera calibration with distortion models and accuracy evaluation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 10, pp. 965-980, 1992.
[17] R. Hartley and A. Zisserman, "Camera Models," in Multiple View Geometry in Computer Vision (Second Edition), Cambridge University Press, 2004, pp. 153-177.
[18] J. F. Henriques, R. Caseiro, P. Martins, and J. Batista, "High-Speed Tracking with Kernelized Correlation Filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 583-596, 2015.
[19] J. Canny, "A Computational Approach to Edge Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-8, no. 6, pp. 679-698, 1986.
[20] "Canny - edge detection | Image Processing #8 | HBY coding academic," HBY coding academic, 2018, May 14. [Online]. Available: https://www.youtube.com/watch?v=PtSgA19sC5g
[21] I. Sobel, “History and Definition of the so-called” Sobel Operator”, more appropriately named the Sobel-Feldman Operator,”, 2015.
[22] R. O. Duda and P. E. Hart, "Use of the Hough Transformation to Detect Lines and Curves in Pictures," Comm. ACM, vol. 15, no. 1, pp. 11-15, 1972.