臺灣博碩士論文加值系統

在全球各大城市中，交通堵塞問題日漸嚴重，也給全球城市帶來了巨大的影響和壓力。交通堵塞將導致二氧化碳排放量增加、燃料消耗劇增以及高峰時段的交通旅行時間變長等諸多社會、經濟和環境問題。為解決這一問題，本博士論文提出了幾個創新之多方向性策略，充分運用且成功融合了交通流之預測、強化學習預測模型、機器學習技術、高級駕駛員輔助系統（ADAS）和車載隨機網絡（VANET）的優勢，以最大化地提高道路利用率並找出最有效的重新路由路徑。

在本博士論文中的幾個策略分別是基於實時交通信息和預測的未來車輛流量，優化路徑選擇進一步增加交通重新路由的準確度與效能。我們在所設計的演算法中將根據車輛之目的地、道路密度和當前交通量通盤考量，進而將最佳化之路徑做為選擇，並藉助動態時間更新演算法，提供了最有效的重新路由方案，極大地幫助用路人選擇最有效的重新路由路徑，從而顯著改善了交通狀況。

另外，我們亦提出基於強化學習的創新獎勵機制，並採用長短期記憶（LSTM）的預測模型來進行改善紅綠燈時向與時長之控制。 此預測模型準確預測未來的道路密度、車速和交通量，並加入創新的獎勵函數，使得我們在動態紅綠燈時長的控制上有卓越的成效。而這種方法確保決策符合環境上之需求並隨時調整，從而提供全面的交通管理解決方案。

通過在 SUMO (Simulation of Urban Mobility) 上進行模擬，我們所提出之幾個策略皆在交通旅行時間、二氧化碳排放量、燃料消耗方面都優於現有文獻中的各種策略，模擬之結果也驗證所提出之幾個策略有顯著地提高交通之效率並減少對環境的影響。毫無疑問地，本博士論文所提出的幾個增強、全面的策略對於解決交通堵塞問題做出了重大貢獻。

The problem of traffic congestion is becoming increasingly severe, bringing substantial impact and stress to global urban environments. Traffic congestion results in increased carbon dioxide emission, escalated fuel consumption, and elongated travel time during peak hours. All of which pose numerous social, economic, and environmental challenges. To address these issues, this dissertation proposes several innovative and multifaceted strategies. These strategies successfully incorporate and integrate traffic flow prediction, reinforcement learning mechanism, machine learning technologies, advanced driver-assistance systems (ADAS), and the advantages of the vehicular ad-hoc network (VANET) with the ultimate goal of maximizing road utilization and identifying the most efficient rerouting paths.

The proposed strategies in this dissertation are designed according to the real-time traffic information and predicted future vehicle flow to optimize path selection and further enhance the accuracy and efficiency of traffic rerouting. These strategies consider the destination of vehicles, road density, and current traffic volume to make comprehensive routing decisions. Furthermore, the dynamic time updating algorithm is incorporated to provide the most efficient rerouting plans, significantly aiding road users in choosing the most effective rerouting paths and substantially improving traffic conditions.

In addition, the strategies proposed in this dissertation utilize a novel reward mechanism in reinforcement learning with a long short-term memory (LSTM) prediction model to improve the control of traffic signal timing and duration. This prediction model accurately forecasts future road density, vehicle speed, and traffic volume and incorporates an innovative reward function, leading to exceptional performance in dynamic traffic light timing control. This proposed method ensures that decisions align with environmental needs and can be adjusted in real-time, thereby providing a comprehensive traffic management solution.

Through simulations conducted on the simulation of urban mobility (SUMO) platform, the proposed strategies in this dissertation outperform those in the literature in terms of travel time, carbon dioxide emissions, and fuel consumption. The simulation results unquestionably demonstrate that these strategies significantly enhance traffic efficiency and mitigate environmental impact. Undoubtedly, the enhanced and comprehensive strategies proposed in this dissertation make significant contributions to solving the problem of traffic congestion.

Contents
Recommendation Letter .................................. i
Approval Letter .................................. ii
Abstract in Chinese .................................. iii
Abstract in English .................................. iv
Acknowledgments .................................. vi
Contents........................................ vii
List of Figures..................................... xi
List of Tables ..................................... xiv
List of Algorithms................................... xv

1 Overview

2 An Improved Traffic Rerouting Strategy Using Real-Time Traffic Information and Decisive Weights
2.1 Related Work
2.1.1 ITS and VANETs
2.1.2 Review Assessment of Closely Related Work
2.2 Proposed System
2.2.1 System Architecture
2.2.2 Travel Time, Road Network Representation, and Capacity
2.2.3 Congestion Detection
2.2.4 Selection of the Rerouted Vehicles
2.3 Proposed Strategy
2.3.1 Congestion Levels Classification and Decisive Weights
2.3.2 Rerouting Strategy
2.4 Performance Analysis
2.4.1 Time Complexity
2.4.2 Communication Overhead
2.5 Performance Evaluation and Discussion
2.5.1 Simulation Setting
2.5.2 Simulation Framework
2.5.3 Results and Analysis from the Simulation
2.6 Concluding Remarks

3 Enhanced Rerouting Mechanism with Machine Learning for Travel Time and CongestionReduction
3.1 Related Work
3.2 Design of Our Proposed Mechanism
3.2.1 Problem Definition
3.2.2 Proposed Mechanism
3.3 Numerical Results and Discussions
3.3.1 Simulation Set-Up
3.3.2 Simulation Results and Discussions
3.4 Concluding Remarks

4 An LSTM-Assisted Proactive Traffic Rerouting Strategy with the Future Situation Prediction
4.1 Related Work
4.2 System Model
4.2.1 System Architecture
4.2.2 Preliminary Regarding LSTM
4.2.3 Operators and Operation of LSTM
4.3 Proposed Strategy
4.3.1 Preprocessing Models
4.3.2 Congestion Detection
4.3.3 Finding a Suitable Path
4.3.4 Resultant Rerouting Strategy
4.4 Numerical Results and Discussions
4.4.1 Simulation Set-Up
4.4.2 Simulation Results and Discussions
4.5 Concluding Remarks

5 LSTM Prediction Model Incorporated Deep Reinforcement Learning based Adaptive Traffic Signal Control in the Intelligent Transportation System
5.1 Related Work
5.1.1 Exploring ITS and VANET
5.1.2 Exploring DRL and TLC
5.1.3 Traffic Rerouting
5.1.4 The Closely Strategies
5.2 Proposed Strategy
5.3 Simulations Results
5.3.1 Simulation Scenario and Closely Comparison
5.4 Concluding Remarks
6 Conclusions
6.1 Future Work

References
ListofPublications

[1] D. Schrank, B. Eisele, and T. Lomax, “2019 Urban Mobility Report,” Texas A&M Transportation Institute, College Station, Texas, USA, Tech. Rep., Aug. 2019.
[2] M. Lee. “the effects of traffic congestion". Accessed on: Jul. 20, 2021. [Online]. Available: https://traveltips.usatoday.com/effects-traffic-congestion-61043.html
[3] S.Mu, Z.Xiong, and Y.Tian “Intelligent Traffic Control System Based on Cloud Computing and Big Data Mining,” IEEE Trans. Ind. Inf., vol. 15, no. 12, pp. 6583–6592, Jul. 2019.
[4] L. Zhu, F. R. Yu, Y. Wang, B. Ning, and T. Tang, “Big Data Analytics in Intelligent Transportation Systems: A Survey,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 1, pp. 383–398, Apr. 2018.
[5] R.BauzaandJ.Gozálvez, “Traffic Congestion Detection in Large-Scale Scenarios Using Vehicle-to-Vehicle Communications,” J. Netw. Comput. Appl., vol. 36, no. 5, pp. 1295–1307, Sep. 2013.
[6] Google Maps. Accessed on: Nov. 2020. [Online]. Available: https://www.google.com/maps
[7] Waze. Accessed on: Nov. 2020. [Online]. Available: https://www.waze.com
[8] Sygic. Accessed on: Nov. 2020. [Online]. Available: https://www.sygic.com
[9] M. Behrisch, L. Bieker, J. Erdmann, and D. Krajzewicz, “SUMO – Simulation of Urban MObility: An Overview,” in Proc. of SIMUL2011, Oct. 2011.
[10] M. Wang, H. Shan, R. Lu, R. Zhang, X.Shen, and F. Bai, “Real-Time Path Planning Based on Hybrid-VANET-Enhanced Transportation System,” IEEE Trans. Veh. Technol., vol. 64, no. 5, pp. 1664–1678, Jul. 2015.
[11] L. Qi, “Research on Intelligent Transportation System Technologies and Applications,” in Proc. of PElTS’08, Aug. 2008, pp. 529–531.
[12] S. An, B. Lee, and D. Shin, “A Survey of Intelligent Transportation Systems,” in Proc. Int. Conf. Comput. Intell. Commun. Syst. Netw., Jul. 2011, pp. 332–337.
[13] E. Faouzi, N.E., H. Leung, and A. Kurian, “Data Fusion in Intelligent Transportation Systems: Progress and Challenges – A Survey,” Inf. Fusion, vol. 12, no. 1, pp. 4–10, Jan. 2011.
[14] J. Yen, “An Algorithm for Finding Shortest Routes from All Source Nodes to A Given Destination in General Networks,” Quart. Appl. Math., vol. 27, no. 4, pp. 526–530, Jan. 1970.
[15] E. W. Dijkstra, “A Note on Two Problems in Connexion with Graphs,” Numer. Math., vol. 1, no. 1, pp. 269–271, Dec. 1959.
[16] R.Bellman,“Onaroutingproblem,”Q.Appl.Math.,vol.16,no.1,pp.87–90,Apr.1958.
[17] J. Pan, I.S. Popa, K. Zeitouni, and C.Borcea, “Proactive Vehicular Traffic Rerouting for Lower Travel Time,” IEEE Trans. Veh. Technol., vol. 62, no. 8, pp. 3551–3568, Oct. 2013.
[18] J. Pan, I.S. Popa, and C.Borcea, “DIVERT: A Distributed Vehicular Traffic Re-Routing System for Congestion Avoidance,” IEEE Trans. Mob. Comput., vol. 16, no. 1, pp. 58–72, Jan. 2017.
[19] C. A. Brennand, A. M. de Souza, G. Maia, A. Boukerche, H. Ramos, A. A. Loureiro, and L. A. Villas, “An Intelligent Transportation System for Detection and Control of Congested Roads in Urban Centers,” in Proc. Int. Symp. Comput. Commun. (ISCC), Jun. 2015, pp. 663–668.
[20] A. M. de Souza, R. S. Yokoyama, L. C. Botega, R. I. Meneguette, and L. A. Villas, “SCORPION: A Solution Using Cooperative Rerouting to Prevent Congestion and Improve Traffic Condition,” in Proc. IEEE Int. Conf. Comput. Inf. Technol. Ubiquitous Comput. Commun. Dependable Autonomic Secure Comput. Pervasive Intell. Comput., Oct. 2015, pp. 497–503.
[21] A.M. deSouza, R.S. Yokoyama, G. Maia, A. Loureiro, and L.A. Villas, “Real-Time Path Planning to Prevent Traffic Jam Through an Intelligent Transportation System,” in Proc. IEEE Symp. Comput. Commun., Jun. 2016, pp. 726–731.
[22] A. Souza, R. Yokoyama, A. B. G. Maia, E. Cerqueira, A. Loureiro, and L. Villas, “ICARUS: Improvement of Traffic Condition through an Alerting and Re-routing System,” Comput. Netw., vol. 110, no. 10, pp. 118–132, Sep. 2016.
[23] M. Rezaei, H. Noori, M. Mohammadkhani Razlighi, and M. Nickray, “ReFOCUS+: Multi-Layers Real-Time Intelligent Route Guidance System With Congestion Detection and Avoidance,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 1, pp. 50–63, Jan. 2021.
[24] M. Chen, X.Yu, and Y.Liu, “PCNN: Deep Convolutional Networks for Short-Term Traffic Congestion Prediction,” IEEE Trans. Intell. Transp. Syst., vol. 19, no. 11, pp. 3550–3559, Jun. 2018.
[25] H.A. Najada and I. Mahgoub, “Anticipation and Alert System of Congestion and Accidents in VANET Using Big Data Analysis for Intelligent Transportation Systems,” in Proc. IEEE Symp. Ser. Comput. Intell. (SSCI), Dec. 2016, pp. 1–8.
[26] S. Majumdar, M. M. Subhani, B. Roullier, A. Anjum, and R. Zhu, “Congestion Prediction for Smart Sustainable Cities Using IoT and Machine Learning Approaches,” Sustain. Cities Soc., vol. 64, p. 102500, Jan. 2021.
[27] A. Mushtaq, I. U. Haq, M. U. Imtiaz, A. Khan, and O. Shafiq, “Traffic Flow Management of Autonomous Vehicles Using Deep Reinforcement Learning and Smart Rerouting,” IEEE Access, vol. 9, pp. 51 005–51 019, Mar. 2021.
[28] Z. Khan, A. Koubaa, and H. Farman, “Smart Route: Internet-of-Vehicles (IoV)-Based Congestion Detection and Avoidance (IoV-Based CDA) Using Rerouting Planning,” Appl. Sci., vol. 10, no. 13, Jun. 2020.
[29] K. Kim, S. Koo, and J. W. Choi, “Analysis on Path Rerouting Algorithm based on V2X Communication for Traffic Flow Improvement,” in Proc. of ICTC’20, Oct. 2020, pp. 251–254.
[30] G. Araujo, M. Queiroz, F. Duarte-Figueiredo, A. Tostes, and A. Loureiro, “CARTIM: A Proposal Toward Identification and Minimization of Vehicular Traffic Congestion for VANET,” in Proc. Int. Symp. Comput. Commun. (ISCC), Jun. 2014, pp. 1–6.
[31] S. Lee, M. Younis, A. Murali, and M. Lee, “Dynamic Local Vehicular Flow Optimization Using Real-Time Traffic Conditions at Multiple Road Intersections,” IEEE Access, vol. 7, pp. 28 137–28 157, Feb. 2019.
[32] D.JiangandL.Delgrossi, “IEEE 802.11p: Towards an International Standard for Wireless Access in Vehicular Environments,” in Proc. IEEE Veh. Technol. Conf., May 2008, pp. 2036–2040.
[33] Z. Mir and F. Filali, “LTE and IEEE 802.11p for Vehicular Networking: A Performance Evaluation,” Eurasip J. Wirel. Commun. Netw., vol. 2014, no. 1, pp. 1–15, May 2014.
[34] J. Gozálvez, M. Sepulcre, and R. Bauza, “IEEE 802.11p Vehicle to Infrastructure Communications in Urban Environments,” IEEE Commun. Mag., vol. 50, no. 5, pp. 176–183, May 2012.
[35] H. Noori and B. Olyaei, “A Novel Study on Beaconing for VANET-based Vehicle to Vehicle Communication:: Probability of Beacon Delivery in Realistic Large-Scale Urban Area Using 802.11p,” in Proc. Int. Conf. Smart Commun. Netw. Technol., vol. 1, Nov. 2013, pp. 1–6.
[36] D. Krajzewicz, Traffic Simulation with SUMO – Simulation of Urban MObility. Springer, New York, NY, Jun. 2010.
[37] F. He, X. Yan, Y. Liu, and L. Ma, “A Traffic Congestion Assessment Method for Urban Road Networks Based on Speed Performance Index,” Procedia Eng., vol. 137, pp. 425–433, Feb. 2016.
[38] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. MIT, 2009.
[39] D. Eppstein,“Finding the k Shortest Paths,” SIAMJ.Comput., vol.28, no.2, pp.652–673, Mar. 1997.
[40] M. Aazam, S. Zeadally, and K. Harras, “Fog Computing Architecture, Evaluation, and Future Research Directions,” IEEE Commun. Mag., vol. 56, no. 5, pp. 46–52, May 2018.
[41] Emission Factors from the Model PHEM for the HBEFA Version 3. Accessed on: Apr. 2021. [Online]. Available: https://www.hbefa.net/e/documents/HBEFA_31_Docu_hot_emissionfactors_ PC_LCV_HDV.pdf
[42] D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent Development and Applications of SUMO–Simulation of Urban MObility,” Int. J. Adv. Syst. Meas., vol. 5, no. 3&4, pp. 128–138, Dec. 2012.
[43] A. Wegener, M. Piórkowski, M. Raya, H. Hellbrück, S. Fischer, and J. Hubaux, “TraCI: An Interface for Coupling Road Traffic and Network Simulators,” in Proc. of CNS’08, Apr. 2008, pp. 155–163.
[44] Veins. Accessed on: Nov. 2020. [Online]. Available: http://veins.car2x.org
[45] W. Arellano and I. Mahgoub, “TrafficModeler Extensions: A Case for Rapid VANET Simulation Using, OMNET++, SUMO, and VEINS,” in Proc. High Capacity Opt. Netw. Emerg./Enabling Tech- nol., Dec. 2013, pp. 109–115.
[46] OpenStreetMap. “openstreetmap". Accessed on: Nov. 2020. [Online]. Available: https: //www.openstreetmap.org/.
[47] N. Rida, M. Ouadoud, A. Hasbi, and S. Chebli, “Adaptive Traffic Light Control System Using Wireless Sensors Networks,” in Proc. IEEE Int. Congr. Inf. Sci. Technol. (CiSt), Oct. 2018, pp. 552–556.
[48] P. Perez-Murueta, A. Gómez-Espinosa, C. Cardenas, and M. Gonzalez-Mendoza, “Deep Learning System for Vehicular Re-Routing and Congestion Avoidance,” Appl. Sci., vol. 9, no. 13, pp. 2717– 2731, Jul. 2019.
[49] D. Montgomery, E. Peck, and G. Vining, Introduction to Linear Regression Analysis, 2021.
[50] S. Mallik, “Intelligent Transportation System,” Int.J.Civ.Eng.,vol.5,no.4,pp.367–372,May2014.
[51] Y. T. Tseng and H. W. Ferng, “An Improved Traffic Rerouting Strategy Using Real-time Traffic Information and Decisive Weights,” IEEE Trans. Veh. Technol., vol. 70, no. 10, pp. 9741–9751, Oct. 2021.
[52] P. Grandinetti, C. Canudas-de-Wit, and F. Garin, “Distributed Optimal Traffic Lights Design for Large-Scale Urban Networks,” IEEE Trans. Control Syst. Technol., vol. 27, no. 3, pp. 950–963, Mar. 2018.
[53] N. Wu, D. Li, Y. Xi, and B. de Schutter, “Distributed Event-Triggered Model Predictive Control for Urban Traffic Lights,” IEEE Trans. Intell. Transp. Syst., Mar. 2020.
[54] B. Pratama, J. Christanto, M.T. Hadyantama, and A. Muis, “Adaptive Traffic Lights through Traffic Density Calculation on Road Pattern,” in Proc. Appl. Sci. Technol., Oct. 2018, pp. 82–86.
[55] P.E. Hart, N.J. Nilsson, and B. Raphael, “A Formal Basis for the Heuristic Determination of Minimum Cost Paths,” IEEE Trans. Syst. Sci. Cybern., vol. 4, no. 2, pp. 100–107, Jul. 1968.
[56] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
[57] B. Flavio, M. Rodolfo, Z. Jiang, and A. Sateesh, “Fog Computing and Its Role in the Internet of Things,” in Proc. MCC Work. Mob. Cloud Comput., Aug. 2012, pp. 13–16.
[58] T. L. Fine, Feedforward Neural Network Methodology. Springer Science & Business Media, Jun. 2006.
[59] M. Schuster and K.K. Paliwal, “Bidirectional Recurrent Neural Networks,” IEEE Trans. Signal Process, vol. 45, no. 11, pp. 2673–2681, Nov. 1997.
[60] G. V. Houdt, C. Mosquera, and G. Nápoles, “A Review on the Long Short-Term Memory Model,” Artif. Intell. Rev., vol. 53, no. 8, pp. 5929–5955, Dec. 2020.
[61] Keras. Accessed on: Jan. 2022. [Online]. Available: https://keras.io/
[62] TensorFlow. Accessed on: Jan. 2022. [Online]. Available: https://www.tensorflow.org/
[63] P. Lara-Benítez, M. Carranza-García, J. M. Luna-Romera, and J. C. Riquelme, “Temporal Convolutional Networks Applied to Energy-Related Time Series Forecasting,” Appl. Sci., vol. 10, no. 7, pp. 2322–2339, Mar. 2020.
[64] L. Changle, Y. Wenwei, M. Guoqiang, and X. Zhigang, “Congestion Propagation Based Bottleneck Identification in Urban Road Networks,” IEEE Trans. Veh. Technol., vol. 69, no. 5, pp. 4827–4841, Mar. 2020.
[65] W. Shangbo, L. Changle, Y. Wenwei, and M. Guoqiang, “Network Capacity Maximization Using Route Choice and Signal Control With Multiple OD Pairs,” IEEE Trans. Intell. Trans. Syst., vol. 21, no. 4, pp. 1595–1611, Apr. 2020.
[66] L. Changhao, Z. Yixiao, Z. Tingting, W. Xuanli, G. Lin, and Z. Qinyu, “High Throughput Vehicle Coordination Strategies at Road Intersections,” IEEE Trans. Veh. Technol., vol. 69, no. 12, pp. 14 341– 14 354, Dec. 2020.
[67] T. Tian, B. Feng, D. Yue, J. Alex, D. Qionghai, and W. Jie, “Cooperative Deep Reinforcement Learning for Large-Scale Traffic Grid Signal Control,” IEEE Trans. Cybern., vol. 50, no. 6, pp. 2687–2700, Jun. 2020.
[68] K. Neetesh, M. Sarthak, G. Vaibhav, and K. Neeraj, “Deep Reinforcement Learning-Based Traffic Light Scheduling Framework for SDN-Enabled Smart Transportation System,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 2411–2421, Mar. 2022.
[69] C. Tianshu, W. Jie, C. Lara, and L. Zhaojian, “Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 3, pp. 1086–1095, Mar. 2020.
[70] L. Xiaoyuan, D. Xunsheng, W. Guiling, and H. Zhu, “A Deep Reinforcement Learning Network for Traffic Light Cycle Control,” IEEE Trans. Veh. Technol., vol. 68, no. 2, pp. 1243–1253, Feb. 2019.
[71] Z. Cao, S. Jiang, J. Zhang, and H.Guo, “A Unified Framework for Vehicle Rerouting and Traffic Light Control to Reduce Traffic Congestion,” IEEE Trans. Veh. Technol., vol. 18, no. 7, pp. 1958–1973, Jul. 2016.
[72] H. Zhu, Z. Wang, F. Yang, Y. Zhou, and X. Luo, “Intelligent Traffic Network Control in the Era of Internet of Vehicles,” IEEE Trans. Veh. Technol., vol. 70, no. 10, pp. 9787–9802, Aug. 2021.