臺灣博碩士論文加值系統

隨著網路和物聯網的進步，REST API通訊協議在線上服務中被廣泛應用。許多網路服務（例如：雲端服務）提供開發人員使用的REST API。因此，進行模糊測試以確保API的穩定性和安全性至關重要。

本研究以HsuanFuzz黑箱REST API模糊測試工具為基礎，在三個方面改進HsuanFuzz的不足。首先，參考Morest中關於分析OpenAPI中的API依賴關係和操作序列生成的方法，提出了一套新方法，能夠自動建立API依賴關係圖，並生成符合數據依賴和CRUD操作的序列，不再需要手動撰寫依賴關係文件。其次，我們減少了序列中重複的操作，優化生成的序列使之更精簡，同時仍能達到最大的程式覆蓋率。最後，我們生成了多種排列組合的合法操作序列，以測試網路服務在不同的測試順序下的適應性。此外，本研究為了進一步提升測試效率，更運用了多執行緒的架構對目標進行請求發送。

With the advancement of the internet and the Internet of Things, REST API communication protocol is widely used in online services. Many internet services, such as cloud services, provide REST APIs for developers to use. Therefore, ensuring the stability and security of the API is crucial, hence the need for fuzzing.

This research is based on the HsuanFuzz REST API black-box fuzzing and improves the shortcomings in three aspects. Firstly, referring to the method in Morest for analyzing the API dependencies in OpenAPI and generating operation sequences, we propose a new method that can automatically generate API dependency graph and produce operation sequences that meet data dependencies and CRUD rule, eliminating the need for manual configuration of API dependencies. Secondly, we reduce duplicated operations in the sequences and optimize the generated sequences to make them more concise while still achieving maximum code coverage. Lastly, we generate multiple combinations of valid operation sequences to test the adaptability of services under different testing orders. Additionally, to further enhance testing efficiency, this research employs a multi-threaded approach to send requests to the target.

中文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
英文摘要 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
目錄. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
圖目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
表目錄 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi
第一章 緒論 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 研究動機 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 研究目標 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 研究貢獻 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
第二章 研究背景 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1 REST API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 REST API 特色 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 OpenAPI 規範 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.4 REST API 操作序列 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.5 模糊測試 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.5.1 REST API 模糊測試 . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6 測試覆蓋標準 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
第三章 相關研究 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.1 RESTler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 RestCT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 RestTestGen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 foRest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.5 HsuanFuzz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.5.1 初始化階段 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.5.2 變異階段 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.5.3 分析階段 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.6 Morest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.6.1 RPG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.7 相關研究比較 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
第四章 方法與實作 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1 OpenAPI 依賴關係解析器 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1.1 建立路徑依賴關係 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1.2 建立 Schema 依賴關係 . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1.3 建立參數依賴關係 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.1.4 建立 API 依賴關係圖 . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 生成操作序列 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2.1 操作序列特性 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2.2 生成操作序列演算法 . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.3 建立測試種子 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3.1 種子結構 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.4 資源池 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.5 多執行緒測試 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
第五章 實驗結果 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.1 實驗環境 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.2 RQ1：生成操作序列的有效性 . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.3 RQ2：操作序列建立方式是否優於 HsuanFuzz . . . . . . . . . . . . . . . . . . 32
5.3.1 APIScryer 與 HsuanFuzz 所生成的操作序列的概念差異 . . . . . . . . 32
5.3.2 比較實際運作對程式覆蓋率的影響 . . . . . . . . . . . . . . . . . . . . 34
5.4 RQ3：是否優於現時的黑箱測試工具 . . . . . . . . . . . . . . . . . . . . . . . 36
5.4.1 RealWorld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
第六章 結論 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
參考文獻 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

[1] 【智能化測試專題】華為雲 API 智能測試工具 - Morest 測試框架. URL: https://bbs.huaweicloud.com/blogs/384195.
[2] Amazon Web Services. URL: https://aws.amazon.com/.
[3] Andrea Arcuri. “Evomaster: Evolutionary multi-context automated system test generation”. In: 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST). IEEE. 2018, pp. 394–397.
[4] Vaggelis Atlidakis, Patrice Godefroid, and Marina Polishchuk. “Checking security properties of cloud service REST APIs”. In: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE. 2020, pp. 387–397.
[5] Vaggelis Atlidakis, Patrice Godefroid, and Marina Polishchuk. “Restler: Stateful rest api fuzzing”. In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE. 2019, pp. 748–758.
[6] Vaggelis Atlidakis et al. “Pythia: grammar-based fuzzing of rest apis with coverage-guided feedback and learning-based mutations”. arXiv preprint arXiv:2005.11498 (2020).
[7] Burp Suite Plugin. URL: https://github.com/orgs/PortSwigger/repositories.
[8] Qiniu Cloud. Goc. URL: https://github.com/qiniu/goc.
[9] Patrice Godefroid, Bo-Yuan Huang, and Marina Polishchuk. “Intelligent REST API data fuzzing”. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2020, pp. 725–736.
[10] Google Cloud Platform. URL: https://cloud.google.com/.
[11] gothinkster. golang-gin-realworld-example-app. URL: https://github.com/gothinkster/golang-gin-realworld-example-app.
[12] Jiaxian Lin et al. foREST: A Tree-based Approach for Fuzzing RESTful APIs. 2022. arXiv: 2203.02906 [cs.SE].
[13] Yi Liu et al. “Morest: model-based RESTful API testing with execution feedback”. In: Proceedings of the 44th International Conference on Software Engineering. 2022, pp. 1406–1417.
[14] Alberto Martin-Lopez, Sergio Segura, and Antonio Ruiz-Cortés. “Test coverage criteria for RESTful web APIs”. In: Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation. 2019, pp. 15–21.
[15] Medium. URL: https://medium.com/.
[16] microsoft. RESTler Annotations. URL: https://github.com/microsoft/restler-fuzzer/blob/main/docs/user-guide/Annotations.md.
[17] Microsoft Azure. URL: https://azure.microsoft.com/.
[18] OpenAI. URL: https://openai.com/.
[19] OpenAI API. URL: https://platform.openai.com/.
[20] OpenAPI. URL: https://www.openapis.org/.
[21] Lalith Rallabhandi. API-fuzzer. URL: https://github.com/Fuzzapi/API-fuzzer.
[22] spaCy. URL: https://spacy.io/.
[23] Swagger. URL: https://swagger.io/.
[24] Swagger. Swagger Petstore. URL: https://github.com/swagger- api/swagger-petstore.
[25] Chung-Hsuan Tsai, Shi-Chun Tsai, and Shih-Kun Huang. “REST API Fuzzing by Coverage Level Guided Blackbox Testing”. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS). IEEE. 2021, pp. 291–300.
[26] Emanuele Viglianisi, Michael Dallago, and Mariano Ceccato. “Resttestgen: automated black-box testing of restful apis”. In: 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE. 2020, pp. 142–152.
[27] Dmitry Vyukov. Randomized testing for Go. URL: https://github.com/dvyukov/go-fuzz.
[28] Huayao Wu et al. “Combinatorial testing of restful apis”. In: Proceedings of the 44th International Conference on Software Engineering. 2022, pp. 426–437.
[29] Mingyuan Wu et al. “One fuzzing strategy to rule them all”. In: Proceedings of the 44th International Conference on Software Engineering. 2022, pp. 1634–1645.
[30] Tai Yue et al. “Ecofuzz: Adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit”. In: Proceedings of the 29th USENIX Conference on Security Symposium. 2020, pp. 2307–2324.
[31] Michal Zalewski. american fuzzy lop. URL: https://lcamtuf.coredump.cx/afl/.