臺灣博碩士論文加值系統

Robot Framework為一個以Python為基礎的自動化驗收測試框架，藉其官方提供多樣化功能的函式庫，語法偏向自然語言容易撰寫及閱讀，並且易於透過Python、Java等程式語言擴充的特性，吸引許多敏捷開發團隊採用。由於跨團隊間的合作普遍，不同的團隊亦會開發同一份測試專案，因此維持測試腳本結果一致性便為重要。然而團隊間執行腳本所處的測試環境各不相同，易導致協助測試結果一致性的輔助關鍵字(assistive keywords)需要在不同測試環境下使用適合的參數設定，而這會限制測試腳本本身的可重複使用性。
本論文提出一個有效提升測試腳本可重複使用性，於不同測試環境中維持測試結果一致性的方法，將測試腳本中輔助關鍵字於該測試環境需增加的等待時間以機器學習預測，使測試腳本不因測試環境因素而需要改變參數設定，達成測試腳本結果一致性(consistency)與可重複使用性(reusability)。

Robot Framework is a test automation and acceptance test framework based on Python. Because of the powerful official libraries, easy-writing and readable syntax, and the easy-extending structure, it attracts more and more agile development team to adopt this framework. Since cross team collaboration becomes more common, different teams will have the opportunity to develop the same test project. In this case, it’s important to keep the test result consistent. However, the test develop environment of executing test scripts are usually different among these teams, and this kind of situation will cause the assistive keywords in test scripts to have different argument settings in different test environments respectively. The reusability of test scripts are limited.
This study will propose a methodology to increase the reusability of test scripts effectively in different test environments. The extended waiting time of assistive keywords will be predicted by machine learning, so that the argument settings will not be indispensable to alter in different test environments. As a result, reusability and consistency will be achieved.

摘要 i
ABSTRACT ii
誌謝 iv
目錄 vi
表目錄 ix
圖目錄 x
第一章 緒論 1
1.1. 研究背景與動機 1
1.2. 研究目標 2
1.3. 論文組織架構 2
第二章 背景知識 3
2.1. Robot Framework 3
2.2. Assistive Keyword 4
2.3. Scikit-Learn 5
2.4.Random Forests 5
第三章 研究方法 7
3.1. 研究方法情境 7
3.2. 網頁背景資料蒐集 9
3.2.1. 影響測試腳本執行結果一致性之因素 9
3.2.2. 蒐集選擇的網頁背景資料特徵 10
3.3. 使用機器學習訓練網頁背景資料 11
3.3.1. 資料前處理 12
3.3.1.1. 皮爾森積差相關分析係數 12
3.3.1.2. 輔助關鍵字執行時間顯示為零的資料 12
3.3.1.3. 虛擬變數 13
3.3.2. 資料訓練 13
3.3.2.1. 機器學習迴歸演算法比較 14
3.3.2.2. 演算法參數調整 15
3.3.3. 訓練資料評價 16
3.3.3.1. 決定係數 17
3.3.3.2. 袋外分數 17
3.3.3.3. 平均絕對誤差 17
3.3.3.4. 均方根誤差 18
3.4. 關鍵字撰寫 18
3.4.1. 關鍵字介面 18
3.4.2. 關鍵字實作概念 19
第四章 設計與實作 20
4.1. 研究方法設計 20
4.2. 網頁背景資料蒐集實作 22
4.2.1. 輔助關鍵字監聽器實作 22
4.2.2. 取得網頁背景資料特徵方法實作 24
4.3. 資料訓練實作 27
4.3.1. 資料前處理實作及模型品質評估 27
4.3.1.1. 皮爾森積差相關分析係數之分析 27
4.3.1.2. 移除輔助關鍵字執行時間顯示為零的資料評估 30
4.3.1.3. 將輔助關鍵字種類轉變成虛擬變數評估 32
4.3.2. 機器學習迴歸演算法參數評估與實作 34
4.3.2.1. 演算法參數評估 34
4.3.2.2. 演算法實作 37
4.4. 關鍵字實作 39
4.4.1. 監聽器實作 39
4.4.2. 關鍵字概念實作 40
4.4.3. 套用關鍵字至Robot Framework測試腳本 41
第五章 案例分析 43
5.1. 驗收測試案例一: 更改網頁介面語言 43
5.2. 驗收測試案例二: 更改使用貨幣種類 48
5.3. 案例結果探討 52
第六章 結論與未來研究方向 53
6.1. 結論 53
6.2. 未來研究方向 53
參考文獻 54

[1] G. Meszaros, xUnit Test Patterns: Refactoring Test Code, 1 edition. Upper Saddle River, NJ: Addison-Wesley, 2007.
[2] “Robot Framework.” [Online]. Available: https://robotframework.org/. [Accessed: 02-May-2019].
[3] “Welcome to Python.org,” Python.org. [Online]. Available: https://www.python.org/. [Accessed: 02-May-2019].
[4] “Acceptance Test Driven Development (ATDD),” Agile Alliance, 17-Dec-2015. .
[5] “瞭解 Java 技術.” [Online]. Available: https://www.java.com/zh_TW/about/. [Accessed: 02-May-2019].
[6] “SeleniumLibrary.” [Online]. Available: http://robotframework.org/SeleniumLibrary/SeleniumLibrary.html. [Accessed: 02-May-2019].
[7] “BuiltIn.” [Online]. Available: http://robotframework.org/robotframework/latest/libraries/BuiltIn.html. [Accessed: 02-May-2019].
[8] “scikit-learn: machine learning in Python — scikit-learn 0.20.3 documentation.” [Online]. Available: https://scikit-learn.org/stable/. [Accessed: 02-May-2019].
[9] “Matplotlib: Python plotting — Matplotlib 3.0.3 documentation.” [Online]. Available: https://matplotlib.org/. [Accessed: 02-May-2019].
[10] “Random forests - classification description.” [Online]. Available: https://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#ooberr. [Accessed: 16-May-2019].
[11] G. James, D. Witten, T. Hastie, and R. Tibshirani, “Bagging, Random Forests, Boosting,” in An Introduction to Statistical Learning, vol. 103, New York, NY: Springer New York, 2013, pp. 316–321.
[12] S. Russell and P. Norvig, “SUPERVISED LEARNING,” in Artificial Intelligence: A Modern Approach, 3 edition., Upper Saddle River: Pearson, 2009, pp. 695–697.
[13] M. Collin, “The Waiting Game,” in Mastering Selenium WebDriver, Packt Publishing, 2015, pp. 87–103.
[14] “JavaScript,” MDN Web Docs. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/JavaScript. [Accessed: 05-May-2019].
[15] “解決內存問題 | Tools for Web Developers,” Google Developers. [Online]. Available: https://developers.google.com/web/tools/chrome-devtools/memory-problems/?hl=zh-tw. [Accessed: 11-May-2019].
[16] “分析運行時性能 | Tools for Web Developers,” Google Developers. [Online]. Available: https://developers.google.com/web/tools/chrome-devtools/rendering-tools/?hl=zh-tw. [Accessed: 02-May-2019].
[17] K. Yeager, “LibGuides: SPSS Tutorials: Pearson Correlation.” [Online]. Available: https://libguides.library.kent.edu/SPSS/PearsonCorr. [Accessed: 18-Jun-2019].
[18] S. Garavaglia, “A Smart Guide to Dummy Variables: Four Applications and a Macro,” p. 10.
[19] “Robot Framework User Guide.” [Online]. Available: http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#id849. [Accessed: 08-May-2019].
[20] A. Géron, Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 1 edition. Beijing ; Boston: O’Reilly Media, 2017.
[21] A. Géron, “Stochastic Gradient Descent,” in Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 1 edition., Beijing ; Boston: O’Reilly Media, 2017, pp. 117–119.
[22] A. Géron, “SVM Regression,” in Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 1 edition., Beijing ; Boston: O’Reilly Media, 2017, pp. 154–156.
[23] A. Géron, “Decision Trees,” in Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 1 edition., Beijing ; Boston: O’Reilly Media, 2017, pp. 167–179.
[24] S. Glantz and B. Slinker, Primer of Applied Regression & Analysis of Variance, 2 edition. New York: McGraw-Hill Education / Medical, 2000.
[25] C. J. Willmott and K. Matsuura, “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance,” Clim. Res., vol. 30, no. 1, pp. 79–82, Dec. 2005.
[26] R. J. Hyndman and A. B. Koehler, “Another look at measures of forecast accuracy,” Int. J. Forecast., vol. 22, no. 4, pp. 679–688, Oct. 2006.
[27] L. Breiman, “OUT-OF-BAG ESTIMATION,” p. 13.
[28] 蕭文龍, “迴歸分析,” in 多變量分析最佳入門實用書, 第二版., 臺灣: 碁峰, 2009, pp. 8-5-8–6.
[29] “OOB Errors for Random Forests — scikit-learn 0.20.3 documentation.” [Online]. Available: https://scikit-learn.org/stable/auto_examples/ensemble/plot_ensemble_oob.html. [Accessed: 02-May-2019].
[30] E. L. Lehmann and G. Casella, Theory of Point Estimation, 2nd edition. New York: Springer, 2003.
[31] 吳彥銘, “將伺服器的CPU 與Memory 使用率結合至Robot Framework 測試報表,” 臺北科技大學, 2019.
[32] “Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more.” [Online]. Available: https://www.amazon.com/. [Accessed: 12-May-2019].
[33] “Window.performance,” MDN Web Docs. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/API/Window/performance. [Accessed: 12-May-2019].
[34] “performance.memory · WebPlatform Docs.” [Online]. Available: https://webplatform.github.io/docs/apis/timing/properties/memory/. [Accessed: 12-May-2019].
[35] “psutil documentation — psutil 5.6.3 documentation.” [Online]. Available: https://psutil.readthedocs.io/en/latest/#. [Accessed: 12-May-2019].
[36] “psutil documentation — psutil 5.6.3 documentation.” [Online]. Available: https://psutil.readthedocs.io/en/latest/#psutil.net_io_counters. [Accessed: 12-May-2019].
[37] “Chrome DevTools Protocol Viewer.” [Online]. Available: https://chromedevtools.github.io/devtools-protocol/. [Accessed: 12-May-2019].
[38] “Navigation Timing.” [Online]. Available: https://www.w3.org/TR/navigation-timing/. [Accessed: 02-May-2019].
[39] “轉譯效能 | Web,” Google Developers. [Online]. Available: https://developers.google.com/web/fundamentals/performance/rendering/?hl=zh-tw. [Accessed: 05-May-2019].
[40] “sklearn.model_selection.GridSearchCV — scikit-learn 0.21.1 documentation.” [Online]. Available: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html#sklearn.model_selection.GridSearchCV. [Accessed: 19-May-2019].
[41] “Mac、PC 和 Linux 平台的全新、快速瀏覽器 | Firefox,” Mozilla. [Online]. Available: https://www.mozilla.org/zh-TW/firefox/. [Accessed: 16-May-2019].