臺灣博碩士論文加值系統

測試圖形使用者介面的應用程式比起測試傳統以程式碼為基礎的程式還要複雜許多，其原因為測試圖形使用者介面程式不但要考慮底層的程式碼的執行。而且為了驗證所有可能由事件組成的圖形使用者介面輸入序列組合，測試人員需要生成大量的圖形使用者介面測試個案來達成其測試的足夠性。但是想要在有限的時間跟資源下，把所有的圖形使用者介面測試個案跑過，並且把找到的錯誤都修復完，可能會造成軟體專案時程的延誤。因此若能在早期測試過程中能儘早執行到找到較多錯誤的測試個案並且立即修復，可降低在測試後期才發現錯誤造成專案延後的風險。近幾年所提出的測試個案優先排序的技術都顯示可以改進在軟體測試過程中錯誤偵測的比率，然而針對圖形使用者介面測試個案的優先排序研究卻較少研究提出。在本論文中，我們提出權重為基礎的方法，產生具有權重的事件流程圖以及圖形使用者介面測試個案，來解決不具有權重的圖形使用者介面測試個案的問題。依據圖形使用者介面測試個案所分配到的權重數值，由高至低、由低至高、或是調整權重的排列方法，得到新的圖形使用者介面測試個案執行的優先順序，並且探討排序過後的錯誤偵測比率。實驗結果說明，權重數值依調整的方式排列會得到較好的錯誤偵測比率。

Testing the correctness of a GUI-based application is more complex than conventional code-based application. The underlying codes of the GUI application have to be tested and the space of possible combinations of events with a large GUI input sequence requires generating numerous test cases to confirm the adequacy of the GUI testing. Running all GUI test cases and then fixing all found bugs within time and budget constraints may delay project completion. Hence, it is important to order the test cases that uncover most faults early in the testing process. Test case prioritization has been used in recent years because they can improve the rate of fault detection during the testing phase. However, few studies have discussed GUI test case prioritization techniques. In this thesis, we propose a weighted event flow graph for solving the non-weighted GUI test case and ranking GUI test cases according to weight scores ordered by high-to-low, low-to-high, and adjusted scores. Our experiments show that the adjusted-weight method obtains better rates of fault detection.

Abstract ii
摘要 iii
誌謝 (Acknowledgement) iv
List of Figures vii
List of Tables ix
Chapter 1 Introduction 1
Chapter 2 Test Case Prioritization and GUI Testing 4
2.1 Test Case Prioritization Problem 4
2.2 Related Works 6
2.3 GUI Testing 7
2.3.1 Overview 7
2.3.2 Event Flow Graph 8
2.3.3 Representation of GUI Test Case 9
Chapter 3 Weight-Based Methods 10
3.1 Create the Weighted Event Flow Graph 10
3.2 Weight-based GUI Test Case Prioritization 15
3.3 Examples of WGTCP 21
Chapter 4 Experiments and Tool Design 24
4.1 Experimental Setup 25
4.2 Experimental Measurement 27
4.3 Experimental Result 28
4.3.1 WordProcessor 29
4.3.2 TerpPaint 31
4.3.3 TerpSheet 32
4.3.4 Calculator 34
4.4 Experimental Analysis 36
4.5 Tool Demonstration 40
Chapter 5 Conclusions 42
References 44

[1] A. Memon, M. L. Soffa, “A Comprehensive Framework for Testing Graphical User Interfaces,” Ph.D. thesis, Department of Computer Science, University of Pittsburgh, 2001.
[2] A. Memon, M. E. Pollack, and M. L. Soffa, “Hierarchical GUI Test Case Generation Using Automated Planning,” IEEE Transactions on Software Engineering, Vol.27, No.2, pp.144-155, 2001.
[3] A. Memon, M. E. Pollack, and M. L. Soffa, “Using A Goal-Driven Approach to Generate Test Cases for GUIs,” Proceedings of the 15th International Conference on Software Engineering, (ICSE′99), pp.257-266, 1999.
[4] L. White, H. Almezen, and N. Alzeidi, “User-Based Testing of GUI Sequences and Their Interactions,” Proceedings of 12th International Symposium on Software Reliability Engineering (ISSRE'01), pp.54-63, 2001.
[5] F. Belli, “Finite-State Testing and Analysis of Graphical User Interfaces,” Proceedings of 12th International Symposium on Software Reliability Engineering (ISSRE'01), pp.34-43, 2001.
[6] J. Chen, S. Subramaniam, “A GUI Environment to Manipulate FSMs for Testing GUI-based Applications in Java,” Proceedings of 34th Annual Hawaii International Conference on System Sciences (HICSS-34), Vo.9, 2001.
[7] S. John, C. Pravir, F. Bob, and P. Andy, “jRapture: A Capture/Replay tool for observation-based testing,” Proceedings of the 2000 ACM SIGSOFT International Symposium on Software Testing and Analysis, pp.158-167, 2000.
[8] G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold, “Test Case Prioritization: An Empirical Study,” Proceedings of the 15th International Conference on Software Engineering, (ICSE′99), pp.179-188, 1999.
[9] A. Spillner, T. Linz, and H. Schaefer, (2007), Software Testing Foundations 2nd Edition: a study guide for the Certified Tester exam, Rocky Nook.
[10] E. Dustin, (2003), Effective Software Testing: 50 Specific Ways to Improve Your Testing, Addison-Wesley.
[11] G. Rothermel, R. H. Untch, C. C hu, and M. J. Harrold, “Prioritizing Test Cases for Regression Testing,” IEEE Transactions on Software Engineering, Vol.27, No10, pp.929-948, 2001.
[12] S. Elbaum, A. G. Malishevsky, and G. Rothermel, “Test Case Prioritization: A Family of Empirical Studies,” IEEE Transactions on Software Engineering, Vol. 28, pp.159-182, 2002.
[13] R. C. Bryce and A. Memon, “Test Suite Prioritization by Interaction Coverage,” Proceedings of the Workshop on Domain Specific Approaches to Test Automation (DoSTA’07), 2007.
[14] F. Belli, M. Eminov, and N. Gokce, “Prioritizing Coverage-Oriented Testing Process – An Adaptive-Learning-Based Approach and Case Study,” Proceedings of the 31st IEEE Annual International Computer Software and Applications Conference (COMPSAC 2007), Vol.02, pp.197-203, 2007.
[15] A. Memon, and M. E. Pollack, “Coverage Criteria for GUI Testing,” ACM SIGSOFT Software Engineering Notes, Vol.26, No.5, 2001.
[16] H. Zhu P. A. V. Hall, and J. H. R. May, “Software Unit Test Coverage and Adequacy,” ACM Computing Surveys (CSUR), Vol.29, No.4, pp.366-427, 1997.
[17] P. Ammann, J.Offutt (2008), Introduction to software testing, Cambridge.
[18] D. Peters, D. L. Parnas, “Generating a test oracle from program documentation: work in progress,” Proceedings of the 1994 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’94), pp.58-65, 1994.
[19] M. J. Harrold, R. Gupta, and M. L. Soffa, “A Methodology for Controlling the Size of a Test Suite,” ACM Transactions on Software Engineering and Methodology (TOSEM), Vol.2, No.3, pp.270-285, 1993.
[20] W. E. Wong, J. R. Horgan, S. London, and A. P. Mathur, “Effect of Test Set Minimization on Fault Detection Effectiveness,” Proceedings of the 17th International Conference on Software Engineering (ICSE’95), pp.41-50, 1995.
[21] G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong, “An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites,” Proceedings of the 14th International Conference on Software Maintenance, (ICSM′98), pp.34-43, 1998.
[22] J. A. Jones and M. J. Harrold, “Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage,” IEEE Transactions on Software Engineering, Vol.29, No.3, pp.195-209, 2003.
[23] D. Leon and A. Podgurski, “A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases,”Proceedings of the International Symposium on Software Reliability Engineering (ISSRE′03), pp.442-456, 2003.
[24] D. Jeffrey and N. Gupta, “Test Case Prioritization Using Relevant Slices,” Proceedings of the 30th IEEE Annual International Computer Software and Applications Conference (COMPSAC 2006), pp.411-420, 2006.
[25] Z. Li, M. Harman and R. M. Hierons, “Search Algorithms for Regression Test Case Prioritization” IEEE Transactions on Software Engineering, Vol.33, No.4, pp.225-237, 2007.
[26] H. Do, G. Rothermel, and A. Kinneer, “Empirical Studies of Test Case Prioritization in a JUnit Testing Environment,” Proceedings of the International Symposium on Software Reliability Engineering (ISSRE′04), pp.113-124, 2004.
[27] J. M. Kim and A. Porter, “A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments,” Proceedings of the 24th International Conference on Software Engineering (ICSE′02), pp.119-129, 2002.
[28] B. Qu, C. Nie, B. Xu, and X. Zhang, “Test Case Prioritization for Black Box Testing,” Proceedings of the 31st IEEE Annual International Computer Software and Applications Conference (COMPSAC 2007), Vol.1, pp.465-474, 2007.
[29] H. Srikanth, “Requirements-Based Test Case Prioritization,” Student Research Forum in 12th International Symposium on the Foundations of Software Engineering, 2004.
[30] H. Srikanth, L. Williams, and J. Osborne, “System Test Case Prioritization of New and Regression Test Cases,” International Symposium of Empirical Software Engineering, 2005.
[31] P. Tonella, P. Avesani, and A. Susi, “Using the Case-Based Ranking Methodology for Test Case Prioritization,” Proceedings of the 22nd International Conference on Software Maintenance (ICSM′06), pp.119-129, 2006.
[32] B. Korel, L. H. Tahat, and M. Harman, “Test Prioritization Using System Models,” Proceedings of the 21st International Conference on Software Maintenance (ICSM′05), pp.559-568, 2005.
[33] K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, and R. S. Roos, “Time-Aware Test Suite Prioritization,” Proceedings of the 2006 International Symposium on Software Testing and Analysis, pp.1-11, 2006.
[34] A. Memon, “GUI Testing: Pitfalls and Process,” IEEE Computer, 2003.
[35] A. Memon, M. E. Pollack, and M. L. Soffa, “Automated Test Oracles for GUIs,” ACM SIGSOFT 8th International Symposium on the Foundations of Software Engineering (FSE-8), pp.30-39, 2000.
[36] A. Memon, I. Banerjee, and A. Nagaraja, “What Test oracle should I use for effective GUI testing?” Proceedings of 18th IEEE International Conference on Automated Software Engineering (ASE’03), pp. 164- 173, 2003.
[37] A. Memon, M. L. Soffa, “Regression testing of GUIs,” Proceedings of the 9th European Software Engineering Conference (ESEC) and 11th ACM SIGSOFT International Symposium on the Foundations of Software Engineering (FSE-11), pp.118-127, 2003.
[38] L White, H Almezen, and S Sastry, “Firewall Regression Testing of GUI Sequences and Their Interactions,” Proceedings of the 19th International Conference on Software Maintenance (ICSM′03), pp.398-409, 2003.
[39] A. Memon, Q, Xie, “Empirical evaluation of the fault-detection effectiveness of smoke regression test cases for GUI-based software,” Proceedings of the 20th International Conference on Software Maintenance (ICSM′04), pp.8-17, 2004.
[40] A. Memon, Q, Xie, “Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software,” IEEE Transactions on Software Engineering, Vol.31, No.10, pp.884-896, 2005.
[41] Q. Xie, “Developing cost-effective model-based techniques for GUI testing,” Proceedings of the International Conference of Software Engineering 2006 (ICSE’06), pp.997-1000, 2006.
[42] M. Ye, B. Feng, Y. Lin, and L. Zhu, “Neural Networks Based Test Cases Selection Strategy for GUI Testing,” Intelligent Control and Automation (WCICA’ 06), Vol.2, pp.5773-5776, 2006.
[43] S. McMaster, A. Memon, “Call-Stack Coverage for GUI Test Suite Reduction,” IEEE Transactions on Software Engineering, Vol.4, No.1, pp.99-115, 2008.
[44] A. Memon, Q, Xie, “Using Transient/Persistent Errors to Develop Automated Test Oracles for Event-driven Software,” Proceedings of the International Conference on Automated Software Engineering (ASE′04), pp.186-195, 2004.
[45] M. Robinson and P. A. Vorobiev, (1999), Swing: A Fast-paced Guide with Production-Quality Code Examples, Manning Publications.
[46] TerpOffice, http://www.cs.umd.edu/~atif/TerpOfficeWeb/, 1 May 2008.
[47] Java Swing Calculator, http://www.beginner-java-tutorial.com/ java-swing- calculator.html, 1 May 2008.
[48] GUITAR, http://www.cs.umd.edu/~atif/GUITARWeb/, 1 May 2008.
[49] Q. Xie and A. Memon, “Studying the Characteristics of a “Good” GUI Test Suite,” Proceedings of the International Symposium on Software Reliability Engineering (ISSRE′06), pp.159-168, 2006.
[50] A. Memon, I. Banerjee, and A. Nagarajan, “GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing,” Proceedings of 10th Working Conference on Reverse Engineering (WCRE’03), pp 260-269, 2003.