臺灣博碩士論文加值系統

這份論文展現了新的網頁程式測試案例的再生方法。為了改善測試案例的錯誤偵測能力，這技術提出關於自動繁衍兩種類型的測試案例：輸入值及行為。與以前的方法不同的是，我們是採用黑箱的方式，因此不需要伺服器端的原始碼，也不需要GUI的狀態轉移圖，很容易就可以使用。我們用此方法對四個真實的網頁程式進行實驗，發現二十二個錯誤，而且這些錯誤都是原來的測試案例沒發現的。試驗結果也顯示此方法比起突變更有效率。

This paper presents a novel approach of test case regeneration for web applications. The proposed technique automatically multiplies original test cases with 2 types of multiplication operators: input value and behavior, to improve the fault detection effectiveness of the original test suite. Different from previous works, our approach is black-box and needs neither information of server-side source code nor transition diagrams of the application’s GUI states, and thus easily applicable. We implemented our approach and conducted experiments using 4 real world web applications. 22 faults in 4 applications were detected, and all of the faults were not revealed by the original test suites. Experimental results also show that the proposed technique achieves better effectiveness in terms of mutation score.

中文摘要 1
ABSTRACT 2
CONTENTS 3
SYMBOLS 6
DEFINITIONS 7
Chapter 1. Introduction 8
1.1 Motivation 9
1.2 Contributions 10
Chapter 2. Related Work 12
2.1 Data Perturbation for XML Services 12
2.2 Generating Source Test Cases 14
Chapter 3. Background 15
3.1 Web Applications 15
3.2 Web Testing 16
Chapter 4. Running Example 17
4.1 Simple Changes to Input Values 20
4.2 Input Value Changes for Breaking HTML Documents 21
4.3 Input Value Changes as Security Attacks 21
4.4 Action Changes as Environment Perturbations 22
4.5 Scenario Change as Cycle Repetition 23
4.6 Crossover from Genetic Programming 24
Chapter 5. Multiplication Algorithms 28
5.1 Multiplication of Input Value 28
5.1.1 EMPTY_INPUT 28
5.1.2 LARGE_INPUT 29
5.1.3 MALFORMED_STRING 29
5.1.4 SQL_INJECTION 30
5.2 Multiplication of Behavior 30
5.2.1 DOUBLE_CLICK 31
5.2.2 SLEEP 31
5.2.3 CYCLE_REPETITION 32
5.2.4 CROSSOVER 34
5.3 Difference of CYCLE_REPETITION and CROSSOVER 35
5.4 The Algorithm of Regenerating test cases 36
Chapter 6. Implementation 39
6.1 Flow of the Architecture 39
6.2 Components of the Architecture 40
6.2.1 SUT 40
6.2.2 Test Script 40
6.2.3 Script Multiplier 40
6.2.4 CTA (Cloud Testing for App) 41
6.2.5 Cloud OS 44
6.2.6 Test Report 45
6.2.7 Test Oracles 45
Chapter 7. Evaluation 47
7.1 Subject Programs 47
7.1.1 Complain Management System [36] 47
7.1.2 Ultra Light Forum (1.1) [37] 49
7.1.3 SchooleMate (1.5.4) [38] 52
7.1.4 Gallery (3.0.9) [39] 58
7.2 Experimental Setup 61
7.3 Experiment 1 – Find Developer’s Bugs 63
7.4 Experiment 2 – Find Inserted Bugs In The Gallery 64
7.5 Threats to Validity 66
7.5.1 Internal threats 66
7.5.2 External threats 67
Chapter 8. Conclusions 68
Reference 71

[1]Alshahwan, N. and Harman, M. 2011. Automated Web Application Testing Using Search Based Software Engineering. Proceedings of the 2011 26th IEEE/ACM International Conference on Automated Software Engineering (Washington, DC, USA, 2011), 3–12.
[2]Alshahwan, N. and Harman, M. 2012. State Aware Test Case Regeneration for Improving Web Application Test Suite Coverage and Fault Detection. Proceedings of the 2012 International Symposium on Software Testing and Analysis (New York, NY, USA, 2012), 45–55.
[3]Ammann, P. and Offutt, J. 2008. Introduction to software testing. Cambridge University Press.
[4]Andrews, A.A., Offutt, J. and Alexander, R.T. 2005. Testing Web applications by modeling with FSMs. Software &; Systems Modeling. 4, 3 (Jul. 2005), 326–345.
[5]Arcuri, A. 2010. Longer is Better: On the Role of Test Sequence Length in Software Testing. Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation (Washington, DC, USA, 2010), 469–478.
[6]Artzi, S., Kiezun, A., Dolby, J., Tip, F., Dig, D., Paradkar, A. and Ernst, M.D. 2010. Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit-State Model Checking. IEEE Transactions on Software Engineering. 36, 4 (Jul. 2010), 474–494.
[7]Binder, R.V. 2000. Testing object-oriented systems: models, patterns, and tools. Addison-Wesley.
[8]Cloud Computing System and Application Services Technologies: https://www.itri.org.tw/eng/econtent/research/research02_02.aspx?sid=26 Accessed: 2014-04-05.
[9]DeMillo, R.A., Lipton, R.J. and Sayward, F.G. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer. 11, 4 (1978), 34–41.
[10]Di Lucca, G.A. and Fasolino, A.R. 2006. Testing Web-based applications: The state of the art and future trends. Information and Software Technology. 48, 12 (Dec. 2006), 1172–1186.
[11]Elbaum, S., Rothermel, G., Karre, S. and Fisher II, M. 2005. Leveraging User-Session Data to Support Web Application Testing. IEEE Trans. Softw. Eng. 31, 3 (Mar. 2005), 187–202.
[12]Goldberg, D.E. 1988. Genetic algorithms in search, optimization, and machine learning. Addison-Wesley.
[13]Halfond, W.G.J. and Orso, A. 2007. Improving Test Case Generation for Web Applications Using Automated Interface Discovery. Proceedings of the the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (New York, NY, USA, 2007), 145–154.
[14]Hamlet, R.G. 1977. Testing Programs with the Aid of a Compiler. IEEE Transactions on Software Engineering. SE-3, 4 (1977), 279–290.
[15]Holland, J.H. 1975. Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. U Michigan Press.
[16]Kieyzun, A., Guo, P.J., Jayaraman, K. and Ernst, M.D. 2009. Automatic creation of SQL Injection and cross-site scripting attacks. IEEE 31st International Conference on Software Engineering, 2009. ICSE 2009 (May 2009), 199–209.
[17]Mesbah, A., van Deursen, A. and Roest, D. 2012. Invariant-Based Automatic Testing of Modern Web Applications. IEEE Transactions on Software Engineering. 38, 1 (Jan. 2012), 35–53.
[18]Myers, G.J., Sandler and Badgett, T. 2011. The art of software testing. John Wiley &; Sons.
[19]Offutt, J. and Xu, W. 2004. Generating Test Cases for Web Services Using Data Perturbation. SIGSOFT Softw. Eng. Notes. 29, 5 (Sep. 2004), 1–10.
[20]Ricca, F. and Tonella, P. 2001. Analysis and Testing of Web Applications. Proceedings of the 23rd International Conference on Software Engineering (Washington, DC, USA, 2001), 25–34.
[21]Robotframework: http://robotframework.org/ Accessed: 2014-04-05.
[22]Santelices, R., Chittimalli, P.K., Apiwattanapong, T., Orso, A. and Harrold, M.J. 2008. Test-Suite Augmentation for Evolving Software. Proceedings of the 2008 23rd IEEE/ACM International Conference on Automated Software Engineering (Washington, DC, USA, 2008), 218–227.
[23]Selenium - Web Browser Automation: http://docs.seleniumhq.org/ Accessed: 2014-04-05.
[24]Sprenkle, S., Gibson, E., Sampath, S. and Pollock, L. 2005. Automated Replay and Failure Detection for Web Applications. Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (New York, NY, USA, 2005), 253–262.
[25]The Open Web Application Security Project (OWASP): https://www.owasp.org/. Accessed: 2014-04-05.
[26]Thummalapenta, S., Lakshmi, K.V., Sinha, S., Sinha, N. and Chandra, S. 2013. Guided Test Generation for Web Applications. Proceedings of the 2013 International Conference on Software Engineering (Piscataway, NJ, USA, 2013), 162–171.
[27]Wassermann, G., Yu, D., Chander, A., Dhurjati, D., Inamura, H. and Su, Z. 2008. Dynamic Test Input Generation for Web Applications. Proceedings of the 2008 International Symposium on Software Testing and Analysis (New York, NY, USA, 2008), 249–260.
[28]XSS Filter Evasion Cheat Sheet: https://www.owasp.org/index.php/XSS_Filter_Evasion_Cheat_Sheet Accessed: 2014-04-05
[29]Xu, W., Offutt, J. and Luo, J. 2005. Testing Web Services by XML Perturbation. 16th IEEE International Symposium on Software Reliability Engineering, 2005. ISSRE 2005 (2005), 257–266.
[30]Yoo, S. and Harman, M. 2012. Test Data Regeneration: Generating New Test Data from Existing Test Data. Softw. Test. Verif. Reliab. 22, 3 (May 2012), 171–201.
[31]Linux: http://www.linux.org/ Accessed: 2013-07.
[32]Apache HTTP Server: https://httpd.apache.org/ Accessed: 2013-07.
[33]MySQL http://dev.mysql.com/downloads/ Accessed: 2013-07.
[34]PHP http://www.php.net/ Accessed: 2013-07.
[35]GlassFish https://glassfish.java.net/ Accessed: 2013-07.
[36]Complain Management System http://sourceforge.net/projects/complain-management-system/?source=directory Accessed 2014-04-05.
[37]Ultra Light Forum http://sourceforge.net/projects/ultralightforum/ Accessed 2014-04-05.
[38]SchoolMate http://sourceforge.net/projects/schoolmate/ Accessed 2014-04-05.
[39]Gallery http://sourceforge.net/projects/gallery/ Accessed 2014-04-05.
[40]IIS http://www.iis.net/ Accessed 2014-07-06
[41]Lijun Shan and Hong Zhu. 2007. Generating Structurally Complex Test Cases By Data Mutation: A Case Study Of Testing An Automated Modelling Tool. The Computer Journal, Vol. 52 No. 5, 2009, 571-588
[42]Xiao Qu, Myra B. Cohen and Katherine M. Woolf. Combinatorial Interaction Regression Testing: A Study of Test Case Generation and Prioritization. Software Maintenance, 2007. ICSM 2007. IEEE International Conference. 255-264.
[43]Xun Yuan, Myra B. Cohen and Atif M. Memon. 2009. GUI Interaction Testing: Incorporating Event Context. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 37, NO. 4, JULY/AUGUST 2011. 559-574.
[44]Roy P. Pargas, Mary Jean Harrold and Robert R. Peck. Test-Data Generation Using Genetic Algorithms. Software Testing, Verification and Reliability, Vol. 9, 1999. 263-282.