臺灣博碩士論文加值系統

基於省電與成本考量，嵌入式系統所配有的運算資源通常有限，要將現有的圖形界面測試技術套用在嵌入式系統軟體開發流程上是非常困難的。隨著智慧型手持裝置(如手機、平板)的普及，在嵌入式系統上進行自動化圖形化介面測試的需求也大幅提高，但現有的測試工具在使用時會有自動化不精確的問題。我們基於現有的測試工具Sikuli以遠端測試架構為基礎實作了圖形化介面自動化測試工具SPAG，以提升測試事件重播的精確度。SPAG將遠端手持式裝置的畫面顯示在本機端，讓使用者可以透過本機端畫面操作手持式裝置並錄製自動化測試項目。我們設計批次事件傳送與智慧等待兩種方法並實作在SPAG中，用以改善滑動等手勢的重播精確度，同時避免待測程式延遲造成的測試失敗。SPAG會自動評估待測程式是否產生延遲，並等待額外的時間讓待測程式完成反應，以提高測試指令的完成機率。基於輔助功能技術，SPAG能錄製流程中自動偵測待測程式狀態的改變，產生驗證指令，並在測試時自動驗證手持式裝置之狀態。我們選定一常用自動化測試工具monkeyrunner來比較測試精確度。在指定待測系統負荷工作量的情況下，SPAG自動化測試的精確度仍然達到90%以上。若依照一般測試流程估計，SPAG 將可以比monkeyrunner少用11%~71%的測試回數。

The embedded systems are built with limited computation resources. Therefore, it is hard to apply conventional GUI testing tools on embedded systems. However, with the popular of smart handheld device, e.g. smart phone and pads, more software run on handheld devices and rising the need of software testing on handheld device. Current testing tools on embedded systems all have defect on reproducing GUI operations, so we design a Smart Phone Automated GUI testing tool (SPAG) based on Sikuli, in order to reduce testing defects. We design Batch Event and Smart Wait method to reproduce GUI operations accurately and ensure the device under test (DUT) can fully process all test operations. We also design the Hybrid Verifying method based on accessibility technology to generate most verify operations and verify them automatically. We use an experiment to compare our SPAG with monkeyrunner on test accuracy and time efficiency, in which SPAG archives 90% testing accuracy with all types of workload while monkeyrunner only has test accuracy of 27%~88%. We use the second experiment to analyze the contribution of our methods. The testing result shows our Smart Wait method brings more improvement on test accuracy.

摘 要 I
Abstract I
誌 謝 II
Contents III
List of Figures V
List of Tables VI
Chapter 1 Introduction 1
Chapter 2 Background and Related Work 4
2.1 Challenges of automated GUI testing 4
2.2 Sikuli 4
2.3 Android Software Testing 5
2.4 Related Work 6
Chapter 3. Definitions and Problem Statement 8
3.1 Definitions 8
3.2 Problem Statement 14
Chapter 4. SPAG Design 15
4.1 Batch Event 16
4.2 Smart Wait 18
4.3 Hybrid Verifying 21
Chapter 5. Implementation 24
5.1 Device under Test 24
5.2 Recording and Replaying Events Sequence 25
Recording input events from live demo. 25
Reproducing Event Sequence with Batch Event 27
5.3 Smart Wait 27
5.4 Hybrid Verifying 28
Chapter 6. Experiment Result 30
6.1 Testbed 30
6.2 Comparison of test accuracy and time efficiency 31
6.3 SPAG Solution analysis 32
Batch Event and Smart Wait 33
Hybrid Verifying 34
Chapter 7. Conclusions and Future Work 36
References 38

[1] (2012). Android SDK Tools: monkeyrunner. Available: http://developer.android.com/guide/developing/tools/monkeyrunner_concepts.html
[2] T. Yeh, T.-H. Chang, and R. C. Miller, "Sikuli: using GUI screenshots for search and automation," presented at the Proceedings of the 22nd annual ACM symposium on User interface software and technology, Victoria, BC, Canada, 2009.
[3] T.-H. Chang, T. Yeh, and R. C. Miller, "GUI testing using computer vision," presented at the Proceedings of the 28th international conference on Human factors in computing systems, Atlanta, Georgia, USA, 2010.
[4] T.-H. Chang, T. Yeh, and R. Miller, "Associating the visual representation of user interfaces with their internal structures and metadata," presented at the Proceedings of the 24th annual ACM symposium on User interface software and technology, Santa Barbara, California, USA, 2011.
[5] Q. Xie and A. M. Memon, "Using a pilot study to derive a GUI model for automated testing," ACM Trans. Softw. Eng. Methodol., vol. 18, pp. 1-35, 2008.
[6] T. Takala, M. Katara, and J. Harty, "Experiences of System-Level Model-Based GUI Testing of an Android Application," in Software Testing, Verification and Verification (ICST), 2011 IEEE Fourth International Conference on Software Testing, Verification and Verification, 2011, pp. 377-386.
[7] L. Zhifang, L. Bin, and G. Xiaopeng, "Test automation on mobile device," presented at the Proceedings of the 5th Workshop on Automation of Software Test, Cape Town, South Africa, 2010.
[8] O.-H. Kwon and S.-M. Hwang, "Mobile GUI Testing Tool based on Image Flow," presented at the Proceedings of the Seventh IEEE/ACIS International Conference on Computer and Information Science (icis 2008), 2008.
[9] J. Bo, L. Xiang, and G. Xiaopeng, "MobileTest: A Tool Supporting Automatic Black Box Test for Software on Smart Mobile Devices," presented at the Proceedings of the Second International Workshop on Automation of Software Test, 2007.
[10] L. Zhi-fang and G. Xiao-peng, "SOA Based Mobile Device Test," presented at the Proceedings of the 2009 Second International Conference on Intelligent Computation Technology and Automation - Volume 04, 2009.
[11] V. R. Vemuri, "Testing Predictive Software in Mobile Devices," presented at the Proceedings of the 2008 International Conference on Software Testing, Verification, and Verification, 2008.
[12] Section 508 of the Rehabilitation Act. Available: www.access-board.gov/508.htm
[13] M. Grechanik, Q. Xie, and C. Fu, "Maintaining and evolving GUI-directed test scripts," presented at the Proceedings of the 31st International Conference on Software Engineering, 2009.
[14] M. Grechanik, Q. Xie, and C. Fu, "Creating GUI Testing Tools Using Accessibility Technologies," presented at the Proceedings of the IEEE International Conference on Software Testing, Verification, and Verification Workshops, 2009.
[15] GestureDetector.SimpleOnGestureListener | Android Developers. Available: http://developer.android.com/reference/android/view/GestureDetector.SimpleOnGestureListener.html