臺灣博碩士論文加值系統

網頁與多媒體技術的整合讓電腦教學邁向了一個新的紀元。在本論文中，我們試圖研發一套超媒體應用程式，希望能在英語寫作學習上提供不同的教學方法與學習感受。有別於傳統靜態網頁的教材呈現方式，我們開發了一套錄音程式將老師的批改過程完整記錄下來。所錄製的結果包含老師的講解聲音與講解動作(如：滑鼠指標、畫筆或文字反白)。在播放時，我們利用多媒體同步技術，將老師原始的批閱過程動態的呈現，學生所看到的將會是具有自動導覽特性的視聽教材。我們相信老師的口語解說與動態的導覽會讓寫作指導過程更加清楚的呈現在學生的面前，並藉此提高學生的學習效率與寫作能力。

由於本系統所產生的超媒體文件包含各種不同的媒體資料，因此媒體間的相互關聯將提供跨媒體存取的重要線索。我們可將媒體關聯性區分為隱喻關係(implicit relation)和明確關係(explicit relation)，前者必須經由運算取得，而後者則可透過直接記錄取得。本論文中，我們試著去記錄媒體的明確關係並進一步探討明確關係中媒體間所具有的隱喻關係，藉此我們便可設計可行的同步機制，將批改作文的過程重建。我們從時間(temporal)、空間(spatial)和內容(content)三方面來探討超媒體系統在同步上可能遭遇的問題並解決之。因此我們設計了數個運算程序來發掘媒體間的關聯，這些資訊將可提供我們在文件導覽和視覺化的呈現上重要的參考依據。

我們的系統目前已在暨大資訊工程系實際上線運作，目的是希望提昇研究生的英文寫作能力。根據學生的意見與他們作文能力的表現，大多數學生對於系統功能都抱持正面看法，而他們每週的寫作技巧也逐漸在進步中。

The integration of Web and multimedia technologies has ushered in a new era for learning. In this thesis, we aim to develop a useful hypermedia application, Web-based Multimedia Annotation (WMA) system, for correcting English compositions. Unlike the traditional hypertext-based lectures, we devised an elaborately capturing tool to record the instructor’s lecturing process. The generated document consists of instructor’s narration, and several types of navigation events (e.g., tele-pointer, pen strokes, and highlight) which are triggered by the instructor during the recording process. At the presentation stage, the recorded lectures can be presented dynamically and synchronously by multimedia synchronization techniques. In contrast to passive navigations of static hypertext documents, the recorded documents offer audiovisual features to activate the lecturing presentation. We believe that the instructor’s narration and guidance make the lecture more comprehensive for learners. The approach is helpful for increasing learners’ learning efficiency and raising their writing ability.
For a composite hypermedia document, media correlations provide important clues to synchronized presentation and cross-media access. The media correlations are classified into implicit relation (retrieved by computing) and explicit relation (recorded or pre-orchestrated by an authoring tool). We show the feasibility to construct a vivid presentation by recording explicit relation and further exploring relations derived from the explicit relation. We describe the synchronization problems in temporal, spatial and content domains that a system may encounter when dealing with hypermedia documents. To facilitate the navigation of the integrated hypermedia documents, we devised several processes for discovering media correlations to provide easy-to-use random access mechanisms and completed visual presentations.
Our system has been served online for raising graduate students’ English writing ability in department of Computer Science and Information Engineering on our campus. According to students’ feedback and performance, they are satisfied with the features of the WMA system, and their writing ability was improved gradually.

CONTENTS I
LIST OF FIGURES IV
LIST OF TABLES VI
CHAPTER I 1
INTRODUCTION AND MOTIVATION 1
1.1 Motivation 2
1.2 Reasons for Capturing Lectures 4
1.3 Overview of the WMA system 5
1.4 Cross-Media Correlation and Synchronization 9
1.5 Authentic Use 11
1.6 Problem Statement 11
1.7 Contributions 13
1.8 Organization 15
CHAPTER II 16
RELATED WORK 16
2.1 Presentation Recording 16
2.1.1 Camera-based Recording 16
2.1.2 Electronic Whiteboard-based Recording 17
2.1.3 Computer-based Recording 18
2.1.4 System Comparison 20
2.2 Writing Environment for Language Learning 22
2.3 Related Techniques for System Implementation 24
2.4 Summary 24
CHAPTER III 26
OVERVIEW OF THE WSML SYSTEM 26
3.1 WSML System Architecture 27
3.2 Speech-Text Alignment Process 29
3.3 Adaptive Synchronization Framework 34
3.4 Summary 40
CHAPTER IV 42
APPLYING WSML SYSTEM TO CHINESE LEARNING 42
4.1 System Framework for CSL Learning 43
4.1.1 WSML Content Repository 44
4.1.2 WSML Browser 45
4.1.3 Exercise Repository 47
4.2 Summary 49
CHAPTER V 50
MULTIMEDIA LECTURE SKIMMING FOR WSML LECTURES 50
5.1 System Framework 51
5.1.1 Multilevel Access Mode 52
5.1.2 Compression Mode 54
5.2 Document Speech Alignment Process 56
5.2.1 Segmentation 58
5.2.2 Binding 59
5.3 Experimental Results 60
5.3.1 Document Speech Alignment 60
5.3.2 Summary Evaluation 60
5.4 Summary 62
CHAPTER VI 63
DESCRIPTION OF THE WMA SYSTEM 63
6.1 Overview of the WMA System 64
6.2 Web-based Multimedia Lecture Recorder (WMLR) 67
6.3 Web-based Multimedia Lecture Viewer (WMLV) 68
6.3.1. Synchronization Mechanism 69
6.3.2. Re-synchronization Mechanism 69
6.3.3 Smoothing Mechanism 70
6.3.4 Experimental Results 72
6.4 System Features 72
6.4.1 Correcting Functions of the Multimedia Lecture Recorder 75
6.4.2 Recording Modes of the Multimedia Lecture Recorder 77
6.4.3 Presentation Modes of Multimedia Lecture Viewer 78
6.5 Summary 80
CHAPTER VII 81
EXPLORING MEDIA CORRELATION AND SYNCHRONIZATION 81
7.1 Syntactic Level Correlation 83
7.1.1 Temporal Correlation 84
7.1.2 Spatial Correlation 84
7.1.3 Content Correlation 86
7.2 Semantic Level Correlation 87
7.2.1 Speech-Event Binding 88
7.2.2 Adaptable Pen Strokes Presentation 89
7.2.3 Erasing Handling 91
7.3 Experimental Results 92
7.3.1 Speech-Event Binding 92
7.3.2 Erasing Handling 94
7.4 Summary 95
CHAPTER VIII 96
SYSTEM EVALUATION 96
8.1 Tasks 96
8.2 Students’ Feedback 97
8.3 Students’ Performance 103
8.4 Summary 104
CHAPTER IX 105
CONCLUSIONS 105
REFERENCES 107
APPENDIX A 114
APPENDIX B 116
EXAMINATION QUESTIONS FOR PRE-TEST 116
Translations 116
Grammar Corrections 116
APPENDIX C 118
EXAMPLES OF CORRECTED COMPOSITIONS 118

[Agius01]Agius, H.W., and Angelides, M.C. (2001). Modeling Content for Semantic-Level Querying of Multimedia. Multimedia Tools and Applications, 15(1), 5-37.
[Arons92]Arons, B. (1992). Techniques, Perception, and Applications of Time-Compressed Speech. In Proceedings of 1992 Conference, American Voice I/O Society, 169-177.
[Arons94a]Arons, B. (1994). Speech Skimmer: Interactively Skimming Recorded Speech. Ph.D. thesis, MIT Media Lab.
[Arons94b]Arons, B. (1994). Pitch-based Emphasis Detection for Segmenting Speech Recordings. In Proceedings of International Conference on Spoken Language Processing, 1931-1934.
[Blako96]Blakowski, G., and Steinmetz, R. (1996). A Media Synchronization Survey: Reference Model, Specification, and Case Studies. IEEE Journal on Selected Areas in Communications 14(1), 5-35.
[Broth01]Brotherton, J. (2001). Enriching Everday Activities through the Automated Capture and Access of Live Experiences. Ph.D. Thesis, Georgia Tech.
[Brusi00]Brusilovsky, P. (2000). Web lectures: Electronic presentations in Web-based instruction. Syllabus 13 (5), 18-23.
[Chen92]Chen, F. R. and Withgott, M. (1992). The Use of Emphasis to Automatically Summarize Spoken Discourse. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 229-233.
[Chen99]Chen, H. Y., Chen, G. Y., and Hong, J. S. (1999). Design of a Web-based Synchronized Multimedia Lecture System for Distance Education. In Proceedings of IEEE International Multimedia Computing and Systems, Florence, Italy, 887-891.
[Chu01]Chu, W.T., Hsu, K.T., and Chen, H.Y. (2001). Design of an Alignment System for Synchronized Speech-Text Presentation. In Proceedings of Distributed Multimedia Systems, Taipei, Taiwan, 86-93.
[Chu05]Chu, W. T., and Chen, H.Y. (2005). Toward Better Retrieval and Presentation by Exploring Cross-media Correlations. Journal of Multimedia Systems 10(3), 183-198.
[Cruz94]Cruz, G., and Hill, R. (1994). Capturing and Playing Multimedia Events with STREAMS. In Proceedings of ACM Multimedia, San Francisco, CA, USA, 193-200.
[Datta01]Datta, A., and Ottmann, T. (2001). Towards a Virtual University. Journal of Universal Computer Science 7(10), 870-885.
[Effel93]Effelsberg, W., Meyer, T. and Steinmetz, R. (1993). A Taxonomy on Multimedia- synchronization. In Proceedings of Fourth Workshop on Future Trends of Distributed Computing Systems, Lisbon, Portugal, 97–103.
[Efl06]Department of CSIE, NCNU. (2006). English as Foreign Language Learning System. See: http://media.csie.ncnu.edu.tw/owl.
[Farkas95]Farkas, D.K., and Poltrock, S.E. (1995). Online editing, mark-up models and the workplace lives of editors and writers. IEEE Transactions on Professional Communication, 38(2), 110-117.
[Fspc06]FormosaSoft Homepage. (2006). PwoerCam & XMS. See: http://www.powercam.com.tw/.
[Gell02]Gellevij, M., van der Meij, H., Jong, T., and Pieters, J. (2002). Visuals in Instruction: Functions of Screen Captures in Software Manuals. In Proceedings of the International Workshop on Dynamic Visualizations and Learning, Tubingen, Germany, 1501-1511.
[Glei02]Gleicher, M. L., Heck, R. M., and Wallick, M. N. (2002). A Framework for Virtual Videography. In Proceedings of the 2nd International Symposium on Smart Graphics, Hawthorne, New York, USA, 9-16.
[Hardo93]Hardock, G., Kurtenbach, G., and Buxton, W. (1993). A marking based interface for collaborative writing. In Proceedings of the 6th annual ACM symposium on User interface software and technology, Atlanta, Georgia, 259-266.
[He99]He, L., Sanocki, E., Gupta, A., and Grudin, J. (1999). Auto-Summarization of Audio-Video Presentations. In Proceedings of the seventh ACM international conference on Multimedia, 489-498.
[Huan01]Huang, X., Acero, A., and Hon, H. W. (2001). Spoken Language Processing: A Guide to Theory, Algorithm and System Development. Chapter 14.
[Ibm05]IBM Corp. (2005). Lotus SmartSuit. See: http://lotus.com/products/product2.nsf/wdocs/sshome.
[Irt06]Internet Related Technologies. (2006). The Amazing ActiveX. See: http://www.irt.org/articles/js178/index.htm.
[Knoy00]Knoy, T. An Editing Workbook for Chinese Technical Writers, 2000.
[Lala03]Lalanne, D., Sire, S., Ingold, R., and Behera, A. (2003). A Research Agenda for Assessing the Utility of Document Annotations in Multimedia Databases of Meeting Recordings. In Proceedings of 3rd International Workshop on Multimedia Data And Document Engineering.
[Lala04]Lalanne, D., Mekhaldi, D., and Ingold, R. (2004). Talking about Documents: Revealing a Missing Link to Multimedia Meeting Archives. In Proceedings of Document Recognition and Retrieval Conference, IS+T SPIE International Symposium on Electronic Imaging, 82-91.
[Latch00]Latchman, H. A., and Gillet, D. (2000). A New Approach in the Use of Multimedia for Technology Enhanced Learning. In Proceedings of the American Control Conference, Chicago, Illinois, USA, 3449-3453.
[Lauer01]Lauer, T., Muller, R., and Ottmann, T. (2001). Animations for Teaching Purposes: Now and Tomorrow. Journal of Universal Computer Science 7(5), 420-433.
[Lauer02]Lauer, T., and Ottmann, T. (2002). Means and Methods in Automatic Courseware Production: Experience and Technical Challenges. In Proceedings of the E-Learn, Montreal, Canada, 553-560.
[Lopre99]Lopresti, D., and Wilfong, G. (1999). Cross-Domain Approximate String Matching. In Proceedings of the String Processing and Information Retrieval Symposium, Cancun, Mexico, 120-127.
[Mekh03]Mekhaldi, D., Lalanne, D., and Ingold, R. (2003). Thematic Alignment of Recorded Speech with Documents. In Proceedings of the ACM Symposium on Document Engineering ,Grenoble, France, 52-54.
[Mspro05]Microsoft Producer Homepage. (2005). Producer 2003. See: http://www.microsoft.com/office/power-point/producer/prodinfo/default.mspx.
[Mswme05]Micorsoft Windows Media Encoder Homepage. (2005). Windows Media Encoder 9 Series. See: http://www.microsoft.com/windows/
windowsmedia/9series/encoder/default.aspx.
[Mukh99]Mukhopadhyay, S., and Smith, B. (1999). Passive Capture and Structuring of Lectures. In Proceedings of the 7th ACM International Conference on Multimedia, Orlando, Florida, USA, 477-487.
[Muller00]Muller, R. and Ottmann, T. (2000). The “Authoring on the Fly” system for Automated Recording and Replay of (tele)presentations. Multimedia Systems 8(3), 158-176.
[Muller02]Muller, R., Ottmann, T., and Zhang, H. (2002). Presentation Recording as a Means to go Virtual for Campus-based Universities. In Proceedings of 13th International Conference on Information Resources Management Association, Seattle, WA, USA, 567-570.
[Neil06]Neil, F. (2006). The Implementation and Extension of the Diff Algorithm proposed by Paul Heckel. See: http://neil.fraser.name/writing/diff/.
[Ogata97]Ogata, H., Yano, Y., and Wakita, R. (1997). COCOA: A Communicative Correction Assisting System for Composition Studies. In Proceedings of International Conference on Computers in Education, Sarawak, Malaysia, 461-468.
[Papa97]Papadias, D., and Theodoridis, Y. (1997). Spatial Relations, Minimum Bounding Rectangles, and Spatial Data Structures. International Journal of Geographic Information System, 11(2), 111-138.
[Real05]RealNetwork Corp. (2005). Accordent’s PresenterONE. See: http://www.realnetworks.com/products-/presenterone/index.html.
[Rowe01]Rowe, L. A., Harley, D., and Pletcher, P. (2001). BIBS: A Lecture Webcasting System. Technical Report, Berkeley Multimedia Research Center, U.C. Berkeley.
[Rui04]Rui, Y., Gupta, A., Grudin J., and He, L. (2004). Automating Lecture Capture and Broadcast: Technology and Videography. Journal of Multimedia Systems 10(1), 3-15.
[Sastry99]Sastry, C. R., Lewis D. P., and Pizano, A. (1999). WEBTOUR: A System to Record and Playback Dynamic Multimedia Annotations on Web Document Content. In Proceedings of the 7th ACM International Conference on Multimedia, Orlando, Florida, USA, 175-178.
[Schar00]Schar, S.G., and Krueger, H. (2000). Using New Learning Technologies with Multimedia. IEEE Multimedia, 7(3), 40-51.
[Schil02]Schillings, V., and Meinel, C. (2002). tele-TASK – Teleteaching Anywhere Solution Kit. In Proceedings of the 30th annual ACM SIGUCCS Conference on User Services, Providence, Rhode Island, USA, 130-133.
[Shi03]Shi, Y., Xie, W., Xu, G., Shi, R., Chen, E., Mao, Y., and Liu, F. (2003). The Smart Classroom: Merging Technologies for Seamless Tele-Education. Journal of IEEE Pervasive Computing 2(2), 47-55.
[Ship03]Shipman, F., Price M., Marshall, C. C., and Golovchinsky, G. (2003). Identifying Useful Passages in Documents based on Annotation Patterns. In Proceedings of 7th European Conference on Research and Advanced Technology for Digital Libraries, 101-112.
[Smart05]Smart Technologies Inc. (2005). SMART Board Interactive Whiteboards. See: http://www2.smarttech.com/st/en-US/Products/
SMART+Boards/.
[Stein96]Steinmetz, R. (1996). Human Perception of Jitter and Media Synchronization. Journal of IEEE Selected Areas in Communications, 14(1), 61-72.
[Stife97]Stifelman, L. J. (1997). The Audio Notebook: Paper and Pen Interaction with Structured Speech. Ph.D. thesis, MIT Media Lab.
[Sugan02]Suganuma, A. (2002). Development of an Automatic Camera Control System for Videoing a Normal Classroom to Realize a Distant Lecture. In Proceedings of E-Learn 2002, Montreal, Canada, 1892-1897.
[Tech05]Techsmith Corp. (2005). Camtasia Studio. See: http://www.techsmith.com/products/studio/default.asp.
[Tpow04]The Purdue OWL. (2004). See:http://owl.english.purdue.edu/.
[W3C]W3C. (2005). Synchronized Multimedia Integration Language (SMIL) Specification. See: http://www.w3.org/TR/REC-smil/.
[Walt04]Walter, Z. (2004). DHTML: Draw Line, Ellipse, Oval, Circle, Poly line, Polygon, Triangle with JavaScript. See: http://www.walterzorn.com/jsgraphics/jsgraphics_e.htm.
[Wang03]Wang, F., Ngo, C. W., and Pong, T. C. (2003). Synchronization of Lecture Videos and Electronic Slides by Video Text Analysis. In Proceedings of the 11th ACM International Conference on Multimedia, Berkeley, California, 315-318.
[Webo06]Webopedia. (2006). What is JavaScript? - A Word Definition From the Webopedia Computer Dictionary. See: http://www.webopedia.com/TERM/J/JavaScript.html.
[Zobel96]Zobel, J., and Dart, P. (1996). Phonetic String Matching: Lessons from Information Retrieval. In Proceedings of the Annual International ACM SIGIR Conference Research and Development in Information Retrieval, 166-172.