臺灣博碩士論文加值系統

由於視訊媒介及網際網路的蓬勃發展與盛行，促使數位學習時代加速來臨，透過此一無遠弗屆的數位學習環境，讓所有使用者可在任何地點、任何時間經由視訊媒介，很容易的擷取各式學習資訊，但是如何提供完善的檢索系統，讓使用者能夠很方便地、很有效地檢索教學視訊串列，以獲得所需學習資訊，是建構完善數位學習環境不容忽視的課題。因為授課老師會在重點部分，花較長時間講解，同時加以註記。基於此一前提，本論文利用電子投影片與教學視訊串列之同步分析，完成手寫註記之偵測，以利建置針對教學視訊之註記投影片檢索系統，藉以協助學生加速獲知重點所在，並在關鍵部分反覆學習，以提升學習效率。
目前雖然有相當多以視覺內容為基礎之教學視訊檢索系統，但大多針對以投影片、主題或文字為基礎，針對註記投影片之檢索系統卻相當貧乏。甚至目前的手寫註記註索系統大都針對以黑板書寫之視訊，而非針對以電子投影片教學之視訊進行處理。所以本論文之目標是提出簡單但有效的針對投影片教學視訊之手寫註記偵測法。
所提方法，首先針對教學視訊串列進行關鍵畫面的擷取，再以投影片特性分析去除非投影片的畫面。接著利用尺度不變特徵比對法，建立投影片和教學視訊畫面的對應關係。再利用單應變換，將投影片反投影至教學視訊畫面，並在其交疊之影像上，根據二值化資訊，進行相關性分析，從而偵測出手寫註記。實驗結果證實所提方法確實有效。

Pervasiveness of streaming media and the Internet has led to the widespread popularity of e-Learning, necessitating an effective means of retrieving lecture videos conveniently. Teachers usually illustrate major pedagogical concepts by taking a considerable amount of time in explanation with handwritten annotations in slides. This work presents a handwritten annotation detection method for e-Learning video streams to facilitating retrieval of annotated slides and increasing students’ learning efficiency.
Although many visual content-based retrieval methods have been proposed for lecture videos in terms of slide, topic, and text, few are specifically designed for handwritten annotations. Moreover, most existing annotation retrieval methods are developed for chalkboard presentation rather than electronic slides. Therefore, the objective of this study was to develop a simple yet effective annotation detection method for lecture videos using slide presentation. The proposed method consists of three stages, slide keyframe extraction, slide/video synchronization, and handwritten annotation detection. Shot boundaries are detected to resample a video stream in a set of keyframes. Non-slide keyframes are then excluded based on slide characteristics. SIFT (Scale Invariant Feature Transformation) matching is used to synchronize lecture slides and videos. Slide images can be then back-projected into the video through homographic transformation. Finally, handwritten annotations are detected using binarization difference. Experimental results demonstrate the feasibility of the proposed method.

Abstract in English ii
Contents iv
List of Figures v
List of Tables vii
Chapter 1 Introduction 1
1.1 Motivation 1
1.2 Survey of related studies 1
1.3 Proposed approach 2
1.4 Organization of the thesis - 3
Chapter 2 Slide Keyframe Extraction 4
2.1 Illumination equalization 4
2.2 Transition detection 5
2.2.1 Feature extraction for transition detection 5
2.2.2 Detecting transition 6
2.3 Slide keyframe detection 8
2.3.1 Binary image projection 9
2.3.2 Detection strategy 10
Chapter 3 Slide-Video Synchronization 11
3.1 SIFT matching 11
3.2 Homographic transformation 14
Chapter 4 Handwritten Annotation Detection 16
Chapter 5 Experimental Results 20
Chapter 6 Conclusions and Future Works 27
References 28

[1]T. Liu and J. R. Kender, “Lecture videos for e-learning: Current research and challenges,” in Proc. IEEE Int’l Symp. Multimedia Software Engineering, 2004, pp. 574–578.
[2]B. Erol and Y. Li, “An overview of technologies for e-meeting and e-lecture,” in Proc. IEEE Int’l Conf. Multimedia Expro, 2005, pp. 6–14.
[3]A. Efrat, A. Amir, K. Barnard, and Q. Fan, “Cross-modality indexing, browsing and search of distance learning media on the web in eBook,” Internet Multimedia Search and Mining, edited by X.-S. Hua, M. Worring, and T.-S. Chua, pp. 1–14.
[4]Q. Fang, A. Amir, K. Barnard, R. Swaminathan, and A. Efrat, “Robust spatiotemporal matching of electronic slides to presentation videos,” IEEE Trans. Image Processing, vol. 20, no. 8, pp. 2315–2328, 2011.
[5]A. Amir, K. Barnard, Q. Efrat, A. Fan, Y. Kharitonova, D. Mathis, R. Swaminathan, J. Torkkola, and A.Winslow, The SLIC Video Browsing System 2005. Available: http://slic.cs.arizona.edu.
[6]X. Wang and M. Kankanhalli, “Robust alignment of presentation videos with slides,” in Proc. Pacific Rim Conf. Multimedia, 2009, pp. 311–322.
[7]X. Wang, S. Ramanathan, and M. Kankanhalli, “A robust framework for aligning lecture slides with video,” in Proc. IEEE Int’l Conf. Image Processing, 2009, pp. 249–252.
[8]G. Gigonzac, P. Pitie, and A. Kokaram, “Electronic slide matching and enhancement of a lecture video,” in Proc. Eur. Conf. Visual Media production, 2007, pp. 1–7.
[9]A. Mavlankar, P. Agrawal, D. Pang, S. Halawa, N.M. Cheung, and B. Girod, “An interactive region-of-interest video streaming system for online lecture viewing,” in Proc. IEEE Int’l Packet Video Workshop, 2010, pp. 64–71.
[10]A. Behera, D. Lalanne, and R. Ingold, “DocMIR: An automatic document-based indexing system for meeting retrieval,” Multimedia Tools and Applications, vol. 37, no. 2, pp. 135–167.
[11]F. Wang, C.W. Ngo, and T.C. Pong, “Structuring low-quality videotaped lectures for cross-reference browsing by video text analysis,” Pattern Recognition, vol. 41, no. 10, pp. 3257–3269, 2008.
[12]T. F. Syeda-Mahmood, “Indexing for topics in videos using foils,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, pp. 312–319.
[13]A. Mittal, P. V. Krishnan, and E. Altman, “Content classification and context-based retrieval system for e-learning,” Educational Technology &; Society, vol. 9, no. 1, pp. 276–291, 2006.
[14]A. Zandifar, R. Duraiswami, and L. S. Davis, “A video-based framework for the analysis of presentation/posters,” Int’l J. Document Analysis and Recognition, vol. 7, nos. 2–3, pp. 178–187, 2005
[15]C. Choudary and T. Lie, “Summarization of visual content in instructional videos,” IEEE Trans. Multimedia, vol. 9, no. 7, pp. 1443–1455, 2007.
[16]M. Liao, R. Yang, and Z. Zhang, “Robust and accurate visual echo cancelation in a full-duplex projector-camera,” IEEE Trans. Pattern Recognition and Machine Intelligence, vol. 30, no. 10, pp. 1831–1840, 2008.
[17]Q. Tung, R. Swaminathan, A. Efrat, and K. Barnard. “Expanding the point automatic enlargement of presentation video elements,” in Proc. ACM Multimedia, 2011.
[18]A. Winslow, Q. Tung, Q. Fang, J. Torkkola, R. Swaminathan, K. Barnard, A. Amir, A. Efrat, and C. Gniady, “Studying on the move – enriched presentation video for mobile devices,” in Proc. IEEE Workshop Mobile Video Delivery, 2009.
[19]D R. Olsen and T. Nielson, “Laser pointer interaction,” in Proc. Conf. Human Factors in Computing Systems, 2001.
[20]J. F. Lapointe and G. Godin , “On-screen laser spot detection for large display interaction,”in Proc. IEEE Int’l Workshop Haptic Audio Environments and their Applications, 2005.
[21]S.Z. Su, S.Y. Chen, S.Z. Li, and D.J. Duh, “Structured local edge pattern moment for pedestrian detection,”, in Proc. Int’l Conf. Image Analysis and Signal Processing, 2010, pp. 556–560.
[22]J.F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679–714, 1986.
[23]Open course source in National Chaio Tung University, http://
[24]D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int’l J. of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
[25]Rob Hess, “SIFT Library” [Online].
Available http://blogs.oregonstate.edu/hess/code/sift/