臺灣博碩士論文加值系統

這篇論文旨在研究並實作以漫畫風格呈現電影內容的摘要。電影運用各種視覺技巧於連續影格上敘述故事的特性，已經在影片內容分析的領域被大量研究。然而目前為止，卻沒有針對電影蘊含故事的視覺特性，製作創意內容摘要的研究成果。我們認為漫畫擁有豐富的視覺敘事語言，是再適合不過的電影摘要形式。因此本文藉由重新建立電影與漫畫之間視覺語言轉換的規則，發展一個能夠迅速且有效產生漫畫風格電影摘要的自動系統。本系統依序運用一次掃描的視頻處理技術處理影片內容，從中選出關鍵影格用以製作漫畫畫格，再以啟發式演算法排版畫格並放置對話氣球，最後使用非擬真算圖技術處理頁面，使其在作為影片資訊摘要的功能以外，還具有視覺上的漫畫風格。這個系統易於實作、運作快速，而且擁有彈性的模組架構，足夠延伸用以處理不同的電影類型和漫畫風格，或擴充以適應各類特定的影片處理技術。

This paper intends to use comics as the form of presentation best for movie content summarization. Movies, with its powerful ability to convey stories and evoke emotions through moving frames, find a significant body of research and application in video content analysis field. However, while this art form has been widely investigated using existing video analysis technology, none of them has been able to produce story content summarization with pleasant or satisfactory results. Indeed, the comic form’s naturally rich visual story-telling vocabulary and vivid imagery is ideal for movie summarization. By re-examining the translation rules between movie and comics, we have been successful in building an effective system that produces comic-styled movie summaries within a reasonable time frame. In our system, a heuristic pictorial layout and balloon placement algorithm is proposed after image processing of keyframes selected by a one-pass video processing. By applying comic style rendering, the generated movie content summary is greatly enhanced in its appearance. The system is easy to implement, fast, and flexible; it can be adapted for use in a variety of movie genres and comic styles, and extended to fit in specific video processing techniques.

謝辭
摘要 i
Abstract iii
Contents v
List of Figures vii
1 Introduction 1
2 Related work 5
2.1 Video summarization 5
2.2 Movie content analysis 7
2.3 Pictorial layout 8
2.4 Comic style 9
3 Translation rules between comics & movie 13
3.1 Panel 13
3.2 Balloon 18
3.3 Page layout 22
4 System 27
4.1 Shot detection 29
4.2 Shot categorization 32
4.3 Keyframes selection 33
4.4 Salience map 35
4.5 Page layout 36
4.6 Balloon placement 38
4.7 Comic-styled rendering 39
5 Results 43
5.1 Experiment 43
5.2 Discussion 67
6 Conclusion 69
Acknowledgments 71
References 73
Contact

B. Adams, C. Dorai, and S. Venkatesh. Automated film rhythm extraction for
scene analysis. In Proc. IEEE ICME ’01, pp 1056–1059, 2001.
J. M. Alderman. Generating comics narrative to summarize wearable computer
data. Master’s thesis, Georgia Institute of Technology, 2006a.
J. M. Alderman. Further resources for generating comics narrative to summarize
wearable computer data, 2006b. [http://dm.lcc.gatech.edu/
~jalderman/comics/].
S. Avidan and A. Shamir. Seam carving for content-aware image resizing. In
ACM SIGGRAPH ’07, 2007.
R. Berdan. Composition and the elements of visual design, 2004.
[http://photoinf.com/General/Robert_Berdan/Composition_and_
the_Elements_of_Visual_Design.htm].
D. Bordwell and K. Thompson. Film Art: An Introduction. McGraw-Hill
Companies, 6 edition, 2003.
J. ’Cali’c and N. W. Campbell. Optimizing layout of video summaries for mobile
devices using visual attention modeling. In Proc. 2nd Intl. Mobile Multimedia
Communications Conf. (MobiMedia 2006), 2006.
J. ’Cali’c, D. P. Gibson, and N. W. Campbell. Efficient layout of comic-like
video summaries. IEEE Trans. Circuits and Systems for Video Technology,
17(7):931–936, 2006.
J. Canny. A computational approach to edge detection. IEEE Trans. Pattern
Analysis and Machine Intelligence, 8(6):679–698, 1986.
Y.-W. Chang. Floorplanning of VLSI design automation. 2006. [http://cc.
ee.ntu.edu.tw/~eda/Course/VLSIDesignAuto/LN/floorplanning.pdf].
L.-Q. Chen, X. Xie, X. Fan, W.-Y. Ma, H. jiang Zhang, and H.-Q. Zhou. A
visual attention model for adapting images on small displays. ACM Multimedia
Systems Journal, 9(4):353–356, 2003.
L.-J. Chiu. Comic-styled photo album layout using simulated annealing. Master’s
thesis, National Taiwan University, Taipei, Taiwan, 2006.
B.-K. Chun, D.-S. Ryu, W.-I. Hwang, and H.-G. Cho. An automated procedure
for word balloon placement in cinema comics. In Proc. 2nd Intl. Symp.
Adcances in Visual Computing, pp 576–585, 2006.
J. P. Collomosse, D. Rowntree, and P. M. Hall. Video analysis for cartoon-style
special effects. In Proc. 14th British Machine Vision Conf., volume 2, pp
749–758, 2003.
J. E. Cutting. Representing motion in a static image: Constraints and parallels
in science, art, and popular culture. Perception Magazine, 31:1165–1193,
2002.
R. Duncan. Toward a theory of comic book communication. [http://www.
hsu.edu/default.aspx?id=3508].
W. Eisner. Comics & Sequential Art. Tamarac: Poorhouse Press, 1985.
J. Geigela and A. Loui. Using genetic algorithms for album page layouts. IEEE
Multimedia, 10(4):16–27, 2003.
A. Girgensohn. A fast layout algorithm for video summaries. In Proc. IEEE
ICME ’03, volume 2, pp 77–80, 2003.
D. B. Goldman, B. Curless, S. M. Seitz, and D. Salesin. Schematic storyboarding
for video visualization and editing. ACM Trans. Graphics (Proc.
SIGGRAPH’06), 25(3), 2006.
C.-J. Hu. A real-time skin-color-enhanced face detection algorithm. Master’s
thesis, National Taiwan University, Taipei, Taiwan, 2007.
W.-I. Hwang, P.-J. Lee, B.-K. Chun, D.-S. Ryu, and H.-G. Cho. Cinema
comics: Cartoon generation from video stream. In Proc. Intl. Conf. Computer
Graphics Theory and Applications, pp 299–304, 2006.
IMSDb. The internet movie script database. [http://www.imsdb.com/].
Intel. Open source computer vision library. Technical Report 123456-001, Intel
Corporation, 2000.
L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for
rapid scene analysis. In IEEE Tans. PAMI ’98, volume 20, pp 1254–1259,
November 1998.

C. Jacobs, W. Li, E. Schrier, D. Bargeron, and D. Salesin. Adaptive grid-based
document layout. In Proc. ACM SIGGRAPH’03, pp 838–847, 2003.
S. Kopf, F. Lampi, T. King, and W. Effelsberg. Automatic scaling and cropping
of videos for devices with limited screen resolution. In Proc ACM
MULTIMEDIA’06, pp 957–958, 2006.
D. Kurlander, T. Skelly, and D. Salesin. Comic chat. In Proc. ACM SIGGRAPH’
96, pp 225–236, 1996.
J. Lasseter. Principles of traditional animation applied to 3D computer animation.
In Proc. ACM SIGGRAPH ’87, 1987.
K.-Y. Lee. Speaker localization for comic-styled movie summarization. Master’s
thesis, National Taiwan University, Taipei, Taiwan, 2007.
Y. Li, T. Zhang, and D. Tretter. An overview of video abstraction techniques.
Technical Report HPL-2001-191, HP Laboratories Palo Alto, 2001.
Y. Li, S.-H. Lee, C.-H. Yeh, and C. C. J. Kuo. Techniques for movie content
analysis and skimming: tutorial and overview on video abstraction techniques.
IEEE Signal Processing Magazine, 23(2):79–89, 2006.
R. Lienhart. Comparison of automatic shot boundary detection algorithems. In
Proc. SPIE Conf. on Storage and Retrieval for Image and Video Databases,
volume 12, pp 290–301, 1999.
R. Lienhart, S. Pfeiffer, and W. Effelsberg. Video abstracting. Commununications
of ACM, 40(12):54–62, 1997.
J.-S. Lin. Full-frame video stabilization by considering capturing intention.
Master’s thesis, National Taiwan University, Taipei, Taiwan, 2007.
A. Lodi, S. Martello, and M. Monaci. Two-dimensional packing problems: A
survey. European Journal of Operational Research, 141(2):241–252, 2002.
S. Lok and S. Feiner. A survey of automated layout techniques for information
presentations. In Proc. 1st Intl. Symp. Smart Graphics, pp 61–68, 2001.
Y.-F. Ma, L. Lu, H.-J. Zhang, and M. Li. A user attention model for video
summarization. In Proc. ACM MULTIMEDIA ’02, pp 533–542, 2002.
S. McCloud. Understanding Comics. Harper Perennial, 1994.
M. Mills, J. Cohen, and Y. Y. Wong. A magnifier tool for video data. In Proc.
ACM SIGCHI ’92, pp 93–98, 1992.
J. Morris. 16 panels that i don’t think work all that well, 2007. [http://
comixtalk.com/16_panels_that_i_dont_think_work_all_that_well].
C. W. Ng and M. R. Lyu. Advise: Advanced digital video information segmentation
engine. In Poster Proc. Intl. World Wide Web Conf., 2002.
Opensubtitles.org. [http://www.opensubtitles.org/en].
Z. Pan and C.-W. Ngo. Structuring home video by snippet detection and
pattern parsing. In Proc. 6th ACM SIGMM Intl. Workshop Multimedia
Information Retrieval, pp 69–76, 2004.
Rivkah. Paneling, pacing, and layout in comics and manga #2, 2006. [http:
//lilrivkah.livejournal.com/169915.html?thread=1435067].
M. Rush. New Media in Art. Thames & Hudson, 2 edition, 2005.
M. Shilman, P. Liang, and P. Viola. Learning non-generative grammatical
models for document analysis. In Proc. IEEE ICCV ’05, pp 962–969, 2005.
Shooters.com. [http://shooter.cn/].
E. Stavrakis. Non-photorealistic computer graphics library. [https://www.
npcglib.org/index.php].
B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs. Automatic thumbnail
cropping and its effectiveness. In Proc. ACM UIST ’03, pp 95–104, 2003.
Y. Taniguchi, A. Akutsu, and Y. Tonomura. Panoramaexcerpts: Extracting
and packing panoramas for video browsing. In Proc. ACM MULTIMEDIA’
97, pp 427–436, 1997.
TIME. TIME magazine, all-time 100 novels: Graphic novels. [http://www.
time.com/time/2005/100books/0,24459,graphic_novels,00.html].
B. T. Truong and S. Venkatesh. Video abstraction: A systematic review and
classification. ACM Trans. Multimedia Computing, Communication and Appication,
3(1):3, 2007.
S. Uchihashi and J. Foote. Summarizing video using a shot importance measure
and a frame-packing algorithm. In Proc. Intl. Conf. Acoustics, Speech, and
Signal Processing, volume 6, pp 3041–3044, 1999.
S. Uchihashi, J. Foote, A. Girgensohn, and J. Boreczky. Video manga: Generating
semantically meaningful video summaries. In Proc. ACM MULTIMEDIA’
99, pp 383–392, 1999.
J. Wang, Y. Xu, H.-Y. Shum, and M. F. Cohen. Video tooning. In ACM
SIGGRAPH ’04, pp 574–583, 2004.
Wikipedia. List of award-winning graphic novels. [http://en.wikipedia.
org/wiki/List_of_graphic_novels:_Award-winning].
H. Winnem‥oller, S. C. Olsen, and B. Gooch. Real-time video abstraction. In
Proc. ACM SIGGRAPH’06, pp 1221–1226, 2006.
W. Wood. 22 panels that always work, 1970s. [http://joeljohnson.com/
archives/2006/08/wally_woods_22.html].
M. M. Yeung and B.-L. Yeo. Time-constrained clustering for segmentation of
video into story units. In Proc. ICPR ’96, volume 3, pp 375–380, 1996.
M. M. Yeung and B.-L. Yeo. Video visualization for compact presentation and
fast browsing of pictorial content. IEEE Trans. Circuits and Systems for
Video Technology, 7(5):771–785, 1997.
J. Zabel. Comics page composition, a. [http://amazingmontage.tripod.
com/page.html].
J. Zabel. Comics theory and comics traditions, b. [http://amazingmontage.
tripod.com/tradition.html].
R. Zabih, J. Miller, and K. Mai. A feature-based algorithm for detecting and
classifying scene breaks. In Proc. ACM MULTIMEDIA’95, pp 189–200,
1995.