臺灣博碩士論文加值系統

全景圖和影片摘要都算是影片後處理領域的重要議題之一。在本篇論文當中,我們主要的目的在於從立體影片中產生一長串的立體全景圖。這裡衍生出來的問題為如何來維持深度資訊和如何來校正邊緣資訊來完整保留邊緣的資訊。先前的方法主要著重在從一般的2D影片中產生全景圖，並且在接圖過程後直接切割不規則區域保留重要區域。跟以往做法不同，本篇著重在經過仔細挑選關鍵影格和深度資訊維持的過程。我們的系統盡可能的在圖片內容減少扭曲現象並且能盡量維持深度資訊。

為了達到令人驚豔的效果，我們的演算法包含了三個步驟。首先，我們從立體影片中挑選具代表性的影格。接著考慮圖片的品質，模糊和搖晃的影格必須避免挑選到，而且要確保關鍵影格彼此之間有涵蓋住。第二步驟我們把這些關鍵影格透過接圖方式銜接在一起。但得到的全景圖會有不規則的邊界。最後我們使用了深度感知的圖片變形技術來對立體圖的左右眼全景圖作用以達到恢復深度感知的資訊並且產生一般常見的長方形型態的全景圖。

透過一連串的實驗過程，總結我們的系統是相當實用於產生令人滿意的立體全景圖，並且展示我們方法的有效程度。

Both of image panorama and video synopsis are vital topics in the field of video post-processing. In this work, we aim at generating a long strip of stereo panoramic view from stereoscopic video to provide synopsis with depth perception. The problem comes from that how to preserve disparity well and rectify panorama without loss of boundary information. Previous approach focus on the generation of panorama from usual 2D video, in addition, only crop unwanted region after stitching step. Unlike previous work, after careful handling on key frame selection and disparity preservation in warping process, our system minimize distortion on the image plane and maintain sense of depth. With an eye to achieving appealing results, our algorithm consists of three steps. First, we select representative frame intelligently from stereoscopic video. Take image quality into consideration, blurry or shaking frame must be avoided and the regional coverage of key frame must be assured. Second, we stitch these key frames into a panorama, which have irregular boundary. Finally, we perform disparity-aware image warping step on left and right panoramic images to restore depth perception of original video and generate rectangular stereo panorama. Following series of experiments, we conclude our work is practicable for generating visually plausible stereoscopic panorama result and demonstrates the effectiveness of our method.

中文摘要 I
ABSTRACT III
誌謝 V
CONTENTS VI
LIST OF FIGURE VII
Chapter 1 INTRODUCTION 1
Chapter 2 RELATED WORK 5
2.1 Panoramic image generation 5
2.2 Image stitch 5
2.3 Image warping and Image retargeting 6
2.4 Stereoscopic image processing 6
2.5 Action synopsis ＆ Video Summary 7
Chapter 3 SYSTEM OVERVIEW 8
Chapter 4 ALGORITHM 10
4.1 Preprocessing: key frame selection 10
4.2 Image stitching 16
4.3 Panorama Rectifying 17
4.3.1 Hybrid mesh 20
4.3.2 Segmentation 21
4.3.3 Saliency detection 22
4.3.4 Image warping energy term 23
4.4 Disparity preserving 27
4.5 Application – Action synopsis 29
Chapter 5 RESULTS 33
Chapter 6 CONCLUSION AND FUTURE WORK 47
REFERENCES 49

[1] Assa, J., et al. (2005). Action synopsis: pose selection and illustration. ACM Trans. Graph. 24(3): 667-676.
[2] Barnes, C., et al. (2010). Video tapestries with continuous temporal zoom. ACM SIGGRAPH 2010 papers. Los Angeles, California, ACM: 1-9.
[3] Bennett, E. P. and L. McMillan (2007). Computational time-lapse video. ACM Trans. Graph. 26(3): 102.
[4] Chen, T., et al. (2012). Visual storylines: Semantic visualization of movie sequence. Computers ＆ Graphics 36(4): 241-249.
[5] Correa, C. D. and K.-L. Ma (2010). Dynamic video narratives. ACM SIGGRAPH 2010 papers. Los Angeles, California, ACM: 1-9.
[6] Dalal, N. and B. Triggs (2005). Histograms of oriented gradients for human detection. Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on.
[7] Kang, H.-W., et al. (2006). Space-Time Video Montage. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2, IEEE Computer Society: 1331-1338.
[8] Barnes, C., et al. (2009). PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3): 1-11.
[9] Goferman, S., et al. (2012). Context-Aware Saliency Detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on 34(10): 1915-1926.
[10] He, K., et al. (2013). Rectangling panoramic images via warping. ACM Trans. Graph. 32(4): 1-10.
[11] Kopf, J., et al. (2012). Quality prediction for image completion. ACM Trans. Graph. 31(6): 1-8.
[12] Lin, S., et al. (2013). Object-Coherence Warping for Stereoscopic Image Retargeting. Circuits and Systems for Video Technology, IEEE Transactions on PP(99): 1-1.
[13] Shewchuk, J. R. (1996). Triangle: Engineering a 2D quality mesh generator and Delaunay triangulator. Applied computational geometry towards geometric engineering, Springer: 203-222.
[14] Shih-Syun, L., et al. (2013). Patch-Based Image Warping for Content-Aware Retargeting. Multimedia, IEEE Transactions on 15(2): 359-368.
[15] Sorkine, O., et al. (2004). Laplacian surface editing. Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing. Nice, France, ACM: 175-184.
[16]Suzuki, S. and K. be (1985). Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics, and Image Processing 30(1): 32-46.
[17] Wexler, Y., et al. (2007). Space-Time Completion of Video. IEEE Trans. Pattern Anal. Mach. Intell. 29(3): 463-476.
[18] Agarwala, A., et al. (2005). Panoramic video textures. ACM Trans. Graph. 24(3): 821-827.
[19] Brown, M. and D. G. Lowe (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision 74(1): 59-73.
[20] Liu, F., et al. (2008). Discovering panoramas in web videos. Proceedings of the 16th ACM international conference on Multimedia. Vancouver, British Columbia, Canada, ACM: 329-338.
[21] Oettermann, S. and D. L. Schneider (1997). The panorama: history of a mass medium, Zone books New York.
[22] Shum, H.-Y. and R. Szeliski (2000). Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment. International Journal of Computer Vision 36(2): 101-130.
[23] Szeliski, R. (2010). Computer vision: algorithms and applications, Springer.
[24] Che-Han, C., et al. (2011). Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images. Multimedia, IEEE Transactions on 13(4): 589-601.
[25] Fischler, M. A. and R. C. Bolles (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6): 381-395.
[26] Lin, S., et al. (2013). Object-Coherence Warping for Stereoscopic Image Retargeting. Circuits and Systems for Video Technology, IEEE Transactions on PP(99): 1-1.
[27] Niu, Y., et al. (2012). Enabling warping on stereoscopic images. ACM Trans. Graph. 31(6): 1-7.
[28] Yan, T., et al. (2013). Seamless stitching of stereo images for generating infinite panoramas. Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology, ACM.
[29] Chen, B. Y., et al. (2008). Capturing Intention‐based Full‐Frame Video Stabilization. Computer Graphics Forum, Wiley Online Library.
[30] Liujun, L., et al. (2011). Key Frames Extraction Algorithm Based on GA. Computational Sciences and Optimization (CSO), 2011 Fourth International Joint Conference on.
[31] Zhuang, Y., et al. (1998). Adaptive key frame extraction using unsupervised clustering. Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on, IEEE.
[32] Zhang, F. and F. Liu Parallax-tolerant Image Stitching. Computer Vision and Pattern Recognition 2014 IEEE Computer Society Conference on.