( 您好!臺灣時間:2021/04/11 19:00
字體大小: 字級放大   字級縮小   預設字形  


研究生(外文):Yung Chieh Chou
論文名稱(外文):Toward More Efficient Multi-Operator Media Retargeting for Digital Images and Videos
指導教授(外文):Po Chyi Su
外文關鍵詞:Multi-OperatorContent-based CroppingSeam CarvingVisual Saliency MapH.264 Motion VectorMotion Feature Map
  • 被引用被引用:1
  • 點閱點閱:70
  • 評分評分:系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔系統版面圖檔
  • 下載下載:1
  • 收藏至我的研究室書目清單書目收藏:0
本研究提出多運算子影像與視訊尺寸調整(retargeting)演算法,目的在於有效率地調整影像畫面至目標解析度,並將演算法延伸應用於視訊。對於數位影像,我們適當地施予基於內容之邊緣裁切(content-based cropping)和縮放(scaling),首先計算影像中的視覺顯著特徵(visual saliency feature),並將影像透過SLIC(Simple Linear Iterative Clustering)演算法切割成較大的超級像素(superpixel),擷取畫面中的前景物作為畫面切割的依據,接著逐一比較視覺特徵圖進行邊緣裁切與等比例縮放。若時間允許,圖縫裁減(seam carving)可被使用讓畫面更接近目標長寬比。圖縫裁減主要計算畫面梯度,採用動態規劃刪除最小能量圖縫並進行圖縫的局部更新,最後定義突出點以限制圖縫數量並決定裁減停止點。對於某些適合的影像,我們亦可增加圖縫來降低畫面直接縮放程度。由實驗結果顯示,我們確實有效率地維持影像主體,演算法也達到較高的實用性。另外,我們將影像處理延伸至視訊資料,考量視訊壓縮域動態資料計算,透過H.264/AVC視訊壓縮編碼時所產生的運動向量(motion vector)和運動補償資訊(motion compensation)判斷鏡頭種類,若為非固定式場景,我們使用邊緣裁切以及縮放的方式處理畫面;若為固定場景,則可使用圖縫裁減機制。為了防止運動中的前景物在裁切過程中被移除而造成失真,我們將壓縮域中的位移向量製作運動特徵圖(motion feature map),結合視覺特徵圖協助圖縫裁減和邊緣裁切。實驗結果顯示我們的方法可以廣泛處理不同種類的鏡頭,在畫面前景物形狀的維持以及背景保留上,亦優於其他視訊畫面調整演算法。
This research presents a multi-operator image retargeting scheme, which can be further expanded to video retargeting. The objective is to effectively and efficiently adjust the image or video frame to the targeted resolution. Given an image or frame, the content-based cropping and scaling will be applied. The visual saliency map is calculated and the superpixels are formed via Simple Linear Iterative Clustering (SLIC) to serve as the reference to extract the visually significant foreground objects. Next, the degree of cropping and scaling will be determined by the saliency map. Seam carving can also be employed to make the resolution closer to the target if the efficiency is not an important issue. Seam caving checks the one-directional gradients and uses dynamic programming to remove the saliency with minimal significance. Local update helps to reduce the computational burden. Saliency points are identified and helps to decide when to stop the seam carving process. For certain images, inserting seams is also useful to decrease the the degree of scaling. Experimental results show that the proposed method does maintain the significant objects of the image and is also more feasible.

For video retargeting, the data in compressed video stream, including the motion vectors and motion compensation, are used to classify the types of shots. If the shot belongs to a fixed scene, seam carving can be applied. Otherwise, only cropping and scaling are used. To avoid removing the foreground objects, the motion feature map is formed, combined with the visual saliency map, to achieve seam carving and cropping. The experimental results shows that the proposed scheme can deal a variety of shots and outperform existing algorithms.

論文摘要 i
Abstract ii
誌謝 iv
目錄 v
附圖目錄 viii
附表目錄 xi
第一章 緒論 1
1.1 研究背景與動機 1
1.2 研究貢獻 3
1.3 論文架構 4
第二章 相關研究 5
2.1 影像畫面調整機制 5
2.1.1邊緣裁切方法 5
2.1.2影像變形方法 6
2.1.3圖縫裁減方法 6
2.1.4多種運算子方法 7
2.2 視訊畫面調整機制 8
2.2.1基於內容之縮放和區塊變形 9
2.2.2加強版圖縫裁減 9
第三章 多運算子影像尺寸調整機制 11
3.1 系統流程 11
3.2 視覺顯著特徵 12
3.3 超級像素切割 15
3.4 基於內容之邊緣裁切 19
3.4.1前景物擷取 19
3.4.2連通數分析 20
3.4.3裁切邊緣分析 22
3.4.4特殊狀況 : 模糊背景切割 24
3.5 圖縫裁減 27
3.5.1 圖縫連接的方法 27
3.5.2 空間關聯度衡量 31
3.5.3 長斜直線偵測 34
3.5.4 顯著特徵點以及圖縫裁減停止點分析 36
3.5.5 圖縫添加 39
3.6 影像調整原則分析 41
第四章 多運算子之視訊調整機制 42
4.1 系統流程 42
4.2 H.264/AVC壓縮域資料分析 43
4.2.1 壓縮域運動預測和運動補償 43
4.2.2 視覺重要度分析 46
4.3 基於內容之視訊畫面調整 49
4.3.1 視訊鏡頭分類 49
4.3.2 固定鏡頭調整機制 51
4.3.3 非固定鏡頭調整機制 53
第五章 實驗結果 56
5.1 影像畫面調整機制 56
5.1.1 視覺顯著圖(VSF) 56
5.1.2 基於內容之影像裁切實驗結果 58
5.1.3 圖縫裁減實驗結果 61
5.1.4 實驗結果比較 63
5.1.5 影像調整演算法品質分析和執行效率比較 69
5.2 視訊畫面調整機制 74
5.2.1 視訊分類實驗結果 74
5.2.2 固定鏡頭畫面調整實驗結果 75
5.2.3 非固定鏡頭畫面調整實驗結果 77
5.2.4 視訊鏡頭連續畫面比較 80
第六章 結論與未來方向 83
6.1 結論 83
6.2 未來方向 84
參考文獻 85

[1] D. Vaquero, M. Turka, K. Pullib, M. Ticob, and N. Gelfandb, "A survey of image retargeting techniques," Proceedings of SPIE the International Society for Optical Engineering, vol. 7798, p. 779814, 2010.
[2] M. Rubinstein, D. Gutierrez, and O. Sorkine, "A comparative study of image retargeting," ACM Transactions on Graphics (TOG), vol. 29, no. 6, p. 160, 2010.
[3] S. Montabone, and A. Soto., "Human detection using a mobile platform and novel features derived from a visual saliency mechanism," Image and Vision Computing, vol. 28, no. 3, pp. 391-402, 2010.
[4] L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, 1998.
[5] M. Zhang, and L. Zhang, "Auto cropping for digital photographs," IEEE International Conference on Multimedia and Expo, p. 4-pp, 2005.
[6] F. Stentiford, "Attention based auto image cropping," The 5th International Conference on Computer Vision Systems, Bielefeld, 2007.
[7] I. S. Amrutha, S. S. Shylaja, S. Natarajan, and K. N. Murthy, "A smart automatic thumbnail cropping based on attention driven regions of interest extraction," Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, ACM, pp. 957-962, 2009.
[8] P. Cheatle, "Automatic image cropping for republishing," IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, pp. 75400O-75400O-9, 2010.
[9] A. Santella , M. Agrawala, D. DeCarlo, D. Salesin, and M. Cohen, "Gaze-based interaction for semi-automatic photo cropping," Proceedings of the SIGCHI conference on Human Factors in computing systems. pp. 771-780, ACM, 2006.
[10] H. Liu, X. Xie, W. Y. Ma, and H. J. Zhang,"Automatic browsing of large pictures on mobile devices,"Proceedings of the eleventh ACM international conference on Multimedia. pp. 148-155, 2003.
[11] B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobjs, "Automatic thumbnail cropping and its effectiveness," Proceedings of the 16th annual ACM symposium on User interface software and technology, 2003.
[12] G. Ciocca, C. Cusano, F. Gasparini, and R.Schettini, "Self-adaptive image cropping for small displays," IEEE Transactions on Consumer Electronics, vol. 53, no. 4, pp. 1622-1627, 2007.
[13] S. Avidan, and A. Shamir, "Seam carving for content-aware image resizing," ACM Transactions on graphics (TOG), vol. 26, no. 3, Aug 2007.
[14] M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Transactions on Graphics (TOG), vol. 27, no. 3, p. 16, 2008.
[15] M. Grundmann, V. Kwatra, M. Han, and I. Essa, "Discontinuous seam-carving for video retargeting," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, June 2010.
[16] D. Domingues, A. Alahi, and P. Vandergheynst, "Stream carving: an adaptive seam carving algorithm," 17th IEEE International Conference on Image Processing (ICIP), pp. 901-904, 2010.
[17] R. Gal, O. Sorkine, and D. Cohen-Or, "Feature-aware texturing," Proceedings of the 17th Eurographics conference on Rendering Techniques. Eurographics Association, June 2006.
[18] L. Wolf, M. Guttmann, and D. Cohen-Or, "Non-homogeneous content-driven video-retargeting," IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, 2007.
[19] Y. S. Wang, C. L. Tai, O. Sorkine, and T. Y. Lee, "Optimized scale-and-stretch for image resizing," ACM Transactions on Graphics (TOG), vol. 27, no. 5, 2008.
[20] S. Hua, G. Chen, H. Wei , and Q. Jiang, "Similarity measure for image resizing using SIFT feature," EURASIP Journal on Image and Video Processing, pp. 1-11, Jan 2012.
[21] M. Rubinstein, A. Shamir, and S. Avidan, "Multi-operator media retargeting," ACM Transactions on Graphics (TOG), vol. 28, no. 3, 2009.
[22] W. Dong, N. Zhou, J. C. Paul, and X. Zhang, "Optimized image resizing using seam carving and scaling," ACM Transactions on Graphics (TOG), vol. 28, no. 5, p.125, 2009.
[23] Y. S. Wang, H. C. Lin, O. Sorkine, and T. Y. Lee, "Motion-based video retargeting with optimized crop-and-warp," ACM Transactions on Graphics (TOG), vol. 29, no. 4, p.90, 2010.
[24] S. Luo, J. Zhang, Q. Zhang, and X. Yuan, "Multi-operator image retargeting with automatic integration of direct and indirect seam carving," Image and Vision Computing, 2012.
[25] W. M. Dong, G. B. Bao, X. P. Zhang, and J. C. Paul, "Fast multi-operator image resizing and evaluation," Journal of Computer Science and Technology, vol. 27, no. 1, pp. 121-134, 2012.
[26] Huan Du, Zhi Liu, Zhiguo Yan, and Yayun Jiang, "Video retargeting based on stretchability aware block scaling," IEEE conference, The Third Research Institute of Ministry of Public Security, 2014.
[27] Lingqiang Ran, Na Lv, and Xiangxu Meng, "Image retargeting based on spring analogy, " International Conference on Progress in Informatics and Computing, 2014.
[28] Yanwen Guo, Feng Liu, Jian Shi,Zhi-Hua Zhou, and Michael Gleicher, "Image retargeting using mesh parametrization," IEEE Transactions on Multimedia, vol. 11, no. 5, August 2009.
[29] Zhiquan Ren, Botao Wang, Yuchen Zhang, Hongkai Xiong, "Visual preserving video retargeting with deformable shape consistency," IEEE conference, Department of Electronic Engineering Shanghai Jiao Tong University, 2013.
[30] BoYan, Kairan Sun, Liu Liu, "Matching-area-based seam carving for video retargeting, " IEEE Transactions on circuits and systems for video technology, vol. 23, no. 2, February 2013.
[31] Wei-Lun Chao, Hsiao-Hang Su, Shao-Yi Chien, Winston Hsu3, and Jian-Jiun Ding, "Coarse-to-fine temporal optimization for video retargeting based on seam carving," International Conference on Multimedia and Expo (ICME), pp. 1-6, 2011.
[32] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk, "SLIC Superpixels," EPFL Technical Report, no. 149300, June 2010.
[33] Jiangyang Zhang, Shangwen Li, C.-C. Jay Kuo, "Compressed-domain video retargeting," IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 797-809, February 2014.
[34] R. Wang, H.-J. Zhang, and Y.-Q. Zhang, "A confidence measure based moving object extraction system built for compressed domain," Circuits and Systems, Proceedings. ISCAS 2000 Geneva, The 2000 IEEE International Symposium, vol. 5, pp. 21–24, May 2000.
[35] Chih-Chung Hsu, Chia-Wen Lin, Yuming Fang, Weisi Lin, "Objective quality assessment for image retargeting based on perceptual distortion and information loss," IEEE Journal of Selected Topics in Signal Processing, vol. 8, no. 3, pp. 377-389, 2014.
[36] RetargetMe Benchmark [Online]. Available: http:// http://people.csail. mit.edu/mrub/retargetme/index.html
[37] Renjie Chen, Daniel Freedman, Zachi Karni, Craig Gotsman, Ligang Liu, "Content-aware image resizing by quadratic programming," Computer Vision and Pattern Recognition Workshops (CVPRW), July 2010.
[38] Z. Karni, D. Freedman, C. Gotsman, "Energy-based image deformation," Eurographics Symposium on Geometry Processing, vol. 28, no.5, pp. 1257-1268, July 2009.
[39] Yuanyuan Ding, Jing Xiao, Jingyi Yu, "Importance filtering for image retargeting," Computer Vision and Pattern Recognition (CVPR), pp. 89-96, June 2011.
[40] NTHU Retargeting Image Dataset (NRID) [Online]. Available: http://www.ee.nthu.edu.tw/cwlin/Retargeting_Quality/NRID.html
[41] Xiph.org Video Test Media [Online]. Available: https://media.xiph.org/video/derf/

註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
第一頁 上一頁 下一頁 最後一頁 top
系統版面圖檔 系統版面圖檔