臺灣博碩士論文加值系統

MPEG-2 視訊標準廣泛的運用於即時影片播放系統中。在視訊傳輸過程裡些微的傳輸錯誤，都可能降低整體的視訊品質。 錯誤隱匿演算法可以有效的隱匿發生在視訊串流中的錯誤區域。本文中我們提出一個時間性錯誤隱匿演算法，利用相鄰畫面的相關性，使用前一個畫面裡的資訊來修補目前畫面的錯誤。本文提出的演算法以Object-Based 動態向量還原錯誤隱匿演算法和Average of Motion Vectors of Vertically Adjacent Macroblock (AVG) 演算法為基礎， Object-Based演算法能有效運用相鄰畫面的相似性找到和錯誤部份相似的區域進行錯誤隱匿並具備低運算量的優點，可以有效運用於即時視訊播放系統。而AVG演算法運用被破壞區塊的上下相鄰動態向量還原錯誤區塊中的動態向量，此方法能夠修補連續Slice中的錯誤。由於這兩種演算法都具有良好的錯誤隱匿品質，我們改良並結合這兩種演算法的優點，並將其運用於即時播放視訊系統和MPEG-2視訊串流。經由實驗模擬，本文提出的演算法可以有效的修補錯誤Slice中的動態向量，提昇視訊品質和影像信號雜訊比(PSNR)而且執行時間短，可以運用於即時播放影片系統。

The MPEG-2 video compression standard is commonly used in real-time video transmission, where a small bit error can cause huge data loss in the video sequence. That is, the errors and data loss degrade the quality of video. The error concealment method is useful to overcome the errors in the corrupted slice of MPEG-2 video stream. In this paper, an efficient error concealment algorithm for MPEG-2 video stream based on the Object-Based motion vector recovery method and Average of Motion vectors of Vertically Adjacent Macroblock (AVG) method is proposed. Our proposed method is a temporal error concealment method for P-frame and B-frame. By using the correlation between the previous and the current frame, the proposed method searches the object in the reference frame to recover the motion vector and the pixel values of the corrupted slice in the current frame. The simulation results indicate that the proposed algorithm can efficiently improve the PSNR values of corrupted video and the computation cost is simple.

TABLE OF CONTENTS
CHINESE ABSTRACT i
ENGLISH ABSTRACT ii
ACKNOWLEDGEMENTS iii
TABLE OF CONTENTS iv
LIST OF FIGURES vi
LIST OF TABLES ix
Chapter 1 MOTIVATION 1
Chapter 2 BACKGROUND AND RELATED WORK 4
2.1 MPEG-2 Encoding Process 4
2.2 MPEG-2 Decoding Process 5
2.3 The Structure of MPEG-2 Video Standard 5
2.3.1 Video Sequence 5
2.3.2 Group of Pictures (GOP) 6
2.3.3 Frame 6
2.3.3.1 Intra Frame (I-Frame) 6
2.3.3.2 Predicted Frame (P-Frame) 6
2.3.3.3 Bidirectional Frame (B-Frame) 7
2.3.4 Block 7
2.3.5 Macroblock(MB) 7
2.3.6 Slice 8
2.4 Introduction to Error Concealment Methods 8
2.4.1 Error Propagation 8
2.4.2 Error Concealment 9
2.4.3 Spatial Error Concealment 10
2.4.3.1 Spatial Interpolation Algorithm 10
2.4.4 Temporal Error Concealment 12
2.4.4.1 Temporal Replacement (TR) 13
2.4.4.2 Average of Motion Vectors of Vertically Adjacent MB (AVG) 13
2.4.4.3 Object-Based Error Concealment Algorithm 15
Chapter 3 PROPOSED METHOD 17
3.1 The Efficient Object-Based Error Concealment Method 17
3.1.1 The Neighboring MVs Checking Process 18
3.1.2 The Reference Block in the Previous Frame 20
3.1.3 Corrupted Slice Recovering Process 21
3.2 The Enhanced AVG Method MB (AVE) 23
Chapter 4 SIMULATION RESULTS 28
4.1 PSNR Distribution with Different Methods 28
4.2 Proposed Method in B Frame Concealment 62
4.3 Proposed Method with Different Threshold 63
Chapter 5 CONCLUSIONS 65
REFERENCE 66
PUBLICATION LIST 69

LIST OF FIGURES
Fig. 2.1 The MPEG-2 encoding block diagram 4
Fig. 2.2 MPEG-2 decoding process 5
Fig. 2.3 Macroblock 4:2:0 format 8
Fig. 2.4 The size of Block and Macroblock 8
Fig. 2.5 The Error Propagation 9
Fig. 2.6 The Spatial Error Concealment 10
Fig. 2.7 The Spatial Interpolation Algorithm 11
Fig. 2.8 The process of the Temporal Error Concealment 12
Fig. 2.9 The process of the Temporal Replacement Method 13
Fig. 2.10 The AVG algorithm 14
Fig. 2.11 The Object-Based Error Concealment Algorithm 15
Fig. 3.1 The Macroblock(MB) checking order 18
Fig. 3.2 The MB above the corrupted slice 18
Fig. 3.3 The MB under the corrupted slice 19
Fig. 3.4 the checking process in the leftmost and rightmost MB 19
Fig. 3.5 The similarity between motion vectors 20
Fig. 3.6 The reference MB above the corrupted slice 21
Fig. 3.7 The reference MB under the corrupted slice 21
Fig. 3.8 The srearch region under the reference MB 22
Fig. 3.9 The search region above the reference MB 22
Fig. 3.10 Proposed block diagram of efficient Object-Based method 25
Fig. 3.11 Proposed block diagram of Enhanced AVG Algorithm (AVE) 26
Fig. 3.12 Flow chart of Proposed method 27
Fig. 4.1 Coastguard under 4% slice loss rate 29
Fig. 4.2 Coastguard under 8% slice loss rate 30
Fig. 4.3 Coastguard under 12% slice loss rate 31
Fig. 4.4 Frame 78 of the Coastguard sequence under 4% slice loss rate 32
Fig. 4.5 Mobile under 4% slice loss rate 33
Fig. 4.6 Mobile under 8 % slice loss rate 34
Fig. 4.7 Mobile under 12% slice loss rate 35
Fig. 4.8 Frame 72 of Mobile sequence under 4% slice loss rate 36
Fig. 4.9 Stefan under 4% slice loss rate 37
Fig. 4.10 Stefan under 8% slice loss rate 38
Fig. 4.11 Stefan under 12% slice loss rate 39
Fig. 4.12 Frame 50 of Stefan sequence under 4% slice loss rate 40
Fig. 4.13 Close-up view the recovered area in Stefan 41
Fig. 4.14 Akiyo under 4% slice loss rate 42
Fig. 4.15 Akiyo under 8% slice loss rate 43
Fig. 4.16 Akiyo under 12% slice loss rate 44
Fig. 4.17 Frame 77 of Akiyo sequence under 4% slice loss rate 45
Fig. 4.18 Frame 80 of Akiyo sequence under 4% slice loss rate 46
Fig. 4.19 Silent under 4% slice loss rate 47
Fig. 4.20 Silent under 8% slice loss rate 48
Fig. 4.21 Silent under 12% slice loss rate 49
Fig. 4.22 Frame 83 of Silent sequence under 4% slice loss rate 50
Fig. 4.23 Mother-Daughter under 4% slice loss rate 51
Fig. 4.24 Mother-Daughter under 8% slice loss rate 52
Fig. 4.25 Mother-Daughter under 12% slice loss rate 53
Fig. 4.26 Frame 56 of Mother-Daughter sequence under 4% slice loss rate 54
Fig. 4.27 Bus under 4% slice loss rate 55
Fig. 4.28 Bus under 8% slice loss rate 56
Fig. 4.29 Bus under 12% slice loss rate 57
Fig. 4.30 Frame 84 of Bus sequence under 4% slice loss rate 58
Fig. 4.31 Proposed method with different Bit rate in IBBPBBP structure 62
Fig. 4.32 Stefan sequence tested with different threshold 63
Fig. 4.33 Silent sequence tested with different threshold 63

LIST OF TABLES
Table-1 Average PSNR (dB) for test sequences under 4% slice loss rate 59
Table-2 Average PSNR (dB) for test sequences under 8% slice loss rate 60
Table-3 Average PSNR (dB) for test sequences under 12 % slice loss rate 61
Table-4 Computational time comparison (slice loss rate 8%)61

[1] ITU-T Rec. H.262 | ISO/IEC 13818-2, “Information technology — Generic coding of moving pictures and associated audio information: Systems”, Second edition ,2000-12-01
[2] http://dcmc.ee.ncku.edu.tw/pdf/course/Reports/MPEG-2_Video_System.pdf
[3] ISO/IEC JTC1/SC29/WG11, “MPEG-4 video verification model version 18.0,” N3908, January 2001.
[4] T. Wiegand, G. J. Sullivan, G. Bjontegaard, A. Luthra, “Overview of the H.264 / AVC Video Coding Standard,” IEEE Transactions on Circuits and Systems for Video Technology, Volume 13, Jul. 2003, pp. 560–576,
[5] ITU-T Recommendation H.261, Video Codec for audiovisual services at px64 kbit/s 1993.
[6] ISO/IEC 14496-10 and ITU-T Rec.H.264, Advanced Video Coding 2003.
[7] P. Frossard, O. Verscheure, “AMISP: a complete content-based MPEG-2 error-resilient scheme,” IEEE Transactions on Circuits and Systems for Video Technology, Volume 11, Issue 9, Sep. 2001, pp. 989-998.
[8] Y.C. Lee, Y. Altunbasak, R.M. Mersereau, “Multiframe error concealment for MPEG-coded video delivery over error-prone networks,” IEEE Transactions on Image Process. Nov 2002, pp.1314–1331.
[9] Y. Guo, Y. Chen, Y.K. Wang, H. Li, M.M Hannuksela, M. Gabbouj, “Error Resilient Coding and Error Concealment in Scalable Video Coding,” IEEE Transactions on Circuits and Systems for Video Technology, Volume 19, Issue 6, June 2009, pp. 781-795.
[10] Z. Wang, Y. L. Yu, D. Zhang, "Best neighborhood matching: An information loss restoration technique for block-based image coding systems," IEEE Transactions on Image Process, Volume 7, no. 7, Jul. 1998, pp. 1056-1061
[11] Z. Wang, J. Ming, B. Fan, “Fast Best Neighborhood Matching Algorithm for Intra Block Error Concealment in H.264/AVC,” Congress on Image and Signal Processing, Volume 1, May 2008, pp. 559-563.
[12] Y. Shi, X. Zhu, J. Xia, H. Yin, “A Fast and Efficient Spatial Error Concealment for Intra-coded Frames,” Congress on Image and Signal Processing, Volume 1, May 2008, pp.264-267
[13] J. W. Suh ,Y. S. Hu, “Error concealment based on directional interpolation,”
IEEE Transactions Consumer Electron, Volume 43, no. 4, Aug 1997, pp. 295–302
[14] J. Feng, K. Lo, H. Mehropour, A. E. Karbowiak, “Error concealment for MPEG video transmissions,” IEEE Transactions Consumer Electron, Volume 43, May 1997,pp. 183–187
[15] H. Sun, K. Challapali, J. Zdepski, “Error concealment in digital simulcast AD-HDTV decoder,” IEEE Transactions Consumer Electron, Volume 38, Aug. 1992, pp. 108–116
[16] W. Suh, Y. S. Ho, “Error concealment techniques for digital TV,” IEEE Transactions on Broadcasting, Volume 48, No. 4, pp. 299–306, Dec. 2002.
[17] J. Zhang, J. F. Arnold, M. R. Frater, “A cell-loss concealment technique for MPEG-2 coded video,” IEEE Transactions on Circuits and Systems for Video Technology, Volume 10, No. 4, June 2000, pp. 659–665
[18] C.H. Hong, “An object-base Motion Vector Recovery Strategy for H.264/AVC,” Institute of Computer & Communication, NCKU, 2007
[19] J.W. Suh, Y. S. Ho, “Motion vector recovery for error concealment,” SPIE Visual Communication and Image Processing, Jan 1999,pp. 667–676
[20] Y. Xu, Y. Zhou, “H.264 video communication based refined error concealment schemes,” IEEE Transactions Consumer Electron, vol. 50, Nov. 2004, no. 4, pp. 1135-1141
[21] S. Tsekeridou, I. Pitas, “MPEG-2 Error Concealment Based on Block-Matching Principles,” IEEE Transactions on Circuits and Systems for Video Technology ,Volume 10,NO.4 , June 2000
[22] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, “The error concealment feature in the H.26L test model, ” IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.
[23] S. Valente, C. Dufour, F. Groliere, D. Snook, “An efficient error concealment implementation for MPEG-4 video streams,” IEEE Transactions Consumer Electron. 47 (3) (2001) 568–578.
[24] Y.K. Wang, M.M. Hannuksela, V. Varsa, A. Hourunranta, M. Gabbouj, “The error concealment feature in the H.26L test model, ” IEEE International Conference on Image Processing, Rochester, NY, USA, 2002, pp. 729–732.
[25] E. Khan, S. Lehmann, H. Gunji, M. Ghanbari, “Iterative error detection and correction of H.263 coded video for wireless networks,” IEEE Transactions on Circuits and Systems for Video Technology, Volume 14, Issue 12, Dec. 2004, pp. 1294-1307.
[26] M.H. Jo, H.N. Kim, W.J. Song,, “Hybrid error concealments based on block content,” Image Processing, IET, Volume 1, Issue 2, June 2007,pp. 141 – 148
[27] J. Cao, F. Li, J. Guo, “Spatial and Temporal Adaptive Error Concealment Algorithm for MPEG-2,” CCECE'2004, Volume 1, May.2004, pp. 285-288
[28] Y. Chen, Y. Hu, O. Au, H. Li ,C.W. Chen, “Video error concealment using spatio-temporal boundary matching and partial differential equation,” IEEE Transactions on Multimedia, Volume. 10, No. 1, Jan. 2008.
[29] W.Y. Kung, C.S. Kim, C.C. Kuo, “Spatial and Temporal Error Concealment Techniques for Video Transmission Over Noisy Channels,” IEEE Transactions on Circuits and Systems for Video Technology, Volume.16, No.7, pp.789-802, July 2006
[30] L.W. Kang, J.J. Leou, “A new hybrid error concealment scheme for MPEG-2 video transmission,” IEEE Workshop on Multimedia Signal Processing, 9-11 Dec. 2002, pp. 29-32