臺灣博碩士論文加值系統

隨著即時數位視訊傳輸在無線通訊上之日益普遍，且大多是在低位
元率的條件下做傳輸，因此無法避免在傳輸過程中，發生資料位元流錯誤或是封包的遺失，因而造成影像的損毀或是遺失，因此視訊系統中的錯誤隱藏便成為重要的課題並且值得我們去做深入的研究與改進。
本篇論文中，我們根據H.264/AVC編解碼系統，提出以整個畫面遺失為基準的錯誤隱藏(Frame Based Error concealment)，除了使用前一個畫面相同位置和周圍四個的移動向量來預測外，我們還在原本沒有移動向量的intra macro-block(I-MB)預測出適當的移動向量來做更佳化的錯誤隱藏。除此之外，在低於我們所設臨界值的低移動向量預測我們還使用H-S光流法來加強畫面品質!
在我們的實驗結果中，無論是主觀或是客觀的比較下，我們的演算
法都可以達到改良的目的，在客觀的比較下，PSNR最多可以達到2dB的改進，而在主觀的比較下，也可以看到畫面的重建是比JM的方法還要好。

Nowadays, real-time video transmission by wireless protocol usually transmits data in low bit-rate environment. So the data loss is inevitable in
this process with relatively high bit error rate and packet loss rate. Since we transmit data at a low-bit rate, a packet loss often results in a frame loss.
Therefore, frame based error concealment (FBEC) of video decoder systems is increasingly important and worthy of research.
In this thesis, we propose a new method to resist whole frame loss. The main idea is to predict suitable motion vectors (MVs) in zero motion inter-macro blocks (PMBs) and intra-macro blocks (IMBs). For low motion MBs, we apply optical flow method to predict suitable motion vector. Other MBs are predicted by neighbor macro-blocks of the previous frames. By applying the proposed algorithm to H.264 decoder, the objective performance measured in PSNR is around 2dB higher than that of reference software
JM10.0. The subjective quality also improves significantly. This method requires only a few extra computing components, and may be easily implemented in hardware.

Contents
摘要 .......... IV
Abstract ............... V
Contents ........... VII
List of Tables .......... IX
List of Figures ........ X
Chapter 1 Introduction ........ 1
1.1 Motivation .......................................................................................................... 1
1.2 Organization of Thesis ....................................................................................... 5
Chapter 2 H.264/AVC Standard and Error Resilience Techniques ............................... 6
2.1 Background of H.264/AVC ................................................................................ 6
2.2 Error resilience tools in H.264/AVC ................................................................ 10
Chapter 3 Frame Based Error Concealment in H.264/AVC ........................................ 18
3. 1 Introduction .................................................................................................... 18
3.2 Macro Block Level Error Concealments ......................................................... 19
3.2.1 Spatial Error Concealment Algorithm ................................................. 19
3.2.2 Temporal Error Concealment Algorithm ............................................. 20
3.3 Frame Level Error Concealments ................................................................. 21
3.3.1 Frame Copy Algorithm ......................................................................... 22
3.3.2 Motion Vector Copy Algorithm ............................................................. 22
3.4 FBEC by Refined Motion Vector Prediction ................................................ 23
3.4.1 Medium5 + DC + ZMMed8 Method ..................................................... 24
3.4.2 Medium5 + DC Method ........................................................................ 24
3.4.3 ZMMed8 Method .................................................................................. 27
3.5 Experiment Results and Discussion ................................................................ 29
3.5.1 Experiments of Proposed Error Concealment Schemes ...................... 29
3.5.2 Comparisons of JM and Proposed Method result ................................ 32
Chapter 4 Refined Low Motion Vector Prediction using Optical Flow
Method .......................................................................................................... 36
4.1 Refined Low Motion Method (RLMM) .......................................................... 36
4.2 Introduction to Optical Flow Equation ........................................................... 37
4.3 Optical Flow Medthod in Block Domain (OFb).............................................. 41
4.4 Optical Flow Method in Pixel Domain (OFp) ................................................. 45
4.5 Experiment Results and Discussion ............................................................... 47
4.5.1 Experiments of Proposed Error Concealment Schemes ...................... 47
VIII
4.5.2 Comparisons of JM and Proposed Method result ................................ 51
Chapter 5 Conclusions and Future Works ................................................................... 60
5.1 Conclusions ...................................................................................................... 60
5.2 Future Works ................................................................................................... 61
References ...................................................................................................................... 62

[1] D.J. Le Gall, “The MPEG video compression algorithm,” Signal Processing: Image Communication, Vol. 4, pp. 129-140, 1992.
[2] ISO/IEC JTC1/SC29/WG11, “MPEG-4 Video Verification Model version 18.0,”N3908, January 2001.
[3] Draft ITU-T Recommendation H.261: Video Codec for Audiovisual Services at Px64 Kits, ITU, 1993.
[4] Draft ITU-T Recommendation H.263: Video Coding for Low Bit-rate Communication, ITU, May 1993.
[5] “Draft ITU-T recommendation and final draft international standard of joint video specification,” in Joint Video Team of ISOIEC MPEG ITUT VCEG, JVTG050, 2003.
[6] T. Wiegand, G. Sullivan and Ajay Luthra, “Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264 |ISO/IEC 14496-10 AVC),” Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T
VCEG, JVT G050r1, Geneva, Switzerland, May 2003.
[7] T. Wiegand and G. Sullivan “Overview of the H.264/AVC Video Coding Standard” IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, pp.560-576, July 2003.
[8] Jinghong Zheng, Lap-Pui Chau,” An efficient spatial domain error concealment method for H.264 video”, Proc. of International Conference on Communications and Signal Processing ,Vol. 1,pp.26 – 30, Dec. 2003.
[9] Y.Wang, M. M. Hannuksela,V.Varsa, A. Hourunranta, and M.Gabbouj, “The error concealment feature in the H.26L test model,” in Proc. Int Conf. Image Processing, vol. 2,sep. 2002, pp. 729-73 63
[10] Jinghong Zheng, Lap-Pui Chau,” Efficient motion vector recovery algorithm for H.264 based on a polynomial model”, IEEE Transactions on Multimedia, Vol. 7, no.
3, pp. 507 – 513, June 2005.
[11] Saurav Bandyopadhyay, Zhenyu Wuj, Purvin Pandit, Jill Boyce, “ Frame Loss Error Concealment for H.264/AVC.” , Joint Video Team of ISO/IEC MPEG & ITU-T VCEG.JVT-P072, July,2005.
[12] Stefano Belfiore, Marco Grangetto, Member, IEEE, Enrico Magli, Member, IEEE, and Gabriella Olmo, Member, IEEE＂Concealment of Whole-Frame Losses for Wireless Low Bit-Rate Video Based on Multiframe Optical Flow Estimation”, IEEE Transaction on Multimedia, VOL. 7, NO. 2, APRIL 2005
[13] P. Baccichet, D. Bagni, A. Chimienti, L.Pezzoni, F.S.Rovati ,＂ Frame concealment for H.264/AVC decoders＂, IEEE Transactions on Consumer Electronics, Vol. 51, no. 1,pp. 227 - 233 , Feb. 2005 .
[14] P. Baccichet, D. Bagni, A. Chimienti, L. Pezzoni, F.S. Rovati, “Frame concealment for H.264/AVC decoders, Digest of Technical Papers. International Conference on Consumer Electronics, pp. 329 –330, Jan. 2005.
[15] “Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC),” in Joint Video
Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVTG050r1, 2003.
[16] S. Kumar, L. Xu, M. K. Mandal, and S. Panchanathan, “Error resiliency schemes in H.264/AVC video coding standard,” Journal of Visual Communication and Image
Representation, vol. 17, no. 2, pp. 425–450, 2006,
[17] S. Wenger, and T. Stockhammer, “An overview on the H.26L NAL concept,” in JVT-B028, Feb 2002.
[18] S. Wenger, “H.264/AVC over IP,” IEEE Trans. Cir. Syst. Video Technol., vol. 13, 64 pp. 645–656, July 2003.
[19] S. Wenger and M. Horowitz, “FMO: flexible macroblock ordering,” in JVT-C089, May 2002.
[20] “Video coding for low bit rate communication annex N,” ITU-T, ITU-T Recommendation H.263 Ver.1, 1995
[21] S. Wenger and T. Stockhammer, “H.26L over IP and H.324 Framework,” ITU-T SG16 Doc. VCEGN52, 2001
[22] T. Stockhammer, M. M. Hannuksela, and T. Wiegand, “H.264/AVC in wireless environments,” IEEE Trans. Circuits Syst. Video Technol., vol.13, pp. 657–673, July
2003.
[23] Jinghong Zheng, Lap-Pui Chau,” An efficient spatial domain error concealment method for H.264 video”, Proc. of International Conference on Communications and Signal Processing ,Vol. 1,pp. 26 – 30, Dec. 2003.
[24] Y.Wang, M. M. Hannuksela,V.Varsa, A. Hourunranta, and M. Gabbouj, “The error concealment feature in the H.26L test model,” in Proc. Int Conf. Image Processing, vol. 2,sep. 2002, pp. 729-73
[25] Jinghong Zheng, Lap-Pui Chau,” Efficient motion vector recovery algorithm for H.264 based on a polynomial model”, IEEE Transactions on Multimedia, Vol. 7, no.
3, pp. 507 – 513, June 2005.
[26] Nasiopoulos, P.; Coria-Mendozal, L.; Mansour, H.; Golikeri, A. “An improved error concealment algorithm for intra-frames in H.264/AVC”, IEEE International Symposium on ISCAS 2005, Vol. 1, pp.320 – 323, 23-26 May 2005.
[27] Yan Zhao; Dong Tian; Hannukasela, M.M.; Gabbouj, M.,”Spatial error concealment based on directional decision and intra prediction”, IEEE International Symposium
on ISCAS 2005, Vol. 3, pp. 2899 – 2902, 23-26 May 2005.
65
[28] Haiying Wang and Jing Lv, “A novel error concealment scheme for intra frames of H.264 video”, IEEE international Workshop on VLSI Design and Video Tech., pp.
300 – 303, 28-30 May 2005.
[29] Nemethova, O.; Al-Moghrabi, A.; Rupp, M., “Flexible Error Concealment for H.264 Based on Directional Interpolation”, International Conference on Wireless
Networks, Communications and Mobile Computing, Vol.2, pp.1255 – 1260, 13-16 June 2005.
[30] Bongsoo Jung; Byeungwoo Jeon; Myung-Don Kim; Bongsue Suh; Song-In Choi, ”Selective temporal error concealment algorithm for H.264/AVC”, 2004 IEEE International Conference on Multimedia and Expo, Vol.1, pp.411 – 414,
27-30 June, 2004.
[31] Pei-Jun Lee; Chen, H.H.; Liang-Gee Chen, “A new error concealment algorithm for H.264 video transmission”, Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, pp.619 – 622, 20-22 Oct. 2004.
[32] Jinghong Zheng; Lap-Pui Chau, ”A temporal error concealment algorithm for H.264 using Lagrange interpolation”, Proceedings of the 2004 International
Symposium on Circuits and Systems, Vol.2, pp.II - 133-6, 23-26 May 2004.
[33] Jinghong Zheng; Lap-Pui Chau, “A temporal error concealment algorithm for H.264 based on plane estimation”, Proceedings of the 2003 Joint Conference of the Fourth International Conference on Information, Communications and Signal Processing, Vol.1, pp.253 – 257, 15-18 Dec. 2003.
[34] Donghyung Kim; Siyoung Yang; Jechang Jeong, “A new temporal error concealment method for H.264 using adaptive block sizes” , IEEE International Conference on Image Processing, Vol. 3, pp.III - 928-31, 11-14 Sept. 2005.
66
[35] Mei-Juan Chen; Wen-Wei Liao; Ming-Chieh Chi, “Robust temporal error concealment for H.264 video decoder”, 2006 Digest of Technical Papers. International Conference on Consumer Electronics, pp.383 – 384, 7-11 Jan. 2006.
[36] Yanling Xu; Yuanhua Zhou, “H.264 video communication based refined error concealment schemes” , IEEE Transactions on Consumer Electronics, Vol.50, Issue
4, pp.1135 – 1141, Nov. 2004.
[37] Myung-Don Kim; Song-In Choi; Sung-Woong Ra, “Hybrid error concealment method for H.264/AVC”, The 7th International Conference on Advanced Communication Technology, Vol.1, pp.408 – 411, 21-23 Feb. 2005.
[38] Agrafiotis, D.; Bull, D.R.; Canagarajah, C.N., “Enhanced Error Concealment With Mode Selection”, IEEE Transactions on Circuits and Systems for Video Technology,
pp.960 – 973, Vol.16, Aug. 2006.
[39] Y. Wang and Q. -F. Zhu, “Error control and concealment for video communication: a review,” Proc. IEEE, Vol. 86(5), pp. 974-997, 1998.
[40] Hsin-Ju Feng and Chih-Hung Kuo, “Frame based Error concealment in H.264/AVC by refined motion prediction,” IEEE Asia Pacific Conference on Circuit and Systems, Singapore, Dec 4-7, 2006, pp 247-250.
[41] G. Roth, R. Sjober, G. Liebl, T. Stockhammer, V. Varsa and M. Karczewicz,“Common test conditions for RTP/IP over 3GPP/3GPP2,” ITU-T SG16 Doc, VCEG-N80, Santa Barbara, CA, USA, September 2001.
[42] A. Tekalp, Digital Video Processing, Prentice Hall, 1995.
[43] B.K.P. Horn and B.G. Schunck, “Determining optical flow,” Artificial Intelligence,
vol. 17, pp. 185-203, 1981.
[44] Belfiore, S.; Grangetto, M.; Magli, E.; Olmo, G., “Concealment of whole-frame losses for wireless low bit-rate video based on multiframe optical flow estimation”,
67 IEEE Transactions on Multimedia, Vol. 7, Issue 2, pp.316 – 329, Apr 2005.
[45] Baccichet, P.; Bagni, D.; Chimienti, A.; Pezzoni, L.; Rovati, F.S.,” Frame
concealment for H.264/AVC decoders”, IEEE Transactions on Consumer Electronics, Vol. 51, Issue 1, pp.227 – 233, Feb. 2005.
[46] Jae-Won Suh; Yo-Sung Ho, “Motion vector recovery using optical flow”, International Conference on Consumer Electronics, 2000. ICCE. pp. 234 – 235, 13-15 June 2000.
[47] Jae-Won Suh; Yo-Sung Ho, “Error concealment technique based on optical flow”, Electronics Letters, Vol. 38, Issue 18, pp.1020 – 1021, 29 Aug. 2002.
[48] Donghyung Kim; Sanghyup Cho; Jechang Jeong, “ A Motion Vector Recovery Algorithm for Temporal Error Concealment using Optical Flow in H.264 Video Coding”, 2006 IEEE International Conference on Multimedia and Expo, pp. 1713 – 1716, July 2006.
[49] Zhi-Heng Zhou; Sheng-Li Xie, “ Error concealment based on robust optical flow”, 2005 International Conference on Communications, Circuits and Systems. Vol.
1, pp.547 – 550, 27-30 May 2005