臺灣博碩士論文加值系統

在這篇論文中提出了可應用於高解析畫質的可調視訊編碼器架構設計。現今視訊編碼標準的發展中，為了滿足各式各樣的網路環境、頻寬大小、視訊裝置的螢幕尺寸、運算能力等，在追求更高的壓縮率之外，更注重的是多樣化的功能性，因此可調視訊編碼(SVC, Scalable Video Coding)愈趨重要。最新的SVC標準是以H.264/MPEG-4 AVC為基礎再作功能性的延伸，同時兼顧了極高的壓縮率和豐富的功能性。
在無線通訊愈來愈成熟且網路多媒體愈來愈普及的今日，FGS (Fine Grain SNR Scalability)的技術愈顯得重要，它可以根據網路狀況而作動態調節，得到相對於頻寬大小的視訊品質，所以非常適合網路串流的應用。但因為要對整張畫面依重要性作為編碼順序的依據，作多次大規模且不規則的掃描編碼，FGS在實作上需要非常可觀的內部記憶體大小或外部記憶體頻寬，在高畫質的應用中，這個問題會隨著畫面變大而變得更加嚴重。我們提出了三種硬體導向的演算法，可以在FGS實作上節省99%的內部記憶體大小或92%的外部記憶體頻寬。此外，在我們的架構中大量採用FGS層之間的硬體共用，大大減少了硬體資源的使用。我們提出的高效能FGS架構設計可即時壓縮每秒30張的HDTV 1920x1080視訊畫面。
熵編碼(Entropy Coding)是根據機率特性對資料作無失真的壓縮，CABAC (Context-based Adaptive Binary Arithmetic Coding)是現今應用於視訊及影像標準中熵編碼的主流，但由於CABAC是以位元為單位的運算，且在位元與位元間有高度的相關性，所以常常為高畫質視訊編碼中的瓶頸所在。我們提出了全新高產出率的架構，且能針對不同需求作彈性的組合和調整，產出率可高達840 Msymbols/sec，為當今最高產出率的CABAC設計。
在這篇論文裡我們提出了第一個SVC標準的視訊品質可調編碼器，裡面採用了最高產出率的CABAC設計，且經由提出的方法可減少99%的內部記憶體大小或92%的外部記憶體頻寬，這個架構設計不只可適用於現今HDTV規格，還能支援未來更高規格的多媒體應用。

An efficient architecture design of high definition SNR scalable video encoder with ultra high throughput CABAC is presented in this thesis. In the recent development of video coding standards, the targets are not only compression efficiency, but also the functionality to meet various requirements of heterogeneous network environments, different display sizes and processing capability of user devices. Therefore, the concept of scalavle video coding (SVC) arises. The scalable extension of H.264/MPEG-4 AVC can provide good compression efficiency and several dimensions of functionalities.
Nowadays, wireless communication is mature and exists everywhere. Fine grain SNR (quality) scalability (FGS) is very important to give arbitrary quality levels according to the available bandwidth. The quality of service (QoS) can be greatly enhanced with FGS, but it introduces large internal memory size or external memory bandwidth in hardware implementation due to multiple frame-level operations. Several novel algorithms are proposed to achieve 99% reduction in internal memory size or 92% reduction in external memory bandwidth. Moreover, layer-wise hardware reuse for FGS layered coding reduces the silicon costs significantly. Our high-performance FGS design can real-time encode HDTV 1920$ imes$1080 sequences with 30 frames per second.
Entropy coding is to compress data based on their probability distribution. Context-based Adaptive Binary Arithmetic Coding (CABAC) is the mainstream of entropy coding in video coding standards. CABAC is usually the bottleneck in high-end video encoder because of its bit-serial processing and high data dependency. Two novel ultra high throughput architectures are proposed: the multi-symbol and ML-decomposed architecture. They are flexible and easily configurable for different requirements. The throughput of 840 Msymbols/sec can be attained. The proposed CABAC design achieves the highest throughput compared to all other works.
In this thesis, the first SNR scalable video encoder of H.264/MPEG-4 AVC scalable extension is implemented. It includes the highest throughput entropy coder up to now, and 92% of off-chip memory bandwidth or 99% of on-chip memory size can be reduced. Therefore, our design is not only for current HDTV applications, but also for much higher specifications in the near future.

1 Introduction 1
1.1 DevelopmentofVideoCoding . . . . . . . . . . . . . . . . . . 1
1.1.1 CompressionEfficiency . . . . . . . . . . . . . . . . . 2
1.1.2 Functionality Enhancement . . . . . . . . . . . . . . . 6
1.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3 DesignMotivation . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.1 FineGrainSNRScalability . . . . . . . . . . . . . . . 9
1.3.2 CABAC . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 ThesisOrganization . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Overview of H.264/MPEG-4 AVC and Scalable Video Coding 13
2.1 H.264/MPEG-4 AVC . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 InterPrediction . . . . . . . . . . . . . . . . . . . . . . 15
2.1.2 IntraPrediction . . . . . . . . . . . . . . . . . . . . . . 17
2.1.3 Transform, Quantization, and Picture Reconstruction . . 17
2.1.4 DeblockingFilter . . . . . . . . . . . . . . . . . . . . . 19
2.1.5 EntropyCoding . . . . . . . . . . . . . . . . . . . . . . 19
2.2 ScalableVideoCoding . . . . . . . . . . . . . . . . . . . . . . 20
2.2.1 TemporalScalability . . . . . . . . . . . . . . . . . . . 22
2.2.2 Spatial Scalability . . . . . . . . . . . . . . . . . . . . 22
2.2.3 SNR (Quality) Scalability . . . . . . . . . . . . . . . . 24
3 Architecture Design of Ultra High Throughput CABAC 27
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.1.1 Binarization . . . . . . . . . . . . . . . . . . . . . . . . 28
3.1.2 ContextModeling . . . . . . . . . . . . . . . . . . . . 30
3.1.3 BinaryArithmeticCoding . . . . . . . . . . . . . . . . 30
3.2 OverallArchitecture . . . . . . . . . . . . . . . . . . . . . . . 31
3.3 Proposed Architecture for Binary Arithmetic Encoder . . . . . . 34
3.3.1 Multi-symbol Architecture . . . . . . . . . . . . . . . . 34
3.3.2 SRAM-basedArchitecture . . . . . . . . . . . . . . . . 44
3.3.3 ML-decomposedArchitecture . . . . . . . . . . . . . . 57
4 Architecture Design of High Definition Fine Grain SNR Scalability 69
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.2 DesignChallenges . . . . . . . . . . . . . . . . . . . . . . . . 74
4.2.1 Highexternalmemorybandwidth . . . . . . . . . . . . 74
4.2.2 Frame-level irregular data access . . . . . . . . . . . . . 75
4.2.3 Highcomputation . . . . . . . . . . . . . . . . . . . . 75
4.3 Proposed Algorithms to Reduce External Memory Bandwidth . 75
4.3.1 Scan bucket algorithm . . . . . . . . . . . . . . . . . . 76
4.3.2 Early context formation with context reduction . . . . . 76
4.3.3 First scan pre-encoding . . . . . . . . . . . . . . . . . . 78
4.4 Proposed Architecture to Reduce Hardware Costs . . . . . . . . 78
4.4.1 Layer-folded reconstructionloop . . . . . . . . . . . . . 79
4.4.2 Scan bucket implementation . . . . . . . . . . . . . . . 79
4.4.3 Enhancement layer arithmetic coder . . . . . . . . . . . 81
4.5 ImplementationResults . . . . . . . . . . . . . . . . . . . . . . 83
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5 Conclusion 87

[1] Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s – Part2: Video, ISO/IEC 11172, 1993.
[2] Information Technology - Generic Coding of Moving Pictures and Associated Audio Information: Video, ISO/IEC 13818-2 and ITU-T Rec. H.262, 1996.
[3] Information Technology - Coding of Audio-Visual Objects - Part 2: Visual, ISO/IEC 14496-2, 1999.
[4] Joint Video Team, Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification, ITU-T Rec. H.264 and ISO/IEC 14496-10 AVC, May 2003.
[5] T. Wedi, “Motion compensation in H.264/AVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 577–586, July 2003.
[6] T. Wiegand and B. Girod, Multi-Frame Motion-Compensated Prediction for Video Transmission, 2002.
[7] T. Wiegand, X. Zhang, and B. Girod, “Long-term memory motioncompensated prediction,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 9, pp. 70–84, Feb. 1999.
[8] M. Flierl and B. Girod, “Generalized B pictures and the draft JVT/H.264 video compression standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 587–597, July 2003.
[9] M. Flierl, T. Wiegand, and B. Girod, “A locally optimal design algorithm for block-based multi-hypothesis motion-compensated prediction,” in Proc. of Data Compression Conf., 1998, pp. 239–248.
[10] H. S. Malvar, A. Hallapuro, M. Karczewicz, and Louis Kerosfsky, “Lowcomplexity transform and quantization in H.264/AVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 598–603, July 2003.
[11] P. List amd A. Joch, J. Lainema, G. Bjontegaard, and M. Karczewicz, “Adaptive deblocking filter,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 598–603, July 2003.
[12] S. Wenger, “H.264/AVC over IP,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 645–656, July 2003.
[13] T. Stockhammer, M. M. Hannuksela, and T. Wiegand, “H.264/AVC in wireless environments,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 657–673, July 2003.
[14] A. Joch, F. Kossentini, H. Schwarz, T. Wiegand, and G. J. Sullivan, “Performance comparison of video coding standards using lagragian coder control,” in Proc. of IEEE International Conference on Image Processing, 2002.
[15] H. Schwarz, D. Marpe, and T. Wiegand, “MCTF and scalability extension of H.264/AVC,” in Proceedings of PCS 2004, Dec. 2004.
[16] H. Schwarz, T. Hinz, H. Kirchhoffer, D. Marpe, and T. Wiegand, “Technical description of the HHI proposal for SVC CE1,” ISO/IEC JTC1/SC29/WG11, Oct. 2004.
[17] H. Schwarz, D. Marpe, and T.Wiegand, “Combined scalability support for the scalable extension of H.264/AVC,” in Proceedings of ICME 2005, July 2005.
[18] H. Schwarz and T. Wiegand, “Overview of the scalable H.264/MPEG4-AVC extension,” in Proceedings of IEEE International Conference on Image Processing (ICIP’06), Oct. 2006.
[19] G. J. Conklin, G. S. Greenbaum, K. O. Lillevold, A. F. Lippman, and Y. A. Reznik, “Video coding for streaming media delivery on the internet,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, pp. 269–281, Mar. 2001.
[20] D. Wu, Y. T. Hou, W. Zhu, Y. Q. Zhang, and J. M. Peha, “Streaming video over the internet: Approaches and directions,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, pp. 282–300, Mar. 2001.
[21] Weiping Li, “Overview of fine granularity scalability in MPEG-4 video standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, no. 3, pp. 301–317, Mar. 2001.
[22] W. Li, F. Ling, and X. Chen, “Fine granularity scalability in MPEG-4 for streaming video,” in Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS), 2000, pp. 299–302.
[23] ISO/IEC JTC1/SC29/WG1 N1855, JPEG 2000 Part I: Final Draft International Standard (ISO/IEC FDIS15444-1), Aug. 2000.
[24] ISO/IEC JTC1/SC29/WG1, JBIG 2 Final Draft Standard, July 1999.
[25] Video Coding for Low Bit Rate Communication, ITU-T Rec. H.263, 1998.
[26] A. Joch, F. Kossentini, H. Schwarz, T.Wiegand, and G. J. Sullivan, “Performance comparison of video coding standards using Lagragian coder control,” in Proceedings of IEEE International Conference on Image Processing (ICIP’02), 2002, pp. 501–504.
[27] T. Wedi and H.G. Musmann, “Motion- and aliasing-compensated prediction for hybrid video coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, pp. 577–587, July 2003.
[28] D. Marpe, H. Schwarz, and T. Wiegand, “Context-based adaptive binary arithmetic coding in the H.264/AVC video compression standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 620–644, July 2003.
[29] ISO/IEC JTC1, “Joint Draft 8 of SVC Amendment,” ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, Doc. JVT-U201, Oct. 2006.
[30] ISO/IEC JTC1, “Joint Scalable Video Model 8.0,” ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6, Doc. JVT-U202, Oct. 2006.
[31] Heiko Schwarz, Detlev Marpe, and Thomas Wiegand, “Hierarchical B pictures,” Joint Video Team, Doc. JVT-P014, July 2005.
[32] Heiko Schwarz, Detlev Marpe, and Thomas Wiegand, “Analysis of hierarchical B pictures and MCTF,” in Proc. of IEEE International Conference on Multimedia and Expo, 2006, pp. 1929–1932.
[33] E. Francois and J. Vieron, “Extended spatial scalability: a generalization of spatial scalability for non-dyadic configurations,” in Proceedings of ICIP 2006, Oct. 2006.
[34] H. Schwarz, T. Hinz, D. Marpe, and T. Wiegand, “Constrained inter-layer prediction for single-loop decoding in spatial scalability,” in Proceedings of ICIP 2005, Sept. 2005.
[35] ISO/IEC 14 496-2/FPDAM4, “Coding of Audio-Visual Objects, Part-2 Visual, Amendment 4: Streaming Video Profile,” ISO/IEC 14 496-2/FPDAM4, July 2000.
[36] Y.-W. Chang, H.-C. Fang, and L.-G. Chen, “High performance two-symbol arithmetic encoder in JPEG 2000,” in Proc. of ISCE, 2004.
[37] R.-R. Osorio and J.-D. Bruguera, “Arithmetic coding architecture for H.264/AVC CABAC compression system,” in Proc. of DSD, 2004.
[38] J. L. Nunez and V. A. Chouliaras, “High-performance arithmetic coding vlsi macro for the h.264 video compression standard,” IEEE Transactions on Consumer Electronics, vol. 51, no. 1, pp. 144–152, Feb. 2005.
[39] H. Shojania and S. Sudharsanan, “A high performance cabac encoder,” in Proc. of NEWCAS, 2005.
[40] Y.-J. Chen, C.-H. Tsai, , and L.-G. Chen, “Analysis and architecture design for multi-symbol arithmetic encoder in H.264/AVC,” in VLSI Design/CAD Symposium, Aug. 2005.
[41] M. Dyer, D. Taubman, and S. Nooshabadi, “Improved throughput arithmetic coder for JPEG2000,” in Proc. of ICIP, 2004.
[42] G. Pastuszak, “A high-performance architecture of arithmetic coder in JPEG2000,” in Proc. of ICME, 2004.
[43] O. Flordal, D. Wu, and D. Liu, “Accelerating CABAC encoding for multistandard media with configurability,” in Proc. of IPDPS, 2006.
[44] L. Li, Y. Song, T. Ikenaga, and S. Goto, “Hardware architecture design of cabac codec for H.264/AVC,” in Proceedings of International Symposium on VLSI Design, Automation, and Test (VLSI-DAT), 2007, pp. 248–251.
[45] J.-L. Chen, Y.-K. Lin, and T.-S. Chang, “A low cost context adaptive arithmetic coder for H.264/MPEG-4 AVC video coding,” in Proceedings of ICASSP, 2007.
[46] P.-S. Liu, J.-W. Chen, and Y.-L. Lin, “A hardwired context-based adaptive binary arithmetic encoder for H.264 advanced video coding,” in Proceedings of International Symposium on VLSI Design, Automation, and Test (VLSI-DAT), 2007, pp. 180–183.
[47] J.-L. Nunez-Yanez, V.-A. Chouliaras, D. Alfonso, and F.-S. Rovati, “Hardware assisted rate distortion optimization with embedded CABAC accelerator for the h.264 advanced video codec,” in IEEE Trans. on Consumer Electronics, 2006.
[48] R. R. Osorio and J. D. Bruguera, “High-throughput architecture for H.264/AVC CABAC compression system,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 11, pp. 1376–1384, Nov. 2006.
[49] Y.-J. Chen, C.-H. Tsai, and L.-G. Chen, “Novel configurable architecture of ML-decomposed binary arithmetic encoder for multimedia applications,” in Proc. of IEEE International Symposium on VLSI Design, Automation and Test, 2007.