跳到主要內容

臺灣博碩士論文加值系統

(3.237.38.244) 您好!臺灣時間:2021/07/26 11:02
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

我願授權國圖
: 
twitterline
研究生:許功炎
研究生(外文):Kun-Yen Hsu
論文名稱:應用於高畫質液晶顯示器之畫面插補演算法與硬體架構設計
論文名稱(外文):Frame Rate Up-Conversion Algorithm and Hardware Architecture Design for High Definition Liquid Crystal Display
指導教授:簡韶逸
學位類別:碩士
校院名稱:國立臺灣大學
系所名稱:電子工程學研究所
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2008
畢業學年度:97
語文別:英文
論文頁數:82
中文關鍵詞:畫面插補演算法
外文關鍵詞:Frame rate up-conversion
相關次數:
  • 被引用被引用:0
  • 點閱點閱:396
  • 評分評分:
  • 下載下載:0
  • 收藏至我的研究室書目清單書目收藏:0
畫面更新率的提升根據不同的應用是將原來的具有較低畫面更新率的視訊訊號提升到較高的畫面更新率. 其中一個例子就是液晶螢幕藉由提高畫面更新率來降低其本身存在的動態模糊問題。
因為液晶螢幕本身是屬於hold-type 的螢幕,亦即每一張畫面都會停留一個畫面的時間這與傳統的CRT 螢幕而有所不同。由於人眼對於移動物體存在著追蹤的與視覺暫留的特性,因此當移動物體在液晶螢幕上顯示時,動態模糊就會被感覺出來。經由研究發現,將液晶螢幕的畫面更新率由原本的60 赫茲提升到120 赫茲可以有效的解決動態模糊的問題。
畫面更新率提升主要是將新的畫面藉由已存在的畫面資料內插出來。其中主要的挑戰是在畫面中存在著複雜的運動時,其中包含攝影機的移動以及前景物體的移動。在文獻中探討中,有三類以區塊為基礎的(block-based)演算法來內插畫面。然而這三類方法只受限於畫面中的移動物體存在區域性的移動。有些演算法亦有討論到攝影機的各類移動,但是卻無法利用簡單的方法將攝影機移動參數準確的估測出來
在這篇論文中, 我們提出了一個名為“ Frame Rate Up-Conversion with Global-to-Local Iterative Motion Compensated Interpolation"的演算法。對於全域的補償內插,其全域參數會先被估測出來,而我們所利用的演算法稱為“Gradient Descent algorithm",而這種演算法可以將全域參數準確的估測出
來。在全域內插補償之後,畫面中屬於移動物體上的區塊會被區分出來。之後這些屬於移動物體的區塊會被所提出的區域性移動內插演算法給內插出來。而所提出的區域內插補償演算法主要可以分成兩個步驟。第一個步驟是將屬於移動物體內部的區塊給內插出來。然而在畫面中區塊與區塊之間是具有高度空間相關性的。因此在第二步驟就會利用已經補好的區塊將剩餘尚未內插出的區塊給內插出來。除此之外,畫面是利用多次性內插的方法給插補出來的。經由實驗發現,所提出的演算法在視覺感官上可以插補出較好的畫面。
對於畫面解析度為1920x1080 的高畫質液晶螢幕,我們所提出的演算法對於硬體實現上是過於複雜的。為了要降低其運算複雜度以及記憶體頻寬需求,在所提出的演算法中所需的記憶體頻寬首先會被分析,除此之外一些方法更被我們所提出,主要目的是希望可以設計出一個更符合硬體實現的演算法。首先在所提出的全域移動參數估測的演算法中,在“Gradient Descent algorithm"中多餘的運算會被去除掉。而在前一章畫面所被估測到的全域移動參數亦會被用來當作預測值,除此之外在畫面中只有部分的像素會被利用來做估測。利用所提出的方法可以將記憶體頻寬的需求量降到只需要原來的1/6,而且可以達到類似的效果。除此之外,畫面中的每個區塊會被分類成是屬於背景部分還是移動物體部分。在本篇論文硬體實作部分,由於所提出的硬體架構是針對1920x1080 的畫面大小去做設計,且畫面更新率可以由原來的60 赫茲提升到120 赫茲,因此高的記憶體頻寬以及高的運算量將是個挑戰。因此在所設計的硬體中應用了高度的硬體平
行化去做加速。在全域性參數估測的部分,以4 倍的硬體來加速。而在全域移動補償的部分,我們所設計的硬體可以達到16 倍的加速。除此之外,針對移動區塊內插的部分亦包含在所提出的硬體架構內.我們所提出的硬體是以Verilog-HDL 來實現並且以SYNOPSYS Design Compiler.來做合成。在所提出的硬體架構中,全部的閘數為1,303k,且總功率消耗在0.9V的供應電壓下為64.474mW. 除此之外,我們設計了第一顆利用高準度全域參數估測並且將畫面更新率由60 赫茲提升到120 赫茲的硬體。.
Frame rate up-conversion (FRUC) is applied for video sequences with lower frame rate and to converts the original frame rate to higher one according to different applications. One example application is motion blur reduction on liquid crystal displays (LCD).
Because LCD is a hold-type display, each frame will be displayed and hold in a period of time which is equal to one frame which is different from CRT. Since human eyes have the characteristic to pursuit moving objects with eye-tracking integration smoothly, the artifact named motion blur will be perceived by human eyes. Frame rate up conversion which converts the frame rate to 120Hz, can reduce the motion blur problem effectively.
Frame rate up conversion is to interpolate the new frames from the original video sequence. The challenge is to deal with the complex motion in a frame, including camera motion and motion of moving objects. There are three kinds of block-based motion compensated interpolating (BMCI) method for frame rate up conversion in the literature. However, the frame interpolation capabilities of these algorithms are only limited to locally translational motion. Some algorithms also consider the camera motion with rotation zoom-in/out can’t be estimated correctly with these simple methods.
In this thesis, a new algorithm “Frame Rate Up-Conversion with Global-to-Local Iterative Motion Compensated Interpolation” is proposed. For the global motion compensated interpolation (GMCI), the global motion parameters are estimated with Gradient Descent algorithm, which has high precision. In the next stage, the frame is interpolated with the estimated parameters. After GMCI, the moving blocks are masked and those masked block will be interpolated with BMCI. There are two stages in the proposed BMCI algorithm. In the first stage, those blocks in the moving object will be interpolated. After GMCI and the first stage, the number of rest moving blocks that have not been interpolated becomes small. Because of the high spatial correlation for moving blocks, the motion vectors (MVs) of these blocks are estimated with the predicted MVs belonging to the neighbored interpolated blocks in the second stage.
In addition, the frame is interpolated with iterative operations. Through experimental result, the proposed algorithm can provide better visual quality.
For high-definition of LCD with the resolution in 1920x1080, the proposed algorithm is too complex for VLSI design. In order to reduce the computation complexity and memory requirement, the memory bandwidth is first analyzed, and several techniques are the proposed to develop a more hardware friendly algorithm. First, in the proposed GME algorithm, the redundant computation of Gradient Descent Algorithm is avoided. The motion parameters of the previous frame are used to estimate global motion parameters with partial pixels and GME is done iteratively in the sub-sampled images. With these techniques, only 1/6 memory bandwidth is required to achieve the similar quality. Furthermore, moving flag masking and probability estimation technique is proposed to decide the interpolation method for each block in advance to reduce the iteration number of the proposed algorithm.
In the VLSI hardware design part of this thesis, because the target specification is for frame size 1920x1080 and from 60 fps to 120fps, high memory bandwidth and high complex computing are required. Highly parallel hardware is designed to execute the proposed algorithm in the limited number of cycles. The hardware is accelerated with four times in the proposed GME hardware architecture. In addition the size of local memory is also analyzed and the proposed GMCI hardware architecture can process 16 pixels in parallel in a cycle with the analysis of position for corresponding pixel in the reference frame. The moving block masking and motion compensated interpolation with two stages are also implemented in our proposed hardware architecture.
The hardware is implemented with Verilog-HDL and synthesized with SYNOPSYS Design Compiler. Faraday 90 nm cell library is adopted to design the hardware. The total gate count is 1,303 k and the usage of local memory size is 98.92kb. In addition, the power is 64.474mW with 0.9V supply voltage. It is the first hardware architecture for FRUC with GME based on Gradient Descent algorithm which estimates global motion parameters in high precision.
致謝……………………………………………………………………………….……i
中文摘要………………………………………………………………………………ii
Abstract ……………………………………………………………………….…......iv
Contents……………………………………………………………………………...vi
List of Figure…………………………………………………………………………ix
List of Table……………………………………………………………………….xii

1. Introduction…………………………………………………………………...….1
1.1 Application of Frame Rate Up Conversion…………………………………..1
1.2 Introduction of Motion Blur on LCD………………………………………...2
1.3 Motion Blur Reduction for LCD……………………………………………..4
1.4 Thesis Organization…………………………………………………………..5

2. Analysis of Block Based Frame Rate-Up Conversion Algorithm……………..6
2.1 Conventional MCI Algorithm………………………………………………..7
2.1.1 Motion Estimation Full Search………………………………………8
2.1.2 Overlapped Block Motion Estimation Full Search…………………..8
2.1.3 MV Assignment and Post-Processing………………………………10
2.1.4 Interpolating Method………………………………………………..11
2.2 Bilateral MCI Algorithm…………………………………………………....14
2.2.1 Bilateral ME………………………………………………………...14
2.2.2 Problem of Bilateral MCI…………………………………………...15
2.3 Optical Flow Based MCI Algorithm………………………………………..16
2.3.1 Problem of Optical Flow Based MCI……………………………….16
2.4 Experimental Result and Comparison………………………………………18
2.5 Summary……………………………………………………………………19

3. Global to Local Iteratively Motion Compensated
Interpolating Algorithm……………………………………………………...…20
3.1 FRUC with Global Motion Compensated Interpolation……………………20
3.1.1 Global Motion Estimation with Gradient Descent Algorithm……...21
3.1.2 Global Motion Compensated Interpolation…………………………23
3.1.3 Summary of GMCI…………………………………………………24
3.2 Proposed Hybrid Algorithm………………………………………………...25
3.2.1 Moving Block Classify……………………………………………..26
3.2.2 Bidirectional OBME………………………………………………..27
3.2.3 Good MV Classify………………………………………………….28
3.2.4 MCI with Neighbor high Correlated MVs………………………….29
3.2.5 Iterative Bidirectional Boundary Matching………………………...30
3.2.6 Operation with Smaller Block Size…………………………………31
3.3 Experimental Result………………………………………………………...31
3.4 Summary……………………………………………………………………34

4. Hardware Oriented Algorithm Design…………………………………...……35
4.1 Specification of Proposed Hardware for FRUC…………………………….35
4.2 Proposed Hardware Oriented Algorithm for FRUC………………………...36
4.3 Hardware Oriented GME…………………………………………...………37
4.3.1 Proposed GME Algorithm Based on Avoiding Redundant Computation Gradient Descent Algorithm………………………….38
4.4 Moving Block Classify……………………………………………………….40
4.4.1 Moving Flag Masking in Reference Frame………………………...41
4.4.2 Similar Probability Estimation in Interpolated Frame……………...42
4.4.3 Moving Block Classify……………………………………………..43
4.4.4 Moving Flag Smoothness…………………………………………...44
4.4.5 Comparison of Similar-Flag-Based and Warping-Frame-Based Algorithm…………………………………………………………...46
4.4.6 Analysis of Memory Bandwidth……………………………………46
4.5 Moving Block Interpolation Stage 1………………………………………..47
4.5.1 Bidirectional Checking and Searching……………………………...49
4.5.2 Header of MB……………………………………………………….50
4.5.3 BMCI of Stage 1……………………………………………………51
4.6 Moving Block Interpolation Stage 2………………………………………..52
4.6.1 MCI with Neighbor High Correlated MV…………………………..52
4.6.2 MCI with Boundary Matching…………………………………...…53
4.7 Analysis of Required Number of Cycle……………………………………...54
4.8 Simulation Result…………………………………………………………….55
4.9 Summary……………………………………………………………………55

5. Hardware Architecture Design and Implementation of FRUC……………...57
5.1 Overview of Proposed FRUC Hardware Architecture……………………...58
5.2 Hardware Architecture of Proposed GME…………………………………...59
5.2.1 3-Taps Filter and Sub-Sample………………………………………59
5.2.2 Initial Matching……………………………………………………..60
5.2.3 Hardware Architecture of Gradient Descent………………………..62
5.2.4 Reference Based Scheduling for Gradient Descent………………...63
5.2.5 Analysis of On-Chip Memory for GME on FRUC…………………64
5.2.6 Interleaved Memory Arrangement………………………………….65
5.2.7 Comparison of GME Hardware with Different Specification………………………………………………………...66
5.3 Hardware Architecture Design for GMCI……………………………………67
5.3.1 Access Multiple Reference Data……………………………………67
5.4 Hardware Architecture of Moving Block Classify…………………………...68
5.4.1 Reference Moving Flag Classify……………………………………68
5.4.2 Similar Probability Estimation for Interpolated Pixel………………69
5.4.3 Moving Block Masking……………………………………………..70
5.5 Hardware Architecture or Moving Block Interpolation Stage 1……………..71
5.5.1 Hardware architecture of Block Based Motion Estimation………...71
5.5.2 Bidirectional Checking……………………………………………...72
5.5.3 BMCI in MBI Stage 1………………………………………………73
5.6 Hardware Architecture of Moving Block Interpolation Stage 2……………..74
5.6.1 Boundary Matching…………………………………………………74
5.6.2 BMCI in MBI Stage 2………………………………………………75
5.7 Implementation Result……………………………………………………….75
5.8 Summary……………………………………………………………………..77
6. Conclusion and Future Work…………………………………………………..78
Reference…………………………………………………………………………….80
[1]A. -M. Hung and T. Nguyen, “A Novel Multi-Stage Motion Vector Processing Method For Motion Compensated Frame Interpolation,” in Proc. of IEEE Int. Conf. on Image Processing, vol. 5, pp. V389-V392, Sept. 2007.
[2]A. K. Michiel, “Temporal Impulse Response And Bandwidth of Display in Relation to Motion Blur,” in SID Symp. Digest, vol. 36, pp. 1578-1581, May 2005.
[3]A. K. Michiel, “Comparison of LCD Motion Blur Reduction Methods Using Temporal Impulse Response and MPRT,” in SID Symp. Digest, vol.37, pp. 1700-1703, June 2006.
[4]K. Taichiro, “Moving Picture Quality Improvement for Hold-type AM-LCDs,” in SID Symp. Digest, vol. 32, pp. 986-989, June 2001.
[5]Y. Kuroki, T. Nishi, K. Seiji, O. Hideki and Y. Shinichi, “Improvement of Motion Image Quality by High Frame Rate,” in SID Symp. Digest, vol. 37, pp. 14-17, June 2006.
[6]T. Chen, “Adaptive Temporal Interpolation Using Bidirectional Motion Estimation And Compensation,” in Proc. of IEEE Int. Conf. on Image Processing, vol. 2, pp. II313-II316, Sept. 2002.
[7]B.-D. Choi, J.-W. Han, C.-S. Kim and S.J Ko, “Motion-Compensated Frame Interpolation Using Bilateral Motion Estimation and Adaptive Overlapped Block Motion Compensation,” IEEE Trans. Circuits Syst. Video Technol., vol. 17, pp.407-416, Apr. 2007.
[8]B.-W. Jeon, G.-I. Lee, S.-H. Lee and R.-H. Park, “Coarse-to-Fine Frame Interpolation for Frame Rate Up-Conversion Using Pyramid Structure”, IEEE Trans. on Consumer Electro., vol. 49, pp. 499-508, Aug. 2003 .
[9]G. Dane and T. Q. Nguyen, “Smooth Motion Vector Resampling For Standard Compatible Video Post-Processing,” in Proc. of IEEE Int. Conf. on Signal, Syst. and Computer, vol. 2, pp. 1731-1735, Nov. 2004.
[10]J. Zhang, L. Sun, S. Yang and Y. Zhong, “Position Prediction Motion-Compensated Interpolation for Frame Rate Up-Conversion Using Temporal Modeling,” in Proc. of IEEE Int. Conf. on Image Processing, vol. 1, pp. I - 53-6, Sept. 2005.
[11]K. Sugiyama, T. Aoki and S. Hangai, “Motion Compensated Frame Rate Conversion Using Normalized Motion Estimation,” in Proc. of IEEE Workshop on Signal Processing Syst. Design and Implementation, pp. 663-558, Nov. 2005.
[12]B.-T. Choi, S.-H. Lee and S.-J. KO, “New Frame Rate Up-Conversion Using Bi-Directional Motion Estimation,” IEEE Trans. on Consumer Electro., vol. 46, pp.603-609, Aug. 2000.
[13]Y. -T. Yang, Y.-S. Tung and J.-L. Wu, “Quality Enhancement of Frame Rate Up-Converted Video by Adaptive Frame Skip and Reliable Motion Extraction,” IEEE Trans. on Circuit Syst. Video Technol., vol. 17, pp. 1700-1713, Dec. 2007.
[14]T.-S. Chong, O.C. Au, W.-S. Chao, T.W. Chan, “Multiple Objective Frame Rate UP Conversion,” in Proc. of IEEE Int. Conf. on Multimedia and Expo., pp.253-256, July 2005.
[15]Z. Jiefu, Y. K. Yu, L. Jiang and Shipeng Li, “A Low Complexity Motion Compensated Frame Interpolation Method,” IEEE Int. Symp. on Circuit and Syst., vol. 5, pp. 4927-930, May 2005.
[16]K. Hilman, H. W. Park and Y. Kim, “Using Motion-Compensated Frame-Rate Conversion for the Correction of 3:2 Pulldown Artifacts in Video Sequences,” IEEE Trans. on Circuit Syst. Video Technol., vol. 10, pp.869-877, Sept. 2000.
[17]S.-H. Lee, O. Kwon and R.-H. Park, “Weighted-Adaptive Motion-Compensated Frame Rate Up-Conversion,” IEEE Trans. on Consumer Electro., vol. 49, pp. 485 – 492, Aug. 2003 .
[18]S.-H. Lee, Y. C. Shin, S. Yang, H.-H. Moon and R.-H. Park, “Adaptive Motion-Compensated Interpolation For Frame Rate Up-Conversion,” IEEE Trans. on Consumer Electro., vol. 18, issue 3, pp. 444-450, Aug. 2002.
[19]F. Shinya and T. Akira, “Motion-Compensated Frame Rate Up-Conversion Based on Block Matching Algorithm with Multi-Size Blocks,” in Proc. of IEEE Int. Conf. on Intelligent Signal Processing and Communication Syst., pp. 353-356, Dec. 2005.
[20]K. Kunio and K. M. Sanjit, “Frame Rate Up-Conversion Considering Multiple Motion,” in Proc. of IEEE Int. Conf. on Image Processing, vol. 1, pp. 127-130, Oct. 1997.
[21]MPEG Video Group, The MPEG-4 Video Standard Verification Model Version 18.0, ISO/IEC JTC 1/SC 29/WG11 N3908, 2001.
[22]D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters,” Journal of the Society for Industrial and Applied Mathematics, vol.3, pp. 431-441, 1963.
[23]C.-Y. Chen, S.-Y. Chien, W.-M. Chao, Y.-W. Hung and L.-G. Chen, “Hardware Architecture for Global Motion Estimation for MPEG-4 Advance Simple Profile,” in Proc. of IEEE Int. Symp. on Circuit and Syst., vol.2, pp. 23-26, May 2004.
[24]H. Yeo and Y. H. Hu, “A Novel Modular Systolic Array Architecture for Full Search Block Matching Motion Estimation,” IEEE Trans. Circuit Syst. Video Technol., vol. 5, no. 5, pp. 407-416, Oct. 1995.
[25]C.-Y. Chen, Analysis and VLSI Architecture of Advanced Temporal Prediction Method in Video Coding Standards, Ph.D. thesis, National Taiwan Univ., May 2006.
[26]Y. W. Huang, Algorithm and Architecture Design for Motion Estimation, H.264/AVC Standard, and Intelligent Video Coding Signal Processing, Ph.D. thesis, National Taiwan Univ., Dec. 2004.
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top