跳到主要內容

臺灣博碩士論文加值系統

(18.97.9.172) 您好!臺灣時間:2025/02/14 02:50
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:黃瀚致
研究生(外文):Huang, Han-Chih
論文名稱:應用於4K Ultra-HD 3D電視之相位式視點合成器設計
論文名稱(外文):VLSI System Implementation of Phase-Based View Synthesis for 4K Ultra-HD 3DTV
指導教授:黃朝宗黃朝宗引用關係
指導教授(外文):Huang, Chao-Tsung
口試委員:張添烜邱瀞德
口試委員(外文):Chang, Tian-SheuanChiu, Ching-Te
口試日期:2018-10-02
學位類別:碩士
校院名稱:國立清華大學
系所名稱:電機工程學系
學門:工程學門
學類:電資工程學類
論文種類:學術論文
論文出版年:2018
畢業學年度:107
語文別:英文
論文頁數:48
中文關鍵詞:視點合成超大型積體電路設計
外文關鍵詞:View synthesisVLSI
相關次數:
  • 被引用被引用:0
  • 點閱點閱:297
  • 評分評分:
  • 下載下載:1
  • 收藏至我的研究室書目清單書目收藏:0
裸視3D 電視是顯示器發展上的一種新趨勢,可以提供觀看者3D 體驗感,
而不需配戴3D 眼鏡。隨著近幾年人類對於螢幕解析度的需求逐漸提升,對於
4K 高解析度3D 顯示器的要求也逐漸提高,然而,受限於有限的硬體設備,我
們無法提供場景中的每個視點給裸視3D 電視,為了解決這個問題,視點合成
技術利用了已存在的兩個視點合成出其餘不存在的虛擬視點。傳統的視點合成
作法利用了深度圖合成虛擬視點,但深度圖往往耗費相當大的計算量才能獲取。
因此,我們提出了一個相位式的視點合成系統,並提出了一個修正不同頻帶中
相位移動的做法來提升合成出視點之品質,並讓整個系統能容忍低解析度之輸
入深度圖,如此一來,我們能在較低的解析度進行深度估算來節省整體地運算
時間。為了達到高畫質以及高解析度的需求,我們提出了一個相位式視點合成
引擎來達成即時運算的需求。
我們的目標是針對4K 高畫質3D 電視提供一個即時的視點合成引擎,在演
算法部分,主要將訊號分解為小波後,經由投影、重建之後來得到合成的視點,
在設計上的挑戰主要在於高吞吐量的小波,對此我們提供了一個高度平行的小
波重建引擎,其內部主要由快速傅立葉轉換(FFT) 的模組而構成,為了減少
FFT 在硬體中所佔的邏輯,我們實作了一個以SRAM 為基底的單一路徑延遲
回溯(Single-path Delay Feedback) 架構,而這個架構的優點是速度快,且所需
的記憶體較少。
我們在台積電40 奈米製程底下合成了我們所設計的相位式視點合成引擎,
此電路總共使用了250 萬的邏輯閘以及228KB 的記憶體,為了達到30fps 的
UHD 顯示器,我們的電路跑在200MHz,在此頻率底下合成時能提供吞吐量達
266Mpixel/s;在FPGA 實作部分,我們最快能跑在80MHz,並提供每秒12 張
的高畫質影片。
Multi-view autostereoscopic 3D TV is a new trend of display devices to provide users an immersive 3D experience without wearing glasses. Owing to the demand of increasing video resolution, 4K Ultra-HD (UHD) 3D display is required in these years. However, we are not able to capture every viewpoint of a scene due to the limited facilities. View synthesis is one of the important techniques to conquer this challenge by interpolating multi-view content from input two-view videos. Traditional methods such as DIBR render novel views according to accurate dense
depth maps which usually require heavy computational complexity. In this thesis, we present a phase-based view synthesis system with a cross-band shift correction scheme that enables the usage of low-resolution disparity maps . The algorithm becomes more robust as it is not sensitive to the resolution of disparity maps.
To achieve the demand for high-resolution and high-quality videos, we designed a VLSI circuit that applies the phase-based view synthesis to support the real-time application.

We aim to introduce a view synthesis engine for UHD 3DTV in real-time VLSI design. In the algorithm, novel views are generated with wavelet-based decomposition, re-projection, and reconstruction. The design challenge for the wavelet computation is the demand of large wavelet throughput. To address this issue, we parallelize the reconstruction engine which is mostly composed of the
FFT engine in our design. To reduce the gate count of an FFT engine, we adpot the SRAM-based Single Path Delay Feedback structure to implement it because this structure has the advantage of high speed and requires less area.

We implemented a VLSI circuit for phase-based view synthesis using TSMC 40nm technology process with 2M gate counts and 228 KB on-chip memory. It runs at clock frequency 200MHz and delivers a throughput of 266Mpixel/s, which is equal to 30 fps for UHD display. We also implemented the algorithm on FPGA at 80MHz. It achieves near real-time performance that provides UHD display at
12 fps.
摘 要......................................i
Abstract...................................ii
Contents..................................iv
List of Figures.............................vi
List of Tables.............................viii

Chapter 1 Introduction 1
1.1 Motivation ............................. 1
1.2 Thesis Organization.............................3
1.3 Contribution .............................3
Chapter 2 Related work 5
2.1 Pixel-based Rendering ............................. 6
2.1.1 Depth-Image-based Rendering(DIBR) ............................. 6
2.1.2 Image Domain Warping(IDW)............................. 6
2.2 Phase-based Motion Processing .............................7
2.2.1 Phase-based Motion Magnification .............................7
2.2.2 Phase-based Frame Interpolation .............................7
2.3 Phase-based View Synthesis .............................9
2.3.1 Phase-based Joint View Expansion .............................9
2.3.2 Eulerian-Lagrangian Stereo-to-Multiview Conversion .............................11
Chapter 3 Algorithm Analysis of Phase-Based View Synthesis 15
3.1 Overview to Eulerian-Lagrangian Stereo-to-Multi-view Conversion . 15
3.1.1 Wavelet Decomposition and Reconstruction .............................16
3.1.2 Wavelet Disparity Refinement ............................. 17
3.1.3 Novel View Synthesis ............................. 18
3.2 Algorithm of Phase-Based View Synthesis for Hardware Implementation
.............................19
3.2.1 Cross-band Shift Correction.............................19
3.2.2 Hardware-Friendly Novel View Synthesis.............................21
3.2.3 Analysis of Pyramid Level Number.............................22
3.2.4 Quality Analysis of Initial Disparity............................. 23
3.3 Quality Comparison to VSRS .............................23
Chapter 4 VLSI Design of Phase-Based View Synthesis.............................27
4.1 System architecture of Phase-Based View Synthesis ............................. 27
4.1.1 Two-Stage Design of Wavelet Pyramid .............................
4.1.2 Row-Based Pipeline Architecture .............................. 28
4.1.3 Wavelet Decomposition Buffer .............................30
4.2 Architecture Design of Wavelet Pyramid .............................30
4.2.1 SRAM-Based Single Path Delay Feedback Architecture .............................32
4.2.2 Decomposition Engine.............................35
4.2.3 Reconstruction Engine ............................. 36
4.2.4 Precision Analysis.............................37
Chapter 5 Implementation Result 41
5.1 TSMC 40 nm Synthesis Result.............................41
5.2 FPGA Implementation Result ............................. 43
Chapter 6 Conclusion and Future Work 47
[1] P. Kellnhofer, P. Didyk, S. P. Wang, P. Sitthi-Amorn, W. T. Freeman,
F. Durand, and W. Matusik, “3DTV at home: eulerian-lagrangian stereo-tomultiview
conversion,” ACM Transactions on Graphics (TOG), vol. 36, no.
4, pp. 146, 2017.
[2] J. Konrad and M. Halle, “3-D displays and signal processing,” IEEE Signal
Processing Magazine, vol. 24, no. 6, pp. 97–111, Nov 2007.
[3] M. Schaffner, P. Greisen, S. Heinzle, F. K. Gürkaynak, H. Kaeslin, and
A. Smolic, “Madmax: A 1080p stereo-to-multiview rendering asic in 65 nm
CMOS based on image domain warping,” in Proceedings of the ESSCIRC
(ESSCIRC), Sept 2013, pp. 61–64.
[4] P. Kauff, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, A. Smolic, and
R. Tanger, “Depth map creation and image-based rendering for advanced
3DTV services providing interoperability and scalability,” Signal Processing:
Image Communication, vol. 22, no. 2, pp. 217–234, 2007.
[5] C. Riechert, F. Zilly, P. Kauff, J. Güther, and R. Schäfer, “Fully automatic
stereo-to-multiview conversion in autostereoscopic displays,” in Proc. IBC,
2012, pp. 8–14.
[6] Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization
via graph cuts,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 23, no. 11, pp. 1222–1239, Nov 2001.
[7] J. Sun, N. Zheng, and H. Shum, “Stereo matching using belief propagation,”
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25,
no. 7, pp. 787–800, July 2003.
[8] O. Veksler, “Stereo correspondence by dynamic programming on a tree,” in
2005 IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR’05), June 2005, vol. 2, pp. 384–390 vol. 2.
[9] H. Hirschmuller, “Accurate and efficient stereo processing by semi-global
matching and mutual information,” in 2005 IEEE Computer Society Conference
on Computer Vision and Pattern Recognition (CVPR’05), June 2005,
vol. 2, pp. 807–814 vol. 2.
[10] P. Didyk, P. Sitthi-Amorn, W. T. Freeman, F. Durand, and W. Matusik,
“Joint view expansion and filtering for automultiscopic 3D displays,” ACM
Transactions on Graphics (TOG), vol. 32, no. 6, pp. 221, 2013.
[11] E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: a flexible
architecture for multi-scale derivative computation,” in Proceesings of International
Conference on Image Processing, Oct 1995, vol. 3, pp. 444–447
vol.3.
[12] W. Y. Chen, Y. L. Chang, H. K. Chiu, S. Y. Chien, and L. G. Chen, “Realtime
depth image based rendering hardware accelerator for advanced three
dimensional television system,” in 2006 IEEE International Conference on
Multimedia and Expo, July 2006, pp. 2069–2072.
[13] Y. R. Horng, Y. C. Tseng, and T. S. Chang, “VLSI architecture for real-time
HD1080p view synthesis engine,” IEEE Transactions on Circuits and Systems
for Video Technology, vol. 21, no. 9, pp. 1329–1340, Sept 2011.
[14] M. Farre, O. Wang, M. Lang, N. Stefanoski, A. Hornung, and A. Smolic,
“Automatic content creation for multiview autostereoscopic displays using
image domain warping,” in IEEE International Conference on Multimedia
and Expo, July 2011, pp. 1–6.
[15] N. Stefanoski, O. Wang, M. Lang, P. Greisen, S. Heinzle, and A. Smolic,
“Automatic view synthesis by image-domain-warping,” IEEE Transactions
on Image Processing, vol. 22, no. 9, pp. 3329–3341, Sept 2013.
[16] M. Schaffner, F. K. Gürkaynak, P. Greisen, H. Kaeslin, L. Benini, and
A. Smolic, “Hybrid ASIC/FPGA system for fully automatic stereo-tomultiview
conversion using IDW,” IEEE Transactions on Circuits and Systems
for Video Technology, vol. 26, no. 11, pp. 2093–2108, Nov 2016.
[17] N. Wadhwa, M. Rubinstein, F. Durand, and W. T. Freeman, “Phase-based
video motion processing,” ACM Transactions on Graphics (TOG), vol. 32,
no. 4, pp. 80, 2013.
[18] E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, “Shiftable
multiscale transforms,” IEEE Transactions on Information Theory, vol. 38,
no. 2, pp. 587–607, March 1992.
[19] S. Meyer, O. Wang, H. Zimmer, M. Grosse, and A. Sorkine-Hornung, “Phasebased
frame interpolation for video,” in 2015 IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), June 2015, pp. 1410–1418.
[20] Y. C. Wang, “Algorithm and VLSI architecture of wavelet-based rendering
engine for stereo-to-multiview conversion system,” 2018.
[21] Q. H. Liu and N. Nguyen, “An accurate algorithm for nonuniform fast fourier
transforms (NUFFT’s),” IEEE Microwave and Guided Wave Letters, vol. 8,
no. 1, pp. 18–20, Jan 1998.
[22] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation
of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp.
297–301, 1965.
[23] S. He and M. Torkelson, “A new approach to pipeline FFT processor,” in
Proceedings of International Conference on Parallel Processing, April 1996,
pp. 766–770.
[24] S. M. Joshi, “FFT architectures: a review,” international Journal of Computer
applications, vol. 116, no. 7, 2015.
[25] E. H. Wold and A. M. Despain, “Pipeline and parallel-pipeline FFT processors
for VLSI implementations,” IEEE Transactions on Computers, vol. C-33, no.
5, pp. 414–426, May 1984.
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top