跳到主要內容

臺灣博碩士論文加值系統

(44.192.94.177) 您好!臺灣時間:2024/07/16 23:51
字體大小: 字級放大   字級縮小   預設字形  
回查詢結果 :::

詳目顯示

: 
twitterline
研究生:許豐育
研究生(外文):Feng-Yu Hsu
論文名稱:聯合多模型提升影像品質:應用於大腦DTI與海馬迴分割
論文名稱(外文):Joint Models for Improving Image Quality: Application to Brain DTI and Hippocampal Segmentation
指導教授:陳志宏陳志宏引用關係
指導教授(外文):Jyh-Horng Chen
口試委員:黃從仁傅楸善林慶波翁駿程
口試委員(外文):Tsung-Ren HuangChiou-Shann FuhChing-Po LinJun-Cheng Weng
口試日期:2022-01-25
學位類別:碩士
校院名稱:國立臺灣大學
系所名稱:生醫電子與資訊學研究所
學門:工程學門
學類:生醫工程學類
論文種類:學術論文
論文出版年:2022
畢業學年度:110
語文別:英文
論文頁數:126
中文關鍵詞:去噪超解析度影像分割深度學習卷積神經網路
外文關鍵詞:DenoisingSuper-ResolutionDeep learningConvolution neural networksegmentation
DOI:10.6342/NTU202200419
相關次數:
  • 被引用被引用:0
  • 點閱點閱:195
  • 評分評分:
  • 下載下載:12
  • 收藏至我的研究室書目清單書目收藏:0
對於研究分析和臨床診斷的準確性,取得具有高分辨率(High Resolution, HR)的核磁造影(Magnetic Resonance Imaging, MRI)圖像非常重要,其提供了重要的結構紋理訊息,有助於早期診斷和後續分析。但是實際上,由於硬件設備、成像時間、信噪比(Signal-to-Noise Ratio, SNR)和運動假影等因素,要獲得高品質的MR圖像相對困難,因此提升影像品質的需求相應而生。
過去已有許多超解析度(Super-Resolution, SR)與去噪(Denoise)相關的技術用於改善MR圖像品質,然而其中透過插值的SR方法,雖然能快速而簡單的執行超解析度,卻時常過度平滑影像,並且通過濾波方式的去噪方式也常造成影像的重要細節丟失。而近年,基於深度學習(Deep Learning, DL)的方法由於更好的模型表達能力,在SR與Denoise問題都達到更加優越的效能。然而許多網路未充分利用影像的先驗知識,以更複雜的網路改進效能,導致可能訓練困難或硬體需求增加。因此我們將聚焦於提升影像的解析度與SNR上,首先將提出的SR模型方法應用在優化海馬迴子區分割上,再結合去噪模型應用於擴散張量造影(Diffusion Tensor Imaging, DTI),改善DTI重建分析以驗證去噪與提升解析度的效果。
首先,我們提出通道分裂邊緣導引殘差網絡(NLCSERN)用於提升人腦T1影像解析度,透過引進影像邊緣以及欠採樣影像,讓模型獲取更多種與輸入相關的影像特徵,並使用三種主要架構改進,在參數量下降約四十七萬的情況下,讓模型更好地學習這些特徵,最終在人腦T1影像的峰值信噪比與結構相似度上,從35.834 dB/0.9148提升至36.45 dB/0.9228。在海馬迴分割方面,低解析度分割結果顯示,在26個子區域中,有22個子區與高解析度的分割結果呈現顯著差異。透過SR模型提升解析度後,我們有效地使其中14個子區域與高解析度的分割結果變為非顯著差異。
我們也收取了DTI影像應用於實驗室已開發的去噪模型(MDNet)與NLCSERN。前者提升B-null影像至少兩倍以上SNR,使FA map平均值與標準差更接近高SNR影像(低SNR影像: 0.1074±0.0977, 去噪影像: 0.0846±0.083與高SNR影像0.0832±0.0865),並且低SNR影像去噪過後,與高SNR影像的神經角度差變得更小,如在external capsule平均角度差從12.57下降至8.23。
NLCSERN則使B-null影像強度分布更接近高解析度影像,其中平均強度的誤差下降63% ,FA map也經由line profile證實許多原本因partial volume未能看出的神經走向,經由提升解析度而重建出來,並且結構如external capsule的神經角度差從16.13度下降至7.81度。
最後我們結合了兩個模型,同時學習去除DTI影像噪音與提升其解析度,重建出SNR兩倍提升且解析度亦兩倍提升的結果,使得在總體造影時間能大大縮小,並且使其在神經追蹤重建後角度差有所下降,如在Capsule從16.99度下降至9.45度。有了我們的聯合去噪與超解析度模型,將能為現代醫療研究提供所需的高品質的人體影像,以利後續相關的精準醫療與研究使用。
To enhance the accuracy of research analysis and clinical diagnosis, High Resolution (HR) Magnetic Resonance Imaging (MRI) images are crucial since they provide sufficient structural texture information for early diagnosis and follow-up analysis. However, due to limitations including hardware equipment, imaging time, Signal-to-Noise Ratio (SNR), and motion artifacts, the acquisition of high-quality MR images is relatively non-efficient. Therefore, the need to improve image quality correspondingly rises.
Throughout the years, numerous super-resolution (SR) and denoise (Denoise) related technologies have been used to improve MR image quality. However, although SR methods through interpolation are fast and simple implementations of super-resolution, they often over-smooth images. On the other hand, denoising methods through filtering often cause the loss of important structural details. Recently, Deep Learning (DL) based methods achieved superior performance than SR and denoise methods due to their better model expression capabilities. Nonetheless, multiple networks also improved their performance with more complex networks instead of making full use of the prior knowledge of images, resulting in possible training difficulties or increased hardware requirements. Therefore, our research focuses on improving the resolution and SNR of the images. First, our proposed SR model is applied to improving the segmentation of hippocampal subfields. Then, the model is combined with the denoising model to enhance Diffusion Tensor Imaging (DTI), and analysis of DTI reconstruction is used to validate the performance of denoising and SR.
First, we proposed the Non-Local Channel Split Edge-guided Residual Network (NLCSERN) to improve the resolution of human brain T1 images. The model obtained more input-related image features by introducing image edges and under-sampled images. An additional architecture improvement further enhanced the model’s learning ability while the number of parameters decreased by about 470,000. Finally, the super-resolved images' peak SNR and structural similarity increased from 35.834 dB/0.9148 to 36.45 dB/0.9228. For segmentation of hippocampal subfields, 22 out of 26 segmentation results of low-resolution images showed a significant difference between results of HR images. With our SR model, we efficiently made 14 subfields that showed significant differences become not significantly different.
We also established a DTI database to apply our denoising model (MDNet) and NLCSERN. The denoising model improved the SNR of B-null images by at least twice, making the FA map average or standard deviation closer to high SNR images (low SNR image: 0.1074±0.0977, denoising image: 0.0846±0.083 and high SNR image 0.0832±0.0865). The model also reduced the angle difference of nerve orientation with high SNR images, e.g., from 12.57 to 8.23 in the external capsule. On the other hand, NLCSERN made the intensity distribution of B-null images closer to high-resolution images, in which the average intensity error was reduced by 63%. Through the line profile results, we also confirmed in the FA map that many neural trends originally not visible due to the partial volume effect can be reconstructed by increasing the resolution. The nerve angle difference of the external capsule dropped from 16.13 to 7.81 for instance.
Finally, we combined the two models to remove DTI image noise for a 2x SNR boost while achieving 2x resolution. The resulting overall imaging time was shortened while the angle difference of nerve tracking significantly reduced, e.g., from 16.99 to 9.45 degrees in the external capsule. With our joint denoising and super-resolution model, we can provide high-quality human images for modern medical research to facilitate follow-up precision medical and research applications.
誌謝..................................................................i
中文摘要..............................................................ii
Abstract...........................................................iiii
Contents.............................................................vi
List of Figures......................................................ix
List of Tables......................................................xii
Chapter 1 Introduction
1.1. Background....................................................1
1.2. Motivation and purpose........................................1
1.3. Magnetic Resonance Imaging (MRI)..............................3
1.3.1. Principle of MRI..............................................3
1.3.2. Application of MRI............................................4
1.3.3. Diffusion Tensor Imaging (DTI)................................6
1.4. Image quality enhancement....................................11
1.4.1. Image Super-Resolution (SR)..................................11
1.4.2. Image Denoising..............................................14
1.5. Deep Learning (DL) model structure...........................16
1.5.1. Convolution neural networks..................................18
1.5.2. PixelShuffle.................................................20
1.5.3. Generative and Adversarial Network (GAN).....................21

Chapter 2 Material and Methods
2.1 Residual-learning SR model design............................23
2.1.1 Channel-splitting............................................26
2.1.2 Edge-guided..................................................29
2.1.3 Non-local Sparse Attention (NLSA)............................31
2.2 Loss Function................................................33
2.3 Training and testing datasets................................37
2.3.1 Subjective cognitive decline (SCD) database..................37
2.3.2 Real pairs of Low and high resolutions human brain database..38
2.3.3 Human brain DTI database.....................................39
2.3.4 Dataset generation steps.....................................40
2.4 Analytical methods and tools.................................44
Chapter 3 Results of Super-Resolution (SR) model design
3.1 SR model performance.........................................51
3.1.1 Improvement of edge-guided...................................53
3.1.2 Improvement of channel splitting.............................55
3.1.3 Improvement of our loss......................................57
3.1.4 Improvement of non-local sparse attention....................59
3.2 Comparison of state-of-art methods...........................60
3.2.1 Model capabilities...........................................60
3.2.2 SR results of real Low Resolution (LR) data..................66
3.2.3 Analysis of equivalent resolution............................70
3.3 Results of hippocampal subfield segmentation.................74

Chapter 4 Results of Super-Resolution and denoised DTI images
4.1 Results of denoising DTI.....................................80
4.2 Results of SR DTI............................................86
4.3 Results of joint denoising and SR DTI........................95
Chapter 5 Discussion
5.1 Training dataset generation.................................106
5.2 Methods of architecture improvement.........................110
5.3 Limitation of our model.....................................113
Chapter 6 Conclusion and Future Work
6.1 Conclusion..................................................116
6.2 Future Work.................................................118
6.2.1 Generation of training sets.................................118
6.2.2 Model improvements..........................................118
6.2.3 For DTI with better image quality...........................119

Reference...........................................................122
1.Zaretskaya, N., Fischl, B., Reuter, M., Renvall, V., & Polimeni, J. R. (2018). Advantages of cortical surface reconstruction using submillimeter 7 T MEMPRAGE. Neuroimage, 165, 11-26.
2.Fan, L. Y., Lai, Y. M., Chen, T. F., Hsu, Y. C., Chen, P. Y., Huang, K. Z., ... & Chiu, M. J. (2018). Diminution of context association memory structure in subjects with subjective cognitive decline. Human brain mapping, 39(6), 2549-2562.
3.Jyh-Horng Chen, et al. Nuclear Magnetic Resonance and Magnetic Resonance Imaging. P.3-16
4.Iglesias, J. E., Augustinack, J. C., Nguyen, K., Player, C. M., Player, A., Wright, M., ... & Alzheimer's Disease Neuroimaging Initiative. (2015). A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: application to adaptive segmentation of in vivo MRI. Neuroimage, 115, 117-137.
5.Frank, L. R. (2002). Characterization of anisotropy in high angular resolution diffusion‐weighted MRI. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 47(6), 1083-1099.
6.Kitajima, K., Kaji, Y., Kuroda, K., & Sugimura, K. (2008). High b-value diffusion-weighted imaging in normal and malignant peripheral zone tissue of the prostate: effect of signal-to-noise ratio. Magnetic Resonance in Medical Sciences, 7(2), 93-99.
7.Mukherjee, P., Berman, J. I., Chung, S. W., Hess, C. P., & Henry, R. G. (2008). Diffusion tensor MR imaging and fiber tractography: theoretic underpinnings. American journal of neuroradiology, 29(4), 632-641.
8.Irani, M., & Peleg, S. (1991). Improving resolution by image registration. CVGIP: Graphical models and image processing, 53(3), 231-239.
9.Stark, H., & Oskoui, P. (1989). High-resolution image recovery from image-plane arrays, using convex projections. JOSA A, 6(11), 1715-1726.
10.Freeman, W. T., Jones, T. R., & Pasztor, E. C. (2002). Example-based super-resolution. IEEE Computer graphics and Applications, 22(2), 56-65.
11.Yang, J., Wright, J., Huang, T., & Ma, Y. (2008, June). Image super-resolution as sparse representation of raw image patches. In 2008 IEEE conference on computer vision and pattern recognition (pp. 1-8). IEEE.
12.Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on image processing, 16(8), 2080-2095.
13.Lee, O. Y., Shin, Y. H., & Kim, J. O. (2019). Multi-perspective discriminators-based generative adversarial network for image super resolution. IEEE Access, 7, 136496-136510.
14.Changlai, G. (2007). Image-denoising method based on wavelet transform and mean filtering. Opto-Electronic Engineering, 1, 19.
15.Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017). Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7), 3142-3155.
16.Zhao, T., McNitt‐Gray, M., & Ruan, D. (2019). A convolutional neural network for ultra‐low‐dose CT denoising and emphysema screening. Medical physics, 46(9), 3941-3950.
17.陳柏廷(2021)。創新邊緣偵測去噪神經網路:提升擴散張量影像訊雜比。國立臺灣大學生醫電子與資訊學研究所碩士論文,台北市。
18.Dong, C., Loy, C. C., & Tang, X. (2016, October). Accelerating the super-resolution convolutional neural network. In European conference on computer vision (pp. 391-407). Springer, Cham.
19.Dong, C., Loy, C. C., He, K., & Tang, X. (2014, September). Learning a deep convolutional network for image super-resolution. In European conference on computer vision (pp. 184-199). Springer, Cham.
20.Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., ... & Shi, W. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4681-4690).
21.Fathi, E., & Shoja, B. M. (2018). Deep neural networks for natural language processing. In Handbook of statistics (Vol. 38, pp. 229-316). Elsevier.
22.Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., ... & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1874-1883).
23.Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27.
24.Lan, R., Sun, L., Liu, Z., Lu, H., Su, Z., Pang, C., & Luo, X. (2020). Cascading and enhanced residual networks for accurate single-image super-resolution. IEEE transactions on cybernetics, 51(1), 115-125.
25.Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125-1134).
26.Zhao, X., Zhang, Y., Zhang, T., & Zou, X. (2019). Channel splitting network for single MR image super-resolution. IEEE Transactions on Image Processing, 28(11), 5649-5662.
27.Lim, B., Son, S., Kim, H., Nah, S., & Mu Lee, K. (2017). Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 136-144).
28.Du, X., & He, Y. (2019). Gradient-guided convolutional neural network for MRI image super-resolution. Applied Sciences, 9(22), 4874.
29.Odena, A., Dumoulin, V., & Olah, C. (2016). Deconvolution and checkerboard artifacts. Distill, 1(10), e3.
30.Wu, E. L., Huang, Y. A., Chiueh, T. D., & Chen, J. H. (2015). Single‐frequency excitation wideband MRI (SE‐WMRI). Medical physics, 42(7), 4320-4328.
31.Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., ... & Change Loy, C. (2018). Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops (pp. 0-0).
32.Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
33.Johnson, J., Alahi, A., & Fei-Fei, L. (2016, October). Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision (pp. 694-711). Springer, Cham.
34.Fritsche, M., Gu, S., & Timofte, R. (2019, October). Frequency separation for real-world super-resolution. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) (pp. 3599-3608). IEEE.
35.Bell-Kligler, S., Shocher, A., & Irani, M. (2019). Blind super-resolution kernel estimation using an internal-gan. arXiv preprint arXiv:1909.06581.
36.Lyu, Q., Shan, H., & Wang, G. (2020). MRI super-resolution with ensemble learning and complementary priors. IEEE Transactions on Computational Imaging, 6, 615-624.
37.Yuan, Y., Liu, S., Zhang, J., Zhang, Y., Dong, C., & Lin, L. (2018). Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 701-710).
38.Ji, X., Cao, Y., Tai, Y., Wang, C., Li, J., & Huang, F. (2020). Real-world super-resolution via kernel estimation and noise injection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 466-467).
39.Tian, C., Xu, Y., & Zuo, W. (2020). Image denoising using deep CNN with batch renormalization. Neural Networks, 121, 461-473.
40.Mei, Y., Fan, Y., & Zhou, Y. (2021). Image Super-Resolution With Non-Local Sparse Attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3517-3526).
41.Xing, W., & Egiazarian, K. (2021). End-to-End Learning for Joint Image Demosaicing, Denoising and Super-Resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3507-3516).
42.Bao, C., Cai, J. F., & Ji, H. (2013). Fast sparsity-based orthogonal dictionary learning for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3384-3391).
43.Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing, 15(12), 3736-3745.
44.Dong, W., Zhang, L., Shi, G., & Li, X. (2012). Nonlocally centralized sparse representation for image restoration. IEEE transactions on Image Processing, 22(4), 1620-1630.
45.Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE transactions on image processing, 19(11), 2861-2873.
46.Fan, Y., Yu, J., Mei, Y., Zhang, Y., Fu, Y., Liu, D., & Huang, T. S. (2020). Neural Sparse Representation for Image Restoration. arXiv preprint arXiv:2006.04357.
47.Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
48.Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612.
49.Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 815-823).
50.Venkatanath, N., Praneeth, D., Bh, M. C., Channappayya, S. S., & Medasani, S. S. (2015, February). Blind image quality evaluation using perception based features. In 2015 Twenty First National Conference on Communications (NCC) (pp. 1-6). IEEE.
51.Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012). No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12), 4695-4708.
52.Do, H., Helbert, D., Bourdon, P., Naudin, M., Guillevin, C., & Guillevin, R. (2021, October). MRI super-resolution using 3D cycle-consistent generative adversarial network. In 2021 Sixth International Conference on Advances in Biomedical Engineering (ICABME) (pp. 85-88). IEEE.
53.Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision (pp. 2223-2232).
54.Tor-Díez, C., Pham, C. H., Meunier, H., Faisan, S., Bloch, I., Bednarek, N., ... & Rousseau, F. (2019, July). Evaluation of cortical segmentation pipelines on clinical neonatal MRI data. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 6553-6556). IEEE.
55.Pham, C. H., Tor-Díez, C., Meunier, H., Bednarek, N., Fablet, R., Passat, N., & Rousseau, F. (2019, April). Simultaneous super-resolution and segmentation using a generative adversarial network: Application to neonatal brain MRI. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) (pp. 991-994). IEEE.
56.Chaudhari, A. S., Stevens, K. J., Wood, J. P., Chakraborty, A. K., Gibbons, E. K., Fang, Z., ... & Hargreaves, B. A. (2020). Utility of deep learning super‐resolution in the context of osteoarthritis MRI biomarkers. Journal of Magnetic Resonance Imaging, 51(3), 768-779.
57.Hu, X., Mu, H., Zhang, X., Wang, Z., Tan, T., & Sun, J. (2019). Meta-SR: A magnification-arbitrary network for super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1575-1584).
連結至畢業學校之論文網頁點我開啟連結
註: 此連結為研究生畢業學校所提供,不一定有電子全文可供下載,若連結有誤,請點選上方之〝勘誤回報〞功能,我們會盡快修正,謝謝!
QRCODE
 
 
 
 
 
                                                                                                                                                                                                                                                                                                                                                                                                               
第一頁 上一頁 下一頁 最後一頁 top
無相關期刊