臺灣博碩士論文加值系統

現階段的影像系統，仍難以在肝臟微創手術中進行準確且即時的定位。為了解決此困境，肝臟微創手術的影像導引系統研究遂成為一個研究探討課題。在影像導引系統中，形變計算為相當關鍵的一個環節，旨在補償肝臟受到器械拉扯、搬動所造成的形變，使導引系統能夠及時定位肝臟內部結構的位置，如血管、腫瘤。過去利用有限元素法的形變計算研究仍面臨問題。其一是手術中的即時影像是受限的，難以提供足夠物理邊界條件；其二是受到準確度與運算速度的權衡限制，且須考慮個體差異。我們選擇利用深度迴歸神經網路，試圖以深度學習的方式進行形變的計算，並達成準確、速度與個體差異的要求。然而，過去研究中以深度學習進行形變計算往往遇到參考座標系的限制，其表現受制於訓練資料的參考坐標系。因此，我們也透過參數化的方式，使得回歸神經網路的表現不受到參考坐標系的影響。
我們透過模擬的形變數據以輔助迴歸神經網路的設計。同時，我們也藉這些形變數據探討了神經網路對於不同形變情境的差異性，是否具備一般化(適應)的能力。這些情境差異性包含：楊氏係數、標的的位置、組織的形狀大小、形變的狀態。透過這些模擬的實驗，我們也提出了訓練此回歸神經網路的建議。接著，我們利用此模擬數據訓練回歸神經網路，並應用於活體外豬肝的形變計算中。然此驗證尚未取得滿意的結果；不過，如果我們直接以活體外豬肝形變的數據訓練此神經網路，可以大幅改善形變計算的準確度。儘管此研究在未來仍有改善的空間，此嘗試確實是了一個嶄新的形變計算方法。

Current imaging modalities still have difficulty locating liver interior structures in minimally invasive liver surgery (MILS) accurately in real-time. Studies on image-guided systems have emerged to solve the positioning problem. In the image-guided system procedures, deformation computation is a critical technique to compensate for liver deformation caused by tool operation. Previous deformation computation methods have yet to resolve limited intra-operative information, providing accurate, real-time positioning, and individual differences at the same time. We tried to solve the dilemma by applying a deep regression network to achieve both of these requirements. Previous deep learning-based methods were still bounded to a pre-trained coordinate system, so we also design a parameterization process to make the network coordinate-invariant.
We conducted simulated experiments to ensure network architecture improvements and explore the network generalization ability in different aspects. Under these controlled experiments, we found the generalization ability and limitation among different aspects of the deformation simulations: Young’s modulus, target positions, size, shape, and deformation. The strategy of training a good network is also provided. In an ex vivo demonstration, the network trained with the simulated data has not yet achieved satisfying results. Excitingly, if we directly trained the network with ex vivo data, the performance is improved. Although improvements are required, this study did investigates the deformation computation in a novel direction.

口試委員會審定書 i
誌謝 ii
中文摘要 iii
ABSTRACT iv
CONTENTS v
LIST OF FIGURES viii
LIST OF TABLES xii
Chapter 1 Introduction 1
1.1 Background 2
1.2 Problem Statements 5
1.3 Aims of the thesis 7
1.4 Thesis Structure 10
Chapter 2 State-of-the-art 11
2.1 Image Registration and Deformation Computation 11
2.2 Deformation Computation Using Neural Networks 13
Chapter 3 Methodology 16
3.1 Formulate the Deformation Computation Problem 17
3.2 Collect Simulated Deformation Data 22
3.2.1 Introduction to Abaqus FEM Simulation 22
3.2.2 Simulation Parameters and Datasets 25
3.2.3 Data Collection Procedure 31
3.3 Collect Ex Vivo Deformation Data 33
3.3.1 The TrakSTAR Electromagnetic Tracking Device 34
3.3.2 The ZED M Stereo Camera 36
3.3.3 Integrate the Deformation Capturing System 40
3.3.4 Evaluate the Deformation Capturing System 42
3.3.5 Liver Deformation Capturing Procedure 45
3.4 The Deep Regression Networks 47
3.4.1 The Naïve Network for Deformation Computation 47
3.4.2 The Pure Parameterized Network 48
3.4.3 Unable-to-Fit-Different-Depth Problem 52
3.4.4 The Difference Network Design 55
3.5 Experiments 57
3.5.1 Improvements of the Deep Regression Network Architecture 57
3.5.2 Young’s Modulus Generalization 58
3.5.3 Size and Shape Generalization 60
3.5.4 Target Position Generalization 61
3.5.5 Deformation Generalization 63
3.5.6 The Ex Vivo Deformation Demonstration 65
Chapter 4 Results and Discussion 67
4.1 Results of Improving the Deep Regression Network 69
4.2 Experiments of Generalization 70
4.2.1 Young’s Modulus Generalization 70
4.2.2 Size and Shape Generalization 72
4.2.3 Target Position Generalization 76
4.2.4 Deformation Generalization 77
4.3 The Ex Vivo Deformation Demonstration 78
4.4 The Error Distributions 80
4.5 Computation Time 85
Chapter 5 Conclusion 87
Chapter 6 Technical Procedures and Derivations 88
6.1 How to Extract Each Deformation Configuration From the Deformation Sequence 88
6.2 The Overall Process for Simulated Data Collection 89
6.3 Calibrate the Stereo Camera 90
6.4 Estimate Sphere Centers in the Stereo Camera Images 91
BIBLIOGRAPHY 95

[1]M. Morino, I. Morra, E. Rosso, C. Miglietta, and C. Garrone, "Laparoscopic vs open hepatic resection: a comparative study," Surgical Endoscopy and Other Interventional Techniques, vol. 17, no. 12, pp. 1914-1918, 2003.
[2]H. Kaneko et al., "Laparoscopic liver resection of hepatocellular carcinoma," The American journal of surgery, vol. 189, no. 2, pp. 190-194, 2005.
[3]G. Belli et al., "Laparoscopic and open treatment of hepatocellular carcinoma in patients with cirrhosis," British Journal of Surgery, vol. 96, no. 9, pp. 1041-1048, 2009.
[4]M. A. Talamini, Advanced therapy in minimally invasive surgery. PMPH-USA, 2006.
[5]K.-H. Chang et al., "Effectiveness of external respiratory surrogates for in vivo liver motion estimation," Medical physics, vol. 39, no. 8, pp. 5293-5301, 2012.
[6]J. S. Heiselman et al., "Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery," Journal of Medical Imaging, vol. 5, no. 2, p. 021203, 2017.
[7]S. Suwelack et al., "Physics‐based shape matching for intraoperative image guidance," Medical physics, vol. 41, no. 11, p. 111901, 2014.
[8]R. Plantefeve, I. Peterlik, N. Haouchine, and S. Cotin, "Patient-specific biomechanical modeling for guidance during minimally-invasive hepatic surgery," Annals of biomedical engineering, vol. 44, no. 1, pp. 139-153, 2016.
[9]R. Modrzejewski, T. Collins, B. Seeliger, A. Bartoli, A. Hostettler, and J. Marescaux, "An in vivo porcine dataset and evaluation methodology to measure soft-body laparoscopic liver registration accuracy with an extended algorithm that handles collisions," International journal of computer assisted radiology and surgery, vol. 14, no. 7, pp. 1237-1245, 2019.
[10]D. Reichard et al., "Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery," International journal of computer assisted radiology and surgery, vol. 12, no. 7, pp. 1101-1110, 2017.
[11]E. Özgür, B. Koo, B. Le Roy, E. Buc, and A. Bartoli, "Preoperative liver registration for augmented monocular laparoscopy using backward–forward biomechanical simulation," International journal of computer assisted radiology and surgery, vol. 13, no. 10, pp. 1629-1640, 2018.
[12]L. W. Clements et al., "Deformation correction for image guided liver surgery: An intraoperative fidelity assessment," Surgery, vol. 162, no. 3, pp. 537-547, 2017.
[13]D. C. Rucker et al., "A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data," IEEE transactions on medical imaging, vol. 33, no. 1, pp. 147-158, 2013.
[14]M. Pfeiffer, C. Riediger, J. Weitz, and S. Speidel, "Learning soft tissue behavior of organs for surgical navigation with convolutional neural networks," International journal of computer assisted radiology and surgery, vol. 14, no. 7, pp. 1147-1155, 2019.
[15]U. Yamamoto, M. Nakao, M. Ohzeki, and T. Matsuda, "Deformation estimation of an elastic object by partial observation using a neural network," arXiv preprint arXiv:1711.10157, 2017.
[16]A. Mendizabal, P. Márquez-Neila, and S. Cotin, "Simulation of hyperelastic materials in real-time using deep learning," Medical image analysis, vol. 59, p. 101569, 2020.
[17]S.-T. Su, M.-C. Ho, J.-Y. Yen, and Y.-Y. Chen, "Featured Surface Matching Method for Liver Image Registration," IEEE Access, vol. 8, pp. 59723-59731, 2020.
[18]H. G. Kenngott et al., "Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging," Surgical endoscopy, vol. 28, no. 3, pp. 933-940, 2014.
[19]I. Peterlík et al., "Fast elastic registration of soft tissues under large deformations," Medical image analysis, vol. 45, pp. 24-40, 2018.
[20]S.-H. Kong et al., "Robust augmented reality registration method for localization of solid organs’ tumors using CT-derived virtual biomechanical model and fluorescent fiducials," Surgical endoscopy, vol. 31, no. 7, pp. 2863-2871, 2017.
[21]J. S. Heiselman, W. R. Jarnagin, and M. I. Miga, "Intraoperative correction of liver deformation using sparse surface and vascular features via linearized iterative boundary reconstruction," IEEE transactions on medical imaging, vol. 39, no. 6, pp. 2223-2234, 2020.
[22]D. M. Cash, M. I. Miga, T. K. Sinha, R. L. Galloway, and W. C. Chapman, "Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements," IEEE transactions on medical imaging, vol. 24, no. 11, pp. 1479-1491, 2005.
[23]P. Dumpuri, L. W. Clements, B. M. Dawant, and M. I. Miga, "Model-updated image-guided liver surgery: preliminary results using surface characterization," Progress in biophysics and molecular biology, vol. 103, no. 2-3, pp. 197-207, 2010.
[24]Z. Wu et al., "3d shapenets: A deep representation for volumetric shapes," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1912-1920.
[25]T. Groueix, M. Fisher, V. G. Kim, B. C. Russell, and M. Aubry, "3d-coded: 3d correspondences by deep deformation," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 230-246.
[26]J. Zhang, Y. Zhong, and C. Gu, "Neural network modelling of soft tissue deformation for surgical simulation," Artificial intelligence in medicine, vol. 97, pp. 61-70, 2019.
[27]S. R. Z. Abdel-Misih and M. Bloomston, "Liver anatomy," (in eng), The Surgical clinics of North America, vol. 90, no. 4, pp. 643-653, 2010/08// 2010.
[28]W.-J. Hsu, "Finite Element Model-Based Simulation of LIver Deformation for Vessel Tracking," Department of Electrical Engineering, National Taiwan University, 2015, 2015.
[29]B. Overmoyer, C. McLaren, and G. Brittenham, "Uniformity of liver density and nonheme (storage) iron distribution," Archives of pathology & laboratory medicine, vol. 111, no. 6, pp. 549-554, 1987.
[30]B. Ahn and J. Kim, "Measurement and characterization of soft tissue behavior with surface deformation and force response under large deformations," Medical image analysis, vol. 14, no. 2, pp. 138-148, 2010.
[31]B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," 1981: Vancouver, British Columbia.
[32]C. M. Bishop, Neural networks for pattern recognition. Oxford university press, 1995.
[33]S. Kaustubh and M. AvatarSatya, "Camera Calibration using OpenCV | Learn OpenCV," ed, 2020.
[34]S. Kaustubh, "Making A Low-Cost Stereo Camera Using OpenCV | Learn OpenCV," ed, 2021.