臺灣博碩士論文加值系統

為了創建全景圖像，利用圖像拼接方法可以將兩個或多個圖像組合在一起。但是，傳統的圖像拼接策略會面對幾個問題的困擾。這些問題導致拼接全景圖像中的偽影和未對準的斷層問題。當用於不同的圖像拼接應用時，它可能導致嚴重的拼接瑕疵。 最佳縫線搜索的縫合帶確定法在現有的圖像拼接框架/流程圖中添加了一個新的處理機制，幫助傳統的圖像拼接策略消除或最小化在全景圖像中產生偽影的可能性。傳統的圖像拼接框架假設接縫線演算法將避免重疊區域中的偽影，並在輸出處提供完美的全景圖像。然而，接縫線演算法的工作原理包括尋找最小能量路徑，最短距離路徑，分割高能量和低能量像素值等，這些都與偽像發現沒有直接關係。因此，當在不同的圖像拼接應用中使用該假設時，不同的圖像場景不能適合每個相鄰的圖像對。這是因為接縫線演算法和拼接影像之間的非直接關係。並且，這種非直接關係，以及無法在視覺上定位和避免偽影導致縫合的全景圖像中的不連續和未對準。另一方面，縫合帶確定法引入了一種全新的中期處理策略，其中使用參數組合來直觀地和直接地定位偽像並且將重疊區域的無偽影區域/縫合帶推薦到最佳接縫線搜索。可用於精確全景圖像拼接的各式演算法。

To create a panorama image, Image Stitching methods are used, which combine two or multiple images together. However, traditional Image Stitching framework is suffering from various issues. These issues lead to the artifacts and misalignments in the stitched panorama image without any warning. It can cause serious damages, when being used in different image stitching applications. “Stitching Strip Determination for Optimal Seamline Search” add a new block into existing image stitching framework/flow-chart, help the traditional image stitching framework to either eliminate or minimize the possibility to incur artifacts in the panorama image. Traditional image stitching framework assumes that seamline algorithms will avoid artifacts in overlapping region and gives perfect panorama image at output. However, Seamline algorithms works on principles such as finding the minimum energy path, shortest distance path, segmenting the high energy and low energy pixel values, etc., which are not directly related to the artifacts finding. Hence, when this assumption is being used in different image stitching applications, different image scenarios cannot fit with each and every adjacent image pair. This happens because of indirect relation between seamline algorithms and artifacts. And, this indirect relation, and inability to visually locate and avoid artifacts lead to discontinuities and misalignments in the stitched panorama image. On the other hand, Stitching Strip Determination method introduced a completely new mid-stage processing strategy, in which combination of parameters are used to visually and directly locate the artifacts and recommend an artifact free region/strip of the overlapping area to the optimal seamline search algorithm for the accurate panorama image stitching.

Chinese Abstract ………………………………………………………… i
English Abstract ………………………………………………………… ii
Acknowledgement ………………………………………………………… iii
Table of Contents ………………………………………………………… iv
List of Tables ………………………………………………………… vi
List of Figures ………………………………………………………… vii
I. Introduction…………………………………………… 1
1.1 Preliminary …………………………………………… 1
II Literature review ……………………………………… 4
2.1 Introduction…………………………………………… 4
2.1.1 Image Acquisition …………………………………… 5
2.1.2 Feature Extraction & Matching ……………………… 5
2.1.3 Homography using RANSAC………………………… 5
2.1.4 Image Alignment……………………………………… 7
2.1.5 Seamline Finding……………………………………… 7
III Stitching Strip Determination………………………… 13
3.1 Artifacts in existing framework ……………………… 14
3.2 Classification of Artifacts Handling strategies ……… 16
3.2.1 Pre-Warping Strategies ……………………………… 16
3.2.2 Post-Warping Strategies……………………………… 16
3.3 Issues with Artifacts Handling strategies …………… 17
3.3.1 Inconsistent results with different image cases ……… 17
3.3.2 Indirectly Handling the artifacts……………………… 22
3.3.3 Subjective Evaluation………………………………… 25
3.3.4 Objective Evaluation ………………………………… 26
3.4 Determination of Stitching Strip……………………… 30
3.4.1 Feature Point Extraction & Matching………………… 35
3.4.2 Intensity Equalization………………………………… 36
3.4.3 Image Segmentation ………………………………… 39
3.4.4 Structural Similarity Index Metric…………………… 40
3.4.4 Stitching Strip Determination………………………… 42
IV. Experiment Results…………………………………… 44
4.1 Introduction…………………………………………… 44
4.2 Panorama Image Construction ……………………… 44
4.2.1 Experimental Case 1 ………………………………… 45
4.2.2 Experimental Case 2 ………………………………… 47
4.2.3 Experimental Case 3 ………………………………… 50
4.2.4 Experimental Case 4 ………………………………… 52
4.2.5 Experimental Case 5 ………………………………… 55
4.2.6 Experimental Case 6 ………………………………… 57
4.2.7 Experimental Case 7 ………………………………… 60
V Conclusion…………………………………………… 63
5.1 Discussion …………………………………………… 63
5.2 Future Work ………………………………………… 65
References …………………………………………… 66

[1] H.-Y. Shum and R. Szeliski, "Construction and refinement of panoramic mosaics with global and local alignment," in Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), 1998: IEEE, pp. 953-956.
[2] K. Li and Y. Liu, "Image stitching method and apparatus," ed: Google Patents, 2013.
[3] H. M. Kalayeh, "Image stitching and related method therefor," ed: Google Patents, 2013.
[4] R. Cutler, "System and method for camera color calibration and image stitching," ed: Google Patents, 2007.
[5] D. Steedly, R. Szeliski, M. Uyttendaele, and M. Cohen, "Image stitching using partially overlapping views of a scene," ed: Google Patents, 2011.
[6] S. Peleg, A. Rav-Acha, and G. Engel, "Method and system for forming a panoramic image of a scene having minimal aspect distortion," ed: Google Patents, 2013.
[7] G. Pettersson, "Image stitching," ed: Google Patents, 2018.
[8] S. Banerjee, P. G. Sheshagiri, P. K. Baheti, A. D. Gupte, and A. V. Rao, "Systems and methods for image stitching," 2017 US.
[9] V. Kwatra, A. Schödl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: image and video synthesis using graph cuts," ACM Transactions on Graphics (ToG), vol. 22, no. 3, pp. 277-286, 2003.
[10] A. Agarwala et al., "Interactive digital photomontage," in ACM Transactions on Graphics (ToG), 2004, vol. 23, no. 3: ACM, pp. 294-302.
[11] P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACM transactions on Graphics, vol. 2, no. 4, pp. 217-236, 1983.
[12] P. Pérez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Transactions on graphics (TOG), vol. 22, no. 3, pp. 313-318, 2003.
[13] A. Eden, M. Uyttendaele, and R. Szeliski, "Seamless image stitching of scenes with large motions and exposure differences," in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), 2006, vol. 2: IEEE, pp. 2498-2505.
[14] J. Gao, Y. Li, T.-J. Chin, and M. S. Brown, "Seam-Driven Image Stitching," in Eurographics (Short Papers), 2013, pp. 45-48.
[15] S. Laaroussi, A. Baataoui, A. Halli, and S. Khalid, "A dynamic mosaicking method for finding an optimal seamline with Canny edge detector," Procedia computer science, vol. 148, pp. 618-626, 2019.
[16] S. Laaroussi, A. Baataoui, A. Halli, and S. Khalid, "A dynamic mosaicking method based on histogram equalization for an improved seamline," Procedia Computer Science, vol. 127, pp. 344-352, 2018.
[17] L. Li, J. Yao, X. Lu, J. Tu, and J. Shan, "Optimal seamline detection for multiple image mosaicking via graph cuts," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 113, pp. 1-16, 2016.
[18] K. Lin, N. Jiang, L.-F. Cheong, M. Do, and J. Lu, "Seagull: Seam-guided local alignment for parallax-tolerant image stitching," in European Conference on Computer Vision, 2016: Springer, pp. 370-385.
[19] Y. Wan, D. Wang, J. Xiao, X. Lai, and J. Xu, "Automatic determination of seamlines for aerial image mosaicking based on vector roads alone," ISPRS journal of photogrammetry and remote sensing, vol. 76, pp. 1-10, 2013.
[20] W.-X. Yan, C.-C. Liu, and J. Hu, "Optimal Seam Line Detection in Laplacian Pyramid Domain for Image Stitching," 電腦學刊, vol. 29, no. 1, pp. 209-219, 2018.
[21] F. Zhang and F. Liu, "Parallax-tolerant image stitching," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3262-3269.
[22] Y. Chen and R. S. Blum, "A new automated quality assessment algorithm for image fusion," Image and vision computing, vol. 27, no. 10, pp. 1421-1432, 2009.
[23] N. Cvejic, D. Bull, and C. Canagarajah, "Metric for multimodal image sensor fusion," Electronics Letters, vol. 43, no. 2, pp. 95-96, 2007.
[24] G. Piella and H. Heijmans, "A new quality metric for image fusion," in Proceedings 2003 International Conference on Image Processing (Cat. No. 03CH37429), 2003, vol. 3: IEEE, pp. III-173.
[25] K. G. Derpanis, "The harris corner detector," York University, pp. 2-3, 2004.
[26] H. Bay, T. Tuytelaars, and L. Van Gool, "Surf: Speeded up robust features," in European conference on computer vision, 2006: Springer, pp. 404-417.
[27] T. Lindeberg, "Scale invariant feature transform," 2012.
[28] D. G. Viswanathan, "Features from accelerated segment test (fast)," ed: nd, 2009.
[29] E. Rublee, V. Rabaud, K. Konolige, and G. R. Bradski, "ORB: An efficient alternative to SIFT or SURF," in ICCV, 2011, vol. 11, no. 1: Citeseer, p. 2.
[30] S. Arya, "A review on image stitching and its different methods," International Journal of Advanced Research in Computer Science and Software Engineering, vol. 5, no. 5, pp. 299-303, 2015.
[31] T. Watanabe, "A fuzzy RANSAC algorithm based on reinforcement learning concept," in 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 2013: IEEE, pp. 1-6.
[32] Wikipidea. (2018). Image Stitching [Online]. Available: https://en.wikipedia.org/wiki/Image_stitching.
[33] J. Chon, H. Kim, and C.-S. Lin, "Seam-line determination for image mosaicking: A technique minimizing the maximum local mismatch and the global cost," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 65, no. 1, pp. 86-92, 2010.
[34] J. Pan, Q. Zhou, and M. Wang, "Seamline determination based on segmentation for urban image mosaicking," IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 8, pp. 1335-1339, 2014.
[35] S. Pang, M. Sun, X. Hu, and Z. Zhang, "SGM-based seamline determination for urban orthophoto mosaicking," ISPRS journal of photogrammetry and remote sensing, vol. 112, pp. 1-12, 2016.
[36] M. Kerschner, "Seamline detection in colour orthoimage mosaicking by use of twin snakes," ISPRS journal of photogrammetry and remote sensing, vol. 56, no. 1, pp. 53-64, 2001.
[37] L. Yu, E.-J. Holden, M. C. Dentith, and H. Zhang, "Towards the automatic selection of optimal seam line locations when merging optical remote-sensing images," International Journal of Remote Sensing, vol. 33, no. 4, pp. 1000-1014, 2012.
[38] S. Mills and P. McLeod, "Global seamline networks for orthomosaic generation via local search," ISPRS journal of photogrammetry and remote sensing, vol. 75, pp. 101-111, 2013.
[39] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models," International journal of computer vision, vol. 1, no. 4, pp. 321-331, 1988.
[40] R. Bellman, "1957, Dynamic programming. Princeton University Press, Princeton, N. J."
[41] E. W. Dijkstra, "A note on two problems in connexion with graphs," Numerische mathematik, vol. 1, no. 1, pp. 269-271, 1959.
[42] Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," in Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, vol. 1: IEEE, pp. 377-384.
[43] N. Cvejic, A. Loza, D. Bull, and N. Canagarajah, "A similarity metric for assessment of image fusion algorithms," International journal of signal processing, vol. 2, no. 3, pp. 178-182, 2005.
[44] M. Hossny, S. Nahavandi, and D. Crieghton, "Feature-based image fusion quality metrics," in International Conference on Intelligent Robotics and Applications, 2008: Springer, pp. 469-478.
[45] P.-w. Wang and B. Liu, "A novel image fusion metric based on multi-scale analysis," in 2008 9th International Conference on Signal Processing, 2008: IEEE, pp. 965-968.
[46] J. Zhao, R. Laganiere, and Z. Liu, "Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement," International Journal of Innovative Computing, Information and Control, vol. 3, no. 6, pp. 1433-1447, 2007.
[47] Y. Zheng, E. A. Essock, B. C. Hansen, and A. M. Haun, "A new metric based on extended spatial frequency and its application to DWT based fusion algorithms," Information Fusion, vol. 8, no. 2, pp. 177-192, 2007.
[48] Q. Wang, Y. Shen, and J. Jin, "Performance evaluation of image fusion techniques," Image fusion: algorithms and applications, vol. 19, pp. 469-492, 2008.
[49] V. C. Chew and F.-L. Lian, "Panorama stitching using overlap area weighted image plane projection and dynamic programming for visual localization," in 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2012: IEEE, pp. 250-255.
[50] A. Román and H. P. Lensch, "Automatic Multiperspective Images," Rendering Techniques, vol. 2, no. 2006, pp. 161-171, 2006.
[51] F. Dornaika and R. Chung, "Mosaicking images with parallax," Signal Processing: Image Communication, vol. 19, no. 8, pp. 771-786, 2004.
[52] S. Peleg, B. Rousso, A. Rav-Acha, and A. Zomet, "Mosaicing on adaptive manifolds," IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 10, pp. 1144-1154, 2000.
[53] A. Rav-Acha, G. Engel, and S. Peleg, "Minimal aspect distortion (MAD) mosaicing of long scenes," International Journal of Computer Vision, vol. 78, no. 2-3, pp. 187-206, 2008.
[54] S. M. Seitz and J. Kim, "Multiperspective imaging," IEEE Computer Graphics and Applications, vol. 23, no. 6, pp. 16-19, 2003.
[55] H.-Y. Shum and R. Szeliski, "Construction of panoramic image mosaics with global and local alignment," in Panoramic vision: Springer, 2001, pp. 227-268.
[56] J. Zaragoza, T.-J. Chin, M. S. Brown, and D. Suter, "As-projective-as-possible image stitching with moving DLT," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 2339-2346.
[57] C. Herrmann et al., "Robust image stitching with multiple registrations," in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 53-67.
[58] B. Han and X. Lin, "A novel hybrid color registration algorithm for image stitching," IEEE Transactions on Consumer Electronics, vol. 52, no. 3, pp. 1129-1134, 2006.
[59] A. Laraqui, A. Baataoui, A. Saaidi, A. Jarrar, M. Masrar, and K. Satori, "Image mosaicing using voronoi diagram," Multimedia Tools and Applications, vol. 76, no. 6, pp. 8803-8829, 2017.
[60] A. Baataoui, A. Laraqui, A. Saaidi, K. Satori, A. Jarrar, and M. Masrar, "Image Mosaicing Using a Self-Calibration Camera," 3D Research, vol. 6, no. 2, p. 19, 2015.
[61] M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," International journal of computer vision, vol. 74, no. 1, pp. 59-73, 2007.
[62] R. Szeliski, H.-Y. Shum, H.-Y. Shum, and H.-Y. Shum, "Creating full view panoramic image mosaics and environment maps," in Proceedings of the 24th annual conference on Computer graphics and interactive techniques, 1997: ACM Press/Addison-Wesley Publishing Co., pp. 251-258.
[63] S. E. Chen, "Quicktime VR: An image-based approach to virtual environment navigation," in Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, 1995: ACM, pp. 29-38.
[64] Q. Zhi and J. R. Cooperstock, "Depth-based image mosaicing for both static and dynamic scenes," in 2008 19th International Conference on Pattern Recognition, 2008: IEEE, pp. 1-4.
[65] A. Bartoli, N. Dalal, and R. Horaud, "Motion panoramas," Computer Animation and Virtual Worlds, vol. 15, no. 5, pp. 501-517, 2004.
[66] H. S. Sawhney and S. Ayer, "Compact representations of videos through dominant and multiple motion estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 814-830, 1996.
[67] M. Irani, S. Hsu, and P. Anandan, "Video compression using mosaic representations," Signal Processing: Image Communication, vol. 7, no. 4-6, pp. 529-552, 1995.
[68] X. Gu et al., "Dynamic image stitching for moving object," in 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2016: IEEE, pp. 1770-1775.
[69] K. S. Bhat, M. Saptharishi, and P. K. Khosla, "Motion detection and segmentation using image mosaics," in 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No. 00TH8532), 2000, vol. 3: IEEE, pp. 1577-1580.
[70] M. Ramachandran and R. Chellappa, "Stabilization and mosaicing of airborne videos," in 2006 International Conference on Image Processing, 2006: IEEE, pp. 345-348.
[71] A. Mills and G. Dudek, "Image stitching with dynamic elements," Image and Vision Computing, vol. 27, no. 10, pp. 1593-1602, 2009.
[72] L. Zeng, S. Zhang, J. Zhang, and Y. Zhang, "Dynamic image mosaic via SIFT and dynamic programming," Machine vision and applications, vol. 25, no. 5, pp. 1271-1282, 2014.
[73] Y. Tang, J. Shin, and H.-C. Liao, "De-ghosting method for image stitching," International Journal of Digital Content Technology and its Applications, vol. 6, no. 18, p. 17, 2012.
[74] Y. Tang and H. Jiang, "Highly efficient image stitching based on energy map," in 2009 2nd International Congress on Image and Signal Processing, 2009: IEEE, pp. 1-5.
[75] B. L. Jovanovski and J. Li, "Image-stitching for dimensioning," 2019.
[76] J. Zheng, Y. Wang, H. Wang, B. Li, and H.-M. Hu, "A Novel Projective-Consistent Plane based Image Stitching Method," IEEE Transactions on Multimedia, 2019.
[77] S. Gong, C. Liu, Y. Ji, B. Zhong, Y. Li, and H. Dong, "Image Stitching," in Advanced Image and Video Processing Using MATLAB: Springer, 2019, pp. 271-327.
[78] C. Zhao, H. Zhang, J. Chen, and W. Fu, "Region-based parallax-tolerant image stitching," in Tenth International Conference on Graphics and Image Processing (ICGIP 2018), 2019, vol. 11069: International Society for Optics and Photonics, p. 1106909.
[79] P. Liu, J. Han, F. Tian, Z. Wu, and J. Wang, "Research on image stitching technology for focal plane array terahertz imaging," in 9th International Symposium on Advanced Optical Manufacturing and Testing Technologies: Optoelectronic Materials and Devices for Sensing and Imaging, 2019, vol. 10843: International Society for Optics and Photonics, p. 108430Z.
[80] R. Nathaniel, D. Rappaport, and I. Goldin, "Method and system for stitching multiple images into a panoramic image," ed: Google Patents, 2015.
[81] Introduction to Dynamic Programming [Online]. Available: http://mirlab.org/jang/books/dcpr/dp.asp?title=8-1%20Introduction%20to%20Dynamic%20Programming%20(%B0%CA%BAA%B3W%B9%BA).
[82] Baeldung. (2018). Dijkstra Algorithm in Java [Online]. Available: https://www.baeldung.com/java-dijkstra.
[83] X. Huang, L. Zhang, and T. Zhu, "Building change detection from multitemporal high-resolution remotely sensed images based on a morphological building index," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 7, no. 1, pp. 105-115, 2013.
[84] (2008). Graph-Cut [Online]. Available: https://www.sciencedirect.com/topics/engineering/graph-cut.
[85] C. S. Xydeas and V. S. Petrovic, "Objective pixel-level image fusion performance measure," in Sensor Fusion: Architectures, Algorithms, and Applications IV, 2000, vol. 4051: International Society for Optics and Photonics, pp. 89-99.
[86] Y. Han, Y. Cai, Y. Cao, and X. Xu, "A new image fusion performance metric based on visual information fidelity," Information fusion, vol. 14, no. 2, pp. 127-135, 2013.
[87] C. Yang, J.-Q. Zhang, X.-R. Wang, and X. Liu, "A novel similarity based quality metric for image fusion," Information Fusion, vol. 9, no. 2, pp. 156-160, 2008.
[88] M. Brown and D. G. Lowe, "Invariant features from interest point groups," in BMVC, 2002, vol. 4.
[89] K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," IEEE transactions on pattern analysis and machine intelligence, vol. 27, no. 10, pp. 1615--1630, 2005.
[90] Wikipedia. (2008). Speeded up robust features [Online]. Available: https://en.wikipedia.org/wiki/Speeded_up_robust_features.
[91] U. o. California. (2010). Histogram Equaliztion [Online]. Available: https://www.math.uci.edu/icamp/courses/math77c/demos/hist_eq.pdf.
[92] Matlab. (2012). Histogram Matching [Online]. Available: https://www.mathworks.com/help/images/ref/imhistmatch.html.
[93] Feature Matching [Online]. Available: https://sites.google.com/a/mines.sdsmt.edu/rijvi_ahmed/computer-vision/project_1/feature-matching.
[94] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE transactions on image processing, vol. 13, no. 4, pp. 600-612, 2004.