臺灣博碩士論文加值系統

隨著全球行動用戶數的逐年增長，許多技術及應用也紛紛遷移到手機上，例如：face morphing、online game、image processing和mobile messenger…等等；本研究的目標是想在行動裝置上，發展一套場景瀏覽系統，使用者只需要在無線網路環境底下，便可以瀏覽有興趣的環境空間。但要在行動裝置上即時繪製複雜的3D場景，卻是一項很大的挑戰，最主要是因為行動裝置本身的計算能力受限，加上仍缺乏3D繪圖運算能力的硬體支援；因此，我們的研究主要便針對運算能力受限的行動裝置，在上面提供以圖像為基礎的空間場景走訪(image-based walkthrough)系統。
我們的系統主要是以Server/Client的架構來建製：HTC P36511為本研究作為Client端使用的實體行動裝置、並以我們的校園環境作為實驗對象。首先伺服器會收集使用者所提供的場景照片，並運用以SIFT (Scale Invariant Feature Transform)為底層技術的環景接圖產生器，透過鄰近照片之間特徵比對的方式將它們接合成全景影像，用此來建製某個區域的場景空間。之後為了配合行動裝置上的硬體限制，在伺服器端發展on-demand transmission機制，對全景圖作適當地分割及壓縮，再將使用者有興趣觀看的場景圖分段分次、正確而即時地傳送到行動裝置上，透過該機制平均能降低53.2%的檔案傳輸量；之後透過OpenGL ES程式資料庫來繪製介面及展示結果，並藉由實作Image Combination加速1.5倍的整體運算時間，最終便能讓使用者自由地在影像虛擬環境中走訪。不同於其他只有固定走訪路徑的相似研究(比如Google Street View)，我們藉由分析各個環景影像內容之間的一致性，建構出較自由及有彈性的走訪路徑。我們的系統在未來能延伸應用在現今路況環境資訊仍顯不足的GPS導航系統上，提供使用者一套擁有更清楚明確路徑資訊及實際環境的導航系統。

As mobile phones became ubiquitous over the past decade, many techniques and applications have been migrated to mobile devices to provide new functionalities such as face morphing, online games, image processing, and mobile messenger. The purpose of this research is to develop a space browsing system, in which user can navigate the scene of interest under a wireless environment on mobile devices. Interactively rendering complex 3D scenes on mobile devices is a challenge, due to that a mobile device is restrained by its limited computing power, memory capacity, and the lack of 3D rendering hardware to accelerate computation. This research therefore focuses on providing an image based approach that renders scenes in handhelds using such limited computing capabilities.

Our system is based on a Server/Client architecture, with the client implemented on HTC P36511 device, utilizing our campus environment as the test bed. In it, Server provides a panorama generator using SIFT (Scale Invariant Feature Transform). After collecting photographs of a scene supplied by user, the generator stitches a series of overlapping photographs into a panorama through detecting feature points between neighboring photographs and constructs the parts of the virtual scene space. To accommodate the limitations of mobile devices, we develop on-demand transmission on server, which cuts the panorama into smaller segments of a suitable size and compresses them. The parts of viewer’s interest are sent to the client, which is rendered in real-time on user interface using OpenGL ES while user browses through the panorama. On-demand transmission decreases at an average of 53.2% data size per transmission. We also implement Image Combination for rendering purposes, which speeds up the whole system’s performance by 1.5 times. Even though the technical approach for designing and developing our system is similar to that of Google Street View, we provide users with flexible and dynamic routes for the walkthrough. Note that most current GPS navigation systems do not provide sufficient scene information. In the future, our system can be applied and ported on such systems to display real world’s scene.

Chapter 1 Introduction................................1
1.1 Research Motivation and Objective.............................................1
1.2 Research Contribution.............................3
Chapter 2 Related Works...............................5
2.1 Image-Based Modeling (IBM)........................5
2.2 Image-Based Rendering (IBR).......................9
2.3 Image Browsing...................................12
2.4 Remote Walkthrough System........................12
2.5 Walkthrough System on Mobile Devices.............13
2.6 PhotoSynth: Microsoft Live Labs..................14
2.7 Google Street View...............................15
2.8 QuickTime VR.....................................16
Chapter 3 System Architecture........................19
3.1 System Overview.........................19
3.2 Preprocessing Unit......................20
3.3 Server..................................22
3.4 Client..................................24
3.5 Client-Server Interaction........................26
Chapter 4 Methodology................................28
4.1 Data Streaming Method............................28
4.2 Detecting Difference of Images...................32
4.3 UL-map...........................................34
4.3.1 UL-map Drawing Flow...................35
Chapter 5 Results and Analysis.......................37
5.1 Data Streaming Performance..............37
5.2 Image Combination.......................38
5.3 Constructing Flexible Walkthrough Routes with Panorama Analysis....................................40
5.4 User Interface..........................44
Chapter 6 Conclusion and Future Work.................47
References...........................................50
Biography............................................56

[1] J.B. Lee, S.K. Lee, and J.D. Kim, T.-Y, “Evaluation of technological innovation in the cellular phone display”, In Proceedings of Portland International Conference on Management of Engineering and Technology, Portland International Conference, pp. 140- 149, 2003.
[2] D.G. Lowe, "Distinctive image features from scale-invariant keypoints”, In International Journal of Computer Vision, pp. 91–110, 2004.
[3] R.I. Hartley and A. Zisserman, “Multiple view geometry”, In Cambridge University Press, Cambridge, UK, 2004.
[4] M. Brown and D.G. Lowe, "Unsupervised 3D object recognition and reconstruction in unordered datasets", In International Conference on 3-D Digital Imaging and Modeling, Ottawa, Canada, 2005.
[5] F. Schaffalitzky and A. Zisserman, “Multi-view matching for unordered image sets, or .How do I organize my holiday snaps?”, In Proceedings of European Conference on Computer Vision, pp. 414-431, 2002.
[6] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and rendering architecture from photographs: A hybrid geometry and image-based approach”, In SIGGRAPH Conf. Proc., 1996.
[7] A.R. Dick, P.H.S. Torr, and R. Cipolla, “Modelling and interpretation of architecture from several images”, In International Journal of Computer Vision, 2004.
[8] S. Teller, M. Antone, Z. Bodnar, M. Bosse, S. Coorg, M. Jethwa, and N. Master, “Calibrated, registered images of an extended urban area”, In International Journal of Computer Vision, pp. 93-107, 2003.
[9] www.cc.gatech.edu/4d-cities
[10] A. Roman, G. Garg, and M. Levoy, “Interactive design of multi-perspective images for visualizing urban landscapes”, In Proceedings of IEEE Visualization, pp. 537-544, 2004.
[11] A. Agarwala, M. Agrawala, M. Cohen, D. Salesin, and R. Szeliski, “Photographing long scenes with multi-viewpoint panoramas”, In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006), 2006.
[12] www.cs.unc.edu/Research/urbanscape
[13] A. Lippman, “Movie maps: An application of the optical videodisc to computer graphics”, In SIGGRAPH Conf. Proc., 1980.
[14] S.M. Seitz, and C.M. Dyer, “View morphing”, In SIGGRAPH Conf. Proc., 1996.
[15] D.G. Aliaga, T. Funkhouser, D. Yanovsky, and I. Carlbom, “Sea of images”, In IEEE Visualization VIS, pp. 331-338, 2002.
[16] L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-quality video view interpolation using a layered representation”, In SIGGRAPH Conf. Proc., pp. 600-608, 2004.
[17] C. Buehler, M. Bosse, L. Mcmillan, S. Gortler, and M. Cohen, “Unstructured lumigraph rendering”, In SIGGRAPH Conf. Proc., pp. 425-432, 2001.
[18] D. Gotz, K. Patel, and D. Manocha, “IRW: An incremental representation for image-based walkthroughs”, In ACM Multimedia, 2002.
[19] T. Takahashi, H. Kawasaki, K. Ikeuchi, and M. Sakauchi, “Arbitrary view position and direction rendering for large-scale scenes”, In IEEE CVPR, pp. 296-303, 2000.
[20] D. Lee and S. Jung, “Capture configuration for image-based street walkthroughs”, In Second International Conference on Cyberworlds (CW), pp. 151, 2003.
[21] Z.P. Hu, Y.J. He, and Z.Y. Ou, “A new IBR approach based on view synthesis for virtual environment rendering”, In IEEE ICAT, pp. 31-35, 2006.
[22] D. Aliaga, D. Yanovsky, T. Funkhouser, and I. Carlbom, “Interactive image-based rendering using feature globalization”, In Proceedings of SIGGRAPH Symposium on Interactive 3D Graphics, pp. 163-170, 2003.
[23] D. Aliaga and I. Carlbom, “Plenoptic stitching: A scalable method for reconstructing interactive walkthroughs”, In ACM SIGGRAPH, pp. 443-450, 2001.
[24] M. Uyttendaele, A. Criminisi, S.B. Kang, S. Winder, R. Szeliski, and R. Hartley, “Image-based interactive exploration of real-world environments”, In IEEE Computer Graphics and Applications, pp. 52-63, 2004.
[25] H.Y. Shum, S.B. Kang, and S.C. Chan, “Survey of image-based representations and compression techniques”, In IEEE Transactions on Circuits and Systems for Video Technology, pp. 1020-1037, 2003.
[26] N. Snavely, S.M. Seitz, and R. Szeliski, "Photo tourism: Exploring photo collections in 3D", In ACM Transactions on Graphics (SIGGRAPH Proceedings), pp. 835-846, 2006.
[27] K. Toyama, R. Logan, and A. Roseway, “Geographic location tags on digital images”, In Proc. Int. Conf. on Multimedia, pp. 156-166, 2003.
[28] M. Naaman, Y.J. Song, A. Paepcke, and H. Garciamolina, “Automatic organization for digital photographs with geographic coordinates”, In Proceedings of ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 53-62, 2004.
[29] N. Mccurdy, and W. Griswold, “A systems architecture for ubiquitous video”, In Proc. Int. Conf. on Mobile Systems, Applications, and Services, pp. 1-14, 2005.
[30] Y. Noimark and D. Cohen-Or, “Streaming scenes to MPEG-4 video enabled devices”, In IEEE Computer Graphics and Applications, pp. 58–64, 2003.
[31] J. Chim, R.W.H. Lau, H. V. Leong, and A. Si, “CyberWalk: A web-based distributed virtual walkthrough environment”, In IEEE Transactions on Multimedia, pp. 503–515, 2003.
[32] K. Engel, O. Sommer, and T. Ertl, “A framework for interactive hardware accelerated remote 3D visualization”, In Proceedings of EG/IEEE TCVG Symposium on Visualization, pp. 167–177, 2000.
[33] R. Bernardini, G. M. Cortelazzo, and G. Tormen, “IBR-based compression for remote visualization”, In Proceedings of 3D Data Processing Visualization and Transmission, pp. 513-519, 2002.
[34] R. Sato, S. Ono, H. Kawasaki, and K. Ikeuchi, “Real-time image-based rendering system for virtual city based on image compression technique and eigen texture method”, In International Conference on Pattern Recognition, pp. 1-4, 2008.
[35] Y. Lei, Z. Jiang, D. Chen, and H. Bao, “Image-based walkthrough over internet on mobile devices”, In Grid and cooperative computing GCC, pp. 728-735, 2004.
[36] Z. Jiang, Y. Mao, Q. Jia, N. Jiang, J. Tao, X. Fang, and H. Bao, “PanoWalk: A remote image-based rendering system for mobile devices”, Advances in Multimedia Information Processing PCM, pp. 641-649, 2006.
[37] Y. Lei, X. Zheng, Z. Jiang, and D. Chen, “Adaptive streaming panoramic video for image based rendering system”, In Proceeding of Networking, Sensing and Control, pp. 153-158, 2005.
[38] J.Y. Zheng, “Scene map on wireless mobile platform”, In Proceedings of the ACM symposium on Applied Computing, pp. 217-218, 2009.
[39] J.Y. Zheng and X. Wang, “View planning for cityscape archiving and visualization”, In Computer Vision ACCV, pp. 303-313, 2007.
[40] A. Mulloni, D. Nadalutti, and L. Chittaro, “Interactive walkthrough of large 3D models of buildings on mobile devices”, In Proceedings of the twelfth international conference on 3D web technology, pp. 17-25, 2007.
[41] http://livelabs.com/photosynth/
[42] http://maps.google.com/help/maps/streetview/
[43]http://ekstreme.com/thingsofsorts/fun-web/chatting-with-a-google-street-view-driver
[44] S.E. Chen, “Quicktime VR: An image-based approach to virtual environment navigation”, In Proc. SIGGRAPH, 1995.
[45] J. Canny, “Computational approach to edge detection”, In IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 679-698, 1986.
[46] C. Harris and M. Stephens, "A combined corner and edge detector”, In Proceedings of the 4th Alvey Vision Conference, pp. 147-151, 1988.
[47] S.N. Sinha, M. Pollefeys, and S.J. Kim, “High-resolution multiscale panoramic mosaics from pan-tilt-zoom cameras”, In Proceedings of Indian Conference on Computer Vision, Graphics and Image Processing ICVGIP, pp. 28-33, 2004.
[48] S.N. Sinha and M. Pollefeys, “Towards calibrating a pan-tilt-zoom camera network”, In Omnidirectional Vision and Camera Networks at ECCV, 2004.