臺灣博碩士論文加值系統

由於雲端運算可提供隨需的服務且有效降低IT設備的成本，近幾年來雲端運算愈來愈受歡迎。虛擬化技術是雲端運算重要的技術之一，它有CPU虛擬化、記憶體虛擬化、I/O虛擬化、儲存虛擬化等等各種虛擬化。虛擬化的核心概念主要是提供一個硬體資源的抽象層，所以它需要額外增加一虛擬層在硬體設備之上，正因為如此，經過虛擬化的硬體效能會降低。在各種虛擬化中以I/O虛擬化效能下降的情況最嚴重，經常在雲端運算成為效能的瓶頸。
InfiniBand 是一種擁有高頻寬、低延遲的高速網路，它廣泛用於高效能運算的領域。在此論文，我們提出一套針對KVM虛擬機器管理者的InfiniBand虛擬化系統─Virt-IB。Virt-IB主要由VM IB library和以virtio為基礎架構的Virt-IB驅動程式所構成。本論文研究兩種做法。第一種做法採用間接執行API的方法，VM IB library提供API介面並傳遞API所需的參數給Virt-IB驅動程式，然後由其在實體機器上執行API後，傳遞回傳值。透過這種方式，基礎Virt-IB的效能只達到實體InfiniBand設備的百分之三十。第二種做法是直接在虛擬機上執行API，然後再間接使用InfiniBand設備進行傳輸。VM IB library不僅提供API介面，同時也具備使用者InfiniBand驅動程式的作用，然後Virt-IB驅動程式提供溝通管道讓VM IB library間接將指令寫入InfiniBand設備。第二種做法的Virt-IB達到實體InfiniBand百分之五十的效能，同時也達到改善I/O虛擬化效能的目的。

With the ability to provide on-demand service and to reduce the IT cost, cloud computing becomes more and more popular recently. Virtualization is one of the important technologies in cloud computing, whose main idea is to provide abstractions of the physical resources. However, such abstraction can cause performance degradation, especially for /IO virtualization, which is usually the performance bottleneck in cloud computing.
InfiniBand is a network system that provides very low latency (less than 5us) and very high bandwidth (multiple Gbps). Due to its excellent performance, InfiniBand is commonly used in high performance computing (HPC) area. In this thesis, we propose Virt-IB for InfiniBand virtualization on Kernel Virtual Machine (KVM). The main components of Virt-IB are VM IB library and Virt-IB driver. Two different Virt-IBs designs were studied and implemented in this thesis. The first design is to execute InfiniBand API indirectly. VM IB library provides API interface and passes all the necessary parameters to Virt-IB driver. Then, Virt-IB driver performs the operation of API. The second design processes the API directly in guest VM and communicates with InfiniBand device indirectly to perform the real operations. VM IB library provides API interface and user-level InfiniBand driver. Virt-IB driver provides a channel for VM IB library to write commands into InfiniBand device indirectly.
Evaluation results show that the first method only achieves about 30% performance of native InfiniBand. And the second method can achieve 50% performance of the native InfiniBand. At the same time, we improve the performance of I/O virtualization.

中文摘要 i
ABSTRACT ii
Acknowledgments iv
TABLE OF CONTENTS v
LIST OF FIGURES vii
Chapter 1 Introduction 1
Chapter 2 Related Work 5
2.1 Overview of Kernel-based Virtual Machine 5
2.2 InfiniBand Architecture 10
2.3 Existing Methods for InfinIBand Virtualization 13
Chapter 3 Design and Implementation 15
3.1 Design overview 15
3.2 Components of Virt-IB 17
3.2.1 VM IB Library 17
3.2.2 Virt-IB Driver 18
3.2.3 Work Flow of Virt-IB 19
3.3 Additional InfiniBand Operations of Virt-IB 21
3.3.1 Resources Creation 21
3.3.2 Memory Registration 21
3.3.3 Posting Work Requests 22
3.3.4 Polling for Completion 22
Chapter 4 The Improved Design and Implementation 24
4.1 The Problem of the Original Design 24
4.2 The Improved Design and Implement of Virt-IB 25
4.3 Differences between Original Virt-IB and Improved Virt-IB from the View of InfiniBand Operations 30
4.3.1 Resources Creation 31
4.3.2 Memory Registration 31
4.3.3 Posting Work Requests 32
4.3.4 Polling for Completion 33
Chapter 5 Performance Results 35
5.1 Experiment Environment Setup 35
5.2 Performance Comparison between Virtual Network and Virt-IB 35
5.3 Performance Results and Analysis for Virt-IB 37
Chapter 6 Conclusion and Future Work 41
Reference 42

[1] "Direct memory access". en.wikipedia.org/wiki/Direct_memory_access.
[2] "Iperf". http://en.wikipedia.org/wiki/Iperf.
[3] "PCI SIG I/O Virtualization (IOV) Specifications".
[4] "QEMU". http://wiki.qemu.org/Main_Page.
[5] "TOP500 List". http://www.top500.org/.
[6] "Virtualization". http://en.wikipedia.org/wiki/Virtualization.
[7] Avi Kivity, Y. K., Dor Laor,Uri Lublin, Anthony Liguori. "kvm: the Linux virtual machine monitor". in Ottawa Linux Symposium ,July 2007.
[8] InfiniBand™ Trade Association, "InfiniBand™ Architecture Specification Volume 1,Release 1.2.1", January 2008.
[9] J. Liu, W. H., B. Abali and D. K. Panda, "High Performance VMM-Bypass I/O in Virtual Machines", in USENIX Annual Technical Conferencepp, 29-42, June 2006.
[10] Jones, M. T. "Virtio: An I/O virtualization framework for Linux". http://www.ibm.com/developerworks/linux/library/l-virtio/.
[11] Lucas Nussbaum, F. A., Olivier Mornard,Jean-Patrick Gelas. "Linux-based virtualization for HPC clusters". in Proceedings of the Linux Symposium, July 2009.
[12] Mellanox InfiniBand. http://mellanox.com/.
[13] Mellanox Technologies. "Mellanox IB-Verbs API (VAPI), Rev.1.00".
[14] OpenFabrics Alliance. https://www.openfabrics.org/index.php.
[15] Peter Mell, T. G., "The NIST Definition of Cloud Computing", National Institute of Standards and Technology: NIST Special Publication, September 2011.
[16] Qumranet Inc., "KVM: Kernel-based Virtualization Driver", 2006.
[17] Russell, R., "virtio: towards a de-facto standard for virtual I/O devices". ACM SIGOPS Operating Systems Review. 42(5),July 2008.
[18] Shafer, J. "I/O virtualization bottlenecks in cloud computing today". in WIOV'10 Proceedings of the 2nd conference on I/O virtualization. 2010.
[19] W. Huang, J. L., B. Abali and D. K. Panda, "A Case for High Performance Computing with Virtual Machines", in Int'l Conference on Supercomputing (ICS), pp. 125-134. June 2006.