臺灣博碩士論文加值系統

專為個人電腦以及小型電腦環境所制定的 SCSI 匯流排架構，在其擴充性以及其排線的最大連接距離上，皆有其限制。隨著利用網路來達到檔案分享的需求增加，各種儲存技術也存在已久。iSCSI (Internet SCSI) 是一種新的通訊協定，使得 SCSI 命令 (command) 可以利用 TCP/IP 通訊協定而在網路上傳遞。相較於NAS ，iSCSI 採用區塊存取 (block-oriented I/O ) 來直接控制 SCSI 設備。iSCSI 使得檔案系統以及資料庫可以直接使用 TCP/IP 網路的服務。相較於 SAN，iSCSI 提供了一個相當低價的選擇以及近乎完全相同的功能。
本論文介紹了 iSCSI 通訊協定的基本觀念，並且完整說明了 iSCSI initiator 端及 iSCSI target 端之設計。我們的平台是根據 iSCSI 通訊協定的第六版草稿建構而成的。我們採用 Linux kernel 2.4 以及 SCSI-II 的設備，成功地開發了一個 iSCSI 的原型平台。所有的程式皆是以 C 語言完成 (約 10,000 行程式碼)。我們的目的在於利用 TCP/IP 網路來實作出這個嶄新的通訊協定，並利用此平台來找出其架構上的限制和效能的瓶頸為何。根據這些經驗，我們可以達到加強現有儲存網路設備機能之目的。
本論文先介紹 DAS，NAS，SAN 和 iSCSI的基本觀念。第二章則詳細敘述與 iSCSI 平台相關 Linux核心部分以及 Linux SCSI 系統。第三章及第四章則分別敘述了 initiator 和 target 端架構和開發考量。在第四章，本論文並且提出了一個在記憶體快取不一致的問題，而其原因是因為檔案系統在開發時所做的一些假設。我們提出了一些建議來處理這個問題。第五章則是 iSCSI 與 NFSv3 效能的比較結果。最後，第六章對本論文做一個總結，並且點出我們未來應繼續研究的方向。

SCSI bus, designed for the PC and small computer environment, has its limitations to scalability and its maximum distance. With the need to share files over networks increasing, several storage technologies have existed for a long time. iSCSI (Internet SCSI) is a new protocol that allows SCSI commands to be sent by exploiting the TCP/IP networks. Compared to the NAS appliances, iSCSI using Block-oriented I/O operations gets the direct control over all low-level SCSI hardware. iSCSI makes local file systems and database systems over TCP/IP networks possible. Compared to SAN, iSCSI provides a much lower-cost option and almost the same functionality.
This thesis presents a fundamental introduction to iSCSI protocol and a comprehensive overview of the iSCSI initiator and target implementations based on version 6 of iSCSI draft. An iSCSI prototype has been implemented successfully on Linux Kernel 2.4.2 with parallel SCSI-II devices. All programs are written in C language (about 10,000 lines of code). Its goal is to implement this emerging protocol on the TCP/IP networks and look for its limitations and performance bottlenecks in order to enhance the functionality of network storage service.
The thesis begins with a background over DAS, NAS, SAN, and iSCSI to give fundamental concepts of storage technologies. The chapter 2 describes the Linux kernel components related to iSCSI platform and Linux SCSI subsystem in detail. The chapter 3 and chapter 4 describe the initiator and target implementations respectively. In chapter 4, it also presents a problem in the cache consistency because of file system assumptions, and gives suggestions to address it. The chapter 5 is experimental results of iSCSI performance compared to NFSv3. Finally chapter 6 figures out the future work and gives a conclusion.

ABSTRACT
CONTENTS
CHAPTER 1 STORAGE TECHNOLOGY 1
1.1 Directed Attached Storage (DAS) 1
1.2 Network Attached Storage (NAS) 3
1.3 Storage Area Network (SAN) 5
1.4 A Emerging Protocol for Storage Networking: iSCSI 6
1.5 iSCSI is A Block-oriented Protocol 8
1.6 A Summary of This Chapter 9
CHAPTER 2 LINUX KERNEL COMPONENTS RELEVANT to iSCSI 10
2.1 Monolithic and Microkernel Operating Systems 10
2.2 Kernel Modules 11
2.3 Linux SCSI Subsystem 12
2.4 Ext2 File System 14
2.5 Block Device Driver and Buffer Cache 17
2.6 Reading An Ext2 Regular File Stored in A SCSI Hard Drive 19
CHAPTER 3 iSCSI INITIATOR 25
3.1 iSCSI Initiator in Linux kernel 2.4 25
3.2 iSCSI Initiator Initialization 28
3.3 iSCSI Initiator in Full-feature Phase 33
3.4 iSCSI Initiator Uninstallation 40
3.5 Step by Step 41
3.6 Summary 42
CHAPTER 4 iSCSI TARGET 43
4.1 iSCSI Target Design Overview 43
4.2 iSCSI Target Initialization 46
4.3 iSCSI Target Session in Login Phase 49
4.4 iSCSI Target Session in Full-feature Phase 51
4.5 iSCSI Target Uninstallation 52
4.6 ReceiveFromInitiator( ) and SendToTarget( ) Functions 53
4.7 Buffer Cache Inconsistency 56
CHAPTER 5 EXPERIMENTAL RESULTS 60
5.1 Performance Comparison between iSCSI and NFSv3 60
5.2 IOzone Benchmark 64
CHAPTER 6 CONCLUSION and FUTURE WORK 82
6.1 Conclusion 82
6.2 Future Work 83
APPENDIX A SCSI I/O INTERFACE 85
A.1 A Broad Overview of SCSI 85
A.2 SCSI-II Architecture and Commands for Devices 87
APPENDIX B DEVELOPMENT PLATFORM 89
REFERENCES 91

Source Code
[1] Linux kernel 2.4.2. http://www.kernel.org/
[2] Kernel httpd accelerator. The directory /linux/net/khttpd/ in Linux source.
[3] Wu-FTP. http://www.wu-ftpd.org/
Specification and Draft
[4] ANSI. “Small Computer System Interface (SCSI) Specification,” ANSI X3.131-1986, 1986.
[5] iSCSI Internet Draft. http://www.ietf.org/ids.by.wg/ips.html
Book
[6] Uresh Vahalia. “Unix Internals: The New Frontiers,” Prentice-Hall Inc., 1996.
[7] Daniel P. Bovet and Macro Cesati. “Understanding Linux Kernel: From I/O Port to Process Management,” O’Reilly & Associates, Inc., 2001.
[8] Scott Maxwell. “Linux Core Kernel Commentary,“ 1st Edition, Coriolis Group, 1999.
[9] Alessandro Rubini and Jonathan Corbet. “Linux Device Drivers,” 2nd Edition, O’Reilly & Associates, Inc., June 2001.
[10] Michael Beck, Harald Bohme, Rober Magnus, Mirko Dziadzka, and Ulrich Kunutz. “Linux Kernel Internals,” 1st Edition, Addison-Wesley Inc., 1999.
[11] Marshall Kirk McKusick, Keith Bostic, Michael J. Karels, and John S. Quarterman. “The Design and Implementation of the 4.4BSD Operating System,” Addison-Wesley Inc., 1996.
[12] Andrew S. Tanenbaum and Albert S. Woodhull. “Operating Systems: Design and Implementation”, 2nd Edition, Prentice-Hall Inc., 1997.
[13] Peter H. Salus. “A Quarter Century of UNIX,” Addison-Wesley, June 1994.
[14] Brent Callaghan. “NFS Illustrated,” Addison-Wesley, 2000.
[15] Friedhelm Schmidt. “The SCSI Bus & IDE Interface: Protocols, Applications & Programming,” 2nd Edition, Addison-Wesley, 1995.
[16] W. Richard Stevens. “UNIX Network Programming Volume 1: Networking APIs: Sockets and XTI,” 2nd Edition, Prentice-Hall Inc., 1998.
[17] W. Richard Stevens. “UNIX Network Programming Volume 2: Interprocess Communications,” 2nd Edition, Prentice-Hall Inc., 1999.
[18] W. Richard Stevens. “TCP/IP Illustrated Volume 1: The Protocols,” Addison-Wesley Inc., 1994.
[19] Gary R. Wright, W. Richard Stevens. “TCP/IP Illustrated Volume 2: The Implementation,” Addison-Wesley Inc., 1995.
[20] W. Richard Stevens. “Advanced Programming in the UNIX Environment,” Addison-Wesley Inc., 1993.
Paper and Thesis
[21] J. Morris, M. Satyanarayanan, M. Conner, J. Howard, D. Rosenthal, and F. Smith. “Andrew: A Distributed Personal Computing Environment,” Communications of the ACM, vol. 29, no. 3, pp. 184-201, March 1986.
[22] M. L. Kazar. “Synchronization and Caching Issues in the Andrew File System,” Proceedings of the USENIX Winter Technical Conference, February 1988.
[23] M. Satyanarayanan. “Coda: A Highly Available File System for a Distributed Workstation Environment,” IEEE Transactions on Computers, 1990.
[24] Brian Pawlowski, Chet Juszczak, Peter Staubach, Carl Smith, Diane Lebel, and David Hitz. “NFS Version 3 Design and Implementation,” Proceedings of the USENIX Summer Technical Conference, June 1994.
[25] R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh and B. Lyon, “Design and Implementation of the Sun Network File system,” Proceedings of the USENIX Summer Technical Conference, June 1985.
[26] R. Macklem. “Not quite NFS, soft cache consistency for NFS,” Proceedings of the USENIX Winter Technical Conference, January 1994.
[27] Uresh Vahalia, Cary G. Gray, and Dennis Ting. “Metadata Logging in an NFS Server,” Proceedings of the USENIX Technical Conference, January 1995.
[28] Chet Juszczak. “Improving the Write Performance of an NFS Server,” Proceedings of the USENIX Winter Technical Conference, January 1994.
[29] M. L. Kazar, et al. “Decorum File System Architectural Overview,” Proceedings of the USENIX Summer Technical Conference, June 1990.
[30] J. H. Hartman and J. K. Ousterhout. “The Zebra Striped Network File System,” 14th SOSP, December 1993.
[31] G. Gibson, et al. “File Server Scaling with Network-Attached Secure Disks,” ACM SIGMETRICS, June 1997.
[32] G. Gibson, et al. “A Cost-Effective, High-Bandwidth Storage Architecture,” ACM Symposium on Operating Systems Principles (ASPLOS), 1998.
[33] Margo Seltzer, Keith Bostic, Marshall Kirk McKusick, and Carl Staelin. “An Implementation of a Log-Structured File System for UNIX,” Proceedings of the USENIX Winter Technical Conference, January 1993.
[34] Ann L. Drapeau. et al. “RAID-II: A High-Bandwidth Network File Server,” Proceedings of the 21st Annual International Symposium on Computer Architecture (ISCA), 1994, pp. 234-244.
[35] Mary G. Baker, John H. Hartman, Michael D. Kupfer, Ken W. Shirriff, and John K. Ousterhout. “Measurements of a Distributed File System,” ASPLOS, 1991.
[36] S. Soltis, G. Erickson, K. Preslan, T. Ruwart, M. O'Keefe. “The Global File System: A File System for Shared Disk Storage,” IEEE Transactions on Parallel and Distributed Systems, October 1997.
[37] L. McVoy and S. Kleiman. "Extent-like Performance from a Unix File System," Proceedings of the USENIX Winter Technical Conference, January 1991.
[38] John Ousterhout. “Why Aren't Operating Systems Getting Faster As Fast as Hardware,” Proceedings of the USENIX Summer Technical Conference, 1990.
[39] Margo Seltzer, Keith A. Smith, Hari Balakrishnan, Jacqueline Chang, Sara McMains, and Venkata Padmanabhan. “File System Logging Versus Clustering: A Performance Comparison,” Proceedings of the USENIX Winter Technical Conference, 1995.
[40] Mark Wittle and Bruce E. Keith. “LADDIS: The Next Generation In NFS File Server Benchmarking,” Proceedings of the USENIX Summer Technical Conference, 1993.
[41] Edward Growchowski. “Emerging Trends in Data Storage on Magnetic Hard Disk Drives,” In Datatech, ICG Publishing, pp. 11-16. September 1998.
[42] Chris Ruemmler and John Wilkes. “An introduction to disk drive modeling,” IEEE Computer, March 1994.
[43] John H. Howard, Michael L. Kazar, Sherri G. Menees, David A. Nichols, M. Satyanarayanan, Robert N. Sidebotham, and Michael J. West. “Scale and Performance in a Distributed File System,” ACM Transactions on Computer Systems, 6(1): 51-81, February 1988.
Network Resource
[44] iSCSI Mail List. http://www.pdl.cmu.edu/mailinglists/ips/mail/maillist.html
[45] Web Site of IBM Storage Networking. http://www.storage.ibm.com/snetwork/nas/
[46] Official Web Site of NASD. http://www.nsic.org/nasd
[47] NAS Manufactures. http://www.storagesearch.com/nas.html
[48] G. Gibson, et al. “Filesystems for Network-Attached Secure Disks,” Technical Report CMU-CS-97-118, July 1997.
[49] G. Gibson, et al. “A Case for Network-Attached Secure Disks,” Technical Report CMU-CS-96-142, 1996.
[50] Official Web Site of Minix. http://www.cs.vu.nl/~ast/minix.html
[51] Linux kernel 2.4. “Linux SCSI-2.4-HOW-TO,” 2001
[52] Linux kernel 2.4. “SCSI-Programming-HOW-TO,” 2001.
[53] Rickart E. Faith. “Writing a SCSI Device Driver for Linux,” 1993.
[54] Dave Rusling. “Linux Kernel Document,” 1996.
[55] Seagate Corporation. “Barracuda 180 (ST1181677FCV)”, March 2002.
[56] Seagate Corporation. “Barracuda ATA IV (ST380021A)”, March 2002.
[57] Seagate Corporation. “DISC Products,” March 2002.
[58] ITcentrix Inc. “Customer Requirements for Storage Networks: Survey Results,” http://www.itcentrix.com/Archive/0007/SANStudy/SurveyResults.htm
[59] Storage Networking Industry Association (SINA). http://www.snia.org/
[60] INTEL. “Intel Architecture Software Developer’s Manual. Volume 3: System Programming,” http://developer.intel.com
[61] INTEL. “Intel Architecture Software Developer’s Manual. Volume 1: Basic Architecture,” http://developer.intel.com
[62] INTEL. “Intel Architecture Software Developer’s Manual. Volume 2: Instruction Set Reference,” http://developer.intel.com
[63] IP Security Protocol (IPSec). http://www.ietf.org/html.charters/ipsec-charter.html
[64] Alessandro Rubini. “Making System Calls from Kernel Space,” http://www.linux-mag.com/2000-11/gear_01.html, GEARHEADS ONLY series of Linux Magazine, November 2000.
[65] Microsoft Windows Driver Development Kits. http://www.microsoft.com/ddk/
[66] IOzone benchmark. http://www.iozone.org/