Expert Reference Series of White Papers
`
`Fundamentals of
`SCSI and iSCSI
`
`1-800-COURSES
`
`www.globalknowledge.com
`
`Ex.1066.001
`
`DELL
`
`

`

`Fundamentals of SCSI and iSCSI
`
`Raymond B. Dooley, Global Knowledge Instructor, CCSI, CCNP, CCDP
`
`Introduction
`The computer world is full of buses. Most people are familiar with the universal serial bus (USB), which works
`fine for storing photos, and music files, but it is not big enough to support the types of file and data retrieval
`functions needed in a corporate data center. To support the retrieval of stored data from a disk, group of disks,
`optic memory array, or tape drives, another type of bus and protocol is required.
`
`Small Computer Systems Interface (SCSI) was originally a proprietary design by Shugart Associates in the 1970s.
`In 1986 the SCSI (pronounced scuzzy) standard was released by the American National Standards Institute
`(ANSI), and since then it has evolved through SCSI-1 and SCSI-2, to the current SCSI-3. This white paper will
`focus on:
`•
`Protocol Overview and Role
`•
`Parallel Bus Technology
`•
`Architecture Model
`•
`SCSI over Fibre Channel
`•
`SCSI Operations
`•
`Commands and Messages
`•
`Building an I/O Request
`•
`Internet SCSI (iSCSI)
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`2
`
`Ex.1066.002
`
`DELL
`
`

`

`SCSI Protocol Overview and Role
`
`Figure 1.
`
`SCSI defines the relationship between initiators (hosts) and targets (disk or tape). The SCSI-3 application resides
`in the host and provides middle-layer OSI Model (session and transport) services for upper layer file manage-
`ment protocols. Several components and options are defined:
`
`The Software initiator uses code in the OS (kernel-resident device driver) which uses the existing infra-
`structure. This is the most common type.
`
`The Hardware initiator uses dedicated hardware, typically in combination with firmware running on the
`hardware to implement SCSI. The hardware is the Host Bus Adapter (HBA), and is more common for servers
`in data centers.
`
`SCSI targets include the following:
`• A single disk
`•
`Just a Bunch of Disks (JBOD)
`• RAID - Random Array of Independent Disks
`•
`Tape Storage
`• Optic Memory
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`3
`
`Ex.1066.003
`
`DELL
`
`

`

`Each individual disk is identified in SCSI as a Logical Unit Number (LUN). SCSI is fast (320 Megabytes per
`second), but limited to 25 meters of cable.
`
`Parallel Bus Technology
`The SCSI parallel bus is either narrow (8 bits) or wide (16 bits), with wide used more often in modern systems.
`Data is sent simultaneously over multiple wires (in parallel, see Figure 2 below).
`
`Figure 2
`
`SCSI is half-duplex meaning data travels in one direction at a time, and only one device can control the bus at
`one time. A device must assume exclusive control over the bus in order to communicate. Another name for this
`is simplex communication (Like a Western Union telegram).
`
`The SCSI initiator selects the SCSI target and sends a command to initiate data transfer. At the end of the
`transfer, the bus is deselected and the bus is free.
`
`Data bits and control bits are sent on separate wires; this is called “out of band” control.
`
`Each SCSI device has a unique Identifier (ID). The initiator is always ID = 7, which is the highest priority. In a
`SCSI target, in addition to the ID, a LUN then identifies the specific target disk. The process requires an exchange
`of commands to and from the two addresses, then the transfer of data.
`
`SCSI-3 Architecture Model
`Figure 3 represents the SCSI-3 architecture model. The model consists of four layers of functionality (from the
`bottom of the graphic working up, with number one at the bottom of the graphic and number four at the top).
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`4
`
`Ex.1066.004
`
`DELL
`
`

`

`
`
`Figure 3.
`
`The physical interconnect layer defines the characteristics of the link between initiator and target.
`
`•
`•
`•
`•
`
`FC is fibre channel
`Serial Storage Architecture (SSA) is common in data centers where scalability is an issue
`IEEE 1394 is the Firewire specification
`SCSI Parallel Interface (SPI) is in every PC (on the far left side of the graphic)
`
`The transport protocol layer defines session management, which is techno-speak for the rules of the conversa-
`tion.
`
`•
`•
`•
`
`SCSI-FCP for fibre channel
`Serial Storage Protocol (SSP) for SAS devices
`Serial Bus Protocol (SBP) for IEEE 1394
`
`The shared command set layer consists of command sets for accessing storage resources (disk arrays, optic ar-
`rays, etc.).
`
`The SCSI Common Access Method (CAM) defines the SCSI drive application programming interface (API). APIs
`are used by application program developers to insure that the application written has proper access to storage.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`5
`
`Ex.1066.005
`
`DELL
`
`

`

`In a medium-to-large enterprise data center, there will be a fibre channel fabric. The SCSI initiator and target will
`communicate over this fabric. SCSI commands, data, and responses are carried in the payload of fibre channel
`frames. Fibre channel can be described as a very fast “pipe,” which never loses any data or becomes congested,
`that provides a conduit for the SCSI commands and traffic.
`
`In SCSI-FCP, the SCSI IDs are mapped to the unique worldwide name in each Fibre Channel port. This is what
`makes the “pipe” work.
`
`SCSI Operations
`SCSI specifies three phases of operation:
`
`1. Phase One is the Command Phase. In this phase the required commands and parameters are sent.
`2.
` Phase Two is the Data Phase. The transfer of data in accordance with the command occurs in this phase.
`3. Phase Three is the Response Phase. Confirmation of command execution is received in this phase.
`
`
`
`Figure 4.
`
`Each SCSI command is formatted as a Command Descriptor Block (CDB). The majority of the SCSI protocol is
`controlled by the initiator, while the target is passive and waits for a command. Only the initiator can initiate a
`SCSI operation by selecting a SCSI target and sending a CDB to it.
`
`If the CDB contains a Read command, the SCSI target moves its heads into position and retrieves the data from
`disk sectors. If the CDB contains a Write command, the SCSI target prepares its buffers and returns an indica-
`tion of ready. When the initiator receives the ready, the data is written to disk.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`6
`
`Ex.1066.006
`
`DELL
`
`

`

`When the operation is complete, the SCSI target returns a response to indicate a successful, or unsuccessful,
`data transfer.
`
`SCSI Commands
`SCSI supports several specific commands for each media type, and primary commands that all devices under-
`stand.
`
`The Session Layer (OSI layer 5) protocols are complex and require more command-level functionality than any
`other layer, with the possible exception of layer 7 (Application).
`
`Since SCSI performs session layer functions for data storage and retrieval, it is also complex. Table 1 shows a
`few of the many available commands, formatted as CDBs:
`
`Command
`Inquiry
`Report LUNs
`Test unit ready
`Report capacity
`Write buffer
`Read buffer
`
`Function
`What device are you?
`How many LUNs do you have?
`Is the LUN available?
`What size is each LUN?
`Write data to target LUN.
`Read data from target LUN.
`
`Table 1.
`
`SCSI Messages are an additional way for the initiator and target to communicate. Some SCSI transmission
`parameters are not tied to a specific command but to the initiator-target relationship. Several parameters,
`including transfer speed, data width, abort, and restore pointers, are established between initiator and target to
`maintain accurate information exchange.
`
`Building an I/O Request
`Figure 5, below, provides an example of an initiator talking to a target.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`7
`
`Ex.1066.007
`
`DELL
`
`

`

`Figure 5.
`
`The process breaks down into the following steps:
`1. The application makes a file I/O request to Volume Manager
`2. Volume Manager maps volume to SCSI ID and target LUN
`3. File System maps files to blocks, makes block I/O request
`4. Command, LBA (Logical Block Address), block count, and LUN sent to SCSI driver
`5. SCSI driver creates CDB
`6. FC (Fibre Channel) driver creates command frame with CDB in payload
`7. FC driver sends command frame to target LUN and awaits response
`
`As errors may occur in this process, the SCSI Retry command causes the entire operation to be repeated.
`
`Internet SCSI (iSCSI)
`SCSI specifications limit data transfer distance to 25 meters, but many data center design parameters require
`more distance between devices. iSCSI (Internet SCSI) allows initiator and target to exchange SCSI commands
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`8
`
`Ex.1066.008
`
`DELL
`
`

`

`using IP networks over TCP ports 860 and 3260. Since iSCSI requires no dedicated cabling (fibre channel) it is
`often viewed as a low-cost alternative to fibre channel (see Table 2 on page 11 for other comparison points).
`
`Figure 6.
`
`There are two major applications for iSCSI:
` Storage Consolidation - Organizations move disparate storage resources from servers around their
`1.
`network to central locations (data centers). This allows for more efficiency in the allocation of storage.
`
`2.
`
` Disaster Recovery - Organizations mirror storage resources from one data center to a remote data
`center, which can serve as a hot standby in the event of a prolonged outage. iSCSI SANs allow entire
`disk arrays to be migrated across a WAN with minimal configuration changes, in effect, making storage
`routable in the same manner as network traffic.
`
`Figure 7
`
`An interface from the SCSI device to the iSCSI world (network) is required. The simplest of these is the standard
`NIC (Network Interface Card). However, as volumes increase, the TCP/IP processing across the interface will
`overload the host.
`
`Another option is to build TCP/IP processing on the NIC, called a TCP Offload Engine (TOE). A TOE can be
`installed as partial offload (everything but error recovery) or full offload. To achieve maximum performance, the
`iSCSI HBA (Host Bus Adapter) is used.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`9
`
`Ex.1066.009
`
`DELL
`
`

`

`Figure 8.
`
`SCSI standards define a client server relationship between the initiator and target. iSCSI defines these as the
`Network Entity.
`
`The network entity contains an iSCSI node (either initiator or target), which is identified by an iSCSI node name.
`There are three formats:
`1.
` iSCSI Qualified Name (IQN) - It is a month and year date followed by a reverse domain name.
`(2010-04.com.acme:storage.tape.sys1.xyz)
`IEEE Extended Unique Identifier (EUI) - 64 bits
`2.
`3. T11 Network Address Authority (NAA) - 64 or 128 bits
`
`IQN names are the most commonly used. If the target node is a storage array, it can contain one or more SCSI
`LUNs.
`
`iSCSI initiator nodes communicate with iSCSI target nodes through network portals, identified by IP address.
`Network portals can be wireless ports.
`
`Table 2 provides a quick comparison of fibre channel and iSCSI as a SAN solution. What it does not show is that
`fibre channel can also run over IP (FCIP), which makes the choices even more challenging. One of the major
`disadvantages of iSCSI in the data center is meeting the scalability and performance required in high-volume
`environments.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`10
`
`Ex.1066.010
`
`DELL
`
`

`

`Fibre Channel
`Designed for enterprise markets
`High bandwidth of 1 Gbps, 2 Gbps, 4 Gbps
`
`Low latency of 2 msec per port
`Large payload of up to 2112 bytes
`
`Low overheads of 5.5% for 1KB payload, 3% for
`2KB payload
`Short distance of 10 km per link for single mode
`fiber
`High cost HBAs and switch ports
`
`iSCSI
`Designed for SMB markets
`Low bandwidth of 1 GigabitEthernet, 10 Giga-
`bitEthernet
`High latency; IP has msec latency
`Smaller payload of up to 1500 bytes; up to 9000
`bytes using jumbo frames
`Higher overheads of 8% for 1KB payload
`
`Long distance; no theoretical limit over IP net-
`works, but high latency
`Low cost: use existing NiC and LAN; iSCSI HBAs
`are expensive
`
`Table 2
`
`Another issue with all SAN solutions is network security. iSCSI supports authentication with CHAP (Challenge
`Handshake Authentication Protocol) but it must be properly implemented to avoid known vulnerabilities.
`
`Also, iSCSI is sent in clear text, and while IPSec may be used for encryption, it has presented problems when
`used with iSCSI. Server Message Encryption (SME) can be used in non-SCSI environments.
`
`Summary
`To recap, SCSI is present in all personal computers and has been a standard for file storage and retrieval for
`many years. It defines an initiator/target client/server relationship which includes addresses, commands, and
`specified operations.
`
`The SCSI model includes numerous interfaces including TCP/IP, fibre channel, directly attached parallel bus, and
`firewire. The many SCSI commands are formatted as CDBs. Fibre Channel and iSCSI are the two major choices
`for building out SCSI over a network. Both are present in modern data centers.
`
`Learn More
`Learn more about how you can improve productivity, enhance efficiency, and sharpen your competitive edge
`through training.
`Data Center Network infrastructure Design v2.0 (DCNID)
`Designing Cisco Storage Networking Solutions 4.2.1 (DCSNS)
`
`Visit www.globalknowledge.com or call 1-800-COURSES (1-800-268-7737) to speak with a Global Knowl-
`edge training advisor.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`11
`
`Ex.1066.011
`
`DELL
`
`

`

`About the Author
`Ray Dooley is a co-founder and CEO of ICM in Redmond, WA. He has been a Global Knowledge instructor since
`1994 and teaches the entire CCNP curriculum, plus advanced classes in MPLS, BGP, ATM, and Cisco Trouble-
`shooting. He has recently become involved in data center and storage network design.
`
`Copyright ©2011 Global Knowledge Training LLC. All rights reserved.
`
`12
`
`Ex.1066.012
`
`DELL
`
`