HIGH PERFORMANCE NETWORKS
`
`Technology and Protocols
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 1
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 1
`
`

`

`HIGH PERFORMANCE NETWORKS
`
`Technology and Protocols
`
`edited
`
`by
`
`Ahmed N. Tantawy
`IBM T. J. Watson Research Center
`
`kl
`
`w
`
`SPRINGER SCIENCE+BUSINESS MEDLA, LLC
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 2
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 2
`
`

`

`Library of Congress Cataloging—in-Puhlication Data
`
`High performance networks. Technology and protocols / edited by Ahmed
`N. Tantawy.
`cm. -- (The Kluwer international series in engineering and
`p.
`computer science ; 237)
`Includes bibliographical references and index.
`ISBN 978-1-4613-6401-6
`ISBN 978-1-4615-3194-4 (eBook)
`DOI 10.1007/978-1-4615-3194-4
`
`2. Computer network protocols.
`1. Computer networks.
`II. Series: Kluwer international
`.
`I. Tantawy, Ahmed N., 1952-
`series in engineering and computer science ; SECS 237.
`TK5105.5.H5217
`1994
`004.6—-d020
`
`93-6142
`CIP
`
`Copyright (0 1994 by Springer Science+Business Media New York
`Originally published by Kluwer Academic Publishers in 1994
`Softcover reprint of the hardcover lst edition 1994
`All rights reserved. No part of this publication may be reproduced, stored in
`a retrieval system or transmitted in any form or by any means, mechanical,
`photo—copying, recording, or otherwise, without the prior written permission of
`the publisher, Springer Science+Business Media, LLC.
`
`Printed on acid-free paper.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 3
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 3
`
`

`

`TABLE OF CONTENTS
`
`PREFACE
`
`I
`
`HIGH PERFORMANCE PROTOCOLS
`
`l.
`
`2.
`
`A Survey of Li ght-Weight Protocols for High-
`Speed Networks
`W. A. Doeringer, H. D. Dykeman,
`M. Kaiserswerth, B. W. Meister,
`H. Rudin & R. Williamson
`
`A Survey of High Performance Protocol
`Implementation Techniques
`D. C. Feldmeier
`
`II
`
`GIGABIT LAN TECHNOLOGY
`
`3.
`
`4.
`
`5.
`
`6.
`
`A Survey of MAC Protocols for High—Speed LANs
`
`A. E. Kamal & B. W. Abeysundara
`
`Optical Network Architectures
`Z. Haas
`
`Fibre Channel
`
`M. W. Sachs
`
`High-Performance Parallel Interface (HIPPI)
`D. E. Tolmie
`
`vii
`
`1
`
`3
`
`29
`
`51
`
`53
`
`85
`
`109
`
`131
`
`III
`
`METROPOLITAN AND WIDE AREA NETWORKS
`
`157
`
`7.
`
`8.
`
`9.
`
`Metropolitan Area Networks
`J. F. Mallenauer
`
`Broadband Integrated Services Digital
`Network (B-ISDN) Standards
`
`M. Zeug
`
`Synchronous Optical Network SONET
`G. Shenoda
`
`INDEX
`
`159
`
`183
`
`205
`
`229
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 4
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 4
`
`

`

`
`
`PREFACE
`
`The world of information processing is going through a major phase of its
`evolution. Networking has been associated with computers since the 1960’s.
`Communicating machines, exchanging information or cooperating to solve
`complex problems, were the dream of many scientists and engineers. Rudi-
`mentary networks and protocols were invented. Local area networks capable
`of carrying a few megabits per second became basic components of corporate
`computing installations in the 1980’s. At the same time, advances in optical
`transmission and switching technologies made it possible to transfer billions
`of bits per second. The availability of this huge bandwidth is making people
`wonder about the seemingly unlimited possibilities of these “fat information
`pipes” A new world where all interesting up-to-date information becomes
`instantaneously available to everyone everywhere is often portrayed to be
`around the comer. New applications are envisioned and their requirements
`are defined.
`
`The new field of High Performance Networking is burgeoning with activities
`at various levels.
`Several frontiers are being explored simultaneously.
`In
`order to achieve more bandwidth and better performance, work is progessing
`in optical transmission, high speed switching and network resource manage-
`ment. Some researchers have started to investigate all-optical networking as a
`promising approach to remove the relatively slow electronics from the
`network infrastructure. This will also introduce a new environment with
`
`unique characteristics that will have a definite impact on network architec-
`tures, topologies, addressing schemes, and protocols.
`
`Protocol design and implementation is another area that continues to attract
`many researchers. The increase in bandwidth and data rates has not been
`matched with equivalent increase in information packet handling at the end
`systems. When computers are attached to high bandwidth links, the real
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 5
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 5
`
`

`

`viii
`
`High Performance Networks
`
`bandwidth usable by the system is immediately throttled by the architectural
`bottlenecks of the network adapters, the number of packets that can be pro-
`cessed per second,and the overall hardware and software system structure.
`This problem has to be solved in order to efficiently utilize the immense
`capabilities of the physical links and enable new bandwidth-hungry applica-
`tions.
`
`Clearly, the problem is complex and multifaceted. Solving one part often
`introduces more complexity to other parts. The goal seems to be clear: pro-
`viding current and future applications with a versatile, flexible and powerful
`means to achieve the connectivity they require.
`The approaches are,
`however, still mostly unclear, unproven, and almost radically different. Two
`major schools of thought emerged. On one side, several groups have declared
`current standard protocol suites -such as TCP/IP- obsolete and started
`designing new models and protocols for high performance communication
`subsystems. On the other hand, some groups have blamed the low perform-
`ance on the inadequate implementation of standard protocols rather than the
`protocols themselves.
`
`Networking is chiefly an experimental science. We can more easily improve
`things when we implement them. Several researchers in industrial and aca-
`demic laboratories are experimenting with new ideas. Most are building pro-
`totypes, measuring performance and gradually refining their concepts. They
`learn invaluable lessons in a field where almost nothing is obvious and old
`solutions becarne justly questionable. Sharing this experience is crucial in
`reaching adequate solutions without wasting time and effort reinventing the
`same concepts and repeating the same mistakes.
`
`At a more practical level, interoperability is a necessity in heterogeneous net-
`works. This translates into an urging need for standardization. The commer-
`cial market needs the security and safety of international standards (especially
`
`In general, standards should be based on some general
`in communications!)
`consensus among interested parties. We are still far from that stage in many
`aspects of this field. There is a need today for some specific technologies.
`People do not have to wait until an optimal solution is found in order to
`standardize it.
`Some standards -such as Synchronous Optical Network
`(SONET), Asynchronous Transfer Mode (ATM), High Performance Parallel
`Interface (HIPPI) and Fibre Channel
`(FCS)- have emerged for various
`reasons. Some of these standards may not live long but they are all inter-
`esting and useful from the technical point of View. Did anyone ever believe
`that ultimate perfection can be achieved? We do the best we can but, fortu-
`nately, we never quit evolving and improving our concepts.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 6
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 6
`
`

`

`Preface
`
`ix
`
`This book is part of a developing series that aims at both introducing the
`underlying concepts of high performance communication systems and pro-
`viding
`the more seasoned researcher with a collection of lessons learned
`through experimentation. A few independent volumes are planned. Each
`volume focuses on a specific aspect of the field.
`
`The first two volumes have been written simultaneously. This volume, High
`Performance Networks: Technology and Protocols, mainly contains survey
`presentations that focus on the technology and protocols used in high per-
`formance networking systems. Chapters include technical overviews of fun-
`damental approaches and established standards. The other volume, High
`Performance Networks: Frontiers and Experience, presents a unique collection
`of some of the most advanced experiments in high performance communi-
`cations in various laboratories across the world. They represent distinct
`points in a wide spectrum of technical aspects, goals and approaches.
`
`This book consists of nine chapters divided into three parts. Part One con-
`tains two surveys. Doeringer et a1. give an overview of the major protocols
`proposed to reduce the processing overhead in high performance networks.
`They compare the techniques used by eight representative protocols at the
`transport layer and discuss their View of the most promising techniques.
`In
`the second chapter, Feldmeier presents a survey of the main techniques used
`in implementing protocols in high performance communication subsystems.
`He discusses the use of special purpose protocol processors and parallel proc-
`essing as means to boost performance.
`
`Part Two is devoted to gigabit networking technology in the local area. A
`survey of Media Access (MAC) protocols proposed for high speed LANs is
`given by Kamal and Abeysundara. They have classified and briefly described
`some twenty different schemes and gave a comprehensive list of references for
`interested readers. Chapter Four explains optical network architectures and
`how one can improve the utilization of the enormous bandwidth available in
`fibers. Haas gives also some examples of optical switches and networks and
`portrays his View of future optical networking.
`
`The two following chapters in Part Two present standards originally devel-
`oped for gigabit speed connections in the traditional I/O paradigm. These
`two standards -FCS and HIPPI- can and are used to build gigabit LANs.
`Sachs gives a brief and accurate presentation of FCS. Tolmie gives a thor~
`ough yet concise presentation of the HIPPI standard.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 7
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 7
`
`

`

`x
`
`High Performance Networks
`
`The focus of Part Three is on high performance Metropolitan and Wide Area
`Networks (MAN/WANs). Mollenauer gives a lively introduction to MANs
`and describes the essentials of the Switched Multimegabit Data Service
`-known as SMDS- and its underlying technology,
`the Distributed Queue
`Dual Bus (DQDB), which has been adopted as the IEEE Standard for MAN
`subnetworks. SMDS is viewed by many as a step on the road to Broadband
`Integrated Services Data Networks (B-ISDN) since they are both based on
`the ATM technology. B-ISDN, considered by many as the ultimate WAN,
`is described in the last two chapters. Zeug presents the B-ISDN standard and
`its reference model, service classes, sublayers and types. Shenoda focuses on
`the physical transmission and describes the SONET standard.
`
`This book contains a unique collection of chapters covering a wide spectrum
`of issues in a technical and concise manner.
`I am greatly indebted to the
`contributing authors, all of whom are among the leading experts in their
`fields. Their willingness not only to share their knowledge and views but also
`to take the time to do so in a clear and well organized fashion is highly com-
`mendable and will be surely very appreciated by the professional community.
`
`Ahmed N. Tantawy
`Yorktown Heights, NY
`March 1993
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 8
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 8
`
`

`

`5
`
`FIBRE CHANNEL
`
`Martin W. Sachs
`IBM Research
`T. 1. Watson Research Center
`P. O. B. 704
`Yorktown Heights, NY 10598
`
`Abstract
`
`Fibre Channel is being developed as an industry-standard transmission
`medium, interconnection network, and logical protocol to support both
`traditional I/O and communications in a local area.
`It will support a
`spectrum of applications requiring either high bandwidth, low cost, or
`both. Mappings are being developed to support several industry
`standard upper level protocols.
`
`Introduction
`Accredited Standards Committee X3's task group X3T9.3 is developing Fibre
`Channel (FC)[l], a standard for a serial lIO channel, to provide a transport vehicle
`for present and future standard upper level protocols. Upper level protocols of
`immediate interest to X3T9.3 are the Intelligent Peripheral Interface Device
`Generic Command Set (IPI3) [2], Small Computer System Interface (SCSI)[3],
`High Performance Parallel Interface Framing Protocol (HIPPI-FP) [4, 5], Internet
`Protocol (IP)[6], and command sets equivalent to that of International Business
`Machines Corp. System/390® lIO[7, 8]. In each case, the logical protocol, rather
`than the physical interface protocol, is being mapped to FC.
`
`As its name indicates, the primary focus of FC is on optical fiber interconnection.
`However, the physical layer definition includes copper coaxial and shielded
`twisted pair interconnections for low cost. short distance interconnection. The
`standard includes bandwidths ranging from 12.5 to 100 megabytes per sec (MB/s).
`
`A. N. Tantawy (ed.), High Performance Networks
`© Kluwer Academic Publishers 1994
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 9
`
`

`

`110
`
`High Performance Networks
`
`Unrepeatered distances up to 10 km are specified, with the maximum depending
`on other physical parameters.
`
`FC is intended to support both classical I/O channel applications (e.g. SCSI and
`IPI3) and local area communications applications (IP). HIPPI framing protocol
`applications are in both categories. The design point of the logical protocols is for
`distances of the order of a few kilometers, for interconnection within a building or
`establishment campus. In addition, care is being taken that FC should efficiently
`support gateways to wide area networks.
`
`In general, proposed applications of FC include both high bandwidth and low cost
`applications. Examples of high bandwidth applications include attachment of
`visualization workstations to supercomputers and attachment of high performance
`disk arrays to both supercomputers and high performance workstations. An
`example of a low cost application is the interconnection of large numbers of low
`cost disk drives within a storage subsystem, in which FC with a serial copper
`transmission medium is expected to provide significant cost reduction compared to
`today's parallel bus interconnections.
`
`Interconnection Topology
`The primary topological elements in FC are fabric, nodes, and N_Ports. The
`topology is illustrated in Figure 1.
`
`Fabric is the term used in FC to denote the medium which interconnects N_Ports.
`The initial emphasis of the FC Committee is on a fabric consisting of space(cid:173)
`division switches for high-performance applications. Work is also in progress on
`fabrics, such as loops, which are more suited to low-cost interconnection. The
`standard also permits two N_Ports to be directly connected by a link, with no
`intervening fabric. A node is an element which contains executing applications
`In general, a node contains a single
`and one or more connections to the fabric.
`instance of an operating system although this is not specified by the standard.
`
`An N_Port (node port) is the embodiment of the function needed at a node to
`connect the node to a fabric. The standard does not specify which FC functions
`are to be implemented in hardware and which are to be implemented in software.
`A typical N_Port can be expected to include a link control facility (e.g. serial
`transmitter and receiver) which connects to the serial link, a direct memory access
`connection to the main memory of the node, and that function, to be described
`subsequently, which controls the information flow on the link and between the
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 10
`
`

`

`Fibre Channel
`
`111
`
`NODE
`
`N_PORTI
`
`IN_PORT
`
`NODE N_PORTt--
`
`FABRIC
`
`--1N_PORT NODE
`
`Figure 1. Fabric, Nodes, and N_Ports
`
`link and the memory of a node. A node controls one or more N_Ports. An
`N_Port is illustrated in Figure 2.
`
`Functional Levels
`FC is divided into five functional levels, named FC-O through FC-4. Of these,
`FC-O, FC-1 and FC-2 are included in the initial "Physical and Signaling Interface"
`(FC-PH) definition[1]. At this time of writing, work is beginning on the FC-3 and
`FC-4 definitions. The functional levels are illustrated in Figure 3.
`
`FC-O defmes the physical level. This includes permitted transmission media,
`optical or electrical specifications of the media, connectors, bit rates, jitter specifi(cid:173)
`cations, and unrepeatered distances.
`
`FC-1 defines the encoding of data and control information on the serial link.
`also includes bit, byte, and word synchronization rules and certain error controls.
`
`It
`
`FC-2 defines the signalling protocol and is roughly equivalent to a data link
`control (DLC) layer in a standard communications protocol. However, with a
`fabric present, the logical link control (LLC) function within the DLC operates
`end to end, between two communicating N_Ports, rather than separately on each
`link.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 11
`
`

`

`112
`
`High Performance Networks
`
`NODE
`MEMORY
`
`FLOW CTL.
`MUL TIPLEX
`
`FIBER
`+-l---+ LINK
`
`Figure 2. Structure of an N]ort. DMA = Direct Memory Access; LCF = Link Control
`Facility.
`
`FC-3 is called the common services level. When defined, it will consist of the
`rules for managing paths between nodes.
`
`FC-4 will define the rules which map the constructs in the upper level protocols to
`the FC-2 and FC-3 primitives. There will be a separate FC-4 definition corre(cid:173)
`sponding to each of the supported protocols. For example, the FC-4 for IP will
`define how IP packets are sent and received using the facilities of FC-2.
`
`Classes of Service
`In order to promote optimum support for the broad range of applications expected
`to use FC, the standard defines multiple classes of service. Each class of service
`consists of fabric rules and specific FC-2 protocols. Three classes are currently
`defined.
`
`Class 1 provides circuit-switched connections, called dedicated connections. Once
`a dedicated connection is made between two N_Ports, they are guaranteed the
`entire link bandwidth. The bandwidth may be used either for a single logical data
`stream or for multiplexed streams.
`In Class 1, every transmission frame is
`acknowledged; the acknowledgements provide end to end flow control and
`detection of lost frames. Class 1 is primarily intended for applications which
`transfer long data streams at high bandwidth. Examples are high performance
`visualization and file transfer.
`
`Class 2 provides high-performance frame switching. Each transmission frame is
`individually routed through the fabric. A given N_Port may be concurrently trans(cid:173)
`ferring data with multiple other N_Ports. Every frame is acknowledged as in
`Class 1. Applications include low-latency message exchanges such as used in
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 12
`
`

`

`Fibre Channel
`
`113
`
`UPPER LEVEL PROTOCOL
`
`FC-4 MAPPING
`FC-3 COMMON SERVICES
`FC-2 LOGICAL SIGNALING
`
`FC-l TRANSMISSION
`FC-0 PHYSICAL
`
`Figure 3. Fe level structure
`
`remote procedure call, and record oriented disk accesses such as used with some
`file server protocols and traditional disk I/O.
`
`Class 3 provides high perfonnance frame switching but without acknowledge(cid:173)
`ments. One application for Class 3 will be transmission of multicast advisory
`messages, such as will be required for configuration management and fabric man(cid:173)
`agement. In these applications, elimination of congestion due to acknowledgement
`traffic is more important than detecting, at the FC-2 level, the occasional loss of a
`message due to the essentially unreliable nature of Class-3 transmission. When
`needed, application-level responses will provide confinnation of message delivery.
`Another potential use of Class 3 is for efficient communication to a router or
`gateway to a wide area network where the transport layer provides the end to end
`flow control and error management which Class 3 lacks.
`
`It is likely that future enhancements of FC will include one or more additional
`classes of service which support newly emerging applications.
`
`FC-O, Physical Characteristics
`FC-O defines the menu of choices for the physical parameters of the link. Support
`of the large variety of FC applications requires a wide range of cost and perfonn(cid:173)
`ance options. The large number of options is a cause of concern with regard to
`interoperability; it may be expected that market forces will eventually limit the set
`of choices which will be in widespread use, especially as the higher perfonnance
`technologies mature and their costs are lowered.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 13
`
`

`

`114
`
`High Perfonnance Networks
`
`Listing all of the pennissible combinations of technology parameters is beyond the
`scope of this article. Reference should be made to the FC-PH specification[l].
`Following are the options for the key parameters of the standard:
`
`• Transmission media: optical fiber, copper coaxial, copper shielded twisted pair
`
`• Transmission rates: 1062.5, 531.25, 265.625, and 131.8125 Mbaud.
`
`• Optical cables
`
`- Single mode: 9 pm
`- Multimode: 50 pm and 62.5 pm
`
`• Optical wavelength
`
`- Single-mode: 1300 nm
`- Multimode: 780 nm
`
`• Optical emitters: Light-emitting diode, laser
`
`• Maximum distances (depending on other options)
`
`- Optical: 500 m - 10 km
`- Electrical: 10-100 m
`
`• Optical connector: SC connector
`
`Fe-I, Transmission Protocol
`The transmission code is an adaptive 8B-10B code with limited run length[9].
`The coding rules enable a receiver to detect all odd-bit errors and a large number
`of other error patterns as code violations. In addition to the encoding of the 256
`8-bit data characters, the code defines a number of additional characters which
`may be used for control functions. Several have unique "comma" properties.
`These characters cannot appear in an error-free data stream as a result of the
`juxtaposition of two data characters. The comma characters can therefore be used
`to enable a receiver to synchronize itself to the character boundaries in the data
`stream.
`
`FC-1 defines the transmission fonnat as a series of 4-byte words (40 bits after
`encoding). It also dermes a number of control words, called ordered sets, which
`are used as frame delimiters, idle words, and for other purposes. Each ordered set
`consists of a particular comma character (the character tenned K28.5) followed by
`three data characters which identify the particular ordered set and are chosen to
`provide a high degree of error immunity. To enable a receiver to maintain syn-
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 14
`
`

`

`Fibre Channel
`
`115
`
`chronization to the word boundaries, a stream of idle words is transmitted between
`frames.
`
`In addition, FC-l defines the rules by which a receiver determines when it is syn(cid:173)
`chronized to character and word boundaries, when it is not synchronized, and how
`it reacquires synchronization. The rules are based on frequency of detection of
`code violations. They provide synchronization stability by avoiding unnecessary
`resynchronization when an isolated bit error occurs.
`
`FC·2, Logical Signalling Protocol
`FC-2 defines the logical signalling protocol. It is roughly equivalent to the LLC
`layer of a standard communications protocol. Areas of protocol defined by FC-2
`include transmission frame format, N-Port addressing, service classes, flow
`control, multiplexing management, initialization, and error detection.
`
`Franne Structure
`All information except certain primitive controls, to be discussed, is transferred in
`frames. The frame format is illustrated in Figure 4. Each frame is bounded by a
`start of frame delimiter and end of frame delimiter. The contents of the frame
`consist of frame header, data field, and 4-byte cyclic redundancy check word
`(CRC).
`
`In addition to bounding the frame, the delimiters are used for certain control func(cid:173)
`tions where the required function must be rapidly identified without requiring
`decoding into the 8-bit domain or checking the frame CRC. Each type of delim(cid:173)
`iter consists of one ordered set. The data characters in the ordered set encode the
`requested control function. Following are the control functions performed by the
`delimiters:
`
`• Start of frame
`
`- Request CIass-l circuit connection
`Indicate first or only frame of sequence of frames (to be discussed below)
`-
`-
`Indicate second through last frame of sequence
`
`• End of frame
`
`- Break CIass-l circuit connection
`-
`Indicate last or only frame of a sequence
`-
`Indicate first through next-to-Iast frame of a sequence
`- Abort frame (disregard contents)
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 15
`
`

`

`116
`
`High Performance Networks
`
`SOF
`
`FRAME HEADER
`
`PAYLOAD
`
`CRC
`
`EOF
`
`Figure 4. Frame Format. SOF = Start of Frame Delimiter; CRC = Cyclic Redundancy
`Check field; EOF = End of Frame Delimiter.
`
`In addition, certain delimiters have separate encodings for each class of service.
`
`The frame header contains various types of addressing and control information
`similar to that found in the usual LLC header. Key elements of the frame header
`include
`
`• 24-bit source and destination N_Port addresses, used for routing through the
`fabric
`
`• Type of upper level protocol to which this frame relates (IPI3, SCSI, etc.)
`
`• Sequence identifier (to be discussed below)
`
`• Exchange identifier (to be discussed below)
`
`• Sequence count (frame sequence number)
`
`• Various other control bits and fields
`
`Frames are classified as link-control frames and data frames. Link-control frames
`include acknowledgements, busy indications, and rejects (error indications). Data
`frames convey the useful information being excbanged by the upper level proto(cid:173)
`cols (e.g. data being read from or written to a disk). In addition, data frames are
`used by a set of supporting upper level protocols called link applications. These
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 16
`
`

`

`Fibre Channel
`
`117
`
`provide various initialization, management, and recovery functions which are per(cid:173)
`formed using basic FC-2 constructs.
`
`The data field contains the useful information, or payload, being conveyed by the
`frame.
`In addition, the data field may contain one or more optional headers
`required by the particular upper level protocol to which the payload belongs. The
`maximum size of the data field, including any optional headers, is 2112 bytes.
`This is a somewhat arbitrary figure which was chosen based on trade-offs among
`factors such as transmission efficiency, CRC coverage, expected costs of trans(cid:173)
`mission, and receiver buffering at the highest bandwidth, etc.
`
`Primitive Sequences
`A primitive sequence is the continuous repetition of a particular ordered set. Con(cid:173)
`tinuous sequences are defined for signalling under conditions in which the use of
`frames is either unreliable or inappropriate. Use of frames is unreliable under
`conditions of high link error rate. Frames are inappropriate, for example, if it is
`likely that the receiver is not synchronized to character and word boundaries, such
`as during link initialization.
`
`Reliable receipt and decoding of a primitive sequence under high link error rate is
`assured by the continuous repetition, combined with the redundancy in the combi(cid:173)
`nation of data characters used for each ordered set. Depending on the protocol, a
`continuous sequence is transmitted either for a fixed length of time or until a spec(cid:173)
`ified primitive sequence is received in response.
`
`The following primitive sequences are defined by FC:
`
`Not Operational Sequence (NOS): An N_Port or port on a switch sends NOS if
`it is unable to detect a proper received signal or to acquire character and word
`synchronization.
`It informs the port at the other end of the link that a trans(cid:173)
`mission or reception problem exists.
`
`Offline Sequence (OLS): An N_Port or port on a switch sends OLS to signal
`that it is about to go off line or power down. OLS is thus an indication to the
`port at the other end of the link that detected errors should be ignored.
`
`Link Reset (LR) and Link Reset Response (LRR): LR and LRR are used in
`interlocked fashion to cause the fabric to remove a dedicated connection, if one
`exists, when the state of the connection is unknown.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 17
`
`

`

`118
`
`High Performance Networks
`
`Information Units, Multiplexing, and How Control
`The main function of FC-2 is to deliver an information unit from the sending
`instance of an upper level protocol in one node to the receiving instance of the
`same upper level protocol in a different node. The content and length of the
`information unit are determined by the upper level protocol; the length of a single
`information unit is essentially unbounded, or may be limited to 232 bytes,
`depending on other parameters. For all practical purposes, then, an information
`unit may be any length defmed by the upper level protocols and a data stream
`may consist of a single information unit or a flow of information units. Informa(cid:173)
`tion units may be delivered in any of the three classes of service, as determined by
`system performance and implementation requirements.
`
`An information unit is transmitted from the sending to the receiving N_Port as the
`payload of a flow of frames, which will be described below. FC-2 is responsible
`for flow control and error detection within the flow of frames and for correct reas(cid:173)
`sembly of the information unit at the receiving N]ort. The definition of FC-2
`permits a fabric to misorder frames in a class 2 and class 3; the FC-2 function at
`the receiving N_Port can correctly reassemble an information unit in spite of mis(cid:173)
`In general, FC-2 will also preserve order of
`ordered delivery by the fabric.
`delivery of information units within a single stream, provided that the upper level
`protocol obeys certain non-mandatory rules.
`
`It will be noted that if FC-2 is viewed as the medium access control (MAC) layer
`of a communications protocol stack, it differs from conventional MAC layers,
`such as those in the IEEE 802 protocol suite[10] or FDDI[ll], in its treatment of
`the information unit.
`In conventional MAC protocols, the maximum transmission
`unit must fit in one physical frame and reassembly of longer streams of informa(cid:173)
`tion is the responsibility of the LLC or transport layer. In order to further its goal
`of high transmission bandwidth, FC places segmentation and reassembly of longer
`data streams in FC-2, where it can be implemented in high speed N_Port function.
`In this regard, FC is similar to IBM's ESCON VO Interface[8] in which segmenta(cid:173)
`tion and reassembly are performed by the ESCON interface (channel) function.
`
`FC-2 defines function which permits multiple independent streams of data to be
`multiplexed by interleaving frames belonging to the different streams. In class I,
`multiple streams may concurrently be transferred in both directions between two
`N_Ports over a dedicated connection. In class 2 and Class 3, a given N_Port may
`also be communicating through the fabric with multiple other N_Ports. Two con(cid:173)
`structs are provided for managing multiplexing; they are called the sequence and
`the exchange. An implementation may make use of one or both for multiplexing
`management.
`
`Oracle-Huawei-NetApp Ex. 1029, pg. 18
`
`

`

`Fibre Channel
`
`119
`
`A sequence is a series of consecutive frames within an exchange (to be explained
`below) which are denoted by the same value of a sequence identifier in the frame
`header. An infonnation unit is transferred as the frame payload of one or more
`sequences.
`In the simplest case, an infonnation unit is transferred as a single
`sequence. Since the FC-2 definition in this area is subject to change, we will
`assume, for the purpose of this article, that an information unit is transferred as a
`single sequence.
`
`An exchange is a relationship between instances of an upper level protocol in two
`nodes which is used to manage a unidirectional or bidirectional flow of related
`In the I/O applications of FC, an exchange is an abstraction
`infonnation units.
`representing a single I/O operation , such as the transfer of a block of data, or a
`chain of related I/O operations which may transfer a stream of blocks between a
`host node and an VO device controller node. Within FC-2, an exchange is said to
`connect an exchange originator with an exchange responder. The frame header of
`every frame associated with a given exchange is labelled by a pair of identifiers,
`one supplied by the originator (originator exchange identifier, OX_ID) and one
`supplied by the responder (responder exchange identifier, RX_ID). T