Network Working Group R. Braden
`Request for Comments: 1009 J. Postel
`Obsoletes: 985 ISI
` June 1987
`
` Requirements for Internet Gateways
`
`Status of this Memo
`
` This document is a formal statement of the requirements to be met by
` gateways used in the Internet system. As such, it is an official
` specification for the Internet community. Distribution of this memo
` is unlimited.
`
` This RFC summarizes the requirements for gateways to be used between
` networks supporting the Internet protocols. While it was written
` specifically to support National Science Foundation research
` programs, the requirements are stated in a general context and are
` applicable throughout the Internet community.
`
` The purpose of this document is to present guidance for vendors
` offering gateway products that might be used or adapted for use in an
` Internet application. It enumerates the protocols required and gives
` references to RFCs and other documents describing the current
` specifications. In a number of cases the specifications are evolving
` and may contain ambiguous or incomplete information. In these cases
` further discussion giving specific guidance is included in this
` document. Specific policy issues relevant to the NSF scientific
` networking community are summarized in an Appendix. As other
` specifications are updated this document will be revised. Vendors
` are encouraged to maintain contact with the Internet research
` community.
`
`1. Introduction
`
` The following material is intended as an introduction and background
` for those unfamiliar with the Internet architecture and the Internet
` gateway model. General background and discussion on the Internet
` architecture and supporting protocol suite can be found in the DDN
` Protocol Handbook [25] and ARPANET Information Brochure [26], see
` also [19, 28, 30, 31].
`
` The Internet protocol architecture was originally developed under
` DARPA sponsorship to meet both military and civilian communication
` requirements [32]. The Internet system presently supports a variety
` of government and government-sponsored operational and research
` activities. In particular, the National Science Foundation (NSF) is
` building a major extension to the Internet to provide user access to
`
`Braden & Postel [Page 1]
`
`Cloudflare - Exhibit 1033, page 1
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` national supercomputer centers and other national scientific
` resources, and to provide a computer networking capability to a large
` number of universities and colleges.
`
` In this document there are many terms that may be obscure to one
` unfamiliar with the Internet protocols. There is not much to be done
` about that but to learn, so dive in. There are a few terms that are
` much abused in general discussion but are carefully and intentionally
` used in this document. These few terms are defined here.
`
` Packet A packet is the unit of transmission on a physical
` network.
`
` Datagram A datagram is the unit of transmission in the IP
` protocol. To cross a particular network a datagram is
` encapsulated inside a packet.
`
` Router A router is a switch that receives data transmission
` units from input interfaces and, depending on the
` addresses in those units, routes them to the
` appropriate output interfaces. There can be routers
` at different levels of protocol. For example,
` Interface Message Processors (IMPs) are packet-level
` routers.
`
` Gateway In the Internet documentation generally, and in this
` document specifically, a gateway is an IP-level
` router. In the Internet community the term has a long
` history of this usage [32].
`
` 1.1. The DARPA Internet Architecture
`
` 1.1.1. Internet Protocols
`
` The Internet system consists of a number of interconnected
` packet networks supporting communication among host computers
` using the Internet protocols. These protocols include the
` Internet Protocol (IP), the Internet Control Message Protocol
` (ICMP), the Transmission Control Protocol (TCP), and
` application protocols depending upon them [22].
`
` All Internet protocols use IP as the basic data transport
` mechanism. IP [1,31] is a datagram, or connectionless,
` internetwork service and includes provision for addressing,
` type-of-service specification, fragmentation and reassembly,
` and security information. ICMP [2] is considered an integral
`
`Braden & Postel [Page 2]
`
`Cloudflare - Exhibit 1033, page 2
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` part of IP, although it is architecturally layered upon IP.
` ICMP provides error reporting, flow control and first-hop
` gateway redirection.
`
` Reliable data delivery is provided in the Internet protocol
` suite by transport-level protocols such as the Transmission
` Control Protocol (TCP), which provides end-end retransmission,
` resequencing and connection control. Transport-level
` connectionless service is provided by the User Datagram
` Protocol (UDP).
`
` 1.1.2. Networks and Gateways
`
` The constituent networks of the Internet system are required
` only to provide packet (connectionless) transport. This
` requires only delivery of individual packets. According to the
` IP service specification, datagrams can be delivered out of
` order, be lost or duplicated and/or contain errors. Reasonable
` performance of the protocols that use IP (e.g., TCP) requires
` an IP datagram loss rate of less than 5%. In those networks
` providing connection-oriented service, the extra reliability
` provided by virtual circuits enhances the end-end robustness of
` the system, but is not necessary for Internet operation.
`
` Constituent networks may generally be divided into two classes:
`
` * Local-Area Networks (LANs)
`
` LANs may have a variety of designs, typically based upon
` buss, ring, or star topologies. In general, a LAN will
` cover a small geographical area (e.g., a single building or
` plant site) and provide high bandwidth with low delays.
`
` * Wide-Area Networks (WANs)
`
` Geographically-dispersed hosts and LANs are interconnected
` by wide-area networks, also called long-haul networks.
` These networks may have a complex internal structure of
` lines and packet-routers (typified by ARPANET), or they may
` be as simple as point-to-point lines.
`
` In the Internet model, constituent networks are connected
` together by IP datagram forwarders which are called "gateways"
` or "IP routers". In this document, every use of the term
` "gateway" is equivalent to "IP router". In current practice,
` gateways are normally realized with packet-switching software
`
`Braden & Postel [Page 3]
`
`Cloudflare - Exhibit 1033, page 3
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` executing on a general-purpose CPU, but special-purpose
` hardware may also be used (and may be required for future
` higher-throughput gateways).
`
` A gateway is connected to two or more networks, appearing to
` each of these networks as a connected host. Thus, it has a
` physical interface and an IP address on each of the connected
` networks. Forwarding an IP datagram generally requires the
` gateway to choose the address of the next-hop gateway or (for
` the final hop) the destination host. This choice, called
` "routing", depends upon a routing data-base within the gateway.
` This routing data-base should be maintained dynamically to
` reflect the current topology of the Internet system; a gateway
` normally accomplishes this by participating in distributed
` routing and reachability algorithms with other gateways.
` Gateways provide datagram transport only, and they seek to
` minimize the state information necessary to sustain this
` service in the interest of routing flexibility and robustness.
`
` Routing devices may also operate at the network level; in this
` memo we will call such devices MAC routers (informally called
` "level-2 routers", and also called "bridges"). The name
` derives from the fact that MAC routers base their routing
` decision on the addresses in the MAC headers; e.g., in IEEE
` 802.3 networks, a MAC router bases its decision on the 48-bit
` addresses in the MAC header. Network segments which are
` connected by MAC routers share the same IP network number,
` i.e., they logically form a single IP network.
`
` Another variation on the simple model of networks connected
` with gateways sometimes occurs: a set of gateways may be
` interconnected with only serial lines, to effectively form a
` network in which the routing is performed at the internetwork
` (IP) level rather than the network level.
`
` 1.1.3. Autonomous Systems
`
` For technical, managerial, and sometimes political reasons, the
` gateways of the Internet system are grouped into collections
` called "autonomous systems" [35]. The gateways included in a
` single autonomous system (AS) are expected to:
`
` * Be under the control of a single operations and
` maintenance (O&M) organization;
`
` * Employ common routing protocols among themselves, to
` maintain their routing data-bases dynamically.
`
`Braden & Postel [Page 4]
`
`Cloudflare - Exhibit 1033, page 4
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` A number of different dynamic routing protocols have been
` developed (see Section 4.1); the particular choice of routing
` protocol within a single AS is generically called an interior
` gateway protocol or IGP.
`
` An IP datagram may have to traverse the gateways of two or more
` ASs to reach its destination, and the ASs must provide each
` other with topology information to allow such forwarding. The
` Exterior Gateway Protocol (EGP) is used for this purpose,
` between gateways of different autonomous systems.
`
` 1.1.4. Addresses and Subnets
`
` An IP datagram carries 32-bit source and destination addresses,
` each of which is partitioned into two parts -- a constituent
` network number and a host number on that network.
` Symbolically:
`
` IP-address ::= { <Network-number>, <Host-number> }
`
` To finally deliver the datagram, the last gateway in its path
` must map the host-number (or "rest") part of an IP address into
` the physical address of a host connection to the constituent
` network.
`
` This simple notion has been extended by the concept of
` "subnets", which were introduced in order to allow arbitrary
` complexity of interconnected LAN structures within an
` organization, while insulating the Internet system against
` explosive growth in network numbers and routing complexity.
` Subnets essentially provide a two-level hierarchical routing
` structure for the Internet system. The subnet extension,
` described in RFC-950 [21], is now a required part of the
` Internet architecture. The basic idea is to partition the
` <host number> field into two parts: a subnet number, and a true
` host number on that subnet.
`
` IP-address ::=
` { <Network-number>, <Subnet-number>, <Host-number> }
`
` The interconnected LANs of an organization will be given the
` same network number but different subnet numbers. The
` distinction between the subnets of such a subnetted network
` must not be visible outside that network. Thus, wide-area
` routing in the rest of the Internet will be based only upon the
` <Network-number> part of the IP destination address; gateways
` outside the network will lump <Subnet-number> and <Host-number>
`
`Braden & Postel [Page 5]
`
`Cloudflare - Exhibit 1033, page 5
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` together to form an uninterpreted "rest" part of the 32-bit IP
` address. Within the subnetted network, the local gateways must
` route on the basis of an extended network number:
`
` { <Network-number>, <Subnet-number> }.
`
` The bit positions containing this extended network number are
` indicated by a 32-bit mask called the "subnet mask" [21]; it is
` recommended but not required that the <Subnet-number> bits be
` contiguous and fall between the <Network-number> and the
` <Host-number> fields. No subnet should be assigned the value
` zero or -1 (all one bits).
`
` Flexible use of the available address space will be
` increasingly important in coping with the anticipated growth of
` the Internet. Thus, we allow a particular subnetted network to
` use more than one subnet mask. Several campuses with very
` large LAN configurations are also creating nested hierarchies
` of subnets, sub-subnets, etc.
`
` There are special considerations for the gateway when a
` connected network provides a broadcast or multicast capability;
` these will be discussed later.
`
` 1.2. The Internet Gateway Model
`
` There are two basic models for interconnecting local-area networks
` and wide-area (or long-haul) networks in the Internet. In the
` first, the local-area network is assigned a network number and all
` gateways in the Internet must know how to route to that network.
` In the second, the local-area network shares (a small part of) the
` address space of the wide-area network. Gateways that support
` this second model are called "address sharing gateways" or
` "transparent gateways". The focus of this memo is on gateways
` that support the first model, but this is not intended to exclude
` the use of transparent gateways.
`
` 1.2.1. Internet Gateways
`
` An Internet gateway is an IP-level router that performs the
` following functions:
`
` 1. Conforms to specific Internet protocols specified in
` this document, including the Internet Protocol (IP),
` Internet Control Message Protocol (ICMP), and others as
` necessary. See Section 2 (Protocols Required).
`
` 2. Interfaces to two or more packet networks. For each
`
`Braden & Postel [Page 6]
`
`Cloudflare - Exhibit 1033, page 6
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` connected network the gateway must implement the
` functions required by that network. These functions
` typically include:
`
` a. encapsulating and decapsulating the IP datagrams with
` the connected network framing (e.g., an Ethernet
` header and checksum);
`
` b. sending and receiving IP datagrams up to the maximum
` size supported by that network, this size is the
` network’s "Maximum Transmission Unit" or "MTU";
`
` c. translating the IP destination address into an
` appropriate network-level address for the connected
` network (e.g., an Ethernet hardware address);
`
` d. responding to the network flow control and error
` indication, if any.
`
` See Section 3 (Constituent Network Interface), for
` details on particular constituent network interfaces.
`
` 3. Receives and forwards Internet datagrams. Important
` issues are buffer management, congestion control, and
` fairness. See Section 4 (Gateway Algorithms).
`
` a. Recognizes various error conditions and generates
` ICMP error and information messages as required.
`
` b. Drops datagrams whose time-to-live fields have
` reached zero.
`
` c. Fragments datagrams when necessary to fit into the
` MTU of the next network.
`
` 4. Chooses a next-hop destination for each IP datagram,
` based on the information in its routing data-base. See
` Section 4 (Gateway Algorithms).
`
` 5. Supports an interior gateway protocol (IGP) to carry out
` distributed routing and reachability algorithms with the
` other gateways in the same autonomous system. In
` addition, some gateways will need to support the
` Exterior Gateway Protocol (EGP) to exchange topological
` information with other autonomous systems. See
` Section 4 (Gateway Algorithms).
`
`Braden & Postel [Page 7]
`
`Cloudflare - Exhibit 1033, page 7
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` 6. Provides system support facilities, including loading,
` debugging, status reporting, exception reporting and
` control. See Section 5 (Operation and Maintenance).
`
` 1.2.2. Embedded Gateways
`
` A gateway may be a stand-alone computer system, dedicated to
` its IP router functions. Alternatively, it is possible to
` embed gateway functionality within a host operating system
` which supports connections to two or more networks. The
` best-known example of an operating system with embedded gateway
` code is the Berkeley BSD system. The embedded gateway feature
` seems to make internetting easy, but it has a number of hidden
` pitfalls:
`
` 1. If a host has only a single constituent-network
` interface, it should not act as a gateway.
`
` For example, hosts with embedded gateway code that
` gratuitously forward broadcast packets or datagrams on
` the same net often cause packet avalanches.
`
` 2. If a (multihomed) host acts as a gateway, it must
` implement ALL the relevant gateway requirements
` contained in this document.
`
` For example, the routing protocol issues (see Sections
` 2.6 and 4.1) and the control and monitoring problems are
` as hard and important for embedded gateways as for
` stand-alone gateways.
`
` Since Internet gateway requirements and
` specifications may change independently of operating
` system changes, an administration that operates an
` embedded gateway in the Internet is strongly advised
` to have an ability to maintain and update the gateway
` code (e.g., this might require gateway code source).
`
` 3. Once a host runs embedded gateway code, it becomes part
` of the Internet system. Thus, errors in software or
` configuration of such a host can hinder communication
` between other hosts. As a consequence, the host
` administrator must lose some autonomy.
`
` In many circumstances, a host administrator will need to
` disable gateway coded embedded in the operating system,
` and any embedded gateway code must be organized so it
` can be easily disabled.
`
`Braden & Postel [Page 8]
`
`Cloudflare - Exhibit 1033, page 8
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` 4. If a host running embedded gateway code is concurrently
` used for other services, the O&M (operation and
` maintenance) requirements for the two modes of use may
` be in serious conflict.
`
` For example, gateway O&M will in many cases be performed
` remotely by an operations center; this may require
` privileged system access which the host administrator
` would not normally want to distribute.
`
` 1.2.3. Transparent Gateways
`
` The basic idea of a transparent gateway is that the hosts on
` the local-area network behind such a gateway share the address
` space of the wide-area network in front of the gateway. In
` certain situations this is a very useful approach and the
` limitations do not present significant drawbacks.
`
` The words "in front" and "behind" indicate one of the
` limitations of this approach: this model of interconnection is
` suitable only for a geographically (and topologically) limited
` stub environment. It requires that there be some form of
` logical addressing in the network level addressing of the
` wide-area network (that is, all the IP addresses in the local
` environment map to a few (usually one) physical address in the
` wide-area network, in a way consistent with the { IP address
` <-> network address } mapping used throughout the wide-area
` network).
`
` Multihoming is possible on one wide-area network, but may
` present routing problems if the interfaces are geographically
` or topologically separated. Multihoming on two (or more)
` wide-area networks is a problem due to the confusion of
` addresses.
`
` The behavior that hosts see from other hosts in what is
` apparently the same network may differ if the transparent
` gateway cannot fully emulate the normal wide-area network
` service. For example, if there were a transparent gateway
` between the ARPANET and an Ethernet, a remote host would not
` receive a Destination Dead message [3] if it sent a datagram to
` an Ethernet host that was powered off.
`
`Braden & Postel [Page 9]
`
`Cloudflare - Exhibit 1033, page 9
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` 1.3. Gateway Characteristics
`
` Every Internet gateway must perform the functions listed above.
` However, a vendor will have many choices on power, complexity, and
` features for a particular gateway product. It may be helpful to
` observe that the Internet system is neither homogeneous nor
` fully-connected. For reasons of technology and geography, it is
` growing into a global-interconnect system plus a "fringe" of LANs
` around the "edge".
`
` * The global-interconnect system is comprised of a number of
` wide-area networks to which are attached gateways of several
` ASs; there are relatively few hosts connected directly to
` it. The global-interconnect system includes the ARPANET,
` the NSFNET "backbone", the various NSF regional and
` consortium networks, other ARPA sponsored networks such as
` the SATNET and the WBNET, and the DCA sponsored MILNET. It
` is anticipated that additional networks sponsored by these
` and other agencies (such as NASA and DOE) will join the
` global-interconnect system.
`
` * Most hosts are connected to LANs, and many organizations
` have clusters of LANs interconnected by local gateways.
` Each such cluster is connected by gateways at one or more
` points into the global-interconnect system. If it is
` connected at only one point, a LAN is known as a "stub"
` network.
`
` Gateways in the global-interconnect system generally require:
`
` * Advanced routing and forwarding algorithms
`
` These gateways need routing algorithms which are highly
` dynamic and also offer type-of-service routing. Congestion
` is still not a completely resolved issue [24]. Improvements
` to the current situation will be implemented soon, as the
` research community is actively working on these issues.
`
` * High availability
`
` These gateways need to be highly reliable, providing 24 hour
` a day, 7 days a week service. In case of failure, they must
` recover quickly.
`
` * Advanced O&M features
`
` These gateways will typically be operated remotely from a
` regional or national monitoring center. In their
`
`Braden & Postel [Page 10]
`
`Cloudflare - Exhibit 1033, page 10
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` interconnect role, they will need to provide sophisticated
` means for monitoring and measuring traffic and other events
` and for diagnosing faults.
`
` * High performance
`
` Although long-haul lines in the Internet today are most
` frequently 56 Kbps, DS1 lines (1.5 Mbps) are of increasing
` importance, and even higher speeds are likely in the future.
` Full-duplex operation is provided at any of these speeds.
`
` The average size of Internet datagrams is rather small, of
` the order of 100 bytes. At DS1 line speeds, the
` per-datagram processing capability of the gateways, rather
` than the line speed, is likely to be the bottleneck. To
` fill a DS1 line with average-sized Internet datagrams, a
` gateway would need to pass -- receive, route, and send --
` 2,000 datagrams per second per interface. That is, a
` gateway which supported 3 DS1 lines and and Ethernet
` interface would need to be able to pass a dazzling 2,000
` datagrams per second in each direction on each of the
` interfaces, or a aggregate throughput of 8,000 datagrams per
` second, in order to fully utilize DS1 lines. This is beyond
` the capability of current gateways.
`
` Note: some vendors count input and output operations
` separately in datagrams per second figures; for these
` vendors, the above example would imply 16,000 datagrams
` per second !
`
` Gateways used in the "LAN fringe" (e.g., campus networks) will
` generally have to meet less stringent requirements for
` performance, availability, and maintenance. These may be high or
` medium-performance devices, probably competitively procured from
` several different vendors and operated by an internal organization
` (e.g., a campus computing center). The design of these gateways
` should emphasize low average delay and good burst performance,
` together with delay and type-of-service sensitive resource
` management. In this environment, there will be less formal O&M,
` more hand-crafted static configurations for special cases, and
` more need for inter-operation with gateways of other vendors. The
` routing mechanism will need to be very flexible, but need not be
` so highly dynamic as in the global-interconnect system.
`
` It is important to realize that Internet gateways normally operate
` in an unattended mode, but that equipment and software faults can
` have a wide-spread (sometimes global) effect. In any environment,
`
`Braden & Postel [Page 11]
`
`Cloudflare - Exhibit 1033, page 11
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` a gateway must be highly robust and able to operate, possibly in a
` degraded state, under conditions of extreme congestion or failure
` of network resources.
`
` Even though the Internet system is not fully-interconnected, many
` parts of the system do need to have redundant connectivity. A
` rich connectivity allows reliable service despite failures of
` communication lines and gateways, and it can also improve service
` by shortening Internet paths and by providing additional capacity.
` The engineering tradeoff between cost and reliability must be made
` for each component of the Internet system.
`
`Braden & Postel [Page 12]
`
`Cloudflare - Exhibit 1033, page 12
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
`2. Protocols Required in Gateways
`
` The Internet architecture uses datagram gateways to interconnect
` constituent networks. This section describes the various protocols
` which a gateway needs to implement.
`
` 2.1. Internet Protocol (IP)
`
` IP is the basic datagram protocol used in the Internet system [19,
` 31]. It is described in RFC-791 [1] and also in MIL-STD-1777 [5]
` as clarified by RFC-963 [36] ([1] and [5] are intended to describe
` the same standard, but in quite different words). The subnet
` extension is described in RFC-950 [21].
`
` With respect to current gateway requirements the following IP
` features can be ignored, although they may be required in the
` future: Type of Service field, Security option, and Stream ID
` option. However, if recognized, the interpretation of these
` quantities must conform to the standard specification.
`
` It is important for gateways to implement both the Loose and
` Strict Source Route options. The Record Route and Timestamp
` options are useful diagnostic tools and must be supported in all
` gateways.
`
` The Internet model requires that a gateway be able to fragment
` datagrams as necessary to match the MTU of the network to which
` they are being forwarded, but reassembly of fragmented datagrams
` is generally left to the destination hosts. Therefore, a gateway
` will not perform reassembly on datagrams it forwards.
`
` However, a gateway will generally receive some IP datagrams
` addressed to itself; for example, these may be ICMP Request/Reply
` messages, routing update messages (see Sections 2.3 and 2.6), or
` for monitoring and control (see Section 5). For these datagrams,
` the gateway will be functioning as a destination host, so it must
` implement IP reassembly in case the datagrams have been fragmented
` by some transit gateway. The destination gateway must have a
` reassembly buffer which is at least as large as the maximum of the
` MTU values for its network interfaces and 576. Note also that it
` is possible for a particular protocol implemented by a host or
` gateway to require a lower bound on reassembly buffer size which
` is larger than 576. Finally, a datagram which is addressed to a
` gateway may use any of that gateway’s IP addresses as destination
` address, regardless of which interface the datagram enters.
`
` There are five classes of IP addresses: Class A through
` Class E [23]. Of these, Class D and Class E addresses are
`
`Braden & Postel [Page 13]
`
`Cloudflare - Exhibit 1033, page 13
`
`

`

`RFC 1009 - Requirements for Internet Gateways June 1987
`
` reserved for experimental use. A gateway which is not
` participating in these experiments must ignore all datagrams with
` a Class D or Class E destination IP address. ICMP Destination
` Unreachable or ICMP Redirect messages must not result from
` receiving such data