EXHIBIT
`
`EXHIBIT
`1006
`
`1006
`
`

`
`(12) United States Patent
`Karol et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 6,628,617 B1
`Sep. 30, 2003
`
`US006628617B1
`
`(54) TECHNIQUE FOR INTERNETWORKING
`TRAFFIC ()N CONNECTIONLESS AND
`CONNECTION_ORIENTED NETWORKS
`
`(75) Inventors: Mark John Karo], Fair Haven, NJ
`(Us); Malathi Veeraraghavan, Atlantic
`,
`H1gh1and5> NJ (Us)
`_
`_
`_
`(73) Asslgneei Lucellt Technologles Inc» Murray H111,
`NJ (Us)
`
`( * ) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U-S-C- 154(k)) by 0 days-
`
`(21) Appl. N0.: 09/261,807
`(22) Filed:
`Mar. 3, 1999
`
`(51) Int. Cl.7 ....................... .. H04L 12/56; H04L 12/64;
`H04L 12/66
`(52) US. Cl. ..................... .. 370/237; 370/466; 370/477;
`370/401; 370/412; 370/352; 370/353; 370/354;
`370/355; 370/356; 370/236; 3700301;
`370/389; 3705951; 370/3955; 37059565
`(58) Field Of Search ............................ .. 370/229, 230.1,
`370/235, 236, 237, 238, 352_353, 354,
`355 356 357 359 389 392 3951 3955
`’ 395,51 5955i 395,65 3957 39572’
`400 401’ 410 4,12 419’ 428 ‘£29 465’
`’
`’
`’
`’
`’ 466 468’ 477’
`’
`’
`
`(56)
`
`References Cited
`
`U-S~ PATENT DOCUMENTS
`6 016 319 A * 1/2000 Kshirsagar et a1. ....... .. 370/399
`6jo55j56l A * 4/2000 Feldman et a1_ __________ __ 370/220
`6,091,725 A * 7/2000 Cheriton et a1.
`370/392
`6,147,989 A * 11/2000 Esaki et a1. ............... .. 370/355
`6,151,319 A * 11/2000 Dommety et a1.
`370/395.52
`6,160,793 A * 12/2000 Ghani et a1. .............. .. 370/236
`
`6,201,792 B1 * 3/2001 Lahat ....................... .. 370/236
`6,252,853 B1 * 6/2001
`370/242
`6,259,699 B1 * 7/2001 Opalka et a1. ............ .. 370/389
`6,317,431 B1 * 11/2001 Hodgkinson et a1. ..... .. 370/392
`6,320,874 B1 : 11/2001 C'rump et a1. ............. .. 370/401
`6,339,594 B1
`1/2002 Civanlar et a1. .......... .. 370/352
`6,343,083 B1 * 1/2002 Mendelson et a1. ....... .. 370/392
`6,343,322 B2 * 1/2002 Nagami et a1. ........ .. 370/3953
`6,343,326 B2 * 1/2002 Acharya et a1. .......... .. 709/238
`6,381,244 B1 * 4/2002 Nishimura et a1.
`370/39521
`6,490,252 B1 * 12/2002 Riggan et a1. ............ .. 370/237
`
`* cued by exammer
`Primary Examinerqiassan KiZOu
`Assistant Examiner—Ahmed Elallam
`(74) Attorney, Agent, or Firm—Matthew J. Hodulik; Barry
`H- Freedman
`(57)
`
`ABSTRACT
`
`Tra?ic On a C0I1I1eCti0I11eSS(CL) network, Such as IP Packets,
`can be routed onto a connection a connection oriented (CO)
`network, Such as an ATM telephony network, When it is
`advantageous to do so from a user or service provider
`viewpoint, without affecting the ability of users to continue
`to use existing applications. Routing is controlled by nodes
`Called CL'CO gateways> With Connectivity ‘0 both the CL
`network and the Co network. When CL traffic originating at
`a source reached these gateway nodes, a decision is made
`whether to continue carrying the information in the CL
`mode, or to redirect the traffic to a CO network. In accor
`dance with one embodiment of the present invention, each
`CL-Co gateway includes hardware and software modules
`that typically comprise (a) interfaces to the Co network, (b)
`interfaces to the CL network (c) a moderately siZed packet
`buffer for temporarily storing packets waiting for CO net
`Work Setup or turnaround; (d) a database for Storing
`forwarding, ?ow control header translation and other
`information, and (e) a Processor Containing logic for COH
`trolling the gateway packet handling operations.
`
`19 Claims, 9 Drawing Sheets
`
`Viptela, Inc. - Exhibit 1006
`Page 1
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 1 0f 9
`
`US 6,628,617 B1
`
`FIG. 1
`
`Viptela, Inc. - Exhibit 1006
`Page 2
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 2 of 9
`
`US 6,628,617 B1
`
`
`
`go...~Emz<Emaozozaza“Em
`
`
`
`
`
`
`
`
`
`X33:zo_m_>_o_._5zH._H._><;so;
`
`
`
`
`
`_£925.:._<._._o_omazowzoza“Ia
`
`
`
`
`
`éosmz._<o._Eom:ozom_._oz>m528
`
`»_§_<m__=255m8zo%8_m_:":8
`
`
`
`$52.52.n_9a-8<":9.
`
`asa:m__~_o-zo_§zzoo
`
`an
`
`an
`
`SE02%
`
`
`
`eazz8m8$\m..a<an
`
`an
`
`SméosmzE;55352Em55:52:2
`
`
`
`m..Q~...~
`
`was
`omn__
`
`en
`
`8.,
`
`V2932n=
`E
`
`8»
`
`:3mmszozazzoo
`
`Viptela, Inc. - Exhibit 1006
`
`Page 3
`
`Viptela, Inc. - Exhibit 1006
`Page 3
`
`
`
`
`

`
`U.S. Patent
`
`D... &
`
`m,
`m m.
`
`9 m
`
`US 6,628,617 B1
`
`w 8% “z: % ..| E8
`N8 \
`
`?gs
`
`as? 4 ...... :r 5
`
`
`
`
`
`§\ . _
`
`
`
`5% EU: on... \ “888% :53
`
`§\ , E. 23 M2:
`
`\ ...... -- 5. §\
`
`N2. 52 ..|l
`
`65E? 6
`
`(1/ ow...
`
`¢ ¢
`
`:05 8 ()0;
`
`<2 5% III
`
`E28 1::
`
`Viptela, Inc. - Exhibit 1006
`Page 4
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 4 0f 9
`
`US 6,628,617 B1
`
`FIG. 5
`
`IP DATAGRAN ARRNES / 501
`
`IS THIS A PACKET
`FROM A FLOW THAT
`NEIDS C0 SERVICE ?
`
`CONSULT FLOW
`DATABASE
`
`f 505
`
`507
`
`< ISTHE
`OUTGOING \ YES
`PORTDITRYVALID?/
`N0
`f 511
`
`509
`/
`PLACE nATAcRAH IN PAcKET
`BUFFER; SEND SIGNAL To
`THE PROTOCOL CONVERTER
`
`FTJINCIITIISN
`
`/ 513
`\FORWARD
`FORWARD OR HOLD ?/
`/
`PLACE DATAGRAM
`IN PACKET BuFFBR
`
`5T7
`CONSULT FORWARDING /
`DATABASE
`
`503
`
`YES
`
`525 /
`CONSULT FORWARDING
`DATABASE
`
`527
`
`IS THERE AN DITRY
`CORRESPONDING TO THE
`HEADER FIEU) VALUES or
`THE INCOMING PAcKET ‘?
`N0
`
`/ 529
`FoRwARo
`DATAGRAN As
`PFR EHTRY
`
`/ 531
`PLACE THE DATAGRAM
`IN PAcKET BuFFBR
`AND SEND SIGNAL To
`SOURCE ROUTING
`
`YES
`
`[521
`FORWARD
`BATAcRAH As
`PERENIRY
`
`519
`
`DOES THE ENTRY FoR THE
`PAcKET HAvE A VALID
`oursomc PORT ENTRY ?
`N0
`[523
`PLACE DATAGRAN IN PACKET
`BUFFER; saw SIGNAL TO THE
`‘SOURCE RouRHc MODULE
`
`Viptela, Inc. - Exhibit 1006
`Page 5
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 5 0f 9
`
`US 6,628,617 B1
`
`FIG. 6
`
`60
`IS THE SYNCHRONIZE
`FLAG SET OR NOT ?
`YES
`/ 607
`SEND SIGNAL To co
`sNTTcH REQUESTING
`coNNEcTToN SEIUP
`609
`/
`NHEN coNNEcTToN IS
`sET UP, SEND SIGNAL
`Tgo?ggl
`
`I
`uPDATE FLOW AND
`HEADER TRANSLATION
`DATABASES
`\
`611
`
`SIGNAL RECHI/ED BY THE GW
`HANDUNG FUNCTION T0 HANDLE
`AN INCOIIING DATAGRAH
`
`/ 601
`
`603
`
`DOES THE PROTOCOL
`FIELD OF THE PACKET
`INDICATE TCP 0R UDP ?
`
`TCP
`
`N0
`
`N0
`
`631
`APPUCATION OF TYPE
`wTTH END-To-END
`HANDsHAKE?
`YES
`
`645
`613
`DATA SEGIIBTI
`FIN/RSI \
`0R FIN/RSI 1’
`SEND SIGNAL To co
`WA
`swTTcH REQUESTING
`CONNECTION saw
`
`/ 617
`GENERAIE A TcP AcK
`INDICAHNG A ZERO
`REcENE wTNDow SIZE
`619
`/
`sEND SIGNAL To co
`sTTTTcH REQUESTING
`coNNEcTToN SETUP
`
`633
`/
`CHECK APPucATToN
`MESSAGE m5
`
`647
`\
`wHEN coNNEcTToN Ts
`sET UP, uPDATE FLOW
`AND HEADER
`TRANSLATION DATADAsEs
`
`635
`
`N0
`
`/ 621
`WHEN CONNECHON ls
`sET UP, cENHTATE A
`NEW ACK wTTH
`NoN-zmo NTNDDN SIZE
`623
`/
`SBID SIGNAL T0
`PROTOCOL CONVERTER
`
`/ 625
`SEND SIGNAL T0 CL PACKET
`FORWARDING ENGINE
`
`IS MESSAGE RELATED To
`OPENING A sEssToN 2
`YES
`
`/ 63’7
`SEND SIGNAL T0 co
`swrTcH REQUESTING
`coNNEcTToN sEwP
`
`639
`/
`NHEN coNNEcTToN IS SET
`UP, sEND sTcNAL To
`PROTOCOL coNvDRTER
`
`/ 64‘
`UPDATE FLOW AND
`HEADER TRANSLATION
`DATADAsEs
`
`Viptela, Inc. - Exhibit 1006
`Page 6
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 6 0f 9
`
`US 6,628,617 B1
`
`FIG. 7
`
`RECEIVES SIGNALS FROM cw / 701
`HANDLING mucnou T0 HANDLE
`A PACKET
`
`/ 703
`
`705
`
`CONSULT FORWARDING DATABASE
`
`YES
`
`DOES THE ENTRY FOR
`THE PACKET HAVE A VALID
`OUTCOINC PORT ENTRY ?
`NO
`
`FORWARD / 709
`DATACRAN AS
`PER ENTRY
`
`PLACE DATACRAII IN PACKET
`BUFFER; SEND TO THE
`SOURCE ROUTING NODULE
`
`/ 707
`
`FIG. 8
`
`ROUTING PROTOCOL (CL 0R co) / 80‘
`UPDATE ARRIVES
`
`UPDATE CORRESPONDING TABLE
`
`/ 803
`
`YES
`
`f 805
`ADVH'ITISING RESOURCES OF C
`NETWORK INTO CL NETWORK
`
`/ 807
`
`GENERATE ROUTING PROTOCOL
`MESSAGE IF NECESSARY
`
`/ 809
`UPDATE INTEGRATED ROUTING TABLE
`
`Viptela, Inc. - Exhibit 1006
`Page 7
`
`

`
`U.S. Patent
`U.S. Patent
`
`Sep. 30, 2003
`Sep. 30, 2003
`
`Sheet 7 0f 9
`Sheet 7 of 9
`
`US 6,628,617 B1
`US 6,628,617 B1
`
`C0NEWIORK
`
`2Q 3 2 5 Q § E
`
`Viptela, Inc. -Exh'
`
`‘
`
`006
`
`ge 8
`
`FIG.9
`
`Viptela, Inc. - Exhibit 1006
`Page 8
`
`

`
`U.S. Patent
`
`Sep. 30, 2003
`
`Sheet 8 0f 9
`
`US 6,628,617 B1
`
`10
`FIG.
`
`/ 901
`
`
`
`IP NETWORIK (CONSISHNG 0F IP ROUHERS)
`
`Viptela, Inc. - Exhibit 1006
`Page 9
`
`

`
`U.S. Patent
`U.S. Patent
`
`Sep. 30, 2003
`Sep. 30, 2003
`
`Sheet 9 0f 9
`Sheet 9 of 9
`
`US 6,628,617 B1
`US 6,628,617 B1
`
`1 8 EZ 5
`
`Viptela, Inc. - Exhibit 1006
`
`Page 10
`
`FIG.11
`
`Viptela, Inc. - Exhibit 1006
`Page 10
`
`

`
`US 6,628,617 B1
`
`1
`TECHNIQUE FOR INTERNETWORKING
`TRAFFIC ON CONNECTIONLESS AND
`CONNECTION-ORIENTED NETWORKS
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to the internet
`Working of connectionless (e.g. Internet Protocol or “IP”)
`and connection oriented (e.g. ATM, MPLS, RSVP)
`networks, and, more particularly, to a technique that alloWs
`datagrams (messages) generated by source endpoints on a
`connectionless (CL) netWork to be redirected to destinations
`on a CL netWork via a communication path that includes a
`connection oriented (CO) netWork operated in a sWitched
`mode (as opposed to a provisioned mode) if there is an
`advantage from the user or service provider prospective.
`
`10
`
`15
`
`2
`CL netWork and destined for a user continue to arrive at the
`gateWay. These packets typically arrive every feW
`microseconds, or less. During the time period When the path
`through the CO netWork toWard the user is being set up,
`typically on the order of milliseconds, many datagrams Will
`arrive at the gateWay, and it is important that these data
`grams not be lost. One possible solution, involving buffering
`of datagrams, has been suggested. HoWever, this solution is
`generally impractical, since the siZe of the buffer Would have
`to be very large.
`
`SUMMARY OF THE INVENTION
`In accordance With the present invention, nodes called
`CL-CO gateWays, are arranged to have connectivity to both
`the CL netWork and the CO netWork. When CL traf?c
`originating at a source reaches these gateWay nodes, a
`decision is made Whether to continue carrying the informa
`tion in the CL mode, or to redirect the traf?c to a CO
`netWork. In accordance With one embodiment of the present
`invention, each CL-CO gateWay includes hardWare and
`softWare modules that typically comprise (a) interfaces to
`the CO netWork, (b) interfaces to the CL netWork (c) a
`moderately siZed packet buffer for temporarily storing pack
`ets Waiting for CO netWork setup or turnaround; (d) a
`database for storing forWarding, ?oW control, header trans
`lation and other information, and (e) a processor containing
`logic for controlling the gateWay packet handling opera
`tions. In accordance With another embodiment of the present
`invention, the gateWay additionally includes a sWitch fabric
`for CO netWorking, a CL packet forWarding engine, and/or
`a protocol converter. Additionally, it is to be noted that the
`gateWay of the present invention can be incorporated Within
`a CL node, but not all nodes of the CL netWork need to have
`the ability to make redirect decisions.
`In order to avoid the need for a large packet buffer and to
`also avoid the loss of datagrams, the gateWay is arranged, in
`one embodiment of the present invention, to generate and
`transmit to the source of the datagrams, a signal requesting
`the source to sloW doWn or stop transmission of datagrams
`during the time period When the gateWay is establishing a
`path through the CO netWork. In another embodiment of the
`invention, datagrams received at the gateWay are turned
`around and continue to be routed at least temporarily
`through the CL netWork during the period When the CO
`netWork path is being set up. As a further alternative,
`application or transport layer signals generated in the source
`are intercepted by the gateWay and used to trigger connec
`tion setup in the CO netWork even before datagrams have
`been transmitted from the source, thereby assuring that
`When datagrams reach the gateWay, there is no loss of
`information.
`In the present invention, the CO netWork can be an MPLS
`(MultiProtocol Label SWitching) or RSVP (Resource reS
`erVation Protocol) based IP netWork, a WDM (Wavelength
`Division Multiplexed) netWork, an ATM (Asynchronous
`Transfer Mode) netWork, or an STM (Synchronous Time
`Multiplexing) netWork, such as the telephony netWork or a
`SONET netWork. The CL netWork is typically, although not
`necessarily, an IP netWork. The present invention is useful,
`for example, in serving the needs of Internet users Who Want
`stricter quality-of-service guarantees for their ?le transfer
`application than is currently offered by the Internet.
`
`65
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIGS. 1 and 2 are diagrams illustrating tWo modes
`(parallel and serial, respectively) of internetWorking CO and
`CL netWorks;
`
`BACKGROUND OF THE INVENTION
`Connectionless (CL) netWorks and connection-oriented
`(CO) netWorks have some fundamental distinguishing fea
`tures. In CL netWorks, no explicit connection setup actions
`are executed prior to transmitting datagrams; instead, data
`grams are routed through a series of interconnected nodes to
`their destinations based on information in their headers. CO
`netWorks operated in the sWitched mode (as opposed to the
`provisioned mode) are those in Which connection setup for
`each call or session is performed prior to information
`transfer. (In a provisioned mode, the path through the
`netWork from a source to a destination is set up in advance,
`i.e., When the netWork is provisioned, and is not changed on
`a short term, i.e., call by call basis). In CO netWorks,
`connection setup actions consist of determining a route for
`the connection, allocating bandWidth (and possibly buffer)
`resources on the links and sWitches on the route, assigning
`and distributing “labels” or “positions” (e.g., time slots or
`Wavelengths) based on Whether the CO netWork is packet or
`circuit sWitched, respectively, and programming connection
`information into sWitch fabrics and endpoints.
`These tWo types of netWorks, CO and CL, enjoy advan
`tages and disadvantages from both the user perspective and
`the service provider perspective. CL netWorks, for instance,
`do not suffer the delay and processing overhead associated
`With connection setup. In contrast, information about the
`connections in CO netWorks helps in providing service
`guarantees and, furthermore, makes it possible to most
`ef?ciently use netWork resources (e.g., bandWidth) by
`“switching” datagrams to appropriate connections as the
`connections are established.
`The need to integrate and exploit the advantages of both
`CL and CO netWorks has long been recogniZed. TWo
`examples are the use of SS7 (Signaling System No. 7) CL
`netWorks in conjunction With the CO telephony netWork,
`and the use of RSVP (Resource reSerVation Protocol) in IP
`netWorks. (See R. Braden, L. Zhang, S. Berson, S. HerZog,
`S. J amin, “Resource ReSerVation Protocol (RSVP) Version
`1 Functional Speci?cation,” IETF RFC 2205, September
`1997) In both of these solutions, applications explicitly
`choose the netWorking mode appropriate to their needs.
`NotWithstanding the need to integrate CL and CO
`netWorks, prior art approaches have heretofore not routed
`traf?c from a source on a CL netWork onto a CO netWork
`operated in the sWitched mode (as opposed to a provisioned
`mode), because of the potential loss of datagrams during the
`time period When a path is being set up in the CO netWork.
`Speci?cally, if a transition betWeen a CL netWork and a CO
`netWork occurs at a gateWay, datagrams originating on the
`
`25
`
`35
`
`45
`
`55
`
`Viptela, Inc. - Exhibit 1006
`Page 11
`
`

`
`US 6,628,617 B1
`
`3
`FIG. 3 is a diagram illustrating different types of CO and
`CL networks with which the present invention may be used;
`FIG. 4 is a block diagram illustrating the internal arrange
`ment of CL-CO gateway arranged in accordance with the
`present invention;
`FIG. 5 is a How diagram illustrating the steps performed
`when the gateway of FIG. 4 performs its packet forwarding
`process;
`FIG. 6 is a How diagram illustrating the steps performed
`when the gateway of FIG. 4 performs its gateway handling
`process;
`FIG. 7 is a How diagram illustrating the steps performed
`in CL router/switch 420 of FIG. 4 when the gateway of FIG.
`4 determines to forward a datagram or store it temporarily in
`buffer 440;
`FIG. 8 is a How diagram illustrating the routing related
`processes performed in the gateway of FIG. 4;
`FIG. 9 illustrates an eXample of the interworking of an IP
`network and a switched CO network;
`FIG. 10 shows the same network as illustrated in FIG. 9,
`from the point of view of the CL network, and illustrates link
`weights in the links between routers; and
`FIG. 11 illustrates two different “types” of paths through
`the network of FIG. 9, in order to eXplain protocol conver
`sion in connection with the present invention.
`
`10
`
`15
`
`25
`
`DETAILED DESCRIPTION
`
`4
`must necessarily include the CO network. Accordingly, the
`gateway must use the CO network to complete the trans
`mission. A gateway arranged in accordance with the present
`invention can be programmed to automatically sense or
`detect the type of network environment that eXists, and
`operate in the appropriate mode. Alternatively, the gateway
`can be appropriately programmed, on installation, to operate
`in a single mode suited to the network environment. Finally,
`it should be mentioned that a single purpose gateway
`dedicated to operate only in one or the other of the modes is
`also within the scope of this invention.
`In the parallel architecture of FIG. 1, the present invention
`solves the problem of how to handle traf?c (user-plane data)
`arriving at the CL-CO gateways until the desired connection
`is established in the CO network, given the “long” call setup
`delays associated with CO networks, by either (a) “turning
`around” a few packets, and continuing to route such packets
`through the CL network using, for example, source routing
`until the connection is set up, or (b) generating and trans
`mitting to the source of the datagrams, a signal requesting
`the source to slow down or stop transmission of datagrams
`during the time period when the gateway is establishing a
`path through the CO network. As a further alternative,
`application or transport layer signals generated in the source
`are intercepted by the gateway and used to trigger connec
`tion setup in the CO network even before datagrams have
`been transmitted from the source, thereby assuring that
`when datagrams reach the gateway, there is no loss of
`information.
`In the serial architecture of FIG. 2, there is no path
`available through the CL on which to “turn around” arriving
`packets. Accordingly, only the options of (a) halting incom
`ing datagrams, or (b) intercepting the application or trans
`port layer signals (sometimes referred to as “set-up
`messages”), can be used.
`In FIG. 1, a source endpoint 101, which is a source of
`datagrams (such as a personal computer, workstation, or
`other processor attached to any information source) is
`directly connected to and served by a node 111 in a CL
`network 110. Traf?c from source endpoint 101 destined for
`destination endpoint 151 (which is directly connected to and
`served by a node 132 in a CL network 130) can be routed in
`at least two different, parallel routes, and this choice of
`routes is re?ected in how the CL-CO gateway 140 operates.
`In the ?rst route, the datagram can follow a path that
`traverses only connectionless nodes, namely from node 111
`possibly through other nodes (not shown) in CL network
`110, but eventually though node 112, which routes traf?c to
`a second CL network 120.
`The datagram continues through various nodes in CL
`network 120, such as nodes 121, 122 and other nodes, not
`shown, and is routed ?nally to a third CL network 130 that
`includes node 132 serving destination endpoint 151. The
`path through the latter network is from node 131 through
`other nodes, not shown, to node 132. The second path that
`a datagram in FIG. 1 can follow eXtends at least partially
`over a CO network 160, using the CL-CO gateways 140 and
`150 of the present invention, which are described in more
`detail below. Here, the path from source endpoint 101 is
`through node 111 in CL network 110 to the ?rst gateway
`140, where the datagrams are processed. When a path has
`been set up in a CO network 160, illustratively from node
`161 through other nodes, not shown, to node 162, the
`datagrams entering CO network 160 from gateway 140 are
`routed to a second CL-CO gateway 150. There, the data
`grams are again processed, and applied to node 132 in CL
`network 130, which node serves destination endpoint 151.
`
`The CL-CO gateway arrangement envisioned by the
`present invention interconnects a CL network and a CO
`network, and allows eXisting CL applications running on CL
`endpoints to continue operating in the CL mode, while, at
`the same time, allowing datagrams (sometimes hereinafter
`called messages) originated on the CL network to be
`transported, for at least a portion of the path between source
`and destination, on the CO network operated in the switched
`mode. This is accomplished without loss of information,
`allowing the network as a whole to exploit the advantages of
`both networking modes (CO and CL). The CL-CO gateways
`of the present invention may be arranged to operate using
`several different strategies, depending upon the architecture
`of the individual CO and CL networks that are intercon
`nected by the gateways. In particular, two architectures for
`internetworking CO and CL networks, called “parallel” and
`“serial”, are illustrated in FIGS. 1 and 2, respectively. The
`parallel con?guration could occur, for eXample, if two
`service providers, one with an IP-router-based network and
`the other with a CO-switch-based network, offer enterprises
`“long-distance” connectivity of their geographically distrib
`uted networks. The serial interconnecting con?guration
`occurs, for instance, when an enterprise decides to route all
`their traf?c through a speci?c service provider who happens
`to use a CO network. Of course, more general internetwork
`ing con?gurations are also possible (e.g., combinations of
`FIGS. 1 and 2), and networks can be envisioned with greater
`numbers of CL-CO and CO-CL transitions.
`The CL-CO gateway of the present invention can operate
`in distinctly different modes, based upon which of the two
`basic internetworking con?gurations, parallel and serial,
`eXists. In the parallel con?guration, since at least two paths
`eXist between the originating and destination CL nodes, one
`using the CL network and the other using the CO network,
`there is always a routing choice, i.e., CL to CO to CL or
`entirely CL. The gateway can make the routing selection
`based on maXimiZing ef?ciency. In the serial con?guration,
`the path between the originating and destination CL nodes
`
`35
`
`45
`
`55
`
`65
`
`Viptela, Inc. - Exhibit 1006
`Page 12
`
`

`
`US 6,628,617 B1
`
`10
`
`15
`
`25
`
`5
`Before proceeding to a description of FIG. 2, it is to be
`noted that in both FIGS. 1 and 2, the source (101, 201) and
`the destination (151, 251) may not connect directly to the
`same CL node (e.g., node 111) that the CL-CO gateway (e.g.
`gateway 140) connects to. Also, the source or destination
`may be directly connected to a CL-CO gateway (e.g.,
`gateway 140) as opposed to being connected through a CL
`node.
`Referring now to FIG. 2, it is to be noted that a datagram
`originating in source 201 has only a single path to destina
`tion 251. The datagram must travel from CL network 210 to
`gateway 240, thence through CO network 260 to gateway
`250, and thence through node 232 in CL network 230 to
`destination 251. Here, CL-CO gateway 240 has no choice in
`this sense: if a datagram arrives from source 201 and is
`destined for destination 251, that datagram must be routed
`onto CO network 260; there is no alternate route that can be
`taken. The arrangement of a CL-CO gateway in the envi
`ronment of a serial architecture of FIG. 2 will be discussed
`further below. First, the arrangement of a CL-CO gateway in
`the environment of the parallel architecture of FIG. 1 will be
`described.
`When a datagram arrives at a CL-CO gateway 140 of FIG.
`1, a determination is made if that packet should be carried
`by CO network 160. Connections are set up through CO
`network 160 for some, but not necessarily all, of the arriving
`CL traf?c. If a CO connection is set up, the path is similar
`to that described above. However, if a CO connection is not
`used, the path might extend from gateway 140 back to node
`112 in CL network 110 via path 1 15, and thence through CL
`networks 120 and 130 to destination endpoint 151. Thus,
`CL-CO gateway 140 handles traf?c both from ?ows for
`which CO connections are set up, as well as continues
`forwarding packets through the CL network if a CO con
`nection is not setup. The decision to set up CO connections
`is made at CL-CO gateway 140, based on the user-speci?ed
`service requirements and the traf?c situation in the CL and
`CO networks.
`Other differences exist between the way the present
`invention operates in the different architectures of FIGS. 1
`and 2, and these differences should be mentioned brie?y.
`First, the issue of how to set up routing tables at the routers
`in the CL network to take advantage of “shorter paths” that
`may exist through the CO network (or to meet users’
`subscribed-to service requirements) arises only for the “par
`allel” con?guration (FIG. 1), since communication paths
`between the CL-CO gateways 140 and 150 exist in both the
`CL networks 110, 120 and 130 and the CO network 160. On
`the other hand, in FIG. 2, there is no choice but to direct the
`traf?c to CO network 260, which implies that the routing
`issue is answered more readily. Second, in FIG. 1, it is also
`important to note that while a connection is being set up in
`CO network 160, communication can still take place
`through the CL networks 110, 120 and 130. Furthermore,
`even after the connection is established in CO network 160,
`55
`data can be allowed to How simultaneously through the CL
`and CO networks if both networks meet the user’s needs.
`It will now be instructive to consider examples of one CL
`network (the IP network) and three CO networks that can
`interoperate in accordance with the present invention. Refer
`ring to FIG. 3, network 300 is an IP (Internet Protocol)
`network comprised of a series of interconnected IP routers
`310. Network 300 is connectionless (CL), because, as stated
`previously, traf?c between endpoints 308 and 309 traverses
`the network without advance set up of a path or route
`through the network. All of the other networks (301, 311 and
`321) shown in FIG. 3 are connection oriented (CO). In these
`
`6
`networks, a path must be set up before traf?c can traverse the
`network from endpoint to endpoint. Network 301 is an ATM
`(Asynchronous Time Multiplexed) network comprised of a
`series of interconnected ATM switches 302. Network 311 is
`an STM (Synchronous Time Multiplexed) network and is
`comprised of STM switches 312 that implement PDH/
`SONET/SDH hierarchies. Network 321 is a WDM
`(Wavelength Division Multiplexed) network, and is com
`prised of WDM optical switches, ADMs and crossconnects
`322. All three CO networks 301, 311 and 321 can be
`operated in the switched mode.
`As indicated above, the CO networks 301, 311 and 321
`with which the present invention can be used, are assumed
`to be operated in the switched mode, rather than the provi
`sioned mode. In the switched mode, the setup actions
`include determining a route for the connection, allocating
`bandwidth (and possibly buffer) resources on the links and
`switches on the route, assigning and distributing “labels” or
`“positions” (e.g., time slots or wavelengths) based on
`whether the CO network is packet or circuit switched,
`respectively, and programming connection information into
`switch fabrics and endpoints. If one or more of these actions
`occurs in response to a request to set up a connection for a
`speci?c data exchange, then the CO network is said to be
`operated in “switched mode.” Otherwise, the CO network is
`said to be operated in “provisioned mode.” For example,
`when SONET is used to create point-to-point links between
`IP routers, then it is operating in a provisioned mode; the
`SONET connections are not set up for speci?c data transfers,
`but rather to just transport IP packets between routers.
`FIG. 4 shows the internal arrangement of CL-CO gateway
`140 in accordance with the present invention. Generally
`speaking, each CL-CO gateway arranged in accordance with
`the present invention includes hardware and software mod
`ules that typically comprise (a) a switch fabric for CO
`networking, shown in FIG. 4 as CO switch 410, (b) a CL
`packet forwarding engine, shown in FIG. 4 as CL router/
`switch 420, (c) a protocol converter 450, (d) a moderately
`siZed packet buffer 440 for temporarily storing packets
`waiting for CO network setup or turnaround; and (e) a
`processor 430 and associated database 431 for controlling
`the gateway packet handling operations and for storing
`forwarding, ?ow control, header translation and other infor
`mation. Input line cards 401 and output line cards 402
`connect the gateway of FIG. 4 to external networks, such
`that datagrams received in input line cards 401 can be
`directed either to CO switch 410 or CL router/switch 420,
`and such that output line cards 402 can receive datagrams
`from either of the last mentioned elements and direct them
`to external networks.
`In FIG. 4, the paths over which user data is communicated
`are shown as solid lines, while the paths over which control
`signals are communicated are shown as dashed lines. As will
`be seen from FIG. 4, and as described in more detail below,
`control over almost all of the elements in FIG. 4 is provided
`by processor 430. Persons skilled in the art will appreciate
`that such control occurs by virtue of stored programs which
`are appropriately arranged to perform the functionality
`described herein. Numerous speci?c programs may be
`devised to accomplish this purpose. While all of the ele
`ments in FIG. 4 can be implemented principally in hardware,
`it is to be understood that some of the elements such as CL
`router/switch 420 and protocol converter 450 may be imple
`mented in software.
`Before proceeding with a detailed description of the
`operation of the gateway of FIG. 4, it will be instructive to
`describe in general some of the functions and characteristics
`
`35
`
`45
`
`65
`
`Viptela, Inc. - Exhibit 1006
`Page 13
`
`

`
`US 6,628,617 B1
`
`7
`of the elements of the gateway. Speci?cally, buffer 440 in
`FIG. 4 is of moderate size, much smaller than Would be
`required in prior art arrangements. This is because in prior
`art arrangements, the buffer Would be required to store all of
`the arriving CL datagrams While a path through the CO
`netWork is being set up. In contrast, buffer 440 needs only
`to be of sufficient siZe to store datagrams that arrive during
`the period When the arriving datagrams are either “turned
`around”, or until “foolers” are generated and acted upon,
`thereby temporarily sloWing doWn or halting the How of neW
`datagrams into the gateWay. Note that the buffers needed for
`the normal operation of CL router/sWitch 420 and CO sWitch
`410 (if the CO netWork is packet-switched) are not shoWn.
`Protocol converter 450 is a typically a softWare imple
`mented process in Which the user payload is extracted from
`an IP datagram, and converted to the CO format, so that it
`can be carried directly on connections in the CO netWork.
`For example, if the CO netWork is an ATM netWork, then
`application data needs to be converted to the AAL5 format
`so that it can be carried through the ATM netWork. Since
`AAL5 performs transport-layer functions, the TCP header
`can also be converted to an AAL5 header in a sWitched
`(rather than provisioned) ATM netWork. In other Words,
`TCP/IP headers are converted into AAL5 /ATM headers. By
`doing this, it appears that TCP connections are terminated at
`the CL-CO gateWays from each endpoint of the
`communication, and a connection of the type supported by
`the CO netWork is set up betWeen the CL-CO gateWays,
`such as gateWays 140 and 150 of FIG. 1.
`Database 431 includes a series of individual databases
`arranged to store information used in various of the func
`tions performed by processor 430, and may include, as an
`example, a datagram forWarding database 432, a flow data
`base 433, and a header translation database 434.
`Datagram forWarding database 432 is the database used in
`typical CL IP routers. It stores the next hop router address
`and outgoing port number corresponding to each destination
`address. Typical ?elds in each record in this database Would
`be: Destination IP address; Next hop router; Outgoing port
`(interface).
`FloW database 433 stores information used to determine
`hoW to handl