`
`
`
`
`
`EXHIBIT
`
`EXHIBIT
`1020
`
`1020
`
`

`

`(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2002/0010866 A1
`McCullough et al.
`(43) Pub. Date:
`Jan. 24, 2002
`
`US 20020010866A1
`
`(54) METHOD AND APPARATUS FOR
`IMPROVING PEER-TO-PEER BANDWIDTH
`
`Publication Classification
`
`BETWEEN REMOTE NETWORKS BY
`COMBINING MULTIPLE CONNECTIONS
`WHICH USE ARBITRARY DATA PATHS
`
`Int. Cl.7 ........................... .. H04L 12/22; H04K 1/00
`(51)
`
`(52) U.S. Cl.
`........... .. 713/201; 709/228
`
`(76)
`
`Inventors: David J. McCullough, Upper
`Brookfield (AU); VVayne Meissner,
`Woolloowin (AU); Craig S.
`Humphrey, A11Ch€11fl0W€f
`(AU);
`Ch1‘i5t0Phe1‘ J- Biggsa Chapel Hill
`(AU)9 Antonio Basilio Merenda>
`Ch ape] Hm (AU)
`Correspondence Address:
`Claude A_ S_ Hamrick, Esq
`()ppENHE1MER WOLFF & DONNELLY LLp
`1400 Page Mill Road
`Palo Alto, CA 94304 (US)
`
`(21) APPL N0‘
`
`09/7409494
`
`(22)
`
`Filed:
`
`Dec. 18, 2000
`
`Related U_s_ Application Data
`
`(63) Non-provisional of provisional
`60/172,369, filed on Dec. 16, 1999.
`
`application No.
`
`(57)
`
`ABSTRACT
`
`A method and apparatus for increasing peer-to-peer band-
`width between remote networks by combining multiple
`connections, which use arbitrary data paths,
`is disclosed.
`The apparatus is a gateway node, which can be a specifically
`designed computer, open computer platform or extensions to
`firmware resident in a router; gateway or remote access
`server. The method includes origin authentication and data
`confidentiality, packet fragmenting, sequencing directed-
`routing, buifering, fragment encapsulation, packet re-assem-
`bly, and additional encapsulation for traversal of firewalls.
`Packet fragments transferred using the method can travel
`along very diverse paths through intervening public or
`private networks before arriving at the peer, which reas-
`sembles them. This eliminates the problems present
`in
`current aggregation schemes used by prior art, which are
`sensitive to the limitations in the infrastructure in the service
`provider’s points of presence.
`
`Connections to
`public network
`
`
`One Link on responder needs a static public
`IP address, all other links can use dynamic
`
`IF, addresses
`
`
`LocalNetwork2
`
`Small Network
`Gateway
`(SNG)
`
`Responder
`
`
`
`
`Multiple fragments travel through Internet
`
`independently of each other and are
`
`aggregated at the destination, not by the
`equipment at each POP
`
`
`
`Viptela, Inc. - Exhibit 1020
`
`Page 1
`
`
`
`LocalNetwork1
`
`Viptela, Inc. - Exhibit 1020
`Page 1
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 1 of 17
`
`US 2002/0010866 A1
`
`14
`
`16
`
`New
`York
`
`I I
`
`I’—22
`I
`I
`I
`I
`
`.
`Chicago
`
`12
`
`_ _ _
`
`_ _ _ _
`
`7
`20
`
`\
`
`\
`
`I
`
`I
`
`I
`
`\
`
`I
`
`I/‘18
`:
`I
`|
`
`\x/
`’/ \‘/28
`\
`
`/
`/’\26
`
`\\
`
`A‘-05!’
`n ees
`g
`
`————-(——- Atlanta
`10
`24
`
`FIG. 1
`
`§Q
`
` /
`
`\
`
`12
`
`\
`
`I
`
`I
`
`/
`
` /
`32
`
`I
`
`I
`
`\
`
`\
`
`\
`
`14
`
`16
`
`‘-03
`Angeles
`
`1 0
`
`Atlanta
`
`FIG. 2
`
`Viptela, Inc. - Exhibit 1020
`
`Page 2
`
`Viptela, Inc. - Exhibit 1020
`Page 2
`
`

`

` PatentApplicationPublication
`29_.IMmMJWwgww_g_.mmmoomM>_o__%n_w
`
`
`
`cozmmemmm_w:cm;o
`
`200
`
`2teehS
`
`
`
`
`
`pmAN_mccm;o+F_m::m:0VExomqBmmmzmmm295
`
`
`
`
`
`Um_o::m;ozn_w_m_xN>__m:w:co=mc_..mm_o2w_m>m=
`
`0..wmmOEH
`
`o
`
`6..uP%mmMN_2c.mm,
`
`Mmm.
`
`V
`
`N3
`
`n_on_Hm.5000E2:
`
`Viptela, Inc. - Exhibit 1020
`Page 3
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 3 of 17
`
`US 2002/0010866 A1
`
`
`
`
`
`o__n:n_ozflmmmumw:Evcoqmm.cov_:_._25
`
`
`
`o_EmS€mm:58wx:__._m£o__m.dmmzuum.n__
`
`
`
`
`
`
`
`wommmfiumn__
`
`vm
`
`
`
`v_§a.oz__memHmm EW_>_v_c_u_Wmu.Euconmom.._3mEc_pl.WAE829W1W.mzmaflao3::E3:030mMx._o>zoz__wEw
`
`
`
`
`
`
`
`
`
`
`
`552:.359$_w>m=mEmEmm:o_Q_.._:_>_
`
`
`
`9:3go:_:o=m.::w¢_o9:umumfimwfimm
`
`Emucm.6:6:08.5>_Emv:mgou:_
`
`
`
`n_on_commEEmEn__:_um
`
`V.05
`
`Viptela, Inc. - Exhibit 1020
`
`Page 4
`
`o__p32mcozomccoo
`vcozzmc
`
`Viptela, Inc. - Exhibit 1020
`Page 4
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 4 of 17
`
`US 2002/0010866 A1
`
`
`
`CEmtoco=9:oEmm:=p-mwco_:8_.__E%_:3:
`
`
`
`
`
`
`
`£=<n_mm 98_.£n_n__
`
`o?mmmm
`
`Va
`
`
`
`NO_.om_E>mozm_
`
`mm
`
`amm
`
`Evmmz
` om
`
`95-9£m$_E8
`
`
`
`mmo_>.ow6mn;._.__wHuww.w.w_+
`
`om_._.<0_._.Zm_I._.D<
`
`mADE
`
`
`
`
`
`EzmxomcoBeam;:9?
`
`
`
`mmmcucmEmoiom=g-mm
`
`
`
`mmeuumn::o_§___$_oEwm
`
`0.05
`
`Viptela, Inc. - Exhibit 1020
`
`Page 5
`
`Viptela, Inc. - Exhibit 1020
`Page 5
`
`
`

`

`n0..uaC.hhnP.n0.na.mlPPAtnetaP
`
`Jan. 24, 2002 Sheet 5 of 17
`
`US 2002/0010866 A1
`
`
`
`mo<n_mmzoF<o_.En_<
`
`E3509:0£_o_.s_2Em
`
`
`
`€o_ton_=o=8__%_$
`
`
`
`comcozomccoo_mo_w>so_
`
`
`
`mm.mo_u_owmr_m__am..mm.
`
`ucmEwU
`
`
`
`:u1I1Uo..m._:wQmo:mn_0._.mmmmmmo_>_.
`
`
`
`
`
`
`
`
`
`Eucoamo._o:mwEmmcm_>_Va:a.«scam_zn_>29:9.
`
`
`
`mo<n_m._mzmmv_
`
`.m
`
`
`_u:mEunco£u_.sc:mEoaimnsm52.82E938.8.8228_§§aEm935.
`
`
`?_O___..._On__0_.._._0v__v%UU<.vE__n_n_n_
`
`
`
`
`n__.9Q.EmSE38Egan.._o:.__p#_“u..owu_m_.mF.__ton__m_mmo.__oomoosom.6:59m_:mumocmn_o_.9«gown.
`c_o;m9££2228m2$§8n:Exc__n_n_n_Bwmmzunm:o:mE¢Emm.u_
`
`
`
`
`mozmzfim.%:o%em_cozmczmwo.mwmon_o._.-coc.2m:_o:m_maumo._.
`
`
`
`
`
`
`m:_._o:_u_n=mczsomu£ow.__n_€0.50::o_ton__mEov_._omm:ms_w_u::m
`
`mwmfibmn__O>_
`
`«KGE
`
`mm:mm:
`
`
`
`v_uE.mn_0mm><._n:0.2905.>._=_uomn:
`
`etmV
`
`C.HIm,
`
`_
`
`.nPmhXE
`
`N6
`la
`0%.
`
`
`
`
`
`u9m_:mo_mocmn_ok.9oc_o:m_mn_om.o._
`
`mm;«m.om:
`
`
`
`
`
`:o_E=m_.Eoo._mmm:m_>_Zn_>
`
`
`
`
`
`
`
`>:__=..Eucoammm.6wmm:_on<n__o__n=n_Eoemm.
`
`
`
`
`
`Eucoamwm+0wxwmEHmcnzwccmmmwficmn__3m>_E.
`
`
`
`
`
`mE£_.om_mco:9co:wucm:ozwo_Ew£:<.
`
`
`
`
`
`w>9_cosgbocmcamcomwwm.
`
`
`
`
`
`O>_BEmamc_mE_m_>_.Emv._O~cozflsmamocwn_O._.o
`
`
`
`
`
`
`
`
`
`.025:v_:_._co_ton_co=mo__oE<.oma:m_>_o=oc:m
`
`
`
`
`
`
`
`
`
`
`
`LBMEE:m.m:s..mn_wmc_QQm_.:Ucm_muo_cmmzsmnEmacs:mmmmww_>_0
`
`
`
`
`
`
`
`Viptela, Inc. - Exhibit 1020
`Page 6
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 6 of 17
`
`US 2002/0010866 A1
`
`
`
`
`
`n_mm8mn__n_mm_8wn__
`
`
`
`.wmwcw_>_m_vc:m_._mmm:mS_mficzm
`
`
`
`
`
`Econ_O._.
`
`
`
`...=._o0n_O._.Econ_O._.Econ_0._.
`
`mn__
`
`_\n=mm:E_
`
`
`
`
`
`Nn_n_n__.n_n_n_Nn_n_n_wn_n_n_
`
`N_8_mEn_F_mo_@En_N_8_mEn___8_mEn_
`
`etmV
`
`mla,
`
`C.nI
`
`_
`
`.HPmhXE
`
`N7
`1a
`one...
`
`Viptela, Inc. - Exhibit 1020
`Page 7
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 7 of 17
`
`US 2002/0010866 A1
`
`Viptela, Inc. - Exhibit 1020
`
`Page 8
`
`FIG.8
`
`182
`
`EncryptedPayload
`
`
`
`For|PSec
`
`L a
`
`s
`'0
`:6
`<1:
`
`I E
`
`IPHeader
`
`Viptela, Inc. - Exhibit 1020
`Page 8
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 8 of 17
`
`US 2002/0010866 A1
`
`START
`
`210
`
`Receive Packet from
`local LAN
`
`I IP TRANSMIT I
`
`214
`
`
`
`
`218
`
`'5 Packet "3860
`encapsulated?
`
`Search lPSec Security Database
`
`YES
`
`230
`Apply IF, Filter rules to VPN
`Data Traffic before Encryption
`
`238
`
`Consult Routing Table
`
`242
`
`Apply IP Filter rules to non-
`VPN traffic but pass lPSec
`
`246
`
`YES
`
`
`
`Packet Destined
`
`for Bundle?
`
`“O
`
`250
`
`Output via PPP Link with IP
`Address from Routing Table
`
`423
`
`
`
`I lPSec ESP Transforms U
`
`FIG. 9A
`
`Viptela, Inc. - Exhibit 1020
`
`Page 9
`
`Viptela, Inc. - Exhibit 1020
`Page 9
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 9 of 17
`
`US 2002/0010866 A1
`
`
` IVC
`Bundle
`
`
`Process
`
`254
`
`TCP
`
`
`
`
`
`
`Encapsulate the
`Data Packet?
`
`Fragment to chosen
`Fragment Length
`
`282
`
`286
`
`Choose IVC for TCP Stream
`
`29
`
`0
`
`TCP Encapsulate and add IP
`and Bundle Headers
`
`270
`
`
`
`258
`
`Choose Inferior Virtual Circuit
`
`262
`
`Fragment Packet up to size of
`MTU
`
`26
`
`Translate |PSec IP Header to
`Match IVC
`
`
`
`NDR uses IP Filter to Match
`IVC address with PPP
`interfaces
`
`
`
`
`274
`
`Forward to correct PPP Link
`
`
`
`Is there more
`
`Data in Packet?
`
`
`
`
`278
`
`N°
`
`START
`
`Yes
`
`
`
`FIG. 9B
`
`Viptela, Inc. - Exhibit 1020
`
`Page 10
`
`Viptela, Inc. - Exhibit 1020
`Page 10
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 10 of 17
`
`US 2002/0010866 A1
`
`
`
`Receive Packets
`
`304
`
`Apply Traffic Filter Rules
`
`312
`
`Forward to Host
`
`308
`
`Packet needs to
`be forwarded?
`
`300
`
`
`
`
`
`320
`
`Destined for appropriate
`application
`
`Type of Packet?
`
`
`
`324
`
`Other TCP
`
`
` Tunnel Data
`Destined for appropriate
`application
`
`
`
`Packet?
`
`Yes
`
`328
`
`Remove IP, TCP and Bundle
`Headers
`
`FIG. 10A
`
`Viptela, Inc. - Exhibit 1020
`
`Page 11
`
`Viptela, Inc. - Exhibit 1020
`Page 11
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 11 of 17
`
`US 2002/0010866 A1
`
` lPSec Flow
`
`Tunnel Data
`Packet?
`
`Exists?
`
`
`
`
`356
`
`Remove ESP
`Header and
`
`
`
`
`
`Decrypt
`
`
`
`Yes
`
`Search for Bundle Match
`
`
`
`
`
`
`
`The Bundle
`Exists?
`
`Translate ESP IP Address to
`VPN Tunnel IP Address
`
`Yes
`
`
`
`3
`
`44
`
`235
`
`Discard Packet
`
`FIG. 10B
`
`Viptela, Inc. - Exhibit 1020
`
`Page 12
`
`Viptela, Inc. - Exhibit 1020
`Page 12
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 12 of 17
`
`US 2002/0010866 A1
`
`360
`
`Data
`
`IV Packet
`
`
`
`364
` Use TCP
`encapsulation
`for tunnel data
`
`
`
`376
`
`368
`
`Add bundle header
`
`Address translation
`
`only
`
`Add TCP Header
`
`380
`
`384
`
`
`
`
`
`
`
`Add IP Header
`
`372
`
`PPP links
`
`FIG. 11
`
`Viptela, Inc. - Exhibit 1020
`
`Page 13
`
`Viptela, Inc. - Exhibit 1020
`Page 13
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 13 of 17
`
`US 2002/0010866 A1
`
`
`
`_8_@EQtam
`
`v_c__
`
`N®._®_..:m_
`
`E._8_mEq
`
`m.w_pm__m>m
`
`
`
`E0:moz92mm
`
`
`
`
`
`mmmznumxc___moo_
`
`n:5282
`
`n:52am2Bmccoo
`
`Emo_Eo£:<
`
`cozowccoo
`
`
`
`26:Emflo
`
`G>@233
`
`
`
`Zn_>Bmzommz
`
`.n__Vm_9mEm.ma
`
`o>_E90
`
`qfimm
`
`3%
`
`Zn_>w_
`
`
`
`was>cmm:_wvmm
`
`55mmxm<
`
`2EmccooBn__.0".
`
`wowwow
`
`o>_390
`
`Q23
`
`3..mozgamm
`
`o>_
`
`=m>>ama
`
`26:5..
`
`
`
`o_%__m>mx:__
`
`2:3zm>5.,%8m
`
`NH.05
`
`90:39
`
`._uE»I0:»:0>_9mo_Emr=:<:mow>
`
`
`
`m::m_xo:_o_.
`
`.m..V
`
`6t
`
`C.HIma
`
`_
`
`anmhXE
`aP
`
`AU2AU1
`416
`
`00
`
`Viptela, Inc. - Exhibit 1020
`Page 14
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 14 of 17
`
`US 2002/0010866 A1
`
`
`
`
`
`_o:.:oonewEma.mwm:v_u<v_mm
`
`ode_>_m._V
`
`V9.
`
`
`
`m.__.6_oom_o§o_>_
`
`
`
`2m>>E__:omEn:_>_E:98Sam
`
`
`
`
`
`Aommkxw.ommwoo.qo._o__>_8592:.
`
`
`
`ND»>mo_om_._oo
`
`ommEI
`
`E38:8.
`
`cozsowxmwnoov
`
`892%Emu
`
`oh»_£oOEuwzd
`
`_>_<mE08#25
`
`$3Em8_§a_mwon
`
`mmw
`
`
`
`umsoo_>_<N?
`
`
`
`
`
`mmw$509homwmowNFmx.<m.o>_Emmmvwm
`
`oomBo.m._oE._2m:m=HwmmmowvmwI....--II
`
`
`
`Efiwam.toQ_=:E
`
`ai
`
`wmwI95eommmor84¢IIw:o:oEoc>wmAmi
`
`
`
`m__LOQ__.®CLO—.=WWEOQ_.W_._®Wo
`
`.m..V
`
`6t
`
`C.HIma
`
`_
`
`anmhXE
`aP
`
`AU2AU1
`516
`
`00
`
`Viptela, Inc. - Exhibit 1020
`Page 15
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 24, 2002 Sheet 15 of 17
`
`US 2002/0010866 A1
`
`
`
`DOUG
` ODDSUUUU(Z
`
` Modemsor ISDNTAs
`
`ISPConnections
`Multiple
`
`FIG.14
`
`Viptela, Inc. - Exhibit 1020
`
`Page 16
`
`Viptela, Inc. - Exhibit 1020
`Page 16
`
`

`

`n0..uaonHhnP.n0.na.mlPPAtnontaP
`
`Jan. 24, 2002 Sheet 16 of 17
`
`US 2002/0010866 A1
`
`
`
`>m>>9.m0aqua
`
`DDDD
`
`D”_
`
`DDDD
`
`
`
`mEoEmE._mm.mmmo2n_
`
`
`
`oo_.cow..oE9E0...
`
`2.05
`
`
`
`oo_t0_m.=c0U
`
`{EI___.—
`
`
`
`wc_..__owwmm.__.._.
`
`u®C._®E_
`
`CO_u0®CCOO
`
`83
`
`
`
`8:5Eosam__msw
`
`
`
`9.2..%_w_on_m_8__8B.2%_:_2
`
`
`
`
`
`Viptela, Inc. - Exhibit 1020
`
`Page 17
`
`Viptela, Inc. - Exhibit 1020
`Page 17
`
`
`
`
`
`
`

`

`mMP
`
`.mM.
`
`71
`
`US 2002/0010866 A1
`
`M.o__m_F=..._wO
`
`
`
`
`
`.m...xmflzn_m_2:_=_>_8__25m
`
`MIMEEE%mEw_ooE8.95
`o_q=_:_>_\.‘
`
`
`2H._V®mW.>ms>2mOv_.o>>_.wZ__mEwNam
`
`
`M,25_So_m¢mcanown
`
`_H_|g_».UA.c_w:w_®::3._.um_:ommm%ww.m.m¢._wo<11%mEzom_..._mo<EohmQ+wwm
`
`
`
`
`
`>m>>2m_0Eiwm
`
`
`
`vzozzwz__mEm
`
`mumBoeom
`
`>m.>>£mO
`
`3.05
`
`Viptela, Inc. - Exhibit 1020
`
`Page 18
`
`Viptela, Inc. - Exhibit 1020
`Page 18
`
`
`

`

`US 2002/0010866 A1
`
`Jan. 24, 2002
`
`METHOD AND APPARATUS FOR IMPROVING
`PEER-TO-PEER BANDWIDTH BETWEEN
`REMOTE NETWORKS BY COMBINING
`MULTIPLE CONNECTIONS WHICH USE
`ARBITRARY DATA PATHS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application claims priority to a U.S. provi-
`sional application entitled “METHOD AND APPARATUS
`FOR IMPROVING PEER-TO-PEER BANDWIDTH
`BETWEEN REMOTE NETWORKS BY COMBINING
`MULTIPLE CONNECTIONS WIIICII USE ARBITRARY
`DATA PATHS” filed on Dec. 16, 1999, Ser. No. 60/172,369,
`which application is hereby incorporated by reference.
`
`FIELD OF THE INVENTION
`
`[0002] The present invention relates generally to intercon-
`necting private peer computer networks securely using a
`public computer network and aggregated multiple links
`between the private networks and the public computer
`network, where the aggregated multiple links improve the
`performance of the connection between the private peer
`computer networks.
`
`DESCRIPTION OF THE RELATED ART
`
`[0003] Businesses today are commonly multi-site opera-
`tions. Even within a given locale, it is very common for a
`business to have several buildings located some appreciable
`distance from each other. However, these businesses must
`stay in close communication not only through their tele-
`phone system but through their computer systems as well.
`Not only is there a requirement for communication among
`the multi-site operation but the communication must be fast,
`reliable, confidential, and, if possible, not too expensive.
`
`[0004] FIG. 1 shows a multi-site operation between Los
`Angeles 10, Chicago 12, New York 14 a11d Atlanta 16, in
`which the Various s'tes communicate by means of dedicated
`point-to-point links 18, 20, 22, 24, 26, 28 that comprise a
`wide-area network 30. Each of the sites typically has
`a private network, such as one or more LANs (not shown in
`FIG. 1), on which i relies for internal communications. The
`point-to-point
`links interconnect
`these private networks,
`with the goal being to have the system appear to the users as
`a single, integrated system. However, to achieve this goal,
`the point-to-point
`inks must operate at high speed. The
`common solution is to use dedicated leased lines, such as T1
`lines, from the public telephone network. These dedicated
`leased lines are fas , reliable and confidential.
`
`[0005] However, a dedicated WAN 30, such as that shown
`in FIG. 1, employing point-to-point leased lines between
`their private netwo ks incurs high telecommunications tar-
`iffs and thus is a costly solution to the multi-site communi-
`cations problem.
`
`[0006] FIG. 2 shows an alternative approach to the prob-
`lem, in which each site 10, 12, 14, 16 is connected to a public
`computer network 32, such as the Internet. This approach
`appears to be a viable alternative, but, in fact, lacks several
`requirements which a solution must meet. First, while the
`cost is low, because only local connect charges are incurred,
`the communications between the sites are not confidential.
`
`Second, the reliability of the computer network is sometimes
`a problem and third, the speed of the interconnection is
`highly variable and often to low for most businesses.
`
`To solve the confidentiality problem, a virtual
`[0007]
`private network (VPN) can be established between the
`multiple sites. A VPN simulates some of the properties of a
`private network in the setting of a public network, such as
`the Internet, by sending data from one private network to the
`other through a tunnel (a secure private path) through the
`public network. A VPN arrangement means that each site
`only needs one network connection so there is a large cost
`saving compared with multiple dedicated circuits. More-
`over, a VPN can connect sites located virtually anywhere in
`the world as long as there is access to the public network.
`
`[0008] However, one problem that still remains even with
`the use of VPNs is the speed of the connection and in many
`cases this speed is limited not by the speed of the public
`network on which the VPN is established but the speed of
`the interconnection between the private site and the public
`network, which is typically not satisfactory for today’s
`businesses.
`
`[0009] A common interconnection between a private site
`and a public network, such as the Internet, is a PSTN dial-up
`connection on which the Point-to-Point Protocol (PPP) is
`run. PPP is a data link protocol that has been designed as the
`Internet standard for connecting (and disconnecting) a pri-
`vate host
`to the Internet Service Provider (ISP). Other
`physical links, such as ADSL and ISDN, can also be used,
`but the protocol remains PPP. These physical links still do
`not solve the speed problem sufliciently. It is highly desir-
`able to have a facility for aggregating the physical links
`between the private host (via a router possibly) and the
`Internet so that high speed and selectable speed connections
`are possible using the common types of physical links that
`are available, the PSTN dial-up link being the most avail-
`able.
`
`[0010] Aprotocol that attempts to fill the need to aggre-
`gate physical
`links for a high speed connection is the
`Multi-Link Point-to-Point Protocol
`(ML-PPP). FIG. 3
`shows ML-PPP being employed primarily by users desiring
`a l1igl1-speed dial-up Internet connections using ISDN. In
`this figure,
`there are two 64 Kbyte per second, ISDN
`B-channels 34, 36 which are aggregated into one 128 Kbyte
`per second channel. These connections couple the private
`network 38 via a router 40 to the public network 32, the
`Internet. For this arrangement to work, the customer pre-
`mises equipment and the ISP POP 42 dial-in equipment must
`both support ML-PPP.
`
`[0011] However, this aggregation solution, while perhaps
`providing some relief to the speed problem, re-introduces
`the confidentiality problem. The protocol does not allow
`users to configure the bundled, dial-up Internet connections
`to securely tunnel private data through the Internet 32
`between a local private network 38 and a remote private
`network 46, which is a requirement for a Virtual Private
`Network (VPN). In other words the confidentiality problem
`now exists between the private local and remote hosts and
`the Internet.
`
`[0012] The Multi-Link PPP scheme creates a further prob-
`lem. This problem, called the “Multi-hnk hunt group split-
`ting problem,” occurs because the ML-PPP was not
`
`Viptela, Inc. - Exhibit 1020
`
`Page 19
`
`Viptela, Inc. - Exhibit 1020
`Page 19
`
`

`

`US 2002/0010866 A1
`
`Jan. 24, 2002
`
`designed to handle an intervening network, such as the
`Internet, between the local private network and the remote
`private network. It was developed primarily to interconnect
`two or more networks directly by multiple point-to-point
`links to improve bandwidth.
`
`[0013] Briefly stated, the problem is that PPP links witl1ir1
`a bundle become dissociated by terminating at multiple
`intervening nodes rather than at a single node. Usually these
`nodes are Network Access Servers (NAS) that receive the
`dial-up calls. ISPs that offer MI.-PPP allow dial-ins to the
`Point-of-Presence (PoP, a switching office of an ISP) using
`the same phone number for all of the links in the bundle. A
`rollover or hunt group of analog lines is commonly used for
`example to route all incoming calls to the available modem
`pools, NASs and routers. The primary and secondary con-
`nections in the Multi—link bundle thus may get established to
`different NAS or remote access concentrators on the internal
`network inside each PoP. The effect is that network nodes
`
`within the public network lose a needed association between
`the links in the bundle.
`
`[0014] An existing protocol has been proposed to fix this
`splitting problem. One of these is the Layer 2 Tunneling
`Protocol (L2TP). LT2P extends the PPP model by allowing
`the link layer (layer 2) and PPP endpoints to reside in
`different devices interconnected by a packet-switched net-
`work. Using L2TP,
`the user has an L2 connection to an
`access concentrator (e.g., modem bank, ADSL, DSLAM)
`and the concentrator tunnels individual PPP frames (frag-
`ments) to a single Network Access Server (NAS). This
`allows the actual processing of PPP packets to be separated
`from the termination of the L2 circuit. The association
`
`between links in the bundle is preserved because the PPP
`fragments are recombined, by means of the tunneling, at a
`single device, the NAS or router.
`
`the Point-to-Point Tunneling
`[0015] Another protocol,
`Protocol (PPTP) has also adopted this approach. However,
`despite these improvements problems still remain. Both
`solutions (L2TP and PPTP) require that the ISPs update their
`NAS software or router firmware in every device and in each
`of their PoPs, in effect placing the burden of aggregating
`PPP fragments on the PoP LAN backbone that interconnects
`the L2 access device and the NAS. This result is simply
`unworkable for several reasons.
`
`[0016] First, placing the burden of aggregating PPP frag-
`ments onto the PoP LAN introduces additional latency and
`possibly performance bottlenecks. Second, all of the ISPs
`PoPs must support ML-PPP with fragment recovery. The
`likelihood of the latter being met, especially where there are
`international tunnel connections and different ISPs, each
`with potentially different equipment,
`is very low. Third,
`ML-PPP configurations and connection types are limited,
`inconsistent or totally non-existent at locations serviced by
`ISPs. Some ISPs offer ML-PPP connections over ISDN
`
`using the Basic Rate Interface (BRI). Some ISPs that offer
`higher speed ISDN connections require that each site have
`a router that
`includes proprietary multi-chassis ML-PPP
`extensions that are consistent with the equipment at their
`PoPs. Sometimes ISDN is not even available to the private
`host or network that needs to connect to the Internet.
`
`[0017] This leaves the operator of the private site or
`network without a guaranteed solution that can easily
`improve bandwidth between remote locations regardless of
`whether they are using analog, digital or a combination of
`connections to the Internet.
`
`[0018] Thus, there is a need a low-cost, high-speed, scal-
`able-speed, confidential connections between the private
`networks of multiple, geographically dispersed sites that
`have the approximately the same characteristics as private,
`high-speed point-to-point
`links
`interconnected between
`those sites.
`
`BRIEF SUMMARY OF THE INVENTION
`
`[0019] The present invention is directed towards such a
`need.
`
`[0020] The present invention establishes a virtual private
`network (VPN) between two edges of a public computer
`network and connects each of these edges to a private
`network to permit communication between the private net-
`works.
`
`[0021] One advantage of the present invention is that it
`provides high speed and scalable bandwidth to businesses
`requiring site-to-site connections between their private
`Local Area Networks.
`
`[0022] Another advantage of the present invention is that
`IP datagrams can be split, recombined and sequenced across
`an arbitrary number of dial-up Internet connections regard-
`less of how the IP packets traverse the Internet and without
`being limited by the equipment at the PoP or any other
`Internet nodes. This makes the present invention indepen-
`dent of the particular ISP’s access equipment so that links
`can be spread across multiple ISPs for increased reliability
`should a P0P fail.
`
`[0023] A further advantage of the present invention is that
`data can be transferred between private networks using a
`variety of connection types between the private network and
`the Internet Service Providers at each location. These con-
`
`nection types include analog modem (PSTN), ISDN, ADSL
`or leased-line T-1 links.
`
`[0024] Yet another advantage of the present invention is
`that a high level of resilience can be maintained because a
`dropped or failed connection can be re-established while the
`VPN is operating.
`
`[0025] Yet another advantage of the present invention is
`that bandwidth is configurable by setting connection
`throughput thresholds and can be tuned for the best perfor-
`mance and the lowest ISP charges.
`
`[0026] Yet another advantage is that the present invention
`can combine multiple Internet connections from a site or
`spread them across a variety of PoPs.
`
`[0027] Yet another advantages is that the present invention
`can operate in a “many to one” scenario in which a large
`number of sites use multiple connections to improve band-
`width between them and a central site that employs one or
`more high-speed connections.
`
`[0028] Yet a further advantage is that the present invention
`can ensure that the tunneled data can traverse the majority of
`routers and firewalls within the Internet successfully, even if
`they restrictive and only allow a set number of protocols to
`pass.
`
`Viptela, Inc. - Exhibit 1020
`
`Page 20
`
`Viptela, Inc. - Exhibit 1020
`Page 20
`
`

`

`US 2002/0010866 A1
`
`Jan. 24, 2002
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0029] These and other features, aspects and advantages of
`the present invention will become better understood with
`regard to the following description, appended claims, and
`accompanying drawings where:
`
`[0030] FIG. 1 shows a multi-site operation between Los
`Angeles, Chicago, New York and Atlanta,
`in which the
`various sites communicate by means of dedicated point—to—
`point links that comprise a wide-area network
`
`[0031] FIG. 2 shows an alternative approach to the prob-
`lem, in which each site is connected to a public computer
`network, such as the Internet;
`
`[0032] FIG. 3 shows ML-PPP being employed primarily
`by users desiring a high-speed dial-up Internet connection
`using ISDN;
`
`[0033] FIG. 4 is a simplified diagram of a system in
`accordance with the present invention;
`
`[0034] FIG. 5 illustrates an IP packet that is secured by the
`IPSec Protocol using ESP services in tunnel mode;
`
`[0035] FIG. 6 shows the fields of a standard IP Packet
`Header. Standard IP fragmentation is used in the present
`invention;
`
`[0036] FIG. 7A illustrates the several blocks that cooper-
`ate to carryout important functions of the present invention;
`
`[0037] FIG. 7B shows the protocol stack for SVC and the
`IVCs that comprise the SVC;
`
`[0038] FIG. 8 shows a fragmented tunnel data packet with
`TCP encapsulation;
`
`[0039] FIGS. 9A and 9B show a flow chart of the process
`for transferring packets from a private LAN, through the
`gateway to the Public Network;
`
`[0040] FIGS. 10A and 10B show a flow chart that illus-
`trates the process of receiving a packet over the VPN;
`
`[0041] FIG. 11 shows a flow chart of the TCP encapsu-
`lation sequence;
`
`[0042] FIG. 12 shows a flow chart of the process for
`negotiating additional IVCs for a SVC;
`
`[0043] FIG. 13 shows a block diagram of a gateway
`system, in accordance with the present invention;
`
`[0044] FIG. 14 shows a typical system that can be sup-
`ported by the Small Network Gateway;
`
`[0045] FIG. 15 shows another typical installation that can
`supported by the SNG; and
`
`[0046] FIG. 16 an alternative embodiment of the present
`invention which includes a standard or industrial server PC
`
`computer for high capacity implementations.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`referred to as a gateway) 60 is connected to one edge of the
`public network 32 by means of one or more links, ILink 1-N
`62, 64, 66, of a first set of links. Each link 62-66 of the first
`set terminates at a one of the PoPs 50, 56 within the public
`network 32. A Responder device (also referred to as a
`gateway) 70 connects at another edge to the public network
`32 by means of one or more links, RL1-N 72, 74, 76, of a
`second set of links. Each link 72-76 of the second set of links
`terminates at one of the PoPs 52, 58 within the public
`network 32. One link that interconnects the Responder and
`the public network must have a static Public IP address, but
`the other links of the second set can use dynamic IP
`addresses. A Virtual Private Network 80 is established
`between the Initiator 60 and the Responder 70 and includes
`one of the first set of links, the public network and one of the
`second set of links. The VPN connects a private network 82
`connected to the Initiator 60 to the a private network 84
`connected to the Responder 70.
`
`[0048] The Virtual Private Network is a tunnel between
`the Initiator and Responder that is implemented using IPSec,
`the Layer 3 security protocol for the Internet, operating in
`tunnel mode. Information is available regarding the Internet
`Security Protocol (IPSec) from IETF (the Internet Engineer-
`ing Task Force, a standards setting body for the Internet).
`However, a brief description of the protocol follows.
`
`[0049] The IPSec Protocol is a protocol to provide security
`services on IP networks. The protocol operates at Level 3,
`the network layer. IPSec provides a choice of two kinds of
`security services, an authentication service and a confiden-
`tiality (security) service. It also provides for an Internet Key
`Exchange that allows parties to negotiate methods of secure
`communication through special exchanges, known as secu-
`rity associations (SA). The parties of the security association
`agree on encryption methods, lock and unlock keys and the
`useful life of the key.
`
`[0050] The authentication service attempts to guarantee
`that the sender is actually the sender named in the transac-
`tion. This service is directed towards prcvcnting impostcrs
`from intruding in a communication process between other
`parties. The IPSec protocol implements the authentication
`service by means of an Authentication Header
`When
`a packet is sent out a hash function is performed over the
`entire packet based on the contents of the packet and a
`known key. The result of the hash is included in the
`Authentication Header. The hash will fail if the contents of
`
`the packet have been altered when the packet is checked by
`the receiver.
`
`[0051] The confidentiality or security service of IPSec
`attempts to ensure that only the two ends involved in the
`communication will be able to decipher the contents of a
`message that has been encrypted for security purposes. The
`IPSec Protocol implements the security service by means of
`the Encapsulating Security Payload (ESP) header. In this
`case, a packet is encrypted using an agreed upon encryption
`algorithm with keys that are known to both the sender and
`the receiver.
`
`[0047] FIG. 4 is a simplified diagram of a system in
`accordance with the present invention. A public computer
`network, such as the Internet 32,
`is represented by the
`cloud-shaped figure. The public network includes one or
`more Points of Presence (P0P) 50, 52, 54, 56, 58 for one or
`more Internet Service Providers. An Initiator device (also
`
`[0052] The IPSec Protocol has two major modes of opera-
`tion, the transport mode and the tunnel mode. The transport
`mode is used to add security to packets traveling between
`two IP systems. The tunnel mode provides security services
`between two IP systems that act as Security Gateways (SG).
`In the tunnel mode an original IP packet is encapsulated in
`
`Viptela, Inc. - Exhibit 1020
`
`Page 21
`
`Viptela, Inc. - Exhibit 1020
`Page 21
`
`

`

`US 2002/0010866 A1
`
`Jan. 24, 2002
`
`an IPSec header and then sent from one security gateway to
`the other gateway which upon receipt of the packet, uses the
`IPSec header for security purposes and recovers the original
`IP packet. Thus IPSec provides level 3 tunneling because the
`payload of the IPSec packet is IP trafiic.
`
`becomes its own smaller packet with its own IP header and
`is routed independently of any other packets. This means
`that fragments can arrive out of order. However, there is
`appropriate information in the IP header to reassemble the
`fragments at the destination.
`
`[0053] FIG. 5 illustrates an IP packet that is secured by the
`IPSec Protocol using ESP services in tunnel mode. The
`diagram shows the portion of the packet 94, 96, 98 that is
`encrypted and the portion of the packet 92, 94, 96, 98 that
`is hashed for authentication. The components of the secured
`IP packet include a New IP header 90, an ESP header, 92 an
`original IP header 94, the IP payload 96, and ESP trailer 98,
`and ESP Authentication trailer 100. This IPSec packet 102
`then can be used to carry IP addresses used on private site
`LANs from one site to another, through the public network,
`in effect, hiding the private source and destination addresses
`of the LAN from users on the public network.
`
`[0054] Thus, the IPSec ESP tunnel mode provides site-to-
`site security between two gateways that are separated by the
`public network. However, the IPSec ESP Tunnel mode does
`not provide a way to treat multiple tunnels between an
`Initiator and the Responder as a unified channel or bundle
`having a bandwidth that is the aggregate of the bandwidth of
`the individual tunnels.
`
`[0055] The present invention provides the facilities to, in
`fact,
`treat multiple tunnels between the Initiator and
`Responder as a unified channel. Such a unified channel is
`called a superior virtual circuit (SVC) and the individual
`tunnels are called inferior virtual circuits (IVCs). An IVC is
`a peer-to-peer connection between an initiator and responder
`that includes a PPP link between the initiator and the public
`network, a connections through the public network, and an
`equivalent PPP link between the responder and the public
`network.
`
`[0056] A necessary condition for treating the IVCs as a
`unified channel is that the packet load must be distributed
`approximately equally over each the IVCs. If this condition
`were not met, some of the IVCs would take most of the load
`causing saturation of those IVCs while other IVCs would
`stand idle. This unbalanced condition would not lead to a
`
`SVC whose bandwidth is approximately the aggregate of the
`individual bandwidths of the IVCs, nor one that would have
`scalable bandwidth.
`
`[0057] One way to balance the packet load over the IVCs
`is to fragment a tunnel source packet and to distribute the
`smaller packets across available IVCs to share the load
`equally. Th