`
`11111111111110111111010111111R Illlillill[1111li11111111111111111111110111111
`
`(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2003/0118029 Al
`Jun. 26, 2003
`Maher, III et al.
`(43) Pub. Date:
`
`(54) METHOD AND APPARATUS FOR
`ENFORCING SERVICE LEVEL
`AGREEMENTS
`
`(76)
`
`Inventors: Robert Daniel Maher III, Plano, TX
`(US); James Robert Deerman, Lucas,
`TX (US); Milton Andre Lie,
`McKinney, TX (US); Mark Warden
`Hervin, Plano, TX (US)
`
`Correspondence Address:
`Craig J. Cox
`Netrake Corporation
`Suite 100
`3000 Technology Drive
`Plano, TX 75074 (US)
`
`(21) Appl. No.:
`
`10/260,768
`
`(22) Filed:
`
`Sep. 30, 2002
`
`Related U.S. Application Data
`
`(63) Continuation of application No. 09/653,521, filed on
`Aug. 31, 2000, now abandoned.
`
`Publication Classification
`
`(51) Int. C1.7
`(52) U.S. Cl.
`
` HO4L 12/28
` 370/395.21; 370/395.43
`
`(57)
`
`ABSTRACT
`
`A network device for enforcing service level agreements is
`described that is able to scan the contents of entire data
`packets including header and payload information. The
`network device includes memory for storing subscriber
`information, policies and statistics. The traffic flow scanning
`processor scans the header and payload information from
`each data packet, which is used to associate each data packet
`with a particular subscriber, classify the type of network
`traffic in the data packet and to enforce the particular policies
`associated with the subscriber. The traffic flow scanning
`processor produces a treatment for the data packet based on
`the scanning. The scanned data packets and the associated
`treatments are then passed to a quality of service processor,
`which modifies the data packets if necessary and enforces
`resource allocation according to the preprogrammed poli-
`cies.
`
`37
`
`/
`
`/
`37
`so l
`
`GO
`
`y- GO
`22
`
`60
`
`37
`
`.
`
`/
`
`/ -38
`
`
`436
`
`ENTERRISE
`..,,,
`rdid .—_._
` 38
`
`''\
`40 \
`'\
`
`1
`
`r 38
`/
`
` Li
`
`18 — 1- 1 ri
`'—r-
`X=..)(
`--1_
`
`17
`
`17
`
`GO
`
`GO
`
`20
`
`/
`
`GO
`/
`/ 22 20
`
`1711
`
`17
`
`17
`
`17
`
`'
`
`GO
`1 -20
`
`X
`
`18 6°
`
`GO
`
`2
`
`X =
`
`X
`
`38
`
`14-\i /
`
`20
`
`HOSTING
`
`60
`
`1\
`
`SERVICE
`PROVIDER
`
`/ 80-
`I
`I
`5°."\-1-----4- 42
`50-H
`RAS —X
`
`
`1:172-44
`
`DNS
`
`X
`_ _
`
`\70ICE OVER IP
`
`x
`
`6U
`GO-N.
`
`MG
`
`22 f
`
`4B
`
`48
`
`48
`
`50
`
`48
`
`50
`
`47
`
`,/
`
`(
`
`j
`
`38
`
`./
`
`PRIVATE
`IP
`NETWORK
`
`22!
`
`\,60
`
`47
`
`PUBLIC
`IP
`NETWORK
`
`/
`
`60
`
`12
`
`10
`
`/
`
`x
`
`16--\ /
`/WEBFARM
`
`II
`
`IDS
`
`26
`30
`
`LB
`
`\
`\
`\ \
`\
`\
`
`
`
`
`
`
`'N..
`
`20
`
`24
`
`28
`
`22
`
`20\1
`
`24
`
`28
`
`LB
`
`IDS
`
`2G
`
`30
`
`I
`I
`
`/
`
`30
`
`/
`
`Cloudflare - Exhibit 1025, page 1
`
`Cloudflare - Exhibit 1025, page 1
`
`
`
`Patent Application Publication
`
`S jo 1 lamIS
`
`TV 6ZOXTTO/£OOZ Sfl
`
`17
`60
`
`WF
`20
`
`WF
`
`WF 17
`
`2><1 GO
`--1 j--20
`
`WF
`
`17
`
` .\
`17
`
`17
`
`I
`
`WF
`
`16 GO
`17
`N--„ 60
`.20\
`'1
`
`22
`
`24-11
`
`LB
`28-)1
` IL
`
`IDS
`
` \ I
`
`26
`
`i I
`30 i I
`J I
`
`30
`
`/./
`
`/
`/,
`
`60116
`1 -\1WEBFARM
`
`2
`
`24
`
`\
`
`12
`
`IDS
`I
`26
`
`30
`
`LB V-28
`
`FIG. I 36
`
`ENTERRISE
`//' )(-38 1-38
`
`f Jo
`
`(-40 \
`
`(('
`/7
`
`17
`
`WF
`GO
`
`60
`/ 22 20-N-
`
`X
`14_J
`20-1
`HOSTING
`GO,/
`
`X
`
`40
`
`38
`
`38
`
`38
`
`37
`
`GO
`
`37
`
`60
`
`/ 18
`so!
`
`80
`22
`\\
`
`34\ 22
`
`
`
`/ 60< SERVICE
`PROVIDER
`42
`
`50
`
`50
`
`50
`
`50
`
`-----2><=
`RAS
`f 44
`DNS
`
`X
`
`X
`
`0
`
`22
`---
`
`PRIVATE
`IP
`NETWORK
`
`22!
`/
`
`\-60
`
`47
`
`(11)
`/ VOICE OVER IP
`
`46~,
`
`-/-5\ \\GO
`
`G
`
`47
`
`\
`
`48
`
`48
`
`—
`48
`
`50
`
`48
`
`50
`
`PUBLIC
`IP
`NETWORK
`
`10
`
`Cloudflare - Exhibit 1025, page 2
`
`Cloudflare - Exhibit 1025, page 2
`
`
`
`Patent Application Publication
`
`S Jo Z lamIS
`
`IV 6Z08110/£00Z Sfl
`
`100
`
`FIG. 2
`
`y-108
`
`
`Jj
`
` y-112
`
`
`
` y-118
`
`102
`
`126
`
`PHY
`INTERFACE
`
`RAM
`
`140
`
`RAM
`
`ez-104
`
`HEADER
`PROCESSOR
`
`PAYLOAD
`ANALYZER
`
`134
`7 132
`110
`126
`
`RAM
`
`QoS ,PROCESSOR
`
`116
`
`136
`138
`126
`
`PM
`
`T — TS
`T T
`126 128 136
`
`108
`
`DBASE
`
`DBASE
`
`I \-114
`
`I
`
`FLOW
`MANAGEMENT
`PROCESSOR
`
`-122
`
`y-124
`
`MICROPROCESSOR
`
`1 142
`
`L.._
`
`PHY
`INTERFACE
`
`120
`
`Cloudflare - Exhibit 1025, page 3
`
`Cloudflare - Exhibit 1025, page 3
`
`
`
`Patent Application Publication
`
`S .0 £ lamIS
`
`IV 6Z08110/£00Z Sfl
`
`FIG. 3
`-------------------------------
`r366
`r366
`J
`i
`r304
`..
`
`STRING
`STRING
`MEMORY
`MEMORY
`
`[,-336
`
`SCRIPT
`MEMORY
`
`
`
`7
`r370 .
`)
`LEAF
`STRING
`MEMORY
`_
`
`"--334
`
`I-3313
`
`SCRIPT H.
`ENGINE
`
`REGISTER
`
`7
`
`CONTEXT
`BUFFERS
`
`SCHEDULER
`
`
`
` /-344
`
`PSE
`INTERFACE
`
`I
`
`362, t 3841
`STRING
`PRE—
` PROCESSOR
`
`GP
`ALU
`
`
`
`3 60
`
`306
`
`V
`LEAF
`STRINGS
`COMPARE
`
`368
`
`316
`
`PACKET
`MEMORY
`CONTROLLER
`1
`
`PACKET
`LOADER
`34Q1
`
`374
`
`• - em.•
`
`
`
`CONTEXT
`BUFFERS
`342'
`
`f-352
`
`)(-350
`
`SESSION
`CAM
`
`LINK
`LIST
`MEMORY
`
`320
`
`112
`
`TRANSMIT
`ENGINE
`
`HOST
`INTERFACE
`
`SESSION
`RAM
`CONTROLLER
`
`
`' 346!
`
`
`
`3481
`
`STRING
`MEMORY
`
`STRING
`MEMORY
`
`\-366
`
`\-366
`
`128
`
`372
`
`SESSION
`RAM
`
`354
`
`Cloudflare - Exhibit 1025, page 4
`
`----
`._ /
`-
`1
`
`302
`
`/ 310
`
`.
`
`
`
` > INTERFACE
`
`
`
`1 — 312
`
`1 6
`REASSEMBLY
`ENGINE
`
`__r im
`- -----
`/ 318
`
`-
`
`LINK LIST r-- ACTIVE
`
`
`
`MEMORY
`I
`
`I
`i
`
`CONTROL
`
`i \ -332
`—1 330—\ V
`... '
`NEXT
`LINK LIST
`FREE
`CONTROLLER
`ENGINE
`
`Cloudflare - Exhibit 1025, page 4
`
`
`
`Patent Application Publication
`
`S Jo 17 13MIS
`
`IV 6Z08110/£00Z Sfl
`
`FIG. 4
`
`/
`
`402
`
`J -408 ROUTE ENGINE CARD
`
`,r-410
`
`HEADER
`PROCESSOR
`
`PAYLOAD
`ANALYZER
`
`_/-412
`
`INGRESS
`00S
`PROCESSOR
`
`EGRESS
`DoS
`PROCESSOR
`
`416
`
`\-418
`
`FLOW
`MANAGEMENT
`PROCESSOR
`
`REC
`
`REC
`
`402'
`
`402'
`
`_/-404
`
`400
`
`SWITCH
`FABRIC
`
`A
`
`MANAGEMENT
`CARD
`
`REC
`L _ _ _ _ _ _ _ _ 40?__ _
`
`_ _
`
`_ _
`
`_ _ _ _ _ _J
`
`Cloudflare - Exhibit 1025, page 5
`
`Cloudflare - Exhibit 1025, page 5
`
`
`
`Patent Application Publication
`
`Jun. 26, 2003 Sheet 5 of 5
`
`US 2003/0118029 Al
`
`500
`
`FIG. 5
`y-502
`
`ASSOCIATE DATA PACKET WITH
`CUSTOMER INFORMATION AND CLASSIFY
`CONTENTS OF DATA PACKET
`
`COMPARE AVAILABLE
`CAPACITY FOR TRAFFIC
`TYPE WITH UNIT CAPACITY
`FOR TRAFFIC TYPE
`
`5°G
`
`500
`
`YES
`
`ENOUGH
`AVAILABLE
`CAPACITY
`
`NO
`
`/-518
`
`CHECK OTHER NRT QUEUES
`FOR AVAILABLE CAPACITY
`
`f -510
`
`SEND DATA PACKET TO
`APPROPRIATE VARIABLE
`BIT RATE NAT-QUEUE
`
`512
`
`DECREMENT AVAILABLE
`CAPACITY BY
`UNIT CAPACITY
`
`518
`
`v-540
`
`AVAILABLE
`CAPACITY IN OTHER
`NRT QUEUE
`
`YES
`
`SEND TO APPROPRIATE
`VBR. NRT QUEUE
`
`520
`
`504
`
`ARE CONTENTS
`REAL TIME OR NON
`REAL TIME
`
`RT
`
`NAT
`
`COMPARE SIZE OF
`DATA PACKET WITH
`AVAILABLE CAPACITY
`
`y-528
`
`528
`
`YES
`
` •••
`
`ENOUGH
`AVAILABLE
`CAPACITY
`
`NO
`
`MARK PACKET
`FOR DELETION
`
`y-534
`
`y-536
`
`FORWARD TO
`QUEUE
`
`NO f -524
`
`SEND TO AVAILABLE
`BIT RATE QUEUE
`FOR BEST EFFORTS
`TREATMENT
`
`DECREMENT AVAILABLE
`CAPACITY OF QUEUE
`
`y-522
`
`540
`
`540
`f-532
`
`DECREMENT AVAILABLE
`CAPACITY BY
`PACKET SIZE
`
`540
`
`f -530
`
`SEND TO APPROPRIATE
`REAL TIME VBR QUEUE
`
`Cloudflare - Exhibit 1025, page 6
`
`Cloudflare - Exhibit 1025, page 6
`
`
`
`US 2003/0118029 Al
`
`1
`
`Jun. 26, 2003
`
`METHOD AND APPARATUS FOR ENFORCING
`SERVICE LEVEL AGREEMENTS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`[0001] This application is a continuation of application
`Ser. No. 09/653,521 which was filed on Aug. 31, 2000.
`
`TECHNICAL FIELD OF THE INVENTION
`
`[0002] The present invention relates to broadband data
`networking equipment. Specifically, the present invention
`relates to a method and network device that is able to
`classify network traffic based on type and application and to
`shape and manage network traffic in order to enforce Service
`Level Agreements.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Almost everyone is using Internet and web-based
`services as a primary means of conducting business. Ser-
`vices such as email, e-commerce, Voice over IP (VoIP), and
`web-browsing have become critical to communication
`within and across organizations. As reliance on network
`based services increase, so do consumer demands for avail-
`ability reliability, and responsiveness of the services. Typi-
`cally, the customers do not care how the service is com-
`posed, to them the quality of service (QoS) is what is
`important. These quality of service expectations are driving
`customers to negotiate guarantees with their service provid-
`ers that will meet customer service requirements for specific
`QoS levels. In order to offer end-to-end QoS guarantees to
`customers, more and more providers and customers are
`entering into Service Level Agreements (SLAs).
`
`[0004] An SLA is a contract between a provider and a
`customer that guarantees specific levels of performance and
`reliability for a certain cost. Traditionally, SLAs have
`included performance guarantees such as response time and
`network availability, in addition to specifying customer
`support and help desk issues. One major problem with
`SLAs, however, is that they are limited to collecting statis-
`tical information on network performance and availability
`since the current state of the art does not allow manipulation
`of the network itself or the data flowing over the network at
`wire speed. Because SLAs are enforced after the fact based
`on statistical information, the only recourse to both provider
`and customer is an adjustment to payments or credits applied
`for future services.
`
`[0005] Technology that would allow real time monitoring
`and dynamic allocation of network resources would allow
`providers and customers to take SLAs and service level
`management (SLM) to the next level. Such a technology
`would identify network resources that were reaching their
`maximum performance and allow the network to dynami-
`cally allocate additional resources, which could be metered
`and billed to the customer. Additionally, the customers
`would not be limited to resources in increments of carrier
`size, such as D3s, Tls or T3s, but instead would be able to
`specify their exact requirement and pay for exactly the
`resources consumed.
`
`[0006] Further, new technology could be incorporated to
`include security features such as prevention of denial of
`service and monitoring for email viruses and worms. This
`
`would allow the provider to differentiate his services from
`other providers and would provide content that could be
`charged for by the provider. The customer would benefit by
`increased availability of their resources as well as being able
`to offload the expense of installing and maintaining security
`equipment to the provider.
`
`[0007] Accordingly, what is needed is a network device
`that can enforce service level agreements by being able to
`recognize network traffic at wire speeds and by dynamically
`modifying the traffic or the network to accommodate per-
`formance and resource policies agreed to between the pro-
`vider and customer. Further, the network device is able to
`provide security for the network that is maintained by the
`provider as a service to the customer.
`
`SUMMARY OF THE INVENTION
`
`[0008] The present invention provides for a network
`device or apparatus that is able to enforce service level
`agreements between providers and customers. The network
`device includes memory, which contains information spe-
`cific to each customer, or subscriber. The memory also
`includes policies defined to enforce the terms of the service
`level agreements such as resource allocation and particular
`service levels, as well as statistics that are kept for each
`subscriber allowing the provider to provide metering and
`billing, as well as to allow the subscriber to keep detailed
`information on the subscribers network usage. The memory
`is connected to a traffic flow scanning processor which is
`operable to scan both the header and payload of all data
`packets flowing through the network device. The traffic flow
`scanning processor scans each packet to associate it with a
`particular subscriber and to identify the type and nature of
`the network traffic. Once the subscriber and type of traffic
`have been identified, the policies for that subscriber can be
`enforced and events or statistics can be logged. This is
`accomplished by the traffic flow scanning processor deter-
`mining a treatment for each data packet based on the
`scanning and preprogrammed policies. This treatment and
`the data packet itself are forwarded to a quality of service
`processor connected to the traffic flow scanning processor.
`The quality of service processor modifies the data packet, if
`necessary, and assigns it to a quality of service queue based
`on the treatment.
`
`[0009] Further, the present invention sets for a method for
`enforcing resource allocation defined by a service level
`agreement. The method associates each data packet with a
`subscriber, or customer, and classifies the data packet by
`traffic type, each traffic type being further classified as either
`real time or non-real time. Once the packet is classified and
`associated with a subscriber, the method checks for available
`bandwidth according to the preprogrammed policies for that
`subscriber. The data packet is then sent to the appropriate
`quality of service queue for transmission back onto the
`network.
`
`[0010] The foregoing has outlined, rather broadly, pre-
`ferred and alternative features of the present invention so
`that those skilled in the art may better understand the
`detailed description of the invention that follows. Additional
`features of the invention will be described hereinafter that
`form the subject of the claims of the invention. Those skilled
`in the art will appreciate that they can readily use the
`disclosed conception and specific embodiment as a basis for
`
`Cloudflare - Exhibit 1025, page 7
`
`Cloudflare - Exhibit 1025, page 7
`
`
`
`US 2003/0118029 Al
`
`Jun. 26, 2003
`
`2
`
`designing or modifying other structures for carrying out the
`same purposes of the present invention. Those skilled in the
`art will also realize that such equivalent constructions do not
`depart from the spirit and scope of the invention in its
`broadest form.
`
`18 of hosting network 14 are usually made at speeds such as
`OC-3 or OC-12 (approx. 150 megabits/sec or 625 megabits/
`sec), while the connection from router 22 of hosting network
`14 to private IP network 12 are on the order OC-48 speeds
`(approx. 2.5 gigabits/sec).
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0011] For a more complete understanding of the present
`invention, reference is now made to the following descrip-
`tions taken in conjunction with the accompanying drawings,
`in which:
`
`[0012] FIG. 1 is a network topology diagram illustrating
`example environments in which the present invention can
`operate;
`
`[0013] FIG. 2 is a block diagram of a "bump-in-the-line"
`network apparatus according to the present invention;
`
`[0014] FIG. 3 is a block diagram of the payload scanning
`engine from FIG. 2; and
`
`[0015] FIG. 4 is a block diagram of a routing network
`apparatus according to the present invention; and
`
`[0016] FIG. 5 is a flow chart illustrating a method accord-
`ing to the present invention for enforcing resource allocation
`according to a Service Level Agreement.
`
`DETAILED DESCRIPTION OF THE DRAWINGS
`
`[0017] Referring now to FIG. 1, a network topology is
`shown which is an example of several network infrastruc-
`tures that connect in some manner to a broader public IP
`network 10 such as the internet. FIG. 1 is in no way meant
`to be a precise network architecture, but only to serve as a
`rough illustration of a variety of network structures which
`can exist on a broadband IP network. Public IP network 10
`can be accessed in a variety of ways. FIG. 1 shows the
`public IP network being accessed through a private IP
`network 12 which can be the IP network of a company such
`as MCI or UUNET which provide private core networks. An
`endless variety of network structures can be connected to
`private IP network 12 in order to access other networks
`connected to private IP network 12 or to access public IP
`network 10.
`
`[0018] One example of a network structure connecting to
`private IP network 12 is hosting network 14. Hosting net-
`work 14 is an example of a network structure that provides
`hosting services for internet websites. These hosting ser-
`vices can be in the form of webfarm 16. Webfarm 16 begins
`with webservers 30 and database 32 which contain the
`webpages, programs and databases associated with a par-
`ticular website such as amazon.com or yahoo.com. Web-
`servers 30 connect to redundant load balancers 28 which
`receive incoming internet traffic and assign it to a particular
`webserver to balance the loads across all of webservers 30.
`Redundant intrusion detection systems 26 and firewalls
`connect to load balancers 28 and provide security for
`webfarm 16. Individual webfarms 16 and 17 connect to
`hosting network 14's switched backbone 18 by means of a
`network of switches 20 and routers 22. Hosting network 14's
`switched backbone 18 is itself made up of a network of
`switches 20 which then connect to one or more routers 22 to
`connect to private IP network 12. Connections between
`individual webfarms 16 and 17 and the switched backbone
`
`[0019] Another example of network structures connecting
`to private IP networks are illustrated with service provider
`network 34. Service provider network 34 is an example of
`a network structure for Internet Service Providers (ISPs) or
`Local Exchange Carriers (LECs) to provide both data and
`voice access to private IP network 12 and public IP network
`10. Service provider network 34 provides services such as
`internet and intranet access for enterprise networks 36 and
`37. Enterprise networks 36 and 37 are, for example, com-
`pany networks such as the company network for Lucent
`Technologies or Merrill Lynch. Each enterprise network,
`such as enterprise network 36, includes a plurality of net-
`work servers and individual workstations connected to a
`switched backbone 18, which can be connected by routers
`22 to service provider network 34.
`
`In addition to internet access for enterprise net-
`[0020]
`works, service provider network 34 provides dial-up internet
`access for individuals or small businesses. Dial-up access is
`provided in service provider network 34 by remote access
`server (RAS) 42, which allows personal computers (PCs) to
`call into service provider network 34 through the public
`switched telephone network (PSTN), not shown. Once a
`connection has been made between the PC 50 and RAS 42
`through the PSTN, PC 50 can then access the private or
`public IP networks 12 and 10.
`
`[0021] Service provider network 34 also provides the
`ability to use the internet to provide voice calls over a data
`network referred to as Voice over IP (VoIP). VoIP networks
`46 and 47 allow IP phones 48 and PCs 50 equipped with the
`proper software to make telephone calls to other phones, or
`PCs connected to the internet or even to regular phones
`connected to the PSTN. VoIP networks, such as VoIP net-
`work 46, include media gateways 52 and other equipment,
`not shown, to collect and concentrate the VoIP calls which
`are sent through service provider network 34 and private and
`public internet 12 and 10 as required. As mentioned, the
`advent of VoIP as well as other real time services such as
`video over the internet make quality of service a priority for
`service providers in order to match the traditional telephone
`service provided by traditional telephone companies.
`
`[0022] Service providers often enter into service level
`agreements with their customers. These service level agree-
`ments set out service and availability requirements, which
`are then monitored and statistics collected. These statistics
`are used to determine whether the service provider met,
`failed to meet, or exceeded the service levels set out in the
`service level agreement. The service provider can then be
`subject to either monetary penalties or rewards for the level
`of service provided.
`
`[0023] Service provider network 34 includes a switched
`backbone 18 formed by switches 20 as well as routers 22
`between it and its end users and between it and private IP
`network 12. Domain name servers 44 and other networking
`equipment, which are not shown, are also included in service
`provider network 34. Similar to hosting network 34, con-
`nection speeds for service provider network 34 can range
`from speeds such as Ti, T3, OC-3 and OC-12 for connecting
`
`Cloudflare - Exhibit 1025, page 8
`
`Cloudflare - Exhibit 1025, page 8
`
`
`
`US 2003/0118029 Al
`
`Jun. 26, 2003
`
`3
`
`to enterprise networks 36 and 37 as well as VoIP networks
`46 and 47 all the way to OC-48 and conceivably even
`OC-192 for connections to the private IP network.
`
`It can easily be seen that aggregation points 60
`[0024]
`exist at the edges of these various network structures where
`data is passed from one network structure to another at
`speeds such as OC-3, OC-12, and OC-48. One major prob-
`lem in the network structures shown in FIG. 1 is the lack of
`any type of intelligence at these aggregation points 60 which
`would allow the network to provide services such as secu-
`rity, metering and quality of service. The intelligence to
`provide these services would require that the network under-
`stand the type of data passing through the aggregation points
`60 and not just the destination and/or source information
`which is currently all that is understood. Understanding the
`type of data, or its contents, including the contents of the
`associated payloads as well as header information, and
`further understanding and maintaining a state awareness
`across each individual traffic flow would allow the network
`to configure itself in real time to bandwidth requirements on
`the network for applications such as VoIP or video where
`quality of service is a fundamental requirement. An intelli-
`gent, or "content aware", network would also be able to
`identify and filter out security problems such as email
`worms, viruses, denial of service (DoS) attacks, and illegal
`hacking in a manner that would be transparent to end users.
`Further, a content aware network would provide for meter-
`ing capabilities by hosting companies and service providers,
`allowing these companies to regulate the amount of band-
`width allotted to individual customers as well as to charge
`precisely for bandwidth and additional features such as
`security.
`
`In accordance with the requirements set forth
`[0025]
`above, the present invention provides for a network device
`that is able to scan, classify, and modify network traffic
`including payload information at speeds of OC-3, OC-12,
`OC-48 and greater thereby providing a "content aware"
`network.
`
`[0026] Referring now to FIG. 2, one embodiment of a
`network apparatus according to the present invention is
`shown. Network apparatus 100, as shown, acts as a "bump-
`in the-line" type device by accepting data received from a
`high-speed network line, processing the data, and then
`placing the data back on the line. Network apparatus 100
`accepts data from the line by means of input physical
`interface 102. Input physical interface 102 can consist of a
`plurality of ports, and can accept any number of network
`speeds and protocols, including such high speeds as OC-3,
`OC-12, OC-48, and protocols including 10/100 Ethernet,
`gigabit Ethernet, and SONET. Input physical interface 102
`takes the data from the physical ports, frames the data, and
`then formats the data for placement on fast-path data bus 126
`which is preferably an industry standard data bus such as a
`POS-PHY Level 3, or an ATM UTOPIA Level 3 type data
`bus.
`
`[0027] Fast-path data bus 126 feeds the data to traffic flow
`scanning processor 140, which includes header processor
`104 and payload analyzer 110. The data is first sent to header
`processor 104, which is operable to perform several opera-
`tions using information contained in the data packet headers.
`Header processor 104 stores the received data packets in
`packet storage memory 106 and scans the header informa-
`
`tion. The header information is scanned to identify the type,
`or protocol, of the data packet, which is used to determine
`routing information as well as to create a session id using
`predetermined attributes of the data packet.
`
`In the preferred embodiment, a session id is created
`[0028]
`using session information consisting of the source address,
`destination address, source port, destination port and proto-
`col, although one skilled in the art would understand that a
`session id could be created using any subset of fields listed
`or any additional fields in the data packet without departing
`from the scope of the present invention. In addition, the
`header information is used to identify the data packet with
`a particular customer or subscriber. When a data packet is
`received that has new session information the header pro-
`cessor creates a unique session id to identify that particular
`traffic flow. Each successive data packet with the same
`session information is assigned the same session id to
`identify each packet within that flow. Session ids are retired
`when the particular traffic flow is ended through an explicit
`action, or when the traffic flow times out, meaning that a data
`packet for that traffic flow has not been received within a
`predetermined amount of time. While the session id is
`discussed herein as being created by the header processor
`104 the session id can be created anywhere in traffic flow
`scanning engine 140 including in payload analyzer 110.
`
`[0029] As will be discussed below, network apparatus 100
`in order to function properly needs to reorder out of order
`data packets and reassemble data packet fragments. Header
`processor 104 is operable to perform the assembly of
`asynchronous transfer mode (ATM) cells into complete data
`packets (PDUs), which could include the stripping of ATM
`header information.
`
`[0030] Header processor 104 is also operable to perform
`routing functions. Routing tables and information can be
`stored in database memory 108. Routing instructions
`received by network apparatus 100 are identified, recorded
`and passed to microprocessor 124 by header processor 104
`so that microprocessor 124 is able to update the routing
`tables in database memory 108 accordingly. While network
`apparatus 100 is referred to as a "bump-in-the-line" appa-
`ratus, The input and the output could be formed by multiple
`lines, for example four OC-12 lines could be connected to
`network apparatus 100 which operates at OC-48 speeds. In
`such a case, "bump-in-the-line" network apparatus 100 will
`have limited routing or switching capabilities between the
`multiple lines, although the switching capability will be less
`than in a conventional router or switch. Additionally, a
`network apparatus can be constructed according to the
`principles of the present invention, which is able to operate
`as a network router or switch. Such an implementation is
`discussed in greater detail with reference to FIG. 4.
`
`[0031] After data packets have been processed by header
`processor 104 the data packets, their associated session id
`and any conclusion formed by the header processor, such as
`routing or QoS information, are sent on fast-data path 126 to
`the other half of traffic flow scanning engine 140, payload
`analyzer 110. The received packets are stored in packet
`storage memory 112 while they are processed by payload
`analyzer 110. Payload analyzer 110 is operable to scan the
`contents of data packets received from header processor 104,
`particularly the payload contents of the data packets,
`although header information can also be scanned as
`
`Cloudflare - Exhibit 1025, page 9
`
`Cloudflare - Exhibit 1025, page 9
`
`
`
`US 2003/0118029 Al
`
`Jun. 26, 2003
`
`4
`
`required. The contents of any or all data packets are com-
`pared to a database of known signatures and if the contents
`of a data packet or packets match a known signature, an
`action associated with that signature and/or session id can be
`taken by network apparatus 100. Additionally, payload ana-
`lyzer 110 is operable to maintain state awareness throughout
`each individual traffic flow. In other words, payload analyzer
`110 maintains a database for each session which stores state
`information related to not only the current data packets from
`a traffic flow, but state information related to the entirety of
`the traffic flow. This allows network apparatus 100 to act on
`not only based on the content of the data packets being
`scanned but also based on the contents of the entire traffic
`flow.
`
`[0032] Payload analyzer 110 is also used to store sub-
`scriber information, policies, events and statistics used in the
`enforcement of service level agreements. The policies,
`which can be customized to an individual subscriber or a
`group of subscribers, can set out service parameters such as
`available bandwidth for the subscriber, available bandwidth
`for certain types of network traffic, real time and non-real
`time, within the subscribers total bandwidth, security level
`for the subscriber, etc. Events allow the tracking of content-
`based occurrences outside the scope of currently kept sta-
`tistics. Event tracking allows a more detailed profile of each
`subscriber to be learned and maintained. The statistics allow
`metering by the provider which can be used, for example, to
`charge the customer for additional services and bandwidth
`provided to the subscriber as well as to keep detailed track
`of the subscribers network activity. The specific operation of
`payload analyzer 110 will be described with reference to
`FIG. 3.
`
`[0033] Once the contents of the packets have been scanned
`and a conclusion, or treatment, is reached by traffic flow
`scanning engine 140, the packets and the associated con-
`clusions of either or both the header processor and the
`payload analyzer are sent to quality of service (QoS) pro-
`cessor 116. QoS processor 116 again stores the packets in its
`own packet storage memory 118 for forwarding. QoS pro-
`cessor 116 is operable to perform the traffic flow manage-
`ment for the stream of data packets processed by network
`apparatus 100. QoS processor contains engines for traffic
`management 126, traffic shaping 128 and packet modifica-
`tion 130.
`
`[0034] QoS processor 116 takes the conclusion, or treat-
`ment, of either or both of header processor 104 and payload
`analyzer 110 and assigns the data packet to one of its internal
`quality of service queues 132 based on the conclusion. The
`quality of service queues 132 can be assigned priority
`relative to one another or can be assigned a maximum or
`minimum percentage of the traffic flow through the device.
`This allows QoS processor to assign the necessary band-
`width to traffic flows such as VoIP, video and other flows
`with high quality and reliability requirements while assign-
`ing remaining bandwidth to traffic flows with low quality
`requirements such as email and general web surfing to low
`priority queues. Information in queues that do not have the
`available bandwidth to transmit all the data currently resid-
`ing in the queue according to the QoS engine is selectively
`discarded thereby removing that data from the traffic flow.
`
`[0035] The quality of service queues 132 also allow net-
`work apparatus 100 to manage network attacks such as
`
`denial of service (DoS) attacks. Network apparatus 100 can
`act to qualify traffic flows by scanning the contents of the
`packets and verifying that the contents contain valid network
`traffic between known sources and destinations. Traffic flows
`that have not been verified because they are from unknown
`sources or because they are new unclassified flows can be
`assigned to a low quality of service queue until the sources
`are verified or the traffic flow classified as valid traffic. Since
`most DoS attacks send either new session information, data
`from spoofed sources, or meaningless data, network appa-
`ratus 100 would assign those traffic flows to low quality
`traffic queues. This ensures that the DoS traffic would
`receive no more than a small percentage (i.e. 5%) of the
`available bandwidth thereby preventing the attacker from
`flooding downstream network equipment.
`
`[0036] The QoS queues 132 in QoS processor 116 (there
`are 65k queues in the present embodiment of the QoS
`processor although any number of queues could be used)
`have multiple associated class of service (CoS) queues
`which feed into schedulers 134 (1024 in the present embodi-
`ment), which feed into logic ports 136 (256 in the present
`embodiment), which send the data to flow control port
`managers 138 (32 is the present embodiment) which can
`correspond to physical egress ports for the network device.
`The traffic management engine 126 and the traffic shaping
`engine 128 determine the operation of the schedulers and
`logic ports in order to maintain traffic flow in accorda