`
`(12) United States Patent
`US 7,296,288 B1
`(10) Patent No.:
`Hill et al.
`(45) Date of Patent:
`Nov. 13, 2007
`
`(54)
`
`(75)
`
`METHODS, APPARATUSES, AND SYSTEMS
`ALLOWING FOR BANDWIDTH
`MANAGEMENT SCHEMES RESPONSIVE TO
`UTILIZATION CHARACTERISTICS
`ASSOCIATED WITH INDIVIDUAL USERS
`
`................ 709/227
`2/2004 Bruck et al.
`6,691,165 B1*
`8/2005 Krautkremer ............ 709/223
`6,934,745 B2 *
`
`1/2003 Burnett et al. ........... 713/153
`2003/0018889 A1*
`
`................. 370/468
`2003/0235209 A1* 12/2003 Garg et al.
`
`* cited by examiner
`
`Inventors: Mark Hill, Los Gatos, CA (US); Guy
`Riddle, Los Gatos, CA (US); Robert E.
`Purvy, San Jose, CA (US)
`
`Primary ExamineriKim Vu
`Assistant Examineriloseph Pan
`(74) Attorney, Agent, or FirmiMark J. Spolyar
`
`(73)
`
`Assignee: Packeteer, Inc., Cupertino, CA (US)
`
`(57)
`
`ABSTRACT
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 788 days.
`
`(21)
`
`Appl. No.: 10/295,391
`
`(22)
`
`Filed:
`
`Nov. 15, 2002
`
`(51)
`
`(52)
`(58)
`
`(56)
`
`Int. Cl.
`
`(2006.01)
`G06F 21/00
`US. Cl.
`........................................... 726/2; 713/194
`Field of Classification Search .................... 713/ 1,
`713/2, 188, 194, 193; 380/200, 201, 255,
`380/277; 726/2, 3, 11715
`See application file for complete search history.
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`1/2002 Morris et al.
`6,339,784 B1*
`6,484,203 B1* 11/2002 Porras et al.
`
`............... 709/204
`............... 709/224
`
`Methods, apparatuses and systems allowing for bandwidth
`management schemes responsive to utilization characteris-
`tics associated with individual users. In one embodiment, the
`present invention allows network administrators to penalize
`users who carry out specific questionable or suspicious
`activities, such as the use of proxy tunnels to disguise the
`true nature of the data flows in order to evade classification
`
`and control by bandwidth management devices. In one
`embodiment, each individual user may be accorded an initial
`suspicion score. Each time the user is associated with a
`questionable or suspicious activity (for example, detecting
`the set up of a connection to an outside HTTP tunnel, or
`peer-to-peer application flow), his or her suspicion score is
`downgraded. Data flows corresponding to users with suffi-
`ciently low suspicion scores, in one embodiment, can be
`treated in a different manner from data flows associated with
`
`other users. For example, different or more rigorous classi-
`fication rules and policies can be applied to the data flows
`associated with suspicious users.
`
`34 Claims, 7 Drawing Sheets
`
`
`
`Computer
`Network
`
`(Outside)
`
` ‘
`(inside)
`
`
`
`
`42
`
`42
`
`EX1026
`
`Palo Alto Networks v. Sable Networks
`
`IPR2020-01712
`
`EX1026
`Palo Alto Networks v. Sable Networks
`IPR2020-01712
`
`
`
`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 1 of 7
`
`US 7,296,288 B1
`
`Network
`
`50 Computer
`
`
`
`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 2 of 7
`
`US 7,296,288 B1
`
`
`
`Administrator
`
`150
`
`
`Interface
`
`1 3 7
`
`Classification
`
`Engine
`
`138
`
`Suspicion
`
`Scoring Module
`
`
`Database
`
`Flow
`
`Database
`
`140
`
`
`
`
`
`Data Packet
`
`Out
`
`
`
`
`Traffic I. Measurement
`
`
`
`
`”1
`
`H ost
`
`Database
`
`
`
`
`
`Flow Control
`
`Module
`
`1 3 2
`
`Data Packet
`In
`
`P acket
`
`Processor
`
`
`Fig._2
`
`
`
`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 3 of 7
`
`US 7,296,288 B1
`
`Receive Data
`
`Packet
`
`New Data
`
`Flow?
`
`Control
`
`Block?
`
`
`Fetch/Update
`Control Block
`
`
`Control Block
` Construct
`
`
`
`Traffic Class
`
`
`
`Identify
`
`P = getControls
`(Traffic Class)
`
`
`Pass Packet to
`Flow Control
`
`Module (P)
`
`
`
`Record Bandwidth
`Utilization Data In
`
`Association with
`Traffic Class
`
`
`
`
`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 4 of 7
`
`US 7,296,288 B1
`
`AccessLink
`
`
`
`
`
`
`
`
`Inbound
`
`fifiE LocalHost
`
`
`
`
`SuspiciousUsers
`
`«fiE [PAddrl
`
`fl lPAder
`EfiEIPAdera
`
`i:
`
`
`
`
`
`
`
`
`
`
`Outbound
`
`,
`
`'65" LocalI-Iost
`SuspiciousUsers
`
`
`
`:fi IPAddrl
`
`a! IPAddr2
`
`l@(IPAddfi
`
`
`iefifl Default
`
`2% Telnet
`
`Fig._4B
`
`ffil LocalHost
`@E SuspiciousUsers
`
`AccessLink
`
`
`
`
`
`
`éfil Default
`
`
`
`Outbound
`
`flfil LocalHost
`
`@l SuspiciousUsers
`
`
`
`:fiE Default
`
`Fig._4A
`
`
`
`US. Patent
`
`Nov. 13, 2007
`
`Sheet 5 of 7
`
`US 7,296,288 B1
`
`302
`
`
`
`
` Instantiate
`Suspicion Scoring
`Object
`
`
`
`
`
`Suspicion Scoring
`
`
`Object
`
`Pickled
`
`Object?
`
`Un-Pickle
`
`
`
`
`
`
`Suspicion Scoring
`Object
`
`Pass Packet to
`
`Fig._5
`
`
`
`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 6 of 7
`
`US 7,296,288 B1
`
`Client Device
`
`
`
` Computer
`
`Network
`
`Tunnel Proxy
`Server
`
`
` Network
`
`Computer
`Network
`
`
`Resource
`
`Fig._6
`
`
`
`US. Patent
`
`Nov. 13,2007
`
`Sheet 7 017
`
`US 7,296,288 B1
`
`“*New Flows Per Minute
`
`*i-t
`
`IP Address
`216.203.49.219
`216.148.237.158
`
`218.148.237.145
`107.158 """"
`10.255.255.255
`10.7154
`10.11.46
`207.46.249.61
`
`10.11.16
`25525512552555
`10711.2
`107.1513
`66.218.71.88"
`10.21.10
`239.255.255.253
`*10.7.15.5"'
`‘
`‘10. 1 . 1.18
`
`110.10.254.74
`101025370
`10.7.3122
`
`Conn
`0
`1
`
`RTT to PS Curr Rate 1 Min Avg ‘Peak Rate
`"80m"
`"2730 " "42o """5
`2730
`14ms
`235k
`1
`19.1k
`235k
`
`Client
`0
`0
`
`Server
`90
`84
`
`"
`
`Failed
`0'"
`0
`
`5
`11
`2
`0
`o """
`0
`
`48ms
`31113
`
`'
`
`***'
`25ms
`
`4871
`2303
`"3108'" 58.9k
`3397
`3464
`654
`190
`' "643'
`' 188
`15.2k
`20.5k
`
`’
`
`,,
`
`'
`
`1
`1
`0
`2
`' 2
`0
`2
`0
`1
`
`0
`0
`0
`
`124111;"
`
`M
`
`.
`
`:
`
`‘
`
`‘
`
`3657
`1735
`o
`549
`492
`0
`39
`485
`1349
`
`0
`0
`0
`
`2905
`430
`0
`252
`4217
`0
`25
`318
`771
`
`54
`0
`0
`
`'
`
`'
`
`‘
`
`i
`'
`,
`
`49k
`310k
`17.6k
`1112
`1112
`220k
`
`17.7k
`5357
`11.1k
`11.3k
`90.6k
`343
`1305
`8787
`2091
`
`345
`'37
`2
`
`:
`
`70 7
`42
`0
`11
`6""
`0
`
`"
`
`64
`"0
`28
`6
`'11
`14
`
`0
`0"
`3
`0
`0
`1
`0
`"1
`1
`
`1
`0
`0
`
`'
`
`11
`4'
`0
`3
`2
`0
`1
`"'0
`0
`
`o
`0
`0
`
`0
`0
`0
`0
`'0 '
`0
`
`0
`'0
`0
`0
`0 '
`0
`0
`"'0
`0
`
`0
`0’"
`0
`
`Table 7
`
`
`
`US 7,296,288 B1
`
`1
`METHODS, APPARATUSES, AND SYSTEMS
`ALLOWING FOR BANDWIDTH
`MANAGEMENT SCHEMES RESPONSIVE TO
`UTILIZATION CHARACTERISTICS
`ASSOCIATED WITH INDIVIDUAL USERS
`
`COPYRIGHT NOTICE
`
`A portion of the disclosure of this patent document
`contains material which is subject to copyright protection.
`The copyright owner has no objection to the facsimile
`reproduction by anyone of the patent document or the patent
`disclosure as it appears in the Patent and Trademark Office
`patent file or records, but otherwise reserves all copyright
`rights whatsoever.
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application makes reference to the following com-
`monly owned US. patent applications and patents, which
`are incorporated herein by reference in their entirety for all
`purposes:
`US. patent application Ser. No. 08/762,828 now US. Pat.
`No. 5,802,106 in the name of Robert L. Packer, entitled
`“Method for Rapid Data Rate Detection in a Packet Com-
`munication Environment Without Data Rate Supervision,”
`US. patent application Ser. No. 08/970,693 now US. Pat.
`No. 6,018,516, in the name of Robert L. Packer, entitled
`“Method for Minimizing Unneeded Retransmission of Pack-
`ets in a Packet Communication Environment Supporting a
`Plurality of Data Link Rates,”
`US. patent application Ser. No. 08/742,994 now US. Pat.
`No. 6,038,216, in the name of Robert L. Packer, entitled
`“Method for Explicit Data Rate Control in a Packet Com-
`munication Environment without Data Rate Supervision,”
`US. patent application Ser. No. 09/977,642 now US. Pat.
`No. 6,046,980, in the name of Robert L. Packer, entitled
`“System for Managing Flow Bandwidth Utilization at Net-
`work, Transport and Application Layers in Store and For-
`ward Network,”
`U.S. patent application Ser. No. 09/106,924 now US. Pat.
`No. 6,115,357, in the name of Robert L. Packer and Brett D.
`Galloway, entitled “Method for Pacing Data Flow in a
`Packet-based Networ ,”
`US. patent application Ser. No. 09/046,776 now US. Pat.
`No. 6,205,120, in the name of Robert L. Packer and Guy
`Riddle, entitled “Method for Transparently Determining and
`Setting an Optimal Minimum Required TCP Window Size,”
`US. patent application Ser. No. 09/479,356 now US. Pat.
`No. 6,285,658, in the name of Robert L. Packer, entitled
`“System for Managing Flow Bandwidth Utilization at Net-
`work, Transport and Application Layers in Store and For-
`ward Network,”
`U.S. patent application Ser. No. 09/198,090 now US. Pat.
`No. 6,412,000, in the name of Guy Riddle and Robert L.
`Packer, entitled “Method for Automatically Classifying
`Traffic in a Packet Communications Networ ,”
`US. patent application Ser. No. 09/198,051, in the name
`of Guy Riddle, entitled “Method for Automatically Deter-
`mining a Traffic Policy in a Packet Communications Net-
`work,”
`U.S. patent application Ser. No. 09/206,772, in the name
`of Robert L. Packer, Brett D. Galloway and Ted Thi, entitled
`“Method for Data Rate Control for Heterogeneous or Peer
`Internetworking,”
`
`2
`
`US. patent application Ser. No. 09/885,750, in the name
`of Scott Hankins and Brett Galloway, entitled “System and
`Method For Dynamically Controlling a Rogue Application
`Through Incremental Bandwidth Restrictions,”
`US. patent application Ser. No. 09/966,538, in the name
`of Guy Riddle, entitled “Dynamic Partitioning of Network
`Resources,”
`in the
`US. patent application Ser. No. 10/039,992,
`Michael J. Quinn and Mary L. Laier, entitled “Method and
`Apparatus for Fast Lookup of Related Classification Entities
`in a Tree-Ordered Classification Hierarchy,”
`US. patent application Ser. No. 10/015,826, in the name
`of Guy Riddle, entitled “Dynamic Tunnel Probing in a
`Communications Network,”
`Us. patent application Ser. No. 10/108,085, in the name
`of Wei-Lung Lai, Jon Eric Okholm, and Michael J. Quinn,
`entitled “Output Scheduling Data Structure Facilitating
`Hierarchical Network Resource Allocation Scheme,”
`US. patent application Ser. No. 10/155,936, in the name
`of Guy Riddle, Robert L. Packer and Mark Hill, entitled
`“Method for Automatically Classifying Traffic with
`Enhanced Hierarchy in a Packet Communications Net-
`work,”
`U.S. patent application Ser. No. 10/177,518, in the name
`of Guy Riddle, entitled “Methods, Apparatuses and Systems
`Allowing for Progressive Network Resource Utilization
`Control Scheme,” and
`US. patent application Ser. No. 10/178,617, in the name
`of Robert E. Purvy, entitled “Methods, Apparatuses and
`Systems Facilitating Analysis of Network Device Perfor-
`mance.”
`
`FIELD OF THE INVENTION
`
`The present invention relates to computer networks and
`bandwidth management, and, more particularly, to methods,
`apparatuses and systems allowing for bandwidth manage-
`ment schemes responsive to the utilization characteristics
`associated with individual users.
`
`BACKGROUND OF THE INVENTION
`
`In order to understand the context of certain embodiments
`
`of the invention, the following provides an explanation of
`certain technical aspects of a packet based telecommunica-
`tions network environment. Intemet/Intranet technology is
`based largely on the TCP/IP protocol suite. At the network
`level, IP provides a “datagram” delivery serviceithat is, IP
`is a protocol allowing for delivery of a datagram or packet
`between two hosts. By contrast, TCP provides a transport
`level service on top of the datagram service allowing for
`guaranteed delivery of a byte stream between two IP hosts.
`In other words, TCP is responsible for ensuring at the
`transmitting host that message data is divided into packets to
`be sent, and for reassembling, at the receiving host, the
`packets back into the complete message.
`TCP has “flow control” mechanisms operative at the end
`stations only to limit the rate at which a TCP endpoint will
`emit data, but it does not employ explicit data rate control.
`The basic flow control mechanism is a “sliding window”, a
`window which by its sliding operation essentially limits the
`amount of unacknowledged transmit data that a transmitter
`is allowed to emit. Another flow control mechanism is a
`
`congestion window, which is a refinement of the sliding
`window scheme involving a conservative expansion to make
`use of the full, allowable window.
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`
`
`US 7,296,288 B1
`
`3
`The sliding window flow control mechanism works in
`conjunction with the Retransmit Timeout Mechanism
`(RTO), which is a timeout to prompt a retransmission of
`unacknowledged data. The timeout length is based on a
`running average of the Round Trip Time (RTT) for acknowl-
`edgment receipt, i.e. if an acknowledgment is not received
`within (typically) the smoothed RTT+4*mean deviation,
`then packet loss is inferred and the data pending acknowl-
`edgment is re-transmitted. Data rate flow control mecha-
`nisms which are operative end-to-end without explicit data
`rate control draw a strong inference of congestion from
`packet loss (inferred, typically, by RTO). TCP end systems,
`for example, will “back-olf,”ii.e., inhibit transmission in
`increasing multiples of the base RTT average as a reaction
`to consecutive packet loss.
`A crude form of bandwidth management in TCP/IP net-
`works (that is, policies operable to allocate available band-
`width from a single logical link to network flows) is accom-
`plished by a combination of TCP end systems and routers
`which queue packets and discard packets when some con-
`gestion threshold is exceeded. The discarded and therefore
`unacknowledged packet serves as a feedback mechanism to
`the TCP transmitter. Routers
`support various queuing
`options to provide for some level of bandwidth manage-
`ment. These options generally provide a rough ability to
`partition and prioritize separate classes of traffic. However,
`configuring these queuing options with any precision or
`without side effects is in fact very difficult, and in some
`cases, not possible. Seemingly simple things, such as the
`length of the queue, have a profound effect on traffic
`characteristics. Discarding packets as a feedback mechanism
`to TCP end systems may cause large, uneven delays per-
`ceptible to interactive users. Moreover, while routers can
`slow down inbound network traffic by dropping packets as
`a feedback mechanism to a TCP transmitter, this method
`often results in retransmission of data packets, wasting
`network traffic and, especially, inbound capacity of a WAN
`link. In addition, routers can only explicitly control out-
`bound traffic and cannot prevent inbound traffic from over-
`utilizing a WAN link. A 5% load or less on outbound traffic
`can correspond to a 100% load on inbound traffic, due to the
`typical imbalance between an outbound stream of acknowl-
`edgments and an inbound stream of data.
`In response, certain data flow rate control mechanisms
`have been developed to provide a means to control and
`optimize efficiency of data transfer as well as allocate
`available bandwidth among a variety of business enterprise
`functionalities. For example, U.S. Pat. No. 6,038,216 dis-
`closes a method for explicit data rate control in a packet-
`based network environment without data rate supervision.
`Data rate control directly moderates the rate of data trans-
`mission from a sending host, resulting in just-in-time data
`transmission to control inbound traffic and reduce the inef-
`
`ficiencies associated with dropped packets. Bandwidth man-
`agement devices allow for explicit data rate control for flows
`associated with a particular
`traffic classification. For
`example, U.S. Pat. No. 6,412,000, above, discloses auto-
`matic classification of network traffic for use in connection
`with bandwidth allocation mechanisms. U.S. Pat. No. 6,046,
`980 discloses systems and methods allowing for application
`layer control of bandwidth utilization in packet-based com-
`puter networks. For example, bandwidth management
`devices allow network administrators to specify policies
`operative to control and/or prioritize the bandwidth allocated
`to individual data flows according to traffic classifications. In
`addition, certain bandwidth management devices, as well as
`certain routers, allow network administrators to specify
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`aggregate bandwidth utilization controls to divide available
`bandwidth into partitions. With some network devices, these
`partitions can be configured to ensure a minimum bandwidth
`and/or cap bandwidth as to a particular class of traffic. An
`administrator specifies a traffic class (such as FTP data, or
`data flows involving a specific user) and the size of the
`reserved virtual linkii.e., minimum guaranteed bandwidth
`and/or maximum bandwidth. Such partitions can be applied
`on a per-application basis (protecting and/or capping band-
`width for all
`traffic associated with an application) or a
`per-user basis (controlling, prioritizing, protecting and/or
`capping bandwidth for a particular user). In addition, certain
`bandwidth management devices allow administrators to
`define a partition hierarchy by configuring one or more
`partitions dividing the access link and further dividing the
`parent partitions into one or more child partitions.
`While the systems and methods discussed above that
`allow for traffic classification and application of bandwidth
`utilization controls on a per-traffic-classification basis oper-
`ate effectively for their intended purposes,
`they possess
`certain limitations. As discussed more fully below, identifi-
`cation of traffic types associated with data flows traversing
`an access link involves the application of matching criteria
`or rules to various characteristics of the data flows. Such
`
`matching criteria can include source and destination IP
`addresses, port numbers, MIME types, etc. After identifica-
`tion of a traffic type corresponding to a data flow, a band-
`width management device associates and subsequently
`applies bandwidth utilization controls (e.g., a policy or
`partition) to the data flow corresponding to the identified
`traffic classification or type. A common use of bandwidth
`management devices is to limit the bandwidth being con-
`sumed by unruly, bandwidth-intensive applications, such as
`peer-to-peer applications (e.g., Kazaa, Napster, etc.). Net-
`work savvy users (such as students in a campus or university
`environment), however, have become aware that such band-
`width management devices have been deployed to limit or
`restrict such unauthorized network traffic. As a result, users
`often attempt to bypass or thwart the bandwidth manage-
`ment scheme effected by such bandwidth management
`devices by creating communications tunnels (proxy tunnels)
`through which unauthorized or restricted network traffic is
`sent. The attributes discernible from the content of these
`tunneled data flows, however, often reveal little information
`about its true nature. For example, commercial HTTP tunnel
`services (such as loopholesoftware.com, Totach.net, and
`http-tunnel.com, etc.) allow users to send all network traffic
`in the form of HTTP traffic through a HTTP tunnel between
`a tunnel client and an HTTP proxy server maintained by the
`tunnel services provider. FIG. 6 illustrates the functionality
`and operation of a typical HTTP proxy tunnel. Client device
`42 includes a client application (such as a peer-to-peer
`application 71) and a tunnel client 72. The client application
`sends data to the tunnel client 72 which tunnels the data over
`
`HTTP to a tunnel proxy server 74. The tunnel proxy server
`74 then forwards the data to the intended destination (here,
`network resource 75), and vice versa. Such HTTP tunnels
`typically feature encryption; accordingly, a bandwidth man-
`agement device 30, encountering the tunneled traffic in this
`form, may not detect the exact nature of the traffic and, in
`fact, classify such data flows as legitimate or regular HTTP
`traffic. Accordingly, these tunneling mechanisms and other
`techniques for evading bandwidth utilization controls imple-
`mented by bandwidth management devices present new
`challenges to network administrators and bandwidth device
`manufacturers desiring to effectively control unauthorized
`or restricted network traffic.
`
`
`
`US 7,296,288 B1
`
`5
`In light of the foregoing, a need in the art exists for
`methods, apparatuses and systems allowing for bandwidth
`management schemes that are responsive to the utilization
`characteristics associated with individual users. A need in
`
`the art further exists for methods, apparatuses and systems
`allowing for detection of questionable or other activities
`designed to evade bandwidth management control schemes
`and, thus, enabling application of more rigorous network
`traffic classification mechanisms and/or disparate bandwidth
`utilization controls. Embodiments of the present invention
`substantially fulfill these needs.
`SUMMARY OF THE INVENTION
`
`The present invention provides methods, apparatuses and
`systems allowing for bandwidth management
`schemes
`responsive to utilization characteristics associated with indi-
`vidual users. In one embodiment,
`the present
`invention
`allows network administrators to penalize users who carry
`out specific questionable or suspicious activities, such as the
`use of proxy tunnels to disguise the true nature of the data
`flows in order to evade classification and control by band-
`width management devices. In one embodiment, each indi-
`vidual user may be accorded an initial suspicion level. Each
`time the user is associated with a questionable or suspicious
`activity (for example, detecting the setup of a connection to
`an outside HTTP tunnel, or peer-to-peer application flow),
`his or her suspicion level is adjusted. Data flows correspond-
`ing to users with sufficiently high suspicion levels, in one
`embodiment, can be treated in a different manner from data
`flows associated with other users. For example, different or
`more rigorous classification rules and bandwidth manage-
`ment policies can be applied to the data flows associated
`with suspicious users. For example, data flows associated
`with suspicious users may be examined more closely in
`order to determine more thoroughly or accurately appropri-
`ate classification rules and/or bandwidth management poli-
`c1es.
`
`DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a functional block diagram illustrating a com-
`puter network environment including a bandwidth manage-
`ment device according to an embodiment of the present
`invention.
`
`FIG. 2 is a functional block diagram setting forth the
`functionality in a bandwidth management device according
`to an embodiment of the present invention.
`FIG. 3 is a flow chart providing a method directed to
`processing data packets to allow for enforcement of band-
`width utilization and other controls on network data flows.
`
`FIG. 4A is a diagram illustrating a traffic classification
`configuration for a given access link according to an
`embodiment of the present invention.
`FIG. 4B is a diagram illustrating a traffic classification
`configuration for a given access link according to another
`embodiment of the present invention.
`FIG. 5 is a flow chart diagram setting forth a method
`directed to the management of suspicion scoring objects
`according to an embodiment of the present invention.
`FIG. 6 is a functional block diagram illustrating a proxy
`tunnel which may be used in attempts to circumvent the
`bandwidth utilization controls implemented by bandwidth
`management devices.
`Table 7 sets forth the data flow metrics, according to an
`embodiment of the present invention, maintained for each
`host associated with data flows traversing a bandwidth
`management device.
`
`6
`DESCRIPTION OF PREFERRED
`
`EMBODIMENT(S)
`
`I. Exemplary Operating Environment
`
`10
`
`15
`
`20
`
`25
`
`FIG. 1 sets forth a packet-based computer network envi-
`ronment including a bandwidth management device 30. As
`FIG. 1 shows, local area computer network 40 interconnects
`several TCP/IP end systems, including client devices 42 and
`server device 44, and provides access to resources operably
`connected to computer network 50 via router 22 and access
`link 21. Access link 21 is a physical and/or logical connec-
`tion between two networks, such as computer network 50
`and local area network 40. Server 28 is a TCP end system
`connected to computer network 50 through router 26 and
`access link 25. Client devices 24 are additional TCP end
`
`systems operably connected to computer network 50 by any
`suitable means, such as through an Internet Services Pro-
`vider (ISP). The computer network environment, including
`computer network 50 is a packet-based communications
`environment, employing TCP/IP protocols, and/or other
`suitable protocols, and has a plurality of interconnected
`digital packet transmission stations or routing nodes. Band-
`width management device 30 is provided between router 22
`and local area computer network 40. Bandwidth manage-
`ment device 30 is operative to classify data flows and,
`depending on the classification, enforce respective band-
`width utilization controls on the data flows to control
`
`30
`
`bandwidth utilization and optimize network application per-
`formance across access link 21.
`
`A. Bandwidth Management Device
`FIG. 2 is a block diagram illustrating functionality,
`according to one embodiment of the present
`invention,
`included in bandwidth management device 30.
`In one
`embodiment, bandwidth management device 30 comprises
`packet processor 131, flow control module 132, measure-
`ment engine 140, traffic classification engine 137, suspicion
`scoring module 138, and administrator interface 150. Packet
`processor 131 is operative to detect new data flows and
`construct data structures including attributes characterizing
`the data flow. Flow control module 132 is operative to
`enforce bandwidth utilization controls on data flows travers-
`
`ing bandwidth management device 30. Traffic classification
`engine 137 is operative to analyze data flow attributes and
`identify traffic classes corresponding to the data flows, as
`discussed more fully below.
`In one embodiment,
`traffic
`classification engine 137 stores traffic classes associated
`with data flows encountered during operation of bandwidth
`management device 30, as well as manually created traffic
`classes and a hierarchical
`traffic class structure,
`if any,
`configured by a network administrator. In one embodiment,
`traffic classification engine 137 stores traffic classes,
`in
`association with pointers to bandwidth utilization controls or
`pointers to data structures defining such bandwidth utiliza-
`tion controls. Suspicion scoring module 138 is operative to
`examine data flows associated with individual users and
`evaluate whether characteristics of the data flows indicate
`
`suspicious activity (e.g., an attempt to evade classification
`and, therefore, configured bandwidth management controls,
`and/or indications that such attempts may be likely). Mea-
`surement engine 140 maintains measurement data relating to
`operation of bandwidth management device 30 to allow for
`monitoring of bandwidth utilization across access link 21
`with respect to a plurality of bandwidth utilization and other
`network statistics on an aggregate and/or per-trafiic-class
`level.
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`
`
`US 7,296,288 B1
`
`7
`Administrator interface 150 facilitates the configuration
`of bandwidth management device 30 to adjust or change
`operational and configuration parameters associated with the
`device. For example, administrator interface 150 allows
`administrators to select identified traffic classes and associ-
`
`ate them with bandwidth utilization controls, such as a
`partition, as well as other controls. Administrator interface
`150 also displays various views associated with a hierarchi-
`cal traffic classification scheme and allows administrators to
`
`traffic classification
`configure or revise the hierarchical
`scheme as discussed more fully below. Administrator inter-
`face 150 can be a command line interface or a graphical user
`interface accessible, for example, through a conventional
`browser on client device 42.
`
`Al. Packet Processing
`In one embodiment, when packet processor 131 encoun-
`ters a new data flow it stores the source and destination IP
`
`addresses contained in the packet headers in host database
`134. Packet processor 131 further constructs a control block
`object
`including attributes characterizing a specific flow
`between two end systems. In one embodiment, a control
`block object contains a flow specification object (or a pointer
`thereto) including such attributes as pointers to the “inside”
`and “outside” IP addresses in host database 134, as well as
`other flow specification parameters, such as inside and
`outside port numbers, service type, protocol type and other
`parameters characterizing the data flow. In one embodiment,
`such parameters can include information gleaned from
`examination of data within layers 2 through 7 of the OSI
`reference model. U.S. Pat. No. 6,046,980, incorporated by
`reference herein, discloses classification of data flows for
`use in a packet-based communications environment. FIG. 1
`illustrates the concept associated with inside and outside
`addresses. As discussed above, in one embodiment, a flow
`specification object
`includes an “inside” and “outside”
`address relative to bandwidth management device 30. See
`FIG. 1. For a TCP/IP packet, packet processor 131 can
`compute the inside and outside addresses based on the
`source and destination addresses of the packet and the
`direction of the packet flow.
`In one embodiment, packet processor 131 creates and
`stores control block objects corresponding to data flows in
`flow database 135. In one embodiment, control block object
`attributes include a pointer to a corresponding flow speci-
`fication object, as well as other flow state parameters, such
`as TCP connection status,
`timing of last packets in the
`inbound and outbound directions, speed information, appar-
`ent round trip time, number of packets, aggregate bytes, etc.
`Control block object attributes further include at least one
`traffic class identifier (or pointer(s) thereto) associated with
`the data flow, as welt as policy parameters (or pointers
`thereto) corresponding to the identified traffic class. In one
`embodiment, control block objects further include a list of
`traffic classes for which measurement data associated with
`
`to
`the data flow should be logged. In one embodiment,
`facilitate association of an existing control block object to
`subsequent packets associated with a data flow or connec-
`tion, flow database 135 further maintains a control block
`hash table including a key comprising a hashed value
`computed from a string comprising the inside IP address,
`outside IP address, inside port number, outside port number,
`and protocol type (e.g., TCP, UDP, etc.) associated with a
`pointer to the corresponding control block object. According
`to this embodiment,
`to identify whether a control block
`object exists for a given data flow, packet processor 131
`hashes the values identified above and scans the hash table
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`for a matching entry. If one exists, packet processor 131
`associates the pointer to the corresponding control block
`object with the data flow.
`As discussed above, host database 134 stores the IP or
`other computer network addresses associated with the end
`systems identified in data flows traversing bandwidth man-
`agement device. In one embodiment, host database 134
`maintains for each IP address, an “inside/outside” flag value
`set to indicate whether the IP address corresponds to an
`“inside” or “outside” system (see above), and a suspicion
`level, score or other value computed by suspicion scoring
`module 138, as described more fully below. In other embodi-
`ments, host database 134 can also be extended to maintain
`other suspicion score values, such as an average suspicion
`score over an interval, the lowest/highest suspicion score
`over an interval, etc. In one embodiment, host database 134
`is maintained in a host object space, which (in one embodi-
`ment) is a computer-readable medium (e.g., Flash memory),
`or an allocated portion thereof, allowing for storage, access
`to and modification of host objects including the attributes
`defining the hosts. In one embodiment, the host object space
`is finite allowing for the storage of only a limited number of
`host objects at any one time. In one embodiment, bandwidth
`management device 30 maintains a Least-Recently Used
`(LRU) list of host objects corresponding to inactive data
`flows based on the last packet time associated with the host
`objects. In one embodiment, packet processor 131 adds
`pointers to the LRU list as the corresponding flows become
`inactive (e.g., when a TCP connection is closed, or a
`threshold period of inactivity is reached, etc.).
`In one
`embodiment, when a new host is detected and the ho st object
`space is depleted, packet processor 131 selects the least-
`recently used host object from the LRU list and overwrites
`the host object space with a new host object.
`A.2. Flow Control Module
`As discussed above, flow control module 132 enforces
`bandwidth utilization controls (and, in some embodiments,
`other policies) on data flows traversing access link 21. A
`bandwidth utilization control for a particular data flow can
`comprise an aggregate bandwidth utilization control, a per-
`flow bandwidth utilization control, or a combination of the
`two. Flow control module 132 can use any suitable func-
`tionality to en