`
`as) United States
`
`
`
`
`
`
`
`
`
`a2) Patent Application Publication 0) Pub. No.: US 2004/0090923 Al
`
`
`
`
`
`(43) Pub. Date:
`May13, 2004
`
`
`Kanet al.
`
`US 20040090923A1
`
`
`
`
`
`
`
`
`(54) NETWORK MONITORING SYSTEM
`RESPONSIVE TO CHANGES IN PACKET
`
`
`ARRIVAL VARIANCE AND MEAN
`
`
`
`
`
`
`
`(76)
`
`
`
`Inventors: Chao Kan, Frisco, TX (US); Aziz
`
`
`
`
`
`
`
`
`
`
`Mohammed,Plano, TX (US); Wei
`
`
`
`
`Hao, Richardson, TX (US); Jimin Shi,
`
`
`Plano, TX (US)
`
`
`Correspondence Address:
`ALCATEL USA
`
`
`INTELLECTUAL PROPERTY DEPARTMENT
`
`
`
`
`
`
`3400 W. PLANO PARKWAY, MS LEGL2
`
`
`
`PLANO, TX 75075 (US)
`
`
`
`
`
`
`10/412,127
`
`
`
`
`
`(21) Appl. No.:
`
`
`
`(22)
`
`Filed:
`
`
`
`Apr. 11, 2003
`
`
`
`Related U.S. Application Data
`
`
`
`
`
`
`
`
`
`
`(60) Provisional application No. 60/424,495,filed on Nov.
`7, 2002.
`
`
`
`
`
`Publication Classification
`
`
`
`
`
`
`
`(51) Unt. C07 caccccccsssssssesnssstnssstvesnetesneee H04J 1/16
`
`
`
`
`
`(52) US. Ch.
`cacsscssssssssssstnsinesnsrstn 370/252; 370/329
`
`
`
`
`
`
`(57)
`
`
`
`ABSTRACT
`
`
`
`
`
`
`
`
`
`
`
`A network monitoring system (10) for monitoring a network
`
`
`
`
`
`
`
`
`along which networktraffic flows in a form of packets. The
`
`
`
`
`
`
`
`
`system comprises circuitry (36, 42) for receiving a packet
`
`
`
`
`
`
`
`communicated along the network and for determining
`
`
`
`
`
`
`
`whetherthe received packetsatisfies a set of conditions. The
`
`
`
`
`
`
`
`system further comprisescircuitry (36/30, 46), responsive to
`
`
`
`
`
`
`
`
`
`
`a determination that the received packetsatisfies the set, for
`
`
`
`
`
`
`determining a measure, wherein the measure is determined
`
`
`
`
`
`
`
`
`over a defined time interval and comprises a ratio of packet
`
`
`
`
`
`
`
`
`arrival variance and a mean of packets arriving during the
`
`
`
`
`
`
`
`time interval and for comparing the measure to a threshold.
`
`
`
`
`
`
`
`Lastly, the system comprisescircuitry (36, 52), responsive to
`
`
`
`
`
`
`
`
`the measure exceeding the threshold, for adjusting network
`resources.
`
`
`40
`
`
`
`
`
`
`
`
`
`
`
`
`CAPTURE PACKET;
`
`
`
`PACKET SATISFY RULE
`
`
`IN RULE SET(S
`
`
`
`
`STORE PACKET INFORMATION,
`
`
`
`INCLUDING TIME OF ARRIVAL,
`
`
`IN FLOW CORRESPONDING
`
`
`
`TO SATISFIED RULE(S)
`
`
`
`
`
`46
`
`
`
`
`
`
`FOR DEFINED INTERVAL, t,
`
`
`
`
`
`DETERMINE IDC FOR EACH FLOW
`
`
`
`
`
`
`IDC > THRESHOLD?
`
`
`
`
`
`QoS MET FOR
`
`
`
`THRESHOLD-EXCEEDING |,
`
`
`PACKET(S)?
`
`
`
`30
`
`
`
`52
`
`
`
`
`RE-ADJUST TRAFFIC
`PARAMETERS
`
`
`
`
`
`
`
`
`
` 44
`
`
`
`
`
`
`Splunk Inc.
`
`Exhibit 1009
`
`Page 1
`
`Splunk Inc. Exhibit 1009 Page 1
`
`

`

`
`
`
`
`
`
`
`Patent Application Publication May 13, 2004 Sheet 1 of 2
`
`
`
`US 2004/0090923 Al
`
`
`
`
`
`
`CORE NETWORK/ROUTER
`
`
`
`Splunk Inc.
`
`Exhibit1009
`
`Page 2
`
`Splunk Inc. Exhibit 1009 Page 2
`
`

`

`
`
`
`
`
`
`Patent Application Publication May 13, 2004 Sheet 2 of 2
`
`
`
`US 2004/0090923 Al
`
`40
`
`
`
`
`
`
`
`
`
`
`
`44
`
`
`
`
`
`
`
`
`
`46
`
`
`
`
`CAPTURE PACKET;
`
`
`
`
`PACKET SATISFY RULE
`
`
`_IN RULE SET(S),
`
`
`
`
`
`
`STORE PACKET INFORMATION,
`
`
`
`
`INCLUDING TIME OF ARRIVAL,
`
`
`
`
`
`IN FLOW CORRESPONDING
`
`
`
`TO SATISFIED RULE(S)
`
`
`
`
`
`
`
`
`FOR DEFINED INTERVAL, t,
`
`
`
`
`DETERMINE IDC FOR EACH FLOW
`
`
`
`
`IDC > THRESHOLD?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`QoS MET FOR
`
`THRESHOLD-EXCEEDING >
`PACKET(S)?_ ~~
`
`
`
`
`
`
`
`90
`
`
`
`
`RE-ADJUST TRAFFIC
`. 52
`
`PARAMETERS
`
`FIG. 3
`
`Splunk Inc.
`
`Exhibit1009
`
`Page 3
`
`Splunk Inc. Exhibit 1009 Page 3
`
`

`

`
`
`US 2004/0090923 Al
`
`
`May 13, 2004
`
`
`
`NETWORK MONITORING SYSTEM RESPONSIVE
`
`
`
`
`TO CHANGESIN PACKET ARRIVAL VARIANCE
`
`
`
`
`
`AND MEAN
`
`
`
`CROSS-REFERENCES TO RELATED
`
`APPLICATIONS
`
`
`
`
`
`
`
`
`
`[0001] This application claims the benefit, under 35
`
`
`
`
`
`
`
`US.C. §119(e)(1), of U.S. Provisional Application No.
`
`
`
`
`
`
`
`60/424,495, filed Nov. 7, 2002, and incorporated herein by
`this reference.
`
`
`
`
`
`STATEMENT REGARDING FEDERALLY
`
`
`
`SPONSORED RESEARCH OR DEVELOPMENT
`
`
`
`
`
`
`
`
`
`
`[0002] Not Applicable.
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`[0003] The present embodiments relate to computer net-
`
`
`
`
`
`
`
`
`
`works and are more particularly directed to a system for
`
`
`
`
`
`
`monitoring network performance and correcting network
`
`
`
`
`
`
`congestion by evaluating changes in packetarrival variance
`
`
`
`
`
`relative to mean packetarrival.
`
`
`
`
`
`
`
`
`[0004] As the numberofusers andtraffic volume continue
`
`
`
`
`
`
`
`
`
`
`to grow on the global Internet and other networks, an
`essential need has arisen to have a set of mechanisms to
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`monitor network performance and to take corrective mea-
`
`
`
`
`
`
`sures in response to falling performance. Such performance
`
`
`
`
`
`
`
`
`may be evaluated in various forms, including but not limited
`
`
`
`
`
`
`
`to detecting and troubleshooting network congestion. Net-
`
`
`
`
`
`
`
`work congestion results from mismatches between network
`
`
`
`
`
`
`
`capacity and network demand. The mismatch may be a
`
`
`
`
`
`
`
`
`long-term one, or at
`instantaneous time scales. Further,
`
`
`
`
`
`
`
`
`network capacity may appear to be ample when using tools
`
`
`
`
`
`
`
`
`look at
`long-term traffic averages; however
`these
`that
`
`
`
`
`
`
`
`
`approaches are not always suitable because a more subtle
`
`
`
`
`
`
`
`
`
`
`problem may arise with short bursts of packets, or peak
`
`
`
`
`
`
`
`demand. With congestion analyses mechanisms,the reliabil-
`
`
`
`
`
`
`
`
`
`
`ity and availability of the network nodes(e.g., IP routers)
`
`
`
`
`
`
`
`
`
`and the given internet paths can be evaluated. This is
`
`
`
`
`
`
`
`
`especially true for Internet Service Providers (“ISPs”) seek-
`
`
`
`
`
`
`
`
`ing to comply with the Service Level Agreements (“SLAs”)
`
`
`
`
`
`
`
`
`that they are now providing to customers. Additionally, such
`
`
`
`
`
`
`
`
`a need is prevalent for the underlying internet protocol
`
`
`
`
`(“IP”) networks in the Internet.
`
`
`
`
`
`
`
`[0005] The Internet is also evolving towards an advanced
`
`
`
`
`
`
`
`
`architecture that seeks to guarantee the quality of service
`
`
`
`
`
`
`
`
`(“QoS”) for real-time applications. QoS permits the control-
`
`
`
`
`
`
`
`ling of what happens to packets whenthere is congestion in
`
`
`
`
`
`
`
`
`
`a network, or more precisely when there is insufficient
`
`
`
`
`
`
`
`
`
`network capacity to deliver all of the offered load without
`
`
`
`
`
`
`
`
`any noticeable queuing delays. One type of QoS framework
`
`
`
`
`
`
`
`seeks to provide hard specific network performance guar-
`
`
`
`
`
`antees to applications such as band-width/delay reservations
`
`
`
`
`
`
`
`
`
`for an imminent or future data flow. Such QoSis usually
`
`
`
`
`
`characterized in terms of ability to guarantee to an applica-
`
`
`
`
`
`
`
`
`tion-specified peak and average band-width,delay, jitter and
`
`
`
`
`
`
`
`packet loss. Another type is to use Class-of-Service (“CoS”)
`
`
`
`
`
`
`such as Differentiated Services (“Diff-Serv”) to representthe
`
`
`
`
`
`
`less ambitious approach of giving preferential treatment to
`
`
`
`
`
`
`
`
`certain kinds of packets, but without making any perfor-
`
`
`mance guarantees.
`
`
`
`
`
`
`
`[0006] During the QoS process to provide services better
`
`
`
`
`
`
`
`thanthe traditional besteffort, network congestion detection
`
`
`
`
`
`
`
`
`
`
`
`
`
`often becomes the starting point for the network perfor-
`
`
`
`
`
`
`
`mance analysis. In the past, a number of congestion detec-
`
`
`
`
`
`
`
`
`
`tion and control schemes have been investigated in data
`
`
`
`
`
`
`
`
`networks. One congestion detection schemeuses the trans-
`
`
`
`
`
`
`
`port-layer protocols to infer congestion from the estimated
`
`
`
`
`
`
`bottleneck service time or from changes in throughput or
`
`
`
`
`
`
`
`end-to-end delay, as well as from packet drops. Specifically,
`
`
`
`
`
`
`
`
`the Internet has traditionally relied on mechanisms in the
`
`
`
`
`
`
`
`Transport Control Protocol (“TCP”), suchassliding window
`control and retransmission timer deficiencies to avoid con-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`gestion. TCP operates to seek excess bandwidth by increas-
`
`
`
`
`
`
`
`
`ing transmission rates until the network becomes congested
`
`
`
`
`
`
`
`
`and then reducing transmission rate once congestion occurs.
`
`
`
`
`
`
`
`
`A few limitations arise from this approach. First, TCP
`
`
`
`
`
`
`
`congestion detection at a first node requires an acknowl-
`
`
`
`
`
`
`
`
`
`edgement from a second node,that is, the increased trans-
`
`
`
`
`
`mission is continued until no acknowledgementis received
`
`
`
`
`
`
`
`from the second node; thus, a feedback communication is
`
`
`
`
`
`
`
`
`
`required from another node and that feedback also utilizes
`
`
`
`
`
`
`
`
`bandwidth on the network. Second, in its effort to identify
`
`
`
`
`
`
`
`bandwidth, TCP necessarily causes the very congestion
`
`
`
`
`
`
`
`which it then seeks to minimize, where the congestion is
`
`
`
`
`
`
`
`
`caused as the TCP increases the bandwidth to a point that
`
`
`
`
`
`
`
`
`exceeds the network capacity. Another type of congestion
`
`
`
`
`
`
`detection schemeis to involve network components such as
`
`
`
`
`
`
`
`routers in the entire process. As most network congestion
`
`
`
`
`
`
`
`occurs in routers, they may be considered an ideal position
`
`
`
`
`
`
`
`
`to monitor network load and congestion and respondthereto
`
`
`
`
`
`
`
`in a control scheme. Such network-based congestion control
`
`
`
`
`
`
`
`uses explicit signaling between routers to provide feedback
`
`
`
`
`
`
`congestion information to a transmitting router, where the
`
`
`
`
`
`
`
`
`transmitting router may then alter its behavior in responseto
`
`
`
`
`
`
`
`
`the feedback, or an overall scheme can change the packet
`
`
`
`
`
`
`
`
`
`processing within one or more routers so as to reduce
`
`
`
`
`
`
`
`
`congestion. In any event, this latter scheme also requires a
`
`
`
`
`
`
`
`form of feedback from a recipient router, thereby increasing
`traffic on the network to accommodate the feedback and also
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`requiring the reliance of the transmitting router on the
`
`
`
`
`integrity of a different router.
`
`
`
`
`
`
`
`
`[0007]
`In view of the above, there arises a need to address
`
`
`
`
`
`
`
`the drawbacks of the prior art, as is accomplished by the
`
`
`
`
`preferred embodiments described below.
`BRIEF SUMMARY OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`[0008]
`In the preferred embodiment, there is a network
`
`
`
`
`
`
`
`monitoring system along which network traffic flows in a
`
`
`
`
`
`
`
`
`form of packets. The system comprises circuitry for receiv-
`
`
`
`
`
`
`
`
`ing a packet communicated along the network and for
`
`
`
`
`
`
`
`
`determining whether the received packetsatisfies a set of
`
`
`
`
`
`
`
`conditions. The system further comprises circuitry, respon-
`
`
`
`
`
`
`
`
`sive to a determination that the received packetsatisfies the
`
`
`
`
`
`
`
`
`set, for determining a measure and circuitry for comparing
`
`
`
`
`
`
`
`the measure to a threshold, wherein the measure is deter-
`
`
`
`
`
`
`
`
`mined over a defined time interval and comprisesa ratio of
`
`
`
`
`
`
`
`packet arrival variance and a mean of packets arriving
`
`
`
`
`
`
`
`
`
`during the time interval. Lastly, the system comprises cir-
`
`
`
`
`
`
`
`cuitry, responsive to the measure exceeding the threshold,
`
`
`
`
`for adjusting network resources.
`
`
`
`
`
`
`
`[0009] Other aspects are also described and claimed.
`
`
`
`
`
`
`
`
`Splunk Inc.
`
`Exhibit1009
`
`Page 4
`
`Splunk Inc. Exhibit 1009 Page 4
`
`

`

`
`
`US 2004/0090923 Al
`
`
`May 13, 2004
`
`
`
`BRIEF DESCRIPTION OF THE SEVERAL
`
`
`
`VIEWS OF THE DRAWING
`
`
`
`
`
`
`
`
`
`[0010] FIG. 1 illustrates a block diagram of a network
`
`
`
`
`
`
`
`system 10 into which the preferred embodiments may be
`
`implemented.
`
`
`
`
`
`
`FIG.2 illustrates a block diagram of each network
`{0011]
`
`
`
`
`monitor NM, through NMgof FIG.1.
`
`
`
`
`
`
`
`FIG.3 illustrates a flow chart of the operation of
`[0012]
`
`
`
`
`
`
`each network monitor NM, through NM,of FIG.2.
`DETAILED DESCRIPTION OF THE
`
`
`INVENTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0013] FIG. 1 illustrates a block diagram of a system 10
`
`
`
`
`
`
`into which the preferred embodiments may be implemented.
`
`
`
`
`
`
`
`System 10 generally includes a number of stations ST,
`
`
`
`
`
`
`
`through ST,, each coupled to a network 20 via a router, and
`
`
`
`
`
`
`
`each operable to send packets as a source or receive packets
`
`
`
`
`
`
`
`as a destination. By way of example, network 20 is an
`
`
`
`
`
`
`
`
`internet protocol (“IP”) network such as the global Internet
`
`
`
`
`
`
`
`
`
`or other [P-using network, where each station and IP net-
`
`
`
`
`
`
`
`
`
`
`worksin general are well knowninthe art. One skilled in the
`
`
`
`
`
`
`
`
`art should appreciate that the use of the IP protocol is by way
`
`
`
`
`
`
`
`
`of illustration, and manyof the various inventive teachings
`
`
`
`
`
`
`
`herein may apply to numerousother protocols, including by
`
`
`
`
`
`
`way of examples asynchronous transfer mode (“ATM”),
`
`
`
`
`
`
`
`
`
`token ring, Novell, Apple Talk,and still others. In any event,
`
`
`
`
`
`
`returning to network 20 as an IP network, and also by way
`
`
`
`
`
`
`
`
`of an example, each station ST, may be constructed and
`
`
`
`
`
`
`
`
`
`function as one of various different types of computing
`
`
`
`
`
`
`devices, all capable of communicating according to the IP
`
`
`
`
`
`
`
`
`
`protocol. Lastly and also by way of example, only four
`
`
`
`
`
`
`stations ST,, are shown so as to simplify theillustration and
`
`
`
`
`
`
`
`
`example, where in reality each such station may be proxi-
`
`
`
`
`
`
`
`
`mate other stations (not shown) and at a geography located
`at a considerable distance from the other illustratedstations.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0014] Continuing with FIG. 1, along the outer periphery
`
`
`
`
`
`
`
`
`of network 20 are shown a number of edge routers ER,
`
`
`
`
`
`
`
`through ER,,, while within network 20 are shown a number
`
`
`
`
`
`
`
`
`
`of core routers CR, through CR,. The terms edge router and
`
`
`
`
`
`
`
`
`
`
`
`core router are known in the art and generally relate to the
`
`
`
`
`
`
`
`function and relative network location of a router. Typically,
`
`
`
`
`
`
`
`edge routers connect
`to remotely located networks and
`
`
`
`
`
`
`
`
`handle considerably less traffic than core routers. In addition
`
`
`
`
`
`
`
`
`and due in part to the relative amountoftraffic handled by
`
`
`
`
`
`
`
`
`core routers, they tend to perform less complex operations
`
`
`
`
`
`
`
`
`on data and instead serve primarily a switching function; in
`
`
`
`
`
`
`
`other words, because of the tremendous amountof through-
`
`
`
`
`
`
`
`
`
`put expected of the core routers, they are typically hardware
`
`
`
`
`
`
`
`
`bound as switching machines and notgiven the capability to
`
`
`
`
`
`
`
`
`provide operations based on the specific data passing
`
`
`
`
`
`
`
`
`
`through the router. Indeed, core routers typically do not
`
`
`
`
`
`
`
`include much in the way of control mechanisms as there
`
`
`
`
`
`
`could be 10,000 or more connections in a single trunk.
`
`
`
`
`
`
`
`
`
`Further, typically core routers do not involve their opera-
`tions with TCPrelated items and instead deal at the IP level
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and below. In contrast, edge routers are able to monitor
`
`
`
`
`
`
`
`various parameters within data packets encountered by the
`
`
`
`
`
`
`
`
`
`respective router. In any event, the various routers in FIG.
`
`
`
`
`
`
`
`
`
`1 are shown merely by way of example, where one skilled
`
`
`
`
`
`
`
`
`
`
`in the art will recognize that a typical network may include
`
`
`
`
`
`
`
`
`quite a different number of both types of routers. Finally,
`
`
`
`
`
`
`
`
`
`note that each core router CR, and each edge router ER,, may
`
`
`
`
`
`
`
`
`
`
`be constructed and function according to the art, with the
`
`
`
`
`
`
`
`
`exception that preferably selected ones of those routers may
`
`
`
`
`
`
`
`include additional functionality for purposes oftraffic con-
`
`
`
`
`
`
`
`
`gestion detection and response based on packet arrival
`
`
`
`
`
`
`
`
`variance and mean as described later. In addition, selected
`
`
`
`
`
`
`
`routers may be further constructed to respond to the traffic
`
`
`
`
`
`
`
`congestion detection that the router determines as well as in
`
`
`
`
`
`
`
`responseto the traffic congestion detection of another router
`
`
`
`
`
`
`
`
`
`in network 20. Moreover, in one approach, core routers may
`
`
`
`
`
`
`
`
`be configured to respond differently than edge routers in the
`
`
`
`
`case of detecting traffic congestion.
`
`
`
`
`
`
`
`[0015] Completing the discussion of FIG.1, note that the
`
`
`
`
`
`
`
`
`
`various stations, edge routers, and core routers therein are
`shownconnected to one another in various fashions and also
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`by way of example. Such connections are intended to
`
`
`
`
`
`
`
`
`
`illustrate an example for later discussion of the preferred
`
`
`
`
`
`
`
`
`
`operation and also to reflect a general depiction of how
`
`
`
`
`
`
`networksare generally established. Thus, each station ST, is
`
`
`
`
`
`
`
`
`shown connected to a single edge router ER,, where that
`
`
`
`
`
`
`
`
`edge router ER, is connected to one or more core routers
`
`
`
`
`
`
`
`
`
`CR,,. The core routers CR,, also by way of example, are
`
`
`
`
`
`
`
`
`shown connected to multiple ones of the other core routers
`
`
`
`
`
`
`
`CR,,. By way of reference, the following Table 1 identifies
`each station and router shown in FIG.1 as well as the other
`
`
`
`
`
`
`
`
`
`
`
`
`
`device(s) to which each is connected.
`TABLE1
`
`connected nodes
`
`
`
`
`station or router
`
`
`
`
`
`ST,
`
`ST,
`
`ST3
`
`ST,
`
`ER,
`
`ER,
`
`ER;
`
`ER,
`
`ERs
`
`ER,
`
`ER,
`
`ER,
`
`ERg
`
`ERio
`ER,
`
`CR,
`
`CR,
`
`CR;
`
`Ry
`
`
`
`
`
`
`
`
`ER,
`
`ERyo
`
`ERs
`
`ER;
`
`ST,; CR,
`
`CR,; CR,
`
`CR,
`
`CR,
`
`ST3; CR,; CR3
`
`
`CRs; CRy
`
`STy; CRy
`
`CRy
`
`CR,
`
`ST; CR,
`
`CR,
`
`ER,; ERy1; ERio; ERy; CR;
`
`
`
`
`CR,; CR,
`
`ER,; ER3; ERy; CRy; CR3;
`
`
`
`
`CRy; ERs
`
`
`ERs; ERs; CRz; CR,; CRy
`
`
`
`
`ER,; ERg; ERo; CR,; CR;
`
`
`
`
`
`CRs; ER,6.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0016] Given the variousillustrated connections as also set
`
`
`
`
`
`
`
`
`forth in Table 1, in general IP packets flow along the various
`
`
`
`
`
`
`
`illustrated paths of network 20, and in groups or in their
`
`
`
`
`
`
`
`
`entirety such packets are often referred to as networktraffic.
`
`
`
`
`
`
`
`
`
`In this regard and as developed below,the preferred embodi-
`
`
`
`
`
`
`
`ments operate to identify and respond to congestion in such
`
`
`
`
`
`
`
`
`network traffic. Finally, note that FIG. 1 may represent a
`
`
`
`
`
`
`
`simplified version of a network or the Internet in that only
`
`
`
`
`
`
`
`
`
`
`
`a few stations and routers are shown, while one skilled in the
`
`
`
`
`
`
`
`
`
`art will readily appreciate that the inventive concepts in this
`
`
`
`
`
`
`document may be applied to a larger numberofstations,
`
`
`
`
`
`
`
`routers, and the network interconnectivity between those
`devices.
`
`
`
`
`
`
`
`{0017] FIG. 1 also illustrates a number of network moni-
`
`
`
`
`
`
`tors NM, through NM, according to the preferred embodi-
`
`
`
`
`Splunk Inc.
`
`Exhibit1009
`
`Page5
`
`Splunk Inc. Exhibit 1009 Page 5
`
`

`

`
`
`US 2004/0090923 Al
`
`
`May 13, 2004
`
`
`
`
`
`
`
`
`
`
`
`ments, where the choice of eight such network monitors is
`
`
`
`
`
`
`
`
`only by way of example given the amountof other hardware
`
`
`
`
`
`
`
`
`
`
`is shown for network 20. As detailed below, each
`that
`
`
`
`
`
`
`
`network monitor NM,, is operable to sample each packetthat
`
`
`
`
`
`
`
`
`is received along the conductor(s) to which the network
`
`
`
`
`
`
`monitor is connected, and if corrective action is deemed as
`
`
`
`
`
`
`
`
`useful then a routing table associated with a router that is
`
`
`
`
`
`
`
`
`also associated with the network monitor NM, may be
`
`
`
`
`
`
`modified to improve network performance. The components
`
`
`
`
`
`
`
`of each network monitor NM,, are described below, but at
`
`
`
`
`
`
`
`
`
`this point the connections of each such monitor are noted in
`
`
`
`the following Table 2:
`
`
`
`network monitor
`
`NM,
`
`NM,
`
`NM;
`
`NM,
`
`NMS;
`
`NM
`
`NM,
`
`NMg
`
`
`TABLE 2
`
`connected nodes
`
`
`
`
`CR,; CR,
`
`CR,; CR;
`
`CRy; CRs
`
`CRs; CRy
`
`CR,; CR; CR3; ERz; ERg;
`
`
`
`
`ERg
`
`CR, ST,
`
`ST; CR,
`
`ERs; ST3
`
`
`
`
`
`
`
`
`
`
`
`
`[0018] FIG. 1 and Table 2 demonstrate that each of
`
`
`
`
`
`network monitors NM, through NM, and NMgis connected
`
`
`
`
`
`
`
`
`to sample packets passing along the conductor(s) between a
`
`
`
`
`
`
`
`pair of nodes, such as between routers or between a router
`
`
`
`
`
`
`
`and a station. However, network monitors NM;, NM,, and
`
`
`
`
`
`
`NM,are each by way ofalternative examples incorporated
`
`
`
`
`
`
`
`
`
`into respective routers CR,, ER,, and ER,,. As a result, each
`
`
`
`
`
`
`
`
`of network monitors NM, NM,, and NM,is able to sample
`
`
`
`
`
`
`
`packets communicated with any of the nodes to which its
`
`
`
`
`
`
`
`respective router is connected; for example with respect to
`
`
`
`
`
`
`
`network monitor NM., it may sample packets communi-
`
`
`
`
`
`
`
`
`cated with respect to any node to which core router CR, is
`
`
`
`
`
`
`
`
`connected, namely, core routers CR,, CR,, CR;, and edge
`
`
`
`
`
`
`
`
`
`routers ER;, ERg, and ER,. Thus, the contrast of network
`
`
`
`
`
`
`
`monitors NM,, NM,, and NM,
`to the other illustrated
`
`
`
`
`
`network monitors NM, through NM,is shown to demon-
`
`
`
`
`
`
`
`
`strate that in the preferred embodiment each network moni-
`
`
`
`
`
`
`
`tor NM, may sample packets as a stand alone entity or may
`
`
`
`
`
`
`
`
`be combined with the hardware and software of an existing
`
`
`
`
`
`
`
`router;
`indeed,
`in the preferred embodiments a network
`
`
`
`
`
`
`monitor NM, also may be combined with network or ele-
`
`
`
`
`
`
`
`
`ment management systems. In any event and by way of
`
`
`
`
`
`
`
`
`introduction to details provided later,
`in the preferred
`
`
`
`
`
`
`
`embodiments the sampling functionality of each network
`
`
`
`
`
`
`monitor NM, permits real-time monitoring, over a defined
`
`
`
`
`
`
`
`
`period of time, of a ratio of the packet arrival variance and
`
`
`
`
`
`
`mean for selected packets, and in response determinations
`
`
`
`
`
`
`
`
`may be made,and actions maybe taken, based on thresholds
`
`
`
`
`
`
`
`exceeded by the ratio, thereby presenting an indication of
`
`
`
`
`likely network traffic congestion.
`
`
`
`
`
`
`
`[0019]
`FIG.2 illustrates a block diagram of each network
`
`
`
`
`
`
`monitor NM, through NM, and NM,of FIG. 1, with the
`
`
`
`
`
`
`further understanding that
`functionally the
`following
`
`
`
`
`
`
`
`description also may be applied to any of network monitors
`
`
`
`
`
`
`
`
`
`NM,, NM,, and NM,, with the addition that certain func-
`
`
`
`
`
`
`
`tionality may be provided by the hardware and software
`
`
`
`
`
`
`
`
`
`already available from each respective router CR,, ER, and
`
`
`
`
`
`
`ER,,. Turning then to FIG. 2, a console 30 is associated
`
`
`
`
`
`
`
`
`with network monitor NM,,, where in the preferred embodi-
`
`
`
`
`
`
`
`
`ment a single such console 30 communicates with multiple
`
`
`
`
`
`
`
`network monitors NM,,. For example, returning briefly to
`
`
`
`
`
`
`FIG. 1, preferably each of network monitors NM,through
`
`
`
`
`
`
`
`
`NM, communicates with a single console 30, where such
`
`
`
`
`
`
`communications also may be by way of packets between
`
`
`
`
`
`
`
`
`console 30 and the network monitors NM,. Console 30 may
`
`
`
`
`
`
`
`
`
`be constructed by one skilled in the art using various forms
`
`
`
`
`
`
`
`
`of hardware and software, where the selection is a matter of
`
`
`
`
`
`
`implementation choice in order to achieve the functionality
`
`
`
`
`
`
`described in this document. Turning to that functionality,
`
`
`
`
`
`console 30 preferably provides an administration (configu-
`
`
`
`
`
`
`
`ration) function and a reporting function. To permit a user to
`
`
`
`
`
`
`
`
`perform these functions, various interfaces may be used such
`
`
`
`
`
`
`as a graphical Web-based configuration tool or a command
`
`
`
`
`
`
`
`
`line tool, where one preferable approach implements con-
`
`
`
`
`
`
`
`
`
`sole 30, by way of example, using the Apache Web server
`
`
`
`
`
`
`
`with the PHP server-side scripting language to provide a
`
`
`
`
`
`
`
`
`
`dynamic interface to a flow store 32. In any event, the user
`
`
`
`
`
`
`
`interface preferably provides different screens for each of
`
`
`
`
`
`
`
`
`the administration and reporting functions. The actual inter-
`
`
`
`
`
`
`
`face and screens, however, may be implemented in various
`
`
`
`
`
`
`
`
`
`forms, where it is desirable for any form that an operator
`
`
`
`
`
`
`may properly control console 30 to perform its administra-
`
`
`
`
`
`
`
`
`
`
`tion and reporting functions with respect to its flow store 32.
`
`
`
`
`
`
`
`
`Preferably, a network administrator or the like uses the
`
`
`
`
`
`administration functionality of console 30 to create a set of
`
`
`
`
`
`
`
`
`rules and to provide those rules, along with a methodology
`
`
`
`
`
`
`
`
`for determining an index dispersion for counts (“IDC”) and
`
`
`
`
`
`
`
`responding thereto, to an RTFM meter 36 described later. By
`
`
`
`
`
`
`
`
`wayofintroduction, the set of rules causes meter 36 to store
`
`
`
`
`
`
`
`
`
`
`packet arrival time for each packet that satisfies one (or
`
`
`
`
`
`
`
`
`
`
`that
`more) of the provided rules,
`is,
`those packets are
`
`
`
`
`
`
`
`
`
`selected from among all packets that pass through the meter
`
`
`
`
`
`
`
`
`
`with the arrival
`time of such selected packets being
`
`
`
`
`
`
`
`recorded; thereafter, meter 36 determines the IDC for the
`
`
`
`
`
`
`
`
`selected packets over a specified time interval t. Also, once
`
`
`
`
`
`
`
`
`
`the system is configured and left to collect packet arrival
`
`
`
`
`
`
`
`
`about the monitored packets, the network administrator can
`
`
`
`
`
`
`
`use the reporting screen to query the information so as to
`
`
`
`
`
`
`
`
`generate network status reports based on the collected
`
`
`
`
`
`
`information and to thereby analyze network congestion.
`
`
`
`
`
`
`[0020] As introduced above, console 30 is connected to a
`
`
`
`
`
`
`
`
`flow store 32, which preferably represents a storage medium
`
`
`
`
`
`
`
`that stores a flow database relating to monitored packets. In
`
`
`
`
`
`
`
`the preferred embodiment, each network monitor NM,
`
`
`
`
`
`
`
`includesits own flow database, although alternative embodi-
`
`
`
`
`
`
`
`
`
`ments may be created where more than one network monitor
`
`
`
`
`
`
`
`
`NM,, shares a common flow database.
`In the preferred
`embodiment the flow database in flow store 32 is an SQL-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`compatible database using the PostgreSQLrelational data-
`
`
`
`
`
`
`base management system, although in alternative embodi-
`
`
`
`
`
`
`
`
`ments various other types of databases may be used in flow
`
`
`
`
`
`
`
`store 32. Using the preferred embodiment as an example,
`
`
`
`
`
`
`
`console 30 communicates with this database through the
`
`
`
`
`
`
`
`
`
`
`Web server’s PHP link to the SQL database. Thus, any
`
`
`
`
`
`
`
`administration and configuration changes made via console
`
`
`
`
`
`
`
`
`
`30 are passed directly to flow store 32. Given the preceding,
`
`
`
`
`
`
`
`
`
`one skilled in the art should appreciate that access to flow
`
`
`
`
`
`
`
`store 32 can be achieved by SQL queries, enabling network
`
`
`
`
`
`
`
`administrators to automate the configuration process or
`
`
`
`
`
`
`
`integrate report generation. As introduced above,
`in the
`
`
`
`
`
`
`
`
`preferred embodiment, flow store 32 also stores what is
`
`
`
`
`
`
`
`
`
`referred to in this documentas a “rule set” (or “rule sets”
`
`
`
`
`
`
`
`
`whenplural), whichis initially provided to the flow store 32
`
`Splunk Inc.
`
`Exhibit1009
`
`Page6é
`
`Splunk Inc. Exhibit 1009 Page 6
`
`

`

`
`
`US 2004/0090923 Al
`
`
`May 13, 2004
`
`
`
`
`
`
`
`
`
`
`from console 30 as part of the administration function and
`
`
`
`
`
`
`
`which is also thereby conveyed to meter 36. As shown by
`
`
`
`
`
`
`
`
`
`example below, each rule set specifies one or morecriteria
`
`
`
`
`
`
`
`against which meter 36 evaluates each incoming packet to
`
`
`
`
`
`
`
`determine if the criteria are satisfied. Additionally, in one
`
`
`
`
`
`
`
`
`
`embodiment, flow store 32 may store the packet arrival
`
`
`
`
`
`
`information for those criteria-satisfying packets in a moni-
`
`
`
`
`
`
`
`
`tored flow so that such information may be evaluated,
`
`
`
`
`
`
`
`including the determination of IDC information and possible
`
`
`
`
`
`
`
`
`responses thereto, by console 30. Moreover and as also
`
`
`
`
`
`
`
`
`discussed below, flow store 32 may store numerousdifferent
`
`
`
`
`
`
`sets of packet arrival information, each corresponding to a
`
`
`
`
`
`
`
`
`different set of flow criteria, that is, corresponding to one of
`
`
`
`
`
`
`
`
`the different specified rule sets. The stored information is
`
`
`
`
`
`
`
`therefore accessible by console 30 and permits other analy-
`
`
`
`
`
`
`
`ses of the flow information so as to provide information and
`
`
`
`
`
`
`
`
`
`reports that are useful for network engineering and manage-
`
`
`ment purposes.
`
`
`
`
`
`
`
`[0021] Continuing with FIG.2, recorder layer 34 provides
`
`
`
`
`
`
`
`
`
`an interface between flow store 32 and a meter 36 (or
`
`
`
`
`
`
`
`
`meters) in use and coupled to the network. Generally, the
`
`
`
`
`
`
`applications of recorder layer 34 can be separated into two
`
`
`
`
`
`
`
`categories: manager applications and meter reading appli-
`
`
`
`
`
`
`
`cations. Manager applications configure meter 36, based on
`
`
`
`
`
`
`
`
`information, including rules in one or morerule sets, in flow
`
`
`
`
`
`
`
`
`
`store 32. Meter reading applications permit the data col-
`
`
`
`
`
`lected and/or determined by meter 36 to be provided in a
`
`
`
`
`
`
`
`
`data format usable by flow store 32 and, indeed, recorder
`
`
`
`
`
`
`
`
`
`
`
`layer 34 facilitates the passage of that data into the flow
`
`
`
`
`
`
`
`
`database of flow store 32. Recorder layer 34 applications
`
`
`
`
`
`
`
`
`may pass information between flow store 32 and the network
`
`
`
`
`
`probes of meter 36 either synchronously or asynchronously.
`
`
`
`
`
`
`
`
`This gives network administrators the flexibility of either
`
`
`
`
`
`
`
`using real-time network flow meters (i.e. NeTraMet) or
`
`
`
`
`
`
`
`
`
`
`importing data from other network analysis tools that are not
`
`
`
`
`
`
`
`
`able to provide real-time data (e.g. Cisco NetFlow data).
`
`
`
`
`
`
`
`[0022]
`In the preferred embodiment, meter 36 is a Real-
`
`
`
`
`
`
`
`Time Traffic Flow Measurement (“RTFM”) meter which is
`
`
`
`
`
`
`
`
`a concept
`from the Internet Engineering Task Force
`
`
`
`
`
`
`
`(“IETF”). As known in the RTFM art, RTFM meters are
`
`
`
`
`
`
`
`previously knownto be used in systems for determining the
`
`
`
`
`
`
`
`service requested by IP packets that are passing through a
`
`
`
`
`
`
`
`network for purposes of collecting revenue, where such a
`
`
`
`
`
`
`
`service is identifiable by the transport port numberspecified
`
`
`
`
`
`
`
`
`
`in each IP packet. For example, RTFM meters are currently
`
`
`
`
`
`
`
`
`being considered for use in systems wherebyan Internet user
`
`
`
`
`
`
`
`
`
`is charged based on the service he or she is using on the
`
`
`
`
`
`
`
`
`Internet; for example, a different fee may be charged to the
`
`
`
`
`
`
`
`
`user for each different Internet service,
`including mail,
`
`
`
`
`
`
`
`
`video, phonecalls, and web browsing. However,as detailed
`
`
`
`
`
`
`
`
`in this document, the preferred embodiment implements the
`
`
`
`
`
`
`
`
`RTFM meter instead to analyze each packet and to deter-
`
`
`
`
`
`
`
`
`
`
`
`mine if the packet satisfies a rule in the rule set and, if so,
`
`
`
`
`
`
`
`
`to store sufficient packet arrival time corresponding to the
`
`
`
`
`
`
`
`
`
`defined interval t so that packet IDC may be determined and
`
`
`
`
`
`
`
`
`used as a basis to indicate, and respond to, network con-
`
`
`
`
`
`
`
`
`gestion. Thus, in real time, meter 36 physically probes the
`
`
`