`
`
`
`
`
`
`
`United States Patent
`(12)
`US 7,313,100 B1
`(10) Patent No.:
`
`
`
`
`
`
`
`
`Turneret al.
`(45) Date of Patent:
`Dec. 25, 2007
`
`
`
`
`US007313100B1
`
`
`
`
`
`
`
`
`
`
`Jr.
`6,182,146 Bl
`1/2001 Graham-Cumming,
`
`
`
`
`
`
`
`6,321,338 B1L*
`11/2001 Porras et al. oe. 726/25
`
`
`
`
`
`(54) NETWORK DEVICE HAVING ACCOUNTING
`
`
`SERVICE CARD
`
`
`
`
`(75)
`
`nventors:
`
`I
`
`
`
`
`
`
`Park,
`Menlo
`Stephen
`urner,
`Menlo Park, CA
`Stephen W Ti
`
`
`
`
`(US); Hsien-Chung Woo, Fremont, CA
`
`
`
`
`
`(US); Sanjay Kalra, San Jose, CA
`
`
`
`
`
`(US); Truman Joe, Mountain View, CA
`
`
`
`
`
`(US); Wendy R Cartee, Los Altos, CA
`(US)
`
`
`
`
`
`ssignee:
`(73) Assig
`
`
`
`
`
`Juniper Networks,
`Juniper N
`ks,
`
`
`CA (US)
`
`
`I
`
`Inc.,
`
`
`
`
`Sunnyvale,
`Sunnyval
`
`
`
`
`(*) Notice:
`
`
`
`
`
`
`
`
`
`Subject to any disclaimer, the term of this
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`
`U.S.C. 154(b) by 1034 days.
`
`
`
`
`(21) Appl. No.: 10/228,150
`
`
`
`
`Filed:
`Aug. 26, 2002
`(22)
`
`
`
`
`Int. Cl.
`
`HOAL 12/26
`
`
`
`(51)
`
`
`
`(2006.01)
`
`
`
`
`
`(Continued)
`
`
`FOREIGN PATENT DOCUMENTS
`
`
`
`WO 9836532 Al *
`8/1998
`WO 2084920 A2 * 10/2002
`
`
`
`
`
`
`
`wo
`WO
`
`
`
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`
`
`
`
`Weaver, A.C. et al., “A Real-Time Monitor for Token Ring Net-
`
`
`
`
`
`works,” Military Communications Conference, 1989. MILCOM
`
`
`
`
`
`
`°89. Oct. 1989. vol. 3. pp. 794-798."
`
`
`
`
`
`
`(Continued)
`
`
`
`
`Primary Examiner—Chi Pham
`Assistant Examiner—Donald L Mills
`
`
`
`
`
`
`
`
`
`(74) Attorney, Agent, or Firm—Shumaker & Sieffert P.A.
`
`
`
`(57)
`
`ABSTRACT
`
`
`
`
`
`
`
`
`
`(52) U.S. Ch wees 370/253; 370/244; 370/252;
`370/392
`
`
`
`
`.
`.
`.
`
`
`
`
`
`(58) Field of Classification Search ................ 370/235,
`
`
`
`
`370/242—244, 250, 252, 253, 389, 392, 396,
`
`
`
`
`. 370/469, 471, 709/223, 224, 229
`
`
`
`
`
`
`
`See application file for complete search history.
`
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`(56)
`
`
`
`
`
`
`
`
`
`
`
`6/1976 Requaetal.
`3,962,681 A
`
`
`
`
`6/1977 Jennyet al.
`4,032,899 A
`
`
`
`
`7/1986 Everett, Jr.
`4,600,319 A
`
`
`
`
`4/1995 Finlay et al.
`5,408,539 A
`
`
`
`
`
`2/1996 Macera et al... 709/249
`5,490,252 A *
`
`
`
`
`
`4/1996 Dobbinsetal.
`5,509,123 A
`
`
`
`
`5,568,471 A * 10/1996 Hershey et al.
`
`
`
`6,011,795 A
`1/2000 Vargheseet al.
`
`
`
`
`6,018,765 A
`1/2000 Duranaet al.
`
`
`
`
`
`6,148,335 A *
`11/2000 Haggard et al... 709/224
`
`
`........00.. 370/245
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A network device integrates accounting functionality for
`
`
`
`
`
`
`
`generation of flow statistics with packet intercept function-
`
`
`
`
`
`
`
`ality to provide a comprehensivetraffic analysis environ-
`
`
`
`
`
`
`
`ment. The device comprises a set of network interface cards
`
`
`
`
`
`
`
`
`to receive packets from a network, and a set of accounting
`
`
`
`
`
`
`
`
`
`service cards to calculate flow statistics for the packets. The
`
`
`
`
`
`
`
`
`device further comprises a control unit to receive the net-
`
`
`
`
`
`
`
`
`
`work packets from the interface cards and distribute the
`
`
`
`
`
`
`
`
`packets to the set of accounting service cards. The account-
`
`
`
`
`
`
`
`
`ing service card comprises an interface for insertion within
`
`
`
`
`
`
`
`
`a slot of a network device. Accounting service cards may be
`
`
`
`
`
`
`
`
`added to easily scale the network device to support higher
`
`
`
`
`
`
`
`bandwidth communication links, such as OC-3, OC-12,
`
`
`
`
`
`
`
`
`OC048 and higher rate links. Additional accounting service
`
`
`
`
`
`
`
`cards may be used for purposes of redundancy to support
`
`
`
`
`
`
`continuous, uninterrupted packet processing and accounting
`in the event of a card failure.
`
`
`
`
`
`
`
`
`
`24 Claims, 9 Drawing Sheets
`
`
`
`PACKET PACKET
`STREAM STREAM
`
`
`A
`B
`
`
`
`
`
`
`
`
`
`ACCOUNTING
`sc
`
`cd
`
`
`
`
`
`
`
`
`
`
`
`TUNNEL
`
`sc40
`
`
`coy
`
`
`
`K—
`
`6
`
`FLOW
`RECORDS
`
`
`SAMPLED
`TRAFFIC
`
`
`
`
`Splunk Inc.
`
`Exhibit 1023
`
`Page 1
`
`VB
`ew
`CONTROL
`82“,
`UNIT
`
`
`ca|ENCRYPTION
`3
`sc
`
`8,
`3B
`
`
`Splunk Inc. Exhibit 1023 Page 1
`
`

`

`
`
`US 7,313,100 B1
`
`Page 2
`
`OTHER PUBLICATIONS
`
`
`
`
`
`
`
`
`
`Dini, P. et al., “Performance Evaluation for Distributed System
`
`Bl
`5/2002
`
`
`
`
`6,392,996
`Hjalmtysson
`
`
`
`
`
`Components,” Proceedings of IEEE Second International Workshop
`Bl
`Wexleretal.
`12/2002
`
`
`
`
`
`
`6,499,088
`
`
`
`
`
`
`
`on Systems Management. Jun. 1996. pp. 20-29.*
`Saito Leeeeeeececcccenrenneee 370/252
`5/2003
`B1*
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`6,563,796
`Integrated Services Adapter, 2000, Cisco Systems, Data Sheet, pp.
`
`
`Uzun
`7/2003
`Bl
`http://www.cisco.com/warp/public/cc/pd/ifaa/svaa/iasvaa/
`1-6,
`
`
`
`
`6,590,898
`
`
`prodlit/ism2__ds.pdf.
`Aukiaetal.
`Bl
`7/2003
`
`
`
`
`6,594,268
`
`
`
`
`
`“The CAIDA WebSite,” www.caida.org/, 2000.
`7/2003
`Bl
`Kloth
`
`
`
`
`6,598,034
`
`
`
`
`“About Endace,” www.endace.com/, 2000.
`5/2004
`Bl
`
`
`
`
`
`
`
`
`
`6,735,201
`Mahajan etal.
`“Cisco IOS NetFlow,” www.cisco.com/warp/public/732/Tech/nmp/
`
`
`Farrell et al.
`6/2004
`Bl
`netflow/index.shtml, 2002.
`
`
`
`
`
`
`6,751,663
`
`
`
`
`
`
`
`
`U.S. Appl. No. 10/188,567, entitled “Adaptive Network Flow
`11/2004
`Bl
`
`
`
`
`
`
`6,826,713
`Beesley et al.
`
`
`
`
`
`
`
`Analysis”, filed Jul. 2, 2002, Scott Mackie.
`
`B2*—1/2006 Jones et al. wee 707/202
`
`
`
`
`
`
`
`6,983,294
`
`
`
`
`
`
`
`
`U.S. Appl. No. 10/228,132, entitled “Adaptive Network Router’,
`
`B2*—1/2006 Luke et al. we. 709/229
`
`
`
`
`
`
`
`
`
`
`
`
`
`6,985,956
`filed Aug. 26, 2002, Wooetal.
`
`
`
`
`
`
`
`
`
`
`B2
`9/2006
`
`
`U.S. Appl. No. 10/228,114, entitled “Network Router Having Inte-
`
`
`
`
`7,114,008
`Jungck etal.
`
`
`
`
`
`
`
`
`
`grated Flow Accounting and Packet Interception”, filed Aug. 26,
`
`10/2002
`2002/014 1343
`Al
`
`
`
`
`Bays
`
`
`
`2002, Wooetal.
`1/2003
`Bullard
`2003/0005 145
`Al
`
`
`
`
`
`
`
`
`
`
`
`U.S. Appl. No. 10/241,785, entitled “Rate-Controlled Transmission
`
`
`
`
`
`
`
`
`
`
`6/2003
`McCollom etal.
`2003/0120769
`Al
`
`
`
`
`
`of Traffic Flow Information”, filed Sep. 10, 2002, Sandeep Jain.
`11/2003
`Kan et al.
`2003/02 14913
`Al
`
`
`
`
`
`
`
`
`
`* cited by examiner
`
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Splunk Inc.
`
`Exhibit 1023
`
`Page 2
`
`Splunk Inc. Exhibit 1023 Page 2
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`
`Sheet 1 of 9
`
`
`
`US 7,313,100 B1
`
`
`
`
`
`
`NETWORK
`
`
`MONITOR
`
`
`
`REAL-TIME
`
`
`
`ACCOUNTING
`
`
`PACKET
`
`
`SERVER
`
`
`ANALYZER
`
`
`10
`
`
`
`
`12
`
`
`
`
`
`
`
`
`
`
`
`Splunk Inc.
`
`Exhibit1023
`
`Page 3
`
`Splunk Inc. Exhibit 1023 Page 3
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 2 of 9
`
`
`
`US 7,313,100 B1
`
`YOLINOWMYOMLAN
`
`yal
`
`OE
`
`ONILNNODOV
`
`
`
`Sa1NdOW
`
`
`
`ci
`
`
`
`OULNOD
`
`
`
`
`
`
`
`éOld
`
`\
`
`/
`
`
`
`
`
`
`
`
`
`\9J>~~
`
`—_WYOMLEN_—
`
`
`
`\voc|\yaLnow||/
`
`“
`
`“
`
`~
`
`Splunk Inc.
`
`Exhibit 1023
`
`Page 4
`
`Splunk Inc. Exhibit 1023 Page 4
`
`
`
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 3 of 9
`
`
`
`US 7,313,100 B1
`
`
`
`
`NETWORK MONITOR
`
`
`
`4
`
`
`Lf
`
`
`21A~
`
`
`21B
`
`
`
`IFC
`
`34
`
`fh
`
`
`
`14
`
`
`
`16
`
`
`
`40
`
`
`ft
`RECOUNTING Se
`
`
`
`
`
`CONTROL UNIT
`
`42
`
`
`
`
`
`ACCOUNTING
`
`Sc
`
`36
`ENCRYPTION
`
`
`
`38
`
`
`
`TUNNNEL SC
`
`
`
`
`
`FIG. 3
`
`Splunk Inc.
`
`Exhibit1023
`
` Page5
`
`Splunk Inc. Exhibit 1023 Page 5
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 4 of 9
`
`
`
`US 7,313,100 B1
`
`
`
`PACKET PACKET
`
`
`STREAM STREAM
`
`
`A
`
`
`
`
`
`ACCOUNTING
`Sc
`
`36
`
`
`A
`<——
`“.e
`
`50
`
`;
`arn
`‘ae
`
`
`
`TUNNEL
`St:
`
`40
`
`~
`
`
`
`= a
`
`CONTROL
`52
`UNIT
`
`42. ——
`
`<a ENCRYPTION
`
`8 38
`
`\62
`
`
`
`B -
`
`
`FLOW
`RECORDS
`
`
`
`
`SAMPLED
`
`TRAFFIC
`
`
`FIG. 4
`
`Splunk Inc.
`
`Exhibit1023
`
` Page6é
`
`Splunk Inc. Exhibit 1023 Page 6
`
`

`

`CARD
`
`
`
`
`
`
`
`
`
`
`
`ACCOUNTING
`
`
`UNIT
`INTERFACE
`
`
`70
`i2
`
`
`68
`
`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 5 of 9
`
`
`
`US 7,313,100 B1
`
`
`
`
`
`
`
` ACCOUNTING SERVICE
`
`
`
`
`66
`
`
`
`
`
`
`
`
`
`
`FIG. 5
`
`Splunk Inc.
`
`Exhibit1023
`
`Page 7
`
`Splunk Inc. Exhibit 1023 Page 7
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 6 of 9
`
`
`
`US 7,313,100 B1
`
`
`
`08
`
`
`
`LINNTOYLNOD
`
`
`
`aYdalnod
`
`9‘SIs
`
`
`
`
`
`NOILVIWHOANINOILWAMOINI
`
`
`
`
`
`ONILNOYONIGUYMYOS
`
`YaL1d—8os636ONLLNNODOV
`
`—9S66NOLLdAMONA
`
`ANIONONILNOY
`
`ONIGUYMYOS
`
`28
`
`OallcOh
`
`¥6
`
`6ANISNA
`
`
`
`ONILNNODDV
`
`
`
`os
`
`98
`
`
`
`OSTANNNNL
`
`88
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Splunk Inc.
`
`Exhibit 1023
`
`Page 8
`
`Splunk Inc. Exhibit 1023 Page 8
`
`
`
`
`
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 7 of 9
`
`
`
`US 7,313,100 B1
`
`
` ACCOUNTING SERVICE
`
`
`
`
`
`
`
`
`112
`
`
`
`
`
`
`
`CARD
` 114
`
`
`
`ACCOUNTING
`
`UNIT
`
`
`111
`
`
`
`
`
`
`
`
`FIG. 7
`
`Splunk Inc.
`
`Exhibit1023
`
` Page9
`
`Splunk Inc. Exhibit 1023 Page 9
`
`

`

`DUPLICATE PACKET STREAMS
`SAMPLED PACKET FLOWS
`ANALYZE PACKET FLOWS
`
`
`
`
`YES
`
`
`
`
`
`
`
`
`DISTRIBUTE PACKETS OF FIRST
`
`
`
`
`STREAM TO ACCOUNTING
`
`
`
`CARDS FOR CALCULATION OF
`
`
`FLOW RECORDS
`
`
`i 124
`
`
`
`
`RECEIVE FLOW RECORDS AND
`
`
`
`ORIGINAL PACKETS FROM
`
`
`
`
`
`
`
`
`
`
`
`ACCOUNTING CARDS
`INFORMATION
`ANALYZE FLOW RECORDS
`
`126
`
`
`FORWARD PACKETS
`
`ACCORDING TO FORWARDING
`
`
`
`
`
`
`128
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`
`
`
`
`Dec. 25, 2007
`
`
`
`
`Sheet 8 of 9
`
`
`
`US 7,313,100 B1
`
`ir 120
`
`
`
`
`RECEIVE NETWORK PACKETS AND
`
`
`
`
`GENERATEFIRST AND SECOND
`
`
`
`
`
`
`
`
`
`
`
`FILTER SECOND PACKET
`
`
`STREAM TO PRODUCE
`
`
`
`
`
`136
`
`
`
`
`
`
`
`
`
`138
`
`
`
`NO
`
`
`
`
`NETWORK
`
`
`CONDITION?
`
`
`440
`
`INFORMATION
`NEIGHBORING ROUTERS
`
`
`UPDATE FORWARDING
`
`
`
`
`142
`
`
`
`
`
`
`
`FORWARD NETWORKATTACK
`
`INFORMATION TO
`
`
`
`
`
`
`
`FIG. 8
`
`Splunk Inc.
`
`Exhibit1023
`
`Page 10
`
`
`
`
`
`130
`
`
`SUSPICIOUS
`
`FLOWS?
`
`
`
`
`
`
`
`
`SUSPICIOUS FLOWS
`
`
`
`
`UPDATEFILTER TO INCLUDE
`
`
`
`
`
`
`
`
`
`
`Splunk Inc. Exhibit 1023 Page 10
`
`

`

`
`U.S. Patent
`
`
`
`Dec. 25, 2007
`
`
`
`
`
`Sheet 9 of 9
`
`
`
`
`
`
`
`
`
`
`US 7,313,100 B1
`
`
`FIG. 9
`
`Splunk Inc.
`
`Exhibit1023
`
`Page 11
`
`Splunk Inc. Exhibit 1023 Page 11
`
`

`

`
`
`US 7,313,100 B1
`
`
`1
`NETWORK DEVICE HAVING ACCOUNTING
`
`
`
`SERVICE CARD
`
`
`
`
`
`TECHNICAL FIELD
`
`
`
`
`
`
`
`
`
`
`
`The invention relates to computer networks and, more
`
`
`
`
`
`
`
`particularly, to techniques for analyzing traffic flow within
`
`
`computer networks.
`
`
`
`
`BACKGROUND
`
`
`
`20
`
`30
`
`35
`
`
`
`
`
`
`
`A computer network is a collection of interconnected
`
`
`
`
`
`
`
`
`computing devices that can exchange data and share
`
`
`
`
`
`
`
`resources. In a packet-based network, such as the Internet,
`
`
`
`
`
`
`
`the computing devices communicate data by dividing the
`
`
`
`
`
`
`
`
`
`data into small blocks called packets, which are individually
`routed across the network from a source device to a desti-
`
`
`
`
`
`
`
`
`nation device. The destination device extracts the data from
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the packets and assembles the data into its original form.
`
`
`
`
`
`
`
`
`
`Dividing the data into packets enables the source device to
`
`
`
`
`
`
`
`
`
`resend only those individual packets that may be lost during
`transmission.
`
`
`
`
`
`
`
`The packets are communicated according to a communi-
`
`
`
`
`
`
`
`
`
`25
`cation protocol
`that defines the format of the packet. A
`
`
`
`
`
`
`
`typical packet, for example,
`includes a header carrying
`
`
`
`
`
`
`
`source and destination information, as well as a payload that
`carries the actual data. The de facto standard for communi-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`cation in conventional packet-based networks, including the
`
`
`
`
`
`Internet, is the Internet Protocol (IP).
`
`
`
`
`
`
`
`
`A system administrator or other user often makes use of
`
`
`
`
`
`
`
`
`a network analyzer to monitor network traffic and debug
`
`
`
`
`
`
`network problems. In general, a network analyzer is a tool
`
`
`
`
`
`
`
`
`
`
`that captures data from a network and presents the data to the
`
`
`
`
`
`
`
`
`user. The network analyzer typically allows the user to
`
`
`
`
`
`
`
`
`
`browse the captured data, and view summary and detail
`
`
`
`
`
`
`
`
`
`information for each packet. Accordingly, the user can view
`
`
`
`
`
`
`
`
`the networktrafic flowing between devices on the network.
`
`
`
`
`
`
`
`
`The information collected during traffic flow analysis may
`
`
`
`
`
`
`
`
`be used for network planning, traffic engineering, network
`
`
`
`
`
`
`
`
`monitoring, usage-based billing and the like. Many conven-
`
`
`
`
`
`
`
`tional network analyzers, such as NetFlow, NeTraMet and
`
`
`
`
`
`
`
`FlowScan, use software applications to collect traffic flow
`information.
`
`
`
`
`
`
`
`
`
`The analyzers typically monitor and collect packets hav-
`
`
`
`
`
`
`
`ing routing information that matchescriteria specified by the
`
`
`
`
`
`
`
`system administrator. The system administrator may specify,
`
`
`
`
`
`
`
`
`for example, source and destination Internet Protocol (IP)
`
`
`
`
`
`
`
`addresses, source and destination port numbers, protocol
`
`
`
`
`
`
`
`
`type, type of service (ToS) and input interface information.
`
`
`
`
`
`
`
`
`The analyzers typically collect packets matching the speci-
`
`
`
`
`
`
`
`
`fied criteria, and construct flow analysis diagrams. Conven-
`
`
`
`
`
`
`
`
`tional network analyzers often make use of sampling tech-
`
`
`
`
`
`
`
`
`niques to selectively sample the packets, and present a
`
`
`
`
`
`
`
`statistically generated view ofthetraflic within the network.
`
`
`
`
`
`
`
`Consequently, the statistics generated by the network ana-
`
`
`
`
`
`
`
`
`lyzer may not only be limited to specified flows, but may be
`
`
`relatively inaccurate.
`
`40
`
`45
`
`50
`
`55
`
`
`2
`
`
`
`
`
`
`
`
`In one embodiment, an apparatus comprises a set of
`
`
`
`
`
`
`
`
`interface cards to receive packets from a network, and a set
`
`
`
`
`
`
`
`
`
`of accounting service cards to calculate flow statistics for the
`
`
`
`
`
`
`
`packets. The apparatus further comprises a control unit to
`
`
`
`
`
`
`
`
`
`receive the packets from the interface cards and distribute
`
`
`
`
`
`
`
`the packets to the set of accounting service cards.
`
`
`
`
`
`
`
`In one embodiment, an accounting service card comprises
`
`
`
`
`
`
`
`
`an interface for insertion within a slot of a network device,
`
`
`
`
`
`
`
`
`and an accounting unit to receive packets from the network
`
`
`
`
`
`
`
`
`
`device via the interface. The accounting unit calculates flow
`
`
`
`
`
`statistics based on the network packets.
`
`
`
`
`
`
`In another embodiment, a method comprises receiving
`
`
`
`
`
`
`
`packets from a network via an interface card of a network
`
`
`
`
`
`
`
`
`device, and distributing the packets to a set of accounting
`service cards of the network device. The method further
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`comprises calculating with the accounting service cards flow
`
`
`
`
`statistics for the packets.
`
`
`
`
`
`
`
`In another embodiment, a method for computing flow
`
`
`
`
`
`
`
`
`statistics within an accounting service card of a network
`
`
`
`
`
`
`
`device comprises receiving packets from a control unit of a
`
`
`
`
`
`
`
`
`network router via an interface, and calculating flow statis-
`
`
`
`
`
`
`
`
`
`tics for the packets. The method further comprises output-
`
`
`
`
`
`
`
`
`
`ting a packet stream carrying the flow statistics and the
`
`
`
`
`
`
`
`
`received packets to the control unit for routing in accordance
`
`
`
`
`
`
`with routing information for the network.
`
`
`
`
`
`
`
`
`The techniques may provide one or more advantages. For
`
`
`
`
`
`
`
`example, according to the principles of the invention, mul-
`
`
`
`
`
`
`
`
`tiple accounting service cards may be addedto easily scale
`
`
`
`
`
`
`
`the network monitor to support monitoring and accounting
`
`
`
`
`
`
`
`for higher bandwidth communication links. Depending upon
`
`
`
`
`
`
`
`
`processing power, two accounting service cards may be used
`
`
`
`
`
`
`
`to provide accounting for a single OC-3 communication
`
`
`
`
`
`
`
`
`
`link, while four cards and sixteen cards may be used to
`
`
`
`
`
`
`
`monitor OC-12 and OC-48 links, respectively. As another
`
`
`
`
`
`
`
`example, eight accounting service cards may be used to
`
`
`
`
`
`
`
`monitor four OC-3 links. Additional accounting service
`
`
`
`
`
`
`
`cards may be used for purposes of redundancy to support
`
`
`
`
`
`
`continuous, uninterrupted packet processing and accounting
`in the event of card failure.
`
`
`
`
`
`
`
`
`
`
`
`
`intercept
`Consequently,
`the flow analysis and packet
`
`
`
`
`
`
`
`features may be readily integrated within a router for a
`
`
`
`
`
`
`
`
`packet-based network. The router may, for example, operate
`
`
`
`
`
`
`
`
`
`as a core router within the Internet to route packets received
`
`
`
`
`
`
`
`
`from high data rate communication links, such as OC-3,
`
`
`
`
`
`
`
`
`OC-12, OC-48, and greater communication links. The router
`
`
`
`
`
`
`
`may integrate accounting functionality to generate flow
`
`
`
`
`
`
`
`records for routed packets, as well as intercept features to
`
`
`
`
`
`
`
`
`
`capture packets for select packet flows. In this manner, the
`
`
`
`
`
`
`
`
`router can adjust routing functions based on the generated
`
`
`
`
`
`
`
`flow records and intercepted packets, thereby dynamically
`
`
`
`
`
`
`
`reacting to network events, such as Denial of Service (DOS)
`
`
`
`
`
`
`attacks and other network security violations.
`The details of one or more embodiments of the invention
`
`
`
`
`
`
`
`
`
`
`
`
`are set forth in the accompanying drawings and the descrip-
`
`
`
`
`
`
`
`tion below. Other features, objects, and advantages of the
`
`
`
`
`
`
`
`invention will be apparent from the description and draw-
`
`
`
`
`
`ings, and from the claims.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SUMMARY
`
`
`
`60
`
`
`
`
`
`
`
`
`
`
`In general, the invention is directed to techniques for
`
`
`
`
`
`
`
`monitoring and analyzing traffic flows within a network. A
`
`
`
`
`
`
`
`network monitor, in accordance with the principles of the
`65
`
`
`
`
`
`
`invention, integrates accounting functionality for generation
`
`
`
`
`
`
`
`
`of flow statistics with packet intercept funtionality to pro-
`
`
`
`
`
`vide a comprehensive traffic analysis environment.
`
`
`
`BRIEF DESCRIPTION OF DRAWINGS
`
`
`
`
`
`
`
`
`
`
`
`FIG.1 illustrates an exemplary system in which a network
`
`
`
`
`
`
`monitor integrates accounting functionality for generation of
`
`
`
`
`
`
`
`
`flow records along with packet intercept functionality to
`
`
`
`
`
`provide a comprehensive traffic analysis environment
`in
`
`
`
`
`
`
`accordance with the principles of the invention.
`SplunkInc.
`Exhibit 1023
`
`Page 12
`
`Splunk Inc. Exhibit 1023 Page 12
`
`

`

`
`3
`
`
`
`
`
`
`FIG.2 is a block diagram illustrating an example embodi-
`
`
`
`
`
`
`
`ment of a network monitor consistent with the principles of
`the invention.
`
`
`
`
`
`
`
`
`FIG.3 is a block diagram illustrating another exemplary
`embodiment of a network monitor in further detail.
`
`
`
`
`
`
`
`
`
`
`
`
`FIG.4 is a block diagram illustrating the flow of packets
`
`
`
`
`
`
`through the various components of a network monitor in
`
`
`
`
`
`
`accordance with the principles of the invention.
`
`
`
`
`
`
`FIG.5 is a block diagram illustrating an example embodi-
`
`
`
`
`
`
`
`ment of an accounting service card in accordance with the
`
`
`
`principles of the invention.
`
`
`
`
`
`
`FIG.6 is a block diagram illustrating an example embodi-
`
`
`
`
`
`
`
`ment of a router that incorporates accounting and intercept
`
`functionality.
`
`
`
`
`
`FIG.7 is a block diagram illustrating another embodiment
`
`
`
`
`of an accounting service card.
`
`
`
`
`
`
`FIG.8 is a flowchart illustrating operation of router that
`
`
`
`
`
`
`
`
`integrates traffic analysis and intercept features with routing
`
`
`
`
`
`
`functionality to dynamically react to network events, such as
`
`
`
`
`
`
`
`
`Denial of Service (DOS) attacks and other network security
`violations.
`
`
`
`
`
`
`FIG. 9 is a schematic diagram illustrating an exemplary
`
`
`
`
`
`
`
`embodimentof a network router that integrates traffic analy-
`
`
`
`
`
`
`
`sis and intercept features with routing functionality.
`
`
`
`DETAILED DESCRIPTION
`
`
`
`
`
`
`
`
`
`
`FIG. 1 illustrates an exemplary system 2 in which a
`
`
`
`
`
`
`network monitor 4 integrates accounting functionality for
`
`
`
`
`
`
`
`generation of flow records with packet intercept function-
`
`
`
`
`
`
`
`ality to provide a comprehensive traffic analysis environ-
`
`
`
`
`
`
`
`
`ment in accordance with the principles of the invention.
`
`
`
`
`
`
`Network monitor 4 is coupled to network 6 for monitoring
`
`
`
`
`
`
`networktraffic. Network 6 may be formed by an intercon-
`
`
`
`
`
`
`nected group of autonomous systems, each representing an
`
`
`
`
`
`
`independent administrative domain having a variety of net-
`
`
`
`
`
`worked resources capable of packet-based communication.
`
`
`
`
`
`
`
`For example, network 6 may include servers, workstations,
`
`
`
`
`
`
`
`
`network printers and fax machines, gateways, routers, and
`
`
`
`
`
`
`
`
`the like. Each autonomous system within network 6 typi-
`
`
`
`
`
`
`
`
`
`cally includes at least one router for sharing routing infor-
`
`
`
`
`
`
`
`
`
`mation with, and forwarding packets to, the other autono-
`
`
`
`
`
`mous systems via communication links.
`
`
`
`
`
`
`
`The term “packet” is used herein to generally describe a
`unit of data communicated between resources in conform-
`
`
`
`
`
`
`
`
`
`
`
`
`
`ance with a communication protocol. The principles of the
`
`
`
`
`
`
`invention may bereadily applied to a variety of protocols,
`
`
`
`
`
`
`
`
`such as the Transmission Control Protocol (TCP), the User
`
`
`
`
`
`
`
`
`Datagram Protocol (UDP), the Internet Protocol (IP), Asyn-
`
`
`
`
`
`
`
`
`chronous Transfer Mode, Frame Relay, and the like. Accord-
`
`
`
`
`
`
`
`
`ingly, “packet” is used to encompass any such unit of data,
`
`
`
`
`
`
`
`
`
`and may be interchanged with the term “cell,” or other
`
`
`
`
`
`
`
`
`similar terms used in such protocols to describe a unit of data
`communicated between resources within the network.
`
`
`
`
`
`
`
`
`
`
`
`
`
`As described, network monitor 4 includes one or more
`
`
`
`
`
`
`
`
`accounting modules that generate accurate flow statistics for
`
`
`
`
`
`
`
`traffic within network 6. More specifically, network monitor
`
`
`
`
`
`
`
`
`
`4 captures packets from one or more links within network 6,
`
`
`
`
`
`
`
`
`
`
`and can generate flow statistics for each packet flow within
`
`
`
`
`
`
`
`
`the link. As network monitor 4 receives packets,
`the
`
`
`
`
`
`
`
`accounting modules associate the network packets with
`
`
`
`
`
`
`
`
`
`respective packet flows, and update the statistics for the
`
`
`
`
`
`
`
`
`packets flows. For example, the accounting modules may
`
`
`
`
`
`
`
`
`maintain an accurate packet count, byte count, source IP
`
`
`
`
`
`
`
`
`address, destination IP address, next hop IP address, input
`
`
`
`
`
`
`total
`interface information, output
`interface information,
`
`
`
`
`
`
`
`
`
`
`octets sent,
`flow start
`flow end time, source and
`time,
`
`
`
`US 7,313,100 B1
`
`
`4
`
`
`
`
`
`
`
`
`
`destination port numbers, TCP flags, IP type of service,
`
`
`
`
`
`
`
`
`originating AS, source address prefix mask bits, destination
`
`
`
`
`
`
`
`
`
`
`
`address prefix mask bits, and the like, for each packet flow.
`
`
`
`
`
`
`The accounting modules provide real-time accounting
`
`
`
`
`
`
`
`capabilities for maintaining accurate flow statistics for all of
`
`
`
`
`
`
`
`the packets received by network monitor4. In particular, as
`
`
`
`
`
`
`
`
`described herein, the accounting modules can monitor and
`
`
`
`
`
`
`
`
`
`
`generate statistics for high traflic rates, even coretraffic rates
`
`
`
`
`
`
`
`
`of the Internet, including OC-3, OC-12, OC-48, and higher
`rates.
`
`
`
`
`
`
`
`
`Network monitor 4 outputs a stream of flow records 14
`
`
`
`
`
`
`
`
`
`that carry flow statistics for the captured packets. Network
`
`
`
`
`
`
`
`
`monitor 4 may, for example, output flow records 14 carrying
`
`
`
`
`
`
`
`
`accounting data for each flow, such as a numberof packets,
`
`
`
`
`
`
`
`
`
`a numberof bytes, a time of capturing a first packet for the
`
`
`
`
`
`
`
`
`
`flow, a time of capturing a most recent packet for the flow,
`
`
`
`
`
`
`
`an incoming interface, an outgoing interface, a source/
`
`
`
`
`
`destination network mask, a source/destination Autonomous
`
`
`
`
`
`
`
`
`
`System (AS) number, and the like. Accounting server 10
`
`
`
`
`
`
`
`
`receives flow records 14, and updates an accounting system
`
`
`
`
`
`
`
`
`based on the flow records for further detailed analysis.
`
`
`
`
`
`
`
`In addition, network monitor 4 provides intercept capa-
`
`
`
`
`
`
`
`bilities that allow a real-time packet analyzer 12 to monitor
`
`
`
`
`
`
`
`specific packet flows within network 4. Network monitor 4
`
`
`
`
`
`
`outputs a stream of packets 16 to real-time packet analyzer
`
`
`
`
`
`
`
`
`12 for further analysis. The stream of packets 16 comprises
`
`
`
`
`
`
`
`
`a subset of the packets captured from network 6. In particu-
`
`
`
`
`
`
`
`
`lar, network monitor 4 intercepts packets for one or more
`
`
`
`
`
`
`
`
`selected packet flows within network 4, and outputs the
`
`
`
`
`
`
`
`intercepted packets as a stream of packets 16. Packet ana-
`
`
`
`
`
`
`
`
`
`lyzer 12 receives the stream of packets 16, and analyzes the
`
`
`
`
`
`
`
`
`packets to identify any suspicious packet
`flows. For
`
`
`
`
`
`
`
`
`example, packet analyzer 12 may identify packet flows
`
`
`
`
`
`
`
`
`
`arising from Denial of Service (DOS) attacks and other
`
`
`
`network security violations.
`
`
`
`
`
`
`
`Asystem administrator may provide intercept information
`
`
`
`
`
`
`
`
`
`to network monitor 4 that specifies a set of packet flows for
`
`
`
`
`
`
`
`which to capture packets. The system administrator may
`
`
`
`
`
`
`
`
`provide the intercept information directly, e.g., via a key-
`
`
`
`
`
`
`
`board, mouse or other input mechanism, to control inter-
`
`
`
`
`
`
`
`ception of packet flows. In addition, an administrator may
`
`
`
`
`
`
`
`remotely provide the routing information to network moni-
`
`
`
`
`
`
`
`tor 4 via a remote managementprotocol. In this manner, the
`
`
`
`
`
`
`
`
`administrator may selectively define the packet flows, and
`
`
`
`
`
`
`
`
`
`packets within a given flow,that are intercepted for analysis.
`
`
`
`
`
`
`
`
`Network monitor 4 may also control the stream of inter-
`
`
`
`
`
`
`
`cepted packets 16 based on feedback from accounting server
`
`
`
`
`
`
`
`
`10. More specifically, accounting server 10 may perform
`
`
`
`
`
`
`
`
`preliminary traffic analysis based on the flow records 14
`
`
`
`
`
`
`
`
`received from network monitor 4, and providesfilter infor-
`
`
`
`
`
`
`
`mation 18 to the network monitor to control the interception
`
`
`
`
`
`
`
`and forwarding of packets flows to packet analyzer 12 for
`
`
`
`
`
`
`
`
`further analysis. In this manner, network monitor 4 inte-
`
`
`
`
`
`
`
`grates accounting functionality for generation of flow
`
`
`
`
`
`
`
`
`records 14 along with packet
`intercept functionality to
`
`
`
`
`
`provide a comprehensivetraflic analysis environment.
`
`
`
`
`
`
`Although illustrated as a stand-alone apparatus, the fea-
`
`
`
`
`
`
`
`tures of network monitor 4 may be integrated within a
`
`
`
`
`
`
`
`network device. For example, as described in detail below,
`
`
`
`
`
`
`
`
`the feature may be integrated within a router. Other network
`
`
`
`
`
`
`
`devices in which the features may be integrated include
`
`
`
`
`
`
`
`gateways, switches, servers, workstations, and the like.
`
`
`
`
`
`
`FIG. 2 is a block diagram illustrating in further detail an
`
`
`
`
`
`
`example embodiment of network monitor 4 coupled to
`
`
`
`
`
`
`communication links 24 of network 6. As illustrated, net-
`
`
`
`
`
`
`
`
`
`work6 includes routers 20A, 20B (“routers 20”) coupled via
`
`
`
`
`
`
`
`communication links 24. Routers 20 may comprise conven-
`SplunkInc.
`Exhibit 1023
`Page 13
`
`
`
`
`
`20
`
`25
`
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`Splunk Inc. Exhibit 1023 Page 13
`
`

`

`
`5
`
`
`
`
`
`
`
`tional routers that forward packets in accordance with a
`
`
`
`
`
`
`topology of network 6. Communication links 24 may com-
`
`
`
`
`
`
`
`links
`prise uni-directional optical
`for carrying packets
`
`
`
`
`
`
`
`
`between routers 20 at high data rates, such as OC-3, OC12,
`
`
`
`
`
`
`
`
`
`OC-48 andgreater rates. Optical splitters 25A, 25B (“optical
`
`
`
`
`
`
`
`splitters 25”) may be inserted within communication links
`
`
`
`
`
`
`
`
`24 to passively collect optical data transmitted and received
`between routers 20.
`
`
`
`
`
`
`
`
`
`
`
`Network monitor 4 includes two ports 26A, 26B for
`
`
`
`
`
`
`
`
`
`receiving the optical data 21A, 21B, respectively, and for-
`
`
`
`
`
`
`
`
`
`
`warding the data in digital form to control unit 28. As
`
`
`
`
`
`
`
`
`discussed in detail, control unit 28 merges the inbound data
`
`
`
`
`
`
`
`
`
`21A, 21B received from ports 26A, 26B, and digitally
`
`
`
`
`
`
`
`
`
`generates two identical packets streams 27A, 27B from the
`
`
`
`
`
`
`
`
`data. Control unit 28 applies filter 30 to packet stream 27A
`
`
`
`
`
`
`
`
`to selectively capture packet flows 16 for forwarding to
`
`
`
`
`
`
`
`
`packet analyzer 12 via output port 26C. In addition, control
`
`
`
`
`
`
`
`
`
`unit 28 distributes packets of the second stream 27B to
`
`
`
`
`
`
`accounting modules 32. Accounting modules 32 generate
`
`
`
`
`
`
`
`
`flow records 14 based on all of the packets of data stream
`
`
`
`
`
`
`
`
`
`27B,i.e., all of the packets received form optical splitters 25,
`
`
`
`
`
`
`
`
`
`and forward flow records 14 to accounting server 10 via
`
`
`
`output port 26D.
`
`
`
`
`
`
`
`Accounting modules 32 may buffer flow records 14 for a
`
`
`
`
`
`
`
`
`
`
`given packet flow until the flow “expires,” i.e., when the
`
`
`
`
`
`
`
`
`accounting modules 32 detect inactivity for the flow for a
`
`
`
`
`
`
`
`configurable period of time, e.g., 30 minutes. Accounting
`
`
`
`
`
`
`
`modules 32 may periodically output batches of flow records
`
`
`
`
`
`
`
`
`
`
`14 forall flows that have recently expired, e.g., every fifteen,
`
`
`
`
`
`
`
`
`
`thirty or sixty seconds. For packet flows that remain active
`
`
`
`
`
`
`
`for long durations, accounting modules 32 may be config-
`
`
`
`
`
`
`
`
`ured to automatically expire the packet flowsafter a defined
`
`
`
`
`
`
`
`
`duration, e.g., 30 or 60 minutes. Upon marking the active
`
`
`
`
`
`
`
`packet flow as expired, accounting modules 32 may output
`
`
`
`
`
`
`
`
`
`
`one or more flow records 14 for the packet flow, and may
`
`
`
`
`
`
`
`
`
`reset the statistics for the packet flow. Alternatively, account-
`
`
`
`
`
`
`
`
`
`ing modules may output flow records 114 withoutresetting
`
`
`
`
`
`
`
`the statistics for the active packet flow.
`
`
`
`
`
`
`FIG.3 is a block diagram illustrating another exemplary
`embodiment of a network monitor 4.
`In the illustrated
`
`
`
`
`
`
`
`
`
`
`
`
`embodiment, network monitor 4 includes a chassis 33 for
`
`
`
`
`
`
`
`
`housing control unit 42. Chassis 33 has a numberofslots
`
`
`
`
`
`
`
`
`(not shown) for receiving a set of cards, including interface
`
`
`
`
`
`
`
`cards (IFCs) 34, accounting service cards (ACCOUNTING
`
`
`
`
`
`SCs) 36, an encryption service card (ENCRYPTION SC) 38,
`
`
`
`
`
`
`
`
`and a tunnel service card (TUNNEL SC) 40. Each card may
`
`
`
`
`
`
`
`
`
`be inserted into a corresponding slot of chassis 33 for
`
`
`
`
`
`
`
`
`electrically coupling the card to control unit 42 via a bus,
`
`
`
`
`
`backplane, or other electrical communication mechanism.
`
`
`
`
`
`
`
`
`Interface cards 34 include ports for receiving inbound
`
`
`
`
`
`
`
`
`
`
`data 21 from optical splitters 25, and for outputting flow
`
`
`
`
`
`
`
`
`records 14 and intercepted packet flows 16. Accordingly,
`
`
`
`
`
`
`
`interface cards 34 include a numberofports (not shown) for
`
`
`
`
`coupling with communication links.
`
`
`
`
`
`
`
`
`Accounting service cards 36 each include one or more
`
`
`
`
`
`
`
`
`accounting modules that generate flow records based on
`
`
`
`
`
`
`
`
`packets received from control unit 42. Each accounting
`
`
`
`
`
`
`
`
`
`service card 36 may, for example,
`include one or more
`
`
`
`
`
`microprocessors, FPGAs, ASICs, or other components. As
`
`
`
`
`
`
`described, control unit 42 distributes packets to accounting
`
`
`
`
`
`
`
`
`
`service cards 36 for accounting and generation of flow
`
`
`
`
`
`
`
`
`records 14. In one embodiment, control] unit 42 distributes
`
`
`
`
`
`
`
`the packets of a common flow to a common accounting
`
`
`
`
`
`
`
`
`service card 36. I