(12) United States Patent
`Brebner
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 6,891,397 B1
`May 10, 2005
`
`USOO6891397 B1
`
`(54) GIGABIT ROUTER ON A SINGLE
`PROGRAMMABLE LOGIC DEVICE
`
`(75) Inventor: Gordon J. Brebner, Monte Sereno, CA
`(US)
`(73) Assignee: Xilinx, Inc., San Jose, CA (US)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 10/420,224
`(22) Filed:
`Apr. 21, 2003
`(51) Int. Cl. .............................................. H03K 19/177
`(52) U.S. Cl. .............................. 326/41; 326/38; 326/39
`(58) Field of Search ................................ 326/37-41, 47
`(56)
`References Cited
`U.S. PATENT DOCUMENTS
`
`6,275,491 B1 *
`6.279,045 B1 *
`
`8/2001 Prasad et al. ............... 370/389
`8/2001 Muthujumaraswathy et al. 710/
`5
`
`OTHER PUBLICATIONS
`http://www-cad.eecS.berkeley.edu/Respep/Research/hsc/;
`downloaded Apr. 18, 2003; pp. 1-6.
`N. Bergmann et al., “Reconfigurable Computing and Reac
`tive Systems'; Proceedings of the 7th Australasian Confer
`ence on Parallel and Real-Time Systems; Springer; Nov.
`29–30, 2000; pp. 171-180.
`F. Braun et al., “Reconfigurable Router Modules Using
`Network Protocol Wrappers'; 11th International Conference
`on Field Programmable Logic and Applications, Springer
`LNCS 2147; Aug. 27–29, 2001; pp. 254-263.
`G. Brebner; “A Virtual Hardware Operating System for the
`Xilinx XC6200”; 6th International Workshop on Field Pro
`grammable Logic and Applications, Springer LNCS 1142,
`Sep. 23–25, 1996; pp. 327–336.
`G. Brebner et al., “Chip-Based Reconfigurable Task Man
`agement'; 11th International Conference on Field Program
`mable Logic and Applications, Springer LNCS 2147; Aug.
`27–29, 2001; pp. 182-191.
`
`G. Brebner; "Single-Chip Gigabit Mixed-Version IP Router
`on Virtex-II Pro”; IEEE Symposium on FPGAs Custom
`Computing Machines; IEEE Computer Society Press; Apr.
`22-24, 2002; pp. 35-44.
`J. Burns et al., “A Dynamic Reconfiguration Run-Time
`System”; Proc. 5th Annual IEEE Symposium on FPGAs
`Custom Computing Machines; IEEE, Apr. 16-18, 1997, pp.
`66-75.
`O. Diessel et al., “Run-Time Compaction of FPGA
`Designs”;7th International Workshop on Field Program
`mable Logic and Applications; Springer LNCS 1304; Sep.
`1–3, 1997; pp. 131-140.
`H. ElGindy et al., “Task Rearrangement on Partially Recon
`figurable FPGAs with Restricted Buffer'; 10th International
`Workshop on Field Programmable Logic and Applications;
`Springer LNCS 1896; Aug. 27–28, 2000; pp. 379–388.
`H. Fallside et al., “Internet Connected FPL'; 10th Interna
`tional Workshop on Field Programmable Logic and Appli
`cations; Springer LNCS 1896; Aug. 27–30, 2000; pp. 48-57.
`D. Harel; “Statecharts: A Visual Formalism for Complex
`Systems';p Science of Computer Programming, 8; Jun.
`1987, pp. 231-274; downloaded from: http://www.wisdom.
`Weizmann.ac.il/-dharel/Scanned. PaperS/Statecharts.pdf.
`A. Silberschattz et al., “ Applied Operating System Con
`cepts”, 1st Edition; New York; published by John Wiley &
`Sons; Copyright 2000; Chapter 5: Threads; pp. 116-117.
`(Continued)
`
`Primary Examiner Anh Q. Tran
`(74) Attorney, Agent, or Firm W. Eric Webostad
`(57)
`ABSTRACT
`Apparatus for network and System on a Single program
`mable logic device is described. The programmable logic
`includes port modules. The port modules have configurable
`logic configured to proceSS communications for routing
`communications. The port modules are configured to the
`process communications for at least one of a plurality of
`protocols.
`
`45 Claims, 16 Drawing Sheets
`
`PL)
`100
`
`Configured
`Logic
`iOS
`
`Port Mus
`
`Port Mcxule
`101-4
`
`
`
`Configured
`Logic
`05
`
`
`
`Configured
`Logic
`105
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 1 of 29
`
`

`

`US 6,891,397 B1
`Page 2
`
`OTHER PUBLICATIONS
`
`G. Brebner; “Multithreading for Logic-Centric Systems';
`12th International Conference on Field Programmable Logic
`and Applications; Montepellier, France; Sep. 2-4, 2002;
`Springer LNCS 2438; pp. 5-14.
`R. Gilligan et al.; Internet Requests for Comments (RFC);
`RFC 2893 “Transition Mechanisms for IPv6 Hosts and
`Routers'; Aug. 2000; Accessible at http://www.rfc-editor.
`org/rfc.html.
`A. Conta; Internet Requests for Comments (RFC); RFC
`2473 “Generic Packet Tunneling in IPv6 Specification”;
`Dec. 19981. Accessible at http://www.rfe-editor.org/
`rfc.html.
`G. Tsirtsis; Internet Requests for Comments (RFC); RFC
`2766 “Network Address Translation-Protocol Translation
`(NAT-PT)”; Feb. 2000; Accessible at http://www.rfe-edi
`tor.org/rfc.html.
`E. Nordmark; Internet Requests for Comments (RFC); RFC
`2765 “Stateless IP/ICMP Translation Algorithm (SIIT)";
`Feb. 2000; Accessible at http://www.rfc-editor.org/rfc.html.
`R. Hinden et al.; Internet Requests for Comments (RFC);
`RFC 2374 “An Aggregatable Global Unicast Address For
`mat”; Jul. 1998; Accessible at http://www.rfe-editor.org/
`rfc.html.
`G. Brebner; "Highly Reconfigurable Communication Pro
`tocol Multiplexing Element for SCOPH'; Reconfigurable
`Technology, Proceedings of SPIE, 4525; Aug. 21-22, 2001;
`pp. 99-106.
`
`J. Lockwood et al., “Reprogrammable Network Packet
`Processing on the Field Programmable Extender (FPX)"; 9th
`International Symposium on Field Programmable Gate
`Arrays; ACM Press 2001; pp. 87-93.
`J. Ditmar et al., “A Dynamically Reconfigurable FPGA
`Based Content Addressable Memory for Internet Protocol
`Characterization'; Proc. 10th International Conference on
`Field Programmable Logic and Applications, Springer
`LNCS 1896; Aug. 27–30, 2000; pp. 19–28.
`S. Guccione et al., “A Reconfigurable Content Addressable
`Memory”; Parallel and Distributed Processing; 15 IPDPS
`2000 Workshop; Springer LNCS 1800; May 1–5, 2000; pp.
`882-889.
`G. Brebner et al., “Runtime Reconfigurable Routing”; Proc.
`12th International Parallel Processing Symposium and 9th
`Symposium Parallel and Distributed Process; Springer
`LNCS 1388; Mar. 30-Apr. 3, 1998; pp. 25–30.
`D. Comer; Chapter 22 The Future IP (IPv6) of Computer
`Networks and Internets; (3rd Edition); Prentice-Hall; 2001;
`pp. 338-349.
`A. Dollas et al., “Architecture and Applications of
`PLATO: . .
`. ”; Proc. 9th IEEE Symposium on Field
`Programmable Custom Computing Machines; IEEE Press;
`Apr. 30-May 2, 2001, pp. Preliminary Proceeding.
`U.S. Appl. No. 10/420,603, filed Apr. 21, 2003, inventor
`Brebner.
`U.S. Appl. No. 10/420,652, filed April 21, 2003, inventor
`Brebner.
`U.S. Appl. No. 10/421,013, filed Apr. 21, 2003, inventor
`Brebner.
`* cited by examiner
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 2 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 1 of 16
`
`US 6,891,397 B1
`
`Configured
`Logic
`105
`
`
`
`PLD
`100
`
`Configured
`Logic
`105
`
`Port Module
`101-1
`
`Port Module
`101-2
`
`Huo
`Module(s)
`102
`
`Embedded
`PrOCeSSOr
`106
`
`Port Module
`101-4
`
`Port Module
`101-3
`
`Configured
`Logic
`105
`
`Configured
`Logic
`105
`
`FIG. 1
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 3 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 2 of 16
`
`US 6,891,397 B1
`
`
`
`Interface
`Module
`
`Network
`interface
`Module
`2014
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 4 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 3 of 16
`
`US 6,891,397 B1
`
`
`
`Mem. Cells
`. 215
`
`:
`
`...
`
`TranSceivers
`
`I
`
`Programmable interconnect
`
`211
`
`FIG. 2B
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 5 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 4 of 16
`
`US 6,891,397 B1
`
`
`
`Uniodirectional Serial
`Inputs and Outputs
`
`PLD
`100
`-
`
`ime
`Domain
`
`Time
`Domain
`38
`
`Packet
`Handler
`
`Microprocessor
`311
`
`FG. 3A
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 6 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 5 of 16
`
`US 6,891,397 B1
`
`
`
`Unidirectional Serial
`Inputs and Outputs
`333
`
`Tithe
`Domain
`331
`
`Time
`Domain
`
`Packet
`Handler
`322-2
`Regs
`333
`
`Microprocessor
`311
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 7 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 6 of 16
`
`US 6,891,397 B1
`
`
`
`
`
`Dr.
`332
`
`Time
`
`D.
`
`Unidirectional Serial
`Inputs and Outputs
`333
`
`
`
`Handler
`322-3
`
`Handler
`322-2
`Regs
`333
`
`Control
`Reg.
`Chain
`319
`
`Process arole a sea--as--
`
`Hub Core Connect
`327
`
`Microprocessor
`311
`
`
`
`FIG. 3C
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 8 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 7 of 16
`
`US 6,891,397 B1
`
`Protocol
`Stack
`400
`-
`
`
`
`Physical Interface Layer
`(LVDS inputs and LVDS Outputs)
`4O1
`Medium Access Control Layer
`(Ethernet Processing)
`4O2
`Transport Protocol Layers
`(TCP/IP Processing)
`403
`
`FIG. 4
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 9 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 8 of 16
`
`US 6,891,397 B1
`
`Y
`
`Packet
`fossing
`Flow
`
`Receive
`Ethernet Packet
`502
`
`
`
`
`
`
`
`
`
`Strean Packet
`Being Received
`511
`
`Broadcast/Sent
`information
`512
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 10 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 9 of 16
`
`US 6,891,397 B1
`
`
`
`
`
`
`
`Broadcast/Sent
`information
`507
`
`
`
`Broadcast
`Packet
`Processing
`
`Broadcast
`Packet
`Processing
`FOW
`
`st
`
`t
`
`
`
`
`
`
`
`
`
`
`
`|
`
`Buffer Information
`521
`
`Complete?
`522
`
`
`
`Change State to
`Ready to Transmit
`524
`Q PortS
`
`Complete?
`522
`
`Change State to
`Ready to Transmit
`
`FIG. 5B
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 11 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 10 of 16
`
`US 6,891,397 B1
`
`
`
`
`
`Advance Buffer
`533
`
`
`
`
`
`
`
`
`
`
`
`
`
`Packet
`Transmission
`FOW
`
`- so
`
`Query Buffer
`531
`
`
`
`Packet
`Waiting?
`532
`
`Transmit Packet (Transmit
`Thread)
`535
`
`FIG. 5C
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 12 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 11 of 16
`
`US 6,891,397 B1
`
`Multithreading Flow
`N so
`
`P
`
`Packet Reception/
`Transmission Thread
`Start
`6O1
`w
`\-
`611
`619. N
`MACISNAP
`L-I - I -
`Header 612 N-1
`J-613
`Recog.
`Thread Encapsulation
`602
`Thread
`603
`
`615 N-1 :
`
`604 IP Address Lookup
`Thread
`605
`
`MAC Header Writing
`Thread
`606
`
`-o-Time
`
`F.G. 6A
`
`Buffer Clock
`Signal
`
`626
`
`626
`
`
`
`
`
`Thread 628
`Clock
`Signal
`627
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 13 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 12 of 16
`
`US 6,891,397 B1
`
`IPv4 Packet?
`702
`
`IPv6 Packet?
`703
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IPv4
`Connection?
`704
`
`IPv6
`Connection?
`
`P Packet
`Conversion Flow
`- 700
`
`IPv6
`Connection?
`707
`
`Connection?
`708
`
`
`
`Encapsulate Packet with
`IPv6
`706
`
`Encapsulate Packet with
`IPv4
`709
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Route Packet
`710
`
`
`
`FIG. 7
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 14 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 13 of 16
`
`US 6,891,397 B1
`
`
`
`Address
`Extraction and
`Matching
`
`Tunnel Lookup
`804
`
`Packet
`Translation
`
`Configured Logic
`800
`
`Packet
`Encapsulation
`803
`
`Fragmentation
`of an Over
`sized Packet
`
`DeCapsulation
`Same Protocol
`
`Decapsulation
`Differing
`Protocols
`
`Packet Handling
`Functions
`
`Physical Layer
`Header
`Recognition
`811
`
`Encapsulation
`814
`
`Protocol
`Recognition
`812
`
`Physica Layer
`Header
`Construction
`815
`
`Address
`Lookup
`813
`
`FIG. 8
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 15 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 14 of 16
`
`US 6,891,397 B1
`
`Routing Table
`
`FIG. 9A
`
`
`
`
`
`IP V4
`(1-bit)
`
`IP W6
`(1-bit)
`
`Router Port
`(4-bits)
`
`Router Port interface
`(10-bits)
`
`Address Lookup
`Table
`p?y 910
`MAC Address
`(48-bits)
`
`912
`
`93 914
`
`915
`
`916
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 16 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 15 of 16
`
`US 6,891,397 B1
`
`Buffer Memory
`1000
`
`-
`
`Flag Space
`
`
`
`Word Space
`1001
`
`Offset
`Space
`1003
`
`Packet
`Space
`1004
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 17 of 29
`
`

`

`U.S. Patent
`
`May 10, 2005
`
`Sheet 16 of 16
`
`US 6,891,397 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`Configured
`Logic
`105-1
`
`Wireless
`Hub
`1102
`
`Configured
`Logic
`105-2
`
`Port Module
`1O
`
`Port Module
`101-2
`
`
`
`
`
`
`
`Port Module
`1O-3
`
`Embedded
`Processor
`106
`
`Port Module
`101-4
`
`
`
`
`
`
`
`
`
`
`
`Configured
`Logic
`105-3
`
`Configured
`Logic
`O5-4
`
`ATM Switch
`1103
`
`FIG. 11
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 18 of 29
`
`

`

`US 6,891,397 B1
`
`1
`GIGABIT ROUTER ON A SINGLE
`PROGRAMMABLE LOGIC DEVICE
`
`1O
`
`25
`
`35
`
`40
`
`15
`
`FIELD OF THE INVENTION
`One or more aspects of the invention relate generally to
`programmable logic devices and, more particularly, to con
`figuring a programmable logic device to provide a router.
`BACKGROUND OF THE INVENTION
`Programmable logic devices (PLDs) exist as a well
`known type of integrated circuit (IC) that may be pro
`grammed by a user to perform Specified logic functions.
`There are different types of programmable logic devices,
`Such as programmable logic arrays (PLAS) and complex
`programmable logic devices (CPLDS). One type of pro
`grammable logic device is called a field programmable gate
`array (FPGA).
`An FPGA conventionally includes an array of config
`urable logic blocks (CLBs) surrounded by a ring of pro
`grammable input/output blocks (IOBs). The CLBs and IOBs
`are interconnected by a programmable interconnect Struc
`ture. The CLBs, IOBs, and interconnect structure are con
`ventionally programmed by loading a Stream of configura
`tion data (bit stream) into internal configuration memory
`cells that define how the CLBs, IOBS, and interconnect
`Structure are configured. Additionally, an FPGA may
`include: embedded memory, Such as block random acceSS
`memories (BRAMs); one or more microprocessors, Some
`times referred to as embedded cores, digital clock managers
`(DCMs); and low voltage differential signaling (LVDS)
`input/output (I/O) interfaces.
`The combination of components on an FPGA may be used
`for System-level integration, Sometimes referred to as
`“System-on-a-chip’ (SoC). Accordingly, System-level inte
`gration with FPGAs is desirable for flexibility and efficiency.
`SUMMARY OF THE INVENTION
`An aspect of the invention is a programmable logic
`device, comprising: port modules having configurable logic
`configured to process communications for routing the com
`munications.
`Another aspect of the invention is a System on a single
`programmable logic integrated circuit, comprising: a hub
`including a microprocessor, ports coupled to the hub, each
`of the ports including buffer memory, a packet handler and
`an input/output interface, the packet handler for each of the
`ports including configurable logic blocks, the configurable
`logic blocks configured to route from the Single program
`mable logic integrated circuit packets received via the
`input/output interface and configured to Send information
`from the packets to the hub for processing with the
`microprocessor, the buffer memory coupled for Storing the
`packets, and buses for respectively coupling the ports to the
`hub, wherein the ports and the hub are coupled to one
`another in a Star topology by the buses.
`Another aspect of the invention is a network, comprising:
`a Single programmable logic device, the Single program
`mable logic device having port modules with configurable
`logic configured to proceSS communications for routing the
`communications, each of the port modules configured to
`process the communications for at least one of a plurality of
`protocols, and network nodes respectively coupled to the
`port modules of the Single programmable logic device, each
`65
`of the network nodes using a respective protocol for com
`munication.
`
`45
`
`50
`
`55
`
`60
`
`2
`BRIEF DESCRIPTION OF THE DRAWINGS
`Accompanying drawing(s) show exemplary embodiment
`(s) in accordance with one or more aspects of the invention;
`however, the accompanying drawing(s) should not be taken
`to limit the invention to the embodiments shown, but are for
`explanation and understanding only.
`FIG. 1 depicts a high-level block diagram of an exemplary
`embodiment of a PLD configured as a router.
`FIG. 2A depicts a block diagram of an exemplary embodi
`ment of a PLD having a unidirectional mesh point-to-point
`topology.
`FIG.2B depicts a block diagram of an exemplary embodi
`ment of a port.
`FIG. 3A depicts a block diagram of an exemplary embodi
`ment of a PLD with a Star topology and a processor centric
`architecture.
`FIG. 3B depicts a block diagram of an exemplary embodi
`ment of a PLD with a bus topology and a logic-centric
`architecture.
`FIG. 3C depicts a block diagram of an exemplary embodi
`ment of a PLD with a point-to-point mesh topology and a
`logic-centric architecture.
`FIG. 4 depicts a block diagram of an exemplary embodi
`ment of a protocol Stack.
`FIG. 5A depicts a flow diagram of an exemplary embodi
`ment of a packet process flow for a port.
`FIG. 5B depicts a flow diagram of an exemplary embodi
`ment of a broadcast packet processing flow.
`FIG. 5C depicts a flow diagram of an exemplary embodi
`ment of a packet transmission flow.
`FIG. 6A depicts a timeline diagram of an exemplary
`embodiment of a multithreading flow.
`FIG. 6B depicts a timing diagram of exemplary embodi
`ments of a buffer clock signal and a thread clock signal.
`FIG. 7 depicts a flow diagram of an exemplary embodi
`ment of a mixed-mode routing flow for a port.
`FIG. 8 depicts a block diagram of an exemplary embodi
`ment of configured logic.
`FIG. 9A depicts a block diagram of an exemplary embodi
`ment of a routing table.
`FIG. 9B depicts block diagram of an exemplary embodi
`ment of a format for an address lookup table.
`FIG. 10 depicts a block diagram of an exemplary embodi
`ment of a buffer memory.
`FIG. 11 depicts a block diagram of an exemplary embodi
`ment of a network.
`
`DETAILED DESCRIPTION OF THE DRAWINGS
`FIG. 1 depicts a high-level block diagram of an exemplary
`embodiment of a PLD 100 configured as a router. At the
`outset, it should be understood that an entire router is
`provided on a single integrated circuit PLD 100.
`PLD 100 includes port modules 101-1, -2, -3 and -4
`(collectively referred to as “ports 101"). PLD 100, as shown,
`includes one or more hub modules 102. For clarity, four
`discrete regions of PLD 100 are shown having respective
`ports 101. However, it should be understood that PLD 100
`may be configured with fewer or more ports 101. Ports 100
`are premised on having configured logic 105, whereas hub
`modules 102 are premised on an embedded microprocessor
`core. Notably, Such a microprocessor core may include
`embedded hardware or embedded firmware or a combina
`tion thereof for a “hard” or “soft' microprocessor. For
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 19 of 29
`
`

`

`US 6,891,397 B1
`
`15
`
`25
`
`35
`
`40
`
`3
`clarity, though more than one embedded microprocessor
`may exist in PLD 100, PLD 100 is described as having a
`Single embedded microprocessor and thus a Single hub
`module (“hub”) 102. Though, for purposes of clarity, PLD
`100 is described in terms of an FPGA having a single
`embedded microprocessor, it will be appreciated that other
`integrated circuits having configurable logic and at least one
`embedded microprocessor may be used.
`PLD 100 is a system-level integration onto a single
`integrated circuit, namely, an SoC architecture. For this
`system-level integration, configurable logic 105 of ports 101
`may be configured for more frequent, leSS control intensive
`operations in contrast to hub 102, which embedded micro
`processor 106 is for leSS frequent, more control intensive
`operations. Accordingly, a majority of packets may be
`entirely handled by ports 101 without ever being received to
`hub 102. In Such a logic-centric architecture, computational
`focus of PLD 100 is placed on configurable logic 105 of
`ports 101, where an embedded microprocessor of hub 102
`Serves as an additional, not a central, System component.
`Alternatively, a more processor-centric architecture may be
`used. Furthermore, an architecture having a combination of
`processor-centric and logic-centric components may be
`used.
`Additionally, as described below, communication
`between ports 101 and one or more devices external to PLD
`100 may be tailored to environmental context. Accordingly,
`ports 101 of PLD 100 may be configured to provide a
`reactive portion of a System, namely, a System that reacts to
`inputs obtained over time. This is in contrast to a transfor
`mational System conventionally associated with running
`programs on a proceSSOr.
`It should be appreciated that each port 101 and each hub
`102 is a respective System, or more particularly a Subsystem
`of the system of PLD 100. Furthermore, hub 102 may
`logically be considered a port as it may have all the
`functionality of a port, and additionally has functionality
`beyond that of a port; however, hub 102 does not have a
`network interface with respect to functionality of hub 102,
`any well-known microprocessor capable of processing net
`work communications may be used. For purposes of clarity,
`gigabit Ethernet and Internet Protocol (IP) digital network
`communications is described, though analog network com
`munications may be used, as well as other combinations of
`data rates and communication protocols. Furthermore,
`though it is assumed that a copper-wired network environ
`ment is used, it should be appreciated that other Signal
`formats (e.g., optical, over-the-air (“wireless”), and the like)
`may be used.
`FIG. 2A is a block diagram of an exemplary embodiment
`of PLD 100 having a unidirectional mesh point-to-point
`topology. With continuing reference to FIG. 1, FIG. 2A is
`described.
`Each port 101-1, -2, -3 and -4 includes a respective
`network interface module 201-1, -2, -3 and -4 (collectively
`referred to as “network interface modules 201”) and
`memory 202-1, -2, -3 and -4 (collectively referred to as
`“memory 202'). For purposes of clearly describing the
`topology of this exemplary embodiment of PLD 100,
`memory 202-1, -2, -3 and -4 is broken out into four sections
`labeled A, B, C and D. For example, memory 202-1 includes
`memory sections 202-1A, -1B, -1C and -1D.
`This exemplary embodiment of PLD 100 includes a hub
`102, namely, embedded microprocessor 203 and memory
`206. Memory 206 associated with embedded microproces
`Sor 203 is broken out into four Sections, namely, memory
`206-1, -2, -3 and -4.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`Connecting ports 101 to one another is done with buses
`205, and connecting ports 101 to hub 102 is done with buses
`204. Buses 205 are broken out into the point-to-point links
`of ports as buses 205-1, -2, -3, -4, -5 and -6. Furthermore,
`buses 205 are unidirectional as illustratively shown.
`Accordingly, each bus 205 is broken out into a directional
`component as indicated by reference letters A and B ascribed
`to each respective bus 205-1, -2, -3, -4, -5 and -6. Thus, for
`example, port 101-1 sends communications to port 101-2 via
`bus 205-1A, and port 101-1 receives communications from
`port 101-2 via bus 205-1B. Memory 202 is disposed for
`receiving incoming communications. Continuing the above
`example, memory 202-2C of port 101-2 buffers incoming
`communications from port 101-1, and memory 202-1C of
`port 101-1 buffers incoming communications from port
`101-2. Notably, for this topology, each port 101 is connected
`to each other port via respective buses 205. In other words,
`ports 101 are in point-to-point communication with one
`another.
`Likewise, buses 204 are broken out into the point-to-point
`links of ports as buses 204-1, -2, -3 and -4. Furthermore,
`buses 204 are unidirectional as illustratively shown.
`Accordingly, each bus 204 is broken out into a directional
`component as indicated by reference letters A and B ascribed
`to each respective bus 204-1, -2, -3 and -4. Thus, for
`example, port 101-1 sends communications to hub 102 via
`bus 204-1B, and port 101-1 receives communications from
`hub 102 via bus 204-1A. Memory 202 is disposed for
`receiving incoming communications. Continuing the above
`example, memory 202-1B of port 101-1 buffers incoming
`communications from hub 102, and memory 206-1 of hub
`102 buffers incoming communications from port 101.
`Notably, for this topology, each port 101 is directly con
`nected to hub 102 via respective buses 204. In other words,
`ports 101 and hub 102 are in point-to-point communication
`with one another.
`FIG. 2B is a block diagram of an exemplary embodiment
`of a port 101. Network interface modules 201-1, -2, -3 and
`-4 include one or more of programmable IOBs 212, one or
`more of DCMS 211, one or more of CLBS 213, one or more
`of configuration memory cells 215, one or more of I/O
`transceivers 216 and a programmable interconnect 211.
`Transceivers need not be used, as Separate receivers and
`transmitters may be used. Notably I/O transceivers may be
`for any of a variety of mediums, Such as wired, wireleSS and
`photonic whether analog or digital. Though, as mentioned
`above, a gigabit router for wired Ethernet communications
`is described, where I/O transceivers are gigabit or multi
`gigabit transceivers. Port 101 may be implemented using an
`FPGA from the VirtexTM-II Pro family of products available
`from Xilinx(E) of San Jose, Calif.
`Though a unidirectional point-to-point mesh topology has
`been described, other topologies may be used, as described
`below. Furthermore, with renewed reference to FIG. 2A and
`continuing reference to FIG. 2B, ports 101 include one or
`more of block random access memories (BRAMs) 214
`coupled to network interface modules 201. BRAMs 214
`provide memories 202, 206 for network interface modules
`201. In the exemplary embodiment described, BRAMs 214
`are dual-port buffers, where there is a BRAM on each of 20
`internal links provided via buses 204, 205. Dual ported
`memory need not be used, as Single ported memory may be
`used.
`Accordingly, each communication port 101 has four “get'
`buffers from which it reads information, Such as packets,
`where each such “get buffer is respectively written to by
`another port 101 or hub 102. Also, hub 102 has four “get”
`
`Ex.1032
`CISCO SYSTEMS, INC. / Page 20 of 29
`
`

`

`5
`buffers from which it reads information, such as packets,
`where each such "get" buffer is respectively written to by a
`port 101.
`Notably, for a VirtexTM-II Pro FPGA implementation,
`buffer word size is 36 bits, and data path widths for a
`point-to-point interconnection network may then be 36 bits
`wide. Of course, longer or shorter bit widths may be used.
`When a write is attempted to a full buffer, the word to be
`written is discarded, and an overflow flag is raised. However,
`there is a commitment mechanism, described below in
`additional detail, for storing complete packets in buffers to
`avoid leaving a packet fragment if a buffer overflow condi
`tions occurs in intra-packet. Each port 101, or hub 102,
`Selects packets for transmission or for further processing, for
`example Selecting on a round robin basis. This round robin
`Selection process is implemented in configured logic 105 at
`ports 101 and by code for processor 106 at hub 102.
`FIG. 3A is a block diagram of an exemplary embodiment
`of a PLD 100 with a star topology and a processor centric
`architecture. In this exemplary embodiment of PLD 100,
`ports 3.16-1, -2, -3 and -4 are coupled to one another via hub
`315 in a star topology. Each port 3.16-1, -2, -3 and -4
`respectively includes: a low-voltage differential signaling
`I/O network interface 301-1, -2, -3 and -4; a packet handler
`302-1, -2, -3 and -4; and buffer memory 303-1, -2, -3 and -4.
`Ports 316-1, -2, -3 and -4, and more particularly dual ported
`buffer memories 303-1, -2, -3 and -4, are connected to hub
`315 via dual port access 320.
`Hub 315 has a microprocessor 311, data on-chip memory
`(OCM) 312, and instruction OCM 313. Hub 315 and ports
`3.16-1, -2, -3 and -4 are serially connected to one another in
`a closed control loop via a device control register (DCR)
`chain 319. DCR chain 319 is used by microprocessor 311 to
`read from and write to control and status registers of packet
`handlers 302-1, -2, -3 and -4. Machine code instructions,
`move to DCR (mtdcr) and move from (mfdcr) may be used
`for reading and writing to control and status registers 333 of
`packet handlers 302-1, -2, -3 and -4.
`Two time domains 317 and 318 are used for this archi
`tecture. Time domain 318 is for operation of microprocessor
`311, and time domain 317 is for transmission of data. So, for
`example, for Serial transmission at gigabit Ethernet rates a
`clock rate of approximately 31.25 MHz may be used,
`whereas operation of microprocessor may be at approxi
`mately 300 MHz. The two time domains may be interfaced
`at dual ported buffer memories 303-1, -2, -3 and -4, where
`information is written in and read out in time domain 318 for
`microprocessor 311 and is written in and read out in time
`domain 317 for packet handlers 302-1, -2, -3 and -4, as well
`as low-voltage differential signaling I/O network interface
`301-1, -2, -3 and -4.
`In this exemplary embodiment, PLD 100 is a processor
`centric architecture. Packet handlers 302 handle physical
`(PHY) layer framing for transmitted packets and recognition
`of PHY layer framing for received packets. However, all
`packets received by PLD 100 pass through hub 315 prior to
`being routed from PLD 100. This is because packet handlers
`302 are only "semi-intelligent,” meaning that packet han
`dlers 302 are not configured for high-level medium access
`protocol (MAC) and Internet Protocol (IP) related functions,
`and consequent packet manipulation and routing, all of
`which is done by hub 315.
`FIG. 3B is a block diagram of an exemplary embodiment
`of a PLD 100 with a bus topology and a logic-centric
`architecture. In this exemplary embodiment of PLD 100,
`ports 326-1, -2, -3 and -4 are coupled to one another via a
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,891,397 B1
`
`6
`bus topology of exchange 329. Each port 326-1, -2, -3 and
`-4 respectively includes: a low-voltage differential signaling
`I/O interface 301-1, -2, -3 and -4; and a packet handler
`322-1, -2, -3 and -4.
`Hub 325 exchanges packets with ports 326 via exchange
`329. Exchange 329 includes a bus topology used to connect
`ports 326 with hub 325. Exchange 329 is an interconnection
`network. Inclusion of exchange 329 moves away from the
`more hub-centric architecture of the exemplary embodiment
`of FIG. 3A. Exchange 329 includes buffer memories 323-1,
`-2, -3 and -4, which are respectively associated with buff
`ering information to ports 326-1, -2, -3 and -4. Exchange
`329 includes buffer memory 328 associated with buffering
`information to and from hub 325. Dual ported buffer memo
`ries 328, and 323-1, -2, -3 and -4 may be used.
`Hub core 330 has a microprocessor 311, data OCM 312,
`and instruction OCM 313. Hub 325 and ports 326-1, -2, -3
`and -4 are serially connected to one another in a closed
`control loop via a DCR chain 319. More particularly, hub
`core 330 is connected to hub core connect 327, and hub core
`connect 327 of hub 325 is serially connected via DCR chain
`319 to ports 326-1, -2, -3 and -4 in a closed loop.
`While inclusion of hub core connect 327 is a slight
`departure for the logic-centric architecture of this embodi
`ment of PLD 100, addition of hub core connect 327 to hub
`325 provides for additional configuration possibilities. For
`example, connection of off-chip or off-PLD 100 memory is
`facilitated by inclusion of hub core connect 327. Also, for
`example, hub core connect 327 provides a mapped memory
`interface between microprocessor 311 and control and status
`registers 333 of ports 326-1, -2, -3 and -4. This facilitates
`program access to Such control and status registers by direct
`memory access (DMA). Hub core connect 327 is used for
`processor local bus (PLB) to on-chip peripheral bus (OPB)
`to DCR bridging. This bridging to DCR chain 319 allows
`microprocessor 311 to have access to control and status
`registers 333 of packet handlers 322. For microprocessor
`311 a PowerPC processor, hub core connect 327 may be an
`IBM Core Connect architecture.
`Notably, hub core connect 327 is optional, and may be
`removed. If hub core connect 327 is not present, access to
`packet buffers of exchange 329 may be done with data-side
`OCM (DSOCM) access from microprocessor 311. If hub
`core connect 327 is not present, packet handlers 322 status
`and control register access may be through mfcdcr and mtdcr
`machine code instructions for direct access to DCR
`registers, as described above.
`Two time domains 331 and 318 are used for this logic
`centric architecture. Time domain 318 is for operation of
`microprocessor 311, and time domain 331 is for transmis
`Sion of data and for dual-threaded operation for massage
`Style functions, as described below. Thus, time domain 331
`may be a dual clock time domain. So, for example, while
`time domain 331 may be for Serial transmission at gigabit
`Ethernet rates a clock rate of approximately 31.25 MHz, it
`may have another cl