`
`US 7,673,072 B2
`Page 2
`
`4,700,185 A
`4,991,133 A
`5,056,058 A
`5,058,110 A
`5,097,442 A
`5,163,131 A
`5,212,778 A
`5,280,477 A
`5,289,580 A
`5,303,344 A
`5,412,782 A
`5,418,912 A
`5,448,566 A
`5,485,579 A
`5,506,966 A
`5,511,169 A
`5,517,668 A
`5,524,250 A
`5,535,375 A
`5,548,730 A
`5,566,170 A
`5,574,919 A
`5,588,121 A
`5,590,328 A
`5,592,622 A
`5,598,410 A
`5,619,650 A
`5,629,933 A
`5,633,780 A
`5,634,099 A
`5,634,127 A
`5,642,482 A
`5,664,114 A
`5,671,355 A
`5,678,060 A
`5,682,534 A
`5,692,130 A
`5,699,317 A
`5,699,350 A
`5,701,434 A
`5,701,516 A
`5,727,142 A
`5,742,765 A
`5,749,095 A
`5,751,715 A
`5,752,078 A
`5,758,084 A
`5,758,089 A
`5,758,186 A
`5,758,194 A
`5,768,618 A
`5,771,349 A
`5,774,660 A
`5,778,013 A
`5,778,419 A
`5,790,804 A
`5,794,061 A
`5,802,258 A
`5,802,580 A
`5,809,328 A
`5,809,527 A
`5,812,775 A
`5,815,646 A
`5,828,835 A
`5,848,293 A
`5,852,721 A *
`5,872,919 A
`5,878,225 A
`5,892,903 A
`5,898,713 A
`5,913,028 A
`5,920,566 A
`5,930,830 A
`
`1011987 Balph et al.
`340/825.5
`211991 Davis et aI .................. 364/900
`1011991 Hirata et al.
`................ 7091230
`1011991 Beach et al.
`............... 370/85.6
`311992 Ward et al. .................... 365/78
`1111992 Row et al. ................... 395/200
`511993 Dally et aI .................. 395/400
`111994 Trapp ........................ 370/85.1
`211994 Latif et al.
`.................. 395/275
`411994 Yokoyama et al. .......... 395/200
`511995 Hausman et al. ............ 395/250
`511995 Christenson ................ 395/200
`911995 Richter et aI ............... 370194.1
`111996 Hitz et al. .............. 395/200.12
`411996 Ban ........................... 395/250
`411996 Suda .......................... 395/280
`511996 Szwerinski et aI.
`.. ....... 395/800
`611996 Chesson et al.
`............. 395/775
`711996 Eshel et aI .................. 3911500
`811996 Young et aI ................. 395/280
`1011996 Bakke et al.
`.................. 370/60
`1111996 Netravali et al. ............ 395/561
`1211996 Reddin et al. .......... 395/200.15
`1211996 Seno et al. .................. 395/675
`111997 Isfeld et al. ............ 395/200.02
`111997 Stone ......................... 370/469
`411997 Bach et al. ............. 395/200.01
`511997 Delp et al.
`.................. 370/411
`511997 Cronin et al.
`............... 3611220
`511997 Andrews et al.
`....... 395/200.07
`511997 Cloud et al.
`................ 395/680
`611997 Pardillos .................. 395/200.2
`911997 Krech, Jr. et al.
`...... 395/200.64
`911997 Collins .................... 395/200.2
`1011997 Yokoyama et al. .......... 7091212
`1011997 Kapoor et al.
`.............. 395/684
`1111997 Shobu et al.
`........... 395/200.12
`1211997 Sartore et al. .......... 395/230.06
`1211997 Kraslavsky ................. 3701254
`1211997 Nakagawa .................. 395/484
`1211997 Cheng et aI ................. 395/842
`311998 Chen .......................... 395/181
`411998 Wong et al. ................. 395/200
`511998 Hagersten ................... 7111141
`511998 Chan et al. .................. 370/455
`511998 Delp et al.
`.................. 395/827
`511998 Silverstein et aI ...... 395/200.58
`511998 Gentry et al.
`.......... 395/200.64
`511998 Hamilton et al. ............ 395/831
`511998 Kuzma ....................... 395/886
`611998 Erickson et al. ............. 395/829
`611998 Picazo, Jr. et aI. ...... 3951188.01
`611998 Brendel et al. ......... 395/200.31
`711998 Jedwab ....................... 714/807
`711998 Hansen et al.
`.............. 7111112
`811998 Osborne ..................... 7091245
`811998 Hansen et al.
`......... 395/800.01
`911998 Chen ..................... 3951182.08
`911998 McAlpine ................... 7111149
`911998 Nogales et aI ............... 395/825
`911998 Cooper et aI ................ 7111133
`911998 Van Seeters et al.
`... 395/200.43
`911998 Purcell et al. ............... 395/163
`1011998 Isfeld et al. .............. 395/200.3
`1211998 Gentry et al.
`............... 395/825
`1211998 Dillon et al. ................ 7091217
`211999 Wakeland et aI ............ 395/200
`311999 Bilansky et aI ......... 395/200.57
`411999 Klaus .................... 3951187.01
`411999 Melzer et al. ................. 371153
`611999 Wang et aI ............. 395/200.33
`711999 Hendel et al. ............... 370/401
`711999 Mendelson et aI .......... 7111171
`
`5,931,918 A
`5,935,205 A
`5,937,169 A
`5,941,969 A
`5,941,972 A
`5,950,203 A
`5,987,022 A
`5,991,299 A
`5,996,013 A
`5,996,024 A
`6,005,849 A
`6,009,478 A
`6,016,513 A
`6,021,446 A
`6,021,507 A
`6,026,452 A
`6,034,963 A
`6,038,562 A
`6,041,058 A
`6,041,381 A
`6,044,438 A
`6,047,323 A
`6,047,356 A
`6,049,528 A
`6,057,863 A
`6,061,368 A
`6,065,096 A
`6,067,569 A
`6,070,200 A
`6,078,733 A
`6,097,734 A
`6,101,555 A
`6,111,673 A
`6,115,615 A
`6,122,670 A
`6,141,701 A
`6,141,705 A
`6,145,017 A
`6,157,944 A
`6,157,955 A
`6,172,980 Bl
`6,173,333 Bl
`6,181,705 Bl
`6,202,105 Bl
`6,223,242 Bl
`6,226,680 Bl
`6,246,683 Bl
`6,247,060 Bl
`6,279,051 Bl
`6,289,023 Bl
`6,298,403 Bl
`6,324,649 Bl
`6,334,153 B2
`6,343,360 Bl
`6,345,301 Bl
`6,345,302 Bl
`6,356,951 Bl
`6,370,599 Bl
`6,385,647 Bl
`6,389,468 Bl
`6,389,479 Bl
`6,393,487 B2
`6,421,742 Bl
`6,421,753 Bl
`6,427,169 Bl
`6,427,171 Bl
`6,427,173 Bl
`6,434,620 Bl
`6,434,651 Bl
`6,449,656 Bl
`6,453,360 Bl
`6,470,415 Bl
`6,473,425 Bl
`
`8/1999 Row et aI .................... 709/300
`8/1999 Murayama et aI ........... 7091216
`8/1999 Connery et al. .......... 395/200.8
`8/1999 Ram et al.
`.................. 710/128
`811999 Hoese et aI.
`.. .............. 710/129
`911999 Stakuis et aI .................. 707/10
`1111999 Geiger et al.
`............... 370/349
`1111999 Radogna et al. ............. 370/392
`1111999 Delp et al.
`.................. 7091226
`1111999 Blumenau ................... 709/301
`12/1999 Roach et aI ................. 370/276
`12/1999 Panner et al.
`.................. 710/5
`112000 Lowe ......................... 7091250
`212000 Gentry et al.
`............... 709/303
`212000 Chen ............................. 714/2
`212000 Pitts ............................ 710/56
`312000 Minami et al. .............. 370/401
`3/2000 Anjur et al. ................... 707/10
`3/2000 Flanders et al. ............. 370/401
`3/2000 Hoese ........................ 710/129
`312000 Olnowich ................... 7111130
`4/2000 Krause ....................... 7091227
`4/2000 Anderson et al.
`........... 7111129
`4/2000 Hendel et aI ................ 3701235
`5/2000 Olarig ........................ 345/520
`5/2000 Hitzelberger ............... 370/537
`5/2000 Dayet al.
`................... 7111114
`5/2000 Khaki et al.
`................ 7091224
`5/2000 Gates et al. ................... 71 0120
`6/2000 Osborne .................. 395/200.8
`8/2000 Gotesman et al. ........... 370/474
`8/2000 Goshey et aI.
`.. ............ 709/321
`8/2000 Chang et al. ................ 359/123
`912000 Ota et aI ..................... 455/553
`912000 Bennett et al. .............. 7091236
`1012000 Whitney ........................ 710/5
`1012000 Anand et al. .................. 710115
`1112000 Ghaffari ........................ 710/5
`1212000 Pedersen
`................ 7091228
`1212000 Narad et al.
`112001 Flanders et al. ............. 370/401
`112001 Jolitz et aI.
`112001 Branstad et al. ............. 370/392
`3/2001 Gates et al. ................... 71 0120
`4/2001 Sheafor et al. .............. 710/132
`5/2001 Boucher et al.
`............. 7091230
`6/2001 Connery et al. ............. 370/392
`6/2001 Boucher et al.
`............. 7091238
`8/2001 Gates et al. ................... 71 0120
`912001 Dowling et al. ............. 370/419
`10/2001 Suri et al.
`................... 710/100
`1112001 Eyres et al. ................. 713/202
`12/2001 Boucher et al.
`............. 7091230
`112002 Feinleib ......................... 713/1
`212002 Burns et al. ................. 7091230
`212002 Bennett et al. .............. 7091236
`3/2002 Gentry et al.
`............... 7091250
`4/2002 Anand et al. .................. 710115
`5/2002 Willis et aI .................. 7091217
`5/2002 Muller et al.
`............... 7091226
`5/2002 Boucher ..................... 7091243
`5/2002 Boucher et al.
`............. 7091238
`7/2002 Tillier ........................... 710/1
`712002 Hoese et aI ................. 710/129
`7/2002 Elzur ......................... 7091224
`7/2002 Craft et al. .................. 7091230
`7/2002 Boucher et al.
`............. 7091238
`8/2002 Boucher et al.
`............. 7091230
`8/2002 Gentry, Jr.
`.. ................ 710/260
`912002 Elzur et aI.
`.. ............... 7091236
`912002 Muller et al.
`............... 7091250
`1012002 Starr et al.
`.................. 7111104
`1012002 Bellaton et al.
`............. 370/392
`
`
`
`US 7,673,072 B2
`Page 3
`
`6,480,489 Bl
`6,487,202 Bl
`6,487,654 B2
`6,490,631 Bl
`6,502,144 Bl
`6,523,119 B2
`6,526,446 Bl
`6,570,884 Bl
`6,591,302 B2
`6,591,310 Bl
`6,648,611 B2
`6,650,640 Bl
`6,657,757 Bl
`6,658,480 B2
`6,678,283 Bl
`6,681,364 Bl
`6,687,758 B2
`6,697,868 B2
`6,751,665 B2
`6,757,746 B2
`6,765,901 Bl
`6,807,581 Bl
`6,842,896 Bl
`6,912,522 B2
`6,938,092 B2
`6,941,386 B2
`6,965,941 B2
`6,996,070 B2
`7,042,898 B2
`7,076,568 B2
`7,089,326 B2
`7,093,099 B2
`7,124,205 B2
`7,133,940 B2
`7,167,926 Bl
`7,167,927 B2
`7,174,393 B2
`7,185,266 B2
`7,191,241 B2
`7,191,318 B2
`7,237,036 B2
`7,254,696 B2
`7,284,070 B2
`200110004354 A
`200110025315 A
`200110013059 Al
`200110014892 Al
`200110014954 Al
`200110048681 Al
`200110053148 Al
`200210073223 Al
`200210112175 Al
`2003/0066011 Al
`2003/0110344 Al
`2003/0165160 Al
`2004/0054814 Al
`2004/0059926 Al
`2004/0153578 Al
`2004/0213290 Al
`2004/0246974 Al
`
`............... 370/389
`1112002 Muller et al.
`1112002 Klausmeier et al. ......... 370/395
`1112002 Dowling ..................... 7121244
`12/2002 Teich et al.
`................. 7091250
`12/2002 Accarie ......................... 710/8
`212003 Pavlin et al ................. 713/192
`212003 Yang et al.
`5/2003 Connery et al. ............. 370/419
`7/2003 Boucher et al.
`............. 7091230
`7/2003 Johnson ... ..... ... ... ... ..... ... 710/3
`1112003 Morse et al .
`................ 417/310
`1112003 Muller et al.
`............... 370/392
`1212003 Chang et al ................. 359/124
`1212003 Boucher et al.
`............. 7091239
`112004 Teplitsky .................... 370/463
`112004 Calvignac et al ............ 7141776
`212004 Craft et al. .................. 7091250
`212004 Craft et al. .................. 7091230
`6/2004 Philbrick et al ............. 7091224
`6/2004 Boucher et al.
`............. 7091250
`7/2004 Johnson et al.
`............. 370/352
`1012004 Starr et al.
`.................. 7091250
`112005 Redding et al.
`............. 717/172
`6/2005 Edgar ............................ 707/2
`8/2005 Burns ......................... 7091230
`912005 Craft et al. .................. 7091250
`1112005 Boucher et al.
`............. 7091230
`212006 Starr et al.
`.................. 3701252
`5/2006 Blightman et al.
`.......... 370/463
`7/2006 Philbrick et al ............. 7091250
`8/2006 Boucher et al.
`............. 7091242
`8/2006 Bodas et al.
`................ 7111206
`1012006 Craft et al. .................. 7091250
`1112006 Blightman et al.
`............ 710122
`112007 Boucher et al.
`............. 7091250
`112007 Philbrick et al ............. 7091250
`212007 Boucher et al.
`............. 7091250
`212007 Blightman et al.
`.......... 7141776
`3/2007 Boucher et al.
`............. 7091230
`3/2007 Tripathyet al. ............. 7121225
`6/2007 Boucher et al.
`............. 7091245
`8/2007 Mittal et al.
`................ 7121210
`1012007 Boucher et al.
`............. 7091250
`112001 Jolitz
`......................... 370/328
`112001 Jolitz
`......................... 7091231
`8/2001 Dawson et al. .............. 7091217
`8/2001 Gaither et al.
`.............. 707/200
`8/2001 Purcell et al. .................. 714/4
`1212001 Bilic et al.
`.................. 370/389
`1212001 Bilic et al.
`.................. 370/389
`6/2002 Darnell et al.
`.............. 7091232
`8/2002 Makofka et al.
`............ 713/200
`412003 Oren .......................... 7141758
`6/2003 Szczepanek et al ......... 7111100
`912003 Minami et al. .............. 370/466
`3/2004 McDaniel
`3/2004 Angelo et al ................ 713/168
`812004 Elzur
`1012004 Johnson et al.
`1212004 Gyugyi et al.
`
`............. 370/469
`370/395.31
`
`FOREIGN PATENT DOCUMENTS
`
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`WO
`
`WO 98119412
`WO 98/50852
`WO 99104343
`WO 99/65219
`WO 00/13091
`WO 01104770
`WO 01105107
`WO 01105116
`WO 01105123
`WO 01140960
`
`5/1998
`1111998
`111999
`12/1999
`312000
`112001
`112001
`112001
`112001
`6/2001
`
`WO
`WO
`
`WO 01159966
`WO 01186430
`
`8/2001
`1112001
`
`OTHER PUBLICATIONS
`
`Internet pages entitled "Hardware Assisted Protocol Processing",
`(which Eugene Feinber is working on), 1 page, printed Nov. 25,1998.
`Zilog product Brief entitled "Z85C30 CMOS SCC Serial Commu(cid:173)
`nication Controller", Zilog Inc., 3 pages, 1997.
`Internet pages of Xpoint Technologies, Inc. entitled "Smart LAN
`Work Requests", 5 pages, printed Dec. 19, 1997.
`Internet pages entitled: Asante and 100BASE-T Fast Ethernet. 7
`pages, printed May 27, 1997.
`the Paragon XP/S-A7
`to
`Internet pages entitled: A Guide
`Supercomputer at Indiana University. 13 pages, printed Dec. 21,
`1998.
`Richard Stevens, "TCP/IP Illustrated, vol. 1, The Protocols", pp.
`325-326 (1994).
`Internet pages entitled: Northridge/Southbridge vs. Intel Hub Archi(cid:173)
`tecture, 4 pages, printed Feb. 19, 2001.
`Gigabit Ethernet Technical Brief, Achieving End-to-End Perfor(cid:173)
`mance. Alteon Networks, Inc., First Edition, Sep. 1996, 15 pages.
`Internet pages directed to Technical Brief on Alteon Ethernet Gigabit
`NIC technology, www.alteon.com.14pages.printed Mar. 15, 1997.
`Via Technologies, Inc. article entitled "VT8501 Apollo MVP4", pp.
`i-iv, 1-11, cover and copyright page, revision 1.3, Feb. 1,2000.
`iReady News Archives article entitled "iReady Rounding Out Man(cid:173)
`agement Team with Two Key Executives", http://www.ireadyco.
`corniarchives/keyexec.html, 2 pages, printed Nov. 28, 1998.
`"Toshiba Delivers First Chips to Make Consumer Devices Internet(cid:173)
`Ready Based On iReady's Design," Press Release Oct. 1998,3 pages,
`printed Nov. 28, 1998.
`Internet pages from iReady Products, web sitehttp://www.ireadyco.
`corniproducts,html, 2 pages, downloaded Nov. 25, 1998.
`iReady News Archives, Toshiba, iReady shipping Internet chip, 1
`page, printed Nov. 25, 1998.
`Interprophet article entitled "Technology", http://www.interprophet.
`cornitechnology.html, 17 pages, printed Mar. 1, 2000.
`iReady Corporation, article entitled "The 1-1000 Internet Tuner", 2
`pages, date unknown.
`iReady article entitled "About Us Introduction", Internet pages
`frornhttp://www.iReadyco.comiabout.html.3pages.printed Nov.
`25, 1998.
`iReady News Archive article entitled "Revolutionary Approach to
`Consumer Electronics Internet Connectivity Funded", San Jose, CA,
`Nov. 20,1997. 2 pages, printed Nov. 2, 1998.
`iReady News Archive article entitled "Seiko Instruments Inc. (SII)
`Introduces World's First Internet-Ready Intelligent LCD Modules
`Based on iReady Technology," Santa Clara, CA and Chiba, Japan,
`Oct. 26, 1998.2 pages, printed Nov. 2, 1998.
`NEWSwatch article entitled "iReady internet Tuner to Web Enable
`Devices", Tuesday, Nov. 5, 1996, printed Nov. 2, 1998,2 pages.
`EETimes article entitled "Tuner for Toshiba, Toshiba Taps iReady for
`Internet Tuner", by David Lammers, 2 pages, printed Nov. 2, 1998.
`"Comparison of Novell Netware and TCPIIP Protocol Architec(cid:173)
`tures", by J.S. Carbone, 19 pages, printed Apr. 10, 1998.
`Adaptec article entitled "AEA-7110C-a DuraSAN product", 11
`pages, printed Oct. 1, 2001.
`iSCSI HBA article entitled "iSCSI and 2Gigabit fibre Channel Host
`Bus Adapters from Emulex, QLogic, Adaptec, JNI", 8 pages, printed
`Oct. 1, 2001.
`iSCSI HBA article entitled "FCE-3210/641O 32 and 64-bit PCI-to(cid:173)
`Fibre Channel HBA", 6 pages, printed Oct. 1, 2001.
`ISCSI.com article entitled "iSCSI Storage", 2 pages, printed Oct. 1,
`2001.
`"Two-Way TCP Traffic Over Rate Controlled Channels: Effects and
`Analysis", by Kalampoukas et aI., IEEE Transactions on Network(cid:173)
`ing, vol. 6, No.6, Dec. 1998, 17 pages.
`IReady News article entitled "Toshiba Delivers First Chips to Make
`Consumer Devices Internet-Ready Based on iReady Design", Santa
`Clara, CA, and Tokyo, Japan, Oct. 14, 1998, printed Nov. 2, 1998,3
`pages.
`
`
`
`US 7,673,072 B2
`Page 4
`
`Internet pages of InterProphet entitled "Frequently Asked Ques(cid:173)
`tions", by Lynne Jolitz, printed Jun. 14,2000,4 pages.
`"File System Design For An NFS File Server Appliance", Article by
`D. Hitz, et ai., 13 pages.
`Adaptec Press Release article entitled "Adaptec Announces
`EtherStorage Technology", 2 pages, May 4, 2000, printed Jun. 14,
`2000.
`Adaptec article entitled "EtherStorage Frequently Asked Questions",
`5 pages, printed Jui. 19,2000.
`Adaptec article entitled "EtherStorage White Paper", 7 pages, printed
`Jui. 19,2000.
`CIBC World Markets article entitled "Computers; Storage", by J.
`Berlino et al., 9 pages, dated Aug. 7, 2000.
`Merrill Lynch article entitled "Storage Futures", by S. Milunovich,
`22 pages, dated May 10, 2000.
`CBS Market Watch article entitled "Montreal Start-Up Battles Data
`Storage Botttieneck", by S. Taylor, dated Mar. 5, 2000, 2 pages,
`printed Mar. 7, 2000.
`Internet-draft article entitled "SCSI/TCP (SCSI over TCP)", by J.
`Satran et al., 38 pages, dated Feb. 2000, printed May 19, 2000.
`Internet pages entitled Technical White Paper-Xpoint's Disk to LAN
`Acceleration Solution for Windows NT Server, printed Jun. 5, 1997,
`15 pages.
`Jato Technologies article entitled Network Accelerator Chip Archi(cid:173)
`tecture, twelve-slide presentation, printed Aug. 19, 1998, 13 pages.
`EETimes article entitled Enterprise System Uses Flexible Spec,
`dated Aug. 10,1998, printed Nov. 25,1998,3 pages.
`Internet pages entitled "Smart Ethernet Network Interface Cards",
`which Berend Ozceri is developing, printed Nov. 25, 1998,2 pages.
`Internet pages of Xaqti corporation entitled "GigaPower Protocol
`Processor Product Review," printed Nov. 25, 1999,4 pages.
`U.S. Appi. No. 601283,896, Titled: CRC Calculations for Out of
`Order PUDs, filed Apr. 12, 2003, Inventor: Amit Oren, Assignee:
`Siliquent Technologies Ltd.
`Internet pages entitled "DART: Fast Application Level Networking
`via Data-Copy Avoidance," by Robert J. Walsh, printed Jun. 3, 1999,
`25 pages.
`Andrew S. Tanenbaum, Computer Networks, Third Edition, 1996,
`ISBN 0-13-349945-6.
`Article from Rice University entitled "LRP: A New Network Sub(cid:173)
`system Architecture for Server Systems", by Peter Druschel and
`Gaurav Banga, 14 pages.
`Internet RFC/STD/FYI/BCP Archives article with heading
`"RFC2140" entitled "TCP Control Block Interdependence", web
`address http://www.faqs.org/rfcs/rfc2140.htrnl.9pages.printed Sep.
`20,2002.
`WindRiver article entitled "Tornado: For Intelligent Network Accel(cid:173)
`eration", copyright Wind River Systems, 2001, 2 pages.
`WindRiver White Paper entitled "Complete TCP/IP Offload for
`High-Speed Ethernet Networks", Copyright Wind River Systems,
`2002, 7 pages.
`Intel article entitled "Solving Server Bottlenecks with Intel Server
`Adapters", Copyright Intel Corporation, 1999,8 pages.
`Schwaderer et ai., IEEE Computer Society Press publication entitled,
`"XTP in VLSI Protocol Decomposition for ASIC Implementation",
`from 15th Conference on Local Computer Networks, 5 pages, Sep.
`30-0ct. 3, 1990.
`Beach, Bob, IEEE Computer Society Press publication entitled,
`"UltraN et: An Architecture for Gigabit Networking", from 15th Con(cid:173)
`ference on Local Computer Networks, 18 pages, Sep. 30-0ct. 3,
`1990.
`
`Chesson et ai., IEEE Syposium Record entitled, "The Protocol
`Engine Chipset", from Hot Chips III, 16 pages, Aug. 26-27,1991.
`Maclean et ai., IEEE Global Telecommunications Conference,
`Globecom '91, presentation entitled, "An Outboard Processor for
`High Performance Implementation of Transport Layer Protocols", 7
`pages, Dec. 2-5, 1991.
`Ross etai., IEEE article entitled "FXI000: Ahigh performance single
`chip Gigabit Ethernet NIC", from Compcon '97 Proceedings, 7
`pages, Feb. 23-26, 1997.
`Strayer et ai., "Ch. 9: The Protocol Engine" from XTP: The Transfer
`Protocol, 12 pages, Jui. 1992.
`Publication entitled "Protocol Engine Handbook", 44 pages, Oct.
`1990.
`Koufopavlou et ai., IEEE Global Telecommunications Conference,
`Globecom '92, presentation entitled, "Parallel TCP for High Perfor(cid:173)
`mance Communication Subsystems", 7 pages, Dec. 6-9, 1992.
`Lilienkamp et ai., Publication entitled "Proposed Host-Front End
`Protocol", 56 pages, Dec. 1984.
`Thia et ai. Publication entitled "High-Speed OSI Protocol Bypass
`Algorithm with Window Flow Control," Protocols for High Speed
`Networks, pp. 53-68, 1993.
`U.S. Appi. No. 601053,240, Titled: TCP/IP Network Accelerator and
`Method of Use, filed Jui. 17, 1997, Inventor: William Jolizt et ai.
`Thia et ai. Publication entitled "A Reduced Operational Protocol
`Engine (ROPE) for a multiple-layer bypass architecture," Protocols
`for High Speed Networks, pp. 224-239, 1995.
`Form 10-K for Exelan, Inc., for the fiscal year ending Dec. 31, 1987
`(10 pages).
`Form 10-K for Exelan, Inc., for the fiscal year ending Dec. 31, 1988
`(10 pages).
`* cited by examiner
`Primary Examiner-Jude J Jean Gilles
`(74) Attorney, Agent, or Firm-Mark Lauer; Silicon Edge
`Law Group, LLP
`
`(57)
`
`ABSTRACT
`
`A system for protocol processing in a computer network has
`an intelligent network interface card (INIC) or communica(cid:173)
`tion processing device (CPD) associated with a host com(cid:173)
`puter. The INIC provides a fast-path that avoids protocol
`processing for most large multi-packet messages, greatly
`accelerating data communication. The INIC also assists the
`host for those message packets that are chosen for processing
`by host software layers. A communication control block for a
`message is defined that allows DMA controllers of the INIC
`to move data, free of headers, directly to or from a destination
`or source in the host. The context is stored in the INIC as a
`communication control block (CCB) that can be passed back
`to the host for message processing by the host. The INIC
`contains specialized hardware circuits that are much faster at
`their specific tasks than a general purpose cpu. A preferred
`embodiment includes a trio of pipelined processors with
`separate processors devoted to transmit, receive and manage(cid:173)
`ment processing, with full duplex communication for four
`fast Ethernet nodes.
`
`21 Claims, 89 Drawing Sheets
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 1 of89
`
`US 7,673,072 B2
`
`20
`r------------------------,
`HOST
`1--/
`I
`
`,
`
`28
`
`-
`
`CPU
`
`r 22
`
`REMOTE
`HOST
`
`25\
`
`(50
`
`42,,-
`
`35~ STORAGE r---
`
`I
`I
`I
`I
`I
`I
`I
`30 I
`~ CPD
`
`1 -
`
`~33
`
`I
`I
`I
`I
`L _____ - - - - - - - - - - - - - - - - -
`
`FIG. 1
`
`( 46
`
`UPPER
`LAYER
`UPPER LAYER 48\
`INTERFACE
`
`STORAGE
`
`"-
`
`35
`
`~8
`
`\
`
`44
`
`CONTEXT
`
`~
`
`52
`'--
`
`54)
`TRANSPORT
`4~
`38"'-
`
`NETWORK
`
`r - DATA LINK
`36
`
`30\
`
`56
`~
`,Ir
`
`INIC/CPD
`
`....
`
`FIG. 2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 8 of89
`
`US 7,673,072 B2
`
`r--------------------------------I
`202/~,_~
`I
`I
`I
`
`300
`"\ 318~
`
`, ' - - - - -
`
`UPPERLAYER
`
`~.I--': ___ -"
`
`"
`
`I
`I
`I
`:
`I
`~III
`- 316
`I ~
`I
`
`:
`:
`I
`I
`I
`,~.
`330
`1--'
`I
`.~I
`325
`r----'
`I
`I
`322~ NETWORK
`I
`I 1--_____ --1
`I
`I
`320~ DATALINK
`DATALINK ~ 312
`I
`I
`I
`I'
`I'
`I
`I
`L ________________________ -t. _____ 1
`'--310
`,w-
`'w-
`
`UPPER LAYER INTERFACE
`
`TRANSPORT
`
`TRANSPORT
`
`NETWORK
`
`~314
`
`240~
`
`306 ~ INIC MINIPORT DRIVER
`
`J
`
`1
`
`INIC
`
`200~
`
`(
`
`210
`
`INIC
`MEMORY ~304
`
`FIG. 10
`
`
`
`
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 11 of 89
`
`US 7,673,072 B2
`
`CLOCK r--r-------L----- L ______ /- ______ L _____ L __
`
`/
`
`/
`I
`I
`I
`I
`I
`I
`
`/
`
`/
`
`500'-,
`,
`
`)
`I
`
`492.....,
`
`CONTROLS FOR FIRST REGISTER SET
`
`490\
`
`505
`
`-../"'
`
`533\
`
`FIRST REGISTER SET
`
`~ I
`I
`I :
`I
`, ,
`I
`>-- f------- - - - - - - - - - - - - - - f-------- ---
`
`RAM FILE
`REGISTER
`f--
`
`498~
`
`,
`INSTRUCTION DECODER
`AND
`OPERAND MULTIPLEXER
`
`496\
`
`SECOND REGISTER SET
`
`/
`
`/
`I
`I
`I
`I
`I
`I
`I
`J
`
`/
`
`, ,
`560,/
`
`I
`I
`
`~ I
`IT
`
`I
`I
`I
`I
`
`I , ,
`) - - -------- ------- - - - - - - - 1-------- - - - - - - - -----
`
`/
`
`/
`I
`I
`I
`STACK
`I
`I EXCHANGE
`I
`I
`~608
`r
`
`ARITHMETIC LOGIC UNIT
`
`QUEUE
`
`602)
`
`503)
`
`r
`
`r
`
`r
`
`r
`
`r
`
`/
`
`/
`
`600/, ,
`
`)
`I
`I
`
`ITI
`
`I
`
`(:"
`470
`
`THIRD REGISTER SET
`501~
`'--~------~----- ~------~------~----- '5..--
`FIG. 14
`
`
`
`~ ___ L_~ ___ ~---~---~--~---~--~--r==1~ __ ~ __ ~ ___ ~ __
`•
`
`500
`
`\
`\
`
`1
`1
`1
`I
`1
`1
`
`r
`
`492:
`
`Ir
`
`SRAM LOAD LOAD
`CTRL CTRL CTRL
`
`502
`
`FLAG
`DEC
`
`508T1
`
`..
`
`•
`
`•
`
`..
`
`533,\
`
`J"
`
`ADDR
`C
`
`r
`ADDR
`ALU FLAG
`FILE
`~ &
`(: r DATA CTX
`CCS REGS
`490 1 520~~~~-LTT~In'-~~
`
`1
`1
`
`RAM FILE
`REGISTER
`
`I- - \- -
`
`________ - - - -1 - -
`
`•
`
`t
`
`•
`
`,
`
`•
`
`Ir
`
`I
`I
`DOUT DOUT
`
`,
`
`ADDR ADDR'
`
`FIG. 15A
`
`~
`7Jl
`•
`~
`~
`~
`
`~ = ~
`
`N
`
`~
`~ :-:
`N o .... o
`
`rFJ =(cid:173)
`('D a ....
`o ....
`
`N
`
`QO
`\0
`
`LOAD
`CTRL
`
`517T1
`..
`
`505 ""'--
`
`~
`
`PC ISTACKIDEBUG
`ADDR
`
`510
`
`535 \
`
`•
`
`Ir
`
`1 •
`
`D~ IINSTR/FETCH
`REG IADDR
`
`-
`
`- -' +- -
`
`- - -I- -
`
`-
`
`- -1- -
`
`-
`
`-
`
`-
`
`-
`
`-
`
`538) I 540) 54471~8]
`•
`•
`•
`
`•
`
`-
`
`-
`
`-
`
`-
`
`d
`rJl
`-....l
`0..,
`-....l
`
`W -= -....l
`N = N
`
`
`
`560-
`
`\
`
`\
`~----~-~- --4----+----~--
`
`ADDR DIN
`C
`C
`RAM FILE
`REGISTER
`
`1--------
`
`SCRATCH
`SRAM
`)
`565
`
`I
`I
`I
`I
`I
`I
`
`r-l DOUT DOUTI(cid:173)
`
`..
`
`ADDR ADDR
`'533) I
`.. *
`INSTRUCTION DECODER
`AND
`OPERAND MULTIPLEXER
`
`r*
`
`: 498)
`I
`I
`I
`I
`I
`I
`:
`
`1
`
`r CTX OPDS CCS
`
`..
`
`,
`
`..
`
`,
`
`..
`FILE ALU ALU ALU TEST FLAG QCH& LIT
`OP
`SEL
`SEL QCMD
`
`----t------~----------
`
`,--t--
`INCR
`
`INCR
`
`568)
`
`570)
`
`505"'--
`
`LOAD k---...
`CTRLf
`"
`
`575)
`
`1
`
`PGM
`CTRL
`
`PC STACK DEBUG
`ADDR
`
`-------- ----~------
`
`~
`7Jl
`•
`~
`~
`~
`
`~ = ~
`
`~
`~ :-:
`J'J
`N o .... o
`
`('D
`('D
`
`(.H
`
`rFJ =(cid:173)
`.....
`....
`o ....
`
`QO
`\0
`
`d
`rJl
`",-.....1
`0'1
`-.....1
`
`"'w = -.....1
`N = N
`
`496: 577
`I
`~-----
`
`1
`
`..
`
`1
`
`1
`
`1
`
`FIG. 1SB
`
`
`
`600 ---"
`
`\
`\
`L_~ ____ ~ ____ ~ ____ ~ ___ ~ ____ ~ ____ + ____ ~ ____ ~ ___ ~ ____ ~ _____ ~ ____ _
`
`~
`7Jl
`•
`~
`~
`~
`
`~ = ~
`
`•
`
`ALU
`
`MUX
`
`QALU
`
`602) I
`1 !
`
`I
`!
`
`QFLGS
`DEST
`&
`FILE ALU ALU OPD TEST FLAG
`CTX OUT CCS RSLT RSLT SEL QADDR
`
`• ttl t ~ STACK
`ITESTI~ IQ~
`160~) I ' 16071 1608) 1
`!
`!
`!
`! 1 ! 1 1 1
`
`EXCHANGE
`
`LOAD~
`CTRL
`
`1
`
`1610) I
`
`505~
`
`LIT PGM
`CTRL
`
`PC STACK DEBUG
`ADDR
`
`N
`
`~
`~ :-:
`N o .... o
`
`~
`
`('D
`('D
`
`.j;o.
`
`rFJ =(cid:173)
`.....
`....
`o ....
`
`QO
`\0
`
`616
`
`I
`L __ _
`
`FIG. 15C
`
`d
`rJl
`-....l
`0..,
`-....l
`
`-....l
`
`W -=
`N = N
`
`
`
`
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 17 of 89
`
`US 7,673,072 B2
`
`9~ LRU
`
`1 ~O 1 ~1 I~I. . 1 ~7 1 ~~ 1 ~ I. .
`FIG.19A
`
`MRU
`R13 R14 R15
`13
`4
`6
`
`9~ LRU
`
`1 ~o 1 ~1 1 ~ I. . I~; 1 ~g 1 ~91· .
`
`MRU
`R13 R14 R15
`6
`9
`4
`
`FIG. 19B
`
`9~ LRU
`
`1 ~o 1 ~1 1 ~ I· . I~; 1 ~8 1 ~ I·
`
`FIG. 19C
`
`9~ LRU
`
`1 ~o 1 ~1 I~I .. I ~71 ~~ I ~ I ..
`FIG. 19D
`
`MRU
`R13 R14 R15
`10
`6
`9
`
`MRU
`R13 R14 R15
`9
`10
`6
`
`
`
`(~
`------------------------------------------------------~
`r I
`
`,--,
`r.
`MUX
`r-~ 0
`
`n
`/. ,D . ,IT
`MUX
`MUX
`;/ ~ 7
`~ 8
`
`MUX
`~ 9
`
`I I
`
`• rr
`
`915
`
`l
`
`~
`7Jl
`•
`~
`~
`~
`
`~ = ~
`
`...- 910
`
`7'r-, r '.
`MUX
`r=: 15
`
`~,....
`
`r
`
`/.
`
`/.
`
`3 t -
`
`RO r- L-I'-- R7 r-
`
`r
`
`2
`
`,
`r
`L......r~-i'-- R8 r- L.../~"--~ R9 r-
`12
`1
`
`•
`
`,
`
`+_ ,
`
`C7
`
`i ,
`C8
`
`J ;
`C9
`
`CO
`~ 944
`949
`954
`939, ~ , ~ r ~ ,
`
`/.
`
`"/
`
`L....../I-i'- R15 r--
`10
`I
`t ;
`C15
`
`"'-900
`
`"'-920
`
`959
`~
`
`N
`
`~
`~ :-:
`N o .... o
`
`~
`
`('D
`('D
`
`QO
`
`rFJ =(cid:173)
`.....
`....
`o ....
`
`QO
`\0
`
`~966
`
`930
`
`LOGIC CIRCUITS
`,
`
`PROCESSOR
`
`I -
`
`935
`
`FIG. 20
`
`470
`
`d
`rJl
`-....l
`0..,
`-....l
`
`-....l
`
`W -=
`N = N
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 19 of 89
`
`US 7,673,072 B2
`
`NETWORK
`
`J:210
`
`INIC 200
`__ 2101
`------¥-----------~----------------
`
`tv21D3
`. ~2~112~
`
`ACCESS
`CTRL
`I 41 0
`r
`2108! 2113 2106
`SEQUENCERS
`2102
`[ RXSEQ I
`
`lCONFIG.J
`
`TXSEQ
`2104
`
`21 05.
`
`:-,.475
`
`PHYSICAL 2100
`r---
`LAYER
`INTERF.
`ASIC 400
`1------- ---------2-----------------,
`I
`I
`j.
`j.
`~2229
`2230)
`I
`I
`:
`j02
`2109--
`QUEUE
`:
`PROCESSOR
`M GR
`470
`: :
`MEDIA
`N
`~X ~ I I
`: :
`2231 2232
`I I
`'--------y-------' I I
`HOST 20
`I I
`11 _______ ..:::' ____ 1
`257
`:
`: ::
`~8 'II
`I
`: : I
`CPU
`, I:
`I
`2 8 :
`:2
`,
`.. PCI
`BUS I~ I ,
`~
`I NT.
`I I
`: :
`: :
`
`,
`I
`:
`STORAGE
`3 5 :
`
`l i L -----------j
`
`I
`
`2228 ~ 442
`SRAM
`')50
`SRAM
`CTRL
`44~ ~ DRAM
`jl~rt~~: } 2110
`CTRL
`L
`'I
`, I
`I r
`---------------------------------~,
`DRAM
`:
`I
`I
`~ 460
`:
`I
`,
`I
`,
`,
`,
`I
`I
`I
`I
`I
`I
`I
`I
`I
`L _____________________________________ ~
`I
`I
`
`I
`I
`I
`I
`
`:
`
`STATUS
`/2223
`
`J
`
`} BUFFER
`2114
`
`FIG. 21
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 20 of 89
`
`US 7,673,072 B2
`
`MAC
`
`QUEUE
`MANAGER
`
`2215, III
`2213 2214 2227
`r'-
`c
`8v
`---~--------------i~
`2200
`r--
`
`f
`
`2210 2209
`2221~
`2105
`I-'
`f--'
`I
`c!
`I
`----~-----~----~---
`I
`I
`I
`:
`QUEUE
`DATA
`I
`I
`PACKET:
`\
`MANAGER
`SYNC
`:
`INTERFACE 1<t---~'---.o;1 ~-,,-r-.L-
`SYNC
`22222
`BUFFER
`:
`I
`\ ~ SEQUENCER
`II ~ : 2230
`2201
`\
`fl
`I
`I
`\ ) l
`\
`I
`2207
`:
`: 2268
`:
`.... 1
`I
`I
`\
`- -:-
`:
`
`2212 "
`
`t---2211
`
`I
`
`/
`
`~2228
`
`4.v ~v
`- - - - - - - - - - - - ;2-0; 22 16-" - r--221 9'
`:
`PACKET
`DATA
`\
`P
`2218
`~
`PROCESSING I SUMMARy]
`ASSEM.BLY /_.r
`I
`I
`REGISTER 3 2220
`SEQUENCER
`\2224:
`i 4= i
`f" '3 .2
`2204
`:
`:
`'-------t--+-.l_---'
`I
`v 2217
`:
`I
`I
`I
`I
`I
`:
`,
`:
`I
`:
`I
`I
`I
`\
`I
`
`I STATUS 1- 2223
`~ PROTOCOL
`ANALYZER
`2203
`
`DMA CONTROL
`2206
`
`l
`1 :
`:
`:
`~---- _____ I ______ --------------------r- ___ ~
`'f::""'2225
`SRAM
`CONTROLLER
`
`\~--------------------~
`
`-c. :::. •• ... '··2226
`
`DRAM
`CONTROLLER
`
`\--2214
`\~S_T_A_TU_s_I~ _________ J
`DATA
`~~
`2223
`
`FIG. 22
`
`
`
`
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 23 of 89
`
`US 7,673,072 B2
`
`Command
`buffer queue
`
`Command pointer
`
`Command pointer
`
`Command pointer
`
`~nse
`buffer queue
`
`Command
`buffer handle
`Status
`
`Command
`buffer handle
`status
`
`Command
`buffer handle
`Status
`
`Command buffers
`
`Command
`buffer handle
`rcp context
`identifier
`Command
`Data pointers
`
`Command
`buffer handle
`TCP context
`identifier
`Command
`Data pointers
`
`Command
`buffer handle
`rcp context
`identifier
`Command
`Data pointers
`
`FIG. 26
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 24 of 89
`
`US 7,673,072 B2
`
`31
`
`o
`
`ERR -
`RCV - - '
`XMf ----'
`
`RMISS
`
`Error bits are sent
`RCV has occured.
`Command has been rompleted
`
`Rev drop occured due to no buffers
`
`FIG. 27
`
`ISR
`IMR
`HBAR
`DBHR
`DBAR
`CBARO
`CBARI
`CBARl
`CBARJ
`CBAR4
`RBAR
`
`\..
`
`0x0
`Ol4
`OXS
`Ole
`OxIO
`()XI4
`()xIS
`OxIC
`000
`004
`0x28
`
`Interrupt status
`Interrupt Mask
`Header Buffer Address
`Data Buffer Handle
`Data Buffer Address
`Command Buffer Address XMTO
`Command Buffer Address XMTI
`Command Buffer Address XMT2
`Command Buffer Address XMTI
`Command Buffer A& RCY
`Response Buffer Address
`
`y
`
`FIG. 28
`
`)
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 25 of 89
`
`US 7,673,072 B2
`
`Mbuf
`
`Mbuf
`
`,---.
`
`Packet Desc
`
`Buffer Desc
`
`BufferDesc
`
`Data
`
`~ Data
`
`FIG. 29
`
`r-------,-(cid:173)
`I
`• •
`:
`•
`• • I l __
`
`ifuet
`
`Ifure
`
`ifaddr
`
`r - -
`
`inifaddr
`
`kaddr dl
`soc
`-
`OO:60:97:DB:9B:A6
`
`H
`
`'-----------.
`
`I
`
`I ~ ifaddr H
`
`sockaddrin
`
`192.100.1.2
`
`I
`
`1
`
`FIG. 30
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 26 of 89
`
`US 7,673,072 B2
`
`Filter Driver
`
`Microsoft
`TCPIIP
`Driver
`
`I
`
`I
`
`3COM
`Miniport
`Driver
`
`ATCP
`Driver
`
`I
`
`INIC
`~rt
`Driver
`
`FIG. 31
`
`Packet Desc
`
`Packet Desc
`
`BufferDesc
`
`BufferDesc
`
`BufferDesc
`
`Header
`Buffer
`
`TCP
`Packet
`
`ARP
`Frame
`
`Example of incoming TCP pkt
`
`Example of incoming ARP Frnme
`
`FIG. 32
`
`FIG. 33
`
`
`
`,---
`
`Addr
`Packet desc
`r - . . .
`mbuf
`
`~ Buffer desc
`. . .
`
`l-
`
`t-
`
`L
`
`~
`
`Packet
`
`Data
`Next9l
`
`Buffer
`J
`
`Header
`Buffer
`
`TCP Fast-path
`
`FIG. 34A
`
`-
`
`-
`
`~
`
`I -
`
`-
`
`Addr
`Packet desc
`. . .
`mbuf
`
`Addr
`Packet desc=O
`. . .
`
`mbuf
`
`~
`7Jl
`•
`~
`~
`~
`
`~ = ~
`
`t
`
`Buffer desc
`. . .
`
`Data
`~ Buffer
`
`Packet
`f
`Data
`Next
`
`Buffer
`
`Header
`Buffer
`
`---.
`
`t -
`
`Data
`Next=Q
`
`t-
`
`Buffer
`
`J
`Data
`Buffer
`
`~
`
`TCP Slow-path
`
`FIG. 34B
`
`ARP Frame
`
`FIG. 34C
`
`N
`
`~
`~ :-:
`N o .... o
`
`~
`
`('D
`('D
`
`rFJ =(cid:173)
`.....
`N
`-....l
`
`o ....
`
`QO
`\0
`
`d
`rJl
`-....l
`0..,
`
`-....l
`
`-....l w -=
`N = N
`
`
`
`u.s. Patent
`
`Mar. 2,2010
`
`Sheet 28 of 89
`
`US 7,673,072 B2
`
`Packe