`
`US 6,240,458 B1
`(10) Patent No:
`2) United States Patent
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`May29, 2001
`(45) Date of Patent:
`Gilbertson
`
`
`
`
`(75)
`
`
`
`
`
`
`
`
`
`
`(54) SYSTEM AND METHOD FOR
`PROGRAMMABLY CONTROLLING DATA
`
`
`
`TRANSFER REQUEST RATES BETWEEN
`
`
`
`
`DATA SOURCES AND DESTINATIONS IN A
`
`
`
`
`DATA PROCESSING SYSTEM
`
`
`
`
`
`
`
`Inventor: Roger Lee Gilbertson, Minneapolis,
`
`MN(US)
`
`
`
`
`
`
`(73) Assignee: Unisys Corporation, Blue Bell, PA
`(US)
`
`Subject to any disclaimer, the term of this
`
`
`
`
`
`
`
`
`
`
`patent is extended or adjusted under 35
`US.C. 154(b) by 0 days.
`
`
`
`
`(*) Notice:
`
`
`
`
`
`
`
`
`(21) Appl. No.: 09/218,211
`4.
`Filed:
`Dec. 22, 1998
`(22)
`
`
`
`
`
`
`
`
`
`
`
`(SL) Mts C0 cececeeeccccssesssessseeecceseessneeeseeeeesaee GO6F 15/16
`
`
`
`
`
`
`
`(52) US. Ch.
`ceecccccesssssssssseseee 709/232; 709/233; 709/234;
`
`
`
`709/235; 710/39; 710/60
`(58) Field of Search
`709/234, 200
`
`
`
`
`
`
`709222,252.243.207,238 335, 732
`
`
`
`
`
`
`
`733 40: 710/263 264. 244. 112 116.
`
`
`
`
`
`
`39. 40 ,52 58-60: 7150. 151: 3701395,
`
`
`
`
`
`
`
`
`,
`,
`,
`,
`,
`AL4
`
`(56)
`
`
`
`
`
`
`References Cited
`
`
`U.S. PATENT DOCUMENTS
`
`
`
`
`1/1985 Agrawalet al.
`4,493,021 *
`.
`
`
`
`5,136,718 *
`8/1992 Haydt .
`
`
`
`
`
`
`6/1993 Webb,Jr.et al.
`5,222,223 *
`9/1996 Guttag et al.
`5,560,030 *
`.
`
`
`
`
`3/1999 AbriU woeeeeeeeteeeeeeeeee 370/413
`5,881,065 *
`
`
`
`
`6,138,192 * 10/2000 Hauisatier ooo... ec ceeeeeeeee 710/100
`
`
`
`
`eo
`.
`
`
`
`cited by examiner
`
`
`
`.
`
`
`
`
`
`
`.
`.
`oo,
`
`
`
`
`Primary Examiner—Ario Etienne
`
`
`
`
`
`(74) Attorney, Agent, or Firm—Charles A. Johnson; Mark
`
`
`
`
`
`T. Starr; Altera Law Group
`(57)
`ABSTRACT
`
`
`A system and method for selectively controlling the inter-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`face throughput of data transfer requests from request
`sources to request destinations. The system and method
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`provide a mannerin whichthe flow of data transfer requests
`
`
`
`
`
`
`
`from request sources to request destinations are controlled.
`
`
`
`
`
`
`
`
`
`The data transfer requests from each of the request sources
`are temporarily stored for future delivery to its addressed
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`request destination. Delivery of the stored data transfer
`
`
`
`
`
`
`
`
`requests to the addressed request destinationis enabled
`
`
`
`
`
`according to a predetermined delivery priority scheme.
`Certain stored data transfer requests are identified to be
`
`
`
`
`
`
`
`
`
`selectively suspended from being prioritized and delivered
`
`
`
`
`
`
`
`to the addressed request destination. The identified data
`
`
`
`
`
`
`
`
`transfer requests are suspended from delivery for a definable
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`period of time. Upon expiration of the definable period of
`
`
`
`
`
`
`
`
`
`time, the suspended data transfer requests, as well as all
`
`
`
`
`
`
`
`
`
`other stored data transfer requests, are enabled for prioriti-
`
`
`
`
`
`
`
`zation and delivery in accordance with the predetermined
`
`
`
`delivery priority scheme.
`
`
`
`
`33 Claims, 14 Drawing Sheets
`
`
`
`WmsLLL
`100
`ye
`110A
`1108
`
`
`
`
`
`
`
`MEMORY STORAGE
`MEMORY STORAGE
`MEMORY STORAGE
`/
`
`
`
`
`
`
`
`UNIT
`UNIT
`UNIT
`
`
`
`(MSU)
`(MSU)
`(MSU)
`\
`
`
`
`\
`“ ~ _ __
`7
`
`13
`__
`=130N
`7130P 430C-
`ee =
`{—
`
`
`
`a130A [Pr__30FEE=a——|
`
`
`
`7300 == 130KSe 1308
`TT ae
`
`
`
`
`
`
`
`yi/ PROCESSING
`PROCESSING
`120B yy 120CHel10.
`
`\ epi|(POD) (POD) (POD)
`
`
`
`
`
`
`MODULE
`MODULE
`MODULE
`MODULE
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`—_-—
`
`
`
`
`
`
`—_
`
`
`
`/
`/
`
`
`
`
`
`
`
`
`
`
`
`
` INPUT/
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`MODULE
`
`(VO)
`
`
`
`
`
`
`
`
`OUTPUT
`MODULE
`(VO)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`SYMMETRICAL MULTIPROCESSING PLATFORM
`
`
`
`
`
`INPUT/
`
`OUTPUT
`
`MODULE
`
`(VO)
`
`
`
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`
`Sheet 1 of 14
`
`
`
`US 6,240,458 B1
`
`ANOWSAW
`soe!—resfaFt
`
`LINN
`
`——T
`
`
`
`ClCOE:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`—_—_———|————eeee—_—eeeeeeee———_————
`
`FINGOW
`
`FINGOW
`
`
`
`
`
`
`
`FINGOWJINGOWJ1INGOWAINGOWFINGOWJINGOW
`
`JINGOW||FINGOW
`LNdLNO||LAdLNO
`
`ALNdNI/LNdNI
`ONISSAOOUd0cONISSAOOUd
`(ov!)(ov!)
`
`
`FJINGOWFINGOW
`
`LAdLNOLAdLNO
`
`/LNANI/LAANI
`ADVYOLS
`
`(oY)(of)
`
`LAdLAOLNdLNO
`
`/LNdNI/LNdNI
`ONISSAOOUdVOC!
`
`LNdLNOLAALNAO
`
`/LAdNI/LNdNI
`ONISSS0Od
`
`
`
`
`
`
`
`~Holy5D0r|4or|F0bl
`
`
`
`
`
`
`
`
`
`001WHO4LV1dONISSSOOULLTINWTWOILSWWAS
`
`L‘Sls
`
`INTEL - 1007
`
`
`
`
`
`INTEL - 1007
`
`
`
`
`
`
`
`AYOWSW
`YHALSNTDYHaLSNTOYaLSMN19HALSMO
`
`
`
`COW)COW)COW)COW)
`
`
`
`
`AYOWAWAYOWSWN
`(aw)qsreANAND
`9rz—VVCTON|;--—JfTtl---
`
`
`|Vv.ivdOd|VOW)YATIOULNOO4_ssauqayv
`
`(NSW)LINNJOVHOLSAYOWSW
`
`
`
`AYOWSW=SS¢“]701907|a9|JOWLNOO
`
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 2 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`
`
`
`
`YVESSOUDVLV
`
`
`
`
`
`
`2S|
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`May29, 2001
`
`Sheet 3 of 14
`
`US 6,240,458 B1
`
`Ssaydqqv
`
`VivddOd
`
`anand
`
`VivdNSW
`
`94d
`
`nsw
`
`VivdNSW/SSayaav
`
`TOHLNODNOILONNS
`
`dgOd-OL-NSWAnand
`
`SSaYadqvV91907
`
`JOYLNOD
`
`Anand
`
`S4yVLVddOd
`vVivdd0d/SSAYqdqvV
`
`TOYLNOONOILONNA
`NSW-0L-dOdTWNOIS
`
`JOVAYALNINSW
`
`&Old
`
`TOXHLNOO
`
`QOd
`
`SSaydqdqv
`
`ANAND
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`U.S. Patent
`
`
`
`
`
`May29, 2001
`
`
`Sheet 4 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`dv‘Sis
`
`adjdja}/gagy;a;ady;qia
`
`NIALLNOILLONNA/SSSYOq0V(<———3WIL)
`Isanoauisanoau
`
`)9qqVvVv
`qY
`IsanoayIsanoay
`
`
`ONIANILVLVG(¢—_——INILL)
`ISanosyisanoay=LSANDAY
`VIVIVIVIVIVIVIV
`
`
`
`VvDis
`
`FIOADHOOT|
`
`ATOADMOOT|
`
`
`
`M9010WALSAS
`
`XN007
`
`NOcOV
`
`
`
`{SandaydOd
`
`TWNOIS
`
`SSNAV
`
`XMvOr
`
`90F/
`
`
`
` 80+/MOOTOWALSAS
`
`LS4N0sYViva
`
`ov7TWNOIS
`
`( <—— TWNOIS)
`
`( <—— TWNOIS)
`
`TWNOISVIVd
`
`SANIT
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`May29, 2001
`
`Sheet 5 of 14
`
`US 6,240,458 B1
`
`TWNODIS1SSNOSYVLIVd
`
`
`ANANDHO1LSS/YOLS
`
`LSANDAYOd
`
`VivdSsHdav
`TWNOIS
`Osu=OAY
`
`ONILNOYAINVAVivd
`AYOLS
`ADVSSAWN
`
`IS01
`
`
`
`
`
`
`
`O1D01ANANDADVSSAW
`
`JYHOLS
`
`ADVSSAW
`
`
`
`
`
`
`
`
`
`08}vivawiWdadacoddqd0d4N
`
`
`uldWd O03YSIVNSISGIOH
`
`VivdWivddad005ddOd4NV
`
`QrivAYldOIAdN-OAY~STVNDISGIOH
`
`
`
`
`
`|91907SNANDsssyaav
`
`
`
`
`
` S‘SlsQ9IDO7ANANDSSayqav
`
`009¢
`
`1901
`
`Viva
`
`QNas
`
`
`
`suav¥3q0040
`
`oe
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 6 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Alrty
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`635Aean=635E
`
`
`
`
`530A
`
`
`
`
`| DATA VALID
`| ROUTING LOGIC
`
`
`
`
`
`
`
`
`
`602)
`
`
`
`CONTROL
`
`LOGIC
`
`DECODED
`
`iv
`
`
`
`
`
`
`MESSAGE
`
`INDICATOR
`
`
`
`
`STORE:
`
`DATA
`INDICATOR
`
`
`
`
`POD
`REQUEST
`
`SIGNAL
`
`
`
`304
`
`
`
`DATA VALID ROUTING LOGIC
`
`
`
`
`
`FIG. 6
`
`
`DATA
`
`REQUEST
`
`SIGNAL
`
`537
`
`
`
`2600
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`May29, 2001
`
`Sheet 7 of 14
`
`US 6,240,458 B1
`
`ANANDHOLAS/AHOLS=HALNIOdTWNDISGIASTYNOISA/VTWNODIS
`
`
`ssayaayOLE
`
`
`
`
`
`AVESTYNOISHOLWOICNITiavNa GIOHINO
`
`ATENTTPCHOLOSIISDol
`ssayqayNOLLONNA
`
`JI901STYNDIS40l4
`ssauday3d0030|aaq00Na
`AYHYNOWLONN|cosuaay
`
`
`
`
`VivaSTYNOISavVivaaHO1S3009304S3NO3YdOdGOH
`
`
`
`ANANDHOLS/AHOLSHOLS4/SYOLSYAGOONA
`96X9104..|ALIHOIYd
`
`ssauaay91907anano
`
`Z0€Gpsplspoe
`HAGOONABCL
`ssayqavviva
`057oe
`——e———~~.————-—--+---7
`
`
`TOXLNOD|ss¥dAday
`dp0Z=
`LIa91907ELSeitn
`VOLSLIMAVOVaY
`Zg
`
`STYNOIS
`
`Ld
`
`—
`
`9S
`
`qaqgoo3d
`QSq00N3
`
`
`
`JOULNODaNano
`
`JOYLNOO
`
`HOLF4/AHO1S
`
`92
`
`STWNOIS
`
`
`
`NOILO31aSanand
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 8 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`esais”
`
`
`
`
`d0de8NOISNVdX3NSWqoes
`
`0028NOISNVdxX4NSWD0€8
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`‘'he)—TnaozsNOISNVdX4NSWss:0€Bv0e8NOISNVdX4NSWvores
`
`
`
`
`AVYAVYf|AVYAVY||F9VHOLSANOLOSHIG||SDVHOLSANOLOSHIC||||SOVHOLSAHOLOaUIG]ANOLOSHId||JOVHOLS
`
`
`
`
`
`
`
`JFOVYHOLSAHOLOAYIOJAOVYOLSAYOLOAYIG
`VivdLOYVivdVivdWLO/ddvViva
`vivamoedv=wlWavivawodv=viva}
`
`
`
`(TOW)HSLSMIOANOWSW02
`
`
`
`ADVYOLSAYOLOAYIGANVYOLSAWOLOAYIC
`
`
`
`
`
`
`
`
`
`g‘DI4
`
`
`
`AVUHVAVY
`
`AVYAVYYY
`
`
`
`
`
`ae eee ewewowecocend
`
`
`
`
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`
`
`
`U.S. Patent
`
`May29, 2001
`
`Sheet 9 of 14
`
`US 6,240,458 B1
`
`
`
`8068008Ot£8eeeeee28088eea8082800082282222Se2228sBeeewweTeeeceweseseseres
`
`
`
`HALSNTOAUOWAWbd°
`
`
`
`Y3ALSN1OAHOWSW
`
`<ILLTS
`
`€SNANOV1VdGOd
`
`0SNANDV1VdGOd
`
`
`
`LAdNILAdNI
`
`
`
`LLIMMALLIGM
`
`
`
`b3NAND04NaNO
`
`706
`
`JOVAYSLNISS
`
`ANANDVIVd
`
`LOANNOOTHSLNI
`
`JI901
`
`Lbe
`
`VOL
`
`py£06
`
`JOVIUSNI
`
`206
`
`IAdNI
`
`ALLIWMA
`
`LAdNI
`
`SLIGM
`
`b3NANO
`
`0anand
`
`6‘Sls
`
`0S¢
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 10 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`ATIEVYENSISNOO
`
`
`
`MO14dLSANDAY
`
`YATIOWULNOOD
`
`
`
`NVOSSINVNAG
`
`8001
`
`
`
`Ol‘Sls
`
`Oc}
`
`isanoay
`
`
`
`MOT4
`
`
`
`HOLAS/AYOLS
`
`anand
`
`
`
`
`
`SA9OYNOSAYNOILVNILSAG
`
`
`
`HO135/SYHOLS
`
`ALIWOldd
`
`
`9ID07
`
`
`91907SNANO
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 11 of 14
`
`
`
`US 6,240,458 B1
`
`
`
`1188|AB STRESS Y 1100
`
`
`
`
`
`
`
`
`HOLD[7:0]
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`COUNTER
`9:0]
`
`1122
`
`
`—j—-
`reG{
`AB?
`
`L-4490,
`
`|
`
`—"s —s w5 = —h wCO
`—
`
`
`
`
`
`
`
`198)1140 |aIC1146| fe
`I! REG
`REShileREG ie[re
`
`
`
`
`Wiwh ey
`|| ABE I (=|
`
`
`w2 foOSjj
`
`
`
`
`
`
`
`
`
`FIG. 11
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 12 of 14
`
`
`US 6,240,458 B1
`
`
`
`IDENTIFY DESTINATION RESOURCES
`
`
`
`
`
`
`
`
`
`DESTINATION RESOURCE
`
`
`
`
`CONFIGURE MODE REGISTERS FOR EACH
`
`
`
`
`
`
`
`DESTINATION RESOURCE
`
`
`
`
`
`SELECT MASKING WAVEFORM BASED ON
`
`
`
`
`CONFIGURED MODE FOR EACH
`
`
`
`
`
`REMOVAL ACTIVITY
`
`
`
`
`SELECT ONE OR MORE DESTINATION
`
`
`RESOURCES TO SUSPEND REQUEST
`
`
`
`
`
`
`1200
`
`
`
`1202
`
`
`
`1204
`
`
`
`1206
`
`
`
`
`
`
`
`
`
`
`
`
`SUSPEND REQUEST REMOVALACTIVITY
`
`
`
`FOR SELECTED DESTINATION
`
`
`RESOURCES ACCORDING TO
`
`
`
`
` 1208
`RESPECTIVE MASKING DUTY CYCLES
`
`
`
`
`
`
`
`
`FIG. 12
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 13 of 14
`
`
`US 6,240,458 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`FIRST POD ASSIGNED TO ROUTINELY
`
`
`
`
`PERFORM LONG DATA TRANSFER
`
`
`
`
`OPERATIONS TO DESTINATION
`
`
`
`RESOURCE A INITIATES DATA
`
`TRANSFER
`
`1300
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`/O MODULE
`
`
`
`
`ASSOCIATED WITH
`
`
`
`
`ENABLE FIRST POD TO
`
`
`
`
`SECOND POD INITIATE
`
`
`
`
`
`INITIATE ANOTHER DATA
`
`
`
`
`
`DATA TRANSFER TO
`
`
`
`TRANSFER
`
`DESTINATION
`
`RESOURCEA?
`
`
`
`
`
`
`
`
`VO DATA
`\_
`
`
`
`
`TRANSFER HAVE
`
`
`
`ASSOCIATED TIMEOUT
`
`RESTRICTION
`
`
`?
`
`
`
`
`
`
`
`
`
`YES
`
`
`
`1306
`
`
`
`SELECTED MODE
`
`
`
`DYNAMICALLY SCAN DESIRED
`
`
`
`MODE INTO PROGRAMMABLE
`
`
`REGISTER CORRESPONDING TO
`
`
`FIRST POD
`
`
`
`
`DISABLE FURTHER DATA
`
`
`
`
`TRANSFERS FROM FIRST POD TO
`
`
`DESTINATION RESOURCE A FOR
`
`
`PERIOD DESIGNATED BY
`
`
`
`FIG. 13
`
`
`1310
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`U.S. Patent
`
`
`
`
`May29, 2001
`
`
`
`
`Sheet 14 of 14
`
`
`US 6,240,458 B1
`
`
`
`
`
`
`
`
`ENABLE A SELECTED ONE OF THE STORE/FETCH
`
`
`
`
`
`QUEUES TOINITIALIZE THE CORRESPONDING MSU
`
`
`
`
`PRELOAD A NUMBEROF STORE/FETCH REQUESTS
`
`
`
`INTO REMAINING STORE/FETCH QUEUES OF
`
`
`CORRESPONDING MSU
`
`
`
`
`
`
`
`
`SET INDEFINITE DATA TRANSFER BLOCK ON
`
`
`REMAINING STORE/FETCH QUEUES
`
`1400
`
`
`
`1402
`
`
`
`1404
`
`
`
`
`
`START MSU CLOCKS
`
`
`
`1406
`
`
`
`
`EXECUTE MSU INITIALIZATION SEQUENCE
`
`
`
`
`
`
`
`
`
`
`DYNAMICALLY SCAN BLOCK RELEASE INTO PROGRAMMABLE
`
`
`
`
`REGISTERS CORRESPONDING TO DESTINATION RESOURCE OF
`
`
`
`REMAINING STORE/FETCH QUEUES
`
`1410
`
`
`
`1408
`
`
`
`
`
`
`
`RELEASE PRELOADED REQUESTS TO DOWNLINE MSU
`
`
`LOGIC STRUCTURES
`
`
`
`1412
`
`
`
`1414
`
`
`
`
`
`1416
`
`
`
`
`
`STOP MSU CLOCKS
`
`
`
`
`
`ASCERTAIN MSU STATE
`
`
`
`
`
`
`
` 1418
`
`
`
`
`
`
`
`
`
`
`
`USER-DEFINED
`
`
`
`
`
`TIME PERIOD EXPIRE
`
`?
`
`
`
`YES
`
`
`
`
`
`
`NO
`
`
`
`
`
`
`
`
`
`
`
`
`
`COMPARE ACTUAL MSU STATE TO
`EXPECTED MSU STATE
`
`
`
`
`FIG. 14
`
`
`
`1420
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`1
`SYSTEM AND METHOD FOR
`
`
`
`PROGRAMMABLY CONTROLLING DATA
`
`
`
`
`
`
`
`TRANSFER REQUEST RATES BETWEEN
`DATA SOURCES AND DESTINATIONS IN A
`
`
`
`
`DATA PROCESSING SYSTEM
`
`
`
`CROSS-REFERENCE TO OTHER PATENT
`
`
`APPLICATIONS
`
`
`
`
`
`
`
`
`
`
`
`The following co-pending patent application of common
`
`
`
`
`
`assignee contains some commondisclosure:
`
`
`
`
`
`
`
`“Multi-Level Priority Control System And Method For
`
`
`
`
`
`
`Managing Concurrently Pending Data Transfer Requests”,
`
`
`
`
`
`
`
`
`filed concurrently herewith with assigned Ser. No. 09/218,
`
`
`
`
`
`
`377, which in incorporated by reference in its entirety;
`
`
`
`
`
`
`
`“Transfer Request Selection Method And Apparatus For
`
`
`
`
`
`Symmetrical Multiprocessor Systems”, filed concurrently
`
`
`
`
`
`
`
`
`herewith with assigned Ser. No. 09/218,210, whichis incor-
`
`
`
`
`
`porated by reference in its entirety; and
`
`
`
`
`
`
`
`“Queueing Architecture And Control System For Data
`
`
`
`
`
`Processing System Having Independently-Operative Data
`
`
`
`
`
`
`
`
`
`And Address Interfaces”, Ser. No. 09/096,822,filed Jun. 12,
`
`
`
`
`
`
`
`
`1998, which is incorporated herein by reference in its
`
`entirety.
`
`
`
`FIELD OF THE INVENTION
`
`
`
`
`
`
`
`
`
`This invention relates generally to data transfer request
`
`
`
`
`
`
`
`management in data processing systems, and more particu-
`
`
`
`
`
`
`
`
`larly to an interface and programmable interface control
`
`
`
`
`
`
`
`
`system and method for selectively providing, and control-
`
`
`
`
`
`
`
`
`
`ling the rate of, data transfer requests to destination
`
`
`
`
`
`
`
`
`resources, thereby providing the ability to manipulate data
`
`
`
`
`
`
`
`throughput under normal operating conditions, and to pro-
`
`
`
`
`
`
`vide a means for performing transaction processing testing.
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`Data processing systems generally include multiple units
`
`
`
`
`
`
`
`such as processing units, memory units, input/output units,
`
`
`
`
`
`
`
`
`
`and the like, which are interconnected over one or more
`
`
`
`
`
`
`
`
`system interfaces. The interfaces provide for the transfer of
`
`
`
`
`
`
`
`
`
`digital signals between the units. Since many of the opera-
`
`
`
`
`
`
`
`
`tions within data processing systems involve suchtransfers,
`
`
`
`
`
`
`
`
`
`the efficiency of the interfaces has a major impact on the
`
`
`
`
`
`
`overall performance of the data processing system.
`
`
`
`
`
`
`
`Manyconventional interfaces used within data processing
`
`
`
`
`
`
`
`
`systems have several types of signal lines, including data
`
`
`
`
`
`
`
`
`
`lines for transferring data signals, and address lines for
`
`
`
`
`
`
`
`transferring address signals. The address lines generally
`
`
`
`
`
`
`
`
`
`provide information indicative of the type of request, and
`
`
`
`
`
`
`indicate a unit and/or a particular addressable
`further
`resource associated within the unit that is involved with the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`request. The data lines provide data signals which are
`
`
`
`
`associated with the request.
`
`
`
`
`
`
`
`
`
`Requests for data transfers may occurat a faster rate than
`
`
`
`
`
`
`
`
`the memory and associated cache coherency logic can
`
`
`
`
`
`
`
`sustain. A buffering technique may be used to queue such
`
`
`
`
`
`
`
`
`requests until they can be processed. However, the queuing
`function can sometimesresult in inefficient and discrimina-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tory request servicing.
`In some cases, one processor’s
`
`
`
`
`
`
`
`requests may be repeatedly processed, while another’s are
`
`
`
`
`
`
`
`left relatively unattended. In other cases, a processor having
`
`
`
`
`
`
`
`
`relatively few requests may needlessly tie up system
`
`
`
`
`
`
`resources by receiving unnecessary request service polls.
`
`
`
`
`
`
`
`
`These situations can reduce available request bandpass, and
`
`
`
`
`
`
`
`
`increase the probability of request stalling or request lock-
`
`
`
`
`
`
`
`
`
`out. To address this issue,
`the buffering technique may
`
`10
`
`15
`
`
`
`20
`
`25
`
`
`
`30
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`
`
`US 6,240,458 B1
`
`
`2
`
`
`
`
`
`
`
`
`include a priority schemeto output the data transfer requests
`
`
`
`
`
`
`
`according to a priority assigned to each of the data transfer
`
`
`
`
`
`
`
`
`requests. One priority scheme knownin the art is known as
`
`
`
`
`
`
`
`
`a “fixed” request priority scheme. Each requester is assigned
`
`
`
`
`
`
`
`
`
`a fixed priority value, and requests are handled according to
`
`
`
`
`
`
`
`
`this associated priority value. Those requests having a high
`
`
`
`
`
`
`
`
`
`fixed priority value are always handled prior to those having
`
`
`
`
`
`
`
`relatively low priority values. Another request priority
`
`
`
`
`
`
`
`scheme is referred to as “snap-fixed”, where input request
`
`
`
`
`
`
`activity is continually or periodically polled. This results in
`
`
`
`
`
`
`
`a captured “snapshot”of the request activity at a given time.
`
`
`
`
`
`
`
`
`All of the captured requests are processed in a fixed order
`
`
`
`
`
`
`
`
`until all requests in the snapshot have been processed, at
`
`
`
`
`
`
`
`which time a new snapshotis taken. A “simple rotational”
`
`
`
`
`
`
`
`priority scheme involves changing the requester priority on
`
`
`
`
`
`
`
`
`
`a periodic basis. For example, the requester priority may be
`
`
`
`
`
`
`changed whenever a request is granted priority. Requester
`
`
`
`
`
`
`
`
`(N-1) movesto priority level (N), requester (N) moves to
`
`
`
`(N+1), and so forth.
`
`
`
`
`
`
`
`
`Regardless of the priority scheme used, there may be
`
`
`
`
`
`
`
`times when the implementedpriority scheme inhibits execu-
`
`
`
`
`
`
`
`tion of a desired system operation. For example, testing of
`
`
`
`
`
`
`
`a complex multiprocessing system having multiple data
`
`
`
`
`
`
`
`
`transfer sources and multiple data transfer destinations can
`
`
`
`
`
`
`
`be incredibly complicated, particularly where test programs
`must be written to simulate transaction “stress” situations.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Such a transaction stress situation may occur during normal
`
`
`
`
`
`
`
`operation where some resources, like memory, are suddenly
`
`
`
`
`
`
`
`
`inundated with pending data transfer requests. When this
`
`
`
`
`
`
`
`
`occurs, memory response times may be reduced, causing the
`
`
`
`
`
`
`
`
`data transaction queuestofill. The requesting modules must
`
`
`
`
`
`
`
`
`
`be able to accommodate this situation to avoid queue
`
`
`
`
`
`
`
`overrun problems,and it is therefore important to be able to
`
`
`
`
`
`
`
`
`simulate and test
`these conditions. Further,
`the memory
`
`
`
`
`
`
`
`
`
`resources must be able to manage and absorb the high
`
`
`
`
`
`
`
`
`volumeof sudden requesttraffic and properly respondto the
`
`
`
`
`
`
`
`requesting modules. Again, these situations require thorough
`
`testing.
`
`
`
`
`
`
`
`In order to prepare test programs to simulate these stress
`
`
`
`
`
`conditions, a detailed knowledge of the entire hardware
`
`
`
`
`
`implementation would be required in order to predict the
`
`
`
`
`
`
`direct effect on system hardware produced bytest program
`
`
`
`
`
`
`
`stimulus. The time, required resources, complexity and cost
`
`
`
`
`
`
`of preparing such test programsis prohibitive.
`
`
`
`
`
`
`
`
`
`It would therefore be desirable to provide an efficient
`
`
`
`
`
`
`
`
`arrangement and method that allows data transfer request
`
`
`
`
`
`
`
`queues to be controlled, or “throttled”, by way of simple
`
`
`
`
`
`
`user-defined parameters. Implemented priority schemes can
`
`
`
`
`
`
`
`be maintained, but can be selectively bypassed to perform
`
`
`
`
`
`
`
`stress tests, or to accommodate peculiar situations which
`
`
`
`
`
`
`
`
`might arise during normal operation. The present invention
`
`
`
`
`
`
`
`
`provides such a solution, and provides these and other
`
`
`
`
`
`
`
`advantages and benefits over the prior art.
`SUMMARYOF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`The present invention relates to a system and method for
`
`
`
`
`
`
`
`selectively controlling the interface throughputof data trans-
`
`
`
`
`
`
`
`fer requests from request sources to request destinations,
`
`
`
`
`
`
`
`thereby providing the ability to manipulate data throughput
`
`
`
`
`
`
`
`under normal operating conditions, and to provide a means
`
`
`
`
`
`for performing transaction processing testing.
`
`
`
`
`
`
`
`In accordance with one embodimentof the invention, a
`
`
`
`
`
`
`
`
`method is provided for controlling the flow of data transfer
`
`
`
`
`
`
`
`requests from various request sources to various request
`
`
`
`
`
`
`
`destinations. Each data transfer request is a request for an
`
`
`
`
`
`
`
`addressed one of the request destinations to supply a data
`
`
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`US 6,240,458 B1
`
`
`3
`
`
`
`
`
`
`
`
`segment to the requesting source. The data transfer requests
`
`
`
`
`
`
`
`
`
`from each of the request sources are temporarily stored for
`
`
`
`
`
`
`
`future delivery to its addressed request destination. Delivery
`
`
`
`
`
`
`
`
`of the stored data transfer requests to the addressed request
`
`
`
`
`
`destination is enabled according to a predetermined delivery
`
`
`
`
`
`
`
`
`priority scheme. Certain ones of the stored data transfer
`
`
`
`
`
`
`
`
`requests are identified to be selectively suspended from
`
`
`
`
`
`
`
`
`being prioritized and delivered to the addressed request
`
`
`
`
`
`
`
`
`destination. These identified data transfer requests are sus-
`
`
`
`
`
`
`
`
`pended from delivery for a definable period of time. During
`
`
`
`
`
`
`
`
`this time, the destination addressed by the suspended data
`
`
`
`
`
`
`
`
`
`transfer requests will not receive any of these requests. Upon
`
`
`
`
`
`
`
`
`
`expiration of the definable period of time, the suspended
`
`
`
`
`
`
`
`
`data transfer requests, as well as all other stored data transfer
`
`
`
`
`
`
`
`
`requests, are enabled for prioritization and delivery in accor-
`
`
`
`
`
`
`
`dance with the predetermined delivery priority scheme. In
`
`
`
`
`
`
`
`
`
`this manner, the suspended data transfer requests will gain
`
`
`
`
`
`
`
`
`priority during the period of suspension, and will thereafter
`
`
`
`
`
`
`
`be provided to the destination according to their respective
`
`priorities.
`
`
`
`
`
`
`
`In accordance with another embodimentof the invention,
`
`
`
`
`
`
`
`
`
`a methodis provided for controlling the flow of data transfer
`
`
`
`
`
`
`requests during normal system operations of a multiprocess-
`
`
`
`
`
`
`
`
`
`ing computing system that has multiple request sources that
`
`
`
`
`
`
`
`provide data transfer requests to multiple request destina-
`
`
`
`
`
`
`
`
`tions. The data transfer requests are prioritized according to
`
`
`
`
`
`
`
`
`a predetermined request dispatch priority scheme. Each data
`
`
`
`
`
`
`
`
`
`transfer request is a request for an addressed one of the
`
`
`
`
`
`
`request destinations to supply a data segment to a respective
`
`
`
`
`
`
`
`one of the request sources. The method includes periodically
`
`
`
`
`
`
`
`performing first data transfer operations between a first
`
`
`
`
`
`
`
`request source and a targeted request destination. A second
`
`
`
`
`
`
`
`data transfer operation is initiated between a second request
`
`
`
`
`
`
`
`
`source and the targeted request destination, wherein the
`
`
`
`
`
`
`
`
`second data transfer operation is subject
`to a response
`
`
`
`
`
`
`
`
`timeout
`limitation. The first data transfer operations are
`
`
`
`
`
`
`
`suspended for a user-defined period upon recognition of
`
`
`
`
`
`
`
`
`
`initiation of the second data transfer operation, and the
`
`
`
`
`
`
`
`
`second data transfer operations are enabled during the
`
`
`
`
`
`
`
`user-defined period. Upon expiration of the user-defined
`
`
`
`
`
`
`
`
`
`
`period, both the first and second data transfer operations are
`
`
`
`
`
`
`
`enabled in accordance with the predetermined request dis-
`
`
`
`patch priority scheme.
`
`
`
`
`
`
`
`In accordance with yet another embodiment of the
`
`
`
`
`
`
`
`invention, a method is provided for controlling the flow of
`
`
`
`
`
`
`
`data transfer requests during offline testing of a multipro-
`
`
`
`
`
`
`
`cessing computing system having a plurality of request
`
`
`
`
`
`
`
`
`sources capable of providing data transfer requests to a
`
`
`
`
`
`
`
`plurality of request destinations in accordance with a pre-
`
`
`
`
`
`
`
`determined request dispatch priority scheme. The multipro-
`
`
`
`
`
`
`
`cessing computing system including a main storage unit
`
`
`
`
`
`
`
`
`having multiple data transfer queues that operate in parallel
`
`
`
`
`
`
`
`
`
`to temporarily store the data transfer requests from the
`
`
`
`
`
`
`
`request sources to the request destinations. The method
`
`
`
`
`
`
`
`
`
`includes selecting a first of the plurality of data transfer
`
`
`
`
`
`
`
`
`queuesto initialize the memory in the main storage unit. A
`
`
`
`
`
`
`
`
`number of known data transfer requests are loaded into
`
`
`
`
`
`
`
`
`second onesof the plurality of data transfer queues, wherein
`
`
`
`
`
`
`
`
`the second onesof the data transfer queues comprise at least
`
`
`
`
`
`
`
`
`
`one of the remaining ones of the data transfer queues not
`
`
`
`
`
`
`
`selected to initialize the memory. Data transfer operations
`
`
`
`
`
`
`
`
`
`are prohibited from the second data transfer queues for a
`
`
`
`
`
`user-defined period. A memory initialization sequence is
`
`
`
`
`
`
`
`
`
`
`executed via the first data transfer queue. The data transfer
`
`
`
`
`
`
`
`
`
`operations are enabled from the second data transfer queues
`
`
`
`
`
`upon expiration of the user-defined period.
`
`
`
`
`
`
`
`In accordance with another aspect of the invention, a data
`
`
`
`
`
`
`
`
`transfer request interface circuit is provided for use in a
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`60
`
`65
`
`
`4
`
`
`
`
`
`
`least one
`multiprocessing computing system having at
`
`
`
`
`
`
`
`
`request source to provide data transfer requests to at least
`
`
`
`
`
`
`
`one request destination. The interface circuit
`includes a
`
`
`
`
`
`
`
`queuing circuit coupled to each of the request sources to
`
`
`
`
`
`
`
`
`receive and temporarily store the data transfer requests. A
`
`
`
`
`
`
`
`
`
`priority logic circuit is coupled to the queuing circuit to
`
`
`
`
`
`
`
`
`prioritize a sequence by which the stored data transfer
`
`
`
`
`
`
`
`
`
`requests are output from the queuing circuit. The priority
`
`
`
`
`
`
`logic operates in accordance with a predetermined priority
`
`
`
`
`
`
`
`algorithm. A maskingregister is coupledto the priority logic
`
`
`
`
`
`
`
`circuit to mask predetermined stored data transfer requests
`
`
`
`
`
`
`
`
`
`from being considered by the priority logic circuit
`in
`
`
`
`
`
`
`
`response to a masking signal pattern provided to the mask-
`
`
`
`
`
`
`
`
`
`ing register. In this manner, the predetermined ones of the
`
`
`
`
`
`
`
`
`
`stored data transfer requests are retained in the queuing
`
`
`
`
`
`
`
`
`
`circuit while the remaining stored data transfer requests are
`
`
`
`
`
`
`
`
`allowed to be prioritized and output from the queuing
`
`
`
`
`
`
`circuit. A configurable request flow controller is coupled to
`
`
`
`
`
`
`
`
`the masking register to generate the masking signal pattern
`
`
`
`
`
`
`in response to user-defined parameters. The user-defined
`
`
`
`
`
`
`
`
`parameters define at least which of the stored data transfer
`
`
`
`
`
`
`
`
`requests are to be masked by the masking register, and the
`
`
`
`
`
`
`
`
`
`
`duration to which the masking signal pattern is to be
`sustained.
`
`
`
`
`
`
`
`
`Still other objects and advantages of the present invention
`
`
`
`
`
`
`
`
`
`will become readily apparent to those skilled in this art from
`
`
`
`
`
`
`
`the following detailed description. As will be realized, the
`
`
`
`
`
`
`
`invention is capable of other and different embodiments, and
`
`
`
`
`
`
`
`
`its details are capable of modification without departing
`
`
`
`
`
`
`
`
`
`from the scope and spirit of the invention. Accordingly, the
`
`
`
`
`
`
`drawing and description are to be regardedasillustrative in
`
`
`
`
`nature, and notasrestrictive.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`The invention is described in connection with the embodi-
`
`
`
`
`
`
`
`
`
`
`
`ments illustrated in the following diagrams.
`
`
`
`
`
`
`FIG. 1 is a block diagram of a Symmetrical Multi-
`
`
`
`
`
`
`Processor (SMP) System Platform in which the principles of
`
`
`
`
`
`the present invention may be applied;
`
`
`
`
`
`
`FIG. 2 is a block diagram of one embodiment of a
`
`
`
`
`Memory Storage Unit (MSU);
`
`
`
`
`
`FIG. 3 is a block diagram of bi-directional MSUInterface
`
`
`
`
`
`
`(MD) and associated interface control logic;
`
`
`
`
`FIG. 4A is a timing diagram of a POD-to-MSUaddress
`
`transfer;
`
`
`
`
`
`
`FIG. 4B is a timing diagram of a POD-to-MSU data
`
`transfer;
`
`
`
`
`
`
`FIG. 5 is a block diagram of one embodiment of the
`
`
`
`Address Queue Logic;
`
`
`
`
`
`
`FIG. 6 is a block diagram of one embodimentof the Data
`
`
`
`Valid Routing Logic;
`
`
`
`
`
`
`FIG. 7 is a block diagram of one embodiment of the
`
`
`
`Store/Fetch Queue Logic;
`
`
`
`
`
`
`FIG. 8 is a block diagram of one embodiment of a
`
`
`
`
`
`
`
`MemoryCluster depicting the various destination resources
`
`
`contained therein;
`
`
`
`
`
`
`FIG. 9 is a block diagram of one embodiment of an MSU
`
`
`
`
`
`
`
`
`illustrating the availability of PODs being the resource
`
`destination;
`
`
`
`
`
`
`
`FIG. 10 is a block diagram illustrating one manner of
`
`
`
`
`
`
`
`configurably controlling the flow of data transfer requests to
`
`
`
`
`
`
`
`particular destination resources in accordance with the
`
`
`present invention;
`
`
`
`
`
`FIG. 11 is a schematic diagram illustrating one embodi-
`
`
`
`
`
`ment of a Configurable Request Flow Controller in accor-
`
`
`
`
`
`dance with the present invention;
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`INTEL - 1007
`
`INTEL - 1007
`
`
`
`
`
`US 6,240,458 B1
`
`
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`
`5
`
`
`
`
`
`
`FIG. 12 is a flow diagram illustrating one embodimentof
`
`
`
`
`
`
`
`
`
`a method for programmably controlling the flow of data
`
`
`
`
`
`
`
`transfer requests in accordance with the present invention;
`
`
`
`
`
`
`FIG. 13 is a flow diagram of an example procedure used
`
`
`
`
`
`
`
`
`during normal operation of the system using the principles
`
`
`
`
`
`of the present invention; and
`
`
`
`
`
`
`FIG. 14 is a flow diagram of an example procedure used
`
`
`
`
`
`
`
`
`during offline testing of the system using the principles of
`
`
`
`the present invention.
`DETAILED DESCRIPTION OF THE
`
`
`ILLUSTRATED EMBODIMENTS
`
`
`
`
`
`
`
`
`
`Generally, the present invention provides a system and
`
`
`
`
`
`
`
`
`method for controlling the throughput of data transfer
`
`
`
`
`
`requests through a source-to-destination interface. The
`
`
`
`
`
`
`invention provides programmable control of the removal of
`
`
`
`
`
`
`
`data transfer requests from queuing structures to destination
`
`
`
`
`
`
`
`
`resources such as memory, and further provides for control
`
`
`
`
`
`
`
`
`
`
`of the rate at which requests are removed from the queuing
`
`
`
`
`
`
`structures, including a complete suspension of data transfers
`
`
`
`
`
`
`
`for a user-defined period. Control of request removalratesis
`
`
`
`
`
`
`
`dynamically configurable, allowing flexibility and ease of
`
`
`
`
`
`
`
`
`use. The present invention facilitates stress testing of both
`
`
`
`
`
`
`
`
`the requester and request receiver, and is available for
`
`
`
`
`
`
`
`system fine-tuning during normal (non-test) system execu-
`tion as well as for extensive offline test execution.
`
`
`
`
`
`
`
`
`
`
`
`
`
`While the present invention is particularly advantageous
`
`
`
`
`
`
`
`in the context of a Symmetrical Multi-Processor (SMP)
`
`
`
`
`
`environment as described below, it will be appreciated by
`
`
`
`
`
`
`
`
`
`those skilled in the art that the invention is equally appli-
`
`
`
`
`
`
`cable to other computing environments requiring manage-
`
`
`
`
`
`
`
`ment of memory,
`I/O, or other
`transaction processing
`
`
`
`
`
`
`requests. Therefore,
`the particular SMP environment
`
`
`
`
`
`
`
`described in the following figures is provided forillustrative
`
`
`
`
`
`
`purposes and to provide a full operational understanding of
`
`
`
`
`
`
`
`
`the invention; however the invention is not limited thereto.
`
`
`
`
`
`
`FIG. 1 is a block diagram of a Symmetrical Multi-
`
`
`
`
`
`
`Processor (SMP) System Platform in which the principles of
`
`
`
`
`