USOO8565033B1
`
`(12) United States Patent
`Manohararajah et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,565,033 B1
`Oct. 22, 2013
`
`(54) METHODS FOR CALIBRATING MEMORY
`INTERFACE CIRCUITRY
`
`(75) Inventors: Valavan Manohararajah, Scarborough
`(CA); Ivan Blunno, Woodbridge (CA);
`Ryan Fung, Mississauga (CA); Navid
`Azizi, Markham (CA)
`(73) Assignee: Altera Corporation, San Jose, CA (US)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 224 days.
`(21) Appl. No.: 13/149,562
`(22) Filed:
`May 31, 2011
`(51) Int. Cl.
`GITC 700
`(52) U.S. Cl.
`USPC ... 365/193:365/129: 365/149; 365/189.011;
`365/189.14; 365/189.05:365/230.06; 365/233.1;
`365/233.13: 365/233.16:365/233.17
`(58) Field of Classification Search
`USPC ........ 365/129, 149, 189.011, 189.14, 189.05,
`365/193,230.06, 233.1, 233.13, 233.16,
`365/233.17
`See application file for complete search history.
`
`(2006.01)
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`7.231,536 B1* 6/2007 Chong et al. .................. T13/400
`7,475,315 B1
`1/2009 Natarajan et al.
`8,117,483 B2 * 2/2012 Welker et al. ................. T13,500
`2007/0239379 A1
`10/2007 Newcomb et al.
`1/2009 Chang et al.
`2009/0031091 A1
`5/2010 Onishi .......................... 365, 193
`2010, 0124131 A1*
`
`2010, 0180 143 A1
`2010/0315119 A1
`2012/0176850 A1*
`2012,02291.86 A1*
`
`7/2010 Ware et al.
`12/2010 Welker et al.
`7/2012 Do et al. ....................... 365, 193
`9/2012 Baba ............................. 327, 161
`
`FOREIGN PATENT DOCUMENTS
`
`WO WO 2011061875 A1 *
`
`5, 2011
`
`OTHER PUBLICATIONS
`
`Burney et al., U.S. Appl. No. 1 1/488, 199, filed Jul. 17, 2005.
`Dastidar et al., U.S. Appl. No. 12/463,358, filed May 8, 2009.
`Fung et al., U.S. Appl. No. 13/149,583, filed May 31, 2011.
`
`* cited by examiner
`Primary Examiner — Fernando Hidalgo
`(74) Attorney, Agent, or Firm — Trey Z Law Group: Jason
`Tsai
`
`ABSTRACT
`(57)
`Integrated circuits may communicate with off-chip memory.
`Such types of integrated circuits may include memory inter
`face circuitry that is used to interface with the off-chip
`memory. The memory interface circuitry may be calibrated
`using a procedure that includes read calibration, write level
`ing, read latency tuning, and write calibration. Read calibra
`tion may serve to ensure proper gating of data strobe signals
`and to center the data strobe signals with respect to read data
`signals. Write leveling ensures that the data strobe signals are
`aligned to system clock signals. Read latency tuning serves to
`adjust read latency to ensure optimum read performance.
`Write calibration may serve to center the data strobe signals
`with respect to write data signals. These calibration opera
`tions may be used to calibrate memory systems Supporting a
`variety of memory communications protocols.
`
`20 Claims, 16 Drawing Sheets
`
`f 23
`
`cAM) AsrATIN
`
`234
`ARANTE WR WONGAA coSA
`error
`Ar", Address conSAN, ANWRNAses
`-23
`RFRN CONRL WREYLINEAA C&NSANTA
`Adress consANANPUsing WRNAsa AN
`JNARESSSRCEHS FR3NE COMANNERA
`-2
`ROR GUARANERAAEL&CATO of CNSATARSS
`-240
`As SCCESSFUL
`EERNME WETER CLE WR
`ArtAWIFT casANWRIT NABLE Stigs
`
`MY
`sije
`
`NTERFACE
`cc:U
`
`stroller
`
`PROKSRAMMABLE
`C3
`Cics
`
`DRESS ARTM
`
`
`
`242
`
`PERFORGARANTEd Write withdAACNSANA"".
`SELECE ARESSBSO (, AND OTHERADRESS
`BTS CONSTAN
`
`PERFORGARANEWRTE WHATACNSTANAT,
`SCAERESS Bilst "t", AN
`HERADDRESS
`is constant
`
`
`
`
`
`
`
`
`
`
`
`Perform controlled write with DATA consTANAT",
`SELECEA)SS3LSE). HiR ONE&MAN
`NTRAAN
`RARSSScCNSAN
`
`-248
`ERFORMGUARANTED ACHECKWEREATAWASWRTTEN
`
`RAWith FFRNSE SINGS FORSECE
`ARSS
`
`AAARSSBS
`8. CABRAE
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 1 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 1 of 16
`
`US 8,565,033 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`rairis
`
`(
`
`)
`
`
`
`
`
`( )
`
`
`
`
`
`PROGRAMASE
`
`(
`
`)
`
`OGC () () OGC
`r
`s a s
`
`V
`
`PROGRAVIASE
`
`(
`
`
`
`)
`
`
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 2 of 26
`
`

`

`U.S. Patent
`
`US 8,565,033 B1
`
`
`
`
`
`
`
`ciwolaevaex,
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 3 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 3 of 16
`
`US 8,565,033 B1
`
`
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 4 of 26
`
`

`

`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 5 of 26
`
`

`

`U.S. Patent
`
`
`
`US 8,565,033 B1
`
`8
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 6 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 6 of 16
`
`US 8,565,033 B1
`
`OO
`
`NREASE Y
`READS & WRES
`
`- O2
`
`REA)
`(CABRAON
`
`
`
`WRE
`EWENG
`
`104.
`
`
`
`108
`
`REA) AENCY
`NNG
`
`
`
`08
`
`NRE
`CABRAON
`
`1C
`
`
`
`RELABLE
`REAS & WRES
`
`
`
`
`
`
`
`
`
`
`
`
`
`F.G. 6
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 7 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 7 of 16
`
`US 8,565,033 B1
`
`
`
`REA CABRAON
`
`SET REASYNC ANCY = MAX
`SE DOf OSPASE = C.
`
`2
`
`2}
`
`CAO KNN AA PATERNS SNG GARANEE ARES
`
`PERFORA, OSENABE CA BRAON TO
`ENSURE CEANOSA. AEC-ES
`
`SCCESS
`
`24
`
`OQIOQS PHASE
`c. JAX
`
`NO
`
`8
`
`NCREMENT DOf OSPASE
`
`ERFO. A REA). Of QS CNRNG O MAXM2
`CiOS MARGN
`
`SCCESS
`
`ADO.NA MEMORY GROUPSO
`BECA BRATE)
`
`CA BRAON
`FA
`
`FIG 7
`
`3.
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 8 of 26
`
`

`

`U.S. Patent
`
`
`
`US 8,565,033 B1
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 9 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 9 of 16
`
`US 8,565,033 B1
`
`
`
`24.
`
`4.
`
`PERFOR READ ES"
`
`READS SCCESSF
`
`- 142
`
`ADS READ-WA BUFFER AENCY UN
`READ SARS O. A
`
`SWEEP DQS Enable TO FND FIRST
`WORKNG PASE
`
`SWEEP DQS Enable TO FIND LAST
`WORKING PHASE (MAYNEED TO INCREMENT
`READ-VAD BUFFER AENCY)
`
`144
`
`6 4.
`
`148
`
`CENTER OQS Enable WHN WORKING RANGE
`
`QS ENABE
`CABRAON
`
`FG. 9
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 10 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 10 of 16
`
`US 8,565,033 B1
`
`
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 11 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 11 of 16
`
`US 8,565,033 B1
`
`
`
`
`
`??ueu SOCI 36ueu OCH
`
`
`
`
`
`~~
`
`${}{}
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 12 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 12 of 16
`
`US 8,565,033 B1
`
`
`
`s
`N
`
`s
`w
`
`s
`
`.
`
`R
`
`ke
`
`2
`..
`ka
`8)
`f
`N
`-------------'
`ac
`N:
`Kr
`t
`so
`e
`ca
`so
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 13 of 26
`
`

`

`U.S. Patent
`
`US 8,565,033 B1
`
`
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 14 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 14 of 16
`
`US 8,565,033 B1
`
`
`
`6
`
`SE RFO. AENCY TO AXV
`
`RECE rC ANCY N REAS For
`A EAS ONE MEMORY GROUP STARTS TO FA
`
`NCREASE AENCY BY ONE O RERN. C.
`WORKENG STATE MAY INCREASE LATENCY BY N
`CYCLES FORADDETIONAL MARGIN)
`
`FA
`
`REAR AENCY NNG
`
`SCCESS
`
`CA BNAON
`A
`
`38
`
`F.G. 14
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 15 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 15 of 16
`
`US 8,565,033 B1
`
`
`
`WRE CABRAON
`
`08
`
`PERFORM, WRTE DOf{S CENTERNGO
`AGN WR. C. Wr OS
`
`22
`
`DEAY DO O. EQUAZE DATA FROM
`REN AA NS
`
`24
`SWEEP DQ TO FIND DQ RANGE (Tsetup)
`
`26
`SWEEP DOS TO FIND DQ RANGE (Thoid)
`
`22)
`
`28
`
`CNR CRS BY DEAYING OC
`SMANEOUS...Y
`
`SCCESS
`
`A
`
`ADDITIONAL MEMORY GROUPS TO
`BECA BRAE)
`
`CABRAON
`A
`
`F.G. 15
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 16 of 26
`
`

`

`U.S. Patent
`
`Oct. 22, 2013
`
`Sheet 16 of 16
`
`US 8.565,033 B1
`
`230
`
`COMMAN) CABRAON
`
`PERFORM GARANEE WREW-E HOCNG OAA CONSTAN
`AT "O", ADDRESS CONSTAN AND WRITE ENABE HG
`
`PERFOR, CONTROLE WREBY ODNG OAA CONSTANTA
`* , ADDRESS CONSTANT AND PUSING WRITE ENABLE AND
`COUMNADRESS SEROBE - G- FOR ONE COMMAN NERVA
`
`PERFORM GUARANTEE) READ A OCATION OF CONSTAN ADDRESS
`
`EERMNE WNETER CONTRO ED WRTE AiaS S{CCESSFU
`AND REPEATWT FFEREN CAS AND WRE ENABE SENGS
`
`ADDRESS CASRAON
`
`-232
`
`242
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`PERFORM GUARANTEE) WRE WTH DATA CONSEAN AT "O",
`SELECTED ADDRESS BT SET TO "O", AND OTHER ADDRESS
`BS CONSTAN
`
`PERFORM. GUARANTEED WRTE WITH AIA CONSEAN A "O",
`SELECTED ADDRESS Bi SE 'O'", AND OTHER ADDRESS
`BTS CONSTAN
`
`PERFORM CONTROLLED WR E WITH DAA CONSTANA",
`SECTS). A RESS 8 PUSEO Gh FOR ONE COMMAND
`NERVAL AND OTHER ADDRESS BETS CONSTANT
`
`PERFORM. GUARANTES) READ TO CHECK WHERE DATANAS WREN
`250
`
`REPEAT WH DFFERENT PASE SETNGS FOR SELECED
`AD}RESS B
`
`ADONALARESS BS
`O BE CA BRAE)
`
`FG 16
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 17 of 26
`
`

`

`US 8,565,033 B1
`
`1.
`METHODS FOR CALIBRATING MEMORY
`INTERFACE CIRCUITRY
`
`BACKGROUND
`
`Programmable integrated circuits are a type of integrated
`circuit that can be configured by a user to implement custom
`logic functions. In a typical scenario, a logic designer uses
`computer-aided design (CAD) tools to design a custom logic
`circuit. When the design process is complete, the CAD tools
`generate configuration data. The configuration data is loaded
`into a programmable integrated circuit to configure the device
`to perform desired logic functions.
`In a typical system, a programmable integrated circuit,
`memory devices, and other electronic components are
`mounted on a printed circuitboard. The programmable inte
`grated circuit includes memory interface circuitry that is used
`to relay data back and forth between the programmable inte
`grated circuit and the memory devices (i.e., the memory inter
`face circuitry is used to read data from and write data into the
`memory devices). When performing Such memory read and
`write operations, the timing of control and data signals is
`critical.
`Because programmable integrated circuits can be config
`ured in many different ways and are installed on many differ
`ent types of boards, the lengths of circuit board traces cou
`pling the programmable integrated circuit to the memory
`devices can vary from one system to another. As a result, it is
`generally not possible to know in advance exactly how data
`and clock paths between a programmable integrated circuit
`and a given memory device will perform. In some systems,
`the data and clock paths may have one set of timing charac
`teristics, whereas in other systems the data and clock paths
`may have a different set of timing characteristics.
`Mismatch (or skew) between the data and clock paths may
`result in degraded setup and hold times. In modern high speed
`memory interface circuitry that use double data rate (DDR)
`transfers (i.e., a data transmission scheme in which data
`toggles on both rising and falling edges of the clock) a small
`amount of skew will result in faulty data transfer during read
`and write operations.
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`SUMMARY
`
`2
`mand signal before a second memory device receives the read
`command signal. The memory interface may be configured to
`send write data (DQ) and data strobe (DQS) signals to the first
`memory device before sending write DQ/DQS signals to the
`second memory device.
`The memory interface circuitry may be calibrated at the
`startup of an integrated circuit to ensure reliable read and
`write operations. Memory interface circuitry calibration pro
`cedures may include read calibration, write leveling, read
`latency tuning, and write calibration. These steps are merely
`illustrative and are not intended to limit the scope of the
`present invention. If desired, these steps may be performed in
`any suitable order. In one suitable embodiment, read latency
`tuning need not be performed. In another suitable embodi
`ment, read calibration is performed before write leveling,
`read latency tuning, and write calibration, whereas write lev
`eling is performed before write calibration.
`Read calibration may serve to calibrate a data strobe enable
`signal and to center the data strobe (DQS) signals with respect
`to the read data (DQ) signals. Calibrating the data strobe
`enable signals ensures that a properly gated DQS reaches the
`capture circuits (e.g., to filter out undesirable glitches in
`DQS). Centering read DQ/DQS involves aligning rising and
`falling edges of the DQS signals to the center of the read DQ
`windows to provide improved read margins (i.e., to provide
`equal setup and hold times) for the different memory devices.
`If desired. DQ/DQS calibration may also involve aligning
`DQS to the edges of the read DQ window.
`Write leveling may serve to align the write data strobe
`signals associated with the different memory devices to the
`system clock (e.g., to align the rising edges of write DQS with
`the rising edges of the reference clock). Aligning the DQS
`signals during write operations may allow data to be properly
`written into the respective memory groups.
`Read latency tuning involves configuring the read-syn
`chronization (read-sync) buffers to optimize for performance.
`The read-sync buffers may be used to provide an adjustable
`read latency. A maximum round trip delay (e.g., the amount of
`time elapsed since the launch of a read command to the time
`read data signals arrive at the read-sync buffer associated with
`the memory device last to receive the read command) may be
`measured during read latency tuning. The read-sync buffers
`may be configured to provide a read latency that is at least one
`cycle greater than the maximum round trip delay, at least two
`cycles greater than the maximum round trip delay, etc.
`Write calibration may serve to align the DQS signal to the
`center of the write DQ window to provide improved write
`margins (i.e., to provide equal setup and hold times) for the
`different memory devices.
`During at least Some of these calibration operations, delay
`may be introduced using programmable delay chains to pro
`vide 50 ps step delays (as an example), using delay-locked
`loops to provided phase delays (e.g., delays that are equal to
`any Suitable fraction of a clock cycle), using buffer circuits to
`provide additional latency (e.g., delays that are equal to an
`integer multiple of a clock cycle), or using other configurable
`delay elements to provide desired tuning accuracy.
`These calibration procedures may be used to calibrate
`memory systems Supporting a variety of memory communi
`cations protocols such as double data rate (DDR), quad data
`rate (QDR), reduced-latency dynamic RAM, and other
`memory communications protocols.
`Further features of the present invention, its nature and
`various advantages will be more apparent from the accompa
`nying drawings and the following detailed description.
`
`Integrated circuits such as programmable integrated cir
`cuits having memory interface circuitry are provided. The
`memory interface circuitry may be used to communicate with
`off-chip memory devices (sometimes referred to as memory
`groups) that are mounted on a circuit board. The memory
`devices and the circuitboard to which the memory devices are
`mounted may collectively be referred to as a memory module.
`Data and data strobe signals may be transmitted between the
`memory devices and the memory interface circuitry. The
`memory interface circuitry may provide system-level control
`signals (e.g., a reference clock signal, address signal, and
`command signal) to the memory devices.
`During read operations, the memory interface circuitry
`may send appropriate system control signals to the memory
`module to read data out of the memory devices. The memory
`devices may output read data and associated data strobe sig
`nals. The read data may be latched using capture circuits and
`may be stored in read-synchronization buffers prior to out
`putting data in parallel.
`During write operations, the memory interface circuitry
`may send appropriate system control signals to the memory
`module to write data into the memory devices. Consider a
`scenario in which a first memory device receives a read com
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 18 of 26
`
`

`

`3
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a diagram of an illustrative programmable inte
`grated circuit in accordance with an embodiment of the
`present invention.
`FIG. 2 is a diagram of illustrative memory interface cir
`cuitry in accordance with an embodiment of the present
`invention.
`FIG. 3 is a diagram of illustrative memory interface cir
`cuitry operating in write leveling mode in accordance with an
`embodiment of the present invention.
`FIG. 4 is a diagram showing an exemplary memory read
`operation in accordance with an embodiment of the present
`invention.
`FIG. 5 is a diagram showing an exemplary memory write
`operation in accordance with an embodiment of the present
`invention.
`FIG. 6 is a flow chart of illustrative steps involved in cali
`brating memory interface circuitry in accordance with an
`embodiment of the present invention.
`FIG. 7 is a flow chart of illustrative steps involved in per
`forming read calibration in accordance with an embodiment
`of the present invention.
`FIG. 8 is an exemplary timing diagram during data strobe
`enable calibration in accordance with an embodiment of the
`present invention.
`FIG. 9 is a flow chart of illustrative steps involved in per
`forming data strobe enable calibration in accordance with an
`embodiment of the present invention.
`FIG. 10 is an exemplary timing diagram during data strobe
`enable calibration in accordance with an embodiment of the
`present invention.
`FIG. 11 is a diagram showing illustrative steps involved in
`performing read data strobe centering in accordance with an
`embodiment of the present invention.
`FIG. 12 is a timing diagram showing illustrative steps
`involved in performing write leveling in accordance with an
`embodiment of the present invention.
`FIG. 13 is an exemplary diagram showing data skew during
`memory read operations in accordance with an embodiment
`of the present invention.
`FIG. 14 is a flow chart of illustrative steps involved in
`performing read latency tuning in accordance with an
`embodiment of the present invention.
`FIG. 15 is a flow chart of illustrative steps involved in
`performing write calibration in accordance with an embodi
`ment of the present invention.
`FIG. 16 is a flow chart of illustrative steps involved in
`performing system control signal calibration in accordance
`with an embodiment of the present invention.
`
`DETAILED DESCRIPTION
`
`Embodiments of the present invention relate to integrated
`circuits that contain memory interface circuitry. The memory
`interface circuitry may be used to interface with off-chip
`memory Such as random-access memory (RAM). The inte
`grated circuits may be digital signal processors, microproces
`sors, application specific integrated circuits, or other Suitable
`integrated circuits. With one Suitable arrangement, the inte
`grated circuits that communicate with memory are program
`mable integrated circuits such as programmable logic device
`integrated circuits or other programmable integrated circuits
`that contain programmable circuitry. The programmable cir
`cuitry can be programmed using configuration data. Pro
`grammable integrated circuits are typically operated in a
`diverse set of system environments. As a result, these inte
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 8,565,033 B1
`
`4
`grated circuits tend to benefit from adjustable timing capa
`bilities of the memory interface circuitry.
`FIG. 1 shows a diagram of an illustrative programmable
`integrated circuit. As shown in FIG. 1, device 10 may have
`input-output (I/O) circuitry 12 for driving signals off of
`device 10 and for receiving signals from other devices via
`input-output pins 14. Interconnection resources 16 Such as
`global and local vertical and horizontal conductive lines and
`buses may be used to route signals on device 10. Intercon
`nection resources 16 include fixed interconnects (conductive
`lines) and programmable interconnects (i.e., programmable
`connections between respective fixed interconnects). Pro
`grammable logic 18 may include combinational and sequen
`tial logic circuitry. For example, programmable logic 18 may
`include look-up tables, registers, and multiplexers. The pro
`grammable logic 18 may be configured to perform a custom
`logic function. The programmable interconnects associated
`with interconnection resources may be considered to be a part
`of programmable logic 18.
`Programmable logic 18 contains programmable elements
`20. Programmable elements 20 may be based on any suitable
`programmable technology, Such as fuses, antifuses, electri
`cally-programmable read-only-memory technology, ran
`dom-access memory cells, mask-programmed elements, etc.
`As an example, programmable elements 20 may be formed
`from memory cells. During programming, configuration data
`is loaded into the memory cells usingpins 14 and input-output
`circuitry 12. The memory cells are typically random-access
`memory (RAM) cells. Because the RAM cells are loaded
`with configuration data, they are sometimes referred to as
`configuration RAM cells (CRAM).
`Programmable element 20 may be used to provide a static
`control output signal for controlling the State of an associated
`logic component in programmable logic 18. The output sig
`nals generated by elements 20 are typically applied to gates of
`metal-oxide-semiconductor (MOS) transistors (sometimes
`referred to as pass gate transistors).
`The circuitry of device 10 may be organized using any
`Suitable architecture. As an example, logic 18 of program
`mable device 10 may be organized in a series of rows and
`columns of larger programmable logic regions, each of which
`contains multiple Smaller logic regions. The logic resources
`of device 10 may be interconnected by interconnection
`resources 16 Such as associated vertical and horizontal con
`ductors. These conductors may include global conductive
`lines that span substantially all of device 10, fractional lines
`Such as half-lines or quarter lines that span part of device 10,
`staggered lines of a particular length (e.g., Sufficient to inter
`connect several logic areas), Smaller local lines, or any other
`Suitable interconnection resource arrangement. If desired, the
`logic of device 10 may be arranged in more levels or layers in
`which multiple large regions are interconnected to form still
`larger portions of logic. Other device arrangements may use
`logic that is not arranged in rows and columns.
`Device 10 may communicate with off-chip memory such
`as memory module 22. Memory module 22 may be a memory
`device sometimes referred to as a single in-line memory
`module (SIMM) or a dual in-line memory module (DIMM).
`Device 10 may be configured to communicate with at least
`two memory modules 22, at least four memory modules 22,
`etc. As shown in FIG. 2, device 10 may include memory
`interface circuitry 24 that serves to relay information between
`memory module 22 and logic circuits 18 that are internal to
`device 10. Memory interface circuitry 24 may include
`memory interface circuit 26, sequencer 30, memory control
`ler 28, multiplexer 32, and other peripheral circuitry.
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 19 of 26
`
`

`

`5
`Memory interface circuit 26 may be coupled to memory
`module 22 through paths 34 and 36. During memory read
`operations, data (DQ) signals and data strobe (DQS) signals
`may be conveyed from memory module 22 to memory inter
`face circuit 26 overpath 34. During memory write operations,
`DQ/DQS may be conveyed from memory interface circuit 26
`to memory module 22 over path 34.
`During read and write operations, control signals such as
`clock CLK, address ADDR, and command CMD signals may
`be conveyed from memory interface circuit 26 to memory
`module 22 over path 36. Signal CLK may serve as a system
`reference clock (e.g., a reference clock to which the DQS
`signals, ADDRR, and CMD should be aligned). Signal CMD
`may be configured to a first value to initiate a read operation,
`to a second value to initiate a write operation, to a third value
`during normal operation, and to other values to initiate any
`desired operations. Signal ADDRR specifies the address
`(e.g., a selected bank address in a memory device) from which
`data is read out during read operations and the address to
`which data is written during write operations.
`Memory interface circuit 26 may serve to perform desired
`data rate conversions and to generate signals that meet timing
`requirements specified by the memory protocol currently
`under use. Memory interface circuit 26 may output informa
`tion gathered during read operations on outputline 38. During
`memory interface calibration processes, read data and related
`signals may be routed to sequencer 30. During normal opera
`tion, read data and related signals may be routed to memory
`controller 28. Sequencer 30 may be coupled between memory
`interface circuit 26 and memory controller 28.
`Memory interface circuit 26 may receive write data and
`related information from multiplexer32. Multiplexer 32 may
`have a first input coupled to memory controller 28 and a
`second input coupled to sequencer 30. Multiplexer 32 may be
`configured to route signals from its first input to its output
`during normal operation, whereas multiplexer 32 may be
`configured to route signals from its second input to its output
`during memory interface calibration processes (e.g., memory
`interface circuit 26 may receive information from sequencer
`30 during read/write calibration operations). Sequencer 30
`40
`arranged using the exemplary configuration of FIG. 2 may
`serve to calibrate the interface between memory module 22
`and circuit 26 at the startup of device 10 (e.g., by sending
`control signals over line 44 to make timing adjustments in
`interface circuit 26).
`Memory controller 28 may provide read data to program
`mable logic circuitry 18 overpath 42, whereas programmable
`logic circuitry 18 may provide write data to memory control
`ler 28 over path 40. Memory controller 28 may be configured
`to generate appropriate control signals corresponding to the
`memory protocol currently under use (e.g., circuit 28 may
`handle memory data management to address desired banks,
`rows, and columns and to perform memory refresh). Memory
`controller 28 may also serve to periodically request recalibra
`tion of memory interface circuit 26 by sending command
`55
`signals over path 29. Generally, memory interface circuit 26
`will be controlled predominately by sequencer 30 during
`memory interface calibration procedures, whereas memory
`interface circuit 26 will be controlled predominately by
`memory controller 28 during normal device operation.
`The arrangement of FIG. 2 is merely illustrative and is not
`intended to limit the scope of the present invention. Integrated
`circuits other than programmable integrated circuits may
`include memory interface circuitry 24 that is used to commu
`nicate with one or more memory modules 22.
`65
`As shown in FIG.3, integrated circuit 10, memory module
`22, and other circuit components (e.g., integrated circuit
`
`50
`
`35
`
`45
`
`60
`
`US 8,565,033 B1
`
`10
`
`15
`
`25
`
`30
`
`6
`chips, Surface mount components, etc.) may be mounted on a
`circuit board (e.g., printed circuit board 50). Board compo
`nents may be interconnected by conductive traces and pack
`aging (e.g., sockets into which integrated circuits are
`mounted) formed on board 50.
`Memory module 22 may include a series of memory
`devices, at least some of which is sometimes referred to as a
`memory group. For example, memory module 22 may
`include first memory group 52-1, second memory group
`52-2,..., and N' memory group 52-N. Memory module 22
`may include at least nine memory groups (as an example).
`Each of the memory groups may contain hundreds or thou
`sands of memory cells (e.g., RAM cells). The memory groups
`may communicate with memory interface circuitry through
`respective signal paths. For example, first memory group
`52-1 may communicate with circuitry 24 by sending data and
`data strobe signals (DQ/DQS) over path 34-1, second
`memory group 52-2 may communicate circuitry 24 by send
`ing DQ/DQS over path 34-2, etc.
`In general, memory access operations are not synchronized
`with system-level control signals CLK/CMD/ADDRR. As a
`result, the DQ signals that are received from the memory
`groups are not phase aligned with any known clock signal in
`device 10. It is therefore necessary to provide DQS clock
`signals with the DQ signals, so that the DQS clocks can be
`used to establish propertiming relationships when processing
`the DQ signals. For example, during a read operation,
`memory interface circuitry 24 uses the DQS clocks in cap
`turing data as it is transmitted over paths 34 from memory 22
`(see, e.g., FIG. 2). In general, the operation of each memory
`group is somewhat independent, so memory module 22 gen
`erates a DQS signal for each of the memory groups.
`The DQS signals for the different memory groups are
`generally not phase aligned with each other (e.g., skew may
`be present among the DQS signals). For example, although
`the DQS signal for a first memory group is edge-aligned with
`the DQ signals in the first memory group, the DQS signal for
`the first memory group and the seventh memory group (as an
`example) need not be in phase with each other.
`Memory interface circuitry 24 may send control signals to
`the memory groups through path 36. Memory module 22 of
`FIG.3 may be a type of memory module that exhibits inherent
`non-Zero layout skew (e.g., the control signals on path.36 may
`arrive at each of the memory groups at different times). For
`example, because of the way path 36 is routed, the control
`signals on path 36 may arrive first at memory group 52-1 and
`then arrive at each Subsequent memory group after some
`delay.
`During read operations, appropriate control signals may be
`sent over path 36 to direct the memory groups to output read
`data. Read data may be generated from the memory groups at
`different times depending on when control signals CLK/
`CMD/ADDR arrive at a particular memory group. For
`example, memory group 52-1 may output read data before
`Subsequent memory group 52-2, memory group 52-2 may
`output read data before Subsequent memory group 52-3,
`memory group 52-3 may output read data before Subsequent
`memory group 52-4, etc. Memory interface circuitry 24 may
`therefore receive read data from the different memory groups
`at Staggered times. Memory interface circuitry 24 may
`include buffer circuitry that can be used to equalize the skew
`among the different memory groups.
`During write operations, care needs to be taken when send
`ing the DQ/DQS signals to the respective memory groups.
`Device 10 may, for example, operate in a write leveling mode
`in which the DQ/DQS signals are sent to the respective
`memory groups at predetermined staggered times. For
`
`Micron et al. - Exhibit 1011
`Micron et al. v. Netlist - IPR2022-00236
`Page 20 of 26
`
`

`

`US 8,565,033 B1
`
`10
`
`15
`
`7
`example, DQ/DQS may be sent over path 34-1 to memory
`group 52-1 at a first point in time, whereas DQ/DQS may be
`sent over path 34-2 to Subsequent memory group 52-2 at a
`second point in time that is later than the first point in time
`(e.g., DQ/DQS for each Subsequent memory group may be
`sent after some adjustable amount of delay). Sending
`DQ/DQS from memory interface circuitry 24 to the memory
`groups using this write leveling approach ensures that the
`DQ/DQS signals and the control signals arrive synchronized
`(e.g., that the DQ/DQS signals and the CLK signal are phase
`aligned).
`FIG. 4 is a diagram showing an illustrative signal flow
`during read operations. As shown in FIG. 4, the memory
`groups on memory module 22 may send read signals to cor
`responding memory interface input-output (I/O) circuits 54 in
`memory interface circuit 26. For example, memory group
`52-1 may send DQ signals overline 34-1A to a first I/O circuit
`54 and may send DQS signals over line 34-1B to the first I/O
`circuit 54. Similarly, memory group 52-2 may send DQ sig
`nals over line 34-2A to a second I/O circuit 54 and may send
`DQS signals over line 34-2B to the second I/O circuit 54, and
`SO. O.
`System control signals CLK/ADDR/CMD may be con
`veyed to the memory groups over path 36. As shown in FIG.
`4, memory group 52-1 may output DQ/DQS signals before
`any other memory group, because group 52-1 receiv