(12) Ulllted States Patent
`Chen et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 8,301,833 B1
`Oct. 30, 2012
`
`US008301833B1
`
`(54) NON-VOLATILE MEMORY MODULE
`
`.
`(75) Inventors‘
`
`-_
`.
`shecclslelll’ walmln’ .CA gELUS _
`6 my - 0 "m9", 1'" me’
`(
`)’
`Scott Milton, Irvme, CA (US); Jayesh
`Bhakta, Cerritos, CA (US)
`
`(73) Assignee: Netlist, Inc., Irvine, CA (US)
`
`( * ) Not1ce:
`
`.
`
`.
`
`.
`
`.
`
`.
`
`Subject' to any d1scla1mer, the term ofth1s
`patent 1s extended or adjusted under 35
`U.S.C. 154(b) by 638 days.
`
`5/1996 Harper, Jr. et a1.
`5,519,663 A
`6,158,015 A 12/2000 Klein
`6,336,174 B1
`1/2002 Li et al.
`6,336,176 B1
`1/2002 Leyda et a1.
`6,487,623 B1
`11/2002 Emerson et a1.
`6,653,507 B1
`12/2003 Chan
`6,799,244 B2
`9/2004 Tanaka et al.
`7,409,590 B2
`8/2008 Moshayedi et al.
`2002/0083368 A1
`6/2002 Abe et al'
`2004/0190210 A1
`9/2004 Leete
`2007/0192627 A1* 8/2007 Oshikiri ...................... .. 713/191
`
`8000,; Cope ““““““““““““““ “ 711/111
`
`2008/0195806 A1,).
`* Citedb examiner
`y
`
`(21) Appl' NO‘: 12/240’916
`-
`_
`(22) F1led.
`
`Sep. 29, 2008
`
`Primary Examiner * Midys Rojas
`(74) Attorney, Agent, or Firm * Nixon Peabody LLP;
`Khaled Shami
`
`Related US. Application Data
`
`(63) Continuation of application No. 12/ 131,873, ?led on
`Jun. 2, 2008, noW abandoned.
`_
`_
`_
`_
`(60) PrOVlslOnal aPPlleatlOn NO- 60/941,586, ?led 011 Jun-
`1, 2007-
`
`(51) Int‘ Cl‘
`(2006-01)
`G06F 12/ 00
`(52) U-s- Cl- ~~~~~~ ~~ 711/104; 711/160; 711/161; 711/162;
`710/10
`(58) Field of Classi?cation Search ................ .. 711/160,
`71 1/161, 162, 104; 710/ 10
`See application ?le for complete search history.
`
`(56)
`
`References Cited
`
`US. PATENT DOCUMENTS
`12/1983 Hoffman
`4,420,821 A
`5/1984 Hoffman
`4,449,205 A
`
`(57)
`
`ABSTRACT
`_
`_
`_
`_
`_
`Certam embod1ments descnbed herem mclude a memory
`system Which can communicate With a host system such as a
`disk controller of a computer system. The memory system
`can include volatile and non-volatile memory and a controller
`Which are con?gured such that the controller backs up the
`volatile memory using the non-volatile memory in the event
`of a trigger condition. In order to poWer the system in the
`event of a poWer failure or reduction, the memory system can
`include a secondary poWer source Which is not a battery and
`may include, for example, a capacitor or capacitor array. The
`memory system can be con?gured such that the operation of
`the volatile memory is not adversely affected by the non
`volatile memory or the controller When the volatile memory is
`interacting With the host system.
`
`30 Claims, 12 Drawing Sheets
`
`.500
`
`5/0
`
`Operate volatile memory at ?rst frequency
`in ?rst mode
`
`.530
`
`Operate non-volatile memory at
`second frequency in second
`mode
`
`Operate volatile memory at
`third frequency in second
`mode
`
`

`
`U.S. Patent
`
`Oct. 30, 2012
`
`Sheet 1 of 12
`
`US 8,301,833 B1
`
`NW
`
`mm.\S
`
`Q\uni
`
`_IIIIIIIIIIIIII
`
`NW
`
`_._o:>>m
`
`__
`
`__W
`
`
`
`
`
` _wmww“xsasx_>_§x_>_M§X25“9229229:2oz<z
`
`<55
`
`
`
`:,_828<_._o:>>w
`
`II|||IIIJ.
`
`QM.
`
`.5n:.:O
`
`
`
`
`
`8mt¢E_s_s_a2m_§_w
`
`
`
`mo._._u<n_<on_m:.m
`
`Q%
`
`3%
`
`NUE
`
`
`
`
`
`

`
`U.S. Patent
`
`Oct. 30, 2012
`
`Sheet 2 of 12
`
`US 8,301,833 B1
`
`NW
`
`1
`
`o._._>>m
`
`E8a8<
`
`<55
`
`H2829:
`
`NDE
`
`

`
`US. Patent
`
`0a. 30, 2012
`
`Sheet 3 0f 12
`
`US 8,301,833 B1
`
`NW
`
`Wm.
`
`502mm?‘
`
`Ska
`
`PZOQEQQ
`
`<20
`
`Qm.
`
`$501
`
`.5950
`
`w DE
`
`

`
`U.S. Patent
`
`Oct. 30, 2012
`
`Sheet 4 of 12
`
`US 8,301,833 B1
`
`mw.0:
`
`§DE
`
`/as\\\
`
`_o.Eo2$2%m
`
`%\
`
`_o_E8\wm2E.m
`
`%%\Sflx
`
`%%\bbvx%h.\
`
`_o_E8\mme%m
`
`_o.Eo2$2%m
`
`Q.\
`
`QR
`
`
`
`
`
`

`
`US. Patent
`
`0a. 30, 2012
`
`Sheet 5 0f 12
`
`US 8,301,833 B1
`
`52mm
`
`$3
`
`6%83353
`
`szcoawmmzunm
`
`%\\
`
`x86
`
`uw DE
`
`

`
`U.S. Patent
`
`Oct. 30, 2012
`
`Sheet 6 of 12
`
`US 8,301,833 B1
`
`>8_8_xozmago
`
`
`W3>:oo\mmm
`
`
`
`<9:
`
`>8_om_
`
`<®nE
`
`N3aoomggm
`
`Eamszoo
`
`m_%o_>_53$
`
`mDE
`
`wees
`
`_>__>_a>z2bimaxK.
`
`;mm._u_
`
`<9:
`
`mass
`
`5%:
`
`<9:
`
`\%\
`
`NE........I_>_>>n_
`
`___Eoo
`
`go:
`
`E293
`
`%>
`
`
`
`

`
`US. Patent
`
`Oct. 30,2012
`
`Sheet 7 of 12
`
`US 8,301,833 B1
`
`ll
`
`Provide ?rst voltage from input power supply
`and second voltage from ?rst power subsystem
`
`200
`
`No
`
`Second condition detected’?
`
`Y
`es
`
`K 220
`
`Provide ?rst voltage and second voltage from
`?rst power subsystem
`
`4
`‘
`
`‘I
`
`K 250
`
`Charge second power subsystem
`
`N0
`
`Third condition detected?
`
`2.50 K
`Provide first voltage and second voltage from
`second power subsystem
`
`FIG. 6
`
`

`
`US. Patent
`
`Oct. 30, 2012
`
`Sheet 8 or 12
`
`US 8,301,833 B1
`
`J00
`
`Communicate data between volatile
`memory and host system in a ?rst mode
`
`/ 5/0
`
`ll
`
`Store a ?rst copy of data from the volatile
`memory to the non-volatile memory when
`in a second mode
`
`/ J20
`
`Restore the ?rst copy of data from the non
`volatile memory to the volatile memory
`
`/ J30
`
`J40
`
`Erase the ?rst copy of data from
`the non-volatile memory
`
`l l
`
`J50
`/
`Copy second copy of data from
`volatile memory to non-volatile
`memory in a second mode
`
`Restore the second copy of data from the
`non-volatile memory to the volatile memory
`
`/ J60
`
`FIG. 7
`
`

`
`U.S. Patent
`
`Oct. 30, 2012
`
`Sheet 9 of 12
`
`US 8,301,833 B1
`
`
`
`
`
`QNVxse3_8_o_m_Eemt_n_
`
`$45%
`II-EIifiéfi1SQ.
`
`
`NM».
`
`Nww.533_§.$U_3:3_>_<%
`
`
`
`2£83.8.9_m_§§.__o.:3:8\.18:8
`
`www\Ew_§_0;-:9:8.1\
`
`90»\2o&a$338wmv
`3%.2N50;1363
`
`max\\mww
`wwwMN».NM».éaEva£208
`
`
`ham».22
`
`mumm~:_.
`
`.8m§m_
`
`maaifi‘IE3
`
`~.E§d__'4:E1"8:932_‘EmejfiAIK\__wu_>_v
`v_om%mm._StEl%\V
`
`95%:Ex2N1:390.3aamm»E»5«X§§|
`3:_>_<%23_8_u33%9138SNmax5m_m__\%NweNI:3\Ea_§%_EfigIFS3E.0my
`.ENU_oaIi23N_.__>_I\ANV_%___ao
`
`
`VxwemuEmmam~C__oIIhbv
`88.5?II»
`
`l053._2_EEoo
`<9:Em252<9:2
`umDE
`
`
`
`
`
`

`
`US. Patent
`
`Oct. 30, 2012
`
`Sheet 10 0f 12
`
`US 8,301,833 B1
`
`500
`
`5/0
`
`/
`
`Operate volatile memory at ?rst frequency
`in ?rst mode
`
`520
`ll
`\
`Operate non-volatile memory at
`second frequency in second
`mode
`
`.530
`/
`ll
`Operate volatile memory at
`third frequency in second
`mode
`
`

`
`US. Patent
`
`0a. 30, 2012
`
`Sheet 11 0112
`
`US 8,301,833 B1
`
`
`
`N zmemww 52%
`
`<OnE
`
`

`
`US. Patent
`
`0a. 30, 2012
`
`Sheet 12 0f 12
`
`US 8,301,833 B1
`
`Z00
`
`Communicate data words between volatile
`memory and host system in ?rst mode
`
`/ 5/0
`
`Store ?rst slice of data word in
`buffer
`
`\
`
`Store second slice of data word In
`buffer
`
`V
`
`WrIte entlre data word from buffer
`to non-volatile memory
`
`FIG. 11
`
`

`
`US 8,301,833 B1
`
`1
`NON-VOLATILE MEMORY MODULE
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application is a continuation of Us. patent applica
`tion Ser. No. 12/131,873, ?led Jun. 2, 2008, Which claims the
`bene?t of Us. Provisional Application No. 60/941 ,586, ?led
`Jun. 1, 2007. Each of these applications is incorporated in its
`entirety by reference herein.
`
`BACKGROUND
`
`Certain types of memory modules comprise a plurality of
`dynamic random-access memory (DRAM) devices mounted
`on a printed circuit board (PCB). These memory modules are
`typically mounted in a memory slot or socket of a computer
`system (e.g., a server system or a personal computer) and are
`accessed by the computer system to provide volatile memory
`to the computer system.
`Volatile memory generally maintains stored information
`only When it is poWered. Batteries have been used to provide
`poWer to volatile memory during poWer failures or interrup
`tions. HoWever, batteries may require maintenance, may need
`to be replaced, are not environmentally friendly, and the sta
`tus of batteries can be dif?cult to monitor.
`Non-volatile memory can generally maintain stored infor
`mation While poWer is not applied to the non-volatile
`memory. In certain circumstances, it can therefore be useful
`to backup volatile memory using non-volatile memory.
`
`SUMMARY
`
`In certain embodiments, a memory system coupled to a
`computer system is provided Which includes a volatile
`memory subsystem, a non-volatile memory subsystem, and a
`controller operatively coupled to the non-volatile memory
`subsystem. The memory system can also include at least one
`circuit con?gured to selectively operatively decouple the con
`troller from the volatile memory subsystem.
`In some embodiments, a poWer module for providing a
`plurality of voltages to a memory system is described. The
`poWer module includes non-volatile and volatile memory,
`and the plurality of voltages include at least a ?rst voltage and
`a second voltage. The poWer module of certain embodiments
`includes an input providing a third voltage to the poWer mod
`ule and a voltage conversion element con?gured to provide
`the second voltage to the memory system. The poWer module
`also includes a ?rst poWer element con?gured to selectively
`provide a fourth voltage to the conversion element. The poWer
`module further includes a second poWer element con?gured
`to selectively provide a ?fth voltage to the conversion ele
`ment. The poWer module can be con?gured to selectively
`provide the ?rst voltage to the memory system either from the
`conversion element or from the input.
`The poWer module can be con?gured to be operated in at
`least three states in certain embodiments. In a ?rst state, the
`?rst voltage is provided to the memory system from the input
`and the fourth voltage is provided to the conversion element
`from the ?rst poWer element. In a second, state the fourth
`voltage is provided to the conversion element from the ?rst
`poWer element and the ?rst voltage is provided to the memory
`system from the conversion element. In a third state, the ?fth
`voltage is provided to the conversion element from the second
`poWer element and the ?rst voltage is provided to the memory
`system from the conversion element.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`A method of providing a ?rst voltage and a second voltage
`to a memory system including volatile and non-volatile
`memory subsystems is provided in certain embodiments. The
`method includes, during a ?rst condition, providing the ?rst
`voltage to the memory system from an input poWer supply
`and providing the second voltage to the memory system from
`a ?rst poWer sub system. The method further includes detect
`ing a second condition and, during the second condition,
`providing the ?rst voltage and the second voltage to the
`memory system from the ?rst poWer sub system. The method
`also includes charging a second poWer subsystem and detect
`ing a third condition. During the third condition, the method
`includes providing the ?rst voltage and the second voltage to
`the memory system from the second poWer subsystem.
`In certain embodiments, a method is provided for control
`ling a memory system operatively coupled to a host system
`and Which includes a volatile memory subsystem and a non
`volatile memory subsystem. The method can include operat
`ing the volatile memory subsystem at a ?rst frequency When
`the memory system is in a ?rst mode of operation in Which
`data is communicated betWeen the volatile memory sub
`system and the host system. In certain embodiments, the
`method further includes operating the non-volatile memory
`sub system at a second frequency When the memory system is
`in a second mode of operation in Which data is communicated
`betWeen the volatile memory subsystem and the non-volatile
`memory subsystem. The method can also include operating
`the volatile memory subsystem at a third frequency When the
`memory system is in the second mode of operation, the third
`frequency less than the ?rst frequency.
`In certain embodiments, a method is provided for control
`ling a memory system operatively coupled to a host system.
`The memory system includes a volatile memory subsystem
`and a non-volatile memory subsystem. In certain embodi
`ments, the method includes communicating data Words
`betWeen the volatile memory subsystem and the host system
`When the memory system is in a ?rst mode of operation. The
`method can further include transferring data Words from the
`volatile memory subsystem to the non-volatile memory sub
`system When the memory system is in a second mode of
`operation. Transferring each data Word can include storing a
`?rst portion of the data Word in a buffer, storing a second
`portion of the data Word in the buffer, and Writing the entire
`data Word from the buffer to the non-volatile memory sub
`system.
`A memory system operatively coupled to a host system is
`provided in certain embodiments. The memory system can
`include a volatile memory subsystem and a non-volatile
`memory subsystem comprising at least 100 percent more
`storage capacity than does the volatile memory subsystem.
`The memory system includes a controller operatively coupled
`to the volatile memory subsystem and operatively coupled to
`the non-volatile memory subsystem, the controller con?g
`ured to alloW data to be communicated betWeen the volatile
`memory subsystem and the host system When the memory
`system is operating in a ?rst state and to alloW data to be
`communicated betWeen the volatile memory subsystem and
`the non-volatile memory subsystem When the memory sys
`tem is operating in a second state.
`A method of controlling a memory system operatively
`coupled to a host system is provided in certain embodiments.
`The memory system includes a volatile memory subsystem
`and a non-volatile memory subsystem. The method can
`include communicating data betWeen the volatile memory
`subsystem and the host system When the memory system is in
`a ?rst mode of operation. The method of certain embodiments
`further includes storing a ?rst copy of data from the volatile
`
`

`
`US 8,301,833 B1
`
`3
`memory subsystem to the non-volatile memory subsystem at
`a ?rst time when the memory system is in a second mode of
`operation. The method may further include restoring the ?rst
`copy of data from the non-volatile memory subsystem to the
`volatile memory subsystem and erasing the ?rst copy of data
`from the non-volatile memory subsystem. In certain embodi
`ments, the method also includes storing a second copy of data
`from the volatile memory subsystem to the non-volatile
`memory subsystem at a second time when the memory sys
`tem is in the second mode of operation, wherein storing the
`second copy begins before the ?rst copy is completely erased
`from the non-volatile memory subsystem.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a block diagram of an example memory system
`compatible with certain embodiments described herein.
`FIG. 2 is a block diagram of an example memory module
`with ECC (error-correcting code) having a volatile memory
`subsystem with nine volatile memory elements and a non
`volatile memory subsystem with ?ve non-volatile memory
`elements in accordance with certain embodiments described
`herein.
`FIG. 3 is a block diagram of an example memory module
`having a microcontroller unit and logic element integrated
`into a single device in accordance with certain embodiments
`described herein.
`FIGS. 4A-4C schematically illustrate example embodi
`ments of memory systems having volatile memory sub
`systems comprising registered dual in-line memory modules
`in accordance with certain embodiments described herein.
`FIG. 5 schematically illustrates an example power module
`of a memory system in accordance with certain embodiments
`described herein.
`FIG. 6 is a ?owchart of an example method of providing a
`?rst voltage and a second voltage to a memory system includ
`ing volatile and non-volatile memory subsystems.
`FIG. 7 is a ?owchart of an example method of controlling
`a memory system operatively coupled to a host system and
`which includes at least 100 percent more storage capacity in
`non-volatile memory than in volatile memory.
`FIG. 8 schematically illustrates an example clock distribu
`tion topology of a memory system in accordance with certain
`embodiments described herein.
`FIG. 9 is a ?owchart of an example method of controlling
`a memory system operatively coupled to a host system, the
`method including operating a volatile memory subsystem at a
`reduced rate in a back-up mode.
`FIG. 10 schematically illustrates an example topology of a
`connection to transfer data slices from two DRAM segments
`of a volatile memory subsystem of a memory system to a
`controller of the memory system.
`FIG. 11 is a ?owchart of an example method of controlling
`a memory system operatively coupled to a host system, the
`method including backing up and/or restoring a volatile
`memory subsystem in slices.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`DETAILED DESCRIPTION
`
`Certain embodiments described herein include a memory
`system which can communicate with a host system such as a
`disk controller of a computer system. The memory system
`can include volatile and non-volatile memory, and a control
`ler. The controller backs up the volatile memory using the
`non-volatile memory in the event of a trigger condition. Trig
`ger conditions can include, for example, a power failure,
`power reduction, request by the host system, etc. In order to
`
`60
`
`65
`
`4
`power the system in the event of a power failure or reduction,
`the memory system can include a secondary power source
`which does not comprise a battery and may include, for
`example, a capacitor or capacitor array.
`In certain embodiments, the memory system can be con
`?gured such that the operation of the volatile memory is not
`adversely affected by the non-volatile memory or by the
`controller when the volatile memory is interacting with the
`host system. For example, one or more isolation devices may
`isolate the non-volatile memory and the controller from the
`volatile memory when the volatile memory is interacting with
`the host system and may allow communication between the
`volatile memory and the non-volatile memory when the data
`of the volatile memory is being restored or backed-up. This
`con?guration generally protects the operation of the volatile
`memory when isolated while providing backup and restore
`capability in the event of a trigger condition, such as a power
`failure.
`In certain embodiments described herein, the memory sys
`tem includes a power module which provides power to the
`various components of the memory system from different
`sources based on a state of the memory system in relation to
`a trigger condition (e. g., a power failure). The power module
`may switch the source of the power to the various components
`in order to e?iciently provide power in the event of the power
`failure. For example, when no power failure is detected, the
`power module may provide power to certain components,
`such as the volatile memory, from system power while charg
`ing a secondary power source (e.g., a capacitor array). In the
`event of a power failure or other trigger condition, the power
`module may power the volatile memory elements using the
`previously charged secondary power source.
`In certain embodiments, the power module transitions rela
`tively smoothly from powering the volatile memory with
`system power to powering it with the secondary power
`source. For example, the memory system may power volatile
`memory with a third power source from the time the memory
`system detects that power failure is likely to occur until the
`time the memory system detects that the power failure has
`actually occurred.
`In certain embodiments, the volatile memory system can
`be operated at a reduced frequency during backup and/or
`restore operations which can improve the e?iciency of the
`system and save power. In some embodiments, during backup
`and/ or restore operations, the volatile memory communicates
`with the non-volatile memory by writing and/ or reading data
`words in bit-wise slices instead of by writing entire words at
`once. In certain embodiments, when each slice is being writ
`ten to or read from the volatile memory the unused slice(s) of
`volatile memory is not active, which can reduce the power
`consumption of the system.
`In yet other embodiments, the non-volatile memory can
`include at least 100 percent more storage capacity than the
`volatile memory. This con?guration can allow the memory
`system to ef?ciently handle subsequent trigger conditions.
`FIG. 1 is a block diagram of an example memory system 10
`compatible with certain embodiments described herein. The
`memory system 10 can be coupled to a host computer system
`and can include a volatile memory subsystem 30, a non
`volatile memory subsystem 40, and a controller 62 opera
`tively coupled to the non-volatile memory subsystem 40. In
`certain embodiments, the memory system 10 includes at least
`one circuit 52 con?gured to selectively operatively decouple
`the controller 62 from the volatile memory subsystem 30.
`In certain embodiments, the memory system 10 comprises
`a memory module. The memory system 10 may comprise a
`printed-circuit board (PCB) 20. In certain embodiments, the
`
`

`
`US 8,301,833 B1
`
`5
`memory system 10 has a memory capacity of 5 1 2-MB, 1-GB,
`2-GB, 4-GB, or 8-GB. Other volatile memory capacities are
`also compatible With certain embodiments described herein.
`In certain embodiments, the memory system 10 has a non
`volatile memory capacity of 512-MB, 1-GB, 2-GB, 4-GB,
`8-GB, 16-GB, or 32-GB. Other non-volatile memory capaci
`ties are also compatible With certain embodiments described
`herein. In addition, memory systems 10 having Widths of 4
`bytes, 8 bytes, 16 bytes, 32 bytes, or 32 bits, 64 bits, 128 bits,
`256 bits, as Well as other Widths (in bytes or in bits), are
`compatible With embodiments described herein. In certain
`embodiments, the PCB 20 has an industry-standard form
`factor. For example, the PCB 20 can have a loW pro?le (LP)
`form factor With a height of 30 millimeters and a Width of
`133.35 millimeters. In certain other embodiments, the PCB
`20 has a very high pro?le (V HP) form factor With a height of
`50 millimeters or more. In certain other embodiments, the
`PCB 20 has a very loW pro?le (VLP) form factorWith a height
`of 18.3 millimeters. Other form factors including, but not
`limited
`to,
`small-outline (SO-DIMM), unbuffered
`(UDIMM), registered (RDIMM), fully-buffered (FBDIMM),
`mini-DIMM, mini-RDIMM, VLP mini-DIMM, micro
`DIMM, and SRAM DIMM are also compatible With certain
`embodiments described herein. For example, in other
`embodiments, certain non-DIMM form factors are possible
`such as, for example, single in-line memory module (SIMM),
`multi-media card (MMC), and small computer system inter
`face (SCSI).
`In certain preferred embodiments, the memory system 10
`is in electrical communication With the host system. In other
`embodiments, the memory system 1 0 may communicate With
`a host system using some other type of communication, such
`as, for example, optical communication. Examples of host
`systems include, but are not limited to, blade servers, 1U
`servers, personal computers (PCs), and other applications in
`Which space is constrained or limited. The memory system 10
`can be in communication With a disk controller of a computer
`system, for example. The PCB 20 can comprise an interface
`22 that is con?gured to be in electrical communication With
`the host system (not shoWn). For example, the interface 22
`can comprise a plurality of edge connections Which ?t into a
`corresponding slot connector of the host system. The inter
`face 22 of certain embodiments provides a conduit for poWer
`voltage as Well as data, address, and control signals betWeen
`the memory system 10 and the host system. For example, the
`interface 22 can comprise a standard 240-pin DDR2 edge
`connector.
`The volatile memory subsystem 30 comprises a plurality of
`volatile memory elements 32 and the non-volatile memory
`subsystem 40 comprises a plurality of non-volatile memory
`elements 42. Certain embodiments described herein advan
`tageously provide non-volatile storage via the non-volatile
`memory subsystem 40 in addition to high-performance (e.g.,
`high speed) storage via the volatile memory subsystem 30. In
`certain embodiments, the ?rst plurality of volatile memory
`elements 32 comprises tWo or more dynamic random-access
`memory (DRAM) elements. Types of DRAM elements 32
`compatible With certain embodiments described herein
`include, but are not limited to, DDR, DDR2, DDR3, and
`synchronous DRAM (SDRAM). For example, in the block
`diagram of FIG. 1, the ?rst memory bank 30 comprises eight
`64Mx8 DDR2 SDRAM elements 32. The volatile memory
`elements 32 may comprise other types of memory elements
`such as static random-access memory (SRAM). In addition,
`volatile memory elements 32 having bit Widths of 4, 8, 16, 32,
`as Well as other bit Widths, are compatible With certain
`embodiments described herein. Volatile memory elements 32
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`compatible With certain embodiments described herein have
`packaging Which include, but are not limited to, thin small
`outline package (TSOP), ball-grid-array (BGA), ?ne-pitch
`BGA (FBGA), micro-BGA (uBGA), mini-BGA (mBGA),
`and chip-scale packaging (CSP).
`In certain embodiments, the second plurality of non-vola
`tile memory elements 42 comprises one or more ?ash
`memory elements. Types of ?ash memory elements 42 com
`patible With certain embodiments described herein include,
`but are not limited to, NOR ?ash, NAND ?ash, ONE-NAND
`?ash, and multi-level cell (MLC). For example, in the block
`diagram of FIG. 1, the second memory bank 40 comprises
`512 MB of ?ash memory organiZed as four 128 Mbx8 NAND
`?ash memory elements 42. In addition, non-volatile memory
`elements 42 having bit Widths of 4, 8, 16, 32, as Well as other
`bit Widths, are compatible With certain embodiments
`described herein. Non-volatile memory elements 42 compat
`ible With certain embodiments described herein have pack
`aging Which include, but are not limited to, thin small-outline
`package (TSOP), ball-grid-array (BGA), ?ne-pitch BGA
`(FBGA), micro-BGA (uBGA), mini-BGA (mBGA), and
`chip-scale packaging (CSP).
`FIG. 2 is a block diagram of an example memory module
`10 With ECC (error-correcting code) having a volatile
`memory subsystem 30 With nine volatile memory elements
`32 and a non-volatile memory subsystem 40 With ?ve non
`volatile memory elements 42 in accordance With certain
`embodiments described herein. The additional memory ele
`ment 32 of the ?rst memory bank 30 and the additional
`memory element 42 of the second memory bank 40 provide
`the ECC capability. In certain other embodiments, the volatile
`memory subsystem 30 comprises other numbers of volatile
`memory elements 32 (e.g., 2, 3, 4, 5, 6, 7, more than 9). In
`certain embodiments, the non-volatile memory sub system 40
`comprises other numbers of non-volatile memory elements
`42 (e.g., 2, 3, more than 5).
`Referring to FIG. 1, in certain embodiments, the logic
`element 70 comprises a ?eld-programmable gate array
`(FPGA). In certain embodiments, the logic element 70 com
`prises an FPGA available from Lattice Semiconductor Cor
`poration Which includes an internal ?ash. In certain other
`embodiments, the logic element 70 comprises an FPGA
`available from another vendor. The internal ?ash can improve
`the speed of the memory system 10 and save physical space.
`Other types of logic elements 70 compatible With certain
`embodiments described herein include, but are not limited to,
`a programmable-logic device (PLD), an application-speci?c
`integrated circuit (ASIC), a custom-designed semiconductor
`device, a complex programmable logic device (CPLD). In
`certain embodiments, the logic element 70 is a custom device.
`In certain embodiments, the logic element 70 comprises vari
`ous discrete electrical elements, While in certain other
`embodiments, the logic element 70 comprises one or more
`integrated circuits. FIG. 3 is a block diagram of an example
`memory module 10 having a microcontroller unit 60 and
`logic element 70 integrated into a single controller 62 in
`accordance With certain embodiments described herein. In
`certain embodiments, the controller 62 includes one or more
`other components. For example, in one embodiment, an
`FPGA Without an internal ?ash is used and the controller 62
`includes a separate ?ash memory component Which stores
`con?guration information to program the FPGA.
`In certain embodiments, the at least one circuit 52 com
`prises one or more sWitches coupled to the volatile memory
`subsystem 30, to the controller 62, and to the host computer
`(e.g., via the interface 22, as schematically illustrated by
`FIGS. 1-3). The one or more sWitches are responsive to sig
`
`

`
`US 8,301,833 B1
`
`7
`nals (e.g., from the controller 62) to selectively operatively
`decouple the controller 62 from the volatile memory sub
`system 30 and to selectively operatively couple the controller
`62 to the volatile memory subsystem 30. In addition, in cer
`tain embodiments, the at least one circuit 52 selectively
`operatively couples and decouples the volatile memory sub
`system 30 and the host system.
`In certain embodiments, the volatile memory sub system 3 0
`can comprise a registered DIMM subsystem comprising one
`or more registers 160 and a plurality of DRAM elements 180,
`as schematically illustrated by FIG. 4A. In certain such
`embodiments, the at least one circuit 52 can comprise one or
`more sWitches 172 coupled to the controller 62 (e.g., logic
`element 70) and to the volatile memory subsystem 30 Which
`can be actuated to couple and decouple the controller 62 to
`and from the volatile memory subsystem 30, respectively.
`The memory system 10 further comprises one or more
`sWitches 170 coupled to the one or more registers 160 and to
`the plurality of DRAM elements 180 as schematically illus
`trated by FIG. 4A. The one or more sWitches 170 can be
`selectively sWitched, thereby selectively operatively coupling
`the volatile memory subsystem 30 to the host system 150. In
`certain other embodiments, as schematically illustrated by
`FIG. 4B, the one or more sWitches 174 are also coupled to the
`one or more registers 160 and to a poWer source 162 for the
`one or more registers 160. The one or more sWitches 174 can
`be selectively sWitched to turn poWer on or off to the one or
`more registers 160, thereby selectively operatively coupling
`the volatile memory subsystem 30 to the host system 150. As
`schematically illustrated by FIG. 4C, in certain embodiments
`the at least one circuit 52 comprises a dynamic on-die termi
`nation (ODT) 176 circuit of the logic element 70. For
`example, the logic element 70 can comprise a dynamic ODT
`circuit 176 Which selectively operatively couples and
`decouples the logic element 70 to and from the volatile
`memory subsystem 30, respectively. In addition, and similar
`to the example embodiment of FIG. 4A described above, the
`one or more sWitches 170 canbe selectively sWitched, thereby
`selectively operatively coupling the volatile memory sub
`system 30 to the host system 150.
`Certain embodiments described herein utiliZe the non
`volatile memory subsystem 40 as a ?ash “mirror” to provide
`backup of the volatile memory subsystem 30 in the event of
`certain system conditions. For example, the non-volatile
`memory subsystem 40 may backup the volatile memory sub
`system 30 in the event of a trigger condition, such as, for
`example, a poWer failure or poWer reduction or a request from
`the host system. In one embodiment, the non-volatile
`memory subsystem 40 holds intermediate data results in a
`noisy system environment When the host computer system is
`engaged in a long computation. In certain embodiments, a
`backup may be performed on a regular basis. For example, in
`one embodiment, the backup may occur every millisecond in
`response to a trigger condition. In certain embodiments, the
`trigger condition occurs When the memory system 10 detects
`that the system voltage is beloW a certain threshold voltage.
`For example, in one embodiment, the threshold voltage is 10
`percent beloW a speci?ed operating voltage. In certain
`embodiments, a trigger condition occurs When the voltage
`goes above a certain threshold value, such as, for example, 10
`percent above a speci?ed operating voltage. In some embodi
`ments, a trigger condition occurs When the voltage goes
`beloW a threshold or above another threshold. In various
`embodiments, a backup and/ or restore operation may occur in
`reboot and/or non-reboot trigger conditions.
`As schematically illustrated by FIGS. 1 and 2, in certain
`embodiments, the controller 62 may comprise a microcon
`
`40
`
`45
`
`20
`
`25
`
`30
`
`35
`
`50
`
`55
`
`60
`
`65
`
`8
`troller unit (MCU) 60 and a logic element 70. In certain
`embodiments, the MCU 60 provides memory management
`for the non-volatile memory subsystem 40 and controls data
`transfer betWeen the volatile memory subsystem 30 and the
`non-volatile memory subsystem 40. The MCU 60 of certain
`embodiments comprises a 16-bit microcontroller, although
`other types of microcontro