I 1111111111111111 11111 111111111111111 111111111111111 IIIII IIIIII IIII IIII IIII
`US008891298B2
`
`c12) United States Patent
`Rao
`
`(IO) Patent No.:
`(45) Date of Patent:
`
`US 8,891,298 B2
`Nov. 18, 2014
`
`(54) LIFETIME MIXED LEVEL NON-VOLATILE
`MEMORY SYSTEM
`
`(75)
`
`Inventor: G. R. Mohan Rao, Richardson, TX (US)
`
`2010/0058018 Al*
`2010/0172179 Al*
`2011/0060870 Al
`2011/0271043 Al *
`
`................... 711/167
`3/2010 Kund et al.
`7/2010 Gorobets et al .......... 365/185.09
`3/2011 Rao
`11/2011 Segal et al.
`
`................... 711/103
`
`(73) Assignee: Greenthread, LLC, Richardson, TX
`(US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 357 days.
`
`(21) Appl. No.: 13/455,267
`
`(22) Filed:
`
`Apr. 25, 2012
`
`(65)
`
`Prior Publication Data
`
`US 2013/0021846 Al
`
`Jan.24,2013
`
`Related U.S. Application Data
`
`(60) Provisional application No. 61/509,257, filed on Jul.
`19,2011.
`
`(51)
`
`(2006.01)
`(2006.01)
`
`Int. Cl.
`GllC 16134
`G06F 12102
`(52) U.S. Cl.
`CPC ........ GllC 1613495 (2013.01); G06F 1210246
`(2013.01); G06F 2212/7202 (2013.01)
`USPC ................. 365/185.03; 365/185.09; 365/201;
`365/148; 365/158; 365/163
`( 58) Field of Classification Search
`USPC ........................................ 365/185.03, 185.09
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`12/2010 Rao
`7,855,916 B2
`...... 365/163
`2009/0268513 Al* 10/2009 DeAmbroggi et al.
`2009/0307418 Al* 12/2009 Chen et al. .................... 711/105
`
`OTHER PUBLICATIONS
`
`Ismael Chang Ghalimi, Intalio, An Intalio White Paper, Cloud Com(cid:173)
`puting is Memory Bound, May 2010.
`Hynix, 32Gb NAND Flash, HY27UKO8BGFM, Product Descrip(cid:173)
`tion Sheet, Feb. 2007.
`Chris Evans, Consultant with Langton Blue, SearchStorage.co.UK,
`Enterprise MLC: How flash vendors are boosting MLC write endur(cid:173)
`ance, Jun. 3, 2011.
`Jesung Kim et al., a Space-Efficient Flash Translation Layer for
`Compactflash Systems, IEEE Transactions on Consumer Electron(cid:173)
`ics, vol. 48, No. 2, May 2002.
`
`(Continued)
`
`Primary Examiner - Harry W Byrne
`Assistant Examiner - Lance Reidlinger
`(74) Attorney, Agent, or Firm - Law Offices of Eugene M.
`Cummings, P.C.
`
`(57)
`
`ABSTRACT
`
`A flash controller for managing at least one MLC non-volatile
`memory module and at least one SLC non-volatile memory
`module. The flash controller is adapted to determine if a range
`of addresses listed by an entry and mapped to said at least one
`MLC non-volatile memory module fails a data integrity test.
`In the event of such a failure, the controller remaps said entry
`to an equivalent range of addresses of said at least one SLC
`non-volatile memory module. The flash controller is further
`adapted to determine which of the blocks in the MLC and
`SLC non-volatile memory modules are accessed most fre(cid:173)
`quently and allocating those blocks that receive frequent
`writes to the SLC non-volatile memory module and those
`blocks that receive infrequent writes to the MLC non-volatile
`memory module.
`
`11 Claims, 5 Drawing Sheets
`
`60 .. -
`
`~I __ ~
`
`1--
`
`60b
`
`62a,,....---
`
`1--
`
`62b
`
`56 - -
`
`t------,-~-------1
`I
`-----
`FTL
`Interface
`" - - -~ - -~ - - - - -~ - - - - s,
`
`- - - 58
`
`Micron Ex. 1001, p. 1
`Micron v. Vervain
`IPR2021-01547
`
`

`

`US 8,891,298 B2
`Page 2
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`Garth Goodson et al., Design Tradeoffs in a Flash Translation Layer,
`HPCA West 2010 (High Perf Comp Arch Conference, Bangalore,
`India.
`Abdul Rub Aamer Mohammed, Improving Hot Data Identification
`for Hybrid SLC/MLC Device; CSci 8980-Advanced Storage Sys(cid:173)
`tems, Spring 2009.
`Yang Hu, Achieving Page-Mapping FTL Performance at Block(cid:173)
`Mapping FTL Cost by Hiding Address Translation, 26th IEEE Sym(cid:173)
`posium on Massive Storage Systems and Technologies (MSST) May
`3-7, 2010.
`Clinton W. Smullen, IV et al., Accelerating Enterprise Solid-State
`Disks with Non-Volatile Merge Caching, 2010 International Green
`Computing Conference, Aug. 15-18, 2010.
`Monolith 3D, Inc., Introducing our monolithic 3D resistive memory
`architecture,
`http://www.monolithic3d.com/2/post/20 l l/06/intro(cid:173)
`ducing-our-3d-resistive-memory-architecture.htrnl, Jun. 27, 2011.
`Song Jiang et al., S-FTL: An Efficient Address Translation for Flash
`Memory by Exploiting Spatial Locality, Proceedings of the MSST
`2011, May 2011.
`Greg Atwood et al., Intel StrataFlash TM Memory Technology Over(cid:173)
`view, Intel Technology Journal Q4 1997.
`Moinuddin K. Qureshi et al., Morphable Memory System: A Robust
`Architecture for Exploiting Multi-Level Phase Change Memories,
`International Symposium on Computer Architecture, Saint-Malo,
`France, Jun. 19-23, 2010.
`Abhishek Rajimwale et al., Block Management in Solid-State
`Devices, Usenix Conference, Jun. 14-19, 2009.
`Brendan Gregg et al., Sun Storage 7000 Unified Storage System
`L2ARC: Second Level Adaptive Replacement Cache, Oracle White
`Paper-Sun Storage 7000 Unified Storage System L2ARC, May
`2010.
`Chunqiang Tang, FVD: a High-Performance Virtual Machine Image
`Format for Cloud, USENIX Conference Jun. 2011.
`Anand Lal Shimpi, AnandTech, The Crucial m4 (Micron C400) SSD
`Review, Mar. 31, 2011.
`Anand Lal Shimpi, AnandTech, The Intel SSD 320 Review: 25nm G3
`is Finally Here, Mar. 28, 2011.
`Micron Technology, Inc., TN-29-42: Wear-Leveling Techniques in
`NAND Flash Devices Introduction, Oct. 2008.
`Qingsong Wei et al., WAFTL: A Workload Adaptive Flash Transla(cid:173)
`tion Layer with Data Partition, IEEE 27th Symposium on Massive
`Storage Systems and Technologies (MSST), May 23-27, 2011.
`Taeho Kgil et al., Improving NAND Flash Based Disk Caches, Inter(cid:173)
`national Symposium on Computer Architecture, Copyright 2008
`IEEE.
`Samsung Electronics Co., Ltd. , 7th International Symposium on
`Advanced Gate Stack Technology, RRAM Technology from an
`Industrial Perspective, Process Development Team/RRAM PJT In(cid:173)
`Gyu Baek, Sep. 2010.
`
`Paolo Pavan et al., Flash Memory Cells-An Overview, Proceedings
`of the IEEE, vol. 85, No. 8, Aug. 1997.
`Yoshihisaiwataetal.,AHigh-DensityNANDEEPROMwithBlock(cid:173)
`Page Programming for Microcomputer Applications, IEEE Journal
`of Solid-State Circuits, vol. 25, No. 2, Apr. 1990.
`Tae-Sung Jung et al., A ll 7-mm2 3.3-V Only 128-Mb Multilevel
`NAND Flash Memory for Mass Storage Applications, IEEE Journal
`of Solid-State Circuits, vol. 31, No. 11, Nov. 1996.
`Masayoshi Ohkawa et al., A 98 mm2 Die Size 3.3V 64-Mb Flash
`Memory with FN-NOR Type Four-Level Cell, IEEE Journal ofSolid(cid:173)
`State Circuits, vol. 31, No. 11, Nov. 1996.
`Masaki Momodomi et al., An Experimental 4-Mbit CMOS EEPROM
`with a NANO-Structured Cell, IEEE Journal of Solid-State Circuits,
`vol. 24, No. 5, Oct. 1989.
`Ken Takeuchi et al., A Multipage Cell Architecture for High-Speed
`Programming Multilevel NAND Flash Memories, IEEE Journal of
`Solid-State Circuits, vol. 33, No. 8, Aug. 1998.
`Shigeru Atsumi et al., A Channel-Erasing 1.8-V-Only 32-Mb NOR
`Flash EEPROM with a Bitline Direct Sensing Scheme, IEEE Journal
`of Solid-State Circuits, vol. 35, No. 11, Nov. 2000.
`Taehee Cho et al., A Dual-Mode NAND Flash Memory: 1-Gb Mul(cid:173)
`tilevel and High-Performance 512-Mb Single-Level Modes, IEEE
`Journal of Solid-State Circuits, vol. 36, No. 11, Nov. 2001.
`Douglas J. Lee et al., Control Logic and Cell Design for a 4K
`NVRAM, IEEE Journal of Solid-State Circuits, vol. SC-18, No. 5,
`Oct. 1983.
`Duane H. Oto et al., High-Voltage Regulation and Process Consid(cid:173)
`erations for High-Density 5 V-Only E2PROM's, IEEE Journal of
`Solid-State Circuits, vol. SC-18, No. 5, Oct. 1983.
`Gheorghe Samachisa et al., A 128K Flash EEPROM Using Double(cid:173)
`Polysilicon Technology, IEEE Journal of Solid-State Circuits, vol.
`SC-22, No. 5, Oct. 1987.
`Nelson Duann, Silicon Motion, Inc., Flash Memory Sununit, SLC &
`MLC Hybrid, Santa Clara, CA, Aug. 2008.
`Simona Boboila et al., Write Endurance in Flash Drives: Measure(cid:173)
`ments and Analysis, Usenix Conference on File and Storage Tech(cid:173)
`nologies, San Jose, CA, Feb.2010.
`Simona Boboila et al. Write Endurance in Flash Drives: Measure(cid:173)
`ments and Analysis Handout at Unsenix Conference on File and
`Storage Technologies, San Jose, CA, Feb. 2010.
`Silicon Systems, Increasing Flash SSD Reliability, StorageSearch.
`com, Apr. 2005.
`Roberto Bez et al., Introduction to Flash Memory, Proceedings of the
`IEEE, vol. 91, No. 4, Apr. 2003.
`Seagate Technology LLC, The Transition to Advanced Format 4K
`Sector Hard Drives, Apr. 2010.
`Tae-Sun Chung et al., A Survey of Flash Translation Layer, Journal of
`Systems Architecture 55, pp. 332-343, 2009.
`Intel, Understanding the Flash Translation Layer (FTL) Specifica(cid:173)
`tion, Dec. 1998.
`
`* cited by examiner
`
`Micron Ex. 1001, p. 2
`Micron v. Vervain
`IPR2021-01547
`
`

`

`U.S. Patent
`
`Nov. 18, 2014
`
`Sheet 1 of 5
`
`US 8,891,298 B2
`
`10
`-
`
`---- 12
`
`Processor
`
`20
`
`I
`
`14
`
`I
`
`16
`
`/
`
`I/0
`
`DRAM
`
`Device
`Controller
`
`18
`
`I/0 /
`
`Disk(s)
`
`/24
`
`(e.g., rotating media-
`magnetic or optical)
`
`MLC
`flash
`
`/
`
`26
`
`SLC
`flash
`
`~
`28
`
`FIG. 1
`
`Micron Ex. 1001, p. 3
`Micron v. Vervain
`IPR2021-01547
`
`

`

`U.S. Patent
`
`Nov. 18, 2014
`
`Sheet 2 of 5
`
`US 8,891,298 B2
`
`LOGICAL
`ADDRESS RANGE
`
`PHYSICAL
`ADDRESS RANGE
`
`RO
`
`MLC/Block 0
`
`RI
`
`MLC/Block 1
`
`Failed Data
`Integrity Test
`
`~
`
`~
`
`, ____________
`:
`R2
`
`, ............ __________ --- --- --- -- ---.
`MLC/Block 2:
`--------------
`MLC/Block 3
`
`R3
`
`R4
`
`RN
`
`MLC/Block 4
`
`MLC/Block N
`
`FIG. 2a
`
`LOGICAL
`ADDRESS RANGE
`
`PHYSICAL
`ADDRESS RANGE
`
`RO
`
`MLC/Block 0
`
`Rl
`MLC/Block 1
`,------------- -------------- I
`:
`R2
`SLC/Block O :
`"------------ --------------
`R3
`MLC/Block 3
`
`Remapping to SLC
`flash module
`
`R4
`
`RN
`
`MLC/Block 4
`
`MLC/Block N
`
`FIG. 2b
`
`Micron Ex. 1001, p. 4
`Micron v. Vervain
`IPR2021-01547
`
`

`

`U.S. Patent
`
`Nov. 18, 2014
`
`Sheet 3 of 5
`
`US 8,891,298 B2
`
`Begin
`
`Read data quantum
`from DRAM into memory
`of device controller
`
`Read logical address
`range and NAND flash
`physical address range
`to which data quantum
`is to be written into
`memory of device
`controller
`
`Combine contents of
`NAND flash memory
`with data quantum to be
`written
`
`Erase NAND flash
`physical address range
`
`Write combined data to
`appropriate NAND flash
`physical range
`
`Read NAND Flash
`physical address range
`into device controller
`memory
`
`100
`
`102
`
`104
`
`106
`
`108
`
`110
`
`112
`
`FIG. 3a
`
`Micron Ex. 1001, p. 5
`Micron v. Vervain
`IPR2021-01547
`
`

`

`U.S. Patent
`
`Nov. 18, 2014
`
`Sheet 4 of 5
`
`US 8,891,298 B2
`
`Compare Data Written to
`NAND FLASH Physical
`Address Range to Data Read
`from NAND FLASH Physical
`Address Range
`
`114
`
`116
`
`120
`
`Identify next
`quantum of
`available SLC
`NAND flash
`
`122
`
`Remap NAND flash
`physical range to
`next available SLC
`NAND flash
`
`126
`
`Success
`
`118
`
`System
`Failure
`
`124
`
`FIG. 3b
`
`Micron Ex. 1001, p. 6
`Micron v. Vervain
`IPR2021-01547
`
`

`

`U.S. Patent
`
`Nov. 18, 2014
`
`Sheet 5 of 5
`
`US 8,891,298 B2
`
`50
`
`60
`a -------
`
`-----
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`I
`I
`I
`I
`I
`I
`
`'
`
`I
`
`I
`I
`I
`
`___,,,
`
`I
`I
`-----------------------
`I
`
`I
`
`MLC
`
`SLC
`
`I
`I
`
`'
`
`I
`I
`I
`I
`I
`I
`I
`
`'
`'
`'
`SLC
`
`I
`
`62a
`
`56
`
`L---'i"
`
`-----
`
`---
`-------
`
`I
`
`FTL
`
`Interface
`
`54
`
`--------- ----------
`
`FIG. 4
`
`I
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`MLC
`
`I
`
`'
`
`I
`I
`I
`I
`I
`I
`
`MLC
`
`SLC
`
`I
`I
`I
`I
`I
`I
`
`---
`I
`I
`-----------------------
`I
`I
`
`~
`1----..
`
`60b
`
`62b
`
`I
`I
`I
`
`I ~
`~
`i--
`'
`'
`'
`
`I
`
`I
`
`I
`
`SLC
`
`I
`
`58
`
`-------
`----- 52
`
`Micron Ex. 1001, p. 7
`Micron v. Vervain
`IPR2021-01547
`
`

`

`US 8,891,298 B2
`
`1
`LIFETIME MIXED LEVEL NON-VOLATILE
`MEMORY SYSTEM
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`2
`These semiconductor technology driven "flash alterna(cid:173)
`tives," i.e., RRAM, PCM, MAGRAM and others, have sev(cid:173)
`eral advantages over any (SLC or MLC) flash because they: 1)
`allow data to be written over existing data ( without prior erase
`5 of existing data), 2) allow for an erase of individual bytes or
`pages (instead of having to erase an entire block), and 3)
`possess superior endurance (1,000,000 write-erase cycles
`compared to typical 100,000 cycles for SLC flash and less
`than 10,000 cycles for MLC flash).
`HDDs have several platters. Each platter contains 250-5,
`000 tracks ( concentric circles). Each track contains 64 to 256
`sectors. Each sector contains 512 bytes of data and has a
`unique "physical (memory) address." A plurality of sectors is
`typically combined to form a "logical block" having a unique
`15 "logical address." This logical address is the address at which
`the logical block of physical sectors appears to reside from the
`perspective of an executing application program. The size of
`each logical block and its logical address (and/or address
`ranges/boundaries) is optimized for the particular operating
`20 system (OS) and software applications executed by the host
`processor. A computer OS organizes data as "files." Each file
`may be located (stored) in either a single logical block or a
`plurality oflogical blocks, and therefore, the location of files
`typically traverses the boundaries of individual (physical)
`25 sectors. Sometimes, a plurality of files has to be combined
`and/or modified, which poses an enormous challenge for the
`memory controller device of a non-volatile memory system.
`SSDs are slowly encroaching on the HDD space and the
`vast majority of NAND flash in enterprise servers utilizes a
`30 SLC architecture, which further comprises a NAND flash
`controller and a flash translation layer (FTL). NAND flash
`devices are generally fragmented into a number of identically
`sized blocks, each of which is further segmented into some
`number of pages. It should be noted that asymmetrical block
`35 sizes, as well as page sizes, are also acceptable within a device
`or a module containing devices. For example, a block may
`comprise 32 to 64 pages, each of which incorporates 2-4 Kbit
`of memory. In addition, the process of writing data to a
`NAND flash memory device is complicated by the fact that,
`40 during normal operation of, for example, single-level storage
`(SLC), erased bits (usually all bits in a block with the value of
`'1 ') can only be changed to the opposite state (usually 'O')
`once before the entire block must be erased. Blocks can only
`be erased in their entirety, and, when erased, are usually
`45 written to '1' bits. However, if an erased block is already
`there, and if the addresses (block, page, etc.) are allowed, data
`can be written immediately; if not, a block has to be erased
`before it can be written to.
`FTL is the driver that works in conjunction with an existing
`operating system ( or, in some embedded applications, as the
`operating system) to make linear flash memory appear to the
`system like a disk drive, i.e., it emulates a HDD. This is
`achieved by creating "virtual" small blocks of data, or sectors,
`out of flash's large erase blocks and managing data on the
`flash so that it appears to be "write in place" when in fact it is
`being stored in different locations in the flash. FTL further
`manages the flash so that there are clean/erased places to store
`data.
`Given the limited number of writes that individual blocks
`within flash devices can tolerate, wear leveling algorithms are
`used within the flash devices (as firmware commonly known
`as FTL or managed by a controller) to attempt to ensure that
`"hot" blocks, i.e., blocks that are frequently written, are not
`rendered unusable much faster than other blocks. This task is
`65 usually performed within a flash translation layer. In most
`cases, the controller maintains a lookup table to translate the
`memory array physical block address (PBA) to the logical
`
`This application is a Non-Provisional Patent Application,
`claiming priority under 35 U.S.C. § 119( e) to U.S. Provisional
`Application Ser. No. 61/509,257, entitled "Improved Life(cid:173)
`time Mixed Level NAND Flash System," filed Jul. 19, 2011, 10
`the complete disclosure thereof being incorporated herein by
`reference. This application also incorporates by reference the
`complete disclosure of U.S. Pat. No. 7,855,916, entitled
`"Nonvolatile Memory Systems with Embedded Fast Read
`and Write Memories," filed on Oct. 22, 2008 by inventor G. R.
`Mohan Rao, and issued on Dec. 21, 2010. This application
`also incorporates by reference the complete disclosure of
`U.S. patent application Ser. No. 12/915,177, entitled "Non(cid:173)
`volatile Memory Systems with Embedded Fast Read and
`Write Memories," filed on Oct. 29, 2010 (US 2011/0060870
`Al) by inventor G. R. Mohan Rao.
`
`TECHNICAL FIELD
`
`This application relates to a system and method for provid(cid:173)
`ing reliable storage through the use of non-volatile memories
`and, more particularly, to a system and method of increasing
`the reliability and lifetime of a NAND flash storage system,
`module, or chip through the use of a combination of single(cid:173)
`level cell (SLC) and multi-level cell (MLC) NAND flash
`storage without substantially raising the cost of the NAND
`flash storage system. The memory in a total non-volatile
`memory system may contain some SRAM (static random(cid:173)
`access memory), DRAM (dynamic RAM), RRAM (resistive
`RAM), PCM (phase change memory), MA GRAM (magnetic
`random-access memory), NAND flash, and one or more
`HDDs (hard disk drives) when storage of the order of several
`terabytes is required. The SLC non-volatile memory can be
`flash, PCM, RRAM, MAGRAM or any other solid-state non(cid:173)
`volatile memory as long as it has endurance that is superior to
`that of MLC flash, and it provides for data access speeds that
`are faster than that of MLC flash or rotating storage media
`(e.g., HDDs).
`
`BACKGROUND OF THE DISCLOSURE
`
`Non-volatile memories provide long-term storage of data.
`More particularly, non-volatile memories can retain the
`stored data even when not powered. Magnetic (rotating) hard
`disk drives (HDD) dominate this storage medium due to 50
`lower cost compared to solid state disks (SSD). Optical (rotat(cid:173)
`ing) disks, tape drives and others have a smaller role in long(cid:173)
`term storage systems. SSDs are preferred for their superior
`performance ( fast access time), mechanical reliability and
`ruggedness, and portability. Flash memory, more specifically 55
`NAND flash, is the dominant SSD medium today.
`RRAM, PCM, MAGRAM and others, will likely play a
`larger role in the future, each of them having theirown advan(cid:173)
`tages and disadvantages. They may ultimately replace flash
`memories, initially for use as a "write buffer" and later to 60
`replace "SLC flash" and "MLC flash." MLC NAND flash is a
`flash memory technology using multiple levels per cell to
`allow more bits to be stored using the same number of tran(cid:173)
`sistors. In SLC NAND flash technology, each cell can exist in
`one of two states, storing one bit ofinformation per cell. Most
`MLC NAND flash memory has four possible states per cell,
`so it can store two bits of information per cell.
`
`Micron Ex. 1001, p. 8
`Micron v. Vervain
`IPR2021-01547
`
`

`

`US 8,891,298 B2
`
`5
`
`4
`maintains an address map comprising a list of individual
`logical address ranges each of which maps to a similar range
`of physical addresses within either the at least one MLC
`module or the at least one SLC module. After each write to
`(flash) memory, the controller conducts a data integrity check
`to ensure that the data was written correctly. When the data
`was not written correctly, the controller modifies the table so
`that the range of addresses on which the write failed is
`remapped to the next available range of physical addresses
`10 within the at least one SLC module. The SLC module can be
`(NAND) flash, PCM, RRAM, MAGRAM or any other solid(cid:173)
`state non-volatile memory as long as it has endurance that is
`superior to that of MLC flash, and it provides for data access
`speeds that are faster than that of MLC flash or rotating
`15 storage media (e.g., HDDs).
`According to another embodiment of the present disclo(cid:173)
`sure, there is provided a system for storing data which com(cid:173)
`prises a controller that is further adapted to determine which
`of the blocks of the plurality of the blocks in the MLC and
`20 SLC non-volatile memory modules are accessed most fre(cid:173)
`quently and wherein the controller segregates those blocks
`that receive frequent writes into the at least one SLC non(cid:173)
`volatile memory module and those blocks that receive infre(cid:173)
`quent writes into the at least one MLC non-volatile module.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The present disclosure will be more fully understood by
`reference to the following detailed description of one or more
`30 preferred embodiments when read in conjunction with the
`accompanying drawings, in which like reference characters
`refer to like parts throughout the views and in which:
`FIG. 1 is a block diagram of a computer system incorpo(cid:173)
`rating one embodiment of the present disclosure;
`FIG. 2 is a drawing depicting a translation table/address
`map in accordance with one embodiment of the present dis(cid:173)
`closure;
`FIGS. 3a and 3b are a flow chart illustrating an exemplary
`method for use in implementing one embodiment of the
`present disclosure; and
`FIG. 4 is a block diagram depicting one embodiment of the
`present disclosure for implementation within a NAND flash
`module.
`
`DETAILED DESCRIPTION OF THE
`ILLUSTRATED EMBODIMENT
`
`3
`block address (LBA) used by the host system. The control(cid:173)
`ler's wear-leveling algorithm determines which physical
`block to use each time data is programmed, eliminating the
`relevance of the physical location of data and enabling data to
`be stored anywhere within the memory array and thus pro-
`longing the service life of the flash memory. Depending on
`the wear-leveling method used, the controller typically either
`writes to the available erased block with the lowest erase
`count ( dynamic wear leveling); or it selects an available target
`block with the lowest overall erase count, erases the block if
`necessary, writes new data to the block, and ensures that
`blocks of static data are moved when their block erase count
`is below a certain threshold (static wear leveling).
`MLC NAND flash SSDs are slowly replacing and/or coex(cid:173)
`isting with SLC NAND flash in newer SSD systems. MLC
`allows a single cell to store multiple bits, and accordingly, to
`assume more than two values; i.e., 'O' or '1 '. Most MLC
`NAND flash architectures allow up to four ( 4) values per cell;
`i.e., '00', '01', '10', or '11'. Generally, MLC NAND flash
`enjoys greater density than SLC NAND flash, at the cost of a
`decrease in access speed and lifetime (endurance). It should
`be noted, however, that even SLC NAND flash has a consid(cid:173)
`erably lower lifetime (endurance) than rotating magnetic
`media (e.g., HDDs), being able to withstand only between
`50,000 and 100,000 writes, and MLC NAND flash has a much 25
`lower lifetime (endurance) than SLC NAND flash, being able
`to withstandonlybetween3,000 and 10,000writes.As is well
`known in the art, any "write" or "program" to a block in
`NAND flash (floating gate) requires an "erase" ( of a block)
`before "write."
`Despite its limitations, there are a number of applications
`that lend themselves to the use ofMLC flash. Generally, MLC
`flash is used in applications where data is read many times
`(but written few times) and physical size is an issue. For
`example, flash memory cards for use in digital cameras would 35
`be a good application of MLC flash, as MLC can provide
`higher density memory at lower cost than SLC memory.
`When a non-volatile storage system combines HDD, SLC
`and MLC (setting aside volatile memory for buffering, cach(cid:173)
`ing etc) in a single (hybrid) system, new improvements and 40
`solutions are required to manage the methods of writing data
`optimally for improved life time (endurance) of flash
`memory. Accordingly, various embodiments of a NAND
`flash storage system that provides long lifetime (endurance)
`storage at low cost are described herein.
`The following description is presented to enable one of
`ordinary skill in the art to make and use the disclosure and is
`provided in the context of a patent application and its require(cid:173)
`ments. Various modifications to the preferred embodiment
`and the generic principles and features described herein will 50
`be readily apparent to those skilled in the art. Thus, the
`present disclosure is not intended to be limited to the embodi(cid:173)
`ments shown, but is to be accorded the widest scope consis(cid:173)
`tent with the principles and features described herein.
`
`45
`
`The present disclosure is directed to the reliable storage of
`data in read and write memory, and, in particular, to the
`reliable storage of data in non-volatile memory, such as, for
`example, NAND flash. Generally, and in particular regard to
`NAND flash memory, two separate banks ofNAND flash are
`maintained by a controller. One bank contains economical
`MLC NAND flash, while a second bank contains high endur-
`55 ance SLC NAND flash. The controller conducts a data integ(cid:173)
`rity test after every write. If a particular address range fails a
`data integrity test, the address range is remapped from MLC
`NAND flash to SLC NAND flash. As the SLC NAND flash is
`used to boost the lifetime (endurance) of the storage system,
`it can be considerably lesser in amount than the MLC NAND
`flash. For example, a system may set SLC NAND flash equal
`to 12.5% or 25% of MLC NAND flash (total non-volatile
`memory storage space=MLC+SLC).
`Turning to the Figures and to FIG. 1 in particular, a com(cid:173)
`puter system 10 depicting one embodiment of the present
`disclosure is shown. A processor 12 is coupled to a device
`controller 14, such as a chipset, using a data link well known
`
`SUMMARY OF THE DISCLOSURE
`
`According to one embodiment of the present disclosure,
`there is provided a system for storing data which comprises at
`least one MLC non-volatile memory module (hereinafter 60
`referred to as "MLC module") and at least one SLC non(cid:173)
`volatile memory module (hereinafter referred to as "SLC
`module"), each module comprises a plurality of individually
`erasable blocks. The data storage system according to one
`embodiment of the present disclosure further comprises a 65
`controller for controlling both the at least one MLC module
`and the at least one SLC module. In particular, the controller
`
`Micron Ex. 1001, p. 9
`Micron v. Vervain
`IPR2021-01547
`
`

`

`US 8,891,298 B2
`
`5
`in the art, such as a parallel bus or packet-based link. The
`device controller 14 provides interface functions to the pro(cid:173)
`cessor 12. In some computer systems, the device controller 14
`may be an integral part of the (host) processor 12. The device
`controller 14 provides a number of input/output ports 16 and 5
`18, such as, for example, serial ports ( e.g., USB ports and
`Firewire ports) and network ports (e.g., Ethernet ports and
`802.11 "Wi-Fi" ports). The device controller 14 may also
`control a bank of, for example, DRAM 20. In addition, the
`device controller 14 controls access to one or more disks 24, 10
`such as, for example, a rotating magnetic disk, or an optical
`disk, as well as two or more types of NAND flash memory.
`One type of NAND flash memory is a MLC NAND flash
`memory module 26. Another type ofNAND flash memory is
`a SLC NAND flash memory module 28.
`The device controller 14 maintains a translation table/ad(cid:173)
`dress map which may include address translations for all
`devices in the computer system. Nonetheless, the discussion
`in the present disclosure will be limited only to NAND flash
`memory modules. In particular, the device controller 14
`maintains a translation table that maps logical computer sys(cid:173)
`tem addresses to physical addresses in each one of the MLC(cid:173)
`and SLC-NAND flash memory modules 26 and 28, respec(cid:173)
`tively. As MLC flash memory is less expensive than SLC flash
`memory, on a cost per bit basis, the translation table will
`initially map all logical NAND flash addresses to the MLC
`NAND flash memory module 26. The address ranges within
`the translation table will assume some minimum quantum,
`such as, for example, one block, although a smaller size, such
`as one page could be used, if the NAND flash has the capa(cid:173)
`bility of erasing the smaller size quantum.
`A "read-modify-write" scheme is used to write data to the
`NAND flash. Data to be written to NAND flash is maintained
`in DRAM 20. After each write to an address within a particu(cid:173)
`lar address range, the device controller 14 will-as time per(cid:173)
`mits-perform a read on the address range to ensure the
`integrity of the written data. If a data integrity test fails, the
`address range is remapped from the MLC NAND flash
`memory module 26 to the next available address range in the
`SLC NAND flash memory module 28.
`FIG. 2 illustrates one embodiment of a translation table/
`address map of the present disclosure. In FIG. 2a, a list of
`logical address ranges (RO-RN) is translated to physical
`address ranges.As illustrated, all of the logical address ranges
`are translated to blocks on the MLC NAND flash memory
`module 26. However, through the application of a data integ(cid:173)
`rity verification check (explained in more detail below) it is
`determined that, for example, address range R2 corresponds
`to failed quanta of data stored in block 2 of the MLC NAND
`flash memory module 26. FIG. 2b shows the quanta of data
`which failed the data integrity verification check ( see FIG. 2a)
`remapped to the next available range of physical addresses
`within the SLC NAND flash memory module 28, in this
`example, SLC/block 0.
`FIGS. 3a and 3b are a flow chart illustrating a method for 55
`utilizing a NAND flash memory system incorporating one
`embodiment of the present disclosure. The method begins in
`a step 100, when a command to write a quantum of data stored
`in DRAM to a particular location in NAND flash memory is
`received. In step 102, the quantum of data is read from DRAM 60
`into memory within the device controller (which acts as the
`memory controller). In step 104, both the logical address
`range and the NAND flash physical address range to which
`the quantum of data is to be written, is read into memory of the
`device controller. In step 106, the quantum of data to be
`written is combined with the contents of the NAND flash
`memory. In step 108, the NAND flash physical address range
`
`6
`to be written is erased. In step 110, the combined data is
`written to the appropriate NAND flash physical address
`range. In step 112 the NAND flash physical address range that
`was written in step 110 is read into device controller memory.
`The flowchart continues in FIG. 3b. In step 114 the NAND
`flash physical address range that was read into device con(cid:173)
`troller memory is compared with the retained data represent(cid:173)
`ing the combination of the previous contents of the physical
`address range and the quantum of data to be written. In step
`116, if the retained data matches the newly stored data in the
`NAND flash memory, the write was a success, and the method
`exits in step 118. However, if the retained data does not match
`the newly stored data in the NAND flash memory, the method
`executes step 120, which identifies the next quantum of avail-
`15 able SLC NAND flash memory addresses. In step 122, a
`check is made to determine if additional SLC NAND flash
`memory is available, and, if not, the NAND flash memory
`system is marked as failed, prompting a system alert step 124.
`However, if additional SLC NAND flash memory is avail-
`20 able, the failed NAND flash physical address range is
`remapped to the next available quantum of SLC NAND flash
`memory in step 126. Execution then returns to step 110,
`where the write is repeated.
`Another application of one embodiment of the present
`25 disclosure, not depicted in any of the drawings, is to allocate
`"hot" blocks; i.e., those blocks that receive frequent writes,
`into the SLC NAND flash memory module 28, while allocat(cid:173)
`ing "cold" blocks; i.e., those blocks that only receive infre(cid:173)
`quent writes, into the MLC NAND flash memory module 26.
`30 This could be accomplished within the device controller 14
`described above, which could simply maintain a count of
`those blocks that are accessed (written to) most frequen