United States Patent (19)
`Bruce et al.
`
`54
`
`EXPANDABLE FLASH-MEMORY MASS
`STORAGE USING SHARED BUDDY LINES
`AND INTERMEDIATE FLASH-BUS
`BETWEEN DEVICE-SPECIFIC BUFFERS
`AND FLASH-INTELLIGENT DMA
`CONTROLLERS
`
`(75)
`
`Inventors: Ricardo H. Bruce, Union City;
`Rolando H. Bruce, South San
`Francisco; Earl T. Cohen, Fremont, all
`of Calif.
`
`Assignee: Bit Microsystems, Inc., Fremont, Calif.
`
`Appl. No.: 939,601
`Filed:
`Sep. 29, 1997
`Related U.S. Application Data
`
`Continuation-in-part of Ser. No. 918,203, Aug. 25, 1997.
`Int. Cl. ................................................... G11C 13700
`U.S. Cl. ................................ 365/185.33; 365/189.01;
`36.5/218
`Field of Search ......................... 365/189.01, 230.01,
`365/185.33, 218
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`6/1988 Sparks ................................ 365/189.01
`4,752,871
`5,297,148 3/1994 Harari et al.......
`... 371/10.2
`5,379,401
`1/1995 Robinson et al.
`... 395/425
`5,388,083 2/1995 Assar et al. ............................. 36.5/218
`5,432,748 7/1995 Hsu et al. .....
`... 365/230.01
`5,448,577 9/1995 Wells et al. ............................ 371/10.1
`5,479,638 12/1995 Assar et al. ............................. 395/430
`5,500.826 3/1996 Hsu et al. .........
`... 365/230.01
`5,509,134 4/1996 Fandrich et al. ........................ 395/430
`5,513,138 4/1996 Manabe et al. ...
`365/185.33
`5,524,231 6/1996 Brown ..................................... 395/428
`5,530,828 6/1996 Kaki et al. .............................. 395/430
`
`
`
`USOO5822251A
`Patent Number:
`11
`(45) Date of Patent:
`
`5,822,251
`Oct. 13, 1998
`
`- - - - - 36.5/218
`
`5.535,328 7/1996 Harari et al. ....................... 395/182.05
`5,559,956 9/1996 Sukegawa .
`... 395/182.06
`5,568,439 10/1996 Harari .........
`5,572,466 11/1996 Sukegawa .......................... 365/185.33
`5,594,883
`1/1997 Pricer ...................................... 395/440
`5,602987 2/1997 Harari et al.....
`395/182.06
`5,603,001
`2/1997 Sukegawa et al. ..................... 395/430
`5,606,529 2/1997 Honma et al. ..
`365/230.01
`5,619,470 4/1997 Fukumoto ............................... 365/228
`5,627,783 5/1997 Miyauchi ........
`... 365/185.33
`5,640,349 6/1997 Kakinuma et al. ................ 365/185.33
`Primary Examiner Terrell W. Fears
`Attorney, Agent, or Firm-Stuart T. Auvinen
`57
`ABSTRACT
`A flash-memory System is expandable. Rather than directly
`connecting individual flash-memory chips to a controller,
`flash buffer chips are used. Each flash buffer chip can
`connect to four banks of flash-memory chips. Chip enables
`for individual chips in a bank are generated from an address
`sent to the flash buffer chips. Two flash-specific DMA
`controllers are provided, each with four DMA state
`machines for controlling the four banks of flash-memory
`chips attached to a flash buffer chip. This allows for four
`way interleaving. Two flash buses connect the two DMA
`controllers to flash buffer chips. The flash bus has a narrow
`byte-wide interface to Send command, address, and data
`bytes from the DMA controller to the flash buffer chips.
`These command, address, and data bytes are then passed
`through the flash buffer chip to the flash-memory chips. Two
`additional command signals on the flash bus are used to
`select and control the flash buffer chips. Busy signals from
`all flash-memory chips in a bank are connected together, and
`the four busy signals from the four banks are time
`multiplexed to a single common busy line for the flash bus.
`The four DMA state machines each monitor one period of
`the busy line, allowing four flash operations to be monitored
`at a time, even though only one data transfer can occur
`acroSS the flash bus.
`
`24 Claims, 8 Drawing Sheets
`
`FLASH
`BUFFER
`CHIP 14
`
`BUFFER
`FLASH -
`CHIP 14Fssy
`
`J
`
`
`
`FASH
`BUS-A
`
`---
`
`L
`
`FLASH - BUFFER
`
`BSY
`
`CHIP 14
`
`20
`
`10
`
`BSY
`
`20 LE
`
`FLASH
`SPECIFIC
`DMA 1650: ---
`
`SCS
`CTLR
`
`Pe. FLASH
`BUFFER
`CHIP 14
`ele
`FLASH
`SPECIFIC
`DMA 12
`---------------
`
`DRAM
`CACHE
`32
`
`LOCAL BUS
`
`LOCAL
`PROC
`MEM 27
`
`t;
`
`26
`
`ROM
`28
`
`30
`
`KIOXIA Ex-1006, Page 1
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 1 of 8
`
`5,822,251
`
`HSV/T-|
`
`WEWN
`
`ZZ
`
`
`
`
`
`KIOXIA Ex-1006, Page 2
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 2 of 8
`
`5,822,251
`
`a1410aas||olsioadds
`Lchoywa
`
`
`
`9¢vce«€=6hGCOSazoz
`
`-HSV14
`
`snd1V900O1
`
`00ud
`
`KIOXIA Ex-1006, Page 3
`
`KIOXIA Ex-1006, Page 3
`
`
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 3 of 8
`
`5,822,251
`
`]]
`
`09
`
`98
`
`
`
`NES) OOIE
`
`
`
`|-NES) OOE·
`
`| 6
`
`SCIWO
`
`SCIWO
`
`
`
`
`
`
`
`we
`
`J
`
`D D
`
`KIOXIA Ex-1006, Page 4
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 4 of 8
`
`5,822,251
`
`“*OLv-0SM
`
`cL
`
`
`
`Z:039pS
`
`YLNDLId-c
`
`054LIHASO5
`=<AS)9
`
`bZvl
`
`OASNA
`
`e€Asng9S
`
`
`
`ZASNA»LASNGAS@-d4
`
`SdIHO
`
`
`
` «oFWAW+-OdMAdMHSVld
`
`—
`
`
`
`Loviva[ezvyols
`
`L-031V1asay
`
`L:0519]82
`
`L-O0INDs4
`
`KIOXIA Ex-1006, Page 5
`
`KIOXIA Ex-1006, Page 5
`
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 5 of 8
`
`5,822,251
`
`0
`CN
`
`EW||
`
`Z89999
`
`|-
`0ÅSTE
`
`| ÅSTE
`
`99
`
`
`
`
`
`KIOXIA Ex-1006, Page 6
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 6 of 8
`
`5,822,251
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`LIV/WWCT-|
`
`CRJOER]
`
`LVLSOER]
`
`KIOXIA Ex-1006, Page 7
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 7 of 8
`
`5,822,251
`
`|HSYJCIV/
`
`ESV/N-IETES
`
`CIWOESVYJE
`
`XHOOTINTI
`
`
`
`
`
`
`
`
`
`
`
`
`OTCIWO LEIS
`| HC|WO LES·
`
`
`
`KIOXIA Ex-1006, Page 8
`
`

`

`U.S. Patent
`
`Oct. 13, 1998
`
`Sheet 8 of 8
`
`5,822,251
`
`CNN ON
`
`ON
`
`ON ONCYN ON
`
`Rsssssss
`Rsssssss
`Rsssssss
`Rsssssss
`
`CNCN ON ON NCN, CNN
`
`CO
`C
`-
`
`:
`
`
`
`Rsssssss
`Rsssssss
`Rsssssss
`Rsssssss
`
`: ,s 3
`:
`
`o
`
`S
`
`:
`
`so O
`it
`C/D SS a
`1- c.5
`
`KIOXIA Ex-1006, Page 9
`
`

`

`1
`EXPANDABLE FLASH-MEMORY MASS
`STORAGE USING SHARED BUDDY LINES
`AND INTERMEDIATE FLASH-BUS
`BETWEEN DEVICE-SPECIFIC BUFFERS
`AND FLASH-INTELLIGENT DMA
`CONTROLLERS
`
`RELATED APPLICATION
`This application is a continuation-in-part of the
`co-pending application for “Unified Re-Map and Cache
`Index Table with Dual Write-Counters for Wear-Leveling of
`Non-Volatile Flash RAM Mass Storage”, filed Aug. 25,
`1997, U.S. Ser. No. 08/918,203.
`BACKGROUND OF THE INVENTION
`1. Field of the Invention
`This invention relates to non-volatile memory Storage
`Systems, and more particularly to expansion of flash
`memory Systems.
`2. Description of the Related Art
`Hard disks have been the traditional Storage medium in
`computer Systems. More recently these hard disks are being
`replaced by non-volatile Semiconductor memory. An array
`of non-volatile random-access memories (NVRAM's) or
`non-volatile flash memories Substitutes for the hard-disk
`Storage. These memory devices use electrically-erasable
`programmable read-only-memory (EEPROM) technology
`for Storage cells. Floating polysilicon gates in these Storage
`cells retain charge and State when power is lost, providing
`non-volatile Storage. These flash-memory Systems are fre
`quently used as a mass-Storage replacement for a hard disk
`on a personal computer and are thus Sometimes referred to
`as a flash "disk', even though no rotating disks are used.
`Hard disks Suffer from their inherent fixed size, since
`expansion of the Storage capacity of a hard disk is not
`possible. An entire new disk assembly is needed to expand
`disk storage. Like DRAM memory, flash memory has the
`potential for expansion. However, current flash-memory
`Systems have not fully realized the potential for expandabil
`ity. Rigid bus architectures often restrict expansion of flash
`memory Systems. Like hard disks, the expansion of these
`flash-memory Systems is limited.
`Slow Flash Operations
`Flash EEPROM chips are divided into pages and blocks.
`A 64 Mbit flash chip typically has 512-byte pages which
`happens to match the sector size for IDE and SCSI hard
`diskS. Rather than writing to or reading from just one word
`in the page, the entire page must be read or written at the
`same time; individual bytes cannot be written. Thus flash
`memory operations are inherently slow Since an entire page
`must be read or written.
`Each page must be cleared of any previous data before
`being written, clearing is accomplished by a flash erase
`cycle. An entire block of pages (typically 16 pages) is erased
`at once. A block of 16 pages must be erased together, while
`all 512 bytes on a page must be written together. Erase is
`therefore significantly slower than read or write operations.
`Interleaving on a page basis increases the effective block
`Size Since Several banks are grouped together. For example,
`a 4:1 page-interleaved System has an effective block size of
`4 blockS. The larger block size can further increase erase
`time. Interleaving is preferably done on a block basis that
`doesn’t increase the effective block size.
`A busy Signal is usually provided by each flash-memory
`chip to indicate when the read, write, or erase operation has
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`5,822,251
`
`2
`completed. The busy Signal allows a local processor to
`continue with other tasks while the flash-memory integrated
`circuit chip performs the flash operation. Flash operations
`generally cannot be stopped once initiated. Erase operations
`are the exception-they can be Suspended and resumed later,
`but generally only one erase operation per flash-memory
`chip can be active or Suspended at a time.
`Busy Signals complicate expansion, as the local processor
`must receive all the busy Signals. AS more flash-memory
`devices are added to the System, the local processor must
`receive additional busy Signals from the additional chips.
`While all the busy Signals can be connected together, this
`can limit performance as parallel flash operations driving the
`Same busy line can interfere with each other, requiring that
`the local processor read Status registers on all the flash chips
`when any device asserts the common busy line. Connecting
`all the busy Signals together generally also means that the
`busy line will stay asserted (busy) as long as any device is
`busy-this limits parallelism by not providing Separate
`indications of when each flash-memory chip has completed
`an operation.
`DMA for Data Transfers
`Direct-memory access (DMA) has been used to facilitate
`data transfers. While DMA is efficient for transfers of raw
`data to a memory, flash-memory chips also require com
`mand and address Sequences to Set up the relatively long
`flash operations. DMA is not well-suited to transfer
`addresses and commands Since it is designed to transfer long
`Strings of data beginning at a starting address through an
`ending address.
`These command and address Sequences are inputted to the
`flash-memory chips through shared data I/O pins, and gen
`erally require special control signals to be asserted to
`distinguish whether the Shared data I/O pins contain data,
`address, or command information. Often only 8 data I/O
`pins are provided, limiting transferS to one byte of a
`command, address, or data per cycle. Thus one or two cycles
`are needed for each command and two or three cycles are
`needed for each address inputted to the flash-memory chips.
`It is desired to modify a DMA controller so that com
`mands and addresses can also be input to a flash-memory
`chip over the shared data/address/command I/O pins. It is
`also desired to allow for expansion of Storage capacity of a
`flash-memory System. A high-performance flash-memory
`System performing flash operations in parallel is desirable,
`even when expansion flash chips are added.
`SUMMARY OF THE INVENTION
`An expandable flash-memory System has a host interface
`that receives requests for access of flash memory. A flash
`Specific direct-memory access (DMA) controller controls
`access of the flash memory. The flash-specific DMA con
`troller generates a Sequence of command bytes followed by
`address bytes to initiate a data transfer from the flash
`memory.
`A flash bus is coupled to the flash-specific DMA control
`ler. The flash bus has shared lines that transmit the Sequence
`of command bytes and address bytes from the flash-specific
`DMA controller. The shared lines also transmit data bytes
`from the flash-specific DMA controller for the host interface.
`Flash buffer chips are coupled to the flash bus. They
`generate control Signals that control flash-memory chips.
`The flash buffer chips pass the Sequence of command bytes
`and address bytes from the flash-specific DMA controller to
`the flash memory.
`A plurality of flash-memory chips are arranged in banks
`that share some of the control signals from a flash buffer
`
`KIOXIA Ex-1006, Page 10
`
`

`

`3
`chip. The flash-me mory chips are non-volatile
`Semiconductor-memory chips that retain data when power is
`no longer applied.
`The flash bus accepts additional flash-memory chips
`attached through the flash buffer chips to expand a Storage
`capacity of the expandable flash-memory System. Thus the
`flash-memory chips are buffered by the flash buffer chips to
`the flash bus and the flash-specific DMA controller.
`In further aspects of the invention the flash bus has
`command lines to indicate transmission of a Select code to
`the flash buffer chips. The flash buffer chips each have a
`Select decoder that receives the select code. Each flashbuffer
`chip generates a plurality of chip-enable control Signals to
`the flash-memory chips. One of the chip-enable control
`Signals is activated to enable a Selected flash-memory chip
`while other chip-enable control signals disable other flash
`memory chips. Thus the chip-enable control signals are
`generated by the flash buffer chips from the select code from
`the flash-specific DMA controller.
`In further aspects of the invention each flash-memory chip
`outputs a busy Signal to indicate when a flash operation is in
`progreSS within the flash-memory chip. All busy Signals
`from flash-memory chips in a bank that is connected to a
`flash buffer chip are connected to a common bank-busy line.
`The bank-busy line for each bank is input to the flash buffer
`chip.
`The flash buffer chip also has a busy combining means
`that receives the bank-busy signals from all banks of flash
`memory chips attached to the flash buffer chip. It combines
`the bank-busy Signals into a merged busy Signal output to the
`flash bus. Thus busy signals from individual flash-memory
`chips are combined by the flash buffer chips.
`In Still further aspects the merged busy signals from all
`flash buffer chips on the flash bus are connected to a
`common flash-bus busy line. The flash-specific DMA con
`troller monitors the common flash-bus busy line to deter
`mine when the flash operation is in progreSS in the flash
`memory chip. Thus all busy Signals are combined into a
`single busy line to the flash-specific DMA controller.
`In further aspects of the invention a Second flash bus is
`coupled to a Second flash-Specific DMA controller. A Second
`plurality of flash buffer chips are coupled to the second flash
`bus. Each flash buffer chip in the Second plurality generates
`control Signals and buffers command and address bytes from
`the second flash bus to additional banks of flash buffer chips.
`Thus the second flash bus accesses the additional banks of
`flash buffer chips.
`In other aspects of the invention a flash-memory Storage
`peripheral has a host interface to a host System and a local
`processor that controls the flash-memory Storage peripheral.
`A read-only memory (ROM) is coupled to the local proces
`Sor. It Stores routines for execution by the local processor.
`The routines include wear-leveling routines for re-mapping
`data from over-used or faulty memory blocks to unused
`memory blockS.
`A cache is coupled to the local processor. It temporarily
`Stores data from the host in Volatile memory that loses data
`when power is disconnected. A first flash-specific DMA
`controller is coupled to the local processor. It generates
`command, address, and data Sequences to a first flash
`memory chip in a format required by the first flash-memory
`chip. A first flash bus is coupled to the first flash-specific
`DMA controller. It transferS data, addresses, and commands
`over shared address/data/command lines. A first flash buffer
`chip is coupled to the first flash bus. It sends the data,
`addresses, and commands from the Shared lines of the first
`flash bus to the first flash-memory chip.
`
`4
`A plurality of other flash-memory chips are arranged into
`banks. The flash-memory chips are non-volatile Semicon
`ductor memory chips that retain data when power is lost.
`Other flash buffer chips are coupled to the first flash bus.
`Each flash buffer chip is coupled to a different plurality of
`banks of the flash-memory chips.
`A second flash-specific DMA controller is coupled to the
`local processor. It generates command, address, and data
`Sequences to a Second flash-memory chip in a format
`required by the Second flash-memory chip. A Second flash
`bus is coupled to the second flash-specific DMA controller.
`It transferS the data, addresses, and commands over shared
`address/data/command lines. A Second flash buffer chip is
`coupled to the Second flash bus. It sends the data, addresses,
`and commands from the shared lines of the second flash bus
`to the second flash-memory chip. Other flash buffer chips are
`coupled to the second flash bus. Each flash buffer chip is
`coupled to a different plurality of banks of the flash-memory
`chips. Thus two flash-specific DMA controllers control
`access of flash-memory chips through flash buffer chips
`connected by two flash buses.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram of a large flash-memory System
`with dual DMA controllers driving dual flash busses.
`FIG. 2 is an expandable flash-disk System using dual
`DMA and dual flash busses.
`FIG. 3 is a diagram of an ASIC chip containing dual DMA
`controllerS modified for controlling flash memories.
`FIG. 4 is a diagram of a flash buffer chip for an expand
`able flash-memory System.
`FIG. 5 is a timing diagram of a time-multiplexed busy
`Signal.
`FIG. 6 highlights the busy signals from the flash-memory
`chips to the flash-specific DMA controller.
`FIG. 7 is a state diagram of the flash-specific DMA state
`machines.
`FIG. 8 is a state diagram of the flash bus state machine.
`FIG. 9 illustrates expansion cards in a flash memory
`System.
`
`DETAILED DESCRIPTION
`The present invention relates to an improvement in flash
`memory Systems. The following description is presented to
`enable one of ordinary skill in the art to make and use the
`invention as provided in the context of a particular applica
`tion and its requirements. Various modifications to the
`preferred embodiment will be apparent to those with skill in
`the art, and the general principles defined herein may be
`applied to other embodiments. Therefore, the present inven
`tion is not intended to be limited to the particular embodi
`ments shown and described, but is to be accorded the widest
`Scope consistent with the principles and novel features
`herein disclosed.
`An expandable flash-memory System uses multiple levels
`of busses. The flash-memory chips are buffered from an
`intermediate flash bus by flash buffer chips that combine
`busy Signals from an array of flash-memory chips. Flash
`Specific direct-memory access (DMA) drives commands,
`addresses, and data out over the flash bus.
`FIG. 1 is a block diagram of a large flash-memory System
`with dual DMA controllers driving dual flash busses. High
`level requests from a local processor are Sent to flash
`memory over buS 22. Each request is translated into a
`
`5,822,251
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`KIOXIA Ex-1006, Page 11
`
`

`

`5,822,251
`
`15
`
`25
`
`35
`
`40
`
`S
`Sequence of commands, address bytes, and data transferS by
`one of the flash-specific DMA controllers 12, 16. DMA
`controllers 12, 16 transfer not only data, as in traditional
`DMA, but also generate the appropriate command and
`address bytes that are transmitted over the Same lines of the
`flash bus as the data.
`Commands, address bytes, and data bytes are transferred
`from DMA controller 12, 16 to flash-memory chips 20 over
`flash busses 10, 18, which have 8 multiplexed data/address/
`command lines. In addition to the 8 shared data/address/
`command lines, flash busses 10, 18 also have 2 lines for an
`encoded command. This 2-bit encoded command is not sent
`to flash-memory chips 20, but is instead used to Select and
`control flash buffer chips 14. Thus each flash bus has an 8-bit
`portion destined for flash buffer chips 14 or for flash
`memory chips 20, and a 2-bit portion sent only for flash
`buffer chips 14.
`DMA controller 12 drives flash bus 10, while DMA
`controller 16 drives second flash bus 18. Flash busses 10, 18
`can operate at the same time, allowing flash operations to be
`initiated and processed in parallel. Each flash buffer chip 14
`can control several banks of flash-memory chips 20. Each
`bank can be separately accessed, allowing many flash opera
`tions to be performed in parallel.
`The busy signals from all flash-memory chips in a bank
`are connected together, So only one flash-memory chip in a
`bank can be active at a time. However, each flash buffer chip
`14 receives Separate busy Signals from each Separate bank;
`in FIG. 1, each flash buffer chip 14 has two banks attached
`and receives two Separate busy Signals. A preferred embodi
`ment has four banks attached to each flash buffer chip 14.
`Each bank has eight flash memory chips.
`An expandable architecture is provided by flash busses
`10, 18. Additional banks of flash-memory chips can be
`added to existing flash buffer chips, or modules of flash
`memory chips with a flash buffer chip can be plugged into
`a flash bus. The flash bus is an intermediate bus that
`facilitates expansion Since any number of flash buffer chips
`can be added.
`Performance is enhanced because two DMA controllers
`are provided, able to launch two new flash operations at
`once. Once launched, flash operations are monitored by the
`flash buffer chips, which receive the busy lines from the
`flash-memory chips. Thus the DMA controller can launch
`additional flash operations to other flash-memory chips. This
`inherent parallelism improves performance by allowing
`multiple flash operations to be performed at the same time.
`Indeed, multiple flash operations can be launched or com
`pleted at the same time since two DMA controllers are
`provided.
`Expandable Flash-Disk System-FIG. 2
`FIG. 2 is an expandable flash-disk System using dual
`DMA and dual flash busses. Banks of flash-memory chips 20
`connect to flash buffer chips 14, which are connected to flash
`busses 10, 18. Expansion occurs when additional flash buffer
`chips are connected to flash busses 10, 18. Since relatively
`few flash buffer chips are connected to each flash bus,
`loading from additional chips is not critical. A great many
`flash-memory chips can be added Since each flash buffer
`chip can connect to four banks of 8 flash-memory chips, a
`total of 32 flash-memory chips per flash buffer chip.
`Flash specific DMA controllers 12, 16 are preferably
`contained in a single Application-Specific Integrated Circuit
`(ASIC) 50. ASIC 50 connects DMA controllers 12, 16 to
`local bus 30. Local bus 30 connects DRAM cache 32, local
`processor 26, and Small-computer System interface (SCSI)
`controller 34 to DMA controllers 12, 16.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`Requests from host 36 are received by SCSI controller 34
`and driven onto local bus 30. Local processor 26 responds
`to these host requests by storing host data into DRAM cache
`32 for writes, or reading data from flash-memory chips 20 or
`from DRAM cache 32 for reads. ROM 28 contains firmware
`code of routines that execute on local processor 26 to
`respond to host requests. Other System-maintenance routines
`are stored on ROM 28, such as wear-leveling and copy-back
`routines. DRAM cache 32 is preferably under firmware
`control by local processor 26, and thus the local processor's
`local memory 27 and DRAM cache 32 may share the same
`physical DRAM chips.
`Accesses of flash-memory chips 20 by local processor 26
`are initiated by local processor 26 Sending a high-level
`command to one of flash-specific DMA controllers 12, 16.
`These DMA controllers then generate Sequences of com
`mand and address bytes, and transfer data. Many Sequences
`may be needed, Such as for block reads and writes. A block
`read requires that many page read Sequences be performed,
`each Sequence generally Sending command and address
`bytes to the flash-memory chips through the flash buffer
`chips. Some flash-memory chips also have a Sequential read
`mode where command and address bytes need only be sent
`for the first page in a Sequence.
`The flash-memory chips being read, written, or erased
`remain busy for Some period of time after the command and
`address bytes are sent. The flash buffer chips combine and
`Send the busy Signals from multiple flash-memory chips to
`the DMA controller. Once the busy signal is de-asserted,
`indicating that the flash-memory chip is ready, the DMA
`controller continues the operation, perhaps reading the data
`or moving on to the next page in the block. Finally, once all
`flash operations have finished, the DMA controller signals to
`local processor 26 that the high-level flash operation is
`complete.
`Flash-Specific DMA-FIG. 3
`FIG. 3 is a diagram of an ASIC chip containing dual DMA
`controllers modified for controlling flash memories. Data
`received from flash busses 10, 18 is buffered by FIFOs 41,
`42 before being sent to the DRAM cache. Having separate
`FIFOs 41, 42 for flash busses 10, 18 allows for independent
`buS operation. One flash bus can be reading from flash
`memory chips while the other is writing to flash-memory
`chips. Flash data is always sent to and from the DRAM
`cache rather than directly from the host to the flash memory.
`Parity circuit 44 adds or checks parity bits for storage in the
`DRAM cache. FIFOs 41, 42 also buffer data from the host
`or DRAM cache before it is written to the flash memory over
`one of flash busses 10, 18.
`Flash bus 10 is controlled by flash-specific DMA control
`ler 12, while flash bus 18 is controlled by flash-specific
`DMA controller 16. Each DMA controller 12, 16 contains
`four state machines 40, one for each of four banks of
`flash-memory chips. Having four State machines allows four
`flash-memory chips to be accessed at once for each flash
`bus, for a total of eight flash accesses. The data can be
`interleaved among the four banks for each flash buffer chip,
`and also interleaved among chips connected to the two flash
`busses. Separate state machines 40 allow the eight flash
`accesses to be in different Stages of completion.
`Flash bus 10 is driven by bus control logic 46 in DMA
`controller 12, while flash bus 18 is driven by bus control
`logic 48 in DMA controller 16. Bus control logic 46, 48
`generates command and address bytes or enables reading
`and writing of FIFOs 41, 42 in response to states entered by
`State machines 40. Arbitration logic (not shown) determines
`
`KIOXIA Ex-1006, Page 12
`
`

`

`5,822,251
`
`15
`
`25
`
`35
`
`40
`
`7
`which state machine 40 currently controls each flash bus,
`Since only one transfer at a time can be performed on a flash
`bus.
`Multiplexers or muxes 83, 85 select address bytes and
`commands from bus control logic 46,48, or data from FIFOs
`41, 42. This allows both commands and data to be sent over
`the same flash bus.
`Error-correcting-code (ECC) generators 91, 93 and ECC
`checkers 95, 97 connect to flash busses 10, 18. Ablock code,
`Such as a Reed-Solomon code, is used to protect entire pages
`of data Stored in the flash-memory chips and to guard against
`more than just simple, Single-bit errors. When data is being
`transferred to the flash-memory chips, ECC generators 91,
`93 generate ECC, and when data is being transferred from
`the flash-memory chips, ECC checkers 95, 97 check the
`ECC and determine if there were any errors, either in the
`flash-memory chips themselves, or possibly in the interface
`to or from them. It is possible to design a combined ECC
`generator/checker unit So that one ECC unit can be used per
`flash bus.
`Flash Buffer Chip-FIG. 4
`FIG. 4 is a diagram of a flash buffer chip for an expand
`able flash-memory system. Flash buffer chips allow a rela
`tively large number of flash-memory chips to be added to the
`System without excessively loading the flash bus. Busy
`signals are combined by the flash buffer chips so that fewer
`busy signals need to be input to the DMA controllers.
`Flash buffer chip 14 connects to a flash bus to receive
`commands, addresses, and data from the flash-specific DMA
`controller. The 8-bit data/address/command portion of the
`flash bus is passed through to output logic 79 and then on to
`the flash-memory chips. The 2-bit Select and command
`portion of the flash bus are decoded by flash-bus interface
`70, which includes state machines to keep track of multi
`cycle command Sequences.
`The flash buffer chips also serve as protocol converters,
`taking the flash bus protocol and converting it to the protocol
`expected by the flash-memory chips. This could be as Simple
`as converting flash bus commands in to the appropriate
`Sequence of Signal transitions to the flash-memory chips, or
`could involve translation of commands or addresses, or even
`more complex Sequencing. In the preferred implementation,
`the commands on the flash bus are kept Similar to those
`expected by the flash-memory chips to minimize the cost of
`conversion and thus keep the flash buffer chips Simple.
`Requests on the flash bus occur in one of two modes:
`initialization mode or normal mode. Selection of the request
`mode is controlled by a bit in the control field sent by the
`DMA controller during a request cycle. Initialization mode
`allows for configuration of the flash buffer chips by writing
`directly to an internal state in the flash buffer chips, while
`normal mode allows for communication with the flash
`memory chips via the flash buffer chips.
`In order to uniquely address each flash buffer chip on a
`flash bus for initialization mode requests, a geographical ID
`is provided which is unique for each flash buffer chip. The
`fixed identifier can be generated by a daisy chain that adds
`one to the value received by each flash buffer chip, when the
`first chip's input is zero. Using this scheme, the flash buffer
`chips on a flash bus can be addressed for initialization mode
`requests using Sequential geographical IDS. Initialization
`mode requests are used to Set Starting address register 62 and
`ending address register 64 in each flash buffer chip So that
`all the flash-memory chips connected to each flash buffer
`chip on the flash bus can be uniquely addressed by normal
`mode requests.
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`Starting address register 62 and ending address register 64
`in a flash buffer chip control the range of addresses for which
`the flash buffer chip will respond to normal mode requests.
`Firmware first configures a first flash buffer chip, whose
`geographical ID is Zero, to respond to a wide range of
`addresses (setting the starting address register 62 to Zero and
`the ending address register 64 to its maximum value).
`Firmware uses normal-mode requests to access flash
`memory chips connected to the first flash buffer chip.
`Firmware can determine how many flash-memory chips are
`actually present. Firmware can then reset ending address
`register 64 on the first flash buffer chip to indicate the last
`flash-memory chip that is actually present. This process is
`then repeated by continuing to a Second flash buffer chip,
`whose geographical ID is one, and Setting its starting address
`register 62 to one more than the ending address register 64
`of the first flash buffer chip. When this process has com
`pleted after initializing all the flash buffer chips, each will
`respond to normal mode requests to a unique and non
`overlapping range of addresses corresponding to the flash
`memory chips actually connected to each flash buffer chip.
`In normal mode, a device address Sent on the flash bus is
`compared to the Starting address register 62 and the ending
`address register 64 in each flash buffer chip by comparators
`66, 68. A device hit is signaled when the device address sent
`on the flash bus falls within the range indicated by the
`Starting address register and the ending address register as
`determined by AND gate 71. When properly initialized, only
`a single flash buffer chip can generate a device hit in
`response to the device address Sent on the flash bus with a
`normal mode request. When enabled by a lock bit in the
`control field sent by the DMA con