(12) United States Patent
`Onufryk et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 8,601,346 B1
`Dec. 3, 2013
`
`USOO86O1346B1
`
`(54) SYSTEMAND METHOD FOR GENERATING
`PARTY DATAN ANONVOLATLE
`MEMORY CONTROLLER BY USINGA
`DISTRIBUTED PROCESSING TECHNIOUE
`
`(75) Inventors: Peter Z. Onufryk, Flanders, NJ (US);
`Inna Levit, Sunnyvale, CA (US)
`
`(73) Assignee: PMC-Sierra US, Inc., Sunnyvale, CA
`(US)
`
`- r
`c
`(*) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 309 days.
`(21) Appl. No.: 13/052,835
`1-1.
`(22) Filed:
`
`Mar 21, 2011
`Related U.S. Application Data
`(60) Provisional application No. 61/374,242, filed on Aug.
`16, 2010.
`
`(2006.01)
`
`(51) Int. Cl.
`GI IC 29/00
`(52) U.S. Cl.
`USPC - - - - - - - - - - - grgrrr. 714/764; 711/103
`(58) Field of Classification Search
`USPC . .
`. .
`. .
`. . . . .
`. . . . 711/103; 714f764
`See application file for complete search history.
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`
`5,875,343 A
`7,620,784 B2
`
`2f1999 Binford et al.
`11/2009 Panabaker
`
`300
`
`5/2010 Yoshida et al.
`7,708,195 B2
`2006:36 R 8.3. is Spa et al.
`al
`2008/0320214 A1* 12/2008 Ma et al. ....................... T11 103
`2009, OO773O2 A1* 3, 2009
`... 711,103
`2010.0185808 A1* 7, 2010 Yu et al. ........................ T11 103
`2010/0262979 A1 10, 2010 Borchers et al.
`2011/O161678 A1* 6, 2011 Niwa ............................ T13, 193
`
`
`
`OTHER PUBLICATIONS
`NVM Express, revision 1.0; Intel Corporation; pp. 103-106 and
`110-114; Jul 12, 2011.
`NVM Express, Revision 1.0; Intel Corporation; Mar. 1, 2011.
`
`* cited by examiner
`Primary Examiner — M. Mujtaba K Chaudry
`(74) Attorney, Agent, or Firm — Kenneth Glass; Stanley J.
`Pawlik; Glass & Associates
`(57)
`ABSTRACT
`A nonvolatile memory controller performs a data stripe
`operation on data blocks by processing a collection of com
`mands. The nonvolatile memory controller includes com
`mand processing units, each of which processes a command
`of the data stripe operation to store a data block into a non
`volatile memory device. A parity calculator in the nonvolatile
`memory controller receives the data blocks of the data stripe
`operation by receiving a sequence of data blocks. The parity
`calculator generates a parity block in a page frame as the
`parity calculator receives the sequence of the data blocks. A
`command processing unit in the nonvolatile memory control
`ler determines when the parity calculator has completed gen
`erating the parity block and writes the parity block to a non
`volatile memory device.
`18 Claims, 12 Drawing Sheets
`
`From Message
`405
`Network 200
`- Command Input->
`Queue
`
`410
`Command
`Distribution Unit
`
`420
`Parity
`Calculator
`
`430
`Context Modulc
`
`435.
`Contcxt
`
`400
`Command Processing
`Module
`
`--
`e
`AIS
`Command
`Processing
`Unit
`
`To
`Nonwolatile
`Memory
`Device 40
`
`X
`
`i.
`From Data
`3.
`Network 205
`-> Frame
`
`f
`To Data
`445
`Network 205
`-- Data Request k
`Routcr
`
`440
`Data Distribution
`Unit
`
`X
`
`To Correction
`w Module 305
`
`
`Ex.1001 / Page 1 of 25Ex.1001 / Page 1 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`
`Ex.1001 / Page 2 of 25Ex.1001 / Page 2 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`
`Ex.1001 / Page 3 of 25Ex.1001 / Page 3 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`US 8,601,346 B1
`
`
`
`0£
`
`
`Ex.1001 / Page 4 of 25Ex.1001 / Page 4 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 4 of 12
`
`US 8,601,346 B1
`
`
`
`
`
`
`
`BhBCI LUOJAI
`
`
`Ex.1001 / Page 5 of 25Ex.1001 / Page 5 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 5 of 12
`
`US 8,601,346 B1
`
`
`
`430
`Context Module
`
`435
`Context
`
`505
`Context State
`
`510
`Flow Identifier
`
`515
`Stripe Count
`
`520
`Context Module
`Controller
`
`FIG. 5
`
`To/From Command
`Input Qucue 405
`
`To/From Parity
`Calculator 430
`
`To/From Command
`Processing Units 415
`
`
`Ex.1001 / Page 6 of 25Ex.1001 / Page 6 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 6 of 12
`
`US 8,601,346 B1
`
`
`
`
`Ex.1001 / Page 7 of 25Ex.1001 / Page 7 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`
`Ex.1001 / Page 8 of 25Ex.1001 / Page 8 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`
`Ex.1001 / Page 9 of 25Ex.1001 / Page 9 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 9 of 12
`
`US 8,601,346 B1
`
`900
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Receive command at command
`distribution unit
`
`905
`
`Page
`available?
`
`N
`
`Data update
`command?
`
`Page frame
`allocated?
`
`Allocate page frame associated
`with context to data stripe
`opcration
`
`925
`
`Initialize stripe count in context
`
`930
`
`Set context to allocated State
`
`935
`
`Data
`update commands
`complete?
`
`N
`
`Distribute command to command
`processing unit
`
`955
`
`
`
`X
`Receive
`additional
`command?
`
`FIG. 9
`
`
`Ex.1001 / Page 10 of 25Ex.1001 / Page 10 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 10 of 12
`
`US 8,601,346 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`1000
`
`Receive command at command
`processing unit
`
`1005
`
`Data stripe
`command?
`
`N
`
`Y
`
`
`
`
`
`Process command
`
`1030
`
`Request block based on data stripe
`command
`
`Receive block in response to
`request for block
`
`Write block to nonvolatile
`memory device
`
`1035
`
`
`
`Receive
`additional
`command?
`
`FIG 10
`
`
`Ex.1001 / Page 11 of 25Ex.1001 / Page 11 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 11 of 12
`
`US 8,601,346 B1
`
`1105
`
`1110
`
`120
`
`1145
`Update parity block in page frame
`asSociated with context
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1100
`
`Receive data block at parity
`calculator
`
`Forward data block to processing
`unit
`
`1 115
`
`Data update
`command?
`
`Y
`
`Identify context associated with
`data update command
`
`Initialized State?
`
`1125
`
`Initialize parity block in page
`frame associated with context
`
`
`
`
`
`
`
`Set context to initialized state
`
`1140
`Update Stripe count of context
`
`1150
`
`
`
`Receive
`additional data
`block?
`
`FIG 11
`
`
`Ex.1001 / Page 12 of 25Ex.1001 / Page 12 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`U.S. Patent
`
`Dec. 3, 2013
`
`Sheet 12 of 12
`
`US 8,601,346 B1
`
`1200
`
`Receive parity Write command at
`parity calculator
`
`1205
`
`Identify context of parity write
`command
`
`1210
`
`Read parity block from page frame
`associated with context
`
`1215
`
`Provide parity block to command
`processing unit
`
`1220
`
`
`
`
`
`
`
`FIG. 12
`
`
`Ex.1001 / Page 13 of 25Ex.1001 / Page 13 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`US 8,601,346 B1
`
`1.
`SYSTEMAND METHOD FOR GENERATING
`PARTY DATAN ANONVOLATLE
`MEMORY CONTROLLER BY USINGA
`DISTRIBUTED PROCESSING TECHNIQUE
`
`CROSS-REFERENCE TO RELATED
`APPLICATION
`
`This application claims benefit of U.S. provisional patent
`application Ser. No. 61/374,242 filed Aug. 16, 2010 and
`entitled “Non Volatile Memory Host Controller Interface
`Device,” which is incorporated herein by reference in its
`entirety.
`
`BACKGROUND
`
`10
`
`15
`
`2
`parity calculator generates the parity block on the fly as the
`command processing units are receiving the data blocks for
`the data stripe operation.
`Because the nonvolatile memory controller performs a data
`stripe operation by using a plurality of command processing
`units, the nonvolatile memory controller performs a data
`stripe operation more quickly than other methods using only
`a single dedicated processor to perform a data Stripe opera
`tion. Moreover, because the parity calculator generates the
`parity block in the page frame as the parity calculator is
`receiving the data blocks for the data stripe operation, the
`parity calculator need not store all the data blocks of the data
`stripe at the same time when generating a parity block. As a
`result, the nonvolatile memory controller consumes less
`power and area than other nonvolatile memory controllers in
`which data blocks of a data stripe operation are stored in
`individual data buffers when computing a parity block.
`A nonvolatile memory controller, in accordance with one
`embodiment, performs a data stripe operation on block
`blocks. The nonvolatile memory controller includes com
`mand processing units and a parity calculator. The parity
`calculator is coupled to the command processing units. Each
`of the command processing units is configured to receive a
`command of a plurality of commands for performing the data
`stripe operation. The commands include data update com
`mands and a parity write command. Each command process
`ing unit receiving a data update command is configured to
`request a data block of the data blocks based on the data
`update command, receive the data block in response to the
`request, and write the data block to a nonvolatile memory
`device. The parity calculator is configured to receive the data
`blocks and generate a parity block based on the data blocks.
`The command processing unit receiving the parity write com
`mand is configured to write the parity block to a nonvolatile
`memory based on the parity write command.
`A nonvolatile memory controller, in accordance with one
`embodiment, includes a command distribution unit, com
`mand processing units, and a parity calculator. The command
`processing units are coupled to the command distribution unit
`and the parity calculator. The command distribution unit is
`configured to receive commands for performing a data stripe
`operation on data blocks. The commands include data update
`commands and a parity write command. The command dis
`tribution unit is further configured to distribute the data
`update commands among at least Some of the command pro
`cessing units. Each of the command processing units receiv
`ing a data update command is configured to request a data
`block based on the data update command, receive the data
`block in response to the request, and write the data block to a
`nonvolatile memory device. The parity calculator includes a
`context memory including a page frame. Moreover, the parity
`calculator is configured to receive the data blocks as a
`sequence of data blocks. Further, the parity calculator is con
`figured to generate a parity block by storing a first data block
`of the sequence of data blocks into the page frame and updat
`ing the data block stored in the page frame with each data
`block following the first data block in the sequence of data
`blocks. The command distribution unit is further configured
`to determine generation of the parity block is complete and
`distribute the parity write command to a command processing
`unit in response to determining the parity block is complete.
`The command processing unit receiving the parity write com
`mand is configured to write the parity block into the nonvola
`tile memory device.
`A method for generating parity data, in accordance with
`one embodiment, includes distributing commands for per
`forming a data stripe operation among a number of command
`
`Ex.1001 / Page 14 of 25Ex.1001 / Page 14 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`Fault-tolerant computing systems often employ data stor
`age techniques for recovery of data lost as a result of a disk
`drive failure. These data storage techniques often involve a
`redundant array of independent disks (RAID). In some of
`these data storage techniques, a disk drive controller performs
`a data stripe operation by dividing data into data blocks and
`generating parity databased on the data blocks. In this way,
`the disk drive controller generates redundancy databased on
`the data blocks. The disk drive controller distributes the data
`blocks and the parity data among an array of disk drives. In
`event of a hardware failure of a disk drive in the array, the disk
`drive controller reconstructs the data in the failed disk drive
`based on the data stored in the other disk drives of the array,
`and stores the reconstructed data into a replacement disk
`drive.
`Because generation of parity data in a RAID operation is
`often a computationally intensive task, some types of disk
`drive controllers include a dedicated processor for generating
`parity data. In these types of disk drive controllers, the dedi
`cated processor stores data blocks into individual data buffers
`of the disk drive controller, executes computing instructions
`to generate parity data, and stores the parity data into yet
`another data buffer. Although use of a dedicated processor for
`generating parity data generally improves throughput of a
`disk drive controller, generation of parity data is still a per
`formance bottleneck in many disk drive controllers. More
`over, the data buffers consume a considerable amount of area
`and power in integrated circuit implementations of the disk
`drive controller.
`
`25
`
`30
`
`35
`
`40
`
`45
`
`SUMMARY
`
`In various embodiments, a nonvolatile memory controller
`performs a data Stripe operation by processing a collection of
`commands. The collection of commands includes data update
`commands and a parity write command. The nonvolatile
`memory controller includes a number of command process
`ing units, each of which receives a command in the collection
`of commands. Each of the command processing units receiv
`ing a data update command requests a data block from a
`controller memory, receives the data block from the controller
`memory through a data path in response to the request, and
`writes the data block to a nonvolatile memory device.
`The parity calculator receives the data blocks as a sequence
`of data blocks through the same data path from which the
`command processing units receive the data blocks. The parity
`calculator generates a parity block by storing a first data block
`of the sequence of data blocks into a page frame and updating
`the data block stored in the page frame with each remaining
`data block in the sequence of data blocks. In this way, the
`
`50
`
`55
`
`60
`
`65
`
`

`

`US 8,601,346 B1
`
`5
`
`10
`
`3
`processing units in the nonvolatile memory controller. The
`commands include a number of data update commands and a
`parity write command. The method further includes generat
`ing data requests by the command processing units based on
`data update commands, and receiving data blocks at the com
`mand processing units and at a parity calculator of the non
`Volatile memory controller in response to the data requests.
`Additionally, the method includes writing the data blocks to
`nonvolatile memory devices by the command processing
`units. The method also includes generating a parity block in a
`page frame of a context memory based on the data blocks.
`Further, the method includes determining generation of the
`parity block is complete and writing the parity block to a
`nonvolatile memory device based on the parity write com
`mand after generation of the parity block is complete.
`15
`Because the method performs a data stripe operation by using
`a plurality of command processing units, the method per
`forms a data Stripe operation more quickly than other meth
`ods using only a single dedicated processor to generate parity
`data.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The accompanying drawings are included to provide a
`further understanding of the invention, and are incorporated
`in and constitute a part of this specification. The drawings
`illustrate embodiments of the invention, and together with the
`description, serve to explain the principles of the invention.
`FIG. 1 is a block diagram of a computing system, in accor
`dance with an embodiment of the present invention.
`FIG. 2 is a block diagram of a network module, in accor
`dance with an embodiment of the present invention.
`FIG. 3 is a block diagram of a control module, in accor
`dance with an embodiment of the present invention.
`FIG. 4 is a block diagram of a storage controller, in accor
`dance with an embodiment of the present invention.
`FIG. 5 is a block diagram of a context module, in accor
`dance with an embodiment of the present invention.
`FIG. 6 is a block diagram of a parity calculator, in accor
`dance with an embodiment of the present invention.
`FIG. 7 is a block diagram of a processing unit, in accor
`dance with an embodiment of the present invention.
`FIG. 8 is a block diagram of a nonvolatile memory device,
`in accordance with an embodiment of the present invention;
`FIG.9 is a flow chart of a portion of a method of generating
`a parity block, in accordance with an embodiment of the
`present invention.
`FIG. 10 is a flow chart of a portion of a method of gener
`ating a parity block, inaccordance with an embodiment of the
`present invention.
`FIG. 11 is a flow chart of a portion of a method of gener
`ating a parity block, inaccordance with an embodiment of the
`present invention.
`FIG. 12 is a flow chart of a portion of a method of gener
`ating a parity block, inaccordance with an embodiment of the
`present invention.
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`DETAILED DESCRIPTION
`
`In various embodiments, a nonvolatile memory controller
`includes command processing units and a parity calculator
`for performing a data stripe operation. Each of the command
`processing units independently requests a data block from a
`controller memory, receives the data block from the controller
`memory through a data path in response to the request, and
`writes the data block to a nonvolatile memory device. The
`parity calculator receives the data blocks as a sequence of data
`
`60
`
`65
`
`4
`blocks through the same data path from which the command
`processing units receive the data blocks. The parity calculator
`generates a parity block by storing a first data block of the
`sequence of data blocks into the page frame and updating the
`data block stored in the page frame with each remaining data
`block in the sequence of data blocks in response to receiving
`the data block. In this way, the parity calculator generates the
`parity block on the fly as the command processing units and
`the parity calculator are receiving the data blocks for the data
`stripe operation.
`FIG. 1 illustrates a computing system 100, in accordance
`with an embodiment of the present invention. The computing
`system 100 includes a host processing unit 110, a communi
`cation network 115, a nonvolatile memory controller 105, and
`a nonvolatile memory device 140. The communication net
`work 115 is coupled (e.g., connected) to the host processing
`unit 110 and the nonvolatile memory controller 105. Addi
`tionally, the nonvolatile memory controller 105 is coupled
`(e.g., connected) to the nonvolatile memory device 140. In
`various embodiments, the nonvolatile memory device 140 is
`a flash storage device and the nonvolatile memory controller
`105 is a flash controller.
`The communication network 115 facilitates communica
`tion between the host processing unit 110 and the nonvolatile
`memory controller 105. For example, the communication
`network 115 may be a packet communication network, Such
`as a Peripheral Component Interconnect Express (PCIe) net
`work. The nonvolatile memory controller 105 manages data
`stored in the nonvolatile memory device 140 and communi
`cates with the host processing unit 110 through the commu
`nication network 115 for transferring data between the host
`processing unit 110 and the nonvolatile memory device 140.
`In various embodiments, the nonvolatile memory controller
`105 processes nonvolatile memory commands generated by
`the host processing unit 110 for controlling operation of the
`nonvolatile memory controller 105. In some embodiments,
`the host processing unit 110 generates Non-Volatile Memory
`Express (NVMe) commands and the nonvolatile memory
`controller 105 processes the (NVMe) commands to manage
`operation of the nonvolatile memory controller 105.
`In various embodiments, the nonvolatile memory control
`ler 105 includes a host controller interface 120, a network
`module 125, a control module 130, a processor module 145,
`and a controller memory 160. The network module 125 is
`coupled (e.g., connected) to the host controller interface 120,
`the control module 130, the processor module 145, and the
`controller memory 160. Additionally, the host controller
`interface 120 is coupled (e.g., connected) to the communica
`tion network 115, and the control module 130 is coupled (e.g.,
`connected) to the nonvolatile memory device 140. Further
`more, the processor module 145 includes processors 150
`coupled (e.g., connected) to the network module 125. In these
`embodiments, each of the host controller interface 120, the
`control module 130, the processors 150, and the controller
`memory 160 is a functional unit of the nonvolatile memory
`controller 105.
`In various embodiments, each of the host controller inter
`face 120, the control module 130, the processor module 145,
`and the controller memory 160 is source node or a destination
`node of the nonvolatile memory controller 105. In this way,
`each functional unit of the nonvolatile memory controller 105
`may be a source node or a destination node. In some embodi
`ments, one or more of the host controller interface 120, the
`control module 130, the processor module 145, and the con
`troller memory 160 is both a source node and a destination
`node of the nonvolatile memory controller 105. In this way, a
`
`Ex.1001 / Page 15 of 25Ex.1001 / Page 15 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`
`
`US 8,601,346 Bl
`
`5
`
`6
`
`
`
`
`functional unit of the nonvolatile memory controller 105 may
`processing status). The network module 125 routes the
`
`
`
`
`be both a source node and a destination node.
`
`
`completion message packet to the processor
`150.
`
`
`communica­The host controller interface 120 facilitates
`
`
`
`
`In various embodiments, the processor module 145 pro­
`
`
`
`tion between the communication network 115 and the func­
`
`
`
`cesses the nonvolatile memory command in the request mes-
`
`
`
`tional units of the nonvolatile memory controller 105 through
`
`
`5 sage packet received from the host controller interface
`120 by
`
`
`the network module 125. The control module 130 manages
`
`
`
`
`generating a request message packet including a data transfer
`
`
`
`the data in the nonvolatile memory device 140. For example,
`
`
`command for transferring data between the host processing
`
`
`the nonvolatile control module 130 may read data from
`
`unit 110 and the controller memory 160, and generating
`
`memory device 140 and write data into the nonvolatile
`
`
`
`another request message packet including a data transfer
`10
`memory device 140.
`
`command for transferring data between the controller
`
`
`data being transferred The controller memory 160 stores
`
`memory 160 and the nonvolatile memory device 140. In this
`
`
`
`memory nonvolatile from the host processing unit 110 to the
`
`
`
`request message way, the processor module 145 generates
`
`
`device 140. Additionally, the controller memory 160 stores
`
`
`
`packets for transferring data in a piecemeal manner between
`
`
`data being transferred from the nonvolatile memory device
`
`
`memory15 the host processing unit 110 and the nonvolatile
`
`140 to the host processing unit 110. In this way, the controller
`
`
`
`
`the device 140 based on the request message packet including
`
`
`
`memory 160 is an intermediate storage location for tempo­
`
`nonvolatile memory command.
`
`
`
`
`rarily storing data being transferred between the host process­
`In some cases, each functional unit receiving a request
`
`
`
`
`
`ing unit 110 and the nonvolatile memory device 140. In vari­
`
`
`
`message packet from the processor module 145 generates one
`
`
`
`a ous embodiments, the controller memory 160 includes
`
`
`
`
`request message 20 or more request data packets based on the
`
`random access memory (RAM), such as a static random
`
`
`
`packet. Each of the request data packets is a request for
`access memory (SRAM) or a dynamic random access
`
`
`
`
`transferring data between functional units of the nonvolatile
`memory (DRAM).
`
`
`
`memory controller the network 105 through module 125 or
`
`
`
`In various embodiments, the host controller interface
`120
`
`
`
`transferring data between the host processing unit 110 and the
`
`
`
`
`retrieves a request packet including a nonvolatile memory
`
`
`
`the control mod­25 host controller interface 120. For example,
`
`
`the command from the host processing unit 110 through
`
`
`
`
`the and provide ule 130 may generate request data packets
`
`a request message communication network 115, generates
`
`
`
`
`
`request data packets to the network module 125 for routing to
`
`
`packet including the nonvolatile memory command based on
`
`
`the controller the controller memory 160. In this example,
`
`
`
`
`
`the request packet, and provides the request message packet
`
`
`to the network module
`
`
`
`
`data including memory 160 generates completion packets 125. In tum, the network module 125
`
`
`
`the comple­30 stored in the controller memory 160 and provides
`
`routes the request message packet to the processor module
`
`
`
`
`
`tion packets to the network module 125 for routing to the
`the nonvolatile145.The processor module 145 processes
`
`
`control module
`
`
`
`
`memory command in the request message packet, generates a
`130.
`In various embodiments, the nonvolatile memory control­
`
`
`
`
`
`
`completion message packet including a completion status
`
`
`
`circuit of an integrated ler 105 is implemented in an integrated
`
`
`
`
`(e.g., a completion entry) based on the request message
`
`
`
`
`
`circuit device which may include an integrated circuit pack­
`
`
`
`
`packet, and provides the completion message packet to the 35
`
`
`
`age containing the integrated circuit. In some embodiments,
`
`
`the network module 125. The network module 125 routes
`
`
`
`
`
`
`
`in completion message packet to the host controller interface the nonvolatile memory controller 105 is implemented
`
`
`
`more than one integrated circuit of an integrated circuit
`120.
`
`
`
`
`
`
`
`
`
`the containing multichip package which may include a device The host controller interface a request packet 120 generates
`
`
`
`
`
`including the completion status based on the completion mes­40 integrated circuits.
`FIG. 2 illustrates the network module 125, in accordance
`
`
`
`sage packet and transmits the request packet to the host pro­
`
`
`
`
`
`
`
`
`
`
`
`The network invention. of the present with an embodiment network the communication cessing unit 110 through 115. In
`
`
`
`
`
`status turn, the host processing unit 110 stores the completion module 125 includes a message network 200 and a data
`
`
`
`
`
`
`
`
`
`message network of the request message packet received from the host control­network 205. Each of the 200 and the data
`
`
`
`
`
`
`
`
`
`host controller to the ( e.g., connected) status to deter­45 network ler interface 120 and processes the completion 205 is coupled
`
`
`
`
`mine the status of processing the nonvolatile memory com­
`
`
`
`module interface 120, the control module 130, the processor
`
`
`mand (i.e., a processing status).
`
`
`(e.g.,145.Additionally, the data network 205 is coupled
`
`
`
`
`
`
`to the controller connected) In various embodiments, a processor module 145 pro­ memory 160.
`such as The message network 200 routes message packets,
`
`
`cesses the nonvolatile memory command in a request mes­
`
`
`
`
`
`
`
`
`
`
`sage packet by generating additional request message pack­50 request message packets and completion message packets,
`
`
`
`
`
`
`
`ets, each of which includes a command. The processor between functional units of the nonvolatile memory control­
`150
`
`
`
`
`
`
`
`
`
`
`the message network embodiments, provides the request message packets to the network module ler 105. In various 200
`
`
`
`
`
`units of the non­among functional message packets routes 125.In tum, the network module 125 routes each of the
`
`
`
`
`
`
`
`inter-processor request message packets received from the processor modulevolatile memory controller 105 by using an
`
`
`
`
`(IPC) protocol. 55 communication memory controller145 to a functional unit of the nonvolatile
`such as data The data network 205 routes data packets,
`
`
`
`
`
`way, thepacket. In this 105 identified in the request message
`
`
`
`
`
`
`
`
`
`
`
`func­packets, between and data completion request packets node and the functionalas a source processor 150 functions
`
`
`
`
`
`
`
`
`unit receiving the request message packet functions as a des­tional units of the nonvolatile memory controller 105. In
`tination node.
`
`
`
`
`pack-various embodiments, the data network 205 routes data
`
`
`
`
`
`
`
`The functional unit receiving a request message packet 60 ets among functional units of the nonvolatile memory con­
`
`
`
`
`
`
`
`communication (IPC) troller the network from the processor 150 through module 125 105 by using an inter-processor
`
`
`
`
`processes the command in the request message packet, gen­protocol.
`
`the control module erates a completion message packet including a completion FIG. 3 illustrates 130, in accordance
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`status based on the request packet, and provides the comple­with an embodiment of the present invention. The control
`
`
`
`
`
`
`
`
`controller a storage tion message packet to the network module 65 module 125. The comple­ 130 includes 300 and a correction
`
`
`
`
`
`
`
`
`
`(e.g., controller tion status of the completion message packet indicates a status module 305. The storage 300 is coupled
`
`
`
`
`
`of the command processed by the functional unit (i.e., a connected) to the network module 125, the nonvolatile
`
`Ex.1001 / Page 16 of 25Ex.1001 / Page 16 of 25
`
`Sandisk Technologies, Inc. et alSandisk Technologies, Inc. et al
`
`

`

`7
`memory device 140, and the correction module 305. The
`correction module 305 is also coupled (e.g., connected) to the
`network module 125.
`The storage controller 300 manages data in the nonvolatile
`memory device 140. For example, the storage controller 300
`may read data from the nonvolatile memory device 140 and
`write data into the nonvolatile memory device 140. The cor
`rection module 305 generates an error correction code (ECC)
`for data to be stored into the nonvolatile memory device 140
`and provides the error correction code to the storage control
`ler 300. In turn, the storage controller 300 writes the data and
`the error correction code associated with the data into the
`nonvolatile memory device 140. Additionally, the storage
`controller 300 reads data and an error correction code asso
`ciated with the data from the nonvolatile memory device 140
`and provides the data and the error correction code to the
`correction module 305. In turn, the correction module 305
`detects data bit errors in the data, if any, and corrects the data
`bit errors based on the error correction code associated with
`the data.
`FIG. 4 illustrates the storage controller 300, in accordance
`with an embodiment of the present invention. The storage
`controller 300 includes a command processing module 400, a
`command input queue 405, a command distribution unit 410.
`a parity calculator 420, a context module 430, a data distri
`bution unit 440, and a data request router 445. The command
`input queue 405 is coupled (e.g., connected) to the message
`network 200 and the command distribution unit 410. Addi
`tionally, the command distribution unit 410 is coupled (e.g.,
`connected) to the command processing unit 415 and the con
`text module 430. The parity calculator 420 is coupled to the
`data network 205, the context module 430, the data distribu
`tion unit 440, and the data request router 445. The data request
`router 445 is coupled (e.g., connected) to the data network
`205 and the command processing module 400. Additionally,
`the command processing module 400 is coupled (e.g., con
`nected) to the data distribution unit 440, the nonvolatile
`memory device 140, and the correction module 305.
`As illustrated in FIG. 4, the command processing module
`400 includes command processing units 415. The command
`40
`processing units 415 process commands for managing data in
`the nonvolatile memory device 140, as is described more fully
`herein. Although three command processing units 415 are
`illustrated in FIG. 4, the command processing module 400
`may have more or fewer than three command processing units
`415 in other embodiments.
`The parity calculator 420 includes page frames 425. More
`over, the parity calculator 420 generates parity blocks in the
`page frames 425, as is described more fully herein. The con
`text module 430 includes contexts 435 corresponding to the
`page frames 425 in the parity calculator 420. Moreover, each
`of