`
`(12) United States Patent
`Perego
`
`USOO6864.896B2
`(10) Patent No.:
`US 6,864,896 B2
`(45) Date of Patent:
`Mar. 8, 2005
`
`(54) SCALABLE UNIFIED MEMORY
`ARCHITECTURE
`
`(75) Inventor: Richard E. Perego, San Jose, CA (US)
`
`5,867,180 A * 2/1999 Katayama et al. .......... 34.5/542
`6,104,417 A * 8/2000 Nielsen et al. ........
`... 34.5/542
`6,208,273 B1 * 3/2001. Dye et al. ..................... 341/51
`6,690,726 B1 * 2/2004 Yavits et al............ 375/240.12
`
`y
`
`- - -
`
`9
`
`(73) Assignee: Rambus Inc., Los Altos, CA (US)
`(*) Notice:
`Subject to any disclaimer, the term of this
`past isits, 6. alusted under 35
`a --
`(b) by
`ayS.
`(21) Appl. No.: 09/858,836
`(22) Filed:
`May 15, 2001
`(65)
`Prior Publication Data
`US 2002/0171652 A1 Nov. 21, 2002
`(51) Int. Cl. .............................................. G06F 15/167
`(52)
`... 3451542; 34.5/543; 34.5/544
`(58) Field of Search ................................. 345/421, 542,
`34.5/519, 536, 522, 506, 505, 531, 532,
`520, 535,543, 544; 710/26; 712/201; 711/209,
`120, 170, 111, 105, 103; 341/51; 375/240.12
`References Cited
`
`(56)
`
`U.S. PATENT DOCUMENTS
`5,325,493 A * 6/1994. Herrell et al. .............. 712/201
`
`* cited by examiner
`
`Primary Examiner Matthew C. Bella
`Assistant Examiner Dalip K. Singh
`(74) Attorney, Agent, or Firm-Lee & Hayes, PLLC
`(57)
`ABSTRACT
`A memory architecture includes a memory controller
`coupled to multiple modules. Each module includes a com
`puting engine coupled to a shared memory. Each computing
`engine is capable of receiving instructions from the memory
`controller and processing the received instructions. The
`shared memory is configured to Store main memory data and
`graphical data. Certain computing engines are capable of
`processing graphical data. The memory controller may
`include a graphics controller that provides instructions to the
`computing engine. An interconnect on each module allows
`multiple modules to be coupled to the memory controller.
`
`34 Claims, 7 Drawing Sheets
`
`3OO v
`
`
`
`304
`
`RENDERING
`ENGINE
`312
`
`
`
`SHARED MAIN
`AND GRAPHICS
`MEMORY 314
`
`CPU/MEMORY
`302 - CONTROLLER SUBSYSTEM
`^ -
`
`CPU
`
`308
`
`310
`
`MEMORY
`CONTROLLER
`GRAPHICS
`CONTROLLER
`
`VIDEO NPUT
`
`DISPLAY
`NTERFACE
`
`306
`
`
`
`WO
`CONTROLLER
`
`Ex. 4, p. 1
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 2 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 1 of 7
`
`US 6,864,896 B2
`
`100 \
`
`108
`
`
`
`
`
`
`
`
`
`
`GRAPHICS
`MEMORY
`
`MEMORY
`CONTROLLER
`
`GRAPHICS
`CONTROLLER
`
`VIDEO
`INPUT
`
`116
`
`DISPLAY
`INTERFACE
`
`106
`
`
`
`I/O
`CONTROLLER
`
`
`
`22, t
`(Prior Art)
`
`Ex. 4, p. 2
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 3 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 2 of 7
`
`US 6,864,896 B2
`
`200 \
`
`204
`
`SHARED MAIN
`MEMORY AND
`GRAPHICS MEMORY
`
`CPU/MEMORY
`202 - CONTROLLER SUBSYSTEM
`
`
`
`MEMORY
`CONTROLLER/
`GRAPHICS
`CONTROLLER
`
`VIDEO INPUT
`
`DISPLAY
`INTERFACE
`
`
`
`
`
`
`
`
`
`
`
`CONTROLLER
`
`
`
`22, 2
`(Prior Art)
`
`Ex. 4, p. 3
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 4 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 3 of 7
`
`US 6,864,896 B2
`
`3OO \
`
`RENDERING
`ENGINE
`312
`
`SHARED MAIN
`AND GRAPHICS
`MEMORY 314
`
`304
`
`
`
`
`
`
`
`CPU/MEMORY
`302 - CONTROLLER SUBSYSTEM
`
`
`
`
`
`
`
`
`
`
`
`MEMORY
`CONTROLLER/
`GRAPHICS
`CONTROLLER
`
`VIDEO NPUT
`
`DISPLAY
`INTERFACE
`
`CONTROLLER
`
`
`
`Ex. 4, p. 4
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 5 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 4 of 7
`
`US 6,864,896 B2
`
`THE MEMORY CONTROLLER/GRAPHICS CONTROLLER
`RECEIVES A PROCESSING TASK
`
`404
`
`THE MEMORY CONTROLLER/GRAPHICS CONTROLLER
`PARTITIONS THE PROCESSING TASK iNTO
`MULTIPLE PORTIONS
`
`
`
`406
`
`THE MEMORY CONTROLLER/GRAPHICS CONTROLLER
`DISTRIBUTES EACH PORTION OF THE PROCESSING
`TASK TO ARENDERING ENGINE
`
`408
`
`NO
`
`ALL RENDERING ENGINES FINISHED
`
`YES
`
`
`
`410
`
`THE MEMORY CONTROLLER/GRAPHICS CONTROLLER
`REASSEMBLES THE FINISHED FORTIONS OF THE TASK
`
`Ex. 4, p. 5
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 6 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 5 of 7
`
`US 6,864,896 B2
`
`
`
`-
`
`HORIZONTAL
`PXEL
`BANDS -
`(ROWs)
`
`-
`
`y
`VERTICAL
`PXEL
`BANDS
`(COLUMNS)
`
`22, 5
`
`Ex. 4, p. 6
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 7 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 6 of 7
`
`US 6,864,896 B2
`
`
`
`MEMORY MODULE
`
`RENDERING
`ENGINE
`
`700
`
`
`
`MEMORY MODULE
`
`RENDERING
`ENGINE
`
`N--
`
`MEMORY DEVICES 704
`
`N --
`
`MEMORY DEVICES 704
`
`INTERCONNECT
`
`INTERCONNECT
`
`708
`
`Ex. 4, p. 7
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 8 of 13
`
`U.S. Patent
`
`Mar. 8, 2005
`
`Sheet 7 of 7
`
`US 6,864,896 B2
`
`800
`
`
`
`MEMORY MODULE
`
`RENDERING
`ENGINE
`
`MEMORY DEVICES 804
`
`806
`
`MEMORY DEVICES 812
`
`RENDERING
`ENGINE
`
`N--
`
`MEMORY DEVICES 804
`
`N--
`
`MEMORY DEVICES 812
`
`808
`
`INTERCONNECT
`
`Ex. 4, p. 8
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 9 of 13
`
`US 6,864,896 B2
`
`1
`SCALABLE UNIFIED MEMORY
`ARCHITECTURE
`
`2
`portion of shared memory 204 and for data stored in the
`graphics memory portion of the shared memory. The shared
`memory 204 may be partitioned statically or dynamically. A
`Static partition allocates a fixed portion of the shared
`memory 204 as “main memory' and the remaining portion
`is the "graphics memory.” A dynamic partition allows the
`allocation of shared memory 204 between main memory and
`graphics memory to change depending on the needs of the
`System. For example, if the graphics memory portion is full,
`and the graphics controller needs additional memory, the
`graphics memory portion may be expanded if a portion of
`the shared memory 204 is not currently in use or if the main
`memory allocation can be reduced.
`Regardless of the System architecture, graphics rendering
`performance is often constrained by the memory bandwidth
`available to the graphics subsystem. In the system of FIG. 1,
`graphics controller 110 interfaces to a dedicated graphics
`memory 112. Cost constraints for the graphics Subsystem
`generally dictate that a limited capacity of dedicated graph
`ics memory 112 must be used. This limited amount of
`memory, in turn, dictates a maximum number of memory
`devices that can be Supported. In Such a memory System, the
`maximum graphics memory bandwidth is the product of the
`number of memory devices and the bandwidth of each
`memory device. Device-level cost constraints and technol
`ogy limitations typically Set the maximum memory device
`bandwidth. Consequently, graphics memory bandwidth, and
`therefore graphics performance, are generally bound by the
`Small number of devices that can reasonably be Supported in
`this type of System configuration.
`Unified memory architectures such as that shown in FIG.
`2, help alleviate cost constraints as described above, and
`generally provide lower cost relative to Systems. Such as that
`shown in FIG. 1. However, memory bandwidth for the
`System of FIG. 2 is generally bound by cost constraints on
`the memory controller/graphics controller 210. Peak
`memory bandwidth for this system is the product of the
`number of conductors on the memory data interface and the
`communication bandwidth per conductor. The communica
`tion bandwidth per conductor is often limited by the choice
`of memory technology and the topology of the main
`memory interconnect. The number of conductors that can be
`used is generally bound by cost constraints on the memory
`controller/graphics controller package or System board
`design. However, the system of FIG. 2 allows theoretical
`aggregate bandwidth to and from the memory devices to
`Scale linearly with System memory capacity, which is typi
`cally much larger than the capacity of dedicated graphics
`memory. The problem is that this aggregate bandwidth
`cannot be exploited due to the limiting factors described
`above relating to bandwidth limitations at the memory
`controller/graphics controller.
`A System architecture which could offer the cost Savings
`advantages of a unified memory architecture, while provid
`ing Scalability options to higher levels of aggregate memory
`bandwidth (and therefore graphics performance) relative to
`Systems using dedicated graphics memory would be advan
`tageous.
`
`TECHNICAL FIELD
`The present invention relates to memory Systems and, in
`particular, to Scalable unified memory architecture Systems
`that Support parallel processing of data.
`BACKGROUND
`Various Systems are available for Storing data in memory
`devices and retrieving data from those memory devices.
`FIG. 1 illustrates a computer architecture 100 in which a
`discrete graphics controller Supports a local graphics
`memory. Computer architecture 100 includes a central pro
`cessing unit (CPU) 102 coupled to a memory controller 104.
`Memory controller 104 is coupled to a main memory 108, an
`I/O controller 106, and a graphics controller 110. The main
`memory is used to Store program instructions which are
`executed by the CPU and data which are referenced during
`the execution of these programs. The graphics controller 110
`is coupled to a discrete graphics memory 112. Graphics
`controller 110 receives video data through a video input 114
`and transmits Video data to other devices through a display
`25
`interface 116.
`The architecture of FIG. 1 includes two separate memo
`ries (main memory 108 and graphics memory 112), each
`controlled by a different controller (memory controller 104
`and graphics controller 110, respectively). Typically, graph
`ics memory 112 includes faster and more expensive memory
`devices, while main memory 108 has a larger Storage
`capacity, but uses slower, leSS expensive memory devices.
`Improvements in integrated circuit design and manufac
`turing technologies allow higher levels of integration,
`thereby allowing an increasing number of Subsystems to be
`integrated into a single device. This increased integration
`reduces the total number of components in a System, Such as
`a computer System. AS Subsystems with high memory per
`formance requirements (Such as graphics Subsystems) are
`combined with the traditional main memory controller, the
`resulting architecture may provide a Single high
`performance main memory interface.
`Another type of computer memory architecture is referred
`to as a unified memory architecture (UMA). In a UMA
`45
`System, the graphics memory is Statically or dynamically
`partitioned off from the main memory pool, thereby Saving
`the cost associated with dedicated graphics memory. UMA
`Systems often employ less total memory capacity than
`Systems using discrete graphics memory to achieve Similar
`levels of graphics performance. UMA Systems typically
`realize additional cost Savings due to the higher levels of
`integration between the memory controller and the graphics
`controller.
`FIG. 2 illustrates another prior art memory system 200
`that uses a unified memory architecture. A CPU/Memory
`Controller subsystem 202 includes a CPU 208 and a
`memory controller and a graphics controller combined into
`a single device 210. The Subsystem 202 represents an
`increased level of integration as compared to the architecture
`of FIG. 1. Subsystem 202 is coupled to a shared memory
`204, which is used as both the main memory and the
`graphics memory. Subsystem 202 is also coupled to an I/O
`controller 206, a video input 212, and a display interface
`214.
`The memory controller/graphics controller 210 controls
`all memory access, both for data Stored in the main memory
`
`35
`
`15
`
`40
`
`50
`
`55
`
`60
`
`65
`
`SUMMARY
`The systems and methods described herein achieve these
`goals by Supporting the capability of locating certain pro
`cessing functions on the memory modules, while providing
`the capability of partitioning tasks among multiple parallel
`functional units or modules.
`In one embodiment, an apparatus includes a memory
`controller coupled to one or more modules. Each installed
`
`Ex. 4, p. 9
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 10 of 13
`
`US 6,864,896 B2
`
`3
`module includes a computing engine coupled to a shared
`memory. The computing engine is configured to receive
`information from the memory controller. The shared
`memory includes Storage for instructions or data, which
`includes a portion of the main memory for a central pro
`cessing unit (CPU).
`In another embodiment, the computing engine is a graph
`ics rendering engine.
`In a particular implementation of the System, the shared
`memory is configured to Store main memory data and
`graphical data.
`Another embodiment provides that the computing engine
`is coupled between the memory controller and the shared
`memory.
`In a particular embodiment, the memory controller
`includes a graphics controller.
`In a described implementation, one or more modules are
`coupled to a memory controller. Each installed module
`includes a computing engine and a shared memory coupled
`to the computing engine.
`
`4
`module 304 includes a rendering engine 312 and a shared
`memory 314 (i.e., main memory and graphics memory). The
`main memory typically contains instructions and/or data
`used by the CPU. The graphics memory typically contains
`instructions and/or data used to process, render, or otherwise
`handle graphical images or graphical information.
`Rendering engine 312 may also be referred to as a
`“compute engine' or a “computing engine.” The shared
`memory 314 typically includes multiple memory devices
`coupled together to form a block of Storage Space. Each
`rendering engine 312 is capable of performing various
`memory acceSS and/or data processing functions. For the
`embodiment shown in FIG. 3, memory controller/graphics
`controller 310 is also coupled to an I/O controller 306 which
`controls the flow of data into and out of the system. An
`optional Video input port 316 provides data to memory
`controller/graphics controller 310 and a display interface
`318 provides data output to one or more devices (such as
`display devices or storage devices). For Systems which
`Support Video input or capture capability, a Video input port
`on the memory controller/graphics controller 310 is one way
`to handle the delivery of Video Source data. Another means
`of delivery of video input data to the system would include
`delivering the data from a peripheral module through the I/O
`controller 306 to device 310.
`In the example of FIG. 3, CPU/memory controller Sub
`system 302 is coupled to four distinct memory modules 304.
`Each memory module includes a rendering engine and a
`shared memory. Each rendering engine 312 is capable of
`performing various data processing functions. Thus, the four
`rendering engines are capable of performing four different
`processing functions Simultaneously (i.e., parallel
`processing). Further, each rendering engine 312 is capable of
`communicating with other rendering engines on other
`memory modules 304.
`The memory controller/graphics controller 310 distributes
`particular processing tasks (Such as graphical processing
`tasks) to the different rendering engines, and performs
`certain processing tasks itself. These tasks may include data
`to be processed and/or instructions to be processed.
`Although four memory modules 304 are shown in FIG. 3,
`alternate System may contain any number of memory mod
`ules coupled to a common memory controller/graphics con
`troller 310. This ability to add and remove memory modules
`304 provides an upgradeable and Scalable memory and
`computing architecture.
`The architecture of FIG. 3 allows the memory controller/
`graphics controller 310 to issue high level primitive com
`mands to the various rendering engines 312, thereby reduc
`ing the volume or bandwidth of data that must be
`communicated between the controller 310 and the memory
`modules 304. Thus, the partitioning of memory among
`multiple memory modules 304 improves graphical data
`throughput relative to Systems in which a Single graphics
`controller performs all processing tasks and reduces band
`width contention with the CPU. This bandwidth reduction
`occurs because the primitive commands typically contain
`Significantly leSS data than the amount of data referenced
`when rendering the primitive. Additionally, the System par
`titioning described allows aggregate bandwidth between the
`rendering engines and the memory devices to be much
`higher than the bandwidth between the controller and
`memory modules. Thus, effective system bandwidth is
`increased for processing graphics taskS.
`FIG. 4 is a flow diagram illustrating a procedure for
`partitioning a processing task into multiple portions and
`
`15
`
`25
`
`35
`
`40
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 illustrates a prior art computer architecture in
`which a discrete graphics controller Supports a local graph
`ics memory.
`FIG. 2 illustrates another prior art memory system that
`uses a unified memory architecture.
`FIG. 3 illustrates an embodiment of a Scalable unified
`memory architecture that Supports parallel processing of
`graphical data.
`FIG. 4 is a flow diagram illustrating a procedure for
`partitioning a processing task into multiple portions and
`distributing the multiple portions to different rendering
`engines.
`FIG. 5 illustrates a graphical rendering Surface divided
`into Sixteen different Sections.
`FIGS. 6 and 7 illustrate different memory modules con
`taining a rendering engine and multiple memory devices.
`FIG. 8 illustrates a memory module containing two dif
`ferent rendering engines and multiple memory devices asso
`ciated with reach rendering engine.
`DETAILED DESCRIPTION
`The architecture described herein provides one or more
`discrete memory modules coupled to a common memory
`controller. Each memory module includes a computing
`engine and a shared memory. Thus, the data processing tasks
`can be distributed among the various computing engines and
`the memory controller to allow parallel processing of dif
`ferent portions of the various processing tasks.
`Various examples are provided herein for purposes of
`explanation. These examples include the processing of
`graphical data by one or more rendering engines on one or
`more memory modules. It will be understood that the
`Systems described herein are not limited to processing
`graphical data. Instead, the Systems described herein may be
`applied to proceSS any type of data from any data Source.
`FIG. 3 illustrates an embodiment of a Scalable unified
`memory architecture 300 that Supports parallel processing of
`graphical data and/or graphical instructions. ACPU/memory
`controller subsystem 302 includes a CPU 308 coupled to a
`memory controller/graphics controller 310. One or more
`memory modules 304 are coupled to memory controller/
`graphics controller 310 in subsystem 302. Each memory
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Ex. 4, p. 10
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 11 of 13
`
`US 6,864,896 B2
`
`S
`distributing the multiple portions to different rendering
`engines. Initially, the memory controller/graphics controller
`receives a processing task (block 402). The memory
`controller/graphics controller partitions the processing task
`into multiple portions (block 404) and distributes each
`portion of the processing task to a rendering engine on a
`memory module (block 406).
`When all rendering engines have finished processing their
`portion of the processing task, the memory controller/
`graphics controller reassembles the finished portions of the
`task (block 410). In certain situations, the memory
`controller/graphics controller may perform certain portions
`of the processing task itself rather than distributing them to
`a rendering engine. In other situations, the processing task
`may be partitioned Such that one or more rendering engines
`perform multiple portions of the processing task. In another
`embodiment, the processing task may be partitioned Such
`that one or more rendering engines are idle (i.e., they do not
`perform any portion of the processing task).
`For typical graphics applications, primitive data (e.g.,
`triangles or polygons) are Sorted by the graphics controller
`or CPU according to the Spatial region of the rendering
`Surface (e.g., the X and y coordinates) covered by that
`primitive. The rendering Surface is generally divided into
`multiple rectangular regions of pixels (or picture elements),
`referred to as “tiles' or “chunks.” Since the tiles generally do
`not overlap spatially, the rendering of the tiles can be
`partitioned among multiple rendering engines.
`An example of the procedure described with respect to
`FIG. 4 will be described with reference to FIG. 5, which
`illustrates a graphical rendering Surface divided into Sixteen
`different sections or “tiles” (four rows and four columns).
`The graphical rendering Surface may be stored in, for
`example, an image buffer or displayed on a display device.
`If a particular image processing task is to be performed on
`the graphical rendering Surface shown in FIG. 5, the
`memory controller/graphics controller may divide the pro
`cessing tasks into different portions based on the Set of tiles
`intersected by the primitive elements of the task. For
`example, four of the Sixteen tiles of the Surface are assigned
`to a first rendering engine (labeled “RE0”) on a first memory
`module. Another four tiles of the Surface are assigned to a
`different rendering engine (labeled “RE1), and so on. This
`arrangement allows the four different rendering engines
`(RE0, RE1, RE2, and RE3) to process different regions of
`the Surface Simultaneously. This parallel processing Signifi
`cantly reduces the processing burden on the memory
`controller/graphics controller. In this example, a significant
`portion of the Set of graphics rendering tasks is performed by
`the four rendering engines, which allows the memory
`controller/graphics controller to perform other tasks while
`these rendering tasks are being processed.
`In an alternate embodiment, each of the rendering engines
`is assigned to process a particular set of rows or columns of
`the rendering Surface, where these rows or columns repre
`sent a band of any size of pixels (at least one pixel wide). AS
`shown in FIG. 5, the rendering surface is divided into two or
`more horizontal pixel bands (also referred to as pixel rows)
`and divided into two or more vertical pixel bands (also
`referred to as pixel columns). The specific example of FIG.
`5 shows a rendering surface divided into four horizontal
`pixel bands and four vertical pixel bands. However, in
`alternate embodiments the rendering Surface may be divided
`into any number of horizontal pixel bands and any number
`of Vertical pixel bands. For example, the rendering Surface
`may be divided into four Sections or "quadrants” (i.e., two
`columns and two rows), in which each of the rendering
`engines is assigned to a particular quadrant of the rendering
`Surface.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`The example discussed above with respect to FIG. 5 is
`related to graphics or image processing. However, the Sys
`tem described herein may be applied to any type of pro
`cessing in which one or more processing functions can be
`divided into multiple tasks that can be performed in parallel
`by multiple computing engines. Other examples include
`floating point numerical processing, digital Signal
`processing, block transferS of data Stored in memory, com
`pression of data, decompression of data, and cache control.
`For example, a task Such as decompression of a compressed
`MPEG (Moving Picture Experts Group) video stream can
`begin by partitioning the Video Stream into multiple regions
`or “blocks. Each block is decompressed individually and
`the results of all block decompressions are combined to form
`a complete Video image. In this example, each block is
`asSociated with a particular rendering engine.
`FIGS. 6 and 7 illustrate different memory modules con
`taining a rendering engine and multiple memory devices.
`The multiple memory devices represent the shared memory
`314 shown in FIG. 3. Although the memory modules shown
`in FIGS. 6 and 7 each contain eight memory devices,
`alternate memory modules may contain any number of
`memory devices coupled to a rendering engine.
`Referring to FIG. 6, a memory module 600 includes eight
`memory devices 604 coupled to a rendering engine 602. The
`rendering engine 602 is also coupled to a module intercon
`nect 606 which couples the memory module 600 to an
`asSociated interconnect on another module, motherboard,
`device, or other system. A memory interconnect 608 couples
`the memory devices 604 to the rendering engine 602. The
`memory interconnect 608 may allow parallel access to
`multiple memory devices 604, and may use either a multi
`drop topology, point-to-point topology, or any combination
`of these topologies.
`Referring to FIG. 7, a memory module 700 includes
`multiple memory devices 704 coupled to a rendering engine
`702. The rendering engine 702 is also coupled to a pair of
`module interconnects 706 and 708, which couple the
`memory module 700 to another module, motherboard,
`device, or System. A memory interconnect 710 couples the
`memory devices 704 to the rendering engine 702 and may
`allow parallel access to multiple memory devices 704. In an
`alternate embodiment, module interconnects 706 and 708
`are connected to form a contiguous interconnect through the
`module 700 connecting the rendering engine 702 to other
`devices in the System.
`AS discussed above, each memory module has its own
`memory interconnect for communicating data between the
`memory module's rendering engine and the various memory
`devices. The memory interconnect on each memory module
`is relatively short and provides significant bandwidth for
`communicating with the multiple memory devices on the
`module. Further, the Simultaneous processing and commu
`nication of data on each memory module reduces the Volume
`or bandwidth of data communicated between the memory
`modules and the memory controller/graphics controller. The
`memory interconnects discussed herein may be a bus, a
`Series of point-to-point links, or any combination of these
`types of interconnects.
`FIG. 8 illustrates a memory module 800 containing two
`different rendering engines 802 and 810, and multiple
`memory devices 804 and 812 associated with reach render
`ing engine. In this embodiment, two different memory
`interconnects 806 and 814 couple the memory devices to the
`rendering engines. The two rendering engines 802 and 810
`are coupled to a common module interconnect 808. Thus,
`
`Ex. 4, p. 11
`
`
`
`Case 6:22-cv-00466-ADA-DTG Document 50-4 Filed 12/19/22 Page 12 of 13
`
`US 6,864,896 B2
`
`15
`
`8
`4. An apparatus as recited in claim 1 wherein the shared
`memory includes at least one memory device.
`5. An apparatus as recited in claim 1 wherein the shared
`memory is configured to Store main memory instructions
`and data in addition to graphical data.
`6. An apparatus as recited in claim 1 wherein the com
`puting engine is coupled between the memory controller and
`the shared memory.
`7. An apparatus as recited in claim 1 wherein the memory
`controller is configured to receive Video data.
`8. An apparatus as recited in claim 1 wherein the memory
`controller includes a graphics controller.
`9. An apparatus as recited in claim 1 wherein each module
`includes a plurality of computing engines.
`10. An apparatus as recited in claim 9 wherein computing
`engines on the same module can communicate with one
`another.
`11. An apparatus as recited in claim 1 wherein computing
`engines on different modules can communicate with one
`another.
`12. An apparatus as recited in claim 1 wherein the
`information received by the computing engine from the
`memory controller includes instructions to be processed by
`the computing engine.
`13. An apparatus as recited in claim 1 wherein the
`information received by the computing engine from the
`memory controller includes data to be processed by the
`computing engine, the data associated with at least one
`Section in a graphical rendering Surface.
`14. An apparatus comprising:
`a memory controller configured to partition processing
`tasks among a plurality of rendering engines,
`a plurality of memory modules coupled to the memory
`controller, each module including:
`at least one rendering engine configured to receive data
`to be processed from the memory controller; and
`a shared memory coupled to the rendering engine,
`wherein the Shared memory includes Storage for a
`portion of the main memory for a central processing
`unit (CPU) coupled to the memory controller.
`15. An apparatus as recited in claim 14 wherein each
`shared memory includes a plurality of memory devices.
`16. An apparatus as recited in claim 14 wherein the
`memory controller is configured to receive video data.
`17. An apparatus as recited in claim 14 wherein each
`partitioned processing task is associated with at least one
`Section of a graphical rendering Surface.
`18. An apparatus as recited in claim 14 further including
`an interconnect configured to couple the plurality of mod
`ules to the memory controller.
`19. An apparatus as recited in claim 14 wherein each
`module includes a plurality of rendering engines.
`20. An apparatus as recited in claim 14 wherein rendering
`engines on different modules can communicate with one
`another.
`21. A module comprising:
`a rendering engine configured to receive information from
`a memory controller, the information associated with at
`least one processing task partitioned by the memory
`controller; and
`a shared memory coupled to the rendering engine,
`wherein the Shared memory is configured to Store main
`memory data as well as graphical data, wherein the
`module is configured to be coupled to or decoupled
`from the memory controller.
`22. A module as recited in claim 21 wherein the rendering
`engine and the shared memory are mounted on a common
`Substrate.
`
`25
`
`7
`rendering engines 802 and 810 can process data and/or tasks
`Simultaneously. In another embodiment, each rendering
`engine 802 and 810 is coupled to an independent module
`interconnect (not shown).
`A particular embodiment of the memory modules shown
`in FIGS. 6, 7, and 8 use multiple “Direct Rambus Channels'
`developed by Rambus Inc. of Mountain View, Calif. The
`Direct Rambus Channels connect the multiple memory
`devices to the associated rendering engine. In this
`embodiment, the memory devices are Rambus DRAMs (or
`RDRAMs(R). However, other memory types and topologies
`may also be used within the scope of this invention. For
`example, memory types Such as Synchronous DRAMs
`(SDRAMs), Double-Data-Rate (DDR) SDRAMs, Fast
`Cycle (FC) SDRAMs, or derivatives of these memory types
`may also be used. These devices may be arranged in a Single
`rank of devices or multiple ranks, where a rank is defined as
`an group of devices which respond to a particular class of
`commands including reads and writes.
`In a particular embodiment of the architecture shown in
`FIG. 3, processing functions that are associated with typical
`graphics controller interface functions Such as Video input
`processing and display output processing are handled by the
`memory controller/graphics controller 310. Processing func
`tions that are associated with high-bandwidth rendering or
`image buffer creation are handled by the rendering engines
`304 on the memory modules.
`Although particular examples are discussed herein as
`having one or two rendering engines, alternate implemen
`tations may include any number of rendering engines or
`computing engines on a particular module. Further, a par
`ticular module may contain any number of memory devices
`coupled to any number of rendering engines and/or com
`puting engines.
`35
`Thus, a System has been described that provides multiple
`discrete memory modules coupled to a common memory
`controller. Each memory module includes a computing
`engine and a shared memory. A data processing task from
`the memory controller can be partitioned among the differ
`ent computing engines to allow parallel processing of the
`various portions of the processing task.
`Although the description above uses language that is
`Specific to Structural features and/or methodological acts, it
`is to be understood that the invention defined in the
`appended claims is not limited to the Specific features or acts
`described. Rather, the Specific features and acts are disclosed
`as exemplary forms of implementing the invention.
`What is claimed is:
`1. An apparatus comprising: