EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Design How-To
`RDIMMs maximize server performance, reliability,
`and scalability
`Tomek Jasionowski, Integrated Device
`Technology
`3/26/2012 06:09 PM EDT
`Post a comment
`
`Some of the most critical factors to consider when choosing enterprise hardware are
`performance, reliability, and scalability. Dynamic random access memory (DRAM) modules
`affect all of these factors. IT managers have two choices of DRAM modules in platforms
`supporting double data rate (DDR3) protocols. These choices are error-correcting code
`(ECC) unregistered dual in-line memory modules (UDIMMs) and registered dual in-line
`memory modules (RDIMMs). In most applications, RDIMMs enable higher performance,
`allow better scalability, and deliver a more comprehensive set of reliability, availability, and
`serviceability (RAS) features than ECC UDIMMs. By using RDIMMs, companies can also
`reduce the total number of servers and related costs of ownership. RDIMMs are the
`preferred choice for most corporate application environments.
`
`RDIMM and DDR3 registers
`ECC UDIMMs are limited to applications needing up to 48-GB memories [1]. If the estimated
`peak DRAM usage over time exceeds 48 GB, RDIMMs are required to avoid memory
`performance bottlenecks. With RDIMMs, systems are scalable to 192-GB capacities [1].
`These high capacities are possible thanks to DDR3 registers embedded in RDIMMs. The
`registers are specialized chips that buffer clock, command, and address signals coming from
`the memory controller (see figure 1). The registers allow integration of more DRAM chips in
`each RDIMM, and more RDIMMs per memory channel, which yields higher performance and
`scalability in end-applications.
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 1 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 1
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Figure 1: The DDR3 register buffers clock, command, and address signals coming from the memory
`controller.
`
`Table 1 describes maximum specifications for RDIMMs and ECC UDIMMs. On the
`application level, the use of RDIMMs with DDR3 registers translates into scalable systems
`with predictable performance over time.
`
`
`Table 1: Maximum specifications for RDIMMs and ECC UDIMMs in servers using the Xeon processor
`5500 series
`
`To better understand how DDR3 registers enable higher performance and quadruple memory
`capacity when compared to ECC UDIMMs, it is necessary to examine how memory modules
`impact line loading. Each memory module, whether RDIMM or ECC UDIMM, communicates
`with a memory controller via data (DQ), data strobe (DQS), address, command, clock, and
`chip select signals. Using a real module design example, a 16-GB RDIMM containing 36 4-
`Gb DRAM chips and including four DRAMs for error correction is organized in four ranks of
`nine DRAM chips each. A memory rank is a group communicating over the 72-bit data bus of
`a memory module, independently selectable via the chip select (CS) signal.
`
`Reviewing the block diagram of the example above (see figure 2), it’s notable that the
`memory controller has one load at the DDR3 register on each clock, command, address, and
`CS line, and four loads on the DQ and DQS lines. Without the DDR3 register, the controller
`would see nine loads on the CS lines. Furthermore, clock, command, and address loads
`increase drastically to 36 due to the direct connection to DRAM chips. These higher loadings
`would cause the memory controller to fail to communicate at a high speed. For this reason,
`ECC UDIMM modules are typically limited to a DRAM capacity that is a factor of four times
`smaller than for RDIMM (see table 2).
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 2 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 2
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Figure 2: 16-GB RDIMM architecture with DDR3 register
`
`Table 2: Loading of memory controller lines with and without DDR3 register
`
`Critical DRAM capacity for enterprise systems
`Increasing DRAM capacity in memory-hungry enterprise platforms with RDIMM modules is
`often the quickest, most efficient and least expensive way to boost server performance. If a
`server does not have enough installed memory to sustain the application, its processors
`need to compensate by resorting to orders-of-magnitude-slower hard drive storage or solid
`state drive storage (see figure 3). This approach significantly extends the computation time,
`even if only a small percentage of memory transactions must be diverted to the storage
`drive.
`
`Consider a system with the following parameters:
`
`SSD storage drive average transfer rate 100 Mbytes/s
`DDR3 memory, 1333 Mbps, 64-bit data bus with transfer rate of 10 GB/s (100 times
`faster than drive transfer)
`DDR3 memory installed on server: 48 GB, 12 UDIMM modules with 4 GB each
`DDR3 memory required by the application: 64 GB
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 3 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 3
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`The result is that the system has to use a 16-GB storage drive memory to meet application
`demand for 64 GB. This virtual memory of 16 GB slows down the whole system. Because
`the storage drive speed is 100 times slower than DDR3 memory transfer, 25% of data
`transfer (16 GB out of 64 GB) via storage drive takes 100 times as long as the same 16 GB
`data transfer via DDR3 memory. Overall, 64-GB memory system performance is slowed
`down 25 times (i.e., 100 times 16 GB/64 GB). Hence, diverting 25% of memory transactions
`to drive storage will introduce up to 25 times longer delays in the data transfer.
`
`By upgrading the system to 64 GB memory with eight 8-GB RDIMMs and removing the
`memory limitations of ECC UDIMM, the system does not need to reroute DRAM data into the
`storage drive (see figure 4). This improves system performance by up to 25 times.
`Furthermore, optimized DRAM capacity helps save power, since storage drive transfers,
`besides being much slower, require much higher wattage per each gigabyte of data transfer.
`
`Figure 3: CPU to memory and storage drive transactions flow diagram
`
`Improving reliability and system uptime with RDIMMs
`In the enterprise equipment space, there is a strong focus on memory RAS requirements,
`which can drastically reduce system downtime and repair costs. Selecting the right memory
`module can make a significant difference on available RAS level. ECC UDIMMs provide a
`limited reliability and are known to cause data corruptions and system crashes due to single
`bit errors and single DRAM failures. RDIMM modules offer a comprehensive RAS solution
`including parity and availability of extended ECC, which minimize these issues. Table 3
`shows a comparison of RAS features in ECC UDIMMs and RDIMMs.
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 4 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 4
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Table 3: Comparison of RAS features in ECC UDIMMs and RDIMMs
`
`Single-bit errors are a key failure type in DRAM communications. A two-and-a-half-year study
`of DIMM modules on tens of thousands of Google servers found DIMM error rates to be
`hundreds to thousands of times higher than thought—a mean of 3,751 single bit errors per
`DIMM per year [3].
`
`Single-bit errors can occur on 64 data and 26 address and command lines interconnecting
`DRAMs and the memory controller. ECC UDIMMs can only detect and correct single-bit
`errors on 64 data lines via ECC feature. If a single-bit error occurs on any of the 26
`command or address lines, ECC UDIMMs will not detect nor report these errors. This error
`detection gap in almost one-third of DRAM interconnects can produce multiple corrupted
`memory operations per year, causing severe corporate data losses, service interruptions,
`server crashes and repair costs.
`
`RDIMMs provide protection against single-bit errors on data, address, and command lines.
`Indeed, RDIMMs use an ECC feature for correcting data errors and a parity feature for
`detecting single errors on address and command lines. If an address or command signal has
`an issue, the RDIMM sends a parity error signal back to the memory controller. The controller
`can then log the event and initiate a corrective sequence such as resending the last
`command.
`
`Another advantage of RDIMMs is that they can support extended ECC, also referred to as
`ChipKill or Chipspare [4]. Extended ECC keeps system operation at full speed even if there
`is a single DRAM chip failure or multi-bit errors from any portion of a single memory chip.
`Together, parity checking and extended ECC minimize system downtime, reduce service
`times and make equipment using RDIMM modules much more reliable than those using ECC
`UDIMMs.
`Memory channel utilization with RDIMMs
`Server manufacturers offering DDR3 memory solutions provide architectures allowing up to
`three DIMMs per channel and up to three channels per CPU core. While RDIMMs permit a
`full usage of all DIMM sockets, ECC UDIMMs are limited to one or two DIMMs per channel.
`The remaining sockets cannot be used and have to be left empty (see figure 4).
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 5 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 5
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Figure 4: Memory channel usage with RDIMMs and ECC UDIMMs
`
`The JEDEC Solid State Technology Association defines RDIMM-related standards. Several
`companies get involved in this process. For example, OEM companies making corporate-
`class systems and microprocessor manufacturers contribute to the standardization of
`RDIMM modules, DRAM modules, and DDR3 registers. In addition, DRAM chip
`manufacturers and DDR3 register manufacturers participate in RDIMM definition phases.
`Figure 5 shows the full ecosystem for defining RDIMM-related specifications. Through
`participation, these enterprises influence the overall quality and industry-wide compatibility of
`the final specification.
`
`Figure 5: DDR3 register specification definers
`
`Key application areas for RDIMMs
`Performance boosts derived from RDIMMs are most visible in memory-hungry applications.
`The first group of these applications falls under high-performance computing environments.
`This includes 3-D imaging, video, and CAD simulations running on professional-grade
`servers used by engineers, architects, designers, and video editors. RDIMMs are also
`indispensable for corporate database programs installed on servers. The most popular
`database programs are enterprise resource planning (ERP) and customer relationship
`management (CRM) software packets, which are used extensively by financial, logistics,
`marketing, and sales teams. Finally, virtualized environments and cloud computing
`applications in data centers gain substantially from DRAM capacity additions. In fact,
`sufficient DRAM capacity can directly translate into reduction of server units in these
`environments.
`
`As an example, if each virtual server requires one CPU core and 48 GB of DRAM, a four-
`core server could accommodate four virtual machines as long as the server includes 192 GB
`of DRAM. ECC UDIMMs enable capacities of up to 48 GB per server using the Xeon
`processor 5500 series, which means a system would require four servers to reach 192 GB
`[1]. By using RDIMMs, IT managers can achieve 192 GB in a single server, realizing
`substantial savings (see figure 6). Added benefits of this approach are power savings and
`space savings drawn from using just a single server unit. Overall, administrators can reduce
`total cost of ownership by using RDIMMs in place of ECC UDIMMs.
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 6 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 6
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`Figure 6: RDIMM technology can reduce hardware cost/footprint by a factor of four in virtualized
`computing platforms compared to ECC-UDIMM-based versions.
`
`RDIMMs also present excellent data bandwidth performance in applications not constrained
`by memory capacity. Based on an Intel study, ECC UDIMMs and RDIMMs produce very
`similar memory bandwidth in a one-DIMM-per-channel configuration using a 6-GB memory
`system (assuming 1-GB modules, two CPUs and three channels per CPU). RDIMMs
`outperform ECC UDIMMs by about 9% in a more typical configuration of two-DIMMs-per-
`channel using a 12-GB or 24-GB memory system (assuming 1-GB or 2-GB modules, two
`CPUs and three channels per CPU). In summary, RDIMMs are appropriate for very low
`memory usage applications and become a higher-performance solution in servers operating
`at capacities as low as 12 GB.
`
`RDIMMs provide several key advantages to enterprise systems. They maximize server
`performance in virtualized, high-performance, and cloud computing applications by delivering
`four times more memory capacity than ECC UDIMMs. RDIMMs also improve performance in
`applications requiring as little as 12 GB of memory in configurations of two or more DIMMs
`per channel. Additionally, these modules deliver an extensive suite of RAS features that
`increases reliability and minimizes downtimes, service times, and corporate data losses, all
`of which reduces the total cost of ownership of DIMMs. Maximized performance and a
`comprehensive set of RAS features make RDIMMs with DDR3 registers a preferred server
`solution for most enterprise applications.
`
`References
`[1] DRAM limits are based on 2Gbit DRAM chips in servers using a popular Intel Xeon
`processor 5500 series. Refer to http://www.intel.com/cd/channel/reseller/asmo-
`na/eng/products/server/410406.htm and http://www.intel.com/cd/channel/reseller/asmo-
`na/eng/products/server/410405.htm. These maximum server memory limits are expected to
`increase to 128 GB for ECC UDIMMs and to 512 GB for RDIMMs on new Intel Romley
`platforms using 4Gbit DRAM chips. Maximum memory limits are four times higher with
`RDIMMs on both servers using the Xeon processor 5500 series or servers based on the
`Romley platform.
`
`[2] Based on a two-core server using the popular Intel Xeon processor 5500 series with six
`memory channels, each populated with two quad-rank 16-GB RDIMMs using 2Gb DRAMs.
`
`[3] “DRAM Errors in the Wild: A Large-Scale Field Study” by Bianca Schroeder, Eduardo
`Pinheiro, Wolf-Dietrich Weber.
`
`[4] Extended ECC, Chipkill or Chipspare is offered by some RDIMM manufacturers. These
`features require motherboards that support them in servers. Please check with your vendors
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 7 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 7
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`

`

`EE Times - RDIMMs maximize server performance, reliability, and scalability
`
`02/07/2017, 12(47
`
`when selecting RDIMMs and motherboards.
`
`About the author
`Tomek Jasionowski is program manager for Enterprise Computing Division of
`Integrated Device Technology Inc., where he is responsible for enterprise-class
`semiconductor product developments.
`
`___________________________
`
`Did you find this article of interest? Then visit the Memory DesignLine, where we update
`daily with design, technology, product, and news articles tailored to fit your world. Too busy to
`go every day? Sign up for our newsletter to get the week's best items delivered to your inbox.
`Just click here and choose the "Manage Newsletters" tab.
`
`EMAIL THIS PRINT COMMENT
`
`More Related Links
`Tight Memory Supply Drives Micron's Record Sales
`ReRAM Goes 3D
`What’s Wrong With 3D NAND?
`Embedded MRAM Can Take the Heat
`Toshiba Fires Back at Western Digital, Seeks $1.1B in Damages
`
`Copyright © 2017 UBM Electronics, A AspenCore company, All rights reserved. Privacy Policy | Terms of
`Service
`
`http://www.eetimes.com/document.asp?doc_id=1279507&print=yes
`
`Page 8 of 8
`
`Polaris Innovations Ltd. Exhibit 2021
`Page 2021 - 8
`IPR2017-00116, Kingston Tech. Co. v. Polaris Innovations Ltd.
`
`