Fully Buffered DIMM (FB-DIMM)
`Server Memory Architecture:
`Capacity, Performance, Reliability, and
`Longevity
`
`Pete Vogt
`Principal Engineer
`Intel Corp
`
`February 18, 2004
`
`1
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`Server Memory Architecture:
`Capacity, Performance, Reliability, and
`Longevity
`
`Pete Vogt
`Principal Engineer
`Intel Corp
`
`February 18, 2004
`
`1
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Agenda
`(cid:121) Server Capacity Problem and Solution
`(cid:121) Performance
`(cid:121) Reliability
`(cid:121) Longevity
`
`2
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`Intel Platform Memory Technology Roadmap
`DDR3
`DDR3
`DDR2 667/800
`
`DDR2 400/533
`
`FB-DIMM
`
`DDR2
`DDR
`
`RDRAM*
`
`SDRAM
`
`DDR266
`
`PC800
`
`DDR333/400
`
`PC1066
`
`PC133
`
`200420042004
`
`2004
`
`2005200520052005
`
`
`
`
`200520052006
`
`2005
`
`(cid:121) DDR2 400/533 support in all main IA segments in 2004
`– with DDR flexibility
`(cid:121) FB-DIMM new server interconnect in 2005
`(cid:121) Low Power SDRAM moving to low power DDR
`(cid:121) RDRAM still used in specific applications
`
`*Other names and brands may be claimed as the property of others
`3
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`FB-DIMM New Server Interconnect
`in 2005
`Why is FB-DIMM Needed in the
`Future?
`
`4
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`DRAM Bit Density; Mb/dev
`2500
`
`2000
`
`DRAM Device Trends
`(cid:121) DRAM bit density follows
`Moore’s law and
`increases 2X every ~2
`years
`
`1500
`
`1000
`
`500
`
`(cid:121) DRAM data rates driven
`by CPU performance
`increase 2X every
`generation (3.35 years)
`
`0
`
`99
`
`00
`
`01
`
`02
`
`07
`06
`05
`04
`03
`Rule of thumb - Source: Pete Vogt
`
`DRAM Data Rate; Mbps
`1200
`
`1000
`
`800
`
`600
`
`400
`
`200
`
`0
`
`99
`
`00
`
`01
`
`07
`06
`05
`04
`03
`02
`Projected historical data - Source: Pete Vogt
`
`DRAM Devices Continue to Scale
`
`5
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`Devices per Channel Trend
`(cid:121) Existing “stub-bus”
`architecture has
`impedance discontinuities
`that effect signal integrity
`
`Stub-bus Topology
`
`DRAM
`
`DRAM
`
`DRAM
`
`DRAM
`
`Mem
`Ctrl
`
`Impedance
`Discontinuities
`
`(cid:121) As the data rate goes up
`the number of devices on
`the channel is going down
`
`Stub-bus Devices/Channel
`160
`140
`120
`100
`80
`60
`40
`20
`0
`
`99
`
`00
`
`01
`
`02
`
`07
`06
`05
`04
`03
`Estimated trend - Source: Pete Vogt
`
`Devices per Channel is Decreasing
`
`6
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`Memory Capacity Trends
`(cid:121) Server performance drives
`2X capacity increase
`demand every ~2 years
`
`Capacity Demand; GB
`35
`30
`25
`20
`15
`10
`
`Demand
`
`99
`
`00
`
`01
`
`02
`
`03
`
`05
`
`Stub-bus Capacity/Ch: GB
`32
`28
`24
`20
`16
`12
`
`07
`06
`05
`04
`Rule of thumb - Source: Pete Vogt
`Demand
`
`Gap is
`growing
`
`Supply
`
`99
`
`00
`
`01
`
`02
`
`07
`06
`05
`04
`03
`Computed trend - Source: Pete Vogt
`
`048
`
`(cid:121) Stub-bus channel capacity
`(density * devices/ch)
`supply has hit a ceiling
`
`Meeting Demand Requires a Change
`
`7
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`FB-DIMM Eliminates the “Stubs”
`(cid:121) FB-DIMM buffers the DRAM
`data pins from the channel
`and uses point-to-point
`links to eliminate the stub
`bus
`
`DRAM
`
`Buffer
`
`DRAM
`
`Buffer
`
`DRAM
`
`Buffer
`
`Mem
`Ctrl
`
`Point-to-Point Links
`
`DRAM
`
`Buffer
`
`FB-DIMM
`
`(cid:121) FB-DIMM capacity scales
`throughout DDR2 & DDR3
`generations
`
`FB-DIMM Capacity/Ch: GB
`35
`30
`25
`20
`15
`10
`
`99
`
`00
`
`01
`
`02
`
`07
`06
`05
`04
`03
`FB-DIMM Capability - Source: Pete Vogt
`
`05
`
`FB-DIMM Meets the Capacity Demand
`
`8
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`FB-DIMM Solution Details
`
`DDR2 connector
`with unique key
`
`Commodity
`DRAMs
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Buffer
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Buffer
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Buffer
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Up to 8
`DIMMs
`• • •
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Buffer
`
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`DRAMDRAM
`
`Serial signaling
`similar to
`PCI-Express
`
`10
`
`14
`
`Memory
`Controller
`
`SMBus
`
`CLK
`Source
`
`Common clock
`source
`
`SMbus access to buffer
`registers
`
`9
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`FB-DIMM Channel Pin Count
`
`Data Path to DIMMs
`Data Path from DIMMs
`Total High-Speed Signals
`Power
`Ground
`Shared Pins (clocks,
`calibration, PLL pwr, test)
`Total Pins
`
`Diff Signals
`10
`14
`
`Pins
`20
`28
`
`48
`6
`12
`~ 3
`
`~ 69
`
`(cid:121) Compare with ~240 pins for DDR2 channel
`
`FB-DIMM Pin Count is 1/3rd of DDR2
`
`10
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`Routing Comparison
`Direct DDR2 Registered DIMMs:
`FB-DIMMs:
`1 Channel, 2 Routing Layers with 3rd layer
`2 Channels, 2 Routing Layers
`required for power
`(includes power delivery)
`Serpentine
`routing is
`complicated
`and uses up
`a lot of
`board area
`
`Fewer
`signals and
`no trace
`length
`matching
`minimizes
`board area
`
`FB-DIMM: Fewer Layers, Less Routing Area
`
`11
`
`Source: Intel Enterprise Architecture Group
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Riser Card
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`~12 inches
`
`DIMM
`
`Logic
`Analyzer
`Riser
`
`LAI buffer
`
`Problem & Solution
`
`~12 inches
`
`Expansion and Visibility
`(cid:121) Repeaters support
`flexible system
`packaging and
`memory riser cards
`
`Repeater
`
`Memory
`Controller
`
`(cid:121) Logic Analyzer Interface
`provides visibility of
`high-speed bus activity
`for system analysis
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`Memory
`Controller
`
`FB-DIMM is a Versatile Solution
`
`12
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Problem & Solution
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`DIMM
`
`FB-DIMM
`Memory
`Controller
`
`Capacity Comparison
`(cid:121) 24x capacity
`– 8GB vs. 192GB
`(cid:121) ~4x bandwidth
`– ~10GB/s vs. ~40GB/s
`(cid:121) ~Lower pin count
`– ~480 vs. ~420
`
`DDR2
`Memory
`Controller
`
`DIMM
`DIMM
`
`DIMM
`DIMM
`
`8GB with 1Gb x4 DRAMs
`~10GB/s of BW w/DDR2-800
`(only 2 ranks per channel)
`
`192GB with 1Gb x4 DRAMs
`~40 GB/s of BW w/DDR2-800
`(2 ranks per DIMM)
`
`FB-DIMM Solves the Server Memory Capacity Problem
`
`13
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`FB-DIMM Performance
`
`14
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`FB-DIMM
`Memory
`Controller
`
`DIMM
`DIMM
`
`DIMM
`DIMM
`
`DDR2
`Memory
`Controller
`
`Performance
`
`Entry Configuration Comparison
`(cid:121) Equal capacity
`– 8GB vs. 8GB
`(cid:121) Better throughput
`– ~6.5GB/s vs. ~8.2GB/s
`(cid:121) Lower pin count
`– ~480 vs. ~280
`
`DIMM
`
`DIMM
`
`DIMM
`
`DIMM
`
`8GB with 1Gb x4 DRAMs
`~6.5GB/s of throughput w/DDR2-800
`(only 2 ranks per channel)
`
`8GB with 1Gb x4 DRAMs
`~8.2GB/s of throughput w/DDR2-800
`(1 rank per DIMM)
`
`Better Throughput with 200 Fewer Pins
`
`15
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Performance
`Entry Configuration Comparison
`
`FB-DIMM provides higher
`sustained throughput
`because of the increased
`number of channels
`
`Inherent serialization
`delay, decreases as
`data rate goes up
`
`FB-DIMM has lower latency
`at higher throughput
`
`2 DDR2 800 channels, 2 ranks per channel
`4 FB-DIMM 800 channels, 1 single rank DIMM per channel
`
`Source: Intel
`
`0
`
`2
`
`4
`
`6
`8
`10
`12
`Sustained Memory Subsystem Throughput (GB/s)
`
`14
`
`16
`
`18
`
`16
`
`Average Memory Read Latency
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Performance
`
`Mid-range Config Comparison
`(cid:121) 4x capacity
`– 8GB vs. 32GB
`(cid:121) ~2.5x throughput
`– ~6.5GB/s vs. ~16.5GB/s
`(cid:121) Lower pin count
`– ~480 vs. ~280
`
`DIMM
`DIMM
`
`DIMM
`DIMM
`
`DIMM
`DIMM
`
`DIMM
`DIMM
`
`FB-DIMM
`Memory
`Controller
`
`4 DIMMs per channel
`doesn’t work
`
`DIMM
`DIMM
`DIMM
`DIMM
`
`DIMM
`DIMM
`DIMM
`DIMM
`
`DDR2
`Memory
`Controller
`
`8GB with 1Gb x4 DRAMs
`~6.5GB/s of throughput w/DDR2-800
`(only 2 ranks per channel)
`
`32GB with 1Gb x4 DRAMs
`~16.5GB/s of throughput w/DDR2-800
`(2 ranks per DIMM)
`
`FB-DIMM Provides 4x the Capacity
`
`17
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Performance
`Mid-range Config Comparison
`
`Latency crossover
`point is earlier
`
`FB-DIMM provides very high
`sustained throughput due to
`simultaneous reads/writes
`
`Slightly larger delay for
`the additional buffer
`
`FB-DIMM has consistent latency
`across a wide throughput range
`Source: Intel
`
`2 DDR2 800 channels, 2 ranks per channel
`4 FB-DIMM 800 channels, 2 dual rank DIMMs per channel
`
`0
`
`2
`
`4
`
`6
`8
`10
`12
`Sustained Memory Subsystem Throughput (GB/s)
`
`14
`
`16
`
`18
`
`18
`
`Average Memory Read Latency
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Performance
`
`Features That Reduce Latency
`(cid:121) Fast pass-through-path in the IO cells
`– The data is not stored and forwarded
`(cid:121) DRAM synchronized to channel clock
`– No asynchronous clock boundary crossings
`(cid:121) Simultaneous Reads and Writes
`– Writes do not block read accesses
`(cid:121) Elimination of read-to-read dead time
`– Reads from different DIMMs have no dead time between
`data transfers
`
`New Channel Features Mitigate Buffer Latency
`
`19
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`FB-DIMM RAS Features
`
`20
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Reliability
`
`Channel Reliability Targets
`(cid:121) Intel’s server goal is >100 years (1142 FITs) per
`Silent Data Corruption
`– The memory channels receive a small portion of this
`total FIT budget
`(cid:121) FB-DIMM is architected for exceptional Silent Data
`Corruption prevention
`– Channel data is protected by a strong CRC
`– Per channel segment SDC FIT rate <0.10 (1,142,000
`years) to support even the highest-RAS servers
`
`21
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Reliability
`
`Improved RAS Features
`(cid:121) CRC protection for commands and data
`(cid:121) Optional bit widths and CRC coverage to cover
`wide range of applications
`(cid:121) Transient bit error detection and retry
`(cid:121) Bit lane fail-over “correction”
`(cid:121) Pass-through-path for high availability
`(cid:121) Hot Add while the channel is active
`(cid:121) Error registers in the buffer for improved fault
`isolation
`Comprehensive RAS Features Deliver a Reliable Solution
`
`22
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`FB-DIMM Longevity
`
`23
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Longevity
`
`FB-DIMM Momentum
`(cid:121) Industry is aligned around FB-DIMM
`– Server OEM suppliers recognize the need for a long-term
`buffered DRAM memory solution
`– DRAM & DIMM vendors committed to delivering FB-
`DIMM products: Samsung, Elpida, Infineon, Micron,
`Hynix, Nanya, Kingston, Smart
`– Complete set of tools for system development
`(cid:121) FB-DIMM is a long-term strategic direction
`– Smooth transition from DDR2 to DDR3 using the same
`connectors and topology
`– Industry investment in the technology leveraged over
`multiple generations
`
`24
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Longevity
`
`FB-DIMM Standardization
`(cid:121) Intel working with JEDEC to establish
`FB-DIMM as an industry standard
`(cid:121) Seven specifications being developed
`– FB-DIMM: Architecture & Protocol
`– FB-DIMM: High-speed Signaling
`– FB-DIMM: Connector (variation of DDR2)
`– FB-DIMM: Module
`– FB-DIMM: Advanced Memory Buffer (AMB)
`– FB-DIMM: SPD EPROM
`– FB-DIMM: Test & Verification
`
`25
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Longevity
`
`FB-DIMM Technology Roadmap
`(cid:121) DDR2 is the first technology intercept
`– Needed part-way through the DDR2 lifetime
`(cid:121) All DDR3 data rates supported
`
`?
`
`FB-DIMM DDR3
`DDR3
`
`FB-DIMM DDR2
`DDR2
`
`DDR
`
`01
`
`02
`
`03
`
`04
`
`05
`
`06
`
`07
`
`08
`
`09
`
`26
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Longevity
`
`FB-DIMM Cost Effectiveness
`(cid:121) Cost competitive environment
`– Multiple buffer vendors and DIMM suppliers
`(cid:121) Eliminates memory expansion hubs
`– Motherboard not burdened with hubs
`(cid:121) Reduces motherboard routing area
`– Only 24 pairs, no serpentine trace matching
`(cid:121) Enables reduced PCB layer count
`– FB-DIMM can be routed in one signal layer
`
`FB-DIMM is a cost effective long-term technology
`
`27
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Summary
`
`Summary
`(cid:121) FB-DIMM solves the long-term server memory
`capacity problem
`(cid:121) Buffer latency is managed with new channel
`features
`(cid:121) Comprehensive RAS features deliver a reliable
`solution
`(cid:121) FB-DIMM is a cost effective long-term technology
`
`FB-DIMM is The Server Memory DIMM
`of the Future
`
`28
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Fully Buffered DIMM Server Memory
`Architecture
`
`Pete Vogt
`
`Intel Corporation
`
`Any Questions?
`
`Please remember to turn in
`Please remember to turn in
`your session survey form.
`your session survey form.
`
`29
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Definitions
`(cid:121) AMB: Advanced Memory Buffer – used to
`implement FB-DIMM
`(cid:121) FB-DIMM: Fully Buffered DIMM
`(cid:121) FIT: Failure in Time (failures in 1 billion hours)
`(cid:121) JEDEC: Joint Electronic Devices Engineering
`Council
`(cid:121) RAS: Reliability/Availability/Serviceability
`(cid:121) RDIMM: Registered DIMM
`(cid:121) SDC: Silent Data Corruption
`
`31
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Fully Buffered DIMM Architecture
`
`(cid:121) Session Outline:
`(cid:121) Server Memory Capacity Problem & Solution
`– Description of the capacity problem facing servers as data rates go up. Graph of
`DRAMs/channel vs. data rate. Description of the high-level FB-DIMM architecture
`that addresses this problem utilizing commodity DRAM devices.
`– Key point: FB-DIMM solves the Server Memory Capacity problem
`(cid:121) FB-DIMM Performance Features
`– Description of architecture features to mitigate the buffer latency; synchronous
`operation, fast pass through path, simultaneous reads/writes, & elimination of
`read-to-read dead time. Loaded latency graph illustrates result.
`– Key point: Buffer latency is managed with new channel features
`(cid:121) FB-DIMM Reliability/Availability/Serviceability (RAS) Features
`– Highlight RAS features in FB-DIMM including bit lane fail-over, reduced pin
`counts, high-availability pass through path, deterministic behavior, strong
`channel CRC protection.
`– Key point: Comprehensive RAS features deliver a reliable solution
`(cid:121) Long-term Server Memory Solution
`– Describe the longevity of FB-DIMM from DDR2-533 to DDR3-1600, indicate
`multiple buffer vendor sources, working within JEDEC to establish as a standard.
`– Key point: FB-DIMM is a cost effective long-term technology
`
`Pete Vogt
`
`32
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
`
`Title:
`Fully Buffered DIMM (FB-DIMM) Server Memory Architecture:
`Capacity, Performance, Reliability, and Longevity
`Abstract:
`What you’ll get from this session:
`– Trend data of the server memory capacity problem
`– Loaded latency curves illustrating FB-DIMM architecture
`features to mitigate inherent buffer delay
`– List of mechanisms that address FB-DIMM reliability
`concerns
`– Roadmap of multiple generations of DRAM support
`
`33
`
`Netlist Ex 2059
`Samsung v Netlist
`IPR2022-00996
`
`
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
