UNITED STATES PATENT AND TRADEMARK OFFICE
`_________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_________________________
`
`REALTEK SEMICONDUCTOR CORP.,
`Petitioner,
`v.
`ATI TECHNOLOGIES ULC
`Patent Owner.
`_________________________
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`_________________________
`
`PATENT OWNER’S RESPONSE
`PURSUANT TO 37 C.F.R. § 42.107(A)
`
`
`
`
`
`
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`Table of Contents
`
`Page
`
`INTRODUCTION ........................................................................................... 1
`I.
`II. GRAPHICS PROCESSING PRIMER ............................................................ 3
`A. Graphics Processors Use Vertex and Pixel Data to Create Display Images . 3
`B. Conventional Graphics Processors Use Separate Dedicated Vertex and
`Pixel Shaders ............................................................................................... 5
`’454 PATENT OVERVIEW ........................................................................... 7
`III.
`A. The ’454 Patent’s Unified Shader Determines Which Data To Process By
`Evaluating Storage Capacity As Seen In Claims 1, 3-10 ............................ 7
`B. The Unified Shader Can Simultaneously Execute Vertex and Pixel
`Operations and Switch Quickly Between Operations at Various Degrees
`of Completion as In Claim 11...................................................................... 9
`C. The Invention of the ’454 Patent Triggers Execution by Transmitting Data
`Rather Than Instructions as in Claim 2 ..................................................... 11
`IV. Asserted Prior Art .......................................................................................... 11
`A. Lindholm ’685 ............................................................................................. 11
`B. Amanatides ................................................................................................... 14
`C. Selzer ............................................................................................................ 15
`PETITIONER HAS FAILED TO PROVE THE CLAIMS OF THE
`V.
`’454 PATENT ARE OBVIOUS UNDER ANY GROUND ................................... 17
`A. Ground 1: Lindholm References .................................................................. 17
`1. Neither Lindholm ’685 Nor Lindholm ’913 Are Prior Art ....................... 17
`a. The ’454 Patent Inventors Conceived of the Inventions At Least by
`August 24, 2001 ................................................................................. 18
`b. The ’454 Patent Inventors Exercised Diligence in Constructively
`Reducing to Practice .......................................................................... 24
`i. The ’454 Patent Was Constructively Reduced to Practice At Least by
`November 20, 2003 ....................................................................... 24
`The Inventors and Other ATI Employees Diligently Worked Every
`Business Day to Reduce the Claimed Invention to Practice ......... 24
`
`ii.
`
`
`
`i
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`c. The Federal Circuit Ruled That the Same R400 Evidence Established
`Diligence ............................................................................................ 27
`d. Petitioner’s Arguments Regarding a Prior ITC Case Not Involving ATI
`Are Incorrect ...................................................................................... 29
`2. No Executing Operations Depending Upon an Amount Of Space
`Available In The Store (Claim 5), Much Less Performing Vertex
`Operations Or Pixel Operations Until Enough Storage Is Available For
`The Other Operation Type (Claims 1, 3, 4) .......................................... 29
`3. No Processor Unit That Performs Vertex Manipulation Operations and
`Pixel Manipulation Operations At Various Degrees Of Completion
`Based On Switching Between Instructions In The Instruction Store
`(Claim 11) .............................................................................................. 32
`4. No Execution of Instructions “In Response to” Receiving Selected Data
`(Claim 2) ................................................................................................ 33
`5. No “Memory” Separate From the “Store” (Claim 6) ................................ 34
`6. No “Control Signal” (Claims 7, 10) or “Arbiter” (Claim 10) ................... 35
`B. Ground 2: Amanatides + Kohn .................................................................... 36
`1. No Executing Operations Depending Upon An Amount Of Space
`Available In The Store (Claim 5), Much Less Performing Vertex
`Operations Or Pixel Operations Until Enough Storage Is Available For
`The Other Operation Type (Claims 1, 3, 4) .......................................... 36
`2. No Processor Unit That Performs Vertex Manipulation Operations And
`Pixel Manipulation Operations At Various Degrees Of Completion
`Based On Switching Between Instructions In The Instruction Store
`(Claim 11) .............................................................................................. 43
`3. No “Selected Data” (Claims 2, 5) .............................................................. 44
`4. Alleged “Selection Circuit” Is Not Within Alleged “Unified Shader” As
`Required (Claims 7, 10) ......................................................................... 46
`5. No “Control Signal” (Claims 7, 10) .......................................................... 49
`6. No “Selection Circuit” or “Arbiter,” “(Claim 10) ..................................... 50
`C. Ground 3: Selzer + Fiske ............................................................................. 52
`1. No Executing Operations Depending Upon An Amount Of Space
`Available In The Store (Claim 5), Much Less Performing Vertex
`
`
`
`ii
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Operations Or Pixel Operations Until Enough Storage Is Available For
`The Other Operation Type (Claims 1, 3, 4) .......................................... 52
`2. No Processor Unit That Performs Vertex Manipulation Operations And
`Pixel Manipulation Operations At Various Degrees Of Completion
`Based On Switching Between Instructions In The Instruction Store
`(Claim 11) .............................................................................................. 57
`3. No Execution of Instructions “In Response to” Receiving Selected Data
`(Claim 2) ................................................................................................ 59
`4. Alleged “Sequencer,” “Instruction Store,” “Circuitry,” “Selection
`Circuit,” and “Arbiter” Are Not Within Alleged “Unified Shader” As
`Required (Claims 2, 5-7, 10, 11) ........................................................... 60
`5. Alleged “Sequencer” And “Instruction Store” Do Not Maintain
`Instructions (Claims 2, 5, and 11) ......................................................... 63
`6. No “Arbiter” (Claim 10) or “Control Signal” (Claims 7, 10) ................... 65
`7. No Motivation to Combine Selzer and Fiske ............................................ 67
`D. Secondary Considerations ............................................................................ 68
`1. Initial Skepticism of ATI’s Unified Shader............................................... 69
`2. Unexpected Results in Developing the Unified Shader ............................ 70
`3. The Unified Shader’s Satisfaction of a Long-Felt Need and Failed
`Attempts by Others ................................................................................ 70
`4. Industry Praise of the Unified Shader in the Xenos GPU ......................... 71
`5. Commercial Success of the Xbox360 Containing ATI’s Unified Shader . 71
`6. Adoption by Others ................................................................................... 72
`VI. CONCLUSION .............................................................................................. 72
`
`
`
`
`
`
`
`
`iii
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`TABLE OF AUTHORITIES
`
` Page(s)
`
`Cases
`ATI Techs. ULC v. Iancu,
`920 F.3d 1362 (Fed. Cir. 2019) ................................................................ 17,18,28
`CAE Screenplates, Inc. v. Heinrich Fiedler Gmbh & Co. Kg,
`224 F.3d 1308 (Fed. Cir. 2000) .......................................................................... 35
`Catalina Mktg. Int’l v. Coolsavings.com, Inc.,
`289 F.3d 801 (Fed. Cir. 2002) ............................................................................ 48
`In re Magnum Oil Tools Int’l, Ltd.,
`829 F.3d 1364 (Fed. Cir. 2016) .......................................................................... 60
`Perfect Surgical Techniques, Inc. v. Olympus Am., Inc.,
`841 F.3d 1004 (Fed. Cir. 2016) .......................................................................... 24
`Scott v. Koyama,
`281 F.3d 1243 (Fed. Cir. 2002) .......................................................................... 25
`Singh v. Brake,
`317 F.3d 1334 (Fed. Cir. 2003) .......................................................................... 18
`Townsend v. Smith,
`36 F.2d 292 (C.C.P.A. 1929) .............................................................................. 18
`
`
`
`iv
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`TABLE OF EXHIBITS
`
`Reference Name
`Declaration of William Mangione-Smith and CV
`IPR2015-00325, Declaration of Dr Wolfe, Sept. 9, 2015
`IPR2015-00325, Declaration of Calvin Watson, Sept. 9, 2015
`IPR2015-00325, Declaration of Lefebvre, Sept. 9, 2015
`IPR2015-00325,-00326, and-00330, Deposition Transcript of
`Calvin Watson, November 4, 2015
`IPR2015-00325, -00326, and -00330, Deposition Transcript of
`Dr. Wolfe, Nov. 10, 2015
`IPR2015-00325,-00326, and -00330, Deposition Transcript of
`Laurent Lefebvre, Nov.13, 2015
`Gruber et al, R400 Shader Processor 2001, Oct. 9, 2015
`R400 Document Library Folder History, November 1, 2000
`through April 8, 2005
`R400 Sequencer Specification (Version 0 4), August 14, 2001
`R400 Sequencer Specification (Version 2 0), September 24,
`2001
`R400 Shader Processor (Version 0 1), Jan. 23, 2001
`R400 Top Level Specification (Version 0 2), March 11, 2001
`R400 Program Logs
`Microsoft Site Visit (Feb. 2003)
`Log of Exhibits and File Location
`R400 Program Review Documentation (Feb. 2003)
`R400 MM Software Status (Feb. 2003)
`R400 I/O Presentation (Feb. 2003)
`R400 I/O Presentation (Feb. 2003)
`R400 August Program Review
`Executive Review - R400 (Oct. 2002)
`R400 Area Estimate
`R400 Executive Review (Sept. 2002)
`GFIXIP 9x SX Micro-Architecture Specification
`WD/IA VGT Micro-Architecture Specification
`GX9 SPI Specification
`
`Ex. No
`EX2001
`EX2002
`EX2003
`EX2004
`EX2005
`
`EX2006
`
`EX2007
`
`EX2008
`EX2009
`
`EX2010
`EX2011
`
`EX2012
`EX2013
`EX2014
`EX2015
`EX2016
`EX2017
`EX2018
`EX2019
`EX2020
`EX2021
`EX2022
`EX2023
`EX2024
`EX2025
`EX2026
`EX2027
`
`
`
`v
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Reference Name
`R400 Top Level Specification (Version 0.2)
`R400 Folder History Log
`R400 Folder History Log
`R400 Folder History Log
`R400 Review PowerPoint
`R400 Review PowerPoint
`R400 Development and Documentation
`Xenos GFX Change History July 2003 - Dec. 2003 Log
`R400 GFX Change History March 2003 - Dec. 2003
`R400 Primitive Assembly
`R400 Primitive Assembly
`Development Documentation
`Development Documentation
`Development Documentation
`R400 Performance Verification
`Development Documentation
`R400 PAD Program Review (Dec. 2002)
`R400 EMU Test Regress History Log
`R400 EMU Test Regress Statistics Log
`R400 Program Review (Dec. 2002)
`R400 Program Review (Dec. 2002)
`Device Development Progress
`Device Development Progress
`Device Development Progress
`Device Development Progress
`R400 Review Status PowerPoint
`TV Specification
`R400 Program Review PowerPoint and Test Results
`Development Documentation
`R400 PowerPoint
`R400 Program Review PowerPoint
`R400 Program Review
`
`Ex. No
`EX2028
`EX2029
`EX2030
`EX2031
`EX2032
`EX2033
`EX2034
`EX2035
`EX2036
`EX2037
`EX2038
`EX2039
`EX2040
`EX2041
`EX2042
`EX2043
`EX2044
`EX2045
`EX2046
`EX2047
`EX2048
`EX2049
`EX2050
`EX2051
`EX2052
`EX2053
`EX2054
`EX2055
`EX2056
`EX2057
`EX2058
`EX2059
`
`
`
`vi
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Reference Name
`Block Development Progress
`R400 Program Review (Dec. 11, 2002)
`R400 Program Review (Dec. 12, 2002)
`Development Documentation
`Development Documentation
`Development Documentation
`R400 Technical Documentation
`R400 Technical Documentation
`R400 GFX Change History March 2003 to December 2003
`Log
`Ikos 2002 Spreadsheet
`R400 - Program Review (Oct. 2002)
`Sqsp Regress Report
`Xenos Sq Change Log
`R400 Regress Testing
`Virtual Logic 3.1 User's Guide
`Virtual Logic 3.5.5 User's Guide
`IKOS Virtual Logic 2.1
`IKON Screenshots
`IKON Screenshots
`GFX9 User’s Guide
`US Patent No. 6,897,871
`Samsung Exynos 5430 Octa SoC
`Samsung Exynos 3 Quad 3470 Processor Database
`HW Emulator First Triangle
`HW Simulator First Triangle
`R400 Regress Testing Logs
`R400 Regress Testing Logs
`R400 IKOS Status
`R400 Program Review
`R400 Program Review Documentation
`PA Check-in History Log
`RB Check-in History Log
`
`Ex. No
`EX2060
`EX2061
`EX2062
`EX2063
`EX2064
`EX2065
`EX2066
`EX2067
`EX2068
`
`EX2069
`EX2070
`EX2071
`EX2072
`EX2073
`EX2074
`EX2075
`EX2076
`EX2077
`EX2078
`EX2079
`EX2080
`EX2081
`EX2082
`EX2083
`EX2084
`EX2085
`EX2086
`EX2087
`EX2088
`EX2089
`EX2090
`EX2091
`
`
`
`vii
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Reference Name
`SC Check-in History Log
`SPI Check-in History Log
`SP Check-in History Log
`SQ Check-in History Log
`SX Check-in History Log
`VGT Check-in History Log
`R400 File Logs 10/1/2002 – 4/17/2003
`Netlist Area Sheets
`IKOS Schematic
`Stephen Morein Depo Transcript (AMD v. LG) May 25, 2017
`Laurent Lefebvre Depo Transcript (337-TA-1044) June 28,
`2017
`IPR2015-00325, Declaration of Laurent Lefebvre
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`IPR2015-00325, EX2018 - EX2056
`IPR2015-00325, EX2057 - EX2071
`Block Change Logs
`Andrew Gruber Depo Transcript (AMD v. LG) July 27, 2017
`IPR2015-00326, EX2001 - EX2002
`IPR2015-00326, EX2003
`IPR2015-00326, EX2004
`IPR2015-00326, EX2005
`IPR2015-00326, EX2006
`IPR2015-00326, EX2007 - EX2072
`IPR2015-00326, EX2073 - EX2118
`Supplemental Declaration of William Mangione-Smith
`Supplemental Declaration of Calvin Watson
`Deposition Transcript of Hanspeter Pfister (02/16/2024)
`
`Ex. No
`EX2092
`EX2093
`EX2094
`EX2095
`EX2096
`EX2097
`EX2098
`EX2099
`EX2100
`EX2101
`EX2102
`
`EX2103
`EX2104
`EX2105
`EX2106
`EX2107
`EX2108
`EX2109
`EX2110
`EX2111
`EX2112
`EX2113
`EX2114
`EX2115
`EX2116
`EX2117
`EX2118
`EX2119
`EX2120
`EX2121
`EX2122
`EX2123
`
`
`
`viii
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Reference Name
`Beyond3D-ATI Xenos: Xbox 360 Graphics Demystified –
`July 26, 2017
`Dictionary of Computing 5th Ed (excerpt)
`Microsoft Computer Dictionary 5th Edition (excerpt)
`How GPUs Work, David Luebke, Greg Humphreys
`2006 Microsoft Corporation Annual Report
`ATI and NVIDIA Proclaim Different Graphics Processors
`Architecture Goals, Anton Shilov (December 23, 2004)
`NVIDIA Chief Architect: Unified Pixel and Vertex Pipelines –
`The Way to Go. NVIDIA Says It Would Make a Chip with
`Unified Pipes “When it Makes Sense” Anton Shilov (July 11,
`2005)
`FORTUNE: Review of the Xobx 360 – Nov. 17, 2005
`TECHSPOT – History of the Modern Graphics Processor, Part
`3 – The Fall of 3Dfx and The Rise of Two Giants, Graham
`Singer (April 10, 2013)
`Working of Xbox 360 – How Xbox 360 Works (February 11,
`2010)
`Xbox 360: the Ars Technica Review, Ben Kuchera
`(November 30, 2005)
`Microsoft Xbox dominates a sluggish 2011 gaming market,
`Nathan Pensky (January 13, 2012)
`Microsoft’s Xbox 360, Sony’s PS3 – A Hardware Discussion,
`Anand Lal Shimpi & Derek Wilson (June 24, 2005)
`Qualcomm History and its GPU (R)evolution – Ryan Shrout
`(June 22, 2015)
`Xbox 360 vs. PlayStation 3: The Hardware Throwdown –
`IGN, Jesse Schedeen (August 26, 2010)
`Xbox 360 and Playstation 3’s Graphics Card (GPU) Compared
`– Red Gaming Tech, CrimsonRayne (March 11, 2013)
`Xbox 360 GPU – NeutralX2 (April 26, 2006)
`Wikipedia - Adreno
`Game Consoles: Global Market share 2006-2010 – Statista
`NVIDIA – Frequently Asked Questions
`
`Ex. No
`EX2124
`
`EX2125
`EX2126
`EX2127
`EX2128
`EX2129
`
`EX2130
`
`EX2131
`EX2132
`
`EX2133
`
`EX2134
`
`EX2135
`
`EX2136
`
`EX2137
`
`EX2138
`
`EX2139
`
`EX2140
`EX2141
`EX2142
`EX2143
`
`
`
`ix
`
`

`

`Ex. No
`EX2144
`
`EX2145
`EX2146
`
`EX2147
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Reference Name
`The Xbox 360 Uncloaked – The Real Story Behind
`Microsoft’s Next-Generation Video Game Console, Dean
`Takahashi
`Andrews - Xbox 360 System Architecture
`Interview with Ken Kutaragi, President of SCEI (3) – “Why I
`worked with NVIDIA on the PS3”
`Scott Wasson, “Details of ATI’s Xbox 360 GPU unveiled,”
`The Tech Report (May 19, 2005)
`
`x
`
`
`
`
`
`

`

`I.
`
`INTRODUCTION
`Petitioner Realtek has failed to prove that any of the challenged claims 1-11
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`of Patent Owner’s ATI’s U.S. Patent No. 8,760,454 (“the ’454 patent”) are
`
`unpatentable.
`
`First, the Lindholm references are not prior art, which the Institution Decision
`
`(“ID”) agreed with, and so Ground 1 fails. The conception of the ’454 patent’s
`
`inventions antedates the Lindholm references’ critical date, and ATI worked
`
`diligently to reduce the inventions to practice, as found by the Federal Circuit as to
`
`a parent patent, regarding the same real-world product.
`
`Second, claims 1 and 3-10 require evaluation of storage space in order to
`
`determine what type of operations to perform, with various levels of specificity
`
`required in the different claims. All three grounds fail to disclose even the broadest
`
`formulation, which the ID agreed with as to Ground 3. All three grounds rely on the
`
`concept of priority and/or load balancing, in which the system first processes data
`
`that has higher priority than other data; it may be that the system has more of a
`
`certain type of data to process, causing that to be higher priority, or it may be a pre-
`
`set preference. But there is no disclosure in any reference that the amount of storage
`
`space for incoming data is evaluated in determining what to choose to process.
`
`Third, claim 11 requires the ability to perform operations at various stages of
`
`completion by switching between instructions stored in the instruction store, i.e.,
`
`
`
`1
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`switching between incomplete operations. None of the grounds disclose that
`
`functionality.
`
`Fourth, claim 2 requires executing instructions in response to receiving
`
`selected data from the data store. All three grounds fail to demonstrate this as well,
`
`because the execution they describe is not done in response to receiving data, but
`
`rather in response to receiving the instructions, nor is the data “selected” from the
`
`data store.
`
`Fifth, the claims require that certain elements are within “unified shader” of
`
`the claims, such as the “sequencer” (claims 2, 5), instruction store (claim 11),
`
`selection circuit (claim 7), arbiter (claim 10), and others, but Ground 2 and 3 point
`
`to items outside the alleged unified shader, and thus the grounds fail as to those
`
`claims.
`
`Sixth, claims 7 and 10 require that an arbiter send a control signal to a
`
`selection circuit, prompting it to send information to a data store. Petitioner
`
`generally fails to identify anything for these elements across the grounds, and what
`
`it does point to contradicts a POSITA’s understanding of the terms and its own
`
`allegations, for example, when it points to “state data” as a “control signal.”
`
`Lastly, all grounds fail because secondary considerations support non-
`
`obviousness of the ’454 Patent.
`
`
`
`2
`
`

`

`II. GRAPHICS PROCESSING PRIMER
`The ’454 Patent issued June 24, 2014, and is titled “Graphics Processing
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`Architecture Employing A Unified Shader.” EX1001. The challenged claims
`
`concern graphic processors that use a “unified shader” to perform both “vertex”
`
`and “pixel” operations. E.g., EX1001 (’454 Patent), 1:32-34, 2:58-61; EX2121
`
`(Mangione-Smith Decl.), ¶15.
`
`A. Graphics Processors Use Vertex and Pixel Data to Create Display
`Images
`Graphics processors are designed to convert a three-dimensional object into
`
`an image for display on a two-dimensional screen. EX2121, ¶16. As part of this
`
`process, a three-dimensional object is rendered as multiple shapes, called
`
`primitives, such as triangles (see below example). Id.
`
`
`Each primitive has three corner points, or three “vertices.” Id., ¶17. To
`
`render a three-dimensional object on a flat screen, each vertex is converted from
`
`three-dimensional coordinates to two-dimensional coordinates as illustrated below.
`
`Id.
`
`
`
`3
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`
`Subsequent processing includes rendering each primitive as two-dimensional
`
`collection of dots called “pixels.” Id., ¶18. The processor uses the two-dimensional
`
`coordinates to determine which pixels fill each primitive (e.g., the XY coordinates
`
`depicted in blue below). Id.
`
`Further processing includes coloring and applying texture for each pixel
`
`
`
`filling the primitive. Id., ¶19.
`
`
`
`4
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`
`B. Conventional Graphics Processors Use Separate Dedicated Vertex
`and Pixel Shaders
`Conventional graphics processors featured a number of circuits, each
`
`configured to perform a single type of transformation. These circuits are known as
`
`
`
`“shaders.” Id., ¶23.
`
`Conventionally, each shader was dedicated to executing a particular type of
`
`operation to transform a particular type of data. Id., ¶24. For example, a “vertex
`
`shader” would execute only vertex operations to transform vertex data. EX1001,
`
`1:63-2:7. But to execute pixel operations to transform pixel data, another, separate
`
`“pixel” shader was needed. Id., 2:8-18. This requirement resulted in drawbacks in
`
`both size and performance. EX2121, ¶25.
`
`First, the need for multiple specialized shaders increased the size of graphics
`
`processors and overall integrated circuit. EX1001, 2:20-29. Second, sequencing
`
`the pixel shader after the vertex shader meant that one shader was dependent on the
`
`
`
`5
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`output of the preceding shader before it could continue processing, leading to idle
`
`downtime and less optimal performance. Id., 2:20-29; EX2121, ¶26.
`
`Such conventional “pipeline” is illustrated below, depicting vertex shader 46
`
`in series with pixel shader 54. Id., ¶27.
`
`EX1001, Fig. 3.
`
`
`
`6
`
`
`
`

`

`III.
`
`’454 PATENT OVERVIEW
`Petitioner challenges claims 1-11. Claims 1-5 and 11 are independent
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`claims. Claims 6-10 depend on claim 5.
`
`A. The ’454 Patent’s Unified Shader Determines Which Data To
`Process By Evaluating Storage Capacity As Seen In Claims 1, 3-
`10
`In contrast to prior approaches (including that of the asserted prior art),
`
`which may use pre-determined priorities or balance workloads based on the
`
`amount of work in the queue for execution, the ’454 Patent describes determining
`
`whether to process a vertex or pixel operation based on available storage space.
`
`EX2121, ¶31.
`
`For example, the patent describes an “arbiter circuit for selecting one of a
`
`plurality of inputs for processing [by the unified shader] . . . wherein the shader
`
`performs one of the vertex operations or pixel operations based on the [arbiter’s]
`
`selected one of the plurality of inputs.” EX1001, 2:58-3:17; EX2121, ¶32. The
`
`arbiter’s selection criteria, or “arbitration scheme,” includes at least the capacity to
`
`store particular types of data. Id. To this end, the data storage (the unified shader’s
`
`register block) is monitored for available space, which informs what data is
`
`selected to be transmitted to the processor to be processed: “if the general purpose
`
`register block 92 does not have enough available space therein to store the
`
`
`
`7
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`incoming vertex data, such information will not be transmitted as the arbitration
`
`scheme of the arbiter 64 is not satisfied.” Id., 5:32-44.
`
`In use, for example, when there is not enough room in the data storage for
`
`vertex data, the processor “continues pixel calculation operations that are to be or
`
`are currently being performed [by] the processor based on instructions maintained
`
`in an instruction store until enough registers within the general purpose register
`
`block become available” to store vertex data. EX1001, Abstract (emphasis added);
`
`EX2121, ¶33.
`
`This is in contrast with the prior art approach of choosing what to process
`
`based on how much of a given type of data is in the queue to be processed, or
`
`based on some pre-set priority or other prioritization scheme.
`
`The claims require this functionality in a variety of different ways, with
`
`different levels of specificity. Id., ¶34.
`
`Claim 1 is the most specific, requiring performing vertex operations until
`
`there is no more room in the general purpose register block for incoming vertex
`
`data, in which case continuing to perform pixel operations until enough space
`
`appears to once again store vertex data. Id. This requires two evaluations: first that
`
`there is no longer room for additional vertex data in the general purpose register
`
`block (at a time when the system is performing vertex operations), and second, that
`
`
`
`8
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`at a later point in time (when the system is performing pixel operations), there is
`
`once again room for additional vertex data in the general purpose register block. Id.
`
`Claims 3 and 4 are broader. Id., ¶35. Claim 3 requires a processor unit to be
`
`“operative to perform pixel calculation operations until enough shared resources
`
`become available and then use the shared resources to perform vertex calculation
`
`operations.” Id. Claim 4 requires the opposite; the processor unit must be
`
`“operative to perform vertex calculation operations until enough shared resources
`
`become available and then use the shared resources to perform pixel calculation
`
`operations.” Id.
`
`Claim 5 is the broadest, requiring that the processor unit “execute vertex
`
`calculation and pixel calculation operations on selected data maintained in a store
`
`depending upon an amount of space available in the store.” Id., ¶36.
`
`As explained below, the prior art references fail to teach even the broadest of
`
`these formulations.
`
`B.
`
`The Unified Shader Can Simultaneously Execute Vertex and Pixel
`Operations and Switch Quickly Between Operations at Various
`Degrees of Completion as In Claim 11
`Further, the patent describes and claims how to allocate the unified shader’s
`
`processing resources among vertex and pixel data in a manner that prevents
`
`backlogs of data waiting to be processed and idle downtime for the shader. Id.
`
`Under the claimed approach, the disclosed unified shader switches back and forth
`
`
`
`9
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`between unfinished operations to prevent downtime and backlogs of data sitting
`
`waiting to be processed. EX1001, 5:23-36; EX2121, ¶37.
`
`To illustrate, when executing a vertex operation, the unified shader may
`
`encounter a vertex instruction that it cannot yet perform because the necessary data
`
`or memory is not yet available. EX1001, 5:36-44; EX2121, ¶38. Rather than
`
`having the unified shader sitting idle, waiting for the necessary data or memory so
`
`that it can proceed with the rest of the vertex operation, the unified shader switches
`
`to pending pixel operations and executes pixel instructions until the idle vertex
`
`operation is ready to proceed. EX1001, 5:32-52.
`
`Thus, the ’454 Patent discloses a unified shader that “has the ability to
`
`simultaneously perform vertex manipulation operations [vertex operations] and
`
`pixel manipulation operations [pixel operations] at various degrees of completion
`
`by being able to freely switch between such program or instructions very quickly.”
`
`Id., 5:32-36 (emphasis added); see id., claim 11; EX2121, ¶39. “[T]hrough the
`
`sharing of resources within the unified shader 62, processing of image data is
`
`enhanced as there is no down time associated with the processor 96.” EX1001,
`
`5:49-52. The ’454 Patent thus discloses and claims a unified shader that can
`
`switch back and forth between executing unfinished vertex and pixel operations.
`
`Id.
`
`
`
`10
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`C. The Invention of the ’454 Patent Triggers Execution by
`Transmitting Data Rather Than Instructions as in Claim 2
`The ’454 patent specifically provides the receipt of selected data is what
`
`causes the processor to perform the corresponding operations: “the shader
`
`performs one of the vertex operations or pixel operations based on the selected one
`
`of the plurality of inputs.” Id., 2:58-3:17; see id., claim 2; EX2121, ¶¶41-42. This
`
`approach is in contrast with the approaches in the asserted references, in which the
`
`shader performs tasks based on the received instructions, not the received data as
`
`claimed. Id. This data-driven approach where the receipt of particular selected
`
`data dictates the execution of instructions enables the invention’s ability to quickly
`
`switch between partially-completed tasks, as the data drives the decision-making
`
`about what to process, rather than the instructions doing so. Id.
`
`IV. ASSERTED PRIOR ART
`A. Lindholm ’685
`Lindholm ’685 regards graphics architecture with a separate pixel input
`
`buffer and vertex input buffer, and several execution pipelines:
`
`
`
`11
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`
`EX1006 (Lindholm ’685), Fig. 2; EX2121, ¶303. An execution pipeline is
`
`
`
`depicted below:
`
`
`
`12
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`
`
`EX1006, Fig. 4. In response to “program instructions,” “programmable
`
`computation units (PCUs) within an Execution Pipeline 240 [] perform operations
`
`such as tessellation, perspective correction, texture mapping, shading, blending,
`
`and the like.” Id., 5:23-29. “Processed samples are output from each Execution
`
`
`
`13
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`Pipeline 240 to a Pixel Output Buffer 270,” and then “[t]he processed samples are
`
`output from Pixel Output Buffer 270 to Raster Analyzer 160.” Id., 5:30-35. After
`
`rasterization, the pixel data is sent to another PCU for pixel processing. Id., Figs,
`
`2, 4, 10:55-11:14.
`
`B. Amanatides
`Amanatides regards graphics architecture in which a “host distributes
`
`primitives in a round-robin manner” to processors, labeled “G/R”:
`
`
`
`EX1007 (Amanatides), Fig. 3, 157; EX2121, ¶304. “Each processor performs the
`
`front-end tasks … for the primitives. It then broadcasts the results to the other
`
`processors to perform the back end operations of rasterization.” EX1007, 157.
`
`The reference also discusses “load balancing” based on “[f]or example, if there is a
`
`lot of texture mapping” tasks to be done and based on prioritization of certain types
`
`of actions. Id., 157, 159.
`
`
`
`14
`
`

`

`C.
`Selzer
`Selzer describes graphics architecture with a traditional pipeline involving a
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`first stage of geometry processing, followed by a second stage of rendering
`
`processing:
`
`
`
`
`
`15
`
`

`

`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`
`
`
`EX1009 (Selzer), Figs. 4.2, 4.4; EX2121, ¶305. “The geometry module performs
`
`the transformation, clipping, polygon and patch subdivision, normal interpolation
`
`and renormalisation and lighting operations.” EX1009, 41. Alternatively, a
`
`rendering module could perform that task. Id. After that step, the geometry
`
`module or rendering module that performed the geometry processing “deliver[s]
`
`processed data to the FIFOs of the appropriate [separate] rendering modules” for
`
`rendering. Id.
`
`
`
`16
`
`

`

`V.
`
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`PETITIONER HAS FAILED TO PROVE THE CLAIMS OF THE ’454
`PATENT ARE OBVIOUS UNDER ANY GROUND
`A. Ground 1: Lindholm References
`1.
`Neither Lindholm ’685 Nor Lindholm ’913 Are Prior