UNITED STATES PATENT AND TRADEMARK OFFICE
`_________________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_________________________
`
`REALTEK SEMICONDUCTOR CORP.,
`Petitioner,
`v.
`ATI TECHNOLOGIES ULC
`Patent Owner.
`_________________________
`Case No. IPR2023-00922
`U.S. Patent No. 8,760,454
`_________________________
`
`PATENT OWNER’S PRELIMINARY RESPONSE
`PURSUANT TO 37 C.F.R. § 42.107(A)
`
`
`
`

`

`
`
`Table of Contents
`INTRODUCTION ......................................................................................... 1 
`I. 
`II.  GRAPHICS PROCESSING PRIMER ........................................................ 3 
`A.  Graphics Processors Use Vertex and Pixel Data to Create Display Images . 3 
`B.  Graphics Processors Use Instruction “Threads” To Transform Vertex and
`Pixel Data ..................................................................................................... 5 
`C.  Conventional Graphics Processors Executed Vertex and Pixel Threads
`Using Separate Vertex Shaders and Pixel Shaders ..................................... 7 
`’454 PATENT OVERVIEW ......................................................................... 9 
`III. 
`A.  The ’454 Patent’s Unified Shader ................................................................ 10 
`B.  The Unified Shader Can Simultaneously Execute Vertex and Pixel Threads
`And Switch Quickly Between Threads At Various Degrees Of
`Completion ................................................................................................ 11 
`C.  The Invention of The ’454 Patent Determines What Data to Process By
`Evaluating Dynamic Storage Capacity ...................................................... 13 
`D.  The Invention Of The ’454 Patent Triggers Execution By Transmitting
`Data Rather Than Instructions ................................................................... 14 
`E.  The ’454 Patent’s Challenged Claims ......................................................... 15 
`1.  All Challenged Claims Require a Unified Shader ..................................... 15 
`2.  Challenged Claims 1 And 3-11 Require Performing Operations Based On
`Dynamically-Monitored Data Storage Capacity ................................... 15 
`3.  Challenged Claim 2 Is Directed to a Unified Shader Processor That
`Executes Pixel or Vertex Thread Instructions In Response To Receiving
`Selected Data From A Register ............................................................. 17 
`IPR Challenged Claims and Grounds ....................................................... 17 
`IV. 
`A.  Lindholm ’685 ............................................................................................. 18 
`B.  Amanatides ................................................................................................... 20 
`C.  Selzer ............................................................................................................ 21 
`V. 
`PETITIONER HAS FAILED TO ESTABLISH THE REQUIRED
`REASONABLE LIKELIHOOD OF SUCCESS FOR ANY
`GROUNDS ................................................................................................... 24 
`A.  Ground 1: Lindholm ’685 + Lindholm ’913 ................................................ 24 
`
`i
`
`

`

`
`
`ii. 
`
`1.  Neither Lindholm ’685 Nor Lindholm ’913 Are Prior Art ....................... 24 
`a.  The ’454 Patent Inventors Conceived of the Inventions At Least By
`August 24, 2001 ................................................................................. 25 
`b.  The ’454 Patent Exercised Diligence In Constructively Reducing To
`Practice From Shortly After Conception Until The Filing Of The
`Parent Patent On November 20, 2003 ............................................... 31 
`i.  The ’454 Patent Was Constructively Reduced To Practice At Least
`by November 20, 2003 .................................................................. 31 
`The Inventors and Other ATI Employees Diligently Worked Every
`Business Day to Reduce the Claimed Invention to Practice ......... 31 
`c.  The Federal Circuit Ruled That the Same R400 Evidence Established
`an Conception Date Before The Lindholm Patents And Diligent
`Reduction To Practice ....................................................................... 35 
`d.  Petitioner’s Arguments Regarding a Prior ITC Case Not Involving ATI
`Are Incorrect ...................................................................................... 37 
`2.  No Unified Shader, Because No Ability to Switch Between Unfinished
`Vertex And Pixel Threads (All Claims) ................................................ 39 
`3.  No Determination of Data to Process Based On Evaluation of Data
`Storage Space (Claims 1, 3-11) ............................................................. 40 
`4.  No Execution of Instructions “In Response to” Receiving Selected Data
`(Claim 2) ................................................................................................ 42 
`B.  Ground 2: Amanatides + Kohn .................................................................... 43 
`1.  No Unified Shader, Because No Ability to Switch Between Unfinished
`Vertex And Pixel Threads (All Claims) ................................................ 44 
`2.  No Determination of Data to Process Based on Evaluation of Data Storage
`Space (Claims 1, 3-11) .......................................................................... 46 
`3.  No “Selected Data” (Claims 2, 5) .............................................................. 48 
`C.  Ground 3: Selzer + Fiske ............................................................................. 49 
`1.  No Unified Shader, Because No Ability to Switch Between Unfinished
`Vertex And Pixel Threads (All Claims) ................................................ 49 
`2.  No Determination of Data to Process Based On Evaluation of Data
`Storage Space (Claims 1, 3-11) ............................................................. 52 
`
`ii
`
`

`

`
`
`3.  No Execution of Instructions “In Response to” Receiving Selected Data
`(Claim 2) ................................................................................................ 53 
`VI.  CONCLUSION ............................................................................................ 53 
`
`
`
`iii
`
`

`

`
`
`TABLE OF AUTHORITIES
`
` Page(s)
`
`Cases
`ATI Techs. ULC v. Iancu,
`920 F.3d 1362 (Fed. Cir. 2019) ........................................................ 24, 36, 37, 38
`
`Certain Consumer Electronics and Display Devices with Graphics
`Processing and Graphics Processing Units Therein,
`Inv. No. 337-TA-932 .................................................................................... 38, 46
`Perfect Surgical Techniques, Inc. v. Olympus Am., Inc.,
`841 F.3d 1004 (Fed. Cir. 2016) .......................................................................... 31
`Scott v. Koyama,
`281 F.3d 1243 (Fed. Cir. 2002) .......................................................................... 31
`Singh v. Brake,
`317 F.3d 1334 (Fed. Cir. 2003) .......................................................................... 25
`Townsend v. Smith,
`36 F.2d 292 (C.C.P.A. 1929) .............................................................................. 25
`Statutes and Other Authorities
`35 U.S.C. § 102(e) ..................................................................................................... 1
`37 C.F.R. §42.24(d) ................................................................................................. 56
`
`iv
`
`

`

`
`
`TABLE OF EXHIBITS
`
`Ex. No
`EX 2001
`EX 2002
`EX 2003
`EX 2004
`EX 2005
`
`EX 2006
`
`EX 2007
`
`EX 2008
`EX 2009
`
`EX 2010
`EX 2011
`
`EX 2012
`EX 2013
`EX 2014
`EX 2015
`EX 2016
`EX 2017
`EX 2018
`EX 2019
`EX 2020
`EX 2021
`EX 2022
`EX 2023
`EX 2024
`EX 2025
`EX 2026
`EX 2027
`
`Reference Name
`Declaration of William Mangione-Smith and CV
`IPR2015-00325, Declaration of Dr Wolfe, Sept. 9, 2015
`IPR2015-00325, Declaration of Calvin Watson, Sept. 9, 2015
`IPR2015-00325, Declaration of Lefebvre, Sept. 9, 2015
`IPR2015-00325,-00326, and-00330, Deposition Transcript of
`Calvin Watson, November 4, 2015
`IPR2015-00325, -00326, and -00330, Deposition Transcript of
`Dr. Wolfe, Nov. 10, 2015
`IPR2015-00325,-00326, and -00330, Deposition Transcript of
`Laurent Lefebvre, Nov.13, 2015
`Gruber et al, R400 Shader Processor 2001, Oct. 9, 2015
`R400 Document Library Folder History, November 1, 2000
`through April 8, 2005
`R400 Sequencer Specification (Version 0 4), August 14, 2001
`R400 Sequencer Specification (Version 2 0), September 24,
`2001
`R400 Shader Processor (Version 0 1), Jan. 23, 2001
`R400 Top Level Specification (Version 0 2), March 11, 2001
`R400 Program Logs
`Microsoft Site Visit (Feb. 2003)
`Log of Exhibits and File Location
`R400 Program Review Documentation (Feb. 2003)
`R400 MM Software Status (Feb. 2003)
`R400 I/O Presentation (Feb. 2003)
`R400 I/O Presentation (Feb. 2003)
`R400 August Program Review
`Executive Review - R400 (Oct. 2002)
`R400 Area Estimate
`R400 Executive Review (Sept. 2002)
`GFIXIP 9x SX Micro-Architecture Specification
`WD/IA VGT Micro-Architecture Specification
`GX9 SPI Specification
`
`v
`
`

`

`Ex. No
`EX 2028
`EX 2029
`EX 2030
`EX 2031
`EX 2032
`EX 2033
`EX 2034
`EX 2035
`EX 2036
`EX 2037
`EX 2038
`EX 2039
`EX 2040
`EX 2041
`EX 2042
`EX 2043
`EX 2044
`EX 2045
`EX 2046
`EX 2047
`EX 2048
`EX 2049
`EX 2050
`EX 2051
`EX 2052
`EX 2053
`EX 2054
`EX 2055
`EX 2056
`EX 2057
`EX 2058
`EX 2059
`
`
`
`Reference Name
`R400 Top Level Specification (Version 0.2)
`R400 Folder History Log
`R400 Folder History Log
`R400 Folder History Log
`R400 Review PowerPoint
`R400 Review PowerPoint
`R400 Development and Documentation
`Xenos GFX Change History July 2003 - Dec. 2003 Log
`R400 GFX Change History March 2003 - Dec. 2003
`R400 Primitive Assembly
`R400 Primitive Assembly
`Development Documentation
`Development Documentation
`Development Documentation
`R400 Performance Verification
`Development Documentation
`R400 PAD Program Review (Dec. 2002)
`R400 EMU Test Regress History Log
`R400 EMU Test Regress Statistics Log
`R400 Program Review (Dec. 2002)
`R400 Program Review (Dec. 2002)
`Device Development Progress
`Device Development Progress
`Device Development Progress
`Device Development Progress
`R400 Review Status PowerPoint
`TV Specification
`R400 Program Review PowerPoint and Test Results
`Development Documentation
`R400 PowerPoint
`R400 Program Review PowerPoint
`R400 Program Review
`
`vi
`
`

`

`Ex. No
`EX 2060
`EX 2061
`EX 2062
`EX 2063
`EX 2064
`EX 2065
`EX 2066
`EX 2067
`EX 2068
`
`EX 2069
`EX 2070
`EX 2071
`EX 2072
`EX 2073
`EX 2074
`EX 2075
`EX 2076
`EX 2077
`EX 2078
`EX 2079
`EX 2080
`EX 2081
`EX 2082
`EX 2083
`EX 2084
`EX 2085
`EX 2086
`EX 2087
`EX 2088
`EX 2089
`EX 2090
`EX 2091
`
`
`
`Reference Name
`Block Development Progress
`R400 Program Review (Dec. 11, 2002)
`R400 Program Review (Dec. 12, 2002)
`Development Documentation
`Development Documentation
`Development Documentation
`R400 Technical Documentation
`R400 Technical Documentation
`R400 GFX Change History March 2003 to December 2003
`Log
`Ikos 2002 Spreadsheet
`R400 - Program Review (Oct. 2002)
`Sqsp Regress Report
`Xenos Sq Change Log
`R400 Regress Testing
`Virtual Logic 3.1 User's Guide
`Virtual Logic 3.5.5 User's Guide
`IKOS Virtual Logic 2.1
`IKON Screenshots
`IKON Screenshots
`GFX9 User's Guide
`US Patent No. 6,897,871
`Samsung Exynos 5430 Octa SoC
`Samsung Exynos 3 Quad 3470 Processor Database
`HW Emulator First Triangle
`HW Simulator First Triangle
`R400 Regress Testing Logs
`R400 Regress Testing Logs
`R400 IKOS Status
`R400 Program Review
`R400 Program Review Documentation
`PA Check-in History Log
`RB Check-in History Log
`
`vii
`
`

`

`
`
`Reference Name
`SC Check-in History Log
`SPI Check-in History Log
`SP Check-in History Log
`SQ Check-in History Log
`SX Check-in History Log
`VGT Check-in History Log
`R400 File Logs 10/1/2002 – 4/17/2003
`Netlist Area Sheets
`IKOS Schematic
`Stephen Morein Depo Transcript (AMD v. LG) May 25, 2017
`Laurent Lefebvre Depo Transcript (337-TA-1044) June 28,
`2017
`IPR2015-00325, Declaration of Laurent Lefebvre
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`Development Documentation
`IPR2015-00325, EX 2018 - EX. 2056
`IPR2015-00325, EX 2057 - EX 2071
`Block Change Logs
`Andrew Gruber Depo Transcript (AMD v. LG) July 27, 2017
`IPR2015-00326, EX 2001 - EX2002
`IPR2015-00326, EX 2003
`IPR2015-00326, EX 2004
`IPR2015-00326, EX 2005
`IPR2015-00326, EX 2006
`IPR2015-00326, EX 2007 - EX 2072
`IPR2015-00326, EX 2073 - EX 2118
`
`Ex. No
`EX 2092
`EX 2093
`EX 2094
`EX 2095
`EX 2096
`EX 2097
`EX 2098
`EX 2099
`EX 2100
`EX 2101
`EX 2102
`
`EX 2103
`EX 2104
`EX 2105
`EX 2106
`EX 2107
`EX 2108
`EX 2109
`EX 2110
`EX 2111
`EX 2112
`EX 2113
`EX 2114
`EX 2115
`EX 2116
`EX 2117
`EX 2118
`EX 2119
`EX 2120
`
`
`
`viii
`
`

`

`
`
`I.
`
`INTRODUCTION
`The Board should deny Realtek’s Petition because it fails to establish the
`
`required reasonable likelihood of success under any of the asserted grounds.
`
`Ground 1: Ground 1 is based on a combination of the Lindholm ’685 and
`
`Lindholm ’913 patent references, under a 35 U.S.C. § 102(e) theory, using the June
`
`2003 filing dates of both references. The ground fails at least because neither
`
`reference is prior art to the ’454 patent. As the substantial evidence here shows,
`
`the ’454 patent was conceived of at least as early as August 24, 2001, and
`
`diligently reduced to practice thereafter. Further, the Federal Circuit held that the
`
`parent to the ’454 patent, which has similar claims, was conceived of before the
`
`Lindholm patents and diligently reduced to practice thereafter and therefore the
`
`Lindholm patents are not prior art, based on essentially the same evidence as here.
`
`The same result as in the Federal Circuit ruling should occur here.
`
`Grounds 1 and 3: Grounds 1 and 3 (based on Selzer and Fiske) fail
`
`substantively as well. First, all challenged claims require a unified shader, which
`
`must be able to switch between processing unfinished vertex and pixel threads.
`
`None of the asserted references in Grounds 1 and 3, alone or in combination,
`
`disclose that requirement. Second, all but one of the claims – i.e., claims 1 and 3-
`
`11 – require determining what data to process based on the amount of free storage
`
`space in a data store. None of the asserted references consider that factor in
`
`1
`
`

`

`
`
`determining what data to process. At best, they determine what to process based
`
`on pre-set priorities for certain data types or based on how much there is to process
`
`of a certain data type, which is insufficient. Third, the remaining claim, claim 2,
`
`requires generating pixel color, vertex position, appearance data “in response to”
`
`data selected from the data store. None of the references disclose this requirement;
`
`to the extent they discuss generating pixel color, vertex position, and appearance
`
`data, each reference discusses doing so in response to receiving instructions, not
`
`data.
`
`Ground 2: Ground 2 is based on Amanatides and Kohn. As with Grounds 1
`
`and 3, described above, the references, alone and in combination, fail to disclose a
`
`unified shader, which must be able to switch between processing unfinished vertex
`
`and pixel threads, and so the ground fails as to all claims. The ground further fails
`
`as to claims 1 and 3-11 for the same reasons as above as well: the references do not
`
`consider current data storage space when determining what data to process. The
`
`ground further fails as to claim 2 because there is no selecting of data within a
`
`general purpose register block, to determine what to process, as required by claim
`
`2. The combination merely transmits data to be processed in a round-robin fashion
`
`based on what is available.
`
`Accordingly, the Board should deny institution of this proceeding.
`
`2
`
`

`

`
`
`II. GRAPHICS PROCESSING PRIMER
`The ’454 patent and its challenged claims generally concern graphic
`
`processors and, more specifically, graphics processors that use a “unified shader”
`
`to perform both “vertex” and “pixel” operations. E.g., EX. 1001 (’454 patent) at
`
`1:32-34, 2:58-61; EX. 2001 (Declaration of Dr. Mangione-Smith) at ¶ 15.
`
`A. Graphics Processors Use Vertex and Pixel Data to Create Display
`Images
`Graphics processors are designed to convert a three-dimensional object into
`
`an image for display on a two-dimensional screen. EX. 2001 at ¶ 16. As part of this
`
`process, a three-dimensional object is rendered as a collection of simple shapes,
`
`called primitives. Id. For example, in the figure below, a three-dimensional
`
`character has been rendered as a collection of triangular primitives. Id.
`
`
`
`3
`
`

`

`
`
`Each triangular primitive is defined by the positions of its three corner
`
`points, i.e., its “vertices.” Id. at ¶ 17. After a three-dimensional object is rendered
`
`as a group of primitives, subsequent processing steps include converting the
`
`vertices of each primitive from three-dimensional coordinates to two-dimensional
`
`coordinates. Id. To illustrate, in the example shown below, a triangular primitive is
`
`being used to render a portion of a sphere and has vertices with three-dimensional
`
`position coordinates along the X-, Y-, and Z- axes. Id. These three-dimensional
`
`vertices coordinates are converted into two-dimensional coordinates (along only
`
`the X- and Y- axes) for display on a two-dimensional screen. Id.
`
`Subsequent processing steps include rendering each primitive as two-
`
`dimensional collection of dots on a screen called “pixels.” Id. at ¶ 18. As part of
`
`this process, the graphics processor uses the two-dimensional vertices coordinates
`
`to determine which pixels fill a particular primitive. Id. In the illustration below,
`
`
`
`4
`
`

`

`
`
`the pixels that fill the primitive defined by the XY coordinates are depicted in blue.
`
`Id.
`
`
`
`
`
`Further processing includes identifying the correct color and texture for each
`
`pixel filling the primitive. Id. at ¶ 19.
`
`B. Graphics Processors Use Instruction “Threads” To Transform
`Vertex and Pixel Data
`Graphics processors use various collections of instructions to transform a
`
`three-dimensional object into a two-dimensional display. Id. at ¶ 20. Each
`
`5
`
`

`

`
`
`collection of instructions is called a “thread.” Id. Each thread contains a sequence
`
`of instructions to be performed one after another. Id. Each instruction consists of
`
`an “OpCode” and an “Operand.” Id. The OpCode indicates which calculation
`
`operation is required by the instruction, such as an arithmetic or logical operation.
`
`Id. The Operand indicates the location of the graphics data upon which the
`
`calculation operation is to be performed. Id.
`
`
`Some threads comprise instructions that manipulate and transform vertex
`
`data. Id. at ¶ 21. These threads are known as “vertex threads,” and can include
`
`instructions that, for example, work together to transform a triangular primitive’s
`
`vertex coordinates from three-dimensional to two-dimensional (as discussed
`
`above), to translate or scale the primitive, to remove any portion of the triangular
`
`primitive that may be off the screen, or to cull primitives that do not project onto
`
`the screen. Id.
`
`Other threads comprise instructions that manipulate and transform pixel
`
`data. Id. at ¶ 22. These threads are known as “pixel threads,” and can include
`
`instructions that, for example, work together to transform pixel data by assigning
`
`6
`
`

`

`
`
`color value to the pixels, determining lighting effects, determining texture
`
`coordinate values, and blending colors. Id.
`
`C. Conventional Graphics Processors Executed Vertex and Pixel
`Threads Using Separate Vertex Shaders and Pixel Shaders
`Conventional graphics processors featured a number of circuits, each
`
`configured to perform a single type of transformation. These circuits are known as
`
`“shaders.” Id. at ¶ 23.
`
`Conventionally, each shader was dedicated to executing a particular type of
`
`thread to transform a particular type of data. Id. at ¶ 24. For example, a “vertex
`
`shader” would execute only vertex threads to transform vertex data, such as, for
`
`example, transforming the three-dimensional coordinates of a primitive’s vertices
`
`in object space into two-dimensional coordinates for a display screen. EX. 1001 at
`
`1:63-2:7. But to execute pixel threads to transform pixel data, another, separate
`
`shader was needed. This type of shader is known as a “pixel shader,” and was
`
`used, for example, determine and apply the color for each pixel making up a
`
`rendered object. Id. at 2:8-18.
`
`Thus, to execute the position and color threads and operations necessary to
`
`transform a three-dimensional object to a two-dimensional display, a conventional
`
`graphics processor needed at least two shaders, one a vertex shader and the other a
`
`pixel shader. Id. at 2:15-18. As described in the ’454 patent, this requirement
`
`7
`
`

`

`
`
`resulted in drawbacks in both size and performance. Ex. 2001 at ¶ 25. First, the
`
`need for multiple specialized shaders increased the size of graphics processors:
`
`Conventional graphics processors require the use of both a vertex
`shader and a pixel shader in order to generate an object. Because both
`types of shaders are required, known graphics processors are relatively
`large in size, with most of the real estate being taken up by the vertex
`and pixel shaders.
`
`Id. at 2:20-29.
`
`Second, sequencing the pixel shader after the vertex shader meant that one
`
`shader was dependent on – and had to wait for – the output of the other before it
`
`could continue processing, leading to idle downtime that slowed down the graphics
`
`processor and resulted in relatively poor performance:
`
`In addition to the real estate penalty associate with conventional
`graphics processors, there is also a corresponding performance penalty
`associated therewith. In conventional graphics processors, the vertex
`shader and the pixel shader are juxtaposed in sequential, pipelined
`fashion, with the vertex shader being positioned before and operating
`on vertex data before the pixel shader can operate on individual pixel
`data.
`
`Id. at 2:20-29; Ex. 2001 at ¶ 26.
`Such a prior art “pipeline” is illustrated in Figure 3 of the ’454 patent, which
`
`depicts a vertex shader 46 in series with pixel shader 54. Id. at ¶ 27.
`
`8
`
`

`

`
`
`
`
`Id. at Fig. 3.
`III.
`’454 PATENT OVERVIEW
`The ’454 patent issued June 24, 2014 and is titled “Graphics Processing
`
`Architecture Employing A Unified Shader.” EX. 1001.
`
`9
`
`

`

`
`
`A. The ’454 Patent’s Unified Shader
`As its title suggests, the ‘454 patent solves the real estate and performance
`
`problems of separate vertex and pixel shaders by providing a shader that is capable
`
`of executing both vertex threads and pixel threads to transform both vertex and
`
`pixel data, respectively. EX. 1001 at 2:58-61; EX. 2001 at ¶ 29. This multipurpose
`
`shader “is referred to as a ‘unified’ shader.” EX. 1001 at 2:58-61.
`
`In an embodiment described in the patent, the unified shader includes a
`
`register, a sequencer, and a processor. Id. at 3:4-10; EX. 2001 at ¶ 30. The unified
`
`shader’s register stores the inputs transmitted to the unified shader, including the
`
`vertex and pixel data that are to be transformed by the unified shader. EX. 1001 at
`
`3:4-5, 4:29-39.
`
`The unified shader’s sequencer stores the individual instructions that, when
`
`sequenced together, comprise the vertex and pixel threads that the unified shader
`
`will use to manipulate and transform the stored vertex and pixel data. Id. at 3:4-10,
`
`4:52-5:31; EX. 2001 at ¶ 31. The ’454 patent refers to those individual instructions
`
`(i.e., the logical and arithmetic operations) as vertex and pixel “calculation
`
`operations.” EX. 1001 at 3:4-7, 3:63-66, 4:42-45, 4:53-60, 5:36-52. The resulting
`
`threads are called vertex and pixel “manipulation operations.” Id. at 3:4-7. 5:19-36.
`
`The unified shader’s processor can execute the vertex and pixel instructions
`
`and threads (stored in the sequencer) on the vertex and pixel data (stored in the
`
`10
`
`

`

`
`
`register). Id. at 3:4-12; EX. 2001 at ¶ 32. The unified shader is thus configured to
`
`perform as both a complete pixel shader and a complete vertex shader in one. EX.
`
`1001 at 2:58-61, 3:10-12, 5:19-31.
`
`B.
`
`The Unified Shader Can Simultaneously Execute Vertex and Pixel
`Threads And Switch Quickly Between Threads At Various
`Degrees Of Completion
`In addition to describing a unified shader that can handle both vertex data
`
`and pixel data, the ’454 patent further addresses issues raised by such a unified
`
`shader. EX. 2001 at ¶ 33. For example, one such issue is how to allocate the
`
`unified shader’s resources among vertex and pixel data in a manner that prevents
`
`backlogs of data waiting to be processed and idle downtime for the shader. Id.
`
`The approach the ’454 patent discloses for increasing efficiencies and
`
`decreasing processing time is to employ a unified shader that can execute vertex
`
`and pixel threads, switching back and forth between unfinished threads to prevent
`
`downtime and backlogs of data sitting waiting to be processed. See, e.g., EX. 1001
`
`at 5:23-36 (“As such, the unified shader 62 of the present invention is able to
`
`perform both the vertex shading and pixel shading operations on data in the context
`
`of a graphics controller based on information passed from the multiplexer . . . . The
`
`unified shader 62 has ability to simultaneously per form vertex manipulation
`
`operations and pixel manipulation operations at various degrees of completion by
`
`11
`
`

`

`
`
`being able to freely switch between Such programs or instructions, maintained in
`
`the instruction store 98, very quickly.”); EX. 2001 at ¶ 34.
`
`For example, when executing the sequence of instructions in a vertex thread,
`
`the unified shader may encounter a vertex calculation operation instruction that it
`
`cannot yet perform because the necessary data or memory is not yet available. EX.
`
`1001 at 5:36-44; EX. 2001 at ¶ 35. Rather than having the unified shader sitting
`
`idle, waiting for the necessary data or memory so that it can execute the next
`
`vertex calculation operation and proceed with the vertex thread, the ’454 patent
`
`discloses the unified shader switching to pending pixel threads and instructions and
`
`executing pixel instructions until the idle vertex thread is ready to proceed. EX.
`
`1001 at 5:32-52.
`
`Thus, the ’454 patent discloses a unified shader that “has the ability to
`
`simultaneously perform vertex manipulation operations [vertex threads] and pixel
`
`manipulation operations [pixel threads] at various degrees of completion by being
`
`able to freely switch between such program or instructions [stored in the unified
`
`shader’s register] very quickly.” Id. at 5:32-36 (emphasis added); EX. 2001 at ¶ 36.
`
`“[T]hrough the sharing or resources within the unified shader 62, processing of
`
`image data is enhanced as there is no down time associated with the processor 96.”
`
`EX. 1001 at 5:49-52. The ’454 patent thus discloses and claims a unified shader
`
`12
`
`

`

`
`
`processor that can switch back and forth between executing unfinished vertex and
`
`pixel threads. Id.
`
`C. The Invention of The ’454 Patent Determines What Data to
`Process By Evaluating Dynamic Storage Capacity
`The claims of the ’454 patent require a “sequencer” and/or an “instruction
`
`store” to “maintain” instructions and require coordination with a “general purpose
`
`register block” and/or “shared resources” for storing data that can be specifically
`
`“selected” based on storage capability; all of which can be switched between “very
`
`quickly,” as explained above. Ex. 2001 at ¶ 37.
`
`In contrast to prior approaches, including that of the prior art asserted here,
`
`which may use pre-determined priorities or balance workloads based on how many
`
`of a particular task is in the queue to execute, the system of the ’454 patent selects
`
`the specific data based on the availability of space in the data storage for the
`
`particular types of data. EX. 2001 at ¶ 38. The specification states that “the present
`
`invention includes an arbiter circuit for selecting one of a plurality of inputs for
`
`processing . . . and a shader, coupled to the arbiter . . . wherein the shader performs
`
`one of the vertex operations or pixel operations based on the selected one of the
`
`plurality of inputs” by the arbiter. Id. at 2:58-3:17. The selection criteria, or
`
`“arbitration scheme,” includes at least the capacity to store particular types of data.
`
`The data storage is monitored for available space, which informs what data is
`
`selected to be transmitted to the processor to be processed: “if the general purpose
`
`13
`
`

`

`
`
`register block 92 does not have enough available space therein to store the
`
`incoming vertex data, such information will not be transmitted as the arbitration
`
`scheme of the arbiter 64 is not satisfied.” Id. at 5:32-44. In use, for example,
`
`when there is not enough room in the data storage for vertex data, the processor
`
`“continues pixel calculation operations that are to be or are currently being
`
`performed [by] the processor based on instructions maintained in an instruction
`
`store until enough registers within the general purpose register block become
`
`available” to store vertex data. Id. at Abstract (emphasis added).
`
`D. The Invention Of The ’454 Patent Triggers Execution By
`Transmitting Data Rather Than Instructions
`Further to the above, it is specifically the receipt of the selected data that
`
`causes the processor to perform the corresponding operations: “the shader
`
`performs one of the vertex operations or pixel operations based on the selected one
`
`of the plurality of inputs.” Id. at 2:58-3:17; EX. 2001 at ¶ 39. This approach is in
`
`contrast with the approach represented in the asserted references, in which the
`
`shader performs tasks based on the received instructions, not the received data as
`
`claimed. This data-driven approach where the receipt of particular selected data
`
`dictates the execution of instructions enables the invention’s ability to quickly
`
`switch between partially-completed tasks, as the data drives the decision-making
`
`about what to process, rather than the instructions doing so.
`
`14
`
`

`

`
`
`E.
`The ’454 Patent’s Challenged Claims
`The Petition challenges claims 1-11 of the ’454 patent. Challenged claims 1,
`
`2, 3, 4, 5, and 11 are independent claims. Challenged claims 6-10 depend directly
`
`or indirectly from independent claim 5.
`
`1.
`All Challenged Claims Require a Unified Shader
`All challenged claims require a unified shader, which as described above,
`
`requires the ability to switch between vertex and pixel operations, including on
`
`unfinished instructions. See Section III.B, supra; EX. 2001 at ¶ 41. Claim 2, for
`
`example, states:
`
`A unified shader, comprising:
`a general purpose register block for maintaining data;
`a processor unit;
`a sequencer, coupled to the general purpose register block and the
`processor unit, the sequencer maintaining instructions operative to
`cause the processor to execute vertex calculation and pixel calculation
`operations on selected data maintained in the general purpose register
`block; and
`wherein the processor unit executes instructions that generate a pixel
`color in response to selected data from the general purpose register
`block and generates vertex position and appearance data in response to
`selected data from the general purpose register block.
`
`EX. 1001 at Claim 2 (emphasis added).
`
`2.
`
`Challenged Claims 1 And 3-11 Require Performing
`Operations Based On Dynamically-Monitored Data Storage
`Capacity
`Claims 1 and 3-11 require that the unified shader be able to monitor storage
`
`space available for data that will be processed and, on that basis, determine what
`
`15
`
`

`

`
`
`data to process, as explained above in Section III.C. EX. 2001 at ¶ 42. For
`
`example, claim 1 states:
`
`A method carried out by a unified shader comprising:
`performing vertex manipulation operations and pixel manipulation
`operations by transmitting vertex data to a general purpose register
`block, and performing vertex operations on the vertex data by a
`processor within the unified shader unless the general purpose
`register block does not have enough available space therein to store
`incoming vertex data; and
`continuing pixel calculation operations that are to be or are currently
`being performed by the processor based on instructions maintained in
`an instruction store until enough registers within the general
`purpose register block become available.
`