UNITED STATES PATENT AND TRADEMARK OFFICE
`_______________
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_______________
`REALTEK SEMICONDUCTOR CORP.,
`Petitioner
`
`v.
`
`ATI TECHNOLOGIES ULC
`Patent Owner
`_______________
`Case No.: IPR2023-00922
`
`U.S. Patent No. 8,760,454
`Issue Date: June 24, 2014
`Title: Graphics Processing Architecture Employing a Unified Shader
`_______________
`
`PETITION FOR INTER PARTES REVIEW
`OF U.S. PATENT NO. 8,760,454
`
`

`

`TABLE OF CONTENTS
`
`Page
`
`INTRODUCTION .......................................................................................... 1
`I.
`II. MANDATORY NOTICES ............................................................................ 1
`A.
`Real Parties-In-Interest ......................................................................... 1
`B.
`Related Matters ..................................................................................... 1
`C.
`Lead and Backup Counsel .................................................................... 3
`D.
`Service Information .............................................................................. 3
`III. PAYMENT OF FEES .................................................................................... 4
`IV. STANDING .................................................................................................... 4
`V.
`THE BOARD SHOULD NOT EXERCISE ITS DISCRETION TO
`DENY INSTITUTION ................................................................................... 4
`VI. REQUEST FOR INTER PARTES REVIEW OF CLAIMS 1-11 OF
`THE ‘454 PATENT ........................................................................................ 5
`A.
`37 C.F.R. §42.104(b)(1): Claims for Which IPR is Requested ........... 5
`B.
`37 C.F.R. §42.104(b)(2): Identification of Prior Art and
`Asserted Grounds for Which IPR is Requested ................................... 6
`Level of Ordinary Skill in the Art ........................................................ 8
`C.
`37 C.F.R. §42.104(b)(5): Evidence Supporting Challenge .................. 8
`D.
`VII. THERE EXISTS A REASONABLE LIKELIHOOD THAT THE
`CHALLENGED CLAIMS ARE UNPATENTABLE ................................... 8
`A.
`Technology Background ...................................................................... 8
`B. Description of the Alleged Invention of the ’454 Patent ................... 13
`C.
`Prosecution History of the ’454 Patent and Alleged Priority of
`Invention ............................................................................................. 15
`D. Ground #1: Claims 1-11 based on the Lindholm Patents .................. 16
`1.
`The Lindholm ’685 Patent ....................................................... 16
`2.
`The Lindholm ’913 Patent ....................................................... 20
`3.
`Combining the Lindholm Patents ............................................ 21
`4.
`Independent Claim 2 ................................................................ 22
`
`i
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`5.
`
`6.
`
`7.
`
`Claim 2 [preamble] – Unified Shader ........................... 23
`a.
`Claim 2[a] – General Purpose Register Block .............. 26
`b.
`Claim 2[b] and [d] – Processor Unit .............................. 27
`c.
`Claim 2[c] - Sequencer .................................................. 29
`d.
`Independent Claim 3 ................................................................ 30
`a.
`Claim 3 [preamble] – Unified Shader ........................... 30
`b.
`Claim 3[a] and [c] – Processor Unit .............................. 30
`c.
`Claim 3[b] – Shared Resources ..................................... 35
`Independent Claim 4 ................................................................ 35
`a.
`Claim 4 [preamble] – Unified Shader ........................... 36
`b.
`Claim 4[a] and [c] – Processor Unit .............................. 36
`c.
`Claim 4[b] – Shared Resources ..................................... 36
`Independent Claim 5 ................................................................ 36
`a.
`Claim 5 [preamble] – Unified Shader ........................... 36
`b.
`Claim 5[a] – Processor Unit .......................................... 37
`c.
`Claim 5[b] – Sequencer ................................................. 37
`Dependent Claim 6................................................................... 37
`8.
`Dependent Claims 7 and 10 ..................................................... 37
`9.
`10. Dependent Claim 8................................................................... 39
`11. Dependent Claim 9................................................................... 39
`a.
`Claim 11 [preamble] – Unified Shader ......................... 39
`b.
`Claim 11[a] and [c] – Processor Unit ............................ 40
`c.
`Claim 11[b] – Instruction Store ..................................... 40
`Ground #2: Claims 1-11 based on the Combination of
`Amanatides and Kohn ........................................................................ 41
`1.
`Independent Claim 2 ................................................................ 45
`a.
`Claim 2[a] – General Purpose Register Block .............. 46
`
`E.
`
`ii
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`2.
`
`3.
`
`Claim 3[a] and [c] – Processor Unit .............................. 53
`b.
`Claim 3[b] – Shared Resources ..................................... 56
`c.
`Independent Claim 4 ................................................................ 57
`b.
`Claim 4[a] and [c] – Processor Unit .............................. 57
`c.
`Claim 4[b] – Shared Resources ..................................... 57
`Independent Claim 5 ................................................................ 57
`a.
`Claim 5 [preamble] – Unified Shader ........................... 57
`b.
`Claim 5[a] – Processor Unit .......................................... 57
`c.
`Claim 5[b] – Sequencer ................................................. 57
`Dependent Claim 6................................................................... 57
`Dependent Claims 7 and 10 ..................................................... 58
`Dependent Claim 8................................................................... 60
`Dependent Claim 9................................................................... 60
`Independent Claim 11 .............................................................. 60
`a.
`Claim 11 [preamble] – Unified Shader ......................... 60
`b.
`Claim 11[a] and [c] – Processor Unit ............................ 60
`c.
`Claim 11[b] – Instruction Store ..................................... 61
`Independent Claim 1 ................................................................ 61
`9.
`Ground #3: Claims 1-11 based on the Combination of Selzer
`and Fiske ............................................................................................. 61
`1.
`Selzer ........................................................................................ 61
`2.
`Fiske ......................................................................................... 63
`3.
`Combining Selzer and Fiske .................................................... 64
`4.
`Independent Claim 2 ................................................................ 66
`a.
`Claim 2 [preamble] – Unified Shader ........................... 66
`b.
`Claim 2[a] – General Purpose Register Block .............. 68
`c.
`Claim 2[b] and [d] – Processor Unit .............................. 70
`
`4.
`5.
`6.
`7.
`8.
`
`F.
`
`iii
`
`

`

`TABLE OF CONTENTS
`(continued)
`
`Page
`
`5.
`
`6.
`
`7.
`
`Claim 2[c] - Sequencer .................................................. 71
`d.
`Independent Claim 3 ................................................................ 72
`a.
`Claim 3 [preamble] – Unified Shader ........................... 72
`b.
`Claim 3[a] and [c] – Processor Unit .............................. 73
`c.
`Claim 3[b] – Shared Resources ..................................... 74
`Independent Claim 4 ................................................................ 74
`a.
`Claim 4 [preamble] – Unified Shader ........................... 74
`b.
`Claim 4[a] and [c] – Processor Unit .............................. 75
`c.
`Claim 4[b] – Shared Resources ..................................... 75
`Independent Claim 5 ................................................................ 75
`a.
`Claim 5 [preamble] – Unified Shader ........................... 75
`b.
`Claim 5[a] – Processor Unit .......................................... 75
`c.
`Claim 5[b] – Sequencer ................................................. 75
`Dependent Claim 6................................................................... 75
`8.
`Dependent Claims 7 and 10 ..................................................... 75
`9.
`10. Dependent Claim 8................................................................... 77
`11. Dependent Claim 9................................................................... 78
`12.
`Independent Claim 11 .............................................................. 78
`a.
`Claim 11 [preamble] – Unified Shader ......................... 78
`b.
`Claim 11[a] and [c] – Processor Unit ............................ 78
`c.
`Claim 11[b] – Instruction Store ..................................... 78
`Independent Claim 1 ................................................................ 78
`13.
`VIII. CONCLUSION ............................................................................................. 79
`
`iv
`
`

`

`TABLE OF AUTHORITIES
`
` Page(s)
`
`Cases
`Advanced Micro Devices, Inc. et al. v. TCL Industries Holdings Co.,
`LTD., et al.,
`Case No. 2:22-cv-00134 ....................................................................................... 2
`ATI Technologies ULC v. Iancu,
`920 F.3d 1362 (Fed. Cir. 2019) .......................................................................... 16
`Certain Graphics Systems, Components Thereof, and Consumer
`Products Containing Same,
`Inv. No. 337-TA-1044 .......................................................................................... 2
`Certain Graphics Systems, Components Thereof, and Digital
`Televisions Containing The Same,
`Inv. No. 337-TA-1318 .......................................................................................... 2
`Unified Patents Inc. v. Bradium Technologies LLC,
`IPR2018-00952, Institution Decision at 15-24 (December 20,
`2018) ..................................................................................................................... 5
`Statutes
`35 U.S.C. § 102 ...................................................................................................... 6, 7
`35 U.S.C. §103 ....................................................................................................... 1, 7
`35 U.S.C. §§ 311-319................................................................................................. 1
`35 U.S.C. § 315 .......................................................................................................... 4
`Other Authorities
`37 C.F.R. §§ 42.1-.80 & 42.100-.123 et seq. ............................................................. 1
`37 C.F.R. § 42.8 ..................................................................................................... 1, 3
`37 C.F.R. §42.103 ...................................................................................................... 4
`37 C.F.R. §42.10 ................................................................................................ 4-6, 8
`
`v
`
`

`

`37 CFR 1.131 ........................................................................................................... 15
`37 CFR LSD ee eescssecssecsseesseeeseeeseesseessseseseseseecsaecaesssesssesssesessassseeeseseseeeseeenees 15
`
`vi
`vi
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`PETITIONER’S EXHIBIT LIST
`
`1009
`
`1010
`
`1008
`
`Exhibit # Reference Name
`1001
`U.S. Patent 8,760,454 to Morein et al.
`1002
`Prosecution History of U.S. Patent 8,760,454
`1003
`Declaration of Hanspeter Pfister, Ph.D.
`1004
`Curriculum Vitae of Hanspeter Pfister, Ph.D.
`1005
`U.S. Patent 7,038,685 to Lindholm et al. (“Lindholm ’685”)
`1006
`U.S. Patent No. 7,015,913 to Lindholm et al. (“Lindholm ’913”)
`1007
`John Amanatides and Edward Szurkowski, A Simple, Flexible,
`Parallel Graphics Architecture, In Proceedings of Graphics
`Interface at 155-160 (Canadian Information Processing Society
`1993) published in Proc. Graphics Interface ’93 in May 1993
`(“Amanatides”)
`Les Kohn and Neal Margulis, Introducing the Intel i860 64-bit
`Microprocessor, IEEE, Volume 9, Issue 4, pages 15-30, August
`1989 (“Kohn”)
`Harald Selzer, Dynamic Load Balancing within a High
`Performance Graphics System, In Proceedings of Rendering,
`Visualization and Rasterization Hardware (Eurographics' 91
`Workshop) at 37-53 (Springer-Verlag 1993) published in 1993
`(“Selzer”) [Library of Congress]
`Stuart Fiske and William J. Dally, Thread prioritization: A Thread
`Scheduling Mechanism for Multiple-Context Parallel Processors,
`In Proceedings of First Symposium on High-Performance
`Computer Architecture, 1995 at 210-221 (IEEE 1995) published in
`1995 (“Fiske”)
`IEEE Xplore bibliography page for Fiske, Thread prioritization: A
`Thread Scheduling Mechanism for Multiple-Context Parallel
`Processors, visited on May 11, 2023
`Declaration of Gordon MacPherson re Authentication of Fiske
`(May 11, 2023)
`IEEE Xplore bibliography page for Kohn, Introducing the Intel
`i860 64-bit Microprocessor visited on May 11, 2023
`
`1011
`
`1012
`
`1013
`
`vii
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`
`1014
`
`1015
`
`1016
`
`Declaration of Gordon MacPherson re Authentication of Kohn
`(May 11, 2023)
`Initial Determination on Violation of Section 337, Certain
`Consumer Electronics and Display Devices With Graphics
`Processing and Graphics Processing Units Therein, 337-TA-932
`(October 9, 2015)
`Harald Selzer, Dynamic Load Balancing within a High
`Performance Graphics System, In Proceedings of Rendering,
`Visualization and Rasterization Hardware (Eurographics' 91
`Workshop) at 37-53 (Springer-Verlag 1993) published in 1993
`(“Selzer”) [University of California, Berkeley, Library]
`
`viii
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`I.
`INTRODUCTION
`
`Pursuant to 35 U.S.C. §§ 311-319 and 37 C.F.R. §§ 42.1-.80 & 42.100-.123
`
`et seq., Petitioner respectfully requests inter partes review of claims 1-11 of U.S.
`
`Patent No. 8,760,454. The challenged claims are unpatentable under 35 U.S.C.
`
`§103 over the prior art publications identified and applied in this Petition.
`
`ATI Technologies ULC (“ATI”) did not invent the first “unified shader,”
`
`which the ’454 patent discloses as a graphics processor that uses a single processor
`
`unit to perform both vertex and pixel operations. Nor did ATI invent the first
`
`unified shader that could be flexibly allocated to pixels or vertices based on
`
`workload. All of this was accomplished years earlier by Lindholm, Amanatides,
`
`Intel, Selzer, and others.
`
`II. MANDATORY NOTICES
`
`Pursuant to 37 C.F.R. § 42.8, Petitioner provides the following mandatory
`
`disclosures:
`
`A.
`
`Real Parties-In-Interest
`
`Petitioner Realtek Semiconductor Corp. (“Realtek”) is the real party-in-
`
`interest.
`
`B.
`
`Related Matters
`
`Pursuant to 37 C.F.R. § 42.8(b)(2), Petitioner submits that the ‘454 Patent is
`
`the subject of a patent infringement lawsuit brought by the Patent Owner, ATI
`
`1
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Technologies ULC, against Petitioner in the following cases that may affect or be
`
`affected by a decision in this proceeding:
`
` Advanced Micro Devices, Inc. et al. v. TCL Industries Holdings Co.,
`
`LTD., et al., Case No. 2:22-cv-00134, now pending in the Eastern
`
`District of Texas. This action is stayed “until final resolution of
`
`Investigation No. 337-TA-1318.” (Dkt. 65)
`
`The ’454 Patent was previously asserted in Certain Graphics Systems, Components
`
`Thereof, and Digital Televisions Containing The Same, Inv. No. 337-TA-1318,
`
`pending in the U.S. International Trade Commission, but the asserted patent claims
`
`were terminated by Order No. 10 on July 14, 2022, upon motion of ATI. The
`
`target date for completion of the investigation is November 7, 2023.
`
`The ’454 Patent was previously the subject of:
`
` IPR2017-01225, which was terminated prior to an institution decision;
`
` Certain Graphics Systems, Components Thereof, and Consumer Products
`
`Containing Same, Inv. No. 337-TA-1044, but the investigation as to the
`
`asserted claims was terminated prior to a determination on the merits;
`
`and
`
` Case No. 1:17-cv-00065 in the District of Delaware, which was
`
`dismissed October 18, 2017.
`
`2
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`C.
`Lead and Backup Counsel
`
`Petitioner designates the following counsel:
`
`LEAD COUNSEL
`Jeffrey Johnson
`Reg. No. 53,078
`ORRICK, HERRINGTON & SUTCLIFFE,
`LLP
`609 Main Street, 40th Floor
`Houston, TX 77002-3106
`Telephone: (713) 658-6400
`Facsimile: (713) 658-6401
`Email: 3J6PTABDocket@orrick.com;
`Realtek-AMD_OHS@orrick.com
`
`BACK-UP COUNSEL
`Christopher J. Higgins
`Reg. No. 66,422
`ORRICK, HERRINGTON & SUTCLIFFE,
`LLP
`1152 15th Street, N.W.
`Washington, DC 20005-1706
`Telephone: (202) 339-8400
`Facsimile: (202) 339-8500
`Email:
`OCHPTABDocket@orrick.com;
`Realtek-AMD_OHS@orrick.com
`Steve Baik
`Reg. No. 42,281
`WHITE HAT LEGAL
`P. O. Box 382
`San Jose, CA 95002
`Telephone: (650) 618-5282
`Email: sbaik@whitehat.legal
`
`D.
`
`Service Information
`
`Pursuant to 37 C.F.R. §42.8(b)(4), Petitioner identifies the following service
`
`information: Petitioner consents to service by electronic mail at the following
`
`addresses: 3J6PTABDocket@orrick.com; OCHPTABDocket@orrick.com;
`
`sbaik@whitehat.legal; and Realtek-AMD_OHS@orrick.com. Petitioner’s Power
`
`of Attorney is attached.
`
`3
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`III. PAYMENT OF FEES
`
`Pursuant to 37 C.F.R. §42.103, $41,500 is being paid at the time of filing
`
`this petition, charged to Orrick, Herrington, & Sutcliffe LLP: Deposit Account 15-
`
`0665. Should any further fees be required by the present Petition, the Patent Trial
`
`and Appeal Board (“the Board”) is hereby authorized to charge the above
`
`referenced Deposit Account.
`
`IV.
`
`STANDING
`
`Pursuant to 37 C.F.R. §42.104(a), Petitioner certifies that (1) the ‘454 Patent
`
`is available for inter partes review; and (2) Petitioner is not barred or estopped
`
`from requesting inter partes review of claims 1-11 of the ‘454 Patent on the
`
`grounds identified in this Petition. In particular, this Petition is timely filed under
`
`35 U.S.C. § 315(b).
`
`V.
`
`THE BOARD SHOULD NOT EXERCISE ITS DISCRETION TO
`DENY INSTITUTION
`
`Although the ’454 Patent is asserted in a case in the Eastern District of
`
`Texas, there is no basis to deny institution under §314. This proceeding is still in
`
`its very early stages. Petitioner has not filed an answer. The action has been
`
`stayed at the request of the Patent Owner “until final resolution of Investigation
`
`No. 337-TA-1318.” Accordingly, there is no effective trial date and none will be
`
`set until sometime after November 7, 2023, the target date for completion of the
`
`4
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Investigation, at the earliest. Petitioner diligently filed this Petition during the stay
`
`and before filing an answer in the district court. Thus, the first, second, third, fifth
`
`and sixth Fintiv factors strongly favor institution.
`
`The Board also should exercise its discretion to institute under §325(d)
`
`because the Examiner neither cited nor discussed the references that form the second
`
`and third grounds for unpatentability. Moreover, Patent Owner did not address the
`
`Examiner’s rejection based on the ’685 patent on its merits and was not able to
`
`overcome the rejection on its merits. And as discussed below, ATI’s evidence of
`
`prior invention was found inadequate in the 337-TA-923 ITC Investigation.
`
`Finally, the Board should not exercise its discretion to deny institution under
`
`§314(a) based on IPR2017-01225. The prior petition was filed April 3, 2017, by a
`
`different petitioner (LG), about six years before Petitioner was accused of infringing
`
`the ’454 patent and is alleged to have notice of ATI’s claims of infringement. See
`
`Unified Patents Inc. v. Bradium Technologies LLC, IPR2018-00952, Institution
`
`Decision at 15-24 (December 20, 2018).
`
`VI. REQUEST FOR INTER PARTES REVIEW OF CLAIMS 1-11 OF
`THE ‘454 PATENT
`
`37 C.F.R. § 42.104(b)(1): Claims for Which IPR is Requested
`A.
`Petitioner requests that the Board finds unpatentable Claims 1-11 of the ‘454
`
`Patent (the “Challenged Claims”). Such relief is justified as the alleged invention
`
`5
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`of the ‘454 Patent was described by others prior to the effective filing date of the
`
`‘454 Patent.
`
`B.
`
`37 C.F.R. § 42.104(b)(2): Identification of Prior Art and Asserted
`Grounds for Which IPR is Requested
`The prior art relied upon in this Petition is:
`
`U.S. Patent No. 7,038,685 (“’685 patent”) was filed on June 30, 2003 and
`
`issued on May 2, 2006. EX1005. It is therefore prior art to the ’454 patent under
`
`at least 35 U.S.C. § 102(e).
`
`U.S. Patent No. 7,015,913 (“’913 patent”) was filed on June 27, 2003 and
`
`issued on March 21, 2006. EX1006. It is therefore prior art to the ’454 patent
`
`under at least 35 U.S.C. § 102(e).
`
`John Amanatides and Edward Szurkowski, A Simple, Flexible, Parallel
`
`Graphics Architecture, In Proceedings of Graphics Interface at 155-160 (Canadian
`
`Information Processing Society 1993) (“Amanatides”) was published in Proc.
`
`Graphics Interface ’93 in May 1993. EX1007. It is therefore prior art to the ’454
`
`Patent under at least 35 U.S.C. § 102(a) and (b).
`
`Les Kohn and Neal Margulis, Introducing the Intel i860 64-bit
`
`Microprocessor, IEEE, Volume 9, Issue 4, pages 15-30, August 1989 (“Kohn”).
`
`EX1008, EX1013, EX1014. It is therefore prior art to the ’454 patent under at
`
`least 35 U.S.C. § 102(a) and (b).
`
`6
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Harald Selzer, Dynamic Load Balancing within a High Performance
`
`Graphics System, In Proceedings of Rendering, Visualization and Rasterization
`
`Hardware (Eurographics' 91 Workshop) at 37-53 (Springer-Verlag 1993)
`
`(“Selzer”) was published in 1993. EX1009, EX1016. It is therefore prior art to
`
`the ’454 patent under at least 35 U.S.C. § 102(a) and (b).
`
`Stuart Fiske and William J. Dally, Thread Prioritization: A Thread
`
`Scheduling Mechanism for Multiple-Context Parallel Processors, In Proceedings
`
`of First Symposium on High-Performance Computer Architecture, 1995 at 210-
`
`221 (IEEE 1995) (“Fiske”) was published in 1995. EX1010, EX1011, EX1012. It
`
`is therefore prior art to the ’454 patent under at least 35 U.S.C. § 102(a) and (b).
`
`The one-year time bar under pre-AIA 35 U.S.C. §102(b) is measured from
`
`the effective U.S. filing date of the ’454 Patent, which is no earlier than
`
`November 20, 2003.
`
`Inter partes review is requested on the following grounds:
`
`Ground 1: The Challenged Claims are rendered obvious under 35 U.S.C.
`
`§ 103(a) by the ’685 patent in combination with the ’913 patent.
`
`Ground 2: The Challenged Claims are rendered obvious under 35 U.S.C.
`
`§ 103(a) by Amanatides in combination with Kohn.
`
`Ground 3: The Challenged Claims are rendered obvious under 35 U.S.C.
`
`§ 103(a) by Selzer in combination with Fiske.
`
`7
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`C.
`Level of Ordinary Skill in the Art
`
`A person of ordinary skill in the field, at the time the ’454 patent was
`
`effectively filed, would have had at least a four-year degree in electrical
`
`engineering, computer engineering, computer science, or a related field and two
`
`years relevant experience in the graphics processing field including developing,
`
`designing, or programming hardware for graphics processing units.
`
`D.
`
`37 C.F.R. § 42.104(b)(5): Evidence Supporting Challenge
`
`The Declaration of Dr. Hanspeter Pfister, Ph.D. (EX1003) and other
`
`supporting evidence in the Exhibit List are filed herewith. Dr. Pfister’s
`
`background and qualifications, and the information provided to him, are discussed
`
`in EX1004.
`
`VII. THERE EXISTS A REASONABLE LIKELIHOOD THAT THE
`CHALLENGED CLAIMS ARE UNPATENTABLE
`A.
`Technology Background
`
`The ’454 patent is directed to the typical functions of a graphics processor,
`
`which is used to generate complex shapes and structures to be displayed on a
`
`screen. EX1001 at 1:38-48; EX1003 ¶37. A graphics processor converts a 3D
`
`object or scene (comprised of points in 3D space called “vertices”) into a 2D image
`
`to be displayed on a computer screen (comprised of “pixels”). Id. Generally, 3D
`
`graphics processing starts with creating a mathematical model of each object. Id.
`
`8
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`The model is then processed through a series of steps, referred to as a “graphics
`
`processing pipeline,” that renders the scene as a 2D image on a display:
`
`In most cases, 3D objects are conceptualized as a series of triangles (called
`
`“primitives”) that cover the surface of an object, such as a teapot:
`
`EX1003 ¶38.
`
`9
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Each point of the primitive is called a “vertex,” and each vertex has certain
`
`properties, which are represented as data. Id. For example, a vertex includes not
`
`just its location, but may also include other information, such as the color of the
`
`object and its material properties (e.g., whether it is reflective). EX1003 ¶¶39-40.
`
`A vertex processor (the “front end” of the graphics pipeline) transforms these
`
`vertices from 3D space into 2D space and determines how lighting and other
`
`conditions in the 3D scene impact the color of the vertices. Id.
`
`After the vertex processing, a step called rasterization determines what
`
`pixels on the 2D screen are covered by each
`
`primitive (at right). Id. ¶¶41-45. At least one
`
`“fragment” is generated for each pixel on the screen
`
`(as a result, the terms “fragment” and “pixel” are
`
`sometimes used interchangeably). Id. The initial values of each pixel (such as its
`
`color) are calculated using the vertex data and interpolation. Id. A pixel processor
`
`(the “back end” of the graphics pipeline) then performs additional operations on
`
`the pixels, which includes determining the final color of each pixel. Id. All of this
`
`information is gathered together for the final image to be displayed on a screen. Id.
`
`10
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`
`Conventionally, separate dedicated vertex processors and pixel processors
`
`(sometimes referred to as “shaders”) were used to process vertex data and pixel
`
`data. EX1003 ¶46. Before the ’454 patent was filed, inventors such as Erik
`
`Lindholm, John Amanatides, and Harald Selzer recognized that the conventional
`
`approach of using separate and dedicated vertex and pixel processors created
`
`inefficiencies. Separate, fixed function vertex and pixel processors could not
`
`adapt to the varying needs of each image, some of which have a lot of vertex data
`
`to process while others have more pixel data. Id. at ¶¶47-48. This was inefficient,
`
`as either vertex processors or pixel processors were often idle while the other type
`
`of processor became backlogged. Id.
`
`What was needed was a way to process both vertex and pixel data through a
`
`single piece of hardware that could process either data type and then prioritize or
`
`balance a common set of resources between the two sample types. The inventor of
`
`the ’685 and ʼ913 Patents, Erik Lindholm, solved the problem. Mr. Lindholm
`
`invented a multithreaded processor unit that could process both vertex and pixel
`
`11
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`graphics data (sometimes called a “unified shader”) depending on available
`
`resources and priorities. Id. ¶48. The invention also allowed the multithreaded
`
`processor unit to dynamically balance or control the number of threads processing
`
`vertices or pixels during operation, depending on the available resources.
`
`Amanatides and Selzer similarly invented unified shaders that could process
`
`vertices and pixels in the same processing units based on workload, available
`
`resources, and processing priorities.
`
`12
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`B.
`Description of the Alleged Invention of the ’454 Patent
`
`Similar to the Lindholm patents, the ’454 patent notes that one drawback to
`
`separate vertex and pixel shader hardware was the increased footprint required on
`
`the processor. EX1003 ¶¶49-50. Thus, the ’454 patent uses the same piece of
`
`hardware, which it also terms a “unified shader,” to run both vertex shader
`
`programs and pixel shader programs. EX1001 at 2:58-3:3. The patent defines
`
`“unified shader” as a shader that is “configured to perform both vertex and pixel
`
`operations.” Id. at 3:10-12. Figure 4 of the ’454 patent is illustrative of the unified
`
`shader.
`
`13
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Within the unified shader is a general-purpose register block for storing
`
`vertex and pixel data to be processed. Id. at 4:29-39. The specification discloses
`
`that there was nothing inventive about the hardware comprising the “unified
`
`shader” itself, which is described as being “a processor (e.g., CPU) 96.” Id. at
`
`4:30-34. In one embodiment, the processor 96 is “logically partitioned into two
`
`sections,” one of which is configured to perform 32-bit floating point arithmetic
`
`operations and another of which is configured to perform scalar operations (e.g.,
`
`log, exponent, reciprocal square root). Id. It was known to a POSITA that vertex
`
`processing typically required floating point operations and pixel processing
`
`required scalar operations. EX1003 ¶¶54-55. The processor unit within the
`
`unified shader can process vertex data concurrently with pixel data and alternate
`
`between the two types of operations. EX1001 at 5:32-36; 6:36-41. Within the
`
`unified shader, the unit that maintains instructions for performing vertex and pixel
`
`operations is called the sequencer. Id. at 4:52-5:5. An arbiter circuit is used to
`
`determine which of a plurality of inputs, such as vertices or pixels, are selected for
`
`processing by the unified shader. Id. at 4:13-28. This determination might be
`
`made by giving priority to vertex processing based on whether the general-purpose
`
`registers have enough available space to store incoming vertex data, and if not,
`
`pixel operations are continued. Id. at 5:41-52.
`
`14
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`C.
`Prosecution History of the ’454 Patent and Alleged Priority of
`Invention
`The application that led to the ’454 patent was filed on May 17, 2011, and
`
`issued on June 14, 2014. The ’454 patent claims priority to an application filed
`
`November 20, 2003. During prosecution, the Examiner rejected the claims of
`
`the ’454 patent three times as anticipated by the Lindholm ’685 patent. EX1002.
`
`ATI never addressed the Lindholm rejection on its merits and was not able to
`
`overcome the rejection on its merits. Id. Instead, ATI filed an inventor
`
`declaration under 37 CFR 1.131, which the Examiner found sufficient in an Office
`
`Action dated August 8, 2013, without substantive analysis.
`
`However, ATI’s evidence of prior invention was subsequently found
`
`insufficient to antedate the ’685 patent after a full adjudication on the merits. In
`
`Certain Consumer Electronics and Display Devices with Graphics Processing and
`
`Graphics Processing Units Therein, Judge Pender found that ATI was unable to
`
`prove conception prior to the effective filing date of the ’685 patent, June 30, 2003,
`
`or actual reduction to practice. Inv. No. 337-TA-932, Final Initial Determination
`
`(October 9, 2015) (EX1015). For example, Judge Pender rejected ATI’s evidence
`
`of earlier conception of claim limitations directed to the processing of graphics
`
`data by assigning a priority to processing vertices or pixels, as well as the
`
`allocation of threads to vertices and pixels based on such a priority. Id. at 140-145.
`
`15
`
`

`

`Patent No. 8,760,454
`Petition for Inter Partes Review
`Judge Pender also found that ATI’s evidence could not support an earlier invention
`
`date for load balancing, i.e., determining whether to process pixel data or vertex
`
`data based on wo