Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 1 of 14 PageID# 14323
`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 1 of 14 Page|D# 14323
`
`
`EXHIBIT
`C
`
`EXHIBIT
`
`
`
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 2 of 14 PageID# 14324
`Case 3:14-CV-OO757-REP-DJN Document 87-11 IIWIIIMIIWHIIIHIIIIMHWIIIHIIIIIIIIIIIIIIHIIIIM “324
`
`US006532013B1
`
`(12) United States Patent
`US 6,532,013 B1
`(10) Patent N0.:
`(45) Date of Patent:
`Papakipos et al.
`Mar. 11, 2003
`
`(54)
`
`(75)
`
`SYSTEM, METHOD AND ARTICLE OF
`MANUFACTURE FOR PIXEL SHADERS FOR
`PROGRAMMABLE SHADING
`
`Inventors: Matthew N. Papakipos, Palo Alto, CA
`(US); David B. Kirk, Santa Clara, CA
`(US); Liang Peng, Sunnyvale, CA
`(US); Harold Robert Feldman Zatz,
`Mountain View, CA (US)
`
`(73)
`
`Assignee: NVIDIA Corporation, Santa Clara, CA
`(US)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 232 days.
`
`(21)
`
`(22)
`
`(51)
`(52)
`(58)
`
`(56)
`
`Appl. No.: 09/585,809
`
`Filed:
`
`May 31, 2000
`
`Int. Cl.7 ................................................ G06T 17/00
`............ 345/426
`
`Field of Search ................................. 345/418, 419,
`345/420, 426, 427, 428
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`.................... 365/49
`2/1991 Duluk, Jr.
`4,996,666 A
`382/236
`7/1996 Chen et al.
`5,535,288 A
`
`.........
`395/119
`11/1996 Duluk, Jr.
`5,572,634 A
`
`11/1996 Duluk, Jr. et al.
`395/121
`5,574,835 A
`
`.......... 395/122
`1/1997 Duluk, Jr. etal.
`5,596,686 A
`9/1997 Duluk, Jr. ........... 395/800.22
`5,669,010 A
`11/1999 Duluk, Jr. ............. 345/441
`5,977,987 A
`
`.
`6,005,582 A * 12/1999 Gabriel et al.
`345/430
`
`2/2000 Jenkins .............
`6,028,608 A *
`345/433
`6,229,553 B1
`5/2001 Duluk, Jr. et al.
`345/506
`6,268,875 B1
`7/2001 Duluk, Jr. et al.
`345/506
`6,285,378 B1
`9/2001 Duluk, Jr.
`............
`345/441
`6,288,730 B1
`9/2001 Duluk, Jr. et al.
`.......... 345/552
`
`
`
`FOREIGN PATENT DOCUMENTS
`
`W0
`W0
`W0
`
`93/23816
`97/05575
`97/05576
`
`11/1993
`2/1997
`2/1997
`
`WO
`WO
`W0
`W0
`W0
`W0
`W0
`WO
`WO
`W0
`
`00/10372
`00/11562
`00/11602
`00/11603
`00/11604
`00/11605
`00/11607
`00/11613
`00/11614
`00/19377
`
`3/2000
`3/2000
`3/2000
`3/2000
`3/2000
`3/2000
`3/2000
`3/2000
`3/2000
`4/2000
`
`OTHER PUBLICATIONS
`
`Lance Williams; “Pyramidal Parametrics”; Jul. 1983; Com-
`puter Graphics, vol. 17, No. 3 Computer Graphics Labora-
`tory, New York Institute of Technology, Old Westbury, NY;
`pp. 1—11.
`James F. Blinn et al., “Texture and Reflection in Computer
`Generated Images”; Oct. 1976; Graphics and Image Pro-
`cessing; Association for Computing Machinery, Inc. pp.
`542—547.
`
`Ned Greene, New York Institute of Technology; “Environ-
`ment Mapping and Other Applications of World Projec-
`tions”; Nov. 1986; The Institute of Electrical and Electronics
`Engineers Inc. Computer Society; pp. 21—29.
`
`* cited by examiner
`
`Primary Examiner—Cliff N. V0
`(74) Attorney, Agent, or Firm—Silicon Valley IP Group,
`LLC; Kevin J. Zilka
`
`(57)
`
`ABSTRACT
`
`A system, method and article of manufacture are provided
`for interweaving shading calculations and texture retrieval
`operations during texture sampling in a graphics pipeline.
`First, a shading calculation is performed in order to generate
`output. Next, texture information is retrieved, and another
`shading calculation is performed using the texture informa-
`tion in order to generate additional output. Texture informa-
`tion may be retrieved and shading calculations may then be
`repeated as desired. Thereafter, the generated output may be
`combined. As such,
`the repeated texture information
`retrieval and shading calculations may be carried out in an
`iterative, programmable manner.
`
`59 Claims, 5 Drawing Sheets
`
`402
`/
`
`‘
`
`SET-UP
`
`PLANE
`EQUATIONS
`
`409
`
`RASTERIZER
`
`404
`
`406
`
`EDGE
`VALUES
`TEXTURE
`COORDINATES
`
`‘
`VERTEX +
`smog:
`PIXEL DATA
`
`
`
`FINAL
`TEXTURE
`COLOR
`
`‘
`
`mam
`
`400
`
`408
`
`407
`
`teams
`
` we
`LOOPBACK
`
`
`
`FILTERED
`
`TEXTURE
`COLOR
`
`/Ho
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 3 of 14 PageID# 14325
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 3 of 14 Page|D# 14325
`
`US. Patent
`
`Mar. 11,2003
`
`Sheet 1 0f5
`
`US 6,532,013 B1
`
`
`
`TRANSFORM ENGINE
`
`
`
`100
`
`102
`
`SET-UP /
`
`
`
`1 04
`
`RASTERIZER /
`
`
`
`TEXTURE MATH
`
`106
`
`
`l
`
`
`LEVEL OF DETAIL (LOD)
`CALCULATOR
`
`108
`
`
`
`1 1O
`
`TEXTURE FETCH /
`
`
`
`
`
`1 12
`i
`TEXTURE FILTER /
`l
`J
` FIGURE 1
`
`TEXTURE COMBINATION ENGINE //
`
`114
`
`(PRIOR ART)
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 4 of 14 PageID# 14326
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 4 of 14 Page|D# 14326
`
`US. Patent
`
`Mar. 11, 2003
`
`Sheet 2 0f 5
`
`US 6,532,013 B1
`
`0
`
`'I
`
`01
`
`202
`
`(0.0)
`
`“’0’
`
`HGUREZ
`
`(PRIOR ART)
`
`200
`
`2
`
`(1,1)
`
`44/302
`
`(0.5, 0.5)
`
`(0,0)
`
`(110)
`
`FIGURE 3
`
`(PRIOR ART)
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 5 of 14 PageID# 14327
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 5 of 14 Page|D# 14327
`
`US. Patent
`
`Mar. 11,2003
`
`Sheet 3 0f5
`
`US 6,532,013 B1
`
`
`
`402
`
`SET-UP /
`
`
`
`
`
`
`
`PLANE
`EQUATIONS
`
`400
`
`404
`
`409
`
`406
`
`408
`
`RASTERBER
`
`EDGE
`
`
`
`VALUES
`
`
`
`
`
`
`
`
`COMBINER
`
`407
`
`‘/
`
`410
`
`FIGURE 4
`
`502
`
`500
`
`e0
`
`e1
`
`e2
`
`04
`
`506
`
`FIGURE 5
`
`SHADER
`
`
`
`TEXTURE
`
`COORDINATES
`
`
`—>
`
`VERTEX +
`LOOPBACK
`TEXTURE
`PIXEL DATA
`FIFO
`
`<——
`
`
`FmTERED
`TEXTURE
`COLOR
`
`HNAL
`TEXTURE
`COLOR
`
`
`
`
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 6 of 14 PageID# 14328
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 6 of 14 Page|D# 14328
`
`US. Patent
`
`Mar. 11,2003
`
`Sheet 4 of 5
`
`US 6,532,013 B1
`
`602
`
`604
`
`FIGURE 6
`
`605
`
`\ YES
`
`C
`“3:333:31: H
`
`I
`—><‘ TEEEEE
`
`
`
`TEXTURE ADDRESS CALCULATION
`(SHADING)
`
`
`
`
`
`TEXTURE ADDRESS CALCULATION
`(SHADING)
`
`
`
` A I DITIONA
`
`SHADING
`
`?
`NO
`
`YES
`
`
`
`600
`
`
`
`NO
`
`YES
`
`TEXTURE
`FETCH
`
`NO
`
`TEXTURE FETCH
`OPERATION
`
`600
`
`TEXTURE ADDRESS CALCULATION
`(S HADING)
`
`
`
`
`YES
`
`I
`
`FETCH
`< TEXTURE
`
`NO
`
`
`
`OPERATION
`TEXTURE FETCH
`
` ~ IDITIONA
`SHADING
`?
` NO
`
`
`
`YES
`
`TEXTURE ADDRESS CALCULATION
`
`(SHADING)
`
`YES
`
`TEXTURE
`FETCH
`
`TEXTURE FETCH
`OPERATION
`
`
`
`
`~ IDITIONA
`SHADING
`7
`
`
`NO
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 7 of 14 PageID# 14329
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 7 of 14 Page|D# 14329
`
`US. Patent
`
`Mar. 11,2003
`
`Sheet 5 0f5
`
`US 6,532,013 B1
`
`607
`
`608
`
`
`
`Figure 6A
`
`FIGURE 7
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 8 of 14 PageID# 14330
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 8 of 14 Page|D# 14330
`
`US 6,532,013 B1
`
`1
`
`SYSTEM, METHOD AND ARTICLE OF
`MANUFACTURE FOR PIXEL SHADERS FOR
`PROGRAMMABLE SHADING
`
`RELATED APPLICATION(S)
`
`This application is related to a co-pending application
`entitled “GRAPHICS PIPELINE INCLUDING COM-
`
`BINER STAGES” filed Mar. 22, 1999 naming David B.
`Kirk, Matthew Papakipos, Shaun Ho, Walter Donovan, and
`Curtis Priem as inventors, and issued as US. Pat. No.
`6,333,744, and which is incorporated herein by reference in
`its entirety.
`1. Field of the Invention
`
`The present invention relates to computer graphics, and
`more particularly to texture sampling in a computer graphics
`processing pipeline.
`2. Background of the Invention
`Recent advances in computer performance have enabled
`graphic systems to provide more realistic graphical images
`using personal computers and home video game computers.
`In such graphic systems, some procedure must be imple-
`mented to “render” or draw graphic primitives to the screen
`of the system. A “graphic primitive” is a basic component of
`a graphic picture, such as a polygon, e.g., a triangle, or a
`vector. All graphic pictures are formed with combinations of
`these graphic primitives. Many procedures may be utilized
`to perform graphic primitive rendering.
`Early graphic systems displayed images representing
`objects having extremely smooth surfaces. That is, textures,
`bumps, scratches, or other surface features were not mod-
`eled. In order to improve the quality of the image, texture
`mapping was developed to model the complexity of real
`world surface images. In general, texture mapping is the
`mapping of an image or a function onto a surface in three
`dimensions. Texture mapping is a relatively efficient tech-
`nique for creating the appearance of a complex image
`without
`the tedium and the high computational cost of
`rendering the actual three dimensional detail that might be
`found on a surface of an object.
`Prior Art FIG. 1 illustrates a graphics pipeline with which
`texture mapping may be performed. As shown, included is
`a transform engine 100, a set-up module 102, a rasterizer
`104, a texture math module 106, a level of detail (LOD)
`calculator 108, a texture fetch module 110, a texture filter
`112, and a texture combination engine 114. It should be
`noted that the transform engine 100 and set-up module 102
`need not necessarily be required in the graphics pipeline of
`a graphics integrated circuit.
`During operation, the transform engine 100 may be used
`to perform scaling, rotation, and projection of a set of three
`dimensional vertices from their local or model coordinates
`
`to the two dimensional window that will be used to display
`the rendered object. The setup module 102 utilizes the world
`space coordinates provided for each triangle to determine
`the two dimensional coordinates at which those vertices are
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`to appear on the two dimensional window. Prior Art FIG. 2
`illustrates the coordinates 200 of the vertices 201 which
`
`60
`
`define a triangle 202. If the vertices 201 of the triangle 202
`are known in screen space, the pixel positions vary along
`scan lines within the triangle 202 in screen space and may
`be determined.
`
`The setup module 102 and the rasterizer module 104
`together use the three dimensional world coordinates to
`determine the position of each pixel contained inside each of
`
`65
`
`2
`the triangles. Prior Art FIG. 3 illustrates a plurality of pixels
`300 identified within the triangle 202 in such a manner. The
`color values of the pixels in the triangle 202 vary from pixel
`to pixel in world space. During use, the setup module 102
`and the rasterizer module 104 generate interpolated colors,
`depth and texture coordinates.
`The setup module 102 and the rasterizer module 104 then
`feed the pixel texture coordinates to the texture math module
`106 to determine the appropriate texture map colors. In
`particular, texture coordinates are generated that reference a
`texture map using texture coordinate interpolation which is
`commonly known to those of ordinary skill in the art. This
`is done for each of the pixels 300 identified in the triangle
`202. Prior Art FIG. 3 illustrates texture coordinates 302 for
`
`the pixels 300 identified within the triangle 202.
`Next, a LOD calculation is performed by the LOD cal-
`culator 108. Occasionally during rendering, one texel, or
`texture element, will correspond directly to a single pixel
`that is displayed on a monitor. In this situation the level of
`detail (LOD) is defined to be equal to zero (0) and the texel
`is neither magnified nor minified. However, the displayed
`image can be a magnified or minified representation of the
`texture map. If the texture map is magnified, multiple pixels
`will represent a single texel. A magnified texture map
`corresponds to a negative LOD value. If the texture map is
`minified, a single pixel represents multiple texels. Aminified
`texture map corresponds to a positive LOD value.
`In
`general, the LOD value corresponds to the number of texels
`in the texture map ‘covered’ by a single pixel.
`The amount of detail stored in different LOD represen-
`tations may be appreciated by drawing an analogy to the
`detail perceived by an observer while observing a texture
`map. For example, very little detail may be perceived by an
`observer while watching an automobile from a distance. On
`the other hand, several details such as doors, windows,
`mirrors will be perceived if the observer is sufficiently close
`to the automobile. A finer level LOD will include such
`additional details, and a courser LOD will not.
`Once the appropriate level of detail of the texture map is
`selected based on the calculated LOD value,
`the texture
`coordinates generated by the texture math module 106 are
`used to fetch the appropriate texture map colors using the
`texture fetch module 110. These texture map colors are then
`filtered by the texture filter module 112. The combiner
`engine 114 combines together the various colors and tex-
`tures fetched by the texture fetch module 110 and filtered by
`the texture filter module 112.
`
`It is important to note that the pipeline described herein-
`above has a linear topology. These and other simplistic
`non-linear pipelines only enable one texture fetch and tex-
`ture calculation per rendering pass. This is a limited design
`that is static in nature. There is thus a need for a pipeline that
`allows for more dynamic texture fetches and shading
`calculations, and in particular, the ability for feeding filter
`results back to influence subsequent texture address calcu-
`lations.
`
`DISCLOSURE OF THE INVENTION
`
`A system, method and article of manufacture are provided
`for interweaving shading calculations and texture retrieval
`operations during texture sampling in a graphics pipeline.
`First, a shading calculation is performed in order to generate
`output,
`i.e. colors or
`texture coordinates. Next,
`texture
`information is retrieved, and another shading calculation is
`performed using the texture information in order to generate
`additional output. Texture information may be retrieved and
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 9 of 14 PageID# 14331
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 9 of 14 Page|D# 14331
`
`US 6,532,013 B1
`
`3
`shading calculations may then be repeated as desired.
`Thereafter, the generated output may be combined. As such,
`the repeated texture information retrieval and shading cal-
`culations may be carried out in an iterative, programmable
`manner.
`
`In one embodiment of the present invention, edge dis-
`tances of a primitive may be calculated, and at least one of
`the shading calculations involves the edge distances.
`Further, the shading calculation may include the calculation
`of a plurality of weighted coefficients from the edge dis-
`tances. As an option, such weighted coefficients may include
`barycentric weights which use parameter values of the
`primitive to perform parameter interpolation.
`the
`In another embodiment of the present
`invention,
`texture information may include filtered texture color infor-
`mation. As an option, the filtered texture value may be used
`as texture coordinates for use in retrieving further texture
`information when the texture information retrieval
`is
`
`repeated. Further, the repeated shading calculation may also
`use the output in order to generate additional output.
`In still another embodiment of the present invention, the
`output may include diffuse output colors, fog output values,
`specular output colors, depth output values, texture color
`output values, a level of detail (LOD) value, and/or a Z-slope
`value. As an option, the shading calculation may include the
`calculation of a level of detail (LOD) which may occur after
`the texture information is retrieved.
`
`In order to accomplish the foregoing, a graphics pipeline
`system may be provided which includes a shading module
`for performing the shading calculation in order to generate
`output. Coupled to the shading module is a texture look-up
`module for retrieving texture information. Further, a feed-
`back loop is coupled between an input and an output of the
`shading module for performing additional shading calcula-
`tions using the texture information from the texture look-up
`module. Also included is a combiner module coupled to the
`output of the shading module for combining the output
`generated by the shading module. In one aspect of the
`present embodiment, at
`least a pair of texture look-up
`modules is coupled to a pair of shading modules which
`together constitute at least four logical modules.
`These and other advantages of the present invention will
`become apparent upon reading the following detailed
`description and studying the various figures of the drawings.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The foregoing and other aspects and advantages are better
`understood from the following detailed description of a
`preferred embodiment of the invention with reference to the
`drawings, in which:
`Prior Art FIG. 1 illustrates a graphics pipeline with which
`texture mapping may be performed;
`Prior Art FIG. 2 illustrates the coordinates of the vertices
`
`which define a triangle;
`Prior Art FIG. 3 illustrates a plurality of pixels identified
`within the triangle of FIG. 2;
`FIG. 4 illustrates a hardware implementation for program-
`mable shading in accordance with one embodiment of the
`present invention;
`FIG. 5 illustrates a graphical representation of the edge
`distances generated by the rasterizer in accordance with one
`embodiment of the present invention;
`FIG. 6 illustrates a flow diagram depicting the manner in
`which the shading calculations are interweaved with the
`texture fetch operations in a plurality of iterations in accor-
`dance with one embodiment of the present invention;
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`FIG. 6A graphically illustrates the derivative values, (ux)
`and (uy), with which a level of detail calculation may be
`performed in accordance with one embodiment of the
`present invention; and
`FIG. 7 illustrates a point within a primitive that is capable
`of being defined by barycentric weights.
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`1—3 illustrate the prior art. FIG. 4 shows an
`FIGS.
`illustrative hardware implementation of the present inven-
`tion. As shown, included is a set-up module 402, a rasterizer
`404, a shader module 406, a texture fetch module 408, and
`a combiner 410 coupled to form a portion of a graphics
`processing pipeline 400. For reasons that will soon become
`apparent, a feedback loop 409 is coupled between an output
`of the shader module 406 to an input thereof. It should be
`noted that the set-up module 402, rasterizer 404, and com-
`biner 410 operate in a conventional manner as set forth
`during reference to FIG. 1. While the combiner 410 may be
`implemented in any desired manner, one exemplary imple-
`mentation is disclosed in a co-pending application entitled
`“GRAPHICS PIPELINE INCLUDING COMBINER
`
`STAGES” filed Mar. 22, 1999 naming David B. Kirk,
`Matthew Papakipos, Shaun Ho, Walter Donovan, and Curtis
`Priem as inventors, and issued as US. Pat. No. : 6,333,744,
`and which is incorporated herein by reference in its entirety.
`With continuing reference to FIG. 4, the various inputs
`and outputs are shown for each of the components. As
`shown, the rasterizer 404 generates edge distances which are
`used by the shader module 406 and texture fetch module
`408.
`
`Also shown in FIG. 4 is an optional feedback first-in
`first-out (FIFO) buffer. When the feedback loop 409 is not
`utilized,
`the temporary data calculated internally by the
`present invention may be dropped before being sent to the
`texture fetch module 408.
`
`As an option, however, the shader module 406 may be
`reused, and some of this data (like the barycentric
`coordinates) may be reused each time a particular group of
`pixels, or “quad,” goes through the shader module 406. If
`new colors are generated during one pass, these colors may
`continuously be associated with the quad on subsequent
`passes. Further, more than one triangle may be processed at
`a time while employing the feedback loop 409, since data
`from several triangles generally appears while waiting for
`the texture fetch module 408 to calculate an individual
`
`texture look-up.
`To address this, the loopback FIFO 407 may be utilized to
`hold barycentric weights, colors from previous passes, tri-
`angle information, and additional scheduling information to
`keep track of what the shader module 406 is supposed to do
`each pass. The FIFO 407 may include a plurality of bits that
`can be reconfigured to store whatever piece of data is
`appropriate. When the texture requests for a particular quad
`are sent to the texture fetch module 408, the associated data
`may also be placed in the FIFO 407. When the texture
`requests complete, the results may be combined with the
`data from the FIFO 407, and a small portion of logic may
`decide whether to send the completed quad to the combiner
`410, or back around for another pass at the shader module
`406.
`
`FIG. 5 illustrates a graphical representation of the edge
`distances generated by the rasterizer 404. As shown, the
`edge distances (e0, e1, e2) represent a perpendicular distance
`500 from an edge 502 of a primitive 504 to a pixel 506
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 10 of 14 PageID# 14332
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 10 of 14 Page|D# 14332
`
`US 6,532,013 B1
`
`5
`
`within the primitive 504. Such edge distances (e0, e1, e2)
`thus identify the location of the pixel 506 within the primi-
`tive 504.
`
`As a function of the shading calculations, various texture
`look-up operations may be carried out utilizing the texture
`look-up module 408 in order to obtain output having appro-
`priate texture map colors. To accomplish this, texture coor-
`dinates may be sent to the texture look-up module 408. In
`response thereto, texture information is received from the
`texture look-up module 408. Such texture information may
`take any form including, but not limited to filtered texture
`color, etc.
`During the course of use, the feedback loop 409 may be
`used for performing another shading calculation using the
`texture information from the texture look-up module 408 in
`order to generate further output. As an option, the texture
`information may include filtered texture look-up values for
`use in retrieving further texture information when the texture
`information retrieval operation is repeated. The present
`invention thus allows repeated texture information retrieval
`and shading calculations in an iterative, programmable-
`manner. In other words, each iteration may be programmed
`to do a desired shading calculation with or without a texture
`look-up, where each subsequent iteration may use results of
`previous texture look-ups for generating further results.
`In one embodiment of the present invention, at least a pair
`of texture look-up modules is coupled to a pair of shading
`modules which together constitute at
`least four logical
`modules. Further, the system may be capable of performing
`both interpolation and shading calculations including pre-
`texture shading calculations and post-texture shading calcu-
`lations
`
`FIG. 6 illustrates a flow diagram depicting the manner in
`which the shading calculations are interweaved with the
`texture fetch operations in a plurality of iterations 600. As
`shown, each iteration 600 includes a shading calculation
`602. During each shading calculation 602, a decision 604
`may be made as to whether a texture look-up may be
`performed. If so,
`texture information may be retrieved
`during a texture look-up operation 605. Also during each
`iteration 600, another decision 606 is made as to whether an
`additional shading operation 602 is to be executed.
`If it is determined in decision 606 that an additional
`
`shading operation 602 is to be performed, another iteration
`600 is executed. On the other hand, if no further shading
`operations are to be executed, the process of FIG. 6 may be
`terminated. During use, the number of iterations may vary
`per the desires of the user.
`As such, decision 604 allows additional texture informa-
`tion to be retrieved in subsequent shading calculations 600
`based on previously retrieved texture information. It should
`be also noted that this may be done on different texture
`maps. In the alternative, it may be decided in decision 604
`to not do a texture look-up, and merely perform a shading
`calculation 600 during a current iteration.
`the
`As mentioned earlier during reference to FIG. 4,
`shading module 406 may carry out many various types.of
`operations in order to produce output of various types based
`on the edge distances (e0, e1, e2) generated by the rasterizer
`404. Such output may include, but are not limited to diffuse
`output colors, fog output values, specular output colors,
`depth output values, texture color output values, a level of
`detail (LOD) value, and/or a Z-slope value. As an option, the
`calculation of a level of detail (LOD) may be performed
`based on the texture information that is previously retrieved.
`In one embodiment, the LOD is calculated numerically.
`For example, the LOD may be calculated in the manner set
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`forth in a co-pending application entitled “SYSTEM,
`METHOD AND ARTICLE OF MANUFACTURE FOR
`
`CALCULATING A LEVEL OF DETAIL (LOD) DURING
`COMPUTER GRAPHICS PROCESSING” filed Mar. 24,
`2000 under application Ser. No. 09/535,045, naming Walt
`Donovan as inventor which is incorporated herein by refer-
`ence in its entirety.
`In particular, four texture coordinates (uO, ul, u2, u3) may
`be first identified on the texture map corresponding to four
`portions of a block of pixels. Such texture coordinates may
`be such that uO is an upper left corner of the block, 111 is an
`upper right corner of the block, u2 is a lower left corner of
`the block, and u3 is a lower right corner of the block. In one
`embodiment, the block may include a 2x2 block of pixels.
`
`Thereafter, a derivative value (ux), or du/dx, and (uy), or
`du/dy, are calculated. FIG. 6A graphically illustrates the
`derivatives values (ux) and (uy). The pixel 607 is mapped in
`a texture coordinate space 608 with the derivatives values
`(ux) and (uy) defined as shown.
`While the derivative values may be calculated in any
`known manner, the derivative value (ux), or du/dx, and (uy),
`or dv/dx, may be estimated using Equations #1.
`
`Equations #1
`
`“x=((“1‘“0)+(“3‘“2))/2
`
`“y=((“2‘“0)+(“3‘“1))/2
`
`It should be noted that similar calculations may be made
`for vxy and pm texture coordinates using the corresponding
`equations for derivatives with respect to the x and y axis.
`With such information, a LOD value is calculated using
`Equations #2.
`
`Equations #2
`
`LOD=1/210g2(max (dx, dy», where
`
`dx=ux2+vx2+px2
`
`dy=uy2+vy2+py2
`
`In order to accomplish the foregoing shading operations
`set forth in FIGS. 4 and 6, perspective corrected barycentric
`weights (g0, g1, g2) may be calculated from the edge
`distances (e0, e1, e2). In a first embodiment, Equations #3 are
`utilized to calculate the perspective barycentric weights (g0,
`g1, g2)
`
`Equations #3
`
`g0=e0*d
`
`g1=el*d
`
`g2=62*d,
`
`where
`
`5=eO*w0*w1+el*w1*w2+62*w2*w0
`
`d=1/5,
`
`where
`
`w0, WI and w2 are the perspective correction factors used to
`perform hyperbolic or perspective corrected interpolation
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 11 of 14 PageID# 14333
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 11 of 14 Page|D# 14333
`
`US 6,532,013 B1
`
`7
`FIG. 7 illustrates a point within a primitive that is capable
`of being defined by barycentric weights (g0, g1, g2). In
`particular, point (p) may be defined by Equation #4 in terms
`of the barycentric weights (g0, g1, g2) and vertices (a, b, c)
`of the primitive. As an option, the perspective barycentric
`weights may use parameter values of the primitive in order
`to perform perspective corrected interpolation of vertex
`parameter values. Optionally,
`the perspective barycentric
`weights may use unclipped parameter values.
`
`Equation #4
`
`p=a *go+b *g1+6*g2
`
`Given the perspective corrected barycentric weights (g0,
`g1, g2), the various shading calculations may be performed
`using the equations set forth in Table 1.
`
`Table 1
`
`Z in Screen Space(depth)—Calculate screen space 2 val-
`ues at the vertices, then interpolate them per-pixel using the
`edge values as non-perspective corrected weights.
`
`ZSO=ZCO/W60; zsl=zcl/wc1; st=ch/wc2.
`
`ZS=(€0250+€1251+82252)/(80+81+82)
`
`Fog—Interpolate the fog range (may be affine xform of 2
`eye, or distance from eye point—both computed per vertex).
`Call fog range from xform fr.
`
`fr=g0*f10+g1*fr1+g2*fr2
`
`then retrieve frac(fr) and run it through either:
`1) no table
`2) exp table
`3) epr2 table
`Note: This table could be implemented as a texture map
`lookup. This would allow one to do an Opengl fog table.
`4 (or any other number of) Non-projective 2-D Texture
`Coordinates—This optimization can only be done if all q’s
`are one. Otherwise,
`the 2 projective case below is per-
`formed.
`
`5=g0*50+g1*51+g2*52
`
`t=g0*t0+g1*t1+g2*t2
`
`2-D Projective or Cube Texture Coordinates
`
`5q=g0 *50+g1*51+g2 *52
`
`tq=g0 *t0+g1 *t1+g2*12
`
`q=g0 *q0+g1*ql+g2 *q2,
`
`where
`
`Cube
`
`qi=1/q
`
`s=5q*qi
`
`t=tq*qi
`
`5r=g0 *50+g1 *51+g2 *52
`
`tr=g0*t0+g1*t1+g2*t2
`
`rr=g0*10+g1 *r1+g2*r2
`
`f=pickmax(5, t, r)
`
`Note: f is a face index beginning at zero (for s). Pick is a
`function that chooses the fth entry from the list passed to it,
`where f is first parameter.
`
`5r=pick(fi tr, rr, 5r)
`
`tr=pick(fi rr,5r, tr)
`
`r=pick(fi5r, tr, rr),
`
`where
`
`ri=1/r
`
`5=5r*ri
`
`t=tr*ri
`
`3-D projective textures
`
`5q=g0 *50+g1*51+g2 *52
`
`tq=g0*t0+g1 *t1+g2 *t2
`
`rq=g0 *10+g1*r1+g2 *r2
`
`q=g0*q0+g1*q1+g2 *ql,
`
`where
`
`qi=1/q
`
`s=5q*qi
`
`t=tq*qi
`
`r=rq*qi
`
`texture
`texture lookup—After the first
`2-D dependent
`lookup, two components of the argb color are reinterpreted
`as texture coordinates, and looked up again.
`Dx6 Bump Mapping—After the first
`texture look-up,
`color r0,g0,b0 is received which is multiplied by 2x2 basis
`matrix, which is constant. s1 and t1 are the interpolated
`texcoords for the second look-up.
`
`51p=m11 *r0+m12*g0+51
`
`tlp=m21*10+m22*g0+t1
`
`After the second texture lookup, received is r1,g1,b1,a1.
`
`f=(b0*m31+m32)
`
`r1p=r1*f
`
`g1p=b1*f
`
`b1p=b1*f
`
`a1p=a1
`
`Polygon Offset—let the notation z(1,0) indicate the 2
`value of the pixel in the bottom right corner of the pixel
`block, or quad. z(0,1) would be the top left. compute the z
`slopes:
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`Case 3:14-cv-00757-REP-DJN Document 87-3 Filed 04/16/15 Page 12 of 14 PageID# 14334
`Case 3:14-cv-OO757-REP-DJN Document 87-3 Filed 04/16/15 Page 12 of 14 Page|D# 14334
`
`DCO=Z(1,0)—Z(0,0)
`
`zyO=z(0,1)—z(0,0)
`
`factor=max(abs(DcO),ab5(zyO))
`
`US 6,532,013 B1
`
`10
`
`zc=a0 *51+b0 *t1+60*r1;
`
`wc=a0*52+b0 *t2+60*r2;
`
`zs=zc/wc
`
`compute a new 2:
`zo=z+factor*zs+units, where factor and units are state.
`Loaded with pipelined state bundles.
`Dot Product-based texture mapping—Using s0, t0, a first
`texture look-up is performed. Call the results a0 b0 c0. Dot
`products are taken between these values and the subsequent
`texture coordinates to generate a new set of texture coordi-
`nates for a subsequent texture lookup:
`
`5p=51 *a0+t1 *b0+r1*60
`
`tp=52*a0+t2 *b0+r2*60
`
`2-D texture lookup performed using (sp, tp).
`or
`
`5p=51 *a0+t1 *b0+r1*60
`
`tp=52*a0+t2 *b0+r2*60
`
`rp=53 *a0+t3 *b0+r3*60
`
`3-D texture lookup performed using (sp, tp, rp) or
`Cube Texture coordinates performed (as above) using (sp,
`tp, rp)
`Reflective Bump Mapping—Using s0, t0, a first texture
`look-up is performed. Call the results hs,ht,hr. this is the
`normal in tangent space.
`interpolate s1, t1, r1.—this is the surface tangent vector in
`eye space
`
`interpolate s2, t2, r2—this is the surface binormal vector.
`interpolate s3, t3, r3—this is the surface normal vector.
`These are used as a basis matrix by which to multiply the
`vector hs,ht,hr. This will give the normal in eye space.
`so,
`
`nx=51 *h5+52 *ht+53 *hr;
`
`ny=t1 *h5+12 *ht+t3 *hr;
`
`nz=r1 *h5+r2 *ht+r3 *hr;
`
`Use the (nx,ny,nz) vector as a cube map lookup for the
`diffuse lighting component.
`Now compute the reflection vector per pixel.
`
`let ne=nx*ex+ny*ey+nz*ez;
`
`let n2=nx*nx+ny *ny+nz*nz;
`
`rx=2 *nx*ne/n2—ex;
`
`ry=2 *ny *ne/nZ—ey;
`
`rz=2 *nz*ne/n2—ez;
`
`Use this reflection vector as a cube map lookup.
`Depth Texture Mapping with Dot Products—Using s0, t0,
`a first texture look-up is performed. Call the results a0, b0,
`c0. Dot products are performed between these values and
`two subsequent sets of texture coordinates to produce 2 clip
`and w clip values. These quotient of these values replaces
`the previously calculated 2 screen value.
`
`Pixel culling—The s, t, r, and q coordinates for a particu-
`lar texture are interpolated per-pixel. Each coordinate is
`individually configured to check for either negative or
`non-negative values. If the texture coordinate matches the
`configuration, the pixel is culled (not drawn).
`Isotropic BRDF—The results of two texture coordinate
`lookups are interpreted as 16-bit numbers, h0, 10 and hl, 11.
`A third texture lookup is performed using (h0, h1, 10—11).
`It should be understood that each of the options set forth
`in the foregoing tables may be adapted to reuse common
`portions of the hardware set forth in FIG. 4. As set forth
`earlier, such hardware is