Hardware Supported Bump Mapping:
`
`A Step towards Higher Quality Real-Time Rendering
`
`I. Emst*, D. Jackel“, H. Riisseler*, O. Wittig*
`
`‘German National Research Center for Information Technology.
`Institute for Computer Architecture and Software Technology (GMD FIRST)
`"University Rostock. Hansestadt Rostock Germany
`
`Abstract
`
`Today's high-end Gouraud renderers produce nicely textured scenes
`by mapping two-dimensional images onto modeled objects in real-
`
`We present a tenderer which textures surfaces in the normal sense of
`the word using bump textures to simulate wrinkled or dimpled color-
`ful surfaces. Using a simplified bump mapping method we first suc-
`ceeded in designing a real-time bump mapping renderer based on the
`high-quality Phong shading model.
`Applying several improvements to our former Phong shading hard-
`wareweareabletowalkthroughperspectivelyconectbumpmapped
`scenes illuminated by colored lightsources.
`This paper describes the main building blocks of the overall architec-
`ture. including reflectance cubes to support a local viewer. a Taylor-
`series based division to calculate homogeneous coordinates and our
`hardware adapted bump mapping method.
`
`-
`
`Keyworrk: bump mapping. reflectance cube. normal vector interpo-
`lation. hardware division.
`local viewer. unlimited colored light-
`sources. Phong shading. real-time rendering
`
`1. Introduction
`
`A contemporary "high-end" graphics workstations has a
`rendering performance of typically one million polygons
`per second in conjunction with hardware supported
`transparency, anti-aliasing, perspective texture mapping,
`image compositing, etc. In spite of the fact that great
`steps have been taken to improve the realistic display of
`very complex objects and scenes. many problems remain
`to be solved in order to achieve
`
`image generation rate of complex scenes (>500.000
`polygons) of at least 30-50 frames per second
`
`- more photorealistic image generation by means of
`hardware supported “normal vector shading” as well
`as improved nanti-aliasing” or ”bump mapping”.
`
`linear scaleability of the rendering performance and
`an improvement of the cost-performance relationship.
`
`In the past decade, many attempts have been undertaken
`to make ..normal vector shading” applicable for hard-
`ware implementations. M. Deering et al.[6] presented a
`simulated VLSI-approach of a- normal vector shader
`(NV8) chip suitable for Phong shading [4]. Bishop and
`Weimar [3] suggested a method which approximates
`Phong’s normal interpolation and light reflection equa-
`tion by a Taylor series expansion. Bishop and Weimar’s
`fast Phong shading reduces the number of computation
`steps per pixel to only 5 additions and one memory
`access. Another alternative. suggested by U. Clausen [5],
`interpolates polar angles instead of normal vectors,
`which has the advantage that the normal vector unit
`length remains unchanged during the interpolation. Our
`normal-vector shading approach [11] is based on the
`approximation of the illumination model geometry by
`using the reflection map method. This shading tech-
`nique, which is used since 1989 in our VISA-systems
`(VISA=VISua1ization Accelerator). operates similar to
`the reflection vector shading hardware of Voorhies and
`Foran [13]. Instead of the single precalculated map of the
`VISA-shader. Voorhies and Foran propose a six-sided
`cube, which incorporates fire specular spread function
`values. Similarly to the reflection map method. each face
`of the cube is indexed by an unnormalized reflection
`vector.
`
`An additional step towards photorealistic image genera-
`tion can be achieved by means of bump-mapping. Half
`of the work of hardware supported bump-mapping is
`done when using a normal vector shader instead of a
`gouraud shader. This paper is focus on the extension of
`our normal vector shading method for achieving a simple
`and efficient hardware bump-mapping support.
`
`First. we discuss the principle of hardware Phong-shad-
`ing. In section 3 the extension of this shading technique
`to achieve bump-mapped surfaces is presented. Finally,
`the hardware implementation aspect of our bump-map-
`
`MEDIATEK, Ex. 1028, Page 1
`IPR2018-00101
`
`

`

`ping method and some simulation results are discussed
`in more detail.
`
`the four neighboring cube elements reduces point sam-
`pling artifacts.
`
`2. Hardware Phong Shading
`
`Phong shading as proposed in [l 1], uses a planar reflec-
`tance map to “pick” the intensity.
`
`The “picking” of the reflectance map is done as follows:
`
`mi’z‘ii «2 = amt-fl
`
`Considering the simplified geometry using parallel light
`and parallel viewer the amount of light reflected by an
`object’s surface is given by r = 'f(al, a2) .
`
`There are some disadvantages to our former approach
`using a reflectance map:
`
`- visible artifacts caused by point sampling the reflec-
`tance map;
`
`- visible distortions at the borders of the atan map
`(angle > 80 degrees);
`
`illumination model allows parallel viewer only;
`
`' costly computation for reflectance map if viewer
`changes.
`
`Solutions to reduce the size of the planar maps are given
`by for example in [12]. [10]. To avoid distortions in the
`border area of the map. it is straightforward to use a
`hemi-cube over one polygon. This algorithm is com-
`monly used for radiosity medrods and to approximate
`normal vector directions. Distortions which occur at the
`
`comers of the herni-cube will be reduced by using an
`arctan map in the range of -45 to 45 degrees. similar to
`the reflectance map approach. Bilinear interpolation of
`
`DeSpite the progress on planar or hemi-cube maps the
`restriction to parallel viewers could not be alleviated.
`One reason for this is that all progress is focused on the
`use of normal vectors to address cube faces. In [8], [13]
`it is proposed to use the reflection of the eye vector on
`the surface to address a six sided cube. Since the reflec-
`
`tion of the viewer is closely related to the specular com-
`ponent of the Phong shading algorithm, this method can
`also be applied to shading. Furthermore a six-sided cube
`contains all possible reflection vector directions around
`a fixed normal vector, the viewer can be local. As a con.
`
`sequence. this method allows for the view point to be
`changed without recomputation of the cube map.
`
`As indicated in [13] it is possible to calculate the reflec—
`tion vector RU without normalization ofthe surface nor-
`mal NU and the eye vector by~multiplying both sides by
`the length of the unnormalized normal vector squared:
`
`RU = 2-NU- (NU-E) -E- (NU-NU)
`
`(EQI)
`
`To calculate the diffuse component, another six sided
`cubemapcanbeusedwhichcanbeindexedbythe
`unnorrnalized normal vector similar to the hemi cube
`
`approach. Calculating colors at the polygon vertices and
`bilinearly interpolating these colors over the polygon is
`not sufficient because highlights in the middle of a trian-
`gle cannot be generated. To reduce cube map space and
`calculation time methods proposed in [12], [10] are used.
`In Figure l the indexing of the cube's faces is illustrated.
`
`
`
`Figure 1: Application of cube faces to calculate phong illumination
`
`MEDIATEK, Ex. 1028, Page 2
`IPR2018-00101
`
`

`

`3. Bump mapping
`
`Bump mapping is an elegant technique to simulate wrin-
`kled or dimpled surfaces without the need to model sur-
`face properties geometrically.
`
`This is done by angular perturbing the surface normals
`according to information given in a two dimensional
`bump map.
`
`Since intensity is mainly a function of the surface nor-
`mal, the local reflection model produces local variations
`on a smooth surface dependent on the perturbation
`defined in the bump map.
`
`Bump mapping is important because it textures a surface
`in the normal sense of the word rather than modulating
`the color of flat surfaces.
`
`3.1. Classic Bump mapping
`
`Blinn [I] first developed a scheme that perturbs the nor.
`mal vector independently of the orientation and position
`ofthe surface. In the case ofanimations, otherwise it is
`obvious that the normal at a particular point must always
`receive the same rate of perturbation. Otherwise the
`bump map detail moves as the object moves. This is
`achieved by aligning the perturbation on a coordinate
`system based on local derivatives.
`
`If 0 (u, v) is a function representing position vectors of
`points 0 onthe surfaceofanobject, wefirstaddasmall
`increment, derived from the bump map B (u, v) to
`define 0‘ (u, v) :
`
`O’(u,v) = 0(u,v) +B(u,v)l%i
`
`Because the displacement function B is small compared
`with its spatial extend by differentiating we get:
`
`N’(u,v) = Oux0v+Bu(I%ix0v)
`(
`N)
`NxN
`+Bv Ouxm +Bqu( NZ)
`
`
`
`(E02)
`
`The last
`dropped.
`
`N N
`
`term Bqu(—'x7j = 0
`W!
`
`and can be
`
`Note that the normal to a surface at a point is given by
`N = 0" x 0v where 0" and 0v arethe partial deriv-
`atives of the surface at point 0 lying in the tangent plane
`(see Figure 2).
`
`Figure 2:
`
`Tangent vectors 0“ , 0‘,
`at a point on a sphere with
`
`normal vector N.
`
`From equation (EQ 2) it can be seen that implementing
`bump mapping in hardware is not possible without sim-
`plifying the calculation scheme.
`
`3.2. Hardware adapted Bump mapping
`
`Using Phong vertex normal interpolation, we already
`know the surface normal at a point. This surface nomal
`N represents the orientation of the surface independent
`of the rotation about N .
`
`Our new tenderer VISA II will use a bump mapping
`scheme closely related to texture mapping.
`
`
`
`texture coordinate system
`
`clip coordinate system
`
`MEDIATEK, Ex. 1028, Page 3
`IPR2018-00101
`
`

`

`The bump map describes the perturbation Au, Av of
`normal [0, 0, 111‘ intexture space u, v, w.
`
`The transformation matrix A carries out this mapping by
`projecting the local polygon coordinate system u’, v', N
`on the texture coordinate system u, v, w .
`
`
`
`Figure 4: Effect of bump map on surface shape
`
`Figure 4 shows a precomputed bump map to simulate
`brick structures. Notice the u and v projections of line 1"
`and 1,, The first line (shape) depicts surface orientations of
`the cut along 1". IV on the brick structure. The second line
`shows the displacement Au, Av. Positive Au, Av val-
`ues represent a displacement of the underlying pixel nor-
`in direction of u, v; negative values represent
`opposite directions. The third line shows the resulting sur-
`face shape and its perturbed surface normals. Notice the
`effect of the displacement between two brick stones to
`simulate the groove.
`
`Transformation of the bump map onto an object is done
`similar to texture mapping. By using homogenous coordi-
`nates the resulting address (5. 5 is the perspectively cor-
`rect pixel index on the bump map.
`
`To guarantee a perturbation in the desired direction u, v,
`the object coordinate system must be transformed such
`that the w -axis equals the surface normal vector direction
`N andthe u -and v-coordinateaxisarerotatedabout
`
`N according to the polygon mapping defined by the mod-
`
`Figure 3 illustrates the mapping of the polygon
`
`[xpyp 21. WI] T.
`
`1'
`
`pixel
`normal
`perturbed
`per
`To
`obtain
`the
`, Au, Av transformed by A is
`N’ = [n’r n‘y,n’z]
`T
`.
`added to the surface normal N = [Br "y. Hz]
`:
`
`n’ =nx+Au-A00+Av-Am
`
`n' =ny+Au.Aw+Av-An
`n’ =nz+Au-A20+Av-A21
`
`The resulting normal is used to calculate the pixel color in
`the Phong illumination model described in the previous
`sections.
`
`4. Proposed Architecture
`
`The scan-converting procedure for triangles is described
`in detail in [11]. Given the triangle’s start coordinate with
`its start attributes and slope increments, the Scan Line Ini-
`tializer computes the relevant data at the start point of
`every new scanline, while the Pixel Calculator interpo-
`[ates the coordinates and attributes within a scanline. This
`
`differs from the former design in the interpolation of an
`extended data set. In particular, we extend the interpolaror
`to homogenous x, y, z, w space, which is necessary for
`achieving perspective texture mapping [9],
`texture-,
`bump-coordinates (u, v) , eye vector [er 2,, e117 .
`
`After raster'ization, the obtained pixel fragment describes
`the position and orientation of small surface elements in
`3D-space, the direction ofreflection, the 5 , 5 texture and
`bump coordinates.
`
`Figure 5 shows a sketch of the proposed architecture.
`
`The non-shaded boxes will be discussed in more detail.
`
`The operation principle of the interpolation-units is pre-
`sented in [11]. To visit the pixels of adjacent triangles
`only once, the edge traverser was slightly modified [7].
`This is essential for color blending.
`
`[x29 yz: 22, W2] To [X3s )3, Z3, W3] T in the Clip COOIdi-
`“1 ”r T
`
`nate system x, y, z, w to the edges
`“3 ”3 T
`“2 ”2 T
`'thetexturecoordinatesys-
`—"‘ —-
`”’2’ w
`’ w3’ W3
`m
`tern u,v,w.
`
`[‘71, wt
`
`,
`
`MEDIATEK, Ex. 1028, Page 4
`IPR2018-00101
`
`

`

`Colorlight
`
`2
`y
`it
`color
`Figure 5: Overall blockdiagram of the rendering engine
`
`The bump engine perturbs the pixel normal by the corre-
`sponding bump map entry Au, Av. The bump map is
`aligned perpendicular to the surface normal. This is done
`by multiplying the bump map entries by A:
`
`
`
`Figure 6: Bump Engine
`
`Kr = 21+ [Au,A\’] -A
`
`The reflectance cubes are addressed by the normal for the
`diffuse color component or by the reflected ray for the
`specular component. The address calculation unit for the
`reflected ray can be foundin [13]. The vector's major axis
`determines die corresponding reflectance cube face.
`Indexing this map is done by dividing the vector’s minor
`axis values by the major axis value.
`
`Selection of reflection map:
`
`[up ny, nz] --) max { lull, |ny|, Inzl} , sign (nx, ny, "2)
`Address function:
`
`X - faces:
`
`n
`n
`1 4
`’ I": ’ |n
`n
`n
`
`[’13,
`’"y
`Y—faces: 4,13%],
`
`Z—faces:
`
`4.1. Division with on chip ROM
`
`Perspective correct texturing and calculating the intersec-
`tions of the normal and reflection vectors with the cube
`
`faces need one division per pixel. A common approach is
`to build the reciprocal ; followed by a multiplication.
`Convergence division algorithms normalize the divisor to
`apositive interval, typically 0,5 5 w < 1 ;positive num-
`bers are shifted up or down by a priority encoded shifter
`code. Negative numbers are first converted to signed pos-
`itive values (complement +1). The result .7 is corrected by
`the exponent of the normalization.
`
`In the fpllowing we focus on the computation of the recip-
`W
`rocal - by Taylor-series approximation [14] about
`w = w,l . After the normalization step, we can write:
`“I
`II-
`II
`w = 0,1w2w3...wm_lw ...w ‘w
`
`wk = 0.1w2w3...wm_1wml...l.
`
`With Aw = w—whso,onecan develop
`
`For a reasonable hardware realization. the number of
`
`terms in the sum has to be small and the on-chip ROM
`implementation has to be memory efficient.
`
`Tnmcating after the third term of the sum and using B as
`the output of the look up table approximating w— , one can
`rewrite the above formula:
`"
`
`é: 8+32-Aw+Ba-sz+error
`
`MEDIATEK, Ex. 1028, Page 5
`IPR2018-00101
`
`

`

`The error is a function of the number of sum-terms t, the
`number of look-up-table entries m —1 and b the number
`of bits per word B . (Remember: 0: are the relevant bits
`of W. = 0.lw2w3...wm_ lwmll)
`
`Figure 7 shows the division by the homogenous factor
`w. With technology currently available, first simula-
`tions estimate an area of ca 25mm2 for this unit. The
`
`division of the vector’s major axis needs less accuracy,
`resulting in a smaller silicon area.
`normalized W
`
`
`
`Figure 7: Divider for WW
`
`ative to the surface normal direction. To calculate the
`
`correct perturbed normals for each per pixel normal,
`recalculation of transformation matrix A for each pixel
`is necessary. Although the perturbation of pixel normals
`varying from the surface normal is not correct, it causes
`only a slight movement of the centroid of the highlight.
`
`In Figure 8, spherical interpolation of a per pixel normal
`(black) is shown. Given the shown lightsource and the
`viewer directions the centroid of the highlight
`lies
`exactly in the middle of the polygon. Applying linear
`interpolation (gray) the normal pointing exactly to the
`lightsource and the viewer is located slightly on the left
`causing the highlight to move to the left. In general this
`effect is not visible in animated scenes.
`
`5L
`
`IE
`
`
`
`eemroldolrdgmght
`
`Figure 8: Spherical interpolation versus linear interpolation
`
`5. Results
`
`One limitation of the bump mapping method described
`above is that all normals on a polygon are perturbed rel-
`
`
`
`
`
`at
`Figure 9: Mexican 1-” Structure carved in tin plane
`
`MEDIATEK, Ex. 1028, Page 6
`IPR2018-00101
`
`

`

`Another disadvantage caused by the calculation of A
`relative to the surface normal is shown in Figure 10:
`
`
`
`Figure 10: False illumination at edge of neighboring poly-
`
`Simulating extreme curvature of adjacent polygon
`meshes by modeling vertex normals leads to machband
`effects at the edges. The vertex normal VI of mesh 1 is
`perturbed dependent on the surface normal (dashed nor-
`mal). The same rate of perturbation is applied to v2 of
`mesh 2 perpendicular to the surface normal of mesh 2
`(dashed gray normal). Assuming the vertex normal
`direction to be the perfect mirror direction, slight varia-
`tions of vertex normals v1 and v2 generate different
`shading intensities. These shading intensity variations
`can be seen clearly along the edge revealing triangula-
`tion structures.
`
`to avoid
`responsibility is
`Therefore the modeler ’s
`extreme curvature simulations or to use finer triangula-
`tion of these scene parts
`
`Figure 9 shows a bluish tin plane illuminated by a purple
`(upper left) and a white (front) lightsource. The plane is
`assembled out of two bump textured triangles. A Mexi-
`
`can head function is used to calculate the normal pertur-
`bation for a 40x40 bump map (see Figure 9 right).
`Applying the bump map to the plane repeatedly leads to
`the desired carving. The zoomed patch in the middle of
`Figure 9 clearly shows correct purple and white high-
`lights given by the illumination geometry.
`
`In Figure 11 the creation of complex images using con-
`ventional textures and bump textures is shown. In our
`example we rebuilt a small part of the “Berlin Wall"
`painting a graffiti texture (Figure 11 middle). To simu-
`late carving and rotting of the wall we created a grey—
`scale image which was converted to a bump map. Notice
`the writing “Berlin Haup(t)stadt" chiseled in the wall
`(see Figure 11 left (bump sketch) and right (resulting
`image». An animation of the “Berlin Wall" example
`scene generates the impression of moving sky trackers
`over the rotten wall.
`
`6. Conclusions and Future Work
`
`We have designed and simulated a real time renderer
`supporting unique features like Phong shading and bump
`textures. Mth the presented renderer we are capable of
`generating animated scenes of higher quality then with
`conventional gouraud shading renderers.
`
`This technique could be improved further by:
`
`-
`
`techniques to simulate spherical interpolation of un-
`normalized normal vectors to avoid slight translations
`of highlights on polygons;
`
`- new hardware adapted methods to avoid artifacts
`caused by angle variations of neighboring polygons;
`
`- artifact reduced MIP mapping for bump textures;
`
`0
`
`implementation of “living“ textures;
`
`
`
`Bump sketch
`
`Figure 11: Simulation of graffiti on rotten wall.
`
`MEDIATEK, Ex. 1028, Page 7
`IPR2018-00101
`
`

`

`
`
`o specification of a set of extensions for the OpenGL
`graphics language to support our unique graphic fea-
`tures.
`
`[14]
`
`Furthermore, it is desirable to switch between conven-
`
`tional texturing. bump texturing, reaction diffusion textur-
`ing and displacement mapping [2].
`
`Hardware", Siggraph Prowdings 1994. pp 163-166
`D. Wong. M. Flynn. “Fast Division Using Accurate Quo-
`tient Approximtions to Reduce me Number of Iterations".
`IEEE TRANSACTION ON COMPUTEIS. VOL41. NO.
`8. pp. 981 - 995, August 1992
`
`7. Acknowledgments
`
`We would like to thank the following people for their sug-
`gestions and contributions to this work: I. Duenow (scan-
`liner and parameter generation), L. Guangming (error
`analysis on division algorithms). T. Le Vin (Verilog simu-
`lation of arithmetic units, including silicon real estate esti-
`mation). and S. Budianto (Register Transfer Level
`Simulation). In addition. we would like to thank the many
`people who have commented on this paper.
`
`Literature
`
`J. F. Blinn. “Simulation ofWrinkled Surfaces Compumr
`Graphics. 120). pp. 286-292. (Pr'oc. SlGGRAPH ‘78)
`3.6. Becker. NJ... Max. “Smooth Iransitlons between
`Bump Rendering Algorithms“. SIGGRAPH ’93. pp 183 -
`190
`
`G. Bishop, D. M. Weimar. “Fast Phong Shading". Computer
`Graphics. Vol. 20, No. 4 , pp. 103 - 106. 1986
`Phong. Bui Thong. “Illumination for Computer Generated
`Pictures”. Commiean'om aftheACM, Vol. 18, No. 6
`(1975). pp. 311-317
`U. Clausen. “Reducing the Phong Shading Method”, Proc.
`Eurographics 89. Eds. W. Hansrnann. ERA Hopgood. W.
`StraBer. pp. 333 - 344. North-Holland. 1989
`M. Deering. S. Winner. 3. Scbediwy. C. Duffy. N. Hunt.
`“'nre Triangle Processor and Normal Vector Shader: A
`VLSI System for High Performance Graphics", Computer
`Graphics. Vol. 22. No. 4; pp 21-30, 1988
`J. Duenow. “Strategien zur Abbildung der OpenGL Soft.
`warebibliothek auf die vorhandene VISA Hardware". inter-
`nal Report. GMD first. Spring 1995
`
`J. D. Foley. A. van Dam. S. K. Feiner, J. F. Hughes: “Com~
`purer Graphics: Principles and Practice”, 2nd Edition. Addi-
`son-Wesley, 1990
`
`Heckbert. H. P. Moreton; “lnmmolarion for Polygon Texture
`MappingandShading”.inSmeoftheArtComputerGraph-
`ice: VisuallzationandModeling; pp 101 - 111. Springer ‘91
`T. 1km. “A walable high-paformance graphics processor.
`GVIP' , The Visud Computer (1995). 11 pp. 121-133,
`Springer 1995.
`D. Iankbl. H. Rilneler. "A Real Time Rendering System
`with Normal Vector Shading". 9th Burographics Workshop
`on Graphics Hardware, Oslo. Norway. 1994, pp 48-57
`.l. T. van Scheltinga. J. 8min M. Bosnia, “Daign ofan on-
`chipreflectance map” , EG Hardware Workshop 1995.
`Maastrich. 28-29 August
`
`D. Voorhies and Jim Foran. “Reflection Vector Shading
`
`MEDIATEK, Ex. 1028, Page 8
`IPR2018-00101
`
`