Hardware Supported Bump Mapping:
`A Step towards Higher Quality Real-Time Rendering
`
`I. Ernst*, D. Jackél**, H. Riisseler*, O. Wittig*
`
`*German National Research Center for Information Technology,
`Institute for Computer Architecture and Software Technology (GMD FIRST)
`**University Rostock, Hansestadt Rostock Germany
`
`Abstract
`
`Today’s high-end Gouraud renderers produce nicely textured scenes
`by mapping two-dimensional images onto modeled objectsin real-
`time.
`
`Wepresent a renderer which textures surfaces in the normal sense of
`the word using bumptextures to simulate wrinkled or dimpled color-
`ful surfaces. Using a simplified bump mapping method we first suc-
`ceeded in designing a real-time bump mapping renderer based on the
`high-quality Phong shading model.
`Applying several improvements to our former Phong shading hard-
`ware we are able to walk through perspectively correct bump mapped
`scenes illuminated by colored lightsources.
`This paper describes the main building blocksof the overall architec-
`ture, including reflectance cubes to support a local viewer, a Taylor-
`series based division to calculate homogeneous coordinates and our
`hardware adapted bump mapping method.
`
`Keywords: bump mapping, reflectance cube, normal vector interpo-
`lation, hardware division,
`local viewer, unlimited colored light-
`sources, Phong shading, real-time rendering
`
`1. Introduction
`
`A contemporary "high-end" graphics workstations has a
`rendering performanceof typically one million polygons
`per second in conjunction with hardware supported
`transparency, anti-aliasing, perspective texture mapping,
`image compositing, etc. In spite of the fact that great
`steps have been taken to improvethe realistic display of
`very complex objects and scenes, many problems remain
`to be solved in order to achieve
`
`*
`
`image generation rate of complex scenes (>500.000
`polygons) of at least 30-50 frames per second
`
`* more photorealistic image generation by means of
`hardware supported “normal vector shading” as well
`as improved ,,anti-aliasing” or ,,bump mapping”.
`linear scaleability of the rendering performance and
`an improvement of the cost-performance relationship.
`
`*
`
`In the past decade, many attempts have been undertaken
`to make ,,normal vector shading” applicable for hard-
`ware implementations. M. Deering et al.[6] presented a
`simulated VLSI-approach of a normal vector shader
`(NVS) chip suitable for Phong shading [4]. Bishop and
`Weimar [3] suggested a method which approximates
`Phong’s normal interpolation and light reflection equa-
`tion by a Taylor series expansion. Bishop and Weimar’s
`fast Phong shading reduces the number of computation
`steps per pixel to only 5 additions and one memory
`access. Anotheralternative, suggested by U. Clausen[5],
`interpolates polar angles instead of normal vectors,
`which has the advantage that the normal vector unit
`length remains unchanged during the interpolation. Our
`normal-vector shading approach [11] is based on the
`approximation of the illumination model geometry by
`using the reflection map method. This shading tech-
`nique, which is used since 1989 in our VISA-systems
`(VISA=VISualization Accelerator), operates similar to
`the reflection vector shading hardware of Voorhies and
`Foran [13]. Instead ofthe single precalculated map of the
`VISA-shader, Voorhies and Foran propose a six-sided
`cube, which incorporates the specular spread function
`values. Similarly to the reflection map method, each face
`of the cube is indexed by an unnormalized reflection
`vector.
`
`An additional step towards photorealistic image genera-
`tion can be achieved by means of bump-mapping. Half
`of the work of hardware supported bump-mapping is
`done when using a normal vector shader instead of a
`gouraud shader. This paper is focus on the extension of
`our normal vector shading method for achieving a simple
`and efficient hardware bump-mapping support.
`
`First, we discuss the principle of hardware Phong-shad-
`ing. In section 3 the extension of this shading technique
`to achieve bump-mapped surfaces is presented. Finally,
`the hardware implementation aspect of our bump-map-
`
`MEDIATEK,Ex. 1028, Page 1
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 1
`IPR2018-00102
`
`

`

`ping method and some simulation results are discussed
`in more detail.
`
`the four neighboring cube elements reduces point sam-
`pling artifacts.
`
`2. Hardware Phong Shading
`
`Phong shading as proposed in [11], uses a planar reflec-
`tance mapto “pick” the intensity.
`
`The “picking”of the reflectance map is done as follows:
`_
`an
`-
`Yn
`a, = alan Zz Oo, = atan Z
`n
`nf
`
`
`
`Considering the simplified geometry using parallel light
`and parallel viewer the amount of light reflected by an
`object’s surface is given by r = f(Q,,,) .
`
`There are some disadvantages to our former approach
`using a reflectance map:
`
`* visible artifacts caused by point sampling the reflec-
`tance map;
`
`visible distortions at the borders of the atan map
`(angle > 80 degrees);
`
`illumination model allows parallel viewer only,
`
`costly computation for reflectance map if viewer
`changes.
`
`*
`*
`
`Solutions to reduce the size of the planar mapsare given
`by for example in [12], [10]. To avoid distortions in the
`border area of the map, it is straightforward to use a
`hemi-cube over one polygon. This algorithm is com-
`monly used for radiosity methods and to approximate
`normal vector directions. Distortions which occurat the
`comers of the hemi-cube will be reduced by using an
`arctan map in the range of -45 to 45 degrees, similar to
`the reflectance map approach. Bilinear interpolation of
`
`Despite the progress on planar or hemi-cube maps the
`restriction to parallel viewers could not be alleviated.
`One reason for this is that all progress is focused on the
`use of normal vectors to address cube faces. In [8], [13]
`it is proposed to use the reflection of the eye vector on
`the surface to address a six sided cube. Since the reflec-
`tion of the vieweris closely related to the specular com-
`ponentof the Phong shading algorithm,this method can
`also be applied to shading. Furthermorea six-sided cube
`contains all possible reflection vector directions around
`a fixed normal vector, the viewer can be local. As a con-
`sequence, this method allows for the view point to be
`changed without recomputation of the cube map.
`
`Asindicated in [13] it is possible to calculate the reflec-
`tion vector Ry Without normalization of the surface nor-
`mal NV, and the eye vector bymultiplying both sides by
`the length of the unnormalized normal vector squared:
`
`Ry = 2:Ny: (Ny®E) -E- (Ny Ny) EQD
`
`To calculate the diffuse component, another six sided
`cube map can be used which can be indexed by the
`unnormalized normal vector similar to the hemi cube
`approach. Calculating colors at the polygon vertices and
`bilinearly interpolating these colors over the polygon is
`not sufficient because highlights in the middle ofa trian-
`gle cannot be generated. To reduce cube map space and
`calculation time methods proposed in [12], [10] are used.
`In Figure 1 the indexing of the cube’s facesis illustrated.
`
`Elements conmin
`
`specular cube
`Figure 1: Application of cube facesto calculate phong illumination
`
`diffuse cube
`
`Elements
`contain
`
`value
`
`MEDIATEK,Ex. 1028, Page 2
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 2
`IPR2018-00102
`
`

`

`3. Bump mapping
`
`Bump mappingis an elegant technique to simulate wrin-
`kled or dimpled surfaces without the need to model sur-
`face properties geometrically.
`
`This is done by angular perturbing the surface normals
`according to information given in a two dimensional
`bump map.
`
`Since intensity is mainly a function of the surface nor-
`mal, the local reflection model produces local variations
`on a smooth surface dependent on the perturbation
`defined in the bump map.
`
`Bump mapping is important becauseit textures a surface
`in the normal sense of the word rather than modulating
`the colorof flat surfaces.
`
`3.1. Classic Bump mapping
`
`Blinn [1] first developed a scheme that perturbs the nor-
`mal vector independently of the orientation and position
`of the surface. In the case of animations, otherwise it is
`obviousthat the normal at a particular point must always
`receive the same rate of perturbation. Otherwise the
`bump map detail moves as the object moves. This is
`achieved by aligning the perturbation on a coordinate
`system based on local derivatives.
`
`Because the displacement function B is small compared
`with its spatial extend by differentiating we get:
`
`N’(u,v) = 0,0, +B, 4 x0,)
`N
`(
`N}
`NXN
`+B, O1% TN] +B,B(72)
`The last
`term 3,8,
`3
`= 0
`dropped.
`
`
`NXN
`hy
`
`(EQ 2)
`
`and can be
`
`Note that the normalto a surface at a point is given by
`N = O,xO,, where O,, and O,, are the partial deriv-
`atives of the surface at point O lying in the tangent plane
`(see Figure 2).
`
`Figure 2:
`Tangent vectors O,, O,
`at a point on a sphere with
`
`normal vector NV.
`
`From equation (EQ 2) it can be seen that implementing
`bump mapping in hardware is not possible without sim-
`plifying the calculation scheme.
`
`If O (u, v) is a function representing position vectors of
`Using Phong vertex normal interpolation, we already
`points O on the surface of an object, wefirst add a small
`know the surface normal at a point. This surface normal
`increment, derived from the bump mapB(4, v) to
`N represents the orientation of the surface independent
`define O’ (u, v):
`of the rotation about N.
`
`3.2. Hardware adapted Bump mapping
`
`O' (u,v) = O(w, Vv) +B (uy)i
`
`Our new renderer VISA II will use a bump mapping
`
`
`
`
`schemeclosely related to texture mapping. YPxijeurvonewsoysuen”
`
`texture coordinate system
`clip coordinate system
`Figure 3: Mapping from clip space to texture space
`
`65
`
`MEDIATEK,Ex. 1028, Page 3
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 3
`IPR2018-00102
`
`

`

`The bump map describes the perturbation Au, Av of
`T.
`in texture space u, v, w.
`
`normal [0, 0,1]
`
`pixelfI{Xx{normals-
`
`per
`
`The transformation matrix A carries out this mapping by
`projecting the local polygon coordinate system u’, v’, N
`on the texture coordinate system 4, Vv, w.
`
`To
`
`obtain
`
`the
`
`perturbed
`
`per
`
`pixel
`
`normal
`
`T
`
`-
`
`N= [n’,, ns, n’ , Au, Av transformed by A is
`added to the surface normal N = [noly n.]
`:
`n= n+ Au-Ag+Av- Apo,
`ny, = n,+Au-Ajyt+Av-A,,
`rvio= n,+Au -Ay t+ Av-A,,
`
`T
`
`perpnel PTV TTT Xt
`normals
`!
`
`Figure 4: Effect of bump map on surface shape
`Figure 4 shows a precomputed bump map to simulate
`brick structures. Notice the u and v projections of line |,
`and L. The first line (shape) depicts surface orientations of
`the cut along 1,,, |, on the brick structure. The secondline
`shows the displacement Au, Av. Positive Au, Av val-
`ues represent a displacement of the underlying pixel nor-
`mal
`in direction of u,v; negative values represent
`opposite directions. The third line showsthe resulting sur-
`face shape and its perturbed surface normals. Notice the
`effect of the displacement between two brick stones to
`simulate the groove.
`
`Transformation of the bump map onto an object is done
`similar to texture mapping. By using homogenouscoordi-
`nates the resulting address - = is the perspectively cor-
`rect pixel index on the bump map.
`
`To guarantee a perturbation in the desired direction u, v,
`the object coordinate system must be transformed such
`that the w -axis equals the surface normal vector direction
`N and the u - and v - coordinate axis are rotated about
`N according to the polygon mapping defined by the mod-
`eler.
`
`Figure 3 illustrates the mapping of the polygon
`[xy 9.2.4, 17,
`
`The resulting normal is used to calculate the pixel color in
`the Phong illumination model described in the previous
`sections.
`
`4. Proposed Architecture
`
`The scan-converting procedure for triangles is described
`in detail in [11]. Given the triangle’s start coordinate with
`its start attributes and slope increments, the Scan Line Ini-
`tializer computes the relevant data at the start point of
`every new scanline, while the Pixel Calculator interpo-
`lates the coordinates and attributes within a scanline. This
`differs from the former design in the interpolation of an
`extended data set. In particular, we extend the interpolator
`to homogenous x, y, Z, w space, which is necessary for
`achieving perspective texture mapping [9],
`texture-,
`bump-coordinates (u,v) , eye vector [e,, eye)
`-
`
`After rasterization, the obtained pixel fragment describes
`the position and orientation of small surface elements in
`*
`:
`.
`ke
`Y¥
`3D-space, the direction of reflection, the a yp texture and
`bump coordinates.
`
`Figure 5 shows a sketch of the proposed architecture.
`
`The non-shaded boxes will be discussed in more detail.
`The operation principle of the interpolation-units is pre-
`sented in [11]. To visit the pixels of adjacent triangles
`only once, the edge traverser was slightly modified [7].
`This is essential for color blending.
`
`[5 Yas 22 Wo] r [x3, Ya 23, Ww] T in the clip coordi-
`uy v7
`to the edges ww]?
`[ts MTT
`.
`in the texture coordinate sys-
`|W,’ Ww;
`
`nate
`
`system 4X, y, Z,W
`Tr
`Hy va)?
`wa’ Ww]?
`tem u,v, Ww.
`
`T
`
`66
`
`MEDIATEK,Ex. 1028, Page 4
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 4
`IPR2018-00102
`
`

`

`Input Parameter Set
`
`Selection of reflection map:
`
`lng tym]
`
` |}. sign (n,, 2,7,)
`
`Address function:
`
`
`uvtmerpolation, celationNormal.&ceVector =ofre=Zzmales
`
`
`X — faces:|1, —5, ae
`
`al’ PP
`
`
`n,
`
`1,
`
`Y~ faces: fa) 1, ih ;
`Z~faces: ot mI1
`
`4.1. Division with on chip ROM
`
`Perspective correct texturing and calculating the intersec-
`tions of the normal and reflection vectors with the cube
`faces need one division per pixel. A common approachis
`to build the reciprocal = followed by a multiplication.
`Convergencedivision algorithms normalize the divisorto
`a positive interval, typically 0,5 < w< 1
`; positive num-
`bers are shifted up or down by a priority encoded shifter
`code. Negative numbers are first converted to signed pos-
`itive values (complement +1). The result = is corrected by
`the exponentof the normalization.
`
`In the following we focus on the computation ofthe recip-
`rocal = by Taylor-series approximation [14] about
`w = w,. After the normalization step, we can write:
`
`w= O,lwyw,. Ww, Wye We,
`
`w, = Ol wows...w,,_yw,,1.-L.
`
`With Aw = w-w,<0, one can develop
`
`the number of
`For a reasonable hardware realization,
`terms in the sum has to be small and the on-chip ROM
`implementation has to be memory efficient.
`
`Truncating after the third term of the sum and ysing B as
`the output of the look up table approximating ww, 7 one can
`rewrite the above formula:
`
`= B+B’-Aw+B°- Aw’ +error
`
`1—
`
`=
`
`= B-(1+B-Aw- (1+B-Aw)) +error
`
`67
`
`MEDIATEK,Ex. 1028, Page 5
`IPR2018-00102
`
`fromBumpETretS
`
`
`materialcomponentKd,Ks,Colorjioht
`
`
`Reflectance cubes
`Addressing-unit
`
`
`
`texturecolor
`
`Zz
`y
`x
`color
`Figure 5: Overall blockdiagram of the rendering engine
`The bump engine perturbs the pixel normal by the corre-
`sponding bump map entry Au, Av . The bump mapis
`aligned perpendicular to the surface normal. This is done
`by multiplying the bump mapentries by A:
`
`
`
`Figure 6: Bump Engine
`
`>
`
`>
`
`N= N+ [Au, Av ‘A
`
`The reflectance cubes are addressed by the normal for the
`diffuse color component or by the reflected ray for the
`specular component. The address calculation unit for the
`reflected ray can be foundin [13]. The vector’s major axis
`determines the corresponding reflectance cube face.
`Indexing this map is done by dividing the vector’s minor
`axis values by the major axis value.
`
`MEDIATEK, Ex. 1028, Page 5
`IPR2018-00102
`
`

`

`ative to the surface normal direction. To calculate the
`correct perturbed normals for each per pixel normal,
`recalculation of transformation matrix A for each pixel
`is necessary. Although the perturbation of pixel normals
`varying from the surface normalis not correct,it causes
`only a slight movementof the centroid of the highlight.
`
`In Figure 8, spherical interpolation of a per pixel normal
`(black) is shown. Given the shown lightsource and the
`viewer directions the centroid of the highlight
`lies
`exactly in the middle of the polygon. Applying linear
`interpolation (gray) the normal pointing exactly to the
`lightsource and the vieweris located slightly on the left
`causing the highlight to moveto theleft. In general this
`effect is not visible in animated scenes.
`
` ei
`
`=awe
`centroid of highlight
`
`Figure 8: Spherical interpolation versus linear interpolation
`
`normalized W
`aboveis that all normals on a polygon are perturbedrel- ac
`
`
`Theerroris a function of the numberof sum-terms f, the
`numberof look-up-table entries m-1 and b the number
`of bits per word B. (Remember: m are the relevantbits
`of Ww, = O,lw5w4...W,,_ Ww, 11)
`
`Figure 7 showsthe division by the homogenousfactor
`w. With technology currently available, first simula-
`tions estimate an area of ca. 25mm’ for this unit. The
`division of the vector’s major axis needs less accuracy,
`resulting in a smaller silicon area.
`
`Figure 7: Divider for 1/w
`
`5. Results
`
`One limitation of the bump mapping method described
`
`Figure 9: Mexican Head. Structure carved in tin plane
`
`68
`
`MEDIATEK,Ex. 1028, Page 6
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 6
`IPR2018-00102
`
`

`

`Another disadvantage caused by the calculation of A
`
`relative to the surface normal is shown in Figure 10:
`
`Figure 10: False illumination at edge of neighboring poly-
`gons.
`
`Simulating extreme curvature of adjacent polygon
`meshes by modeling vertex normals leads to machband
`effects at the edges. The vertex normal v, of mesh 1 is
`perturbed dependent on the surface normal (dashed nor-
`mal). The same rate of perturbation is applied to v, of
`mesh 2 perpendicular to the surface normal of mesh 2
`(dashed gray normal). Assuming the vertex normal
`direction to be the perfect mirror direction, slight varia-
`tions of vertex normals v, and v, generate different
`shading intensities. These shading intensity variations
`can be seen clearly along the edge revealing triangula-
`tion structures.
`
`to avoid
`Therefore the modeler’s responsibility is
`extreme curvature simulations or to use finer triangula-
`tion of these scene parts
`
`Figure 9 showsa bluish tin plane illuminated by a purple
`(upper left) and a white (front) lightsource. The planeis
`assembled out of two bumptextured triangles. A Mexi-
`
`can head functionis used to calculate the normal pertur-
`bation for a 40x40 bump map (see Figure 9 right).
`Applying the bump mapto the plane repeatedly leads to
`the desired carving. The zoomed patch in the middle of
`Figure 9 clearly shows correct purple and white high-
`lights given by the illumination geometry.
`
`In Figure 11 the creation of complex images using con-
`ventional textures and bump textures is shown. In our
`example we rebuilt a small part of the “Berlin Wall”
`painting a graffiti texture (Figure 11 middle). To simu-
`late carving and rotting of the wali we created a grey-
`scale image which was converted to a bump map. Notice
`the writing “Berlin Haup(t)stadt” chiseled in the wall
`(see Figure 11 left (bump sketch) and right (resulting
`image)). An animation of the “Berlin Wall” example
`scene generates the impression of moving sky trackers
`overthe rotten wall.
`
`6. Conclusions and Future Work
`
`We have designed and simulated a real time renderer
`supporting unique features like Phong shading and bump
`textures. With the presented renderer weare capable of
`generating animated scenes of higher quality then with
`conventional gouraud shading renderers.
`
`This technique could be improved further by:
`
`*
`
`techniques to simulate spherical interpolation of un-
`normalized normal vectors to avoid slight translations
`of highlights on polygons;
`
`* new hardware adapted methods to avoid artifacts
`caused by angle variations of neighboring polygons;
`
`* artifact reduced MIP mapping for bumptextures;
`
`*
`
`implementation of “living“ textures;
`
`eS
`
`Bumpsketch
`
`Figure 11: Simulation of graffiti on rotten wall.
`
`MEDIATEK,Ex. 1028, Page 7
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 7
`IPR2018-00102
`
`

`

`
`
`Hardware”, Siggraph Proceedings 1994, pp 163-166
`D. Wong, M.Flynn,“Fast Division Using Accurate Quo-
`tient Approximations to Reduce the NumberofIterations”,
`IEEE TRANSACTION ON COMPUTERS, VOL.41, NO.
`8, pp. 981 - 995, August 1992
`
`* specification of a set of extensions for the OpenGL
`graphics language to support our unique graphic fea-
`tures.
`
`[14]
`
`Furthermore, it is desirable to switch between conven-
`tional texturing, bump texturing, reaction diffusion textur-
`ing and displacement mapping [2].
`
`7. Acknowledgments
`
`We would like to thank the following people for their sug-
`gestions and contributions to this work: J. Duenow (scan-
`liner and parameter generation), L. Guangming (error
`analysis on division algorithms), T. Le Vin (Verilog simu-
`lation of arithmetic units, including silicon real estate esti-
`mation), and S. Budianto (Register Transfer Level
`Simulation). In addition, we would like to thank the many
`people who have commented on this paper.
`
`Literature
`
`{]
`
`[2]
`
`[3]
`
`[4]
`
`{5]
`
`[6]
`
`[7]
`
`[8]
`
`[9]
`
`{10}
`
`{lJ
`
`[12]
`
`[13]
`
`J. FBlinn, “Simulation of Wrinkied Surfaces”, Computer
`Graphics, 12(3), pp. 286-292, (Proc. SIGGRAPH °78)
`B.G. Becker, N.L. Max, “Smooth Transitions between
`Bump Rendering Algorithms”, SIGGRAPH ’93, pp 183 -
`190
`
`G. Bishop, D. M. Weimar, “Fast Phong Shading”, Computer
`Graphics, Vol. 20, No. 4, pp. 103 - 106, 1986
`Phong,Bui Thong.“Illumination for Computer Generated
`Pictures”. Communications ofthe ACM, Vol. 18, No. 6
`(1975), pp. 311-317
`U. Clausen, “Reducing the Phong Shading Method”, Proc.
`Eurographics 89, Eds. W. Hansmann, F.R.A Hopgood, W.
`StraBer; pp. 333 - 344, North-Holland, 1989
`M.Deering, S. Winner, B. Schediwy, C. Duffy, N. Hunt,
`“The Triangle Processor and Normal Vector Shader: A
`VLSI System for High Performance Graphics’, Computer
`Graphics, Vol. 22, No. 4; pp 21-30, 1988
`J. Duenow, “Strategien zur Abbildung der OpenGL Soft-
`warebibliothek auf die vorhandene VISA Hardware”, inter-
`nal Report, GMD First, Spring 1995
`J. D, Foley, A. van Dam,S. K. Feiner, J. F Hughes: “Com-
`puter Graphics: Principles and Practice”, 2nd Edition, Addi-
`son-Wesley, 1990
`Heckbert, H. P. Moreton; “Interpolation for Polygon Texture
`Mapping and Shading”, in State of the Art Computer Graph-
`ics: Visualization and Modeling; pp 101 - 111, Springer ‘91
`T. Ikedo, “A scalabie high-performance graphics processor:
`GVIP”, The Visual Computer (1995), 11 pp. 121-133,
`Springer 1995,
`D. Jackél, H. Risseler, “A Real Time Rendering System
`with Normal Vector Shading”, 9th Eurographics Workshop
`on Graphics Hardware, Oslo, Norway, 1994, pp 48-57
`J. T. van Scheltinga, J. Smit, M. Bosma,“Design of an on-
`chip reflectance map” , EG Hardware Workshop 1995,
`Maastrich, 28-29 August
`D. Voorhies and Jim Foran,“Reflection Vector Shading
`
`70
`
`MEDIATEK,Ex. 1028, Page 8
`IPR2018-00102
`
`MEDIATEK, Ex. 1028, Page 8
`IPR2018-00102
`
`