Hardware Supported Bump Mapping:
`A Step towards Higher Quality Real-Time Rendering
`
`I. Ernst*, D. Jackél**, H. Riisseler*, O. Wittig*
`
`*German National Research Center for Information Technology,
`Institute for Computer Architecture and Software Technology (GMD FIRST)
`**University Rostock, Hansestadt Rostock Germany
`
`Abstract
`
`Today's high-end Gouraud renderers produce nicely textured scenes
`by mapping two-dimensional images onto modeled objects in real-
`
`Wepresent a renderer which textures surfaces in the normal sense of
`the word using bumptextures to simulate wrinkled or dimpled color-
`ful surfaces. Using a simplified bump mapping method wefirst suc-
`ceeded in designing a real-time bump mapping renderer based on the
`high-quality Phong shading model.
`.
`Applying several improvements to our former Phong shading hard-
`ware we are able to walk through perspectively correct bump mapped
`scenes illuminated by colored lightsources.
`This paper describes the main building blocks of the overall architec-
`ture, including reflectance cubes to support a local viewer, a Taylor-
`series based division to calculate homogeneous coordinates and our
`hardware adapted bump mapping method.
`
`Keywords: bump mapping, reflectance cube, normal vector interpo-
`lation, hardware division,
`local viewer, unlimited colored light-
`sources, Phong shading,real-time rendering
`
`1. Introduction
`
`A contemporary "high-end" graphics workstations has a
`rendering performance oftypically one million polygons
`per second in conjunction with hardware supported
`transparency, anti-aliasing, perspective texture mapping,
`image compositing, etc. In spite of the fact that great
`steps have been taken to improvethe realistic display of
`very complex objects and scenes, many problems remain
`to be solved in order to achieve
`
`image generation rate of complex scenes (>500.000
`polygons) of at least 30-50 frames per second
`* more photorealistic image generation by means of
`hardware supported “normal vector shading” as well
`as improved ,,anti-aliasing” or ,,bump mapping”.
`linear scaleability of the rendering performance and
`an improvementof the cost-performance relationship.
`
`In the past decade, many attempts have been undertaken
`to make ,,normal vector shading” applicable for hard-
`ware implementations. M. Deering et al.[6] presented a
`simulated VLSI-approach of a normal vector shader
`(NVS)chip suitable for Phong shading [4]. Bishop and
`Weimar (3] suggested a method which approximates
`Phong’s normal interpolation and light reflection equa-
`tion by a Taylor series expansion. Bishop and Weimar’s
`fast Phong shading reduces the number of computation
`steps per pixel to only 5 additions and one memory
`access. Another alternative, suggested by U. Clausen [5],
`interpolates polar angles instead of normal vectors,
`which has the advantage that the normal vector unit
`length remains unchanged during the interpolation. Our
`normal-vector shading approach [11] is based on the
`approximation of the illumination model geometry by
`using the reflection map method. This shading tech-
`nique, which is used since 1989 in our VISA-systems
`(VISA=VISualization Accelerator), operates similar to
`the reflection vector shading hardware of Voorhies and
`Foran [13]. Instead ofthe single precalculated map of the
`VISA-shader, Voorhies and Foran propose a six-sided
`cube, which incorporates the specular spread function
`values. Similarly to the reflection map method,each face
`of the cube is indexed by an unnormalized reflection
`vector.
`
`An additional step towards photorealistic image genera-
`tion can be achieved by means of bump-mapping. Half
`of the work of hardware supported bump-mapping is
`done when using a normal vector shader instead of a
`gouraud shader. This paper is focus on the extension of
`our normal vector shading method for achieving a simple
`and efficient hardware bump-mapping support.
`
`First, we discuss the principle of hardware Phong-shad-
`ing. In section 3 the extension of this shading technique
`to achieve bump-mapped surfaces is presented. Finally,
`the hardware implementation aspect of our bump-map-
`
`MEDIATEK, Ex. 1028, Page 1
`IPR2018-00101
`
`

`

`ping method and somesimulation results are discussed
`in more detail.
`
`the four neighboring cube elements reduces point sam-
`pling artifacts.
`
`2. Hardware Phong Shading
`Phong shading as proposed in [11}, uses a planar reflec-
`tance map to “pick” the intensity.
`
`Despite the progress on planar or hemi-cube maps the
`restriction to parallel viewers could not be alleviated.
`One reason for this is that all progress is focused on the
`use of normal vectors to address cube faces. In [8], [13]
`it is proposed to use the reflection of the eye vector on
`The “picking” of the reflectance map is done as follows:
`the surface to address a six sided cube. Since the reflec-
`Xp Yn
`tion of the vieweris closely related to the specular com-
`a, = atan z,
`,@, = atan z,
`ponent of the Phong shading algorithm, this method can
`also be applied to shading. Furthermoreasix-sided cube
`Considering the simplified geometry using parallel light
`containsall possible reflection vector directions around
`and parallel viewer the amountof light reflected by an
`a fixed normal vector, the viewer can be local. As a con-
`object’s surface is given by r = f(O,, 0.) .
`sequence, this method allows for the view point to be
`changed without recomputation of the cube map.
`
`There are some disadvantages to our former approach
`using a reflectance map:
`
`° visible artifacts caused by point sampling the reflec-
`tance map;
`visible distortions at the borders of the atan map
`(angle > 80 degrees);
`
`illumination modelallowsparallel viewer only;
`
`* costly computation for reflectance map if viewer
`changes.
`Solutions to reducethe size of the planar maps are given
`by for example in [12], [10]. To avoid distortions in the
`border area of the map,it is straightforward to use a
`hemi-cube over one polygon. This algorithm is com-
`monly used for radiosity methods and to approximate
`normal vector directions. Distortions which occur at the
`comers of the hemi-cube will be reduced by using an
`arctan map in the range of -45 to 45 degrees, similar to
`
`As indicated in [13] it is possible to calculate the reflec-
`tion vector R,, without normalization ofthe surface nor-
`mal N,, and the eye vector by-multiplying both sides by
`the length of the unnormalized normal vector squared:
`
`€Q0
`Ry = 2-Ny: (Ny®E) -E- (Ny Ny)
`To calculate the diffuse component, another six sided
`cube map can be used which can be indexed by the
`unnormalized normal vector similar to the hemi cube
`approach. Calculating colors at the polygon vertices and
`bilinearly interpolating these colors over the polygon is
`not sufficient because highlights in the middle ofa trian-
`gle cannot be generated. To reduce cube map space and
`calculation time methods proposed in [12], [10] are used.
`In Figure 1 the indexing of the cube’s facesis illustrated.
`
`the reflectance map approach. Bilinear interpolation of
`
`
`
`diffuse cube
`
`Figure 1: Application of cube faces to calculate phong illumination
`
`MEDIATEK, Ex. 1028, Page 2
`IPR2018-00101
`
`

`

`Because the displacementfunction B is small compared
`with its spatial extend by differentiating we get:
`
`(EQ 2)
`
`N’(u,v) = O,xO, +B(%xo,)
`+B{o,x) +BBP)IM
`The last
`term BBOS
`Jeo and can be
`dropped.
`IM?
`
`Note that the normal to a surface at a point is given by
`N = O,,xO,, where O, and O,are the partial deriv-
`atives ofthe surface at point O lying in the tangent plane
`(see Figure 2).
`
`Figure 2:
`Tangent vectors O,, O,
`at a point on a sphere with
`
`normal vector N.
`
`From equation (EQ 2) it can be seen that implementing
`bump mapping in hardware is not possible without sim-
`plifying the calculation scheme.
`
`3.2. Hardware adapted Bump mapping
`
`Using Phong vertex normal interpolation, we already
`know the surface normal at a point. This surface normal
`N represents the orientation of the surface independent
`of the rotation about N .
`
`Our new renderer VISA II will use a bump mapping
`scheme closely related to texture mapping.
`
`3. Bump mapping
`Bumpmappingis an elegant technique to simulate wrin-
`kled or dimpled surfaces without the need to model sur-
`face properties geometrically.
`
`This is done by angular perturbing the surface normals
`according to information given in a two dimensional
`bump map.
`
`Since intensity is mainly a function of the surface nor-
`mal,the local reflection model produces local variations
`on a smooth surface dependent on the perturbation
`defined in the bump map.
`
`Bump mappingis important because it textures a surface
`in the normal sense of the word rather than modulating
`the color of flat surfaces.
`
`3.1. Classic Bump mapping
`
`Blinn (1) first developed a scheme that perturbs the nor-
`mal vector independently of the orientation and position
`of the surface. In the case of animations, otherwise it is
`obviousthat the normal at a particular point must always
`receive the same rate of perturbation. Otherwise the
`bump map detail moves as the object moves. This is
`achieved by aligning the perturbation on a coordinate
`system based on local derivatives.
`
`If O (u, v) is a function representing position vectors of
`points O on the surface of an object, wefirst add a small
`increment, derived from the bump map
`_B (u, v) to
`define O’ (u,v):
`
`O’(u,v) = O(u, v) +B (uv) i
`
`
`
`ypxuyeuuoTeuGysueN”
`
`texture coordinate system
`
`clip coordinate system
`
`MEDIATEK, Ex. 1028, Page 3
`IPR2018-00101
`
`

`

`The transformation matrix A carries out this mapping by
`projecting the local polygon coordinate system u’, v’, N
`on the texture coordinate system u, v, w.
`
`To
`
`obtain
`
`the
`
`perturbed
`
`per
`
`pixel
`
`normal
`
`T
`
`:
`
`T
`
`N= [n'y n'y n’ , Au, Av transformed by A is
`added to the surface normal N = [n,, ny n,|
`:
`n= n,+Au-Ag + Av- Ap,
`n’
`ny + Au-Ajy+Av-A),
`nv
`n,+Au-Ayy)+Av-A,,
`
`The resulting normal is used to calculate the pixel color in
`the Phong illumination model described in the previous
`sections.
`
`4. Proposed Architecture
`The scan-converting procedure for triangles is described
`in detail in [11]. Given the triangle’s start coordinate with
`its start attributes and slope increments,the Scan Line Ini-
`tializer computes the relevant data at the start point of
`every new scanline, while the Pixel Calculator interpo-
`lates the coordinates and attributes within a scanline. This
`differs from the former design in the interpolation of an
`extended data set. In particular, we extend the interpolator
`to homogenous x, y, Z, w space, which is necessary for
`achieving perspective texture mapping [9],
`texture-,
`bump-coordinates (u, v) , eye vector [eee] .
`
`After rasterization, the obtained pixel fragment describes
`the position and orientation of small surface elements in
`3D-space, the direction ofreflection, the < , = texture and
`bumpcoordinates.
`
`Figure 5 showsa sketch of the proposed architecture.
`
`The non-shaded boxes will be discussed in more detail.
`The operation principle of the interpolation-units is pre-
`sented in [11]. To visit the pixels of adjacent triangles
`only once, the edge traverser was slightly modified [7].
`This is essential for color blending.
`
`The bump map describes the perturbation Au, Av of
`normal [0, 0, 1] r in texture space u, v, W.
`
`
`
`perpixel
`bormals
`
`t
`
`Figure 4: Effect of bump map on surface shape
`Figure 4 shows a precomputed bump map to simulate
`brick structures. Notice the u and v projections ofline 1,
`and L,. The first line (shape) depicts surface orientations of
`the cut along 1,,, |, on the brick structure. The secondline
`shows the displacement Au, Av. Positive Au, Av val-
`ues represent a displacementof the underlying pixel nor-
`in direction of u,v; negative values represent
`opposite directions. The third line shows the resulting sur-
`face shape and its perturbed surface normals. Notice the
`effect of the displacement between two brick stones to
`simulate the groove.
`
`Transformation of the bump map onto an object is done
`similar to texture mapping. By using homogenous coordi-
`nates the resulting address =. = is the perspectively cor-
`rect pixel index on the bump map.
`
`To guarantee a perturbationin the desired direction u, v,
`the object coordinate system must be transformed such
`that the w -axis equals the surface normal vector direction
`N and the u - and v - coordinate axis are rotated about
`N accordingto the polygon mapping defined by the mod-
`
`Figure 3 illustrates the mapping of the polygon
`[x,o 42420, ]7,
`
`[X55 Yo 2» Ww] r, [x3, 32 Zap Wa] T inthe clip coordi-
`
`system xX, y,Z,w to the edges
`My Va)?
`[Ms v3)"
`Ww,’ Ww
`° Ww,’ W3
`tem u,v, W.
`
`,
`in the texture coordinate sys-
`
`zm ayww,
`
`MEDIATEK, Ex. 1028, Page 4
`IPR2018-00101
`
`

`

`Colorlight
`
`Zz
`y
`x
`color
`Figure 5: Overall blockdiagram of the rendering engine
`The bump engine perturbs the pixel normal by the corre-
`sponding bump map entry Au, Av . The bump map is
`aligned perpendicular to the surface normal. This is done
`by multiplying the bump map entries by A:
`
`
`
`Figure 6: Bump Engine
`
`v = Ne [Au, Av] ‘A
`
`The reflectance cubes are addressed by the normal for the
`diffuse color componentor by the reflected ray for the
`specular component. The address calculation unit for the
`reflected ray can be foundin [13]. The vector’s major axis
`determines the corresponding reflectance cube face.
`Indexing this map is done by dividing the vector’s minor
`axis values by the major axis value.
`
`Selection of reflection map:
`
`[ny ny, n — max { \n,), |n,|, \n.|} Sign (7,,%,, 1.)
`
`Address function:
`
`n
`
`n
`
`X tor ” |Bay
`Y-faces:| — fti [ny
`Z—faces:| =fi ln
`
`non
`
`4.1. Division with on chip ROM
`
`Perspective correct texturing and calculating the intersec-
`tions of the normal and reflection vectors with the cube
`faces need one division per pixel. A common approachis
`to build the reciprocal = followed by a multiplication.
`Convergence division algorithms normalize the divisor to
`a positive interval, typically 0,5 S$ w <1 ; positive num-
`bers are shifted up or down by a priority encoded shifter
`code. Negative numbers are first converted to signed pos-
`itive values (complement +1). The result = is corrected by
`the exponent of the normalization.
`
`In the following we focus on the computationof the recip-
`rocal = by Taylor-series approximation [14] about
`w = w,. After the normalization step, we can write:
`w = 0,lwiw...We_ Wa WWm ax
`n
`
`
`w, = 0,lwow,...w,,_,w,1...1.
`
`With Aw = w-w,s0, one can develop
`1_ 1
`Aw Aw bw
`wow. tat yt Tt
`
`For a reasonable hardware realization, the number of
`terms in the sum has to be small and the on-chip ROM
`implementation has to be memory efficient.
`
`Truncating after the third term ofthe sum and ysing B as
`the outputof the look up table approximating — , one can
`rewrite the above formula:
`s
`
`2 = B+B’-Aw+B’- Aw’ +error
`
`MEDIATEK, Ex. 1028, Page 5
`IPR2018-00101
`
`

`

`Theerror is a function of the number of sum-termsf, the
`numberof look-up-table entries m~1 and b the number
`of bits per word B. (Remember: m are the relevantbits
`of w, = 0,lw,w,...w,,_.w,,11)
`
`Figure 7 showsthe division by the homogenousfactor
`w. With technology currently available, first simula-
`tions estimate an area of ca. 25mm’ for this unit. The
`division of the vector’s major axis needs less accuracy,
`resulting in a smallersilicon area.
`
`ative to the surface normal direction. To calculate the
`correct perturbed normals for each per pixel normal,
`recalculation of transformation matrix A for each pixel
`is necessary. Although the perturbation of pixel normals
`varying from the surface normalis not correct, it causes
`only a slight movementof the centroid of the highlight.
`
`In Figure 8, spherical interpolation of a per pixe] normal
`(black) is shown. Given the shown lightsource and the
`viewer directions the centroid of the highlight
`lies
`exactly in the middle of the polygon. Applying linear
`interpolation (gray) the normal pointing exactly to the
`lightsource and the vieweris located slightly on the left
`causing the highlight to moveto theleft. In generalthis
`effect is not visible in animated scenes.
`
`centroid of highlight
`
`
`
`normalized W
`aboveis that all normals on a polygon are perturbedrel-
`
`
`Figure 8: Spherical interpolation versus linear interpolation
`
`Figure 7: Divider for 1/w
`
`5. Results
`
`Onelimitation of the bump mapping method described
`
`ac
`Figure 9: Mexican Head Structure carved in tin plane
`
`MEDIATEK, Ex. 1028, Page 6
`IPR2018-00101
`
`

`

`Another disadvantage caused by the calculation of A
`
`relative to the surface normal is shownin Figure 10:
`
`Figure 10: False illumination at edge of neighboring poly-
`
`Simulating extreme curvature of adjacent polygon
`meshes by modeling vertex normals leads to machband
`effects at the edges. The vertex normal v, of mesh 1 is
`perturbed dependent on the surface normal (dashed nor-
`mal). The samerate of perturbation is applied to v. of
`mesh 2 perpendicular to the surface normal of mesh 2
`(dashed gray normal). Assuming the vertex normal
`direction to be the perfect mirrordirection, slight varia-
`tions of vertex normals v, and v. generate different
`shading intensities. These shading intensity variations
`can be seen clearly along the edge revealing triangula-
`tion structures.
`
`to avoid
`responsibility is
`Therefore the modeler’s
`extreme curvature simulations or to use finer triangula-
`tion of these scene parts
`
`Figure 9 showsa bluishtin planeilluminated by a purple
`(upperleft) and a white (front) lightsource. The planeis
`assembled out of two bumptextured triangles. A Mexi-
`
`can head function is used to calculate the normal pertur-
`bation for a 40x40 bump map (see Figure 9 right).
`Applying the bump mapto the plane repeatedly leads to
`the desired carving. The zoomed patch in the middle of
`Figure 9 clearly shows correct purple and white high-
`lights given by the illumination geometry.
`
`In Figure 11 the creation of complex images using con-
`ventional textures and bumptextures is shown. In our
`example we rebuilt a small part of the “Berlin Wall”
`painting a graffiti texture (Figure 11 middle). To simu-
`late carving and rotting of the wall we created a grey-
`scale image which was converted to a bump map. Notice
`the writing “Berlin Haup(t)stadt” chiseled in the wall
`(see Figure 11 left (bump sketch) and right (resulting
`image)). An animation of the “Berlin Wail” example
`scene generates the impression of moving sky trackers
`over the rotten wall.
`
`6. Conclusions and Future Work
`
`We have designed and simulated a real time renderer
`supporting unique features like Phong shading and bump
`textures. With the presented renderer we are capable of
`generating animated scenes of higher quality then with
`conventional gouraud shading renderers.
`
`This technique could be improvedfurther by:
`
`*
`
`techniques to simulate spherical interpolation of un-
`normalized normal vectors to avoid slight translations
`of highlights on polygons;
`
`* new hardware adapted methods to avoid artifacts
`caused by angle variations of neighboring polygons;
`
`* artifact reduced MIP mapping for bumptextures;
`*
`
`implementation of “livingtextures;
`
`Texture
`
`Figure 11: Simulation of graffiti on rotten wall.
`
`MEDIATEK, Ex. 1028, Page 7
`IPR2018-00101
`
`

`

`
`
`Hardware”, Siggraph Proceedings 1994, pp 163-166
`D. Wong, M. Flynn, “Fast Division Using Accurate Quo-
`tient Approximations to Reduce the Number ofIterations”,
`IEEE TRANSACTION ON COMPUTERS,VOL.41, NO.
`8, pp. 981 - 995, August 1992
`
`© specification of a set of extensions for the OpenGL
`graphics language to support our unique graphic fea-
`tures.
`
`[14}
`
`Furthermore, it is desirable to switch between conven-
`tional texturing, bumptexturing, reaction diffusion textur-
`ing and displacement mapping [2].
`
`7. Acknowledgments
`We would like to thank the following people for their sug-
`gestions and contributions to this work: J. Duenow (scan-
`liner and parameter generation), L. Guangming (error
`analysis on division algorithms), T. Le Vin (Verilog simu-
`lation of arithmetic units, includingsilicon real estate esti-
`mation), and S. Budianto (Register Transfer Level
`Simulation). In addition, we would like to thank the many
`people who have commented onthis paper.
`
`Literature
`
`J. F. Blinn,“Simulation of Wrinkled Surfaces”, Computer
`Graphics, 12(3), pp. 286-292, (Proc. SIGGRAPH ‘78)
`B.G.Becker, N.L. Max, “Smooth Transitions between
`Bump Rendering Algorithms“, SIGGRAPH '93,pp 183 -
`190
`
`G. Bishop, D. M. Weimar,“Fast Phong Shading”, Computer
`Graphics, Vol. 20, No. 4, pp. 103 - 106, 1986
`Phong, Bui Thong.“Illumination for Computer Generated
`Pictures”. Communications ofthe ACM, Vol. 18, No. 6
`(1975), pp. 311-317
`U.Clausen, “Reducing the Phong Shading Method”, Proc.
`Eurographics 89, Eds. W. Hansmann, F.R.A Hopgood, W.
`StraBer; pp. 333 - 344, North-Holland, 1989
`M.Deering, S. Winner, B. Schediwy, C. Duffy, N. Hunt,
`“The Triangle Processor and Normal Vector Shader: A
`VLSI System for High Performance Graphics”, Computer
`Graphics, Vol. 22, No. 4; pp 21-30, 1988
`J. Duenow,“Strategien zur Abbildung der OpenGLSoft-
`warebibliothek auf die vorhandene VISA Hardware”, inter-
`nal Report, GMD First, Spring 1995
`J. D. Foley, A. van Dam, S.K. Feiner, J. F. Hughes: “Com-
`puter Graphics: Principles and Practice”, 2nd Edition, Addi-
`son-Wesley, 1990
`Heckbert, H. P. Moreton; “Interpolation for Polygon Texture
`Mapping and Shading”, in State of the Art Computer Graph-
`ics: Visualization and Modeling; pp 101 - 111, Springer ‘91
`T. Ikedo,“A scalable high-performance graphics
`GVIP”, The Visual Computer (1995), 11 pp. 121-133,
`Springer 1995.
`D.Jackél, H. Risseler, “A Real Time Rendering System
`with Normal Vector Shading”, 9th Eurographics Workshop
`on Graphics Hardware, Oslo, Norway, 1994, pp 48-57
`J. T. van Scheltinga, J. Smit, M. Bosma, “Design of an on-
`chip reflectance map” , EG Hardware Workshop 1995,
`Maastrich, 28-29 August
`
`MEDIATEK, Ex. 1028, Page 8
`IPR2018-00101
`
`