Illumination dependent refinement of multiresolution meshes

Illumination dependent refinement of multiresolution meshes Reinhard Klein,Andreas Schilling and Wolfgang StraßerWilhelm-Schickard-Institut,GRIS,Universit¨a t T¨u bingen,GermanyAbstractState of the art multiresolution modeling allows to se-lectively refine a coarse mesh of an object on the visually important parts.In this way it is possible to render the ge-ometry of a given object accurately with a minimum number of triangles.There are also approaches that use cones of normal vectors to perform illumination dependent selective refinement.This includes the correct rendering of highlights and transitions between lit and unlit areas of an object.The drawback of these approaches is that using only the normal cone for the detection gives only a very poor criteria to de-tect the problematic cases.Therefore,too much refinement has to be performed which in most cases prevents real time update rates between successive frames.In this paper we describe a new approach for better esti-mation of normal deviations between different levels of de-tail.This allows for accurate lighting with a minimum num-ber of triangles.1Introduction and previous workRecently multiresolution modeling(MRM)for simpli-cial geometric models(any dimension,non-orientable,non-manifold,non-regular)gain massive interest.Multireso-lution models provide the basis to handle,visualize and transmit over the network and edit very large datasets [14,2,9,1,4,5,12,13,15].The generation of a MRM of an object generally involves a sequence of local simplification operations like vertex re-moval,edge collapse,triangle collapse and vertex cluster-ing.The sequence of local simplification operations de-fines a sequence of coarser and coarser approximations of the original model,the MRM.In order to be able to refine a coarse approximation of the model,the inverse local simpli-fication operations are needed and the MRM must contain 1Universit¨a t T¨u bingen,Auf der Morgenstelle10/C9,72076T¨ubingen, Germany.E-mail:andreas reinhard@gris.uni-tuebingen.dehttp:www.gris.uni-tuebingen.de the necessary information.Selective refinement becomes possible if the sequence defined by the simplification pro-cess is changed.This is only possible under certain circum-stances which guarantee that the parts of the mesh onwhich Figure1.Example of a hierarchy of modifi-cation operations.The left side shows dif-ferent level of detail triangulations.On theright side the corresponding directed acyclicgraph.Operation2can only be performedafter operation1.Operation4requires op-erations1and2to be performedfirst but isindependent of operation3.the operation is performed are present in the current mesh. This requirement automatically defines a hierarchy of mod-ification operations and the corresponding triangles.This hierarchy can be considered as a directed acyclic graph of modification operations or the associated triangles.Figure 1shows an example.For illumination depended selective refinement the sur-face normals are essential.For the decision if the mesh at a certain location has to be refined further,the normals con-tained in the actual coarse approximation cannot be used. Information about the normals in the corresponding area of the original model(which can be totally different)is re-quired.Therefore,in previous work the variation of the normals in a certain area is represented by a cone of nor-mals represented by a vector together with an angle.The normal cone conservatively bounds all the normals of all the triangles in the corresponding area.This correspond-ing area is defined by several authors[16,5,10]as all the triangles in the original mesh which are descendents of the area being considered in the simplification hierarchy.How-ever,the dependencies introduced by the simplification hi-erarchy are not related to the geometry of the relevant area. Therefore,in general the area considered for the calcula-tion of the normal cone comprises in many cases more than twice the actual needed area.This leads to much larger nor-mal cones than required and therefore to unnecessary refine-ments.Therefore,we suggested in[6]not to use the hierar-chy to determine the corresponding area but instead to use a geometric correspondence.Despite of this improvement there still remains a general problem when a normal cone is used:The cone of normals does not tell anything about the distribution of the normals in the relevant area.Figure2 shows the result of lighting dependent simplification using normal cones.Note that in the area of the highlight the car bonnet is refined down to triangles of pixel size although this would be unnecessary especially if Phong-interpolation is used.If we knew that the normals over the area of a triangle can be sufficiently well approximated by a linear interpo-lation of the vertex normal(which occurs very often)the refinement could be avoided in the following cases: always,if Phong interpolation is used,if Gouraud interpolation is used and a linear brightness distribution can be assumed.In the remainder of the paper,we will introduce a suitable measure to be stored within the MRM,that allows us to detect these cases during the refinement process at render-ing time and to obtain substantially smaller approximating meshes for the same high image quality.2The simplification algorithmThe mesh simplification algorithm always starts with the finest triangulation in3D space approximating the original model surface.In the following this starting triangulation is denoted with,indicating that it contains vertices.The simplification algorithm successively simplifies the starting triangulation by removing vertices from the cur-rent triangulation and retriangulating the resulting hole or by collapsing edges or triangles.This is done until no fur-ther vertices can be removed from the simplified triangu-lation without exceeding a predefined distance between the original triangulation and the simplified one.If after a sin-gle simplification step the simplified triangulation contains vertices it is denoted byThe order in which the simplification steps are per-formed is determined by a priority queue.For every sin-gle vertex of the simplified mesh an error value is computed and stored.This value describes the potential er-ror,that is the modified one-sided Hausdorff distance that would occur between the simplified and the original mesh if the simplification step was performed.At the beginning of the algorithm this error value is computed for every vertex, edge or triangle in the original mesh.The vertices,edges or triangles of the original mesh are then sorted into a prior-ity queue according to their potential error.Afterwards,in each simplification step we successively eliminate the ver-tex,edge or triangle with the smallest potential error on top of the priority queue.If a simplification step is actually per-formed the potential errors of the neighbor elements are re-computed and the priority queue is updated.2.1Generating information about the normals 2.1.1The normals in a simplified triangulation During the simplification process the normals at the ver-tices must be recalculated to avoid alias problems due to undersampling,see Figure3.After a simplification step the normals at the vertices incident to the modified area are given bywhere enumerates all triangles incident to vertex.1 Note that as a consequence of thisfiltering procedure,the normals at the vertices of a given triangle may change dur-ing the simplification process.1In order to model sharp edges several normals at a vertex are neces-sary.These cases can be handled in a similar way taking into account only the respective incident triangles.Figure2.The upper two pictures show the illumination of a car bonnet without and with adaptive triangulation.The lower two pictures show the corresponding triangulations.Data courtesy of Volkswagen AG.Figure3.Alias problems due to sampling ofnormals.2.1.2Determination of deviation anglesTo get information about the deviation of the normals,the correspondence map between the vertices and triangles ofthe original triangulation and the triangles of the simpli-fied triangulation used for the distance calculations[9]is exploited.For a given triangle of the sim-plified triangulation this map delivers all verticesof the original triangulation that are nearer to than to any other triangle of the simplified triangulation.Each of these vertices corresponding to is projected onto the triangle In addition to these vertices we determine all vertices which are connected to one of the vertices via an edge and project them onto the plane defined by The pro-jected vertices are denoted with They are connected by the same edges as their counterparts in the original trian-gulation.For each projected triangle of which at least one vertex does not belong to the two intersection points and of their projected edges with the edges of tri-angle are determined and corresponding normal vectors and of these points in the original mesh are in-terpolated,see Figure4.In the next step for each of these vertices and intersection points the barycentric co-ordinates with respect to are computed,and using these coordinates,linearly interpolated normals based on the current normals at the vertices are computed. In a last step the deviation angles between the original normals and corresponding interpolated normals in the vertices and intersection points are computed.After computing all these angles a linear deviation functionis computed,such that The function is de-fined by its values at the verticesvV k2k3Figure4.Determination of deviation anglesby projecting original vertices on plane of tri-angle.2.1.3Deviation of normals due to normalfilteringAs described above,normalfiltering is performed by recal-culating the normals at the vertices of a simplified triangula-tion based on the adjacent triangles.Thisfiltering does not only occur during the simplification process but also during the extraction of selectively refined meshes from the MRM. During the recalculation of these normals,different constel-lations of adjacent triangles must be taken into account.The deviation between the normal vector used to determine the deviation angles(see previous paragraph)and an actual normal vector varies.To get a conservative estimation for this deviation after the simplification process all possible normals have to be computed.This can be done with a sim-ple recursive algorithm operating on the directed graph of the hierarchy.3The extraction algorithmThe goal of the multiresolution model is to minimize the number of polygons that are sent to the rendering pipeline, without affecting the quality of the resulting image.To achieve this,the multiresolution model allows the extraction of selectively refined models either by coarsening or refin-ing the current triangulation.The decision whether a given triangle may remain in the triangulation or if its surrounding area must be refined further or if a coarser triangulation of its surrounding area is possible,is the most important step in this process.For an implementation this can be reduced to the single question of whether a given triangle fulfills the quality requirements or not[8].To answer this question we have to investigate certain characteristics of that part of the original surface that corresponds to the triangle in question. In the following we call this part of the original surface cor-responding surface for simplicity.Theflow diagram of thealgorithm we use for this decision is shown infigure5.3.1Handling backfacing surface partsThefirst question in the algorithm is to decide if a trian-gle and its corresponding part on the orig-inal surface is on the backside with respect to the current viewing position.It can easily be answered in a conserva-tive way using the normals at the vertices of the triangle together with the corresponding deviation an-gles.Let be the vectors pointing from the vertices to the observer then we get the following criteria.Ifthen the surface is backfacing and the triangle need not be refined any more.In the other cases we proceed with check-ing the geometric and shading errors.3.2Controlling the geometric errorTo control the geometric error the distance(Hausdorff-distance)between original and simplified triangulation[9] must be converted to a screen space error.Several ap-proaches to this problem have been published and can be used[7,16,5,10].A further significant reduction can be achieved by avoiding the refinement outside of the view frustum.However,refinement outside the view frustum is inevitable to support fast changes of the viewing direction in VR-applications.3.3Controlling the shading errorsIn the following we assume that the Blinn-Phong shading model is used.For one light source it is given by:if;else,where and are the ambient,diffuse and specu-lar material coefficients,the corresponding lu-minaces,is the direction from the surface point to the light source,and the so called halfway vector be-tween the vector to the light source and the vector to the observer.3.3.1Only ambient light contributionsThefirst test determines for a given triangle if the cor-responding surface faces away from the light source and therefore,gets only(constant)ambient lighting.To per-form this test we determine if the normal cones at the ver-tices defined by the normals at the vertices andBackside?YesYesYesYesYesYesYesYesDo not refineDo not refineDo not refineDo not refineDo not refineNoNoNoNoNoNoNoGeometry ok?Projected TriangleSmaller than pixel?RefineRefineRefineFacing awayfrom light sourceLinear Phong Inter-polation sufficient?Phong inter-polation used?Brightness suf-ficiently linear?Lin.approx.ofnormals sufficient?α*dL/dsmall enough?αFigure5.Flow diagram of the extraction algo-rithm.the corresponding deviation angles are facing away from the direction to the light source.Letbe the vectors pointing from the vertices to the light source then we get the following criteria.Ifonly the constant ambient term has to be considered and the triangle is not refined further2.3.3.2Is linear Phong interpolation sufficient?In the next test we determine if linear Phong interpolation on the given triangle adequately describes the illumination of the corresponding surface.As a criteria we assume that it can be tolerated if the correct color in a pixel is replaced by a color that would be correct on a point,that is located in a one pixel-wide neighborhood of the pixel itself.In other words,the deviation angle,introduced by the varying nor-mals on the corresponding surface must be less than the minimal deviation between Phong interpolated normals of two points that are one(screen-)pixels apart3.Let respectively,be the partial derivatives of in the directions of(screen-)and and the deriva-tive of in the direction of.Now,we have to guarantee that a deviation of the normal by an angle in an arbitrary direction is always less than the minimum ab-solute deviation value determined by and.To get (and with that)we could solve the corresponding eigen-value problem,see Figure6.To avoid this computation we use the simple conservative bound for(and therefore for )based on a projection in the normal vector space, see Figure6.If we get the following formula Otherwise,is replaced by For sake of simplicity,for and in this calculation the Phong increments can be used.Up to this point the angular deviations are con-trolled.Therefore,the resulting meshes cannot only be used for Phong-illumination but also in ray-tracing applications, or with environment mapping which is of special interest in the CAD-industry to visualize the quality and smoothness of free-form surfaces.If linear interpolation of the normals is not sufficient, there still may be cases where refinement may be unneces-sary,since the gradient of the intensity may be so small that 2Although cases can be constructed where these nine tests are re-quired,they can normally be reduced to three testswith only marginal effects on the resulting illu-mination3To be correct the deviation of the halfway vector should also be con-sidered.However,we neglect the deviation of this vector in a one pixel-wide neighbourhood.Figure6.Finding the smallest deviation in di-rectional deviation space.in larger neighborhoods of the pixel or even on the whole triangle the differences of intensities cannot be recognized by an observer.This cases are detected in a subsequent test. For this purpose let us assume that we use a Phong model. Than the original normals deviate from the interpolated nor-mals at most by an angle of Now we have tofind an up-per bound for the changes in the brightness that might be caused by a variation of the normals by an angle of in ar-bitrary direction.To end up this we consider the derivatives of the diffuse term and of the specular term,whereand Then the changes that can be caused by an angle are bounded bywhere andAsis a monotonic function of the maximum deviation can be found at where is the angle between and at the darkest(diffuse)point in the triangle.For simplicity we make the reasonable assumption that the curvature of the lines of constant intensity never change sign4.Under this assumption the darkest point in the triangle will be one of its vertices.For the specular term things are more complicated asis not monotonic any more,see Figure7.4Note that this is not the case,if we have multiple highlights on one Phong-interpolated surface.But this is only the case,if the viewing point is close to the focal point or close to a caustics and then only slight changes of the viewing position will change the illumination dramatically.Errors introduced by coarser triangulations because of this assumption are hardly noticeable,even in animations.Figure7.Plot of the function.Thederivative has a maximum absolute value atFigure8.The minimal angle between thecone of the normals and the cone of halfwayvectors deliver a bound for at the brightestpoint.On the range the maximum derivative occurs at.But this value is only of interest if.In the other cases the maximum derivative occurs either at(brightest point)or (darkest point).Again,the darkest point is lo-cated at one of the vertices and can be used to determine if Tofind the brightest point is more compli-cated.Therefore,we determine using the cone of the normals and the cone of the halfway vectors of the triangle [6],see Figure8.In both cases it is now easy to check if the changes of the brightness on the triangle are acceptable or not.3.3.3Is Gouraud interpolation sufficientAfter performing the Phong test,we know that the nor-mals of the corresponding surface are sufficiently approx-imated by the interpolated normals.If we have a renderer that supports Phong interpolation we are done.In the other case we suppose that we have graphics hardware supporting Gouraud shading.Now,the only task left is to determine, if the Phong interpolation can be adequately replaced by a simple Gouraud interpolation.This task has been investi-gated by[3,11].Although their algorithms do not cover all possible cases in our context the algorithm of Harrison et. al.[3]delivers acceptable results.4Conclusions and future workIn this paper we describe an algorithm for lighting-dependent refinement of multiresolution surface models. The given algorithm is not restricted to a special represen-tation of the multiresolution model and can be used in com-bination with all currently available MRMs,as long as the normal deviations are calculated as described in the paper. It is thefirst algorithm that can be used to check if Phong-interpolation on a triangle is sufficient.Moreover,it can be used to control refinement in combination with environment mapping,which is demanded by the automobile industry in order to check the quality and smoothness of their surfaces. Last but not least,the algorithm can be applied to produce optimally simplified surface models of glossy surfaces for ray-tracing with normal interpolation.Despite of the already available algorithms to check if Gouraud-interpolation can replace Phong-interpolation fur-ther work in this area is needed to handle all cases without artifacts.5AcknowledgementsWe wish to thank two of our anonymous reviewers,who provided us with exceptionally thorough reviews. References[1] A.Ciampalini,P.Cignoni,C.Montani,and R.Scopigno.Multiresolution decimation based on global error.Techni-cal Report CNUCE:C96-021,Istituto per l’Elaborazione dell’Informazione-Consiglio Nazionale delle Ricerche, Pisa,ITALY,July1996.[2]J.Cohen,A.Varshney,D.Manocha,G.Turk,H.Weber,P.Agarwal,F.P.Brooks,Jr.,and W.Wright.Simplification envelopes.In H.Rushmeier,editor,SIGGRAPH96Confer-ence Proceedings,Annual Conference Series,pages119–128.ACM SIGGRAPH,Addison Wesley,Aug.1996.held in New Orleans,Louisiana,04-09August1996.[3]K.Harrison,D.A.P.Mitchell,and A.H.Watt.The H-test,a method of high speed interpolative shading.In N.Magnenat-Thalmann and D.Thalmann,editors,New Trends in Computer Graphics(Proceedings of CG Interna-tional’88),pages106–116.Springer-Verlag,1988.[4]H.Hoppe.Progressive meshes.In Computer GraphicsProceedings,Annual Conference Series,1996(ACM SIG-GRAPH’96Proceedings),pages99–108,1996.[5]H.Hoppe.View-dependent refinement of progressivemeshes.In Computer Graphics Proceedings,Annual Con-ference Series,1997(ACM SIGGRAPH’97Proceedings), pages189–198,1997.[6]R.Klein.Multiresolution representations for surfacesputers&Graphics,22(1),Jan.1998.to ap-pear.[7]R.Klein,D.Cohen-Or,and T.H¨u ttner.Incremental view-dependent multiresolution triangulation of terrain.In Pacific Graphics’97Proceedings.IEEE Computer Society,IEEE Computer Society Press,1997.[8]R.Klein and J.Kr¨a mer.Multiresolution representations forsurface meshes.In W.Straßer,editor,Proceedings of the Spring Conferece on Computer Graphics,pages57–66,June 1997.held in Budmerice,Slovakia,05-08June1997. [9]R.Klein,G.Liebich,and W.Straßer.Mesh reduction witherror control.In R.Yagel,editor,Visualization96.ACM, November1996.[10] D.Luebke and C.Erikson.View-dependent simplificationof arbitrary polygonal environments.In T.Whitted,edi-tor,SIGGRAPH97Conference Proceedings,Annual Con-ference Series,pages199–208.ACM SIGGRAPH,Addison Wesley,Aug.1997.ISBN0-89791-896-7.[11]J.P¨o psel and C.Hornung.Highlight shading:Lightingand shading in a PHIGS+/puters& Graphics,14(1):55–64,1990.[12]R.Ronfard and J.Rossignac.Full-range approximationof triangulated puter Graphics Forum, 15(3):C67–C76,C462,Sept.1996.[13]J.Rossignac and P.Borrel.Multi-resolution3d approxi-mation for rendering complex scences.In B.Falcidieno and T.L.Kunii,editors,Modeling in Computer Graphics: Methods and Applications,pages455–465.Springer Verlag, 1993.[14]W.J.Schroeder,J.A.Zarge,and W.E.Lorensen.Decima-tion of triangle meshes.In E.E.Catmull,editor,Computer Graphics(SIGGRAPH’92Proceedings),volume26,pages 65–70,July1992.[15]G.Turk.Re-tiling polygonal surfaces.In E.E.Catmull,editor,Computer Graphics(SIGGRAPH’92Proceedings), volume26,pages55–64,July1992.[16]J.C.Xia and A.Varshney.Dynamic view-dependent sim-plification for polygonal models.In H.Rushmeier,editor, Visualization’96Proceedings,volume30(4),Aug.1996.。