A GPU-BASED INTERACTIVE TOOL FOR FREE-FORM SURFACE DEFORMATION
BOCCALATTE, Alessandro (1) PERSIANI, Franco (2)
(1)
Università di Bologna
Facoltà di Ingegneria e-mail: boccalatte@ingfo.unibo.it
(2)
Università di Bologna
Facoltà di Ingegneria e-mail: franco.persiani@unibo.it ABSTRACT The Graphics Processing Units (GPU) recently developed by 3D industry manufacturers have reached very valuable performance levels while offering also a high programmability of their hardware pipelines. Now graphics programmers can directly configure the geometry and texture processing pipelines, by substituting standard per-vertex and per-pixel operations with their own programs. Using programmable graphics hardware usually results in a big performance gain, since the workload needed for geometry computation moves from the CPU to the graphics processor. In this work we present a software tool for free-form surface defomation based on a so called “vertex-program” running on the graphics processor. The software is provided with an interaction mechanism which is based on hand-tracking and a two-handed direct manipulation model, while the deforming tools can be interactively reshaped by modifying some key parameters describing their geometry. With this software it is possible to perform real-time deforming on surfaces represented by very large polygonal meshes (hundreds of thousands of triangles), since each vertex displacement is computed and executed directly on the graphics pipeline. Besides, the per-vertex approach does not rely on any “a priori” knowledge of the mesh representing the surface; this means the system operates in a geometry-indipendent way and therefore it is able to apply the deforming operations to any set of polygon, regardless of how vertices are connected to each other. Usability and performances of this software are investigated. Key words: Computational Geometry, Shape Modeling, Virtual Sculpting, Real-Time Simulation. Grupo temàtico: computational geometry
1. Introduction Shape modeling has been a very active research area in the field of Computer Graphics. One of the most common approach consists in trying to reach the desired shape of an object by deforming a surface initially set to some predefined shape. For instance many traditional techniques allow to modify the shape of a NURBS-like patch by adding and or moving its control points. More recent methods are based on the concept of Virtual Sculpting and FreeForm Deforming (FFD), where users can interact with a volumetric virtual body or with its external surface in a manner that is similar to playing with real world deformable materials. A widely studied case is represented by the so-called “virtual clay”, that many researchers have been working on with several different methods. In virtual sculpting and FFD there are three main approaches: surface models [5, 6], physicallybased volume models [3, 4] and simplified volume models [1, 2]. Phisically-based volume models are probably the ones where the virtual materals show the most accurate and realistic behavior, but are computationally very expensive. Simplified volume models approaches try to emulate the behavior of deformable materials by adopting very simple mathematical models that can capture the essential features of a elasticplastic behavior. They usually adopt scalar functions that represent some scalar field [1] distribution or some kind of B-Spline trivariate functions to parametrically deform space with few control points [2]. Surface models represent the most direct and intuitive way to deform surface; these methods apply some scalar functions [5] or matrix transformations [6] directly on the vertices of the polygon mesh representing the surface. The main drawback with this approach is that they usually require complex algorithms for real-time mesh refinement when applying large deformations. This is essentially due to the relatively small number of polygons that a mesh can have in order to allow a sufficient rendering speed and a good interactivity; a higher number of polygons would allow much larger deformation with good looking results, but rendering speed and interactivity would be strongly diminished. In our work we present a simple and efficient method for shape modeling of very large polyogonal meshes (up to 1 milion triangles) which exploits the power of modern graphics hardware available on consumer PCs. The core of any video board consists of a Graphics Processing Unit (GPU) that is able to draw a large number of geometric primitives per second (triangles, points, lines, etc.) by computing a set of standard operations such as lighting, rototranslations, scaling, texture-mapping, perspective trasforms, etc. These standard operations take place in two hardware blocks: the vertex pipeline (vertex shader) and the fragment pipeline (pixel shader). In recent programmable GPUs, these fixed operations can be replaced by variable user-defined sets of instructions, thus allowing for customizable behaviour of the graphics board. It is therefore possible to compile and execute custom vertex programs and/or fragment programs directly on the GPU. One advantage of such an approach is the possibility to move part of the computational workload from the CPU to the GPU, with big performance gain. Moreover the latest graphics chips can render milions of primitives per second and have a good level of parallelism, thanks to the SIMD (Single Instruction Multiple Data) architecture of their cores. Our work makes use of a few vertex programs to compute and apply displacements to the vertices of a generic polygonal mesh. Our vertex programs are also designed to perform realtime re-calculation of normals at each vertex and to simulate smooth shading accordingly. 2. Work development In the following, we first describe the architecture of our graphical framework for shape modeling, then we discuss the mathematics used for geometry processing, lately we describe the GPU-based implementation and finally some results and considerations.
2.1 Framework architecture The software we developed consists of a graphical framework for 3D visualization and interaction. An initial surface (Fig. 1) is shown to users at startup and is ready to be deformed by handling the provided virtual tools. The surface can be represented by any polygonal mesh (polygon soup), but in our experiments we used to start with a sphere made up of many hundreds of thousands triangles. Two main deforming tools are provided and users can switch between them at any time, but only one at a time can be used. Each tool defines a vector field that might be thought like a sort of force field and that is able to make surfaces deform under its action. The force field is basically characterized by a center point and an intensity function decaying with distance. Each vertex of the polygon mesh reacts to the force field by changing its position according to the local field direction and intensity.
Fig. 1 – Overview of the software (the shape shown has been obtained from a sphere)
As said before, vertex displacements are computed directly on the GPU, so the CPU is only responsible for passing the original position of vertices to the graphics boards. Deforming tools can be moved, scaled and rotated in 3D space by either linking their position and orientation to the 6 degrees of freedom of user’s tracked right hand or alternatively by using a standard mouse-based interaction. At the same time it is possible to change the point of view (rotation, translation and zoom in/out) by grabbing the virtual world with the tracked left hand, or once again with the more traditional (but less efficient and more tedious) mouse. Since both hands can be tracked with magnetic devices (Polhemous in our case), interaction turns out to be very fast and intuitive. 2.2 Mathematical model Each deforming tools defines a vector field in space: the first tool consists of an ellipsoidal force field and is well suitable for deformations that are mainly localized around a point on the surface, while the second is a torus-shaped field and aims to produce deformations that should tend to be axial-symmetric.
Fig. 2 – Vector field main features
In both cases the field is defined (Fig. 2) by its volumetric distribution and by a vector defining a main direction; the volumetric distribution actually sets the current active domain, which is the portion of space subjected to a non-zero force intensity. Every point located out of this region is not interested by the field ( the local intensity is zero ). The active domain extension can be set by the user by changing few shape parameters as it will be explained later. The main direction is instead a vector having different meanings in the two cases: in the first tool (ellipsoid) it defines the active force direction, that is the displacement direction, while in the second tool (torus) it represents the axes of cylindrical symmetry (axial-symmetry) for the torus itself. Vertex displacements are generated as a sort of reaction to the local field intensity and direction. The value of the displacement D at location (x,y,z) is given by the product of a scalar value A representing intensity and a normalized vector n representing direction.
Fig. 3 – Field Influence function
The intensity function itself is obtained as the product of a gain value and an influence function. D(x,y,z) = gain(x,y,z) * influence(x,y,z) * n(x,y,z); In our first approach we started by setting the influence factor to a discontinue function equal to unity inside the active domain, and zero outside; this did not lead to good looking results, so we chose to use a diffferent function (Fig. 3) of the form 1/(1+(d/d0)4), where d is the distance from the field center and d0 is its value at the active domain boundary, which approximated the previous one but that was not discontinue at the boundary of the active domain. When applied to compute vertex displacements, this function shows much smoother and better looking behavior. The gain function instead depends on the type of tool, as well as the normalized directional vector n. Ellipsoidal tool: in this case the gain is equal to the distance of the point (vertex) from the plane orthogonal to the main direction, which is the local field direction. It is possible to have either a “pulling” or a “pushing” effect of the field on the vertices of the surface; to swicth between them users just need to rotate the main direction by 180 degrees; this can be accomplished by twisting the right hand (or a virtual sphere in the mouse-based interface), but the same result can
be more quicly achieved by simply switching to the “action-reverse” mode with a simple button press.
Fig. 4 – Action reverse in ellipsoidal tool
In Fig. 4 we show the effects of the ellipsoidal tool on the shape with either direct or reverse modes enabled. The shape and size of the ellipsoidal active domain can be set by modifying the three axial lengths of the ellipsoid (Fig. 5). Of course if we set them all to the same value, the ellipsoid becomes a sphere. A global scaling mechanism is also provided, so that we can change the overall size of the tool without loosing the ratios already set between the three axial lengths. Toroidal tool: in this case the gain is equal to the distance of a point from the circular axes of the torus, while the local field direction is set to be radial and belonging to the circle plane. The influence function is in the same form of the ellipsoid case but the distance d is here meant as the distance from the circular axes of the torus. In fact the active domain for the torus is the volume enclosed by the torus itself.
Fig. 5 – Changing ellipsoide shape parameters
We can observe that if we change the main direction by rotating it of 180 degrees, the torus plane does not change, and as a consequence switching to action-reverse mode has no practical effects in this case. On the other hand it is still possible to switch from a pushing deformation to a pulling one (and viceversa), by moving the inner radius from a positive to a negative value. Fig. 6 shows the basic effects of the torus-shaped tool on a spherical surface.
Fig. 6 – Torus push and pull effects
2.2 GPU implemetation We developed the software in C++, using OpenGL API for graphics and the “Nvidia-CG” shading language to write the GPU-based programs. The field equations of our mathematical model have been implemented in a pair of vertex programs, one for the ellipsoidal tool and the other for the toroidal one. For each vertex program the input data are defined by the vertex position and its normal in “model-space” coordinate (as passed in by the OpenGL application), and by some “vertex-indipendent” (also known as “uniform”) parameters such as the current “model-view” matrix (telling us how the model is placed with respect to the camera), the “projection” matrix (to take into account the field of view settings) and the “model-space” tool transform matrix expressing the orientation, scale (size) and position of the tool with respect to the surface. The model-space is the frame of reference of our 3D surface that is going to be deformed and is of course different from the model-view space which is the frame of reference of the viewer. With this parameters it is possible to compute all the parameters needed by the field equations in order to find the displacement for the currently processed vertex. The output vertex position is obtained by summing the input coordinates with the displacement found, by converting it into a 4-components vector in homogeneous coordinates and by multiplying this vector by the modelview and projection matrices in order to have the final vertex position in screen coordinates. All the above calculations are performed only if a preliminary distance test is passed by the current vertex. Indeed for vertices that are very far from the active domain, displacements are not computed and the vertex program directly jumps to the final steps, allowing to save a big amount of time. Besides, it is quite easy to compute how normals change during deformation; the vertex programs indeed compute the displacement not only for the current vertex position (x,y,z), but also for a couple of auxiliary points that are located in a small volume about the current vertex, namely at (x+dx1, y+dy1, z+dz1) and (x+dx2, y+dy2, z+dz2). By comparing the displacements at the three locations (vertex and auxiliary points), it is therefore possible to extimate how the local surface is going to be reoriented; indeed the normal is given by the cross-product of the two vectors that express the differences between the two auxiliary displacements and the vertex one. Finally, output colors are computed using a standard lighting model that simulates diffuse lights by computing the dot product of the light direction with the surface normal. Therefore, vertex programs provide not only real-time mesh deformation, but also a correct shading of the surface itself. 2.3 Performance and Usability Considerations For what concerns performances, we must notice that vertex programs can change the output position of the vertices with respect to their original values, but cannot permanently replace the old positions with the new ones. This means that the graphics board can write temporary results on the frame-buffer (i.e. the screen) but cannot save them on memory. Users can see the realtime deformation effects while moving the tools near the surface, but deformations simply stop existing as soon as the tool is moved away from the surface. This is due to the fact that on current GPUs, vertex and fragment programs cannot write (save) any data to the local video memory, since they only have “read” permission. In future graphics hardware there will be both read and write access to memory from within vertex and fragment programs, but at present time we are forced to use the external OpenGL application to save the temporary results. This is accomplished by running a slightly modified copy of the vertex program also on the CPU; however the CPU vertex processing happens only when the user wants to save the latest deformation before going on with its job. Of course the CPU needs much more time to perform this task than the GPU, but this does not represent a real problem since the saving operation is not frequently needed. Typical usage requires a saving operation once every 5 - 10 seconds, and the time needed is typically not very long (far less than one second), although not comparable to the much shorter time required by the graphics processor to perform the same operations. So, while the graphics processor actually computes vertex displacements at every frame, allowing
visible real-time deformations, the CPU is instead in charge of saving partial results to system and video memory at a user-defined frequency, which is however very low as said before. The combination of intense GPU usage to show deformations “in progress” with a low CPU usage to obtain permanent deformations, results in a high performance, user-friendly tool for shape modeling. In Fig. 7 the some examples are shown, taken from our tests.
Fig. 7 – Examples of shapes obtained from a sphere
Our tests were performed on a standard PC equipped with a 2.4 GHz Pentium4 CPU and a QUADRO FX3000 graphics board with a 256MB video memory. The test surface was a spherical mesh made up of one milion triangles. The mesh is compiled into a standard OpenGL display-list at software start time and at every “saving” step, in order to avoid performance bottle-necks that would arise otherwise, due to passing vertices information from system to video memory. Our preliminary tests showed that using a display-list increased the rendering speed at least by a factor of 5. The rendering frame rate with vertex programs enabled is variable and is depending on the type of tool used and on its proximity to the surface. In all cases however it ranges from a minimum of about 14 to a maximum that is over 30 frames per second, always maintaining the interaction and the usability at very good levels. 3. Conclusions We have designed a software implementing GPU-based tools for real-time surface deformation. Two kinds of tools have been designed, an ellisoidal and a toroidal one, allowing for different deforming effetcs. Each tool has been implemented as a force field able to deform a nearby surface. Tools shape and size can be easily modified by changing few parameters. Vertex displacements and normals update are performed by a vertex program implemented in CG shading language and executed in the graphics pipeline. These tools resulted to be able to perform editing operations on very large polygonal meshes while maintaining very high rendering speed. Using very large meshes in real-time allows for extremely fine shape control and quite large deformations. Furthermore, two-handed interaction makes editing operations very easy and intuitive, allowing many degrees of freedom and the control over tools shape parameters. References [1] DEWAELE, Guillaume. Interactive Global and Local Deformation for Virtual Clay. Pacific Graphics, Canmore, Canada, October 2003. [2] RAVIV, Alon. Three-Dimensional Sculpting Via Zero Sets of Scalar Trivariate Functions. Computer-Aided Design, 32(8-9):513-526, August 2000. [3] McDONNEL, K. T. Virtual Clay: A Real-Time Sculpting System with Haptic Toolkits. 2001 ACM Symposium on Interactive 3D Graphics (2001). [4] DACHILLE, Frank. Haptic Scultping of Dynamic Surfaces. 1999 ACM Symposium on Interactive 3D Graphics (April 1999), 103-110.
[5] BILL, James R. Computer Sculpting of Polygonal Models Using Virtual Tools. In Tech. Rep. UCSC-CRL-94- 27, Baskin Center for Computer Engineering and Information Sciences, University of California, Santa Cruz, US, July 1994. [6] LLAMAS, Ignacio. Twister: A Space-Warp Operator for the Two-Handed Editing of3D Shapes. Proceedings of ACM SIGGRAPH 2003, San Diego, California, August 2003. [7] Thomas W. SEDERBERG, Free-Form Deformation of Solid Geometric Models. Computer Graphics (proceedings of SIGGRAPH ‘86), volume 20, pages 151-160, Aug. 1986. [8] KIPFER, Peter. UberFlow: A GPU-based Particle Engine. Proceedings of Eurographics Symposium on Graphics Hardware (2004). [9] LOSASSO, F. Smooth Geometry Images. Proceedings of Eurographics Symposium on Geometry Processing (2003).