INTRO 3 PROBLEM DESCRIPTION 3 SOLUTION OVERVIEW 4 FOUNDATIONS 5 Transformations 5 3D modeling - Graphics 5 STL Files 5 Voxels 6 Marching cubes 6 Data storage 7 Octree Object 7 Haptics Environment 7 Software Overview 8 OpenHaptics Toolkit 9 METHOD 10 Extending HDAPI 10 Hardware Used 11 Software Overview 11 3D Objects 12 Collision Detection 12 Forces 13 Assumptions 13 3D objects 13 Collision Detection 14 Technical Specifics 14 Kinematic Chains 14 3D Object 15 Collision Detection 16 Force Rendering 16 TCP Repositioning 17 Alignment of Workpiece 18 RESULTS 18 DISCUSSION 21 CONCLUSION 22 RECOMMENDATIONS 22 REFERENCES 24 APPENDIX A 25 Kinematic Chains 25 APPENDIX B 26 Terminology 26 APPENDIX C 27 User Manual 27 Recompiling for .Net 2003 27 Setup 28 Intro This project describes a new technique for rapid prototyping. It blends manual labor and computerized feedback to assist the physical milling process. A prototype system was created that utilizes a haptic, or force feedback, device to assist the user in the milling process. Haptics provides a physical touch when working in a computer environment. Combining sight with touch enhances the computer interaction. Haptic devices have been used in the design phase with success. Designers in a haptic environment can benefit from the haptics when creating 3D objects. Features such as corners, faces and even textures can be simulated. The surface can be pulled or pushed in software packages simulating “virtual clay”. The process described would benefit a company that uses the haptic device for 3D design. This could also been seen as an incentive for companies to purchase such a device to offset the cost to benefit. This extends the capabilities of a company by allowing in-house rapid prototyping. The turnover time between design and getting the finished prototype could be reduced. The system could allow a designer to manufacture a prototype and reduce the need for a skilled machinist. Engineers, inventors and artists could have the opportunity to experiment and have greater control over their designs. A prototype system was developed that demonstrates the different possibilities of blending manual and computerized techniques. It provides a glimpse into the future of haptics and rapid prototyping. Problem Description This project will explore using a semi-automated process as a new form of rapid prototyping. Subtractive fabrication is one type of rapid prototyping. Subtractive fabrication starts with a stock material and removes material via CNC machining. CNC machining is ideal for simple geometric shapes but complex designs can take weeks of machining time. Additive fabrication, on the other hand, is ideal for complex designs but can be expensive and require specialized equipment. This project will incorporate haptic feedback into a subtractive fabrication method. The purpose is to increase flexibility by allowing complex designs to be created by milling techniques. CNC machining is cost-effective for large batch operations and it can take time to setup and can require repositioning the work piece. Some shapes are also too difficult to produce using standard CNC machines. This means that skilled labor and valuable resources are often tied up when creating a prototype. Haptics have been used in the design phase of rapid prototyping. Programs designed for CAD/CAM environments exists that incorporate haptics. A possible extension is to allow designers to double as machinists. This system will demonstrate a system simple enough for designers to produce prototypes. Designers could create a rough prototype to get a feel for a design without having the cost in time and money that other techniques require. With improvements in tolerance the system could provide a low cost replacement to existing methods. Using this process could quicken the time to production by providing a relatively cheap in-house prototype capabilities. Solution Overview This project will create a virtual environment to assist an average user to manufacture a prototype from a stored file. The environment will provide a physical and visual support system for the material removal process. Support will be generated from the haptic feedback as well as the graphic display. A Phantom Omni haptic device’s capabilities will be extended by attaching a drilling device. The haptic device is normally used to provide feedback to the user to simulate virtual objects. A user holds the device in hand and responds to feedback when they move a cursor in a virtual system. With a drill attachment the user will remove material from a real world block (stock material) to recreate the virtual object. The software and hardware will combine to provide forces to prevent the user from removing /desired material. The question to be answered is, if it is possible to feel a virtual object is it possible to create a real world object? An environment needs to simulate a dynamic environment, both collision detection and deformable objects. The software will track and mirror real world drilling progress. An object will have to represent the real world stock material, i.e. a block of wood. This object needs to allow for a changing shape while be drawn efficiently and realistically. It will also need to prevent collisions with the desired shape. The forces rendered will have to prevent the drill bit from removing desired material. Providing these features requires a broad range of topics in the computer field. It includes resolving issues when integrating the APIs for the graphics, collision detection, a mathematics package and the haptic device API. It explores multiple data structures, such as voxels and octrees. It relies heavily on linear algebra and homogeneous transformations for the graphics, collision detection and haptics. Finally, multiple threads need to communicate as they each compete for processor time to perform in real-time. The hardware used was a Phantom Omni (haptic device) from Sensable Technologies, a computer with Firewire capabilities and a handheld drilling machine with a flexible extension shaft that connects to the drill bits. The haptic device is the basic model with 6 Degrees of Freedom (DOF) and is capable of rendering forces along 3 translational axes x, y, z. The flexible shaft with the drill bit is attached to the last rotational joint of the haptic device, the stylus pen. The computer reads in data from the haptic device and sends commands to the haptic device. Foundations Transformations 4x4 Homogeneous Transformation Matrices allow one matrix to specify both the rotation and position of a system. They can represent how to move between coordinate systems. Figure 1 demonstrates the transformation from the W coordinate system to the H coordinate system H W T Figure 1 In robotics, when moving between coordinate systems, rotations are applied first, then transformations. 3D modeling - Graphics STL Files STL files are commonly used in rapid prototyping. 3D objects can be stored in STL (STereoLithography) files. An ASCII STL file consists of a list of triangle faces that represents the 3D object. Each triangle face has a normal vector and also 3 vertexes representing each point in the triangle. This pattern is repeated until all the faces are represented. solid name facet normal 0.0 -1.0 0.0 outer loop vertex 0.0 0.0 0.0 vertex 1.0 0.0 0.0 vertex 0.0 0.0 1.0 endloop endfacet facet normal 0.0 0.0 -1.0 : : endfacet endsolid  Figure 2 Stereolithography (STL) files are a commonly used in rapid prototyping to store 3D data. Other file types are available including TRI and POLY that were developed for the UNC SWIFT collision detection package. Voxels The term voxel comes from volume element and is the 3-dimension extension of a pixel. Voxels are often used to simulate 3D data such as cat scans and MRI images. Voxels represent a volume object. Voxelization is the process of using voxels to represent stored data. Objects that are represented using polygonal shapes can be converted into a voxel object although this can cause the object to have a boxy appearance depending on the resolution. Marching cubes Marching cubes technique is a method of displaying 3D objects. This technique was originally developed to display 3D medical data, such as cat scans and MRIs . This type of data is known as volume data. Layers of 2D pictures are combined to represent a 3D object. Voxels represent a volume and are drawn as cubes. The original data must be converted to a voxel image. A problem arises when only part of a voxel should be drawn, i.e. only half of the voxel is inside of the object it represents. Marching cubes tracks vertex information of a cube. A combination of triangles is used to represent the shape. Drawing is based on the vertices that lie within the object. Figure 3  Data storage Octree Object Octrees are object-oriented data structures that recursively partition its structure. Octrees are extensions of quadtrees. A quadtree is a 2- dimension data storage technique similar to binary trees. Although instead of having 2 children, each quadtree node has 4 children. The children represent the squares remaining when dividing the current node into 4 equal sub squares. This process is repeated until the desired depth is reached. The octree uses the same principle but is moved into 3 dimensions. (Include picture/ hierarchy) This structure can reduce the number of branches visited when searching. Branches that are empty can be skipped. Higher branches keep track of children branches that are empty. Branches with a value of empty can be skipped thus reducing large sections and saving time. Haptics Environment The Phantom Omni is a small, lightweight desktop unit. The Phantom Omni is not a traditional robot. It has been compared to a robot in reverse (cite). The user holds the stylus pen in one hand and controls the movement. The haptics simulates a virtual object that is controlled by software. The haptic device is dependent on software to render forces. A force is when the haptic device resists or changes the movement of the stylus pen. A user interacts with a scene using the stylus pen. Objects can be simulated and felt by using through the stylus pen. The haptic device responds to user movements. It returns position and orientation data to the software and based on this information the software commands the forces the haptic device should render. Objects are loaded into the virtual environment, one usually being a sphere to represent a cursor. Forces occur when collisions occur. The user moves the haptic device, which in turn, moves the cursor on screen. When the virtual objects interact, the software can generate force to simulate this. Forces can simulate the contact between two surfaces, such as a sphere with a wall. The force would allow the user to feel a virtual object. Software Overview Typical programs contain three threads. The main console is the parent thread that initializes two other threads. The graphics thread is created by OpenGL and runs between 30-60 Hz. This thread handles updating the onscreen graphics and handles keyboard and mouse interrupts by registering callbacks with GLUT, a utility library for OpenGL. The other thread is the haptic thread or scheduler. This thread can run between 800 - 1600Hz and handles collision detection and force rendering. The scheduler runs at least 1000Hz for a realistic experience. (cite) (Open haptic toolkit) Communication between the graphics thread and the scheduler is achieved by using callback functions defined by the scheduler. Synchronous methods are available to pass objects between threads in a thread-safe manner. An object containing position and state data can returned from the scheduler via a synchronous callback. The graphics thread controls what the user sees. The scheduler uses collision detection to control what the user feels. Graphics and force rendering are the two main parts of visual based haptic programs. Objects can be loaded and drawn using OpenGL commands. Sensable Technologies offers a development kit that provides application program interfaces (API) to communicate with the haptic device. A high level APIs can provide collision detection and force rendering based on OpenGL. A low level API can be used to allow a programmer custom collision detection and force rendering. OpenHaptics Toolkit The OpenHaptics Toolkit (OHTK) is a developer’s kit that provides an interface with the haptic device. It is made of two main parts, the Haptic Device API (HDAPI) and the Haptic Library API (HLAPI). The HDAPI provides a low-level interface and the HLAPI is built to run on top of and alongside of the HDAPI. The HLAPI is intended to be written alongside the OpenGL API. The OHTK also provides linear algebra data structures and a basic set of mathematic functions. Figure OHTK has a two-layered approach to haptics. The bottom layer, HDAPI, hooks into the hardware level but still provides a simple interface. The HLAPI, haptics library, is built on top of the HDAPI and works alongside of OpenGL. The HLAPI offers the ability to quickly add haptics to existing programs while the HDAPI allows full control over the haptic device. HLAPI The HLAPI provides a high-level haptic interface and is designed to be familiar to OpenGL API programmers. It allows significant reuse of existing OpenGL code and greatly simplifies synchronization of the haptics and graphics threads. It also provides collision detection and force rendering. Force rendering and collision detection are handled by setting options with the HLAPI. The HLAPI uses the OpenGL buffer to extract the shapes and render forces. The programmer must provide two things: it must indicate to the HLAPI the area of code that draws the virtual objects and set material properties of an object. The shapes that will be rendered with haptics must be registered to the HLAPI but the HLAPI will calculate and render the forces automatically. HDAPI The HDAPI provides low-level access to the haptic device. It enables programmers to render forces directly and provides convenient utility features and debugging aids. (cite) If the HLAPI is not used then collision detection and force rendering must be implemented by the programmer. Method Extending HDAPI Change: Move TCP force render point Must use HDAPI Must implement collision detection Must calculate/handle force rendering Need for offset alignment Addition: Read in coord system of Work Piece Read in STL files Octree Display The user controls the haptic device with a drill attached to remove material from a real world-milling object (block material). Hardware Used King Craft hand held drill model KBM-160F capable of 35000rpm. One drill bit was a spherical cutting tool with a 3.5 mm radius. The drill has a flexible extension shaft that allows the drilling tool to move with the haptic device. The drilling tool is connected to the haptic device by a fabricated clamp. Two tool tips are used; a sphere cutting bit and a pointed drill bit. The pointed drill bit is used to calibrate the software by reading in positions and determining the offset and also to read in the dimensions of the stock material. Software Overview The software environment will mirror real world conditions while providing additional information to the user. This is a virtual environment that assists the average user to manufacture a prototype without special skills. The environment will control a support system that assists the material removal process. Support will be generated from the haptic feedback as well as the graphic display. The software is built on existing application program interfaces (API). Interfacing with the haptic device is accomplished by using the Open Haptics Toolkit (OHTK) available from Sensable Technologies with a developer’s license. The system also uses an API for collision detection (SWIFT) an When using HLAPI the haptics thread does not need to be modified. Write the program from a OpenGL point of view. The Haptics commands wrap around OpenGL commands finding collisions, shapes and textures. Want to render forces at tip of drill, not normal position. HLAPI does not support this. Based on this principle, the software will keep the drill bit from penetrating a virtual object. This will allow the user to remove material This project strictly used the HDAPI. HDAPI requires implementing collision detection. The two main modifications are the addition of collision detection and the manual force rendering. This thread interacts with the haptic device using methods provided by the HDAPI. Other threads can interact with the scheduler by calling methods… 3D Objects Octree Object Each node has children and tracks the changes of the children. For a simple example using a 3-value system the values would be empty, partially filled or completely filled. The leaf nodes would have the property of being either empty or completely filled. The branches that are empty aren’t drawn. At the branch level it would set its value depending on its children. If all the children are completely filled then the current branch value would be also completely filled. It is empty when all the child nodes are empty and partially filled when only a few children are filled. STL/SWIFT Object Collision Detection Using the HDAPI requires implementing collision detection. Collision detection is occurs at two levels. Contact can occur between the cursor object and the octree object or between the cursor object and the STL object. The collision with the octree determines if the cursor object has penetrated the shell of the octree object. The collision detection with the STL object tries to prevent the cursor object from penetrating the STL object. Octree Object – A problem arises when introducing deformable objects because its shape changes over the course of the program. The deformable object (octree object) will represent the real-world stock material. Initially it will have the same dimensions and orientation and it will resemble the current state of the stock material throughout the milling process. (The simple solution is to use Voxel technology) This is where the octree comes in. Each node has children and tracks the changes of the children. For a simple example using a 3-value system the values would be empty, partially filled or completely filled. The leaf nodes would have the property of being either empty or completely filled. The branches that are empty aren’t drawn. At the branch level it would set its value depending on its children. If all the children are completely filled then the current branch value would be also completely filled. It is empty when all the child nodes are empty and partially filled when only a few children are filled. Forces Assumptions 3D objects STL Files STL files do not have to be watertight meaning holes, gaps and inconsistencies are allowed. SWIFT requires that all objects must be watertight. Some STL files will cause errors when loading into the SWIFT package. Collision Detection STL Object using SWIFT The object loaded from the STL file is a static nonmoving object and the object’s shape does not change. It may be repositioned but it is still seen by the collision detection algorithm as a static object. This means that the collision detection package can precompute the collision space. Octree All vectors from the origin of a sphere with the length of the radius will fall along the edge of the sphere. A vector S from a point outside of the sphere to the origin will cross the border of the sphere. If the radius of the sphere is subtracted from the vector S, this will return the distance from the initial position to the border of the sphere. This distance to the edge of the sphere will provide collision detection. Using this technique we can calculate the distance from the center point of a leaf node to the edge of the sphere. A vector from the center point of the leaf node to the center point of the sphere is created. The vector is then shortened by the length of the radius. The shortened vector is placed at the center point of the leaf node. If the position lies within the coordinates of the leaf node then a collision occurs. Make a vector from the center of the cube to the center of the sphere. Subtract the radius of the sphere from the vector. Add the vector to the coordinates of the center point of the cube. If the endpoint of the vector is inside of the cube then a collision has occurred. Technical Specifics Kinematic Chains What is returned from haptics is rotation point of stylus in the World coordinate system. Coordinate systems Haptics in World. Tool Offset in Haptics Cursor(TCP) to Stock Material in World( Tool Offset to octree) Translation from default haptic TCP to desired Tooltip * haptic Transformation Matrix from world center point to TCP * inverted Transformation Matrix from world center point to origin of stock material. 3D Object STL Objects The program then uses OpenGL calllists to draw the STL object. CallLists are used for the cursor and STL objects because they have static shapes. CallLists compile the graphics code and allow the programmer to draw an object using one line of code. The objects are first read in and loaded into a temporary storage variables. The display list stores the shape after it has compiled the code. The program does not need to store the object after the shape is given to openGl. Drawing the shape is then only one line of code. Opengl lists were used to store the shape loaded from a file then delete the data structure. One line of code then draws the opengl list. Octree Object & Marching Cubes The octree object uses the marching cubes technique: to graphically smooth dynamic objects by examining the vertices of a cube. (cite)There are 2^8 possible combinations of a cube’s vertices and there are at most 5 triangles that can be drawn for a cube. SEE www. For examples of the marching cubes There is a difference between branches and leaf nodes. Leaf nodes are seen as vertices of a cube. Each branch has 8 nodes with three values for a node. Removed, inside or on the border. For displaying the marching cubes the leaf nodes are the corner points. A value of inside is seen as part of the object. A value of removed or on the border are seen as a removed vertice and not drawn. The reason for having a removed and on the border value is to reduce the number of branches to visit. There are 3 states. Inside Not drawn Inside Drawn Removed Not drawn Searches don’t go down paths where the child branch is removed. Paths also shouldn’t be visited when nothing should be drawn, i.e. the inside of the cube where it would not be visible. The on the border data type informs the upper branch that this sub branch should be visited. Collision Detection Collision detection occurs at two levels. Contact can occur between the cursor object and the octree object or between the cursor object and the STL object. The collision with the octree determines if the cursor object has penetrated the shell of the octree object. The collision detection with the STL object tries to prevent the cursor object from penetrating the STL object. Octree The drill bit is spherical and collides with a cube. The distance from the center point of the sphere to the outer shell of the sphere is always the radius. The algorithm finds intersections at the surface of the sphere, not as a solid. This means that only the outer shell of the sphere is used for collision detection. SWIFT The University of North Carolina has developed a collision detection package called SWIFT that is available for free as long as it is for non-commercial use. The integration of this package requires using their custom file types or writing custom file readers. The package offers a wide range of features. Force Rendering Forces are generated by the software and rendered in two possible cases. Force smoothing has been applied to try to soften the changes between forces. The haptic device can have problems in the difference between the last force and the current force is too large. Common errors are a buzzing noise, the device “kicking” or that is doesn’t render any force feedback. The software tracks the last two forces vectors rendered to the device and the desired force. The smoothing takes an average of these three forces. Priority Instrument Software Real World 1 Octree Distance between cursor Drill bit is within object and STL object is specified less than specified distance of distance. desired shape 2 SWIFT Collision occurs between Drill bit touches the cursor object and the the stock octree material Table Forces The Phantom Omni is capable of a .8 N continuous force and 3.3 N of peak force rendering. Octree The force generated by the collision with the octree is a resistance force. The program creates a force in the opposite direction of the motion of the cursor. When a collision occurs with the octree the contact leaf is removed. Forces do not accumulate when multiples leaves are removed. SWIFT The collision between the cursor object and the STL object takes precedence over the collision between the cursor object and the octree. This force must repel the drill bit from removing material that will make up the end shape. The SWIFT collision detection package returns the distance to collision and gives a surface normal vector from the STL object in the direction of the cursor object. The force is generated in this direction. When the cursor penetrates the surface, force-rendering stops. This provides the user with immediate feedback that the current position is an error. TCP Repositioning Forces must be rendered at the end of the drill bit. This requires W calculating the transformation matrix representing the offset TCP T, the transformation from the TCP to the World coordinate system. A technique was provided by Professor Horsch that calculated the transformation from 3 input points. This is based on solving an overdetermined system. W H W TCP T= TCP T H T Alignment of Workpiece A right-handed coordinate system can be created from two vectors. The two vectors are first made orthogonal if they aren’t already. A coordinate system must contain three vectors that are orthogonal to each other. The third vector is created by calculating the cross product of the first two vectors. The cross product of two vectors creates a vector orthogonal to the two vectors. The vectors must be read in from the stock material. The two vectors to create the coordinate system are read in from 3 positions. An origin position is picked and the remaining points are in the direction of the x- and y-axis. This process has is combined with reading the dimensions of the stock material. The origin point should be the lower corner point starting at 0. Three more points are then read in. The x- and y-axes are read in for calculating the coordinate system. The points should also represent the maximum dimension in the axis. The final point is the z dimension along the negative z-axis. Results STL Files Drillbit. -- 1 mm diameter centered around origin CenteredSphere – 3mm Diameter Torus -- 5.8mm diameter x 1.9mm thickness Preliminary tests were conducted to assess the capabilities of the system. Simple shapes were used for testing. This included a sphere and a torus. Testing of more elaborate shapes would have been attempted if the initial testing had proven promising. Four tests. The first two tests were small shapes and the second two shapes were large ones. Test 1: Torus Octree Depth: 5 Calibration Points: 3 ~1:15 Dim: 40x40x14 High Density 50x50x40 Progress during pause Finished Product Test 2: small Sphere ~1 hour. Desired diameter 15 mm. High Density 30x30x30 Test 3: Large Torus – Failed ~45 minutes through the milling process. The process was less than ¼ complete when it crashed. The program crashed when turning off force feedback in software. This was the largest attempted shape using a block of medium density stock material 80x80x40. Test 4: Large Sphere ~1:15 hours Offset Calibration Points: 5 Octree Depth: 6 45 mm diameter High Density 60x60x60 Observations: After a rough shape was created and material was clear the forces were easily determined. There were occasions that drilling errors were not found until the process of drilling was further along. It was hard to determine when the process was finished. Force feedback for collisions with the octree could not be felt. The drilling process created too much vibration to feel the feedback rendered when a collision with the octree occurred. The testing had the noise of the drill and a vacuum to collect the removed debris. The initial force feedback is hard to determine when drilling. There were occasions when the contact with the STL object could not be felt. When an error is made, force rendering stops. This gives the user immediate feedback that an error was made. Interpretation: Average Milling Time: Tests would assess: Force/Visual Feedback The accuracy of reproducing a shape Tool Offset: Ultra sensitive Implications: Work piece dimension errors. Render Forces at inconsistent positions/ depending at angle Partial Solution: Lock a degree of freedom, roll axis or spinning of haptic pen. This returns better results Misc. Topics Tool Calibration Read in Work piece Discussion: Voxel structure took too much processor time to draw. The haptics thread needs to run at 1000 Hz. There were 128^3 cells to check for drawing each cycle. The octree was introduced to speed up the drawing with the tradeoff of using more memory. Getting offset of drill bit inaccurate. Testing showed that the resolution of the rotational angles was not high enough. Initial testing showed promise. The first test was a simple torus with a diameter of 35 mm. The milling process takes 30 – 60 min. for a small shape. The shape resembles a torus. The front face looks smoother than the back face. While milling it was hard to feel the force feedback. Most errors were caused when the force rendering stopped. Force rendering should increase theoretically to infinity as the distance between the drill bit and the desired shape decreases. The haptic device is not capable of this and it is possible to push the haptic device beyond the threshold. When this happens, the force feedback goes from very strong to non-existent. In most cases this causes a hole in the desired shape. There is also a tolerance issue with the force feedback. The haptic device is set to render forces when the distance to the STL object is less than 5 mm. It can be hard to determine where the STL object is. The drill bit can penetrate the desired shape because the user can overpower the haptic forces. Evolved/Purposed Techniques: 1. Rough Shaping Start removing material on the top face then slowly working down evenly. This would rely on removing material until force feedback was encountered then moving to another area. This would create a rough shape. The fine detailing would be saved until the end. 2. Visual Based The user knows the shape of the STL object. The user can pick a feature near the edge from the virtual object and removing material until force feedback is determined. Then the user relies on force feedback to guide the drilling. The user constantly checks with the virtual display to understand the shape of the area. Tests 1 and 4 demonstrated the capabilities of this system. A smaller sphere Must be researched Stability errors, overheating, dust/debris System uses 100% processor resources. What is causing this. Fatigue Known Issues: There is an unresolved issue with OpenGL. Occasionally the octree object is not drawn correctly using the marching cubes technique. This is apparent when the object is rotated; the back face is not rendered. This might be caused by incorrect OpenGL setup options. Conclusion This project could allow the average user to create products without complex training. Designers and inventors could create their own prototypes. The support provided by the force feedback should prevent the worker from removing any of the material from the desired shape. The visual feedback would allow the user to focus on areas that require special attention. This allows the worker to interact with the design rather than focusing on the manufacturing. This project combined various areas of computer science and offered a glimpse into the future of haptics. Haptics are being incorporated into CAD/CAM environments, 3D design techniques and various areas of the medical field. Future uses and research includes generating computer controlled milling paths, simulating and performing medical procedures, and free form 3D object creation, i.e. virtual clay. Recommendations Density A density property exists in the octree structure but is currently unused. This is when collisions occur with a leaf node and should be viewed as object persistence. Currently, leaf nodes are removed after a collision between a leaf node and the cursor object occurs. The possible extension is to modify a counter when collisions occur and only remove the leaf when the counter expires. Graphics improvements The current program displays the maximum haptics workspace as the screen dimensions. This should be changed to better fit the drilling area. A zoom feature is not implemented and the current camera angle rotations are controlled by the keyboard. Ideally the haptic device should control this. The haptics could be used to position the viewing angle by capturing the motion of the haptics endpoint. Rotations could also be obtained from the orientation of the haptic device. Larger Device Stronger force feedback is needed to prevent removing desired material. Currently the forces provide the user with the knowledge that the desired shape is near. The user can overpower the force feedback. A larger haptic device from SensAble Technologies might provide larger forces. The program should be easily scalable to work with a larger device by SensAble. The other devices are capable of rendering stronger forces. The levels of force would have to be adjusted in an extended program. Most other features are similar between all SensAble devices because they all can use the OpenHaptics Toolkit and the HDAPI. Voxelization It is possible to precompute the collision space of for the STL object. This could remove using the SWIFT collision detection package with the tradeoff being that the resolution of the octree would determine the accuracy of the final shape. References  http://www.csit.fsu.edu/~burkardt/data/stla/stla.html STLA Files August 31, 2005  SWIFT documentation  MARCHING CUBES: A HIGH RESOLUTION 3D SURFACE CONSTRUCTION ALGORITHM  http://www.exaflop.org/docs/marchcubes/ind.html The Marching Cubes Algorithm, August 2, 2005  http://www.answers.com/main/ntquery;?tname=glut-1  http://www.sensable.com/products/phantom_ghost/OpenHapticsToolkit- intro.asp kaufmanvolumegraphics.pdf Introduction to Volume Graphics Arie E. Kaufman Oomeslnc97.pdf 3D Shape Representation: Transforming Polygons into Voxels Voxelization technique Volvis00.pdf Shape-based Volumetric Collision Detection 3D TOUCH™ SDK OPENHAPTICS™ TOOLKIT PROGRAMMER’S GUIDE 2004 SensAble Technologies OtaduyJainSudLinvis04, Haptic Display of Interaction between Textured Models sculptingpicture Real-time Haptic Sculpting in Virtual Volume Space Appendix A Kinematic Chains Coordinate System of World (W) -- This is identity 4x4 matrix representing Haptic device (0,0,0) Haptic (H) – Center of Rotation calculated and returned from HDAPI Tool Tip (TCP) -- Calibrated from reading in points WorkPiece (WP) -- Read in from calibration subroutine Kinematic Chain Tool Tip to World Coordinates W H W TCP T = TCP T H T Tool Tip to WorkPiece WP W W 1 TCP T = TCP T W P T T = Tool Tip x Stylus x inverse(WorkPiece) Appendix B Terminology Callback -- Function that is set to run on the scheduler. These can set responses to user events or query device state. Graphics Thread -- see Scheduler Haptic Device -- Phantom Omni 6 DOF device. Can render force feedback in 3 dimensions. Haptic Thread -- see Scheduler HDAPI --Haptic Device API. This allows low a level interface with the haptic device. HLAPI -- Haptic Library API. This builds on top of the HDAPI and allows collision detection based on openGL methods. Homogeneous Matrix Transformation– In graphics and robotics a 4x4 matrix is used to describe both the rotation and the position of a system. 3x3 is the rotation matrix with the last column being the position vector [x y z 1] T and the last row always being [0 0 0 1]. OpenGl and the Haptics Library both use column major order when referencing arrays. Octree -- Storage technique for representing 3D objects. Cubes are recursively divided into 8 equal sized cubes. There are multiple levels each level having 8 children. This storage technique is meant to enhance performance by reducing cells to check. OpenGl -- A graphics API that allows 2 and 3 dimensional manipulation. multi-language multi-platform. Column Major order Scheduler -- This is an infinite loop that responds to events based on callbacks. Events can be from user devices or based on idle time. The scheduler will execute at a set frequency. OpenGl and the HDAPI use seperate schedulers. OpenGl is around 60 Hz while the Haptic scheduler is at 1000 Hz. STL file -- STereoLithography. A triangle mesh that forms a shell of a solid object. All faces are listed. Each face has a normal vector and then 3 vectors with 3 coordinates each. STL Object -- This represents the desired shape in the virtual environment consisting of openGL triangles. The triangle data is loaded from an STL file. Forces should be rendered to prevent the tool tip to penetrate this shape. Stock Material -- Real world drilling material made from Styrofoam. This will be simulated by the Octree Object SWIFT -- University of North Carolina Chapel Hill's collision detection package. This package uses Row Major order TCP -- Tool Center Point. This is the endpoint of the kinematic chain representing the tooltip of the Haptic device. Appendix C User Manual Recompiling for .Net 2003 Open Haptics ToolKit Available with Developer’s License  provides installation, step-by-step information to build projects Additional install information http://dsc.sensable.com/3dtouch/openhaptics/faqs.asp The source code for HLU and HDU must be recompiled. The Snap Contraints and the Haptics Mouse utilities can be recompiled but it is not required for running this project. Open the project file for HLU and convert to the newer format. Go to the Build/Batch Build and build all. Close this project and repeat this for the HDU. The new rebuild library files should be copied to a new location, for example $(3DTOUCH_BASE)\lib. There are two library files to be copied from each project. One is located in the Release subfolder and the other is in the Debug folder. Set an environment variable to the path to the license file. SWIFT++ Available for Download http://www.cs.unc.edu/~geom/SWIFT++/ The project requires using a multi-threaded DLL runtime library because the OpenHaptics Toolkit requires this. The example SWIFT file is set to build using the single threaded runtime library. To test the SWIFT library the example project was build using the multi-threaded DLL option. To successfully build the example and the thesis project with the SWIFT library included, the SWIFT package must be rebuilt. Rebuilding Source Code to work with Haptics 1. Open Project file swift.dsw 2. Change Build Configuration from Debug to Release 3. Change Project Properties->C/C++->Code Generation->Runtime Library * Multi-threaded DLL (/MD) 4. Build Project * If Link errors occur see below Including the SWIFT library into Existing Projects 1. Check that Project Properties * C/C++->Code Generation->Runtime Library is set to Multi- threaded DLL (/MD) C/C++->General->Additional Include Directories add path to swift.h header file. (ex. ..\swift\include\ ) * Linker->General->Additional Library Directories add path to the rebuilt swift.lib file. (ex. ...\swift\release\ ) * Linker->Input->Additional Dependencies add swift.lib 2. Include swift.h Possible Error Messages VC++ .Net 2003 If Link errors occur and complain about missing files it is caused by switching the runtime libraries. The missing files are a part of Visual Studio 6. If you don't have Visual Studio 6, trying searching the internet for the files or find someone with a copy of these files. Copy the Library and DLL files from a Visual Studio Directory to the windows/system32 directory or place them in the project folder. Setup Hardware 1. Phantom Omni 2. Computer with Firewire Port 3. Stock material, (Styrofoam) 4. 2 types of drill bits a. spherical for cutting b. pointed for reading in position data 5. Drilling device with flexible extension shaft 6. System to stabilize work piece and haptic device a. vice a. Clamps to prevent the haptic device moving 7. Fitting to attach flexible extension shaft to omni Initialization Routine There is a sequence of events that must be followed before the actual milling process can begin. After the program is running the user must load the shape of the drilling bit (cursor object). The user must input the radius of the tool tip before loading the object. The next step is initializing the haptic device. This can be done automatically but was left as a debugging tool. This assures that the haptic device is connected to the project and gathers data for the graphics thread. When this process is complete, the haptics thread (scheduler) is running at 1000Hz. With the haptics scheduler running, the initialization process is complete. The next step is to calibrate the tool tip. The haptic device returns the tool center point (TCP) inside of the stylus pen. The TCP must be moved to the center of the drill bit, assuming a spherical drill bit. The user selects the number of points to calibrate the tool ranging from 3 – 9. This is when the operator uses the pointed drill bit. A sharp point is required because the drilling tool will rotate around a static point. The user rotates the drilling tool while keeping contact with one point. The user pushes button 1 on the stylus pen for each of the selected calibration points. When all points are read in the software calculates the offset and uses it to transform the TCP to the correct position. The same pointed drill bit is used to read in the positions of the Styrofoam work piece (stock material). The stock material must be aligned so the positive x-axis runs from left to right, the y-axis from the floor to the ceiling and the z-axis from the haptic device to the user. When the Align Workpiece routine is initiated the user must enter 4 points. The first point is the bottom left corner closest to the user. This is the origin point. To read in the point move the drill bit to the origin point and then presses and releases button 1 on the stylus pen. A user then moves to the far right, bottom corner that is closest to them to read in the x position. The y point requires reading in the top left closest corner point. The z point is the bottom left corner closest to the haptic device in the –z direction. After aligning the stock material the setup process is nearly complete. The last step is loading the 3D object from a stored file. The user can select the resolution of the octree and the scaling factor. The scaling factor should only be used in debugging situations since the object would normally be created with the desired dimensions. After the options are set the user can load a desired STL file. The STL object that is loaded must fit within the dimensions of the stock material. If nothing happens in the software, it is likely that the scaling was incorrect or the object was too large.