Document Sample
rD Powered By Docstoc
					INTRO                      3



FOUNDATIONS                5

Transformations             5

3D modeling - Graphics      5
  STL Files                 5
  Voxels                    6
  Marching cubes            6

Data storage                7
  Octree Object             7

Haptics Environment         7
  Software Overview         8
  OpenHaptics Toolkit       9

METHOD                     10

Extending HDAPI            10
  Hardware Used            11
  Software Overview        11
  3D Objects               12
  Collision Detection      12
  Forces                   13

Assumptions                13
  3D objects               13
  Collision Detection      14

Technical Specifics        14
  Kinematic Chains         14
  3D Object                15
  Collision Detection      16
  Force Rendering          16
  TCP Repositioning        17
  Alignment of Workpiece   18

RESULTS                    18

DISCUSSION                 21

CONCLUSION                 22
RECOMMENDATIONS               22

REFERENCES                    24

APPENDIX A                    25

Kinematic Chains              25

APPENDIX B                    26

Terminology                   26

APPENDIX C                    27

User Manual                   27
  Recompiling for .Net 2003   27
  Setup                       28
         This project describes a new technique for rapid prototyping. It blends
manual labor and computerized feedback to assist the physical milling process. A
prototype system was created that utilizes a haptic, or force feedback, device to
assist the user in the milling process. Haptics provides a physical touch when
working in a computer environment. Combining sight with touch enhances the
computer interaction.

        Haptic devices have been used in the design phase with success.
Designers in a haptic environment can benefit from the haptics when creating 3D
objects. Features such as corners, faces and even textures can be simulated. The
surface can be pulled or pushed in software packages simulating “virtual clay”.

       The process described would benefit a company that uses the haptic
device for 3D design. This could also been seen as an incentive for companies to
purchase such a device to offset the cost to benefit. This extends the capabilities
of a company by allowing in-house rapid prototyping. The turnover time between
design and getting the finished prototype could be reduced. The system could
allow a designer to manufacture a prototype and reduce the need for a skilled
machinist. Engineers, inventors and artists could have the opportunity to
experiment and have greater control over their designs.

        A prototype system was developed that demonstrates the different
possibilities of blending manual and computerized techniques. It provides a
glimpse into the future of haptics and rapid prototyping.

Problem Description
        This project will explore using a semi-automated process as a new form of
rapid prototyping. Subtractive fabrication is one type of rapid prototyping.
Subtractive fabrication starts with a stock material and removes material via CNC
machining. CNC machining is ideal for simple geometric shapes but complex
designs can take weeks of machining time. Additive fabrication, on the other
hand, is ideal for complex designs but can be expensive and require specialized

       This project will incorporate haptic feedback into a subtractive fabrication
method. The purpose is to increase flexibility by allowing complex designs to be
created by milling techniques. CNC machining is cost-effective for large batch
operations and it can take time to setup and can require repositioning the work
piece. Some shapes are also too difficult to produce using standard CNC
machines. This means that skilled labor and valuable resources are often tied up
when creating a prototype.

       Haptics have been used in the design phase of rapid prototyping.
Programs designed for CAD/CAM environments exists that incorporate haptics.
A possible extension is to allow designers to double as machinists. This system
will demonstrate a system simple enough for designers to produce prototypes.

        Designers could create a rough prototype to get a feel for a design without
having the cost in time and money that other techniques require. With
improvements in tolerance the system could provide a low cost replacement to
existing methods. Using this process could quicken the time to production by
providing a relatively cheap in-house prototype capabilities.

Solution Overview
       This project will create a virtual environment to assist an average user to
manufacture a prototype from a stored file. The environment will provide a
physical and visual support system for the material removal process. Support will
be generated from the haptic feedback as well as the graphic display.

         A Phantom Omni haptic device’s capabilities will be extended by
attaching a drilling device. The haptic device is normally used to provide
feedback to the user to simulate virtual objects. A user holds the device in hand
and responds to feedback when they move a cursor in a virtual system. With a
drill attachment the user will remove material from a real world block (stock
material) to recreate the virtual object. The software and hardware will combine
to provide forces to prevent the user from removing /desired material. The
question to be answered is, if it is possible to feel a virtual object is it possible to
create a real world object?

         An environment needs to simulate a dynamic environment, both collision
detection and deformable objects. The software will track and mirror real world
drilling progress. An object will have to represent the real world stock material,
i.e. a block of wood. This object needs to allow for a changing shape while be
drawn efficiently and realistically. It will also need to prevent collisions with the
desired shape. The forces rendered will have to prevent the drill bit from
removing desired material.

        Providing these features requires a broad range of topics in the computer
field. It includes resolving issues when integrating the APIs for the graphics,
collision detection, a mathematics package and the haptic device API. It explores
multiple data structures, such as voxels and octrees. It relies heavily on linear
algebra and homogeneous transformations for the graphics, collision detection
and haptics. Finally, multiple threads need to communicate as they each compete
for processor time to perform in real-time.

         The hardware used was a Phantom Omni (haptic device) from Sensable
Technologies, a computer with Firewire capabilities and a handheld drilling
machine with a flexible extension shaft that connects to the drill bits. The haptic
device is the basic model with 6 Degrees of Freedom (DOF) and is capable of
rendering forces along 3 translational axes x, y, z. The flexible shaft with the drill
bit is attached to the last rotational joint of the haptic device, the stylus pen. The
computer reads in data from the haptic device and sends commands to the haptic


       4x4 Homogeneous Transformation Matrices allow one matrix to specify
       both the rotation and position of a system. They can represent how to
       move between coordinate systems. Figure 1 demonstrates the
       transformation from the W coordinate system to the H coordinate system

                                                W   T
                                              Figure 1

       In robotics, when moving between coordinate systems, rotations are
       applied first, then transformations.

3D modeling - Graphics
       STL Files

       STL files are commonly used in rapid prototyping. 3D objects can be
       stored in STL (STereoLithography) files. An ASCII STL file consists of a
       list of triangle faces that represents the 3D object. Each triangle face has a
       normal vector and also 3 vertexes representing each point in the triangle.
       This pattern is repeated until all the faces are represented.

       solid name
         facet normal 0.0 -1.0 0.0
            outer loop
                vertex 0.0 0.0 0.0
                vertex 1.0 0.0 0.0
                vertex 0.0 0.0 1.0
  facet normal 0.0 0.0 -1.0
endsolid [1]
                               Figure 2

Stereolithography (STL) files are a commonly used in rapid prototyping to
store 3D data. Other file types are available including TRI and POLY that
were developed for the UNC SWIFT collision detection package.

The term voxel comes from volume element and is the 3-dimension
extension of a pixel. Voxels are often used to simulate 3D data such as cat
scans and MRI images. Voxels represent a volume object.

Voxelization is the process of using voxels to represent stored data.
Objects that are represented using polygonal shapes can be converted into
a voxel object although this can cause the object to have a boxy
appearance depending on the resolution.

Marching cubes

Marching cubes technique is a method of displaying 3D objects. This
technique was originally developed to display 3D medical data, such as cat
scans and MRIs [3]. This type of data is known as volume data. Layers of
2D pictures are combined to represent a 3D object.

Voxels represent a volume and are drawn as cubes. The original data must
be converted to a voxel image. A problem arises when only part of a
voxel should be drawn, i.e. only half of the voxel is inside of the object it
represents. Marching cubes tracks vertex information of a cube. A
combination of triangles is used to represent the shape. Drawing is based
on the vertices that lie within the object.
                                        Figure 3 [4]

Data storage

     Octree Object

     Octrees are object-oriented data structures that recursively partition its
     structure. Octrees are extensions of quadtrees. A quadtree is a 2-
     dimension data storage technique similar to binary trees. Although instead
     of having 2 children, each quadtree node has 4 children. The children
     represent the squares remaining when dividing the current node into 4
     equal sub squares. This process is repeated until the desired depth is
     reached. The octree uses the same principle but is moved into 3

               (Include picture/ hierarchy)

     This structure can reduce the number of branches visited when searching.
     Branches that are empty can be skipped. Higher branches keep track of
     children branches that are empty. Branches with a value of empty can be
     skipped thus reducing large sections and saving time.

Haptics Environment

     The Phantom Omni is a small, lightweight desktop unit. The Phantom
     Omni is not a traditional robot. It has been compared to a robot in reverse
     (cite). The user holds the stylus pen in one hand and controls the
     movement. The haptics simulates a virtual object that is controlled by
The haptic device is dependent on software to render forces. A force is
when the haptic device resists or changes the movement of the stylus pen.
A user interacts with a scene using the stylus pen. Objects can be
simulated and felt by using through the stylus pen.

The haptic device responds to user movements. It returns position and
orientation data to the software and based on this information the software
commands the forces the haptic device should render. Objects are loaded
into the virtual environment, one usually being a sphere to represent a
cursor. Forces occur when collisions occur.

The user moves the haptic device, which in turn, moves the cursor on
screen. When the virtual objects interact, the software can generate force
to simulate this. Forces can simulate the contact between two surfaces,
such as a sphere with a wall. The force would allow the user to feel a
virtual object.

Software Overview

Typical programs contain three threads. The main console is the parent
thread that initializes two other threads. The graphics thread is created by
OpenGL and runs between 30-60 Hz. This thread handles updating the
onscreen graphics and handles keyboard and mouse interrupts by
registering callbacks with GLUT, a utility library for OpenGL. The other
thread is the haptic thread or scheduler. This thread can run between 800 -
1600Hz and handles collision detection and force rendering. The
scheduler runs at least 1000Hz for a realistic experience. (cite) (Open
haptic toolkit)

Communication between the graphics thread and the scheduler is achieved
by using callback functions defined by the scheduler. Synchronous
methods are available to pass objects between threads in a thread-safe
manner. An object containing position and state data can returned from
the scheduler via a synchronous callback. The graphics thread controls
what the user sees. The scheduler uses collision detection to control what
the user feels.

Graphics and force rendering are the two main parts of visual based haptic
programs. Objects can be loaded and drawn using OpenGL commands.
Sensable Technologies offers a development kit that provides application
program interfaces (API) to communicate with the haptic device. A high
level APIs can provide collision detection and force rendering based on
OpenGL. A low level API can be used to allow a programmer custom
collision detection and force rendering.

OpenHaptics Toolkit

The OpenHaptics Toolkit (OHTK) is a developer’s kit that provides an
interface with the haptic device. It is made of two main parts, the Haptic
Device API (HDAPI) and the Haptic Library API (HLAPI). The HDAPI
provides a low-level interface and the HLAPI is built to run on top of and
alongside of the HDAPI. The HLAPI is intended to be written alongside
the OpenGL API. The OHTK also provides linear algebra data structures
and a basic set of mathematic functions.

      OHTK has a two-layered approach to haptics. The bottom layer, HDAPI,
      hooks into the hardware level but still provides a simple interface. The
      HLAPI, haptics library, is built on top of the HDAPI and works alongside
      of OpenGL. The HLAPI offers the ability to quickly add haptics to
      existing programs while the HDAPI allows full control over the haptic


      The HLAPI provides a high-level haptic interface and is designed to be
      familiar to OpenGL API programmers. It allows significant reuse of
      existing OpenGL code and greatly simplifies synchronization of the
      haptics and graphics threads. It also provides collision detection and force

      Force rendering and collision detection are handled by setting options with
      the HLAPI. The HLAPI uses the OpenGL buffer to extract the shapes and
      render forces. The programmer must provide two things: it must indicate
      to the HLAPI the area of code that draws the virtual objects and set
      material properties of an object. The shapes that will be rendered with
      haptics must be registered to the HLAPI but the HLAPI will calculate and
      render the forces automatically.


      The HDAPI provides low-level access to the haptic device. It enables
      programmers to render forces directly and provides convenient utility
      features and debugging aids. (cite[6]) If the HLAPI is not used then
      collision detection and force rendering must be implemented by the

Extending HDAPI

     Move TCP force render point
           Must use HDAPI
                  Must implement collision detection
                  Must calculate/handle force rendering
           Need for offset alignment

       Read in coord system of Work Piece
       Read in STL files

Octree Display

       The user controls the haptic device with a drill attached to remove
material from a real world-milling object (block material).

       Hardware Used

       King Craft hand held drill model KBM-160F capable of 35000rpm. One
       drill bit was a spherical cutting tool with a 3.5 mm radius. The drill has a
       flexible extension shaft that allows the drilling tool to move with the
       haptic device. The drilling tool is connected to the haptic device by a
       fabricated clamp.

       Two tool tips are used; a sphere cutting bit and a pointed drill bit. The
       pointed drill bit is used to calibrate the software by reading in positions
       and determining the offset and also to read in the dimensions of the stock

       Software Overview

       The software environment will mirror real world conditions while
       providing additional information to the user. This is a virtual environment
       that assists the average user to manufacture a prototype without special
       skills. The environment will control a support system that assists the
       material removal process. Support will be generated from the haptic
       feedback as well as the graphic display.
     The software is built on existing application program interfaces (API).
     Interfacing with the haptic device is accomplished by using the Open
     Haptics Toolkit (OHTK) available from Sensable Technologies with a
     developer’s license. The system also uses an API for collision detection
     (SWIFT) an

     When using HLAPI the haptics thread does not need to be modified.
     Write the program from a OpenGL point of view. The Haptics commands
     wrap around OpenGL commands finding collisions, shapes and textures.

     Want to render forces at tip of drill, not normal position. HLAPI does
     not support this. Based on this principle, the software will keep the drill
     bit from penetrating a virtual object. This will allow the user to remove

     This project strictly used the HDAPI. HDAPI requires implementing
     collision detection. The two main modifications are the addition of
     collision detection and the manual force rendering.

     This thread interacts with the haptic device using methods provided by the
     HDAPI. Other threads can interact with the scheduler by calling

3D Objects

     Octree Object

     Each node has children and tracks the changes of the children. For a
     simple example using a 3-value system the values would be empty,
     partially filled or completely filled. The leaf nodes would have the
     property of being either empty or completely filled. The branches that are
     empty aren’t drawn. At the branch level it would set its value depending
     on its children. If all the children are completely filled then the current
     branch value would be also completely filled. It is empty when all the
     child nodes are empty and partially filled when only a few children are

     STL/SWIFT Object

Collision Detection
    Using the HDAPI requires implementing collision detection. Collision
    detection is occurs at two levels. Contact can occur between the cursor
    object and the octree object or between the cursor object and the STL
    object. The collision with the octree determines if the cursor object has
    penetrated the shell of the octree object. The collision detection with the
    STL object tries to prevent the cursor object from penetrating the STL

    Octree Object –

    A problem arises when introducing deformable objects because its shape
    changes over the course of the program. The deformable object (octree
    object) will represent the real-world stock material. Initially it will have
    the same dimensions and orientation and it will resemble the current state
    of the stock material throughout the milling process. (The simple solution
    is to use Voxel technology) This is where the octree comes in.

    Each node has children and tracks the changes of the children. For a
    simple example using a 3-value system the values would be empty,
    partially filled or completely filled. The leaf nodes would have the
    property of being either empty or completely filled. The branches that are
    empty aren’t drawn. At the branch level it would set its value depending
    on its children. If all the children are completely filled then the current
    branch value would be also completely filled. It is empty when all the
    child nodes are empty and partially filled when only a few children are


    3D objects

           STL Files

           STL files do not have to be watertight meaning holes, gaps and
           inconsistencies are allowed. SWIFT requires that all objects must
           be watertight. Some STL files will cause errors when loading into
           the SWIFT package.
     Collision Detection

            STL Object using SWIFT

            The object loaded from the STL file is a static nonmoving object
            and the object’s shape does not change. It may be repositioned but
            it is still seen by the collision detection algorithm as a static object.
            This means that the collision detection package can precompute
            the collision space.


            All vectors from the origin of a sphere with the length of the radius
            will fall along the edge of the sphere. A vector S from a point
            outside of the sphere to the origin will cross the border of the
            sphere. If the radius of the sphere is subtracted from the vector S,
            this will return the distance from the initial position to the border
            of the sphere. This distance to the edge of the sphere will provide
            collision detection.

            Using this technique we can calculate the distance from the center
            point of a leaf node to the edge of the sphere. A vector from the
            center point of the leaf node to the center point of the sphere is
            created. The vector is then shortened by the length of the radius.
            The shortened vector is placed at the center point of the leaf node.
            If the position lies within the coordinates of the leaf node then a
            collision occurs.

            Make a vector from the center of the cube to the center of the
            sphere. Subtract the radius of the sphere from the vector. Add the
            vector to the coordinates of the center point of the cube. If the
            endpoint of the vector is inside of the cube then a collision has

Technical Specifics
     Kinematic Chains

     What is returned from haptics is rotation point of stylus in the World
     coordinate system.
     Coordinate systems
     Haptics in World.
     Tool Offset in Haptics
     Cursor(TCP) to Stock Material in World( Tool Offset to octree)
Translation from default haptic TCP to desired Tooltip * haptic
Transformation Matrix from world center point to TCP * inverted
Transformation Matrix from world center point to origin of stock material.

3D Object

       STL Objects

       The program then uses OpenGL calllists to draw the STL object.
       CallLists are used for the cursor and STL objects because they
       have static shapes. CallLists compile the graphics code and allow
       the programmer to draw an object using one line of code. The
       objects are first read in and loaded into a temporary storage
       variables. The display list stores the shape after it has compiled
       the code. The program does not need to store the object after the
       shape is given to openGl. Drawing the shape is then only one line
       of code.

       Opengl lists were used to store the shape loaded from a file then
       delete the data structure. One line of code then draws the opengl

       Octree Object & Marching Cubes

       The octree object uses the marching cubes technique: to
       graphically smooth dynamic objects by examining the vertices of a
       cube. (cite)There are 2^8 possible combinations of a cube’s
       vertices and there are at most 5 triangles that can be drawn for a
       cube. SEE www. For examples of the marching cubes

       There is a difference between branches and leaf nodes. Leaf nodes
       are seen as vertices of a cube. Each branch has 8 nodes with three
       values for a node. Removed, inside or on the border. For
       displaying the marching cubes the leaf nodes are the corner points.
       A value of inside is seen as part of the object. A value of removed
       or on the border are seen as a removed vertice and not drawn.

       The reason for having a removed and on the border value is to
       reduce the number of branches to visit. There are 3 states.

              Inside         Not drawn
              Inside         Drawn
              Removed        Not drawn
           Searches don’t go down paths where the child branch is removed.
           Paths also shouldn’t be visited when nothing should be drawn, i.e.
           the inside of the cube where it would not be visible. The on the
           border data type informs the upper branch that this sub branch
           should be visited.

   Collision Detection

           Collision detection occurs at two levels. Contact can occur
   between the cursor object and the octree object or between the cursor
   object and the STL object. The collision with the octree determines if the
   cursor object has penetrated the shell of the octree object. The collision
   detection with the STL object tries to prevent the cursor object from
   penetrating the STL object.


           The drill bit is spherical and collides with a cube. The distance
           from the center point of the sphere to the outer shell of the sphere
           is always the radius. The algorithm finds intersections at the
           surface of the sphere, not as a solid. This means that only the outer
           shell of the sphere is used for collision detection.


           The University of North Carolina has developed a collision
           detection package called SWIFT that is available for free as long as
           it is for non-commercial use. The integration of this package
           requires using their custom file types or writing custom file
           readers. The package offers a wide range of features.

   Force Rendering

           Forces are generated by the software and rendered in two possible
   cases. Force smoothing has been applied to try to soften the changes
   between forces. The haptic device can have problems in the difference
   between the last force and the current force is too large. Common errors
   are a buzzing noise, the device “kicking” or that is doesn’t render any
   force feedback. The software tracks the last two forces vectors rendered
   to the device and the desired force. The smoothing takes an average of
   these three forces.
Priority    Instrument             Software                Real World
 1       Octree        Distance between cursor       Drill bit is within
                       object and STL object is      specified
                       less than specified           distance of
                       distance.                     desired shape
 2       SWIFT         Collision occurs between      Drill bit touches
                       the cursor object and the     the stock
                       octree                        material

                               Table Forces

       The Phantom Omni is capable of a .8 N continuous force and 3.3 N
of peak force rendering.


       The force generated by the collision with the octree is a resistance
       force. The program creates a force in the opposite direction of the
       motion of the cursor. When a collision occurs with the octree the
       contact leaf is removed. Forces do not accumulate when multiples
       leaves are removed.


       The collision between the cursor object and the STL object takes
       precedence over the collision between the cursor object and the
       octree. This force must repel the drill bit from removing material
       that will make up the end shape. The SWIFT collision detection
       package returns the distance to collision and gives a surface normal
       vector from the STL object in the direction of the cursor object.
       The force is generated in this direction.

       When the cursor penetrates the surface, force-rendering stops.
       This provides the user with immediate feedback that the current
       position is an error.

TCP Repositioning

        Forces must be rendered at the end of the drill bit. This requires
calculating the transformation matrix representing the offset TCP T, the
transformation from the TCP to the World coordinate system. A
technique was provided by Professor Horsch that calculated the
transformation from 3 input points. This is based on solving an
overdetermined system.
                                  W          H       W
                                TCP   T=   TCP   T   H   T

       Alignment of Workpiece

               A right-handed coordinate system can be created from two vectors.
       The two vectors are first made orthogonal if they aren’t already. A
       coordinate system must contain three vectors that are orthogonal to each
       other. The third vector is created by calculating the cross product of the
       first two vectors. The cross product of two vectors creates a vector
       orthogonal to the two vectors.
               The vectors must be read in from the stock material. The two
       vectors to create the coordinate system are read in from 3 positions. An
       origin position is picked and the remaining points are in the direction of
       the x- and y-axis.
               This process has is combined with reading the dimensions of the
       stock material. The origin point should be the lower corner point starting
       at 0. Three more points are then read in. The x- and y-axes are read in for
       calculating the coordinate system. The points should also represent the
       maximum dimension in the axis. The final point is the z dimension along
       the negative z-axis.


       STL Files
       Drillbit.    -- 1 mm diameter centered around origin
       CenteredSphere – 3mm Diameter
       Torus        -- 5.8mm diameter x 1.9mm thickness

       Preliminary tests were conducted to assess the capabilities of the system.
Simple shapes were used for testing. This included a sphere and a torus. Testing
of more elaborate shapes would have been attempted if the initial testing had
proven promising.

Four tests.

The first two tests were small shapes and the second two shapes were large ones.

       Test 1: Torus Octree Depth: 5             Calibration Points: 3
               ~1:15 Dim: 40x40x14               High Density 50x50x40
                    Progress during pause

                          Finished Product

Test 2: small Sphere
        ~1 hour. Desired diameter 15 mm. High Density 30x30x30
       Test 3: Large Torus – Failed
               ~45 minutes through the milling process. The process was less
               than ¼ complete when it crashed. The program crashed when
               turning off force feedback in software. This was the largest
               attempted shape using a block of medium density stock material

       Test 4: Large Sphere
               ~1:15 hours Offset Calibration Points: 5 Octree Depth: 6
               45 mm diameter     High Density 60x60x60


        After a rough shape was created and material was clear the forces were
easily determined. There were occasions that drilling errors were not found until
the process of drilling was further along. It was hard to determine when the
process was finished.

        Force feedback for collisions with the octree could not be felt. The
drilling process created too much vibration to feel the feedback rendered when a
collision with the octree occurred. The testing had the noise of the drill and a
vacuum to collect the removed debris.

       The initial force feedback is hard to determine when drilling. There were
occasions when the contact with the STL object could not be felt.

      When an error is made, force rendering stops. This gives the user
immediate feedback that an error was made.


       Average Milling Time:

       Tests would assess:
              Force/Visual Feedback
              The accuracy of reproducing a shape

                Tool Offset:
                      Ultra sensitive

              Work piece dimension errors.
              Render Forces at inconsistent positions/ depending at angle

       Partial Solution:
               Lock a degree of freedom, roll axis or spinning of haptic pen.
                       This returns better results

Misc. Topics

       Tool Calibration
       Read in Work piece


       Voxel structure took too much processor time to draw. The haptics thread
needs to run at 1000 Hz. There were 128^3 cells to check for drawing each cycle.
The octree was introduced to speed up the drawing with the tradeoff of using
more memory.

Getting offset of drill bit inaccurate. Testing showed that the resolution of the
rotational angles was not high enough.

         Initial testing showed promise. The first test was a simple torus with a
diameter of 35 mm. The milling process takes 30 – 60 min. for a small shape.
The shape resembles a torus. The front face looks smoother than the back face.
While milling it was hard to feel the force feedback.
         Most errors were caused when the force rendering stopped. Force
rendering should increase theoretically to infinity as the distance between the drill
bit and the desired shape decreases. The haptic device is not capable of this and it
is possible to push the haptic device beyond the threshold. When this happens,
the force feedback goes from very strong to non-existent. In most cases this
causes a hole in the desired shape.
         There is also a tolerance issue with the force feedback. The haptic device
is set to render forces when the distance to the STL object is less than 5 mm. It
can be hard to determine where the STL object is. The drill bit can penetrate the
desired shape because the user can overpower the haptic forces.

       Evolved/Purposed Techniques:

       1. Rough Shaping

           Start removing material on the top face then slowly working down
           evenly. This would rely on removing material until force feedback
           was encountered then moving to another area. This would create a
           rough shape. The fine detailing would be saved until the end.

       2. Visual Based

           The user knows the shape of the STL object. The user can pick a
           feature near the edge from the virtual object and removing material
            until force feedback is determined. Then the user relies on force
            feedback to guide the drilling. The user constantly checks with the
            virtual display to understand the shape of the area.

         Tests 1 and 4 demonstrated the capabilities of this system. A smaller
         Must be researched
               Stability errors, overheating, dust/debris
               System uses 100% processor resources. What is causing this.

         Known Issues:

                There is an unresolved issue with OpenGL. Occasionally the
                octree object is not drawn correctly using the marching cubes
                technique. This is apparent when the object is rotated; the back
                face is not rendered. This might be caused by incorrect OpenGL
                setup options.

        This project could allow the average user to create products without
complex training. Designers and inventors could create their own prototypes.
The support provided by the force feedback should prevent the worker from
removing any of the material from the desired shape. The visual feedback would
allow the user to focus on areas that require special attention. This allows the
worker to interact with the design rather than focusing on the manufacturing.

        This project combined various areas of computer science and offered a
glimpse into the future of haptics. Haptics are being incorporated into
CAD/CAM environments, 3D design techniques and various areas of the medical
field. Future uses and research includes generating computer controlled milling
paths, simulating and performing medical procedures, and free form 3D object
creation, i.e. virtual clay.


        A density property exists in the octree structure but is currently unused.
This is when collisions occur with a leaf node and should be viewed as object
persistence. Currently, leaf nodes are removed after a collision between a leaf
node and the cursor object occurs. The possible extension is to modify a counter
when collisions occur and only remove the leaf when the counter expires.
       Graphics improvements

        The current program displays the maximum haptics workspace as the
screen dimensions. This should be changed to better fit the drilling area. A zoom
feature is not implemented and the current camera angle rotations are controlled
by the keyboard. Ideally the haptic device should control this. The haptics could
be used to position the viewing angle by capturing the motion of the haptics
endpoint. Rotations could also be obtained from the orientation of the haptic

       Larger Device

       Stronger force feedback is needed to prevent removing desired material.
Currently the forces provide the user with the knowledge that the desired shape is
near. The user can overpower the force feedback. A larger haptic device from
SensAble Technologies might provide larger forces.

        The program should be easily scalable to work with a larger device by
SensAble. The other devices are capable of rendering stronger forces. The levels
of force would have to be adjusted in an extended program. Most other features
are similar between all SensAble devices because they all can use the
OpenHaptics Toolkit and the HDAPI.


        It is possible to precompute the collision space of for the STL object. This
could remove using the SWIFT collision detection package with the tradeoff
being that the resolution of the octree would determine the accuracy of the final
[1] STLA Files August 31,
[2] SWIFT documentation
[4] The Marching Cubes
Algorithm, August 2, 2005

kaufmanvolumegraphics.pdf Introduction to Volume Graphics
Arie E. Kaufman

Oomeslnc97.pdf 3D Shape Representation: Transforming Polygons into
Voxelization technique

Volvis00.pdf Shape-based Volumetric Collision Detection

2004 SensAble Technologies

OtaduyJainSudLinvis04, Haptic Display of Interaction between Textured Models

sculptingpicture Real-time Haptic Sculpting in Virtual Volume Space
Appendix A
Kinematic Chains

    Coordinate System of

    World (W) -- This is identity 4x4 matrix representing Haptic device
    Haptic (H) – Center of Rotation calculated and returned from
    Tool Tip (TCP) -- Calibrated from reading in points
    WorkPiece (WP) -- Read in from calibration subroutine

    Kinematic Chain

                      Tool Tip to World Coordinates
                               W       H   W
                             TCP T = TCP T H T

                        Tool Tip to WorkPiece
                           WP       W    W    1
                          TCP T = TCP T W P T

               T = Tool Tip x Stylus x inverse(WorkPiece)
Appendix B

    Callback -- Function that is set to run on the scheduler. These can set
           responses to user events or query device state.
    Graphics Thread -- see Scheduler
    Haptic Device -- Phantom Omni 6 DOF device. Can render force
           feedback in 3 dimensions.
    Haptic Thread -- see Scheduler
    HDAPI --Haptic Device API. This allows low a level interface with the
           haptic device.
    HLAPI -- Haptic Library API. This builds on top of the HDAPI and
           allows collision detection based on openGL methods.
    Homogeneous Matrix Transformation– In graphics and robotics a 4x4
           matrix is used to describe both the rotation and the position of a
           system. 3x3 is the rotation matrix with the last column being the
           position vector [x y z 1] T and the last row always being [0 0 0 1].
           OpenGl and the Haptics Library both use column major order
           when referencing arrays.
    Octree -- Storage technique for representing 3D objects. Cubes are
           recursively divided into 8 equal sized cubes. There are multiple
           levels each level having 8 children. This storage technique is
           meant to enhance performance by reducing cells to check.
    OpenGl -- A graphics API that allows 2 and 3 dimensional manipulation.
           multi-language multi-platform. Column Major order
    Scheduler -- This is an infinite loop that responds to events based on
           callbacks. Events can be from user devices or based on idle time.
           The scheduler will execute at a set frequency. OpenGl and the
           HDAPI use seperate schedulers. OpenGl is around 60 Hz while the
           Haptic scheduler is at 1000 Hz.
    STL file -- STereoLithography. A triangle mesh that forms a shell of a
           solid object. All faces are listed. Each face has a normal vector and
           then 3 vectors with 3 coordinates each.
    STL Object -- This represents the desired shape in the virtual
           environment consisting of openGL triangles. The triangle data is
           loaded from an STL file. Forces should be rendered to prevent the
           tool tip to penetrate this shape.
    Stock Material -- Real world drilling material made from Styrofoam.
           This will be simulated by the Octree Object
    SWIFT -- University of North Carolina Chapel Hill's collision detection
           package. This package uses Row Major order
    TCP -- Tool Center Point. This is the endpoint of the kinematic chain
           representing the tooltip of the Haptic device.
Appendix C
User Manual
Recompiling for .Net 2003

       Open Haptics ToolKit                   Available with Developer’s License

        [6] provides installation, step-by-step information to build projects
        Additional install information

               The source code for HLU and HDU must be recompiled. The
       Snap Contraints and the Haptics Mouse utilities can be recompiled but it is
       not required for running this project. Open the project file for HLU and
       convert to the newer format. Go to the Build/Batch Build and build all.
       Close this project and repeat this for the HDU.
               The new rebuild library files should be copied to a new location,
       for example $(3DTOUCH_BASE)\lib. There are two library files to be
       copied from each project. One is located in the Release subfolder and the
       other is in the Debug folder.

       Set an environment variable to the path to the license file.

       SWIFT++                                        Available for Download

               The project requires using a multi-threaded DLL runtime library
       because the OpenHaptics Toolkit requires this. The example SWIFT file is
       set to build using the single threaded runtime library. To test the SWIFT
       library the example project was build using the multi-threaded DLL
       option. To successfully build the example and the thesis project with the
       SWIFT library included, the SWIFT package must be rebuilt.

       Rebuilding Source Code to work with Haptics
        1. Open Project file swift.dsw
        2. Change Build Configuration from Debug to Release
        3. Change Project Properties->C/C++->Code Generation->Runtime
            * Multi-threaded DLL (/MD)
        4. Build Project
            * If Link errors occur see below
        Including the SWIFT library into Existing Projects
          1. Check that Project Properties
               * C/C++->Code Generation->Runtime Library is set to Multi-
                         threaded DLL (/MD)
                C/C++->General->Additional Include Directories add path to
                         swift.h header file.
                         (ex. ..\swift\include\ )
               * Linker->General->Additional Library Directories add path to the
        rebuilt swift.lib file. (ex. ...\swift\release\ )
               * Linker->Input->Additional Dependencies add swift.lib
          2. Include swift.h

        Possible Error Messages VC++ .Net 2003

                If Link errors occur and complain about missing files it is caused
        by switching the runtime libraries. The missing files are a part of Visual
        Studio 6. If you don't have Visual Studio 6, trying searching the internet
        for the files or find someone with a copy of these files. Copy the Library
        and DLL files from a Visual Studio Directory to the windows/system32
        directory or place them in the project folder.


        1.   Phantom Omni
        2.   Computer with Firewire Port
        3.   Stock material, (Styrofoam)
        4.   2 types of drill bits
             a. spherical for cutting
             b. pointed for reading in position data
        5.   Drilling device with flexible extension shaft
        6.   System to stabilize work piece and haptic device
             a. vice
        a.   Clamps to prevent the haptic device moving
        7.   Fitting to attach flexible extension shaft to omni

        Initialization Routine

                There is a sequence of events that must be followed before the
        actual milling process can begin. After the program is running the user
        must load the shape of the drilling bit (cursor object). The user must input
        the radius of the tool tip before loading the object.
                The next step is initializing the haptic device. This can be done
        automatically but was left as a debugging tool. This assures that the haptic
device is connected to the project and gathers data for the graphics thread.
When this process is complete, the haptics thread (scheduler) is running at
        With the haptics scheduler running, the initialization process is
complete. The next step is to calibrate the tool tip. The haptic device
returns the tool center point (TCP) inside of the stylus pen. The TCP must
be moved to the center of the drill bit, assuming a spherical drill bit.
        The user selects the number of points to calibrate the tool ranging
from 3 – 9. This is when the operator uses the pointed drill bit. A sharp
point is required because the drilling tool will rotate around a static point.
The user rotates the drilling tool while keeping contact with one point.
The user pushes button 1 on the stylus pen for each of the selected
calibration points. When all points are read in the software calculates the
offset and uses it to transform the TCP to the correct position.
        The same pointed drill bit is used to read in the positions of the
Styrofoam work piece (stock material). The stock material must be
aligned so the positive x-axis runs from left to right, the y-axis from the
floor to the ceiling and the z-axis from the haptic device to the user.
        When the Align Workpiece routine is initiated the user must enter
4 points. The first point is the bottom left corner closest to the user. This
is the origin point. To read in the point move the drill bit to the origin
point and then presses and releases button 1 on the stylus pen. A user then
moves to the far right, bottom corner that is closest to them to read in the x
position. The y point requires reading in the top left closest corner point.
The z point is the bottom left corner closest to the haptic device in the –z
direction. After aligning the stock material the setup process is nearly
        The last step is loading the 3D object from a stored file. The user
can select the resolution of the octree and the scaling factor. The scaling
factor should only be used in debugging situations since the object would
normally be created with the desired dimensions. After the options are set
the user can load a desired STL file. The STL object that is loaded must
fit within the dimensions of the stock material. If nothing happens in the
software, it is likely that the scaling was incorrect or the object was too

Shared By: