ArcGIS Geoprocessing Python Scri

Document Sample
ArcGIS Geoprocessing Python Scri Powered By Docstoc
        ArcGIS Geoprocessing:
Python Scripting - Advanced Techniques

           Nathan Warmerdam
Workshop Outline

 • The Geoprocessor in scripting

 • Cursors
   – Reading and writing geometry

 • Creating and using objects for parameter values

 • Effective error handling

 • Working with ArcGIS Server geoprocessing services
   – Submitting jobs and getting results

 • Additions of note in 9.3
The geoprocessor

                                        Pythonic    Cross-
                                                    Cross-    Supported
                                                   Platform    Versions

 import arcgisscripting
 gp = arcgisscripting.create(9.3)
      arcgisscripting.create(9.3)                                9.3

  import arcgisscripting                                       9.2, 9.3
  gp = arcgisscripting.create()

 import win32com.client
 i     t i 32      li t                                        9.0, 9.1,
                                                               90 91
 gp = win32com.client.Dispatch(“esri…
      win32com.client.Dispatch(“esri…                          9.2, 9.3
 • htt // bh l       i    /    i d kt /9 3/i d      f ?id 842& id

   The di  t h bj t        b      di      i t f th         i ti
 • Th dispatch object may be used in a variety of other scripting
   languages (Perl, VBScript, JScript)
The geoprocessor

 Advantages of using a “9.3” geoprocessor …

 1. Boolean properties are true Booleans (not strings or a
 1 B l             i            B l      (       i
    0 / 1)

 2. List functions return Python lists (not enumerations)

 3 All t l return a result object
 3.    tools t          lt bj t

 4. Extent and point properties are returned as extent
    and point objects (not delimited strings)

 5 R t      Unicode i t d f t i      (      t non-
 5. Returns U i d instead of strings (supports non-
    ASCII characters)
Accessing Data with Cursors

 • Cursors allow record by record access of field values

    – Moving values from one table or feature class to another

              if row1.parcelid == row2.parcelid:
                    1            2
                 row1.owner = row2.owner

    – Accessing properties of a feature’s geometry

              print row.Shape.Area

                                                         Dev Summit 2009 Tech Sessions   5
Available Cursors

  • Search Cursor - data access (read only)

  • Update Cursor - data modification (read, write)

  • Insert Cursor- data addition (write only)

                                                  Dev Summit 2009 Tech Sessions   6
Accessing Data with a Search Cursor

                  PathToData, WhereClause,
  • SearchCursor (PathToData, WhereClause,
    SpatialReference, FieldList, SortFields):
    SpatialReference, FieldList, SortFields): SC Object

  • From the Search Cursor object you get a Row
      Fields            d          ti   f th       bj t
    – Fi ld are accessed as properties of the row object
    – Use the row object’s GetValue method if your field name is a
  • When done, delete row and cursor objects to remove
    data locks
  • Use options to limit the records
    – For example, the “where clause” is the same as using a
      definition query on a layer

                                                         Dev Summit 2009 Tech Sessions   7
Accessing Data with a Search Cursor

     import arcgisscripting
     gp = arcgisscripting.create(9.3)

     typefield = gp.GetParameterAsText(0)

          gp             (   /    /      y g /       )
     sc = gp.SearchCursor(“D:/data/reedley.gdb/roads”)
     row = sc.Next()

     # Print road name and road type
     while row:
         if row.GetValue(typefield) == “hwy”:
            print “Modifying HWY:” + row.Name
            # do stuff
            row = sc.Next()

     del row
     del sc

                                                   Dev Summit 2009 Tech Sessions   8
Reading Feature Geometry

                           Dev Summit 2009 Tech Sessions   9
Reading Feature Geometry

 • Knowing the hierarchy of geometry is important
   – A feature class is made of features
   – A feature is made of parts
   – A part is made of points
 • In Python terms
   – A single part feature looks like this
          [pnt, pnt, pnt]
        [ [pnt, pnt, pnt] ]
   – A multipart polygon feature looks like this
          [pnt, pnt, pnt],[pnt, pnt, pnt]
        [ [pnt, pnt, pnt],[pnt, pnt, pnt] ]
   – A single part polygon feature with a hole (inner ring) looks like
          [pnt, pnt, pnt, pnt, pnt, pnt]
        [ [pnt, pnt, pnt, ,pnt, pnt, pnt] ]

                                                           Dev Summit 2009 Tech Sessions   10
Reading Feature Geometry

  • Use the PartCount property to get the number of
    parts for a feature
  • Use the GetPart method to retrieve the desired part
  • A null point is used as a separator between rings
    (holes) in a polygon part

         # Reading lines
         x = 0
         while x < feat.PartCount:
             roadArray = feat.GetPart(x)
             pnt = roadArray.Next()
             while pnt:
                 print pnt.X, pnt.Y
                   t    RoadArray.Next()
                 pnt = R dA       N t()
             x = x + 1
                                               Dev Summit 2009 Tech Sessions   11
Reading Feature Geometry

 • Need coordinate information in a different coordinate

 • Features may be projected on-the-fly using the Spatial
   Reference parameter

    Create   SR          o    projection
  # C eate a S object from a p oject o file e
  SR = gp.CreateObject(“spatialreference”)
  SR.CreateFromFile(”c:/NAD 1983 UTM Zone 10N.prj”)

  # Create search cursor, using GCS spatial reference
  rows = gp.SearchCursor("D:/data.gdb/roads","", SR)
  row = rows.Next()

                                                Dev Summit 2009 Tech Sessions   12
Writing Feature Geometry

 • To create new features use an Insert cursor
              gp             (   /     g /       )
       rows = gp.InsertCursor("D:/data.gdb/roads“)
       row = rows.NewRow()

 • An Update cursor can be used to replace existing

 • Use the Point and Array objects to create feature parts

 • A part may be used to set a geometry field
    – A multipart feature is an array containing other arrays, where
      each array is a part

                                                           Dev Summit 2009 Tech Sessions   13
Writing Feature Geometry

   # Open an insert cursor for the feature class
   cur = gp.InsertCursor(“D:/data.gdb/roads”)

     Create array and point objects
   # C   t          d   i t bj t
   lineArray = gp.CreateObject("Array")
   pnt = gp.CreateObject("Point")

   # Add two points to the array
   pnt.X, pnt.Y = 358331, 5273193
   pnt.X, pnt.Y = 358337, 5272830

   # Create a new row, or feature, in the feature class
   feat = cur.NewRow()

   # Set the geometry of the new feature to the array of points
   feat.Shape = lineArray

   # Insert the feature

                                                          Dev Summit 2009 Tech Sessions   14
Writing Feature Geometry

              y        geometry is checked before it is
 • The validity of the g      y
    – Problems such as invalid ring order are corrected automatically
    – Uses the same process used by the Repair Geometry tool

 • Writing features requires a schema lock on the data
    – Use the TestSchemaLock method to check

                                                         Dev Summit 2009 Tech Sessions   15
Tools and Parameters

 • A geoprocessing tool
   has a collection of
          t      l
   parameter values

 • Parameters tell the tool
   what to do

                              Dev Summit 2009 Tech Sessions   16
Using Objects to Set Parameters

 • Most tool parameters are easy to define with a string or
   a number
   – Such as a data path or buffer distance

   Other     t          t      t d fi     ith    ti
 • Oth parameters are not easy to define with a string
   – Such as a spatial reference or field mapping

 • Geoprocessing objects can be used to set these

 • Many ways to create geoprocessing objects
   – The CreateObject method
   – The Describe object, has properties that return Objects
                                                          Dev Summit 2009 Tech Sessions   17
Using Objects to Set Parameters

 • Use the SpatialReference object as a parameter value

import arcgisscripting
gp = arcgisscripting create(9 3)

# Describe a dataset with a specific Spatial Reference
       gp.Describe("D:/data/reedley.gdb/city boundary")
desc = gp.Describe( D:/data/reedley.gdb/city_boundary )

# Create the FDS using the describe object's SR Object
gp                     (               y g   ,         ,

                                               Dev Summit 2009 Tech Sessions   18
Using Objects to Set Parameters

• Some parameters accept multiple values (Multivalue)

• Multivalue parameters may be expressed in two ways:
   – As a string ("value1;value2;value3")
   – As a Value Table

• Use LoadFromString method to
  load a multivalue string

 Use ExportToString t create an
•U E       tT St i to     t
 output argument value
   – Useful for script tools

                                               Dev Summit 2009 Tech Sessions   19
Using Objects to Set Parameters

 • Overlay tools, such as Union use Value Tables with
   multiple columns
   – Difficult to parse multivalue string when more than one column
     is used

     import arcgisscripting
     gp = arcgisscripting.create(9.3)

     # Create a value table with 2 columns
     vt = gp.createobject('valuetable', 2)

     # Add 3 feature classes with ranks to the value table
     vt.addrow( c:/base/counties.shp 1 )
     vt addrow("c:/base/counties shp 1")
     vt.addrow("c:/base/parcels.shp 1")
     vt.addrow("'c:/base/state boundary.shp' 2")

     # Run Union using the value table as input
     gp.union(vt, "c:/output/landinfo.shp")

                                                        Dev Summit 2009 Tech Sessions   20
Error Handling

 • Syntax errors
    – Detected when compiled

 • Execution errors
    – Add error handling routines to catch errors
    – Tool error messages can be retrieved from the geoprocessor

 • Errors you raise (or assert) yourself
    – Use Raise and Assert statements to trigger specific exceptions
    – Especially useful when using tools and native methods
                            g                     g
    – Provides more meaningful context for messages

                                                         Dev Summit 2009 Tech Sessions   21
Error Handling
   import arcgisscripting
   gp = arcgisscripting.create(9.3)

   # Get input and output feature classes
   fc = gp.GetParameterAsText(0)
   outfc = gp.GetParameterAsText(1)

          # If the input has point feature raise an error
          if gp.Describe(fc).ShapeType == "Point":
              raise Exception, "Feature Class should not be point"
              gp.FeatureToPoint(fc, outfc)

   # Geoprocessing Errors will be caught here
   except arcgisscripting.ExecuteError:
       print gp.GetMessages(2)

   # Other errors will get caught here
   except Exception, ErrorDesc:
       print ErrorDesc.message

                                                            Dev Summit 2009 Tech Sessions   22
Error Handling

  • Use the traceback module to get Python errors

    import traceback, sys
        t the traceback object
    # get th t     b k bj t
    tb = sys.exc_info()[2]

   # tbinfo contains the failure’s line number and the line’s code
   tbi f = t     b k f     t tb(tb)[0]

   print tbinfo        # provides where the error occurred
   print sys.exc_type # provides the type of error
     i t           l         id   the
   print sys.exc_value # provides th error message

                                                         Dev Summit 2009 Tech Sessions   23

Cursors, Complex Objects and Error Handling

                                       Dev Summit 2009 Tech Sessions   24
What is a Geoprocessing Service?

 • Geoprocessing functionality published to ArcGIS
   Se e
    – Tasks in a service can be any script or model tool created in
      ArcGIS Desktop

 • Geoprocessing services can be used by several clients
    – Desktop, Engine, Explorer, Server web clients, Python Scripting

 • Geoprocessing services have two modes of execution
    – Asynchronous – Your code must ping server for status
    – Synchronous – Your code waits for server to finish

                                                           Dev Summit 2009 Tech Sessions   25
Accessing Geoprocessing Services

 • A geoprocessing service can be thought of as a remote
    – Can be added to your script with the AddToolbox method
    – Tasks availabe from the service are then used like normal tools


                                Service          Folder     GP
                                 URL                      Service

gp AddToolbox(“flame7;GP/SurfaceModels")
gp.AddToolbox( flame7;GP/SurfaceModels )
gp.AddToolbox( flame7;GP/SurfaceModels")

               Local Host

                                                          Dev Summit 2009 Tech Sessions   26
Working with Geoprocessing Services

     p        g p         g
 • Input to a geoprocessing service can be:
    – Record set
    – Feature set
    – Raster dataset
    – String, Double, Long, Date, File, Linear Unit, Boolean

 • Feature sets and record sets
    – Can be created as objects via our scripting module
    – Lightweight representations of feature classes and tables
                input                  object,
    – To use as input, create an empty object and load data in

                                                           Dev Summit 2009 Tech Sessions   27
Working with Geoprocessing Services

• When a server tool is
  executed it returns a result

                j          y
• The result object is how you
  interact with the server

• The GetOutput method
  retrieves data from the

• The GetMapImageURL
  method retrieves an output’s
  map service as an image
                                      Dev Summit 2009 Tech Sessions   28

Geoprocessing Services

                         Dev Summit 2009 Tech Sessions   29
Accessing a Result’s Output

import arcgisscripting
import time
gp = arcgisscripting.create(9.3)

inFeatSet = gp.CreateObject("featureset")
results = gp.BufferPoints(inFeatSet, "5 feet")

# Service is asynchronous. Wait until the job has completed
while results.Status < 4:

# Get an output Feature Set back and save locally
outFeatSet = result.GetOutput(0)
gp.Copy eatu es(out eatSet, C:/te p/base.gdb/to e s_bu e )
gp.CopyFeatures(outFeatSet,"C:/temp/base.gdb/towers buffer")

                                                    Dev Summit 2009 Tech Sessions   30
Additions of note at 9.3

AddFieldDelimiters Method

 • The syntax used to build an SQL expression varies on
   the data source
 • The AddFieldDelimiters method removes the guess
   work in getting the appropriate field delimiters in your
   SQL expression

   crimeFC = “c:/data/crime mdb/crime”

   # Use AddFieldDelimiters to add correct delimiters
   dField = gp.AddFieldDelimiters(crimeFC, “BEAT”)
            gp                   (       ,       )
   query = dField + “ = 1”

   gp.Select(crimeFC, “c:/data/crime.mdb/beat1”, query)

Using geometry directly in Geoprocessing Tools

 • In 9.3 you can use geometry from the row directly in a
   geoproceeing tool

  import arcgisscripting, os
  gp = arcgisscripting create()

  fc = "F:/data/USCounties.shp"

  sc = gp.searchcursor(fc)
  row =

  while row:
      name = + "_" + row.state
      feat = row.shape
      # use the geometry directly in copy features
      gp.copyfeatures(feat, "F:/data/counties.gdb/%s" % name)
      row =
                                                  Dev Summit 2009 Tech Sessions   33
ArcSDESQLExecute Object

• For users that are experienced using SQL against
  ArcSDE geodatabases and would like a more
  integrated approach to sending their SQL to the
  i t    t d         ht      di th i       t th
  ArcSDE database

• Supports most SQL statements
  – DDL – eg. CREATE, DROP,
  – SQL functions with ST_GEOMETRY spatial type for Oracle,
  Informix and PostgreSQL
  – Oracle Spatial functions against SDO_GEOMETRY spatial type

                                                      Dev Summit 2009 Tech Sessions   34
ArcSDESQLExecute Object

 • Very important to understand the impact of issuing
   SQL Statements against Geodatabase objects.

 • Please read the ArcGIS “Working with geodatabases
   using SQL” book for guidance.

                                                Dev Summit 2009 Tech Sessions   35
Help Resources

 • Scripting / Tool documentation
  Geoprocessing Resource C
 •G                      Center

 • Python References
   – Learning Python by Mark Lutz
   –C      Python b Wesley J. Chun
     Core P th    by W l J Ch
   – Dive Into Python by Mark Pilgrim

 • Python Organization

                                                 Dev Summit 2009 Tech Sessions   36
     Thank you

…please don’t forget to fill out an evaluation form!

                                            Dev Summit 2009 Tech Sessions   37