Intranet CATIE by nikeborome


									  GIS Data: Processing
ArcToolbox, Geoprocessing and

           Spring 2007

      GISC 6382 Applied GIS UT-Dallas Ron Briggs
ArcToolbox, Geoprocessing, ModelBuilder : Overview

  Greatly enhanced in ArcGIS 9---it’s the big change over 8.x
  • ArcToolbox is dockable window in ArcCatalog or
    ArcMap, ( was a separate module in version 8)
  • Has command line interface available for traditionalists
    and/or power users
  • Tools support all data types (coverages, shapefiles, gdb,
    raster) rather than primarily coverages
  • Greatly expanded set of geoprocessing tools
     – Available for almost all the analyses in original, command line
       ArInfo 7, plus more
  • Supports multi- step workflow via Modelbuilder and/or

                    GISC 6382 Applied GIS UT-Dallas Ron Briggs
         What is geoprocessing?
• the modification and analysis of spatial (geographic) data.

                                                              CAD   GDB

           Projections                          Conversion


                Spatial                     Data
               analysis                  management

                 GISC 6382 Applied GIS UT-Dallas Ron Briggs
 What’s Geoprocessing used for?

–Data processing/ data automation: Creation,
 Preparation, Conversion, Management
–Data analysis/spatial modeling: Suitability,
 Sensitivity, Capacity, Risk

           GISC 6382 Applied GIS UT-Dallas Ron Briggs
     Accessing geoprocessing tools
• Geoprocessing is carried out using tools stored in ArcToolbox
   – A dockable window available in ArcMap, ArcCatalog, ArcGlobe, or ArcScene
• Tools run processes

                                                                  Opens ArcToolbox

                     GISC 6382 Applied GIS UT-Dallas Ron Briggs
               Inside of ArcToolbox
• Toolbox: Container
  for toolsets and tools
  – Note: cannot have a
    toolbox within a toolbox       Toolset
• Toolset: Logical
  container of tools
  and other toolsets                     Tools
  (i.e. folder)
• Tool: Single
  operation (includes
  dialogs, models,
  and scripts)
                    GISC 6382 Applied GIS UT-Dallas Ron Briggs
        Supported data types
• The tools work with many data types
                    Coverage      Geodatabase

        Shapefile                                      Tables

                                                             Layer file

    Raster                                                      Behavior


                GISC 6382 Applied GIS UT-Dallas Ron Briggs
   The tools work with many data types
             …Caution # 1!!!
• Often there are different tools for different data
  types. Often there is one:
   – For shapefiles and geodatabse features classes
   – Another for coverages
   – Another for raster datasets
• And they have the same name!
   – For example, there is a separate Clip tool for each of
     these input data types
   – Similarly, there is a separate Define tool for each of
     these input data types
                GISC 6382 Applied GIS UT-Dallas Ron Briggs
       Tools with the same name
• Some tools have the same name
   – Must distinguish between these tools in scripts
   – Be sure to get the correct tool for the data type you are processing

                                                                  Clips a

                                                          Clips a

                              Clips a
                        GISC 6382 Applied GIS UT-Dallas Ron Briggs
 The tools work with many data types
           …Caution # 2 !!!
• In a few instances, you will find certain
  tools are available only for coverages or
  only for shapefiles (even though logically
  they could by applied to both)
  – For example, Thiessen polygon creation only
    available for coverages as of 9.1
     • Go to ArcToolbox>Coverage Tools>Analysis>Proximity
  – Solution is to convert data to the appropriate format to
    run the tool

               GISC 6382 Applied GIS UT-Dallas Ron Briggs
          A special note on the
        Coverages Tools toolbox
• Only available if you have ArcInfo
  Workstation installed
• The toolset titles (Analysis,
  Conversion, Data Management)
  duplicate many of the toolboxes in
  ArcToolbox as a whole
   – You need to use these tools to process
   – In some cases, there are capabilities not
     available for other data types (e.g.
     Thiessen polygon creation)

                  GISC 6382 Applied GIS UT-Dallas Ron Briggs
  System (Esri provided) toolboxes
• All tools are organized
  into toolboxes
• System toolboxes
  are provided by ESRI.
  Number depends on
  –License level (ArcVIEW,
   ArcEditor, ArcInfo)
  –Extensions licensed
• and you can create
  your own toolboxes
  (and tools)GISC 6382 Applied GIS UT-Dallas Ron Briggs
  Tool Counts—well over 500 now
• Counts vary depending on license/extensions
  – ArcView: 102 tools
  – ArcEditor: 104 tools
  – ArcInfo: 251 tools
  – Spatial Analyst extension: 158 tools
  – 3D Analyst extension: 45 tools
  – Geostatistical Analyst extension: 1 tool
• A total of 455 tools came with 9.0
  – Feature class tools – 120
  – Spatial Analyst tools – 185
  – Geocoding tools – 5
  – 3D Analyst tools– 20                          Counts as of 9.0.
  – Linear Referencing–10                         9.1 has an additional 171 tools
                                                  9.2 has even more tools
  – Coverage tools – 57
                GISC 6382 Applied GIS UT-Dallas Ron Briggs
             The geoprocessing framework:
                 four ways to run tools 2

                                                              Command line

1                                                                               3
    Dialog                                                              Models


                                                         4                      14
                 GISC 6382 Applied GIS UT-Dallas Ron Briggs
           Selecting the way to run tools:
          Choosing between the four ways
Selection depends on a user’s needs and abilities
  1. As a dialog inside of any ArcGIS application
     •   Simple, one at a time processing (GIS for dummies?)
     •   Good for ad hoc processing
  2. As a command from a command line
     •   Very efficient for command line afficianados
     •   Supports intelligent auto-completion of commands
  3. As a process in a model (Modelbuilder)
     •   An ―executable flow chart‖ for GIS processing
     •   Powerful multi-step processing, yet simple to create and use
  4. As a function call in a script or program
     •   Similar to AML in ArcInfo 7, but avoids learning proprietary
     •   provides decision making and looping to support batch processing
In this course we will use #1 and #3
                    GISC 6382 Applied GIS UT-Dallas Ron Briggs
• all tools have one or more parameters
  – pieces of information that the tool needs in order
    to run
• some parameters are required,
  – such as the name of the input data set
• some parameters are optional
• most parameters have a default set, which
  – you should always review, and
  – change if it’s inappropriate for your application

              GISC 6382 Applied GIS UT-Dallas Ron Briggs
              Environment settings
• ―Customizable defaults‖
• Common parameters that are applied to all
  tools within a geoprocessing session
• Settings work (―persist‖) in all
  environments (dialogs, command line,
  models, scripts,)
   General settings
     Current workspace
     Output coordinate system

   Geodatabase settings                      Raster settings
     XY Domain                                   Cell size
     M and Z Domains                             Mask
                                             Geodatabase raster
   Coverage environment
      Comparison between prj files
      Precision for new coverages                                   17
                       GISC 6382 Applied GIS UT-Dallas Ron Briggs
  Four levels of environment settings
• User Profile
   – The settings will persist throughout
     all applications loaded by the user
• Application Level
   – The settings will apply to all tools
     executed from that application
• For a specific model or script
   – Settings apply to all tools within this
     specific model or script.
• For an individual tool
   – Settings apply to a specific tool in a
     model or script

        Lower order (e.g. individual tool) settings
        override higher order (e.g. user profile) settings
                       GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Using Modelbuilder

 ArcGIS 9.2 has some major
enhancements to Modelbuilder:

     GISC 6382 Applied GIS UT-Dallas Ron Briggs
        Why Use Modelbuilder?
•   document the processing steps (workflow) that
    you carried out
    –   You will forget
    –   You may need to do it again
•   re-run the analysis to experiment with different
    parameters to see how results are affected
•   rerun the analysis if you find a mistake
•   carry out repetitive, multi-step tasks
•   provide other users (e.g. operations staff) with
    an exact workflow that they can replicate

                GISC 6382 Applied GIS UT-Dallas Ron Briggs
 The Elements of ModelBuilder
• Model diagram window
  – Input variable
  – Tool
  – Derived data variable
• Toolbar
  –   Add Data or Tools
  –   Layout
  –   Zoom and Pan
  –   Add connection
  –   Run

                   GISC 6382 Applied GIS UT-Dallas Ron Briggs
   Building a model: constructing
• Create a new model
• Drag tool into model
• Drag data into
  model or onto a tool
• Link data and tool

                GISC 6382 Applied GIS UT-Dallas Ron Briggs
  Building a model: running and documenting
• Save and rename
  the model
• Run model
• Document the

             GISC 6382 Applied GIS UT-Dallas Ron Briggs
 Setting model parameters

Or double click on the tool’s process
box to open its parameter dialog
                      GISC 6382 Applied GIS UT-Dallas Ron Briggs
           Element States
• Not ready to run
(parameters not set)

• Ready to run

• Has been run

            GISC 6382 Applied GIS UT-Dallas Ron Briggs
An example,
which we will
 now build

         GISC 6382 Applied GIS UT-Dallas Ron Briggs

ESRI GIS Software

   GISC 6382 Applied GIS UT-Dallas Ron Briggs
    Environmental Systems Research Institute, Inc.
              (ESRI)    Redlands, CA
• Privately held by Jack & Laura Dangermond
• One of the 50 largest software companies in the world
• Leader in GIS with at least 1/3rd of market
• 1 million users (2002) and 2,600 employees
• Originator of commercial GIS in 1981 with release of ArcInfo
• Released their first GUI (graphics user interface) product, ArcView, in
  1991 using proprietary Avenue programming language (for MS
  Windows, UNIX, Macintosh)
• These two product lines (ArcView 3 and ArcInfo 7) combined together
  in ArcGIS v.8 released in 2000
     – complete rewrite based on Microsoft COM/Active X software objects
         • Only run on MS Windows
     – completely different interface from earlier ArcView and ArcInfo
     – old, Avenue based, ArcView still available as ArcView 3.3,
         • But Old Avenue scripts will not run in ArcGIS 8
     – old command line ArcInfo 7 still available as ArcGIS Workstation
         • Old AML scripts will run in ArcGIS 9
• ArcGIS 9 released in 2004
• Current 2007 release is ArcGIS 9.2                                       28
                         GISC 6382 Applied GIS UT-Dallas Ron Briggs
        ESRI Product Line-up: ArcGIS client products
ArcReader (―adobe acrobat‖ for maps) & ArcExplorer (spatial data viewer)
    –     Free viewers for geographic data.
ArcGIS 9.x Desktop: two primary modules (MS NT/2000/XP only)
           1.   ArcMap: for data display, map production, spatial analysis, data editing
           2.   ArcCatalog: for data management and preview
           ArcToolbox, for specialized data conversions and analyses, available as a window in both
    Available capabilities within these modules are “tiered”
           •       ArcView: viewing, map production, spatial analysis, basic editing
           •       ArcEditor: ArcView, plus specialized editing
           •       ArcInfo: ArcView & ArcEditor plus special analyses and conversions
    Extensions: for special apps.: Spatial Analyst, 3D Analyst, Geostatistics, Business Analyst,
         Network Analyst.
    ArcObjects: build specialized capabilities within ArcMap or ArcCatalog using VB for Applications
ArcGIS Workstation (for UNIX and MS NT/2000/XP)
    –          the old command line ArcInfo 7.1
ArcGIS Engine (MS NT/2000/XP)
    –          Set of embeddable GIS components (ArcObjects software objects) for use in building custom
    –          Runs under Windows, Unix and Linux, with support for Java, C++, COM and .NET
    –          Replaces MapObjects which were based upon a previous generation of GIS objects
        ArcGIS 8 released 2000 to integrate two previous standalone products: ArcView and ArcInfo
        ArcGIS 9 released 2004 providing the full capability that should have been in ArcGIS 8!!!
           --full support for all data types (coverages, shapefiles, geodatabases)
           --full support for all previous geoprocessing analyses
           --Modelbuilder for scripting and repetitive processing
           --ArcEngine for building custom applications
        ArcView 3.3 (the predecessor to ArcGIS 8.x) the only GUI option for UNIX.
        ESRI Product Line-up: ArcGIS server products (Spring 2007)

ArcGIS Server
    –           Permits the creation of server-based specialized GIS applications
    –           Provides full range of GIS capabilities to a user without a desktop GIS
ArcIMS (Internet Map Server)
    –           Software to develop Internet server-based mapping and basic analysis
    –           Provides maps and simple query to the user without a desktop GIS
SDE (Spatial Database Engine)
    –           middleware to support spatial data storage in standard DBMS
    –           Supports all major industry databases:
            •      Oracle, SQL-Server, IBM DB2, Ingres
Note: SDE and ArcIMS now distributed as part of ArcGIS Server package as of 9.2

ArcGIS Services
    –           Server based applications built and operated by ESRI or its partners
                and made available on the Internet for subscription
    –           Normally charged on a ―per transaction‖ basis, but can be flat fee
    –           presumably built using ArcGIS Server

1/13/2006                     GISC 6382 Applied GIS UT-Dallas Ron Briggs
                   Other ESRI Products:
• ArcPAD
   – Mapping on PDAs (―handhelds‖) with Windows CE operating system
• ArcLogistics Route
   – Specialized business application for delivery routing
• ArcFM
   – water and telecom: industry specific facilities management
• ArcGIS Extensions
   –   Spatial Analyst: raster data analysis
   –   3D Analyst: 3-dimensional data display
   –   Geostatistics: surface analysis
   –   Business Analyst: marketing and site selection
   – Survey Analyst: update of ArcInfo COGO (coordinate geometry) module
   – Network Analyst: network routing algorithims; shortest path, etc.
   – Maplex: automated, high quality labeling for maps (now part of ArcGIS 9)
   – Publisher: creates .MXP maps for reading with ArcReader
   Extensions work irrespective of ArcView/ArcEditor/ArcInfo tier
• BusinessMap:
   – $99 standalone business mapping (originally Richardson-based MapLynx)
                       GISC 6382 Applied GIS UT-Dallas Ron Briggs
                                      ArcGIS System
                                                                   Consistent interface
  c:\ ArcGIS Workstation                          Clients          Increasing capability

       ArcInfo          ArcEditor               ArcView        $    ArcExplorer              Browser
        ArcMap               ArcMap              ArcMap
        ArcCatalog           ArcCatalog          ArcCatalog
        ArcToolbox           ArcToolbox          ArcToolbox

 ArcEngine/                                                              ArcServer Services
 ArcObjects                                   ArcIMS Services
 Development &             ArcSDE Services

                               Files                     Multi-user Geodatabases
                     (Personal Geodatabase,
                     Shapefiles, Coverages,               (in Oracle, SQL Server,           Source: ESRI with mods.
Handheld/Wireless    Grids, tins, etc)                         IBM DBII, etc)
Some additional detail…

    GISC 6382 Applied GIS UT-Dallas Ron Briggs
       Discontinued Products
• ArcCAD
  – CAD product from ESRI
• PC ArcInfo
  –   1st effort at PC based GIS
  –   DOS based, command-line driven
  –   Data not compatible with ArcInfo UNIX
  –   Replaced by ArcInfo 8 and ArcView 3.2
• DAK (Data Automation Kit)
  – Subset of PC ArcInfo for data preparation for ArcView
• Atlas/GIS
  – once a leader in PC-based mapping
  – Bought by ESRI in 1996 & discontinued in 2001
               GISC 6382 Applied GIS UT-Dallas Ron Briggs
                 ArcGIS Version 8/9
With Version 8 & 9, now have two flavors:
   – Largest Microsoft COM/ActiveX application to date
   – Full GUI interface
   – Customization via Visual Basic for Applications
   – New data base concepts: Geodatabase
   – Runs on XP/2000/NT only
   – no UNIX version available
   – classic, command-line ArcInfo with AMLs (Arc Macro Language) for
   – same as version 7 and earlier, with minor enhancements
   – the only option for UNIX, but also available on MS XP
   – With release of ArcGIS 9, little reason to use unless under UNIX
      • Now, all capabilities have been moved to ArcGIS

                   GISC 6382 Applied GIS UT-Dallas Ron Briggs
  ArcGIS Desktop Primary Characteristics
• GUI-based tools
   – ArcCatalog, ArcMap, ArcToolbox
• Geodatabase spatial data model supported in
  standard database environments
   – MS Access (.msb) for personal applications
   – File-based geodatabase with 9.2
   – Any industry db via SDE for multi-user applications
• Modeling of real world as intelligent objects
   – Houses, poles, not points, lines, polygons
• COM/ActiveX components (ArcEngine) for
  embedding geography in other applications
ArcInfo7: simple data        complex applications
ArcInfo8/9: intelligent data    simpler applications
                 GISC 6382 Applied GIS UT-Dallas Ron Briggs
       ArcGIS 9 Desktop Modules
ArcCatalog (schema editor, with VISIO generation)
• The base application for ArcInfo Desktop
• Windows Explorer-like interface
• for organizing access to data and metadata
• For launching other Desktop apps: MAP and TOOLBOX
ArcMap (object editor)
• GUI for map creation and spatial data editing
• ArcPlot/ArcEdit (from ArcInfo v. 7) & ArcView 3.2
  View/Layout combined
• Map projections on the fly (not via conversion as in AV)
ArcToolbox (geoprocessor)
• An interface to geoprocessing tools
• In ArcGIS 8 it was a separate module
   – In ArcGIS Release 9 it’s an integrated window in ArcCatalog or
     ArcMap                                                         37
                  GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS Desktop Capability Tiers:
Each tier has the same interfaces (ArcCatalog, ArcMap,
  and ArcToolbox), but an increasing set of capabilities
  are available within them (and $ price rises
– viewing, map production, spatial analysis, basic editing
– ArcView, plus topologic editing of geodatabases and
  editing of coverages
– ArcView and ArcEditor, plus more geoprocessing
  analysis, conversions, and full support for coverages.
– Old, command line ArcInfo including AML support
               GISC 6382 Applied GIS UT-Dallas Ron Briggs
 ArcGIS 9.0 versus ArcGIS 8.3
• Main differences from 8.x are in ArcToolbox
  – ArcToolbox built into ArcCatalog and ArcMap rather
    than a separate module
  – All ArcToolbox tools support all data types
    (geodatabase, shapefiles, coverages)
      • 8.3 primarily support coverages
  – ModelBuilder diagrammatic modeling tool
      • Invaluable for tracking and replicating geoprocessing steps
  – New scripting capability for repetitive actions
      • Python, JScript and VBScript--simpler to use than VB for
        Applications, the only alternative in 8.x
      • Old aml (arc macro language from ArcInfo 7) also supported

 What ArcGIS 8 should have been when it was first released!
 Incorporates just about everything from ArcInfo 7/ArcView 3.         39
                 GISC 6382 Applied GIS UT-Dallas Ron Briggs
    ArcGIS 9.1 & 9.2 versus ArcGIS 9.0
For Spring 2007 we will use ArcGIS 9.2
   Ormsby text contains copy of 9.0
    Gorr and Kurland contains copy of 9.1

Main differences in 9.1 from 9.0
• General performance improvement and user enhancements (e.g. pause
• Additional geoprocessing tools (63 in ArcView, 86 in ArcEditor, 22 in ArcInfo)
• Improved labeling of features on maps (Big change here was in 9.0)
• Support for linear referencing
• Network Analyst extension for transportation networks (shortest path, etc..)

Main differences in 9.2 from 9.1
• Geodatabase and map documents (.mxd files) incompatible!
• File based geodatabase
     – Faster, larger capacity (x 1,000) than MS Access-based personal geodatabase
     – Higher precision of coordinate storage
•   Identity/info. and measurement tools rewritten and greatly enhanced
•   Graphing re-written
     – Supports brushing (linking) between graph & map (as in Anselin’s GeoDA) software)
•   Supports cartographic representation
     – Can re-position features on map without changing coordinate values in geodatabase
And it will all keep changing!

                                          Internet                  Pervasive

                         Workstation                      • Small Hardware
                  Mini                                    • Wireless Internet
                                     Source: ESRI, Inc.   • Interoperable
      Mainframe                                           • Embedded
                  GISC 6382 Applied GIS UT-Dallas Ron Briggs

To top