Docstoc

IMG - PDS Project Management - Nasa

Document Sample
IMG - PDS Project Management - Nasa Powered By Docstoc
					PDS Imaging Node Status

Mission Interface, Archive and
      Delivery Services
            PDSMC April 3 & 4, 2008
               Washington, DC



   Lisa Gaddis (USGS) and Sue LaVoie (JPL)



                                        PDSMC April 3, 2008   1
Overview: Imaging Node (IMG) Activities                                                            (FY08+)

Mission Interface
    •   Data Nodes: THEMIS (ASU), HiRISE (UA), LROC (ASU)
    •   Cassini (ISS, VIMS, RADAR)
    •   Mars Exploration Rovers (Cameras – Descent, Haz, MicroImager, Nav, Pan)
    •   Mars Global Surveyor (MOC)
    •   Mars Reconnaissance Orbiter (HiRISE, CTX, MARCI, ONC)
    •   Messenger (MDIS)
    •   Mars Express (HRSC)
    •   Phoenix (MECA-OM, MECA-AFM, RAT, SSI, ‘MARDI’)
    •   Chandrayaan-1 Moon Mineralogy Mapper (M3)
    •   Lunar Reconnaissance Orbiter (LROC, LAMP)
    •   LCROSS
    •   Mars Science Laboratory (MASTCAM, MAHLI, MARDI)

Systems: Online Systems & Support Software
    •   Online access to planetary image data (~9 TB JPL, ~7 TB USGS, ~5 TB ASU, ~18 TB UA = ~39 TB)
              • Atlas II, Map-a-Planet, Photojournal (~2 TB/mo of MER, Cassini, MRO)
    •   Deliver improved ancillary data (pointing, position, geometry)
              • Unified Planetary Coordinates (UPC) Database

Baseline Services
    •   Provide expert assistance to PDS and users for cartographic and scientific data analysis
    •   Restorations include Lunar Orbiter, Apollo cameras

PDS ‘Community Service’
    •   Serve as PDS representative on NASA ‘cartography’ working groups



                                                                                       PDSMC April 3, 2008   2
               Data Delivery Services
– IMG Web site
  • Enhanced automation for faster & easier updates
  • Continued sprucing per advice from Mark Rose (Ames) and users
  • Mission-specific interface points to online image data and other Node
    collections
– Map-a-Planet
  • Web Map Services (WMS) interface delivers derived PDS products
     – New: (Moon) Clementine NIR, Lunar Orbiter, Lunar Prospector; (Mars) MGS
       TES albedo and thermal inertia; (Jupiter/Io) Galileo SSI color
– Atlas II (aka ‘New’ Planetary Image Atlas)
  • Includes image data from all past and current planetary missions
     – Updated monthly with Cassini, THEMIS, HiRISE, MER, etc.
  • Single- and multi-mission searches and bulk download
  • Map-based searches now being implemented (Google Toolkit)
     – Will use improved geometry data from UPC
– Unified Planetary Coordinates (UPC) Database
  • Delivers image metadata with unified coordinates and improved geometry
  • Geometry ‘backbone’ for Atlas II, ODE (GEO), ISIS, (ILIADS?) etc.


                                                             PDSMC April 3, 2008   3
                                   Map-A-Planet
• Capabilities and Features:
  – Delivers derived PDS map products
      • Viking Orbiter Mars Digital Image Mosaic (MDIM)
        v. 2.1
      • MGS/MOC WA (256 p/deg), MOLA (128 p/deg)
      • Clementine UVVIS and NIR mosaics (11 bands)
  – Uses WMS to make maps of spatially
    referenced data from geographic info in
    labels
  – Multiple output file formats
      • GIS ‘world file’ and common image formats
      • ISIS detached labels as input and output
      • Multiple image projections
           –   Simple cylindrical
           –   Mercator
           –   Sinusoidal
           –   Polar Stereographic (implementation in progress)
      • 32-bit output formats (new!)
           – MGS/TES albedo and thermal inertia data
  – Products linked to NASA Nomenclature
    database and USGS planetary maps
                                                                                     See http://mapaplanet.org
                                                         OGC WMS – Open Geospatial Consortium Web Map Service(s)

                                                                                      PDSMC April 3, 2008   4
               MAP: In Development
• Add option for full image download
• Add JPEG2000 format for output files
   – Uses OGC Geospatial Data Abstraction Library (GDAL) for data translation
     from PDS
   – Smaller file sizes for large image products and mosaics
        • HiRISE and LROC
• Output ISIS formatted files from PDS input
   – Create ISIS detached label to augment PDS label for products
• Part of USGS ‘integrated Web services’ plan to simplify access to
  PDS products and allow ‘on-the-fly’ processing
   – Will directly link PDS and ISIS products to a variety of desired output formats
   – Uses OGC-compliant s/w and data formats to ensure interoperability
• New data added (or soon to be added):
   –   Mars MOLA (128 pixels/deg) global mosaic (highest available resolution)
   –   Mars MOC (256 pix/deg) WA global mosaic
   –   Mars TES global derived data (albedo, thermal inertia @ 8 pix/deg)
   –   Lunar Prospector derived Fe, Ti, Th, H, etc. (5 and/or 0.5 deg)
   –   Clementine NIR 6-band global mosaic
   –   Lunar Orbiter photographic global mosaic (512 pix/deg)
   –   Merged Galileo SSI and Voyager color mosaic for Io
                                                                     See http://mapaplanet.org

                                                                    PDSMC April 3, 2008   5
                                         Atlas II
     • Atlas II has 4 distinct layers and functions
          – Data Access via Web Service (uses OODT)
               • Performs searches across all data sets in the system
               • Handles semantics translation (e.g., East or West longitude, SCET vs
                 FDS) then searches metadata from repository’s OODT Profile Server
                 or SQL database
          – User Interface via ‘Forms’
               • The human-interface part that allows tailored requests
          – Processing Capability
               • Uses JPEG2000 software and server
                     – Facilitates HiRISE and LROC deliveries
          – Retrieval (uses OODT)
               • Data download via OODT Product server or ‘wget’ to Web Service
               • Shopping-cart style bulk data ordering
               • Allows geographic data searches


OODT - Object Oriented Data Technology                    See: http://pdsimg.jpl.nasa.gov/Missions/index.html


                                                                                PDSMC April 3, 2008   6
                 Atlas II Status and Plans
Status:
• New missions supported
    – Cassini, MER, MGS, Messenger, MRO (HiRISE, CTX, and MARCI) now operational
• Support for both mission-specific and multi-mission queries
    – The same code base is used for both
    – Common parameters of each logical grouping (data set) are dynamically found and
      displayed and searched across missions

Plans:
• System modifications
    –   Migrate meta-data from Sybase to MySQL to increase performance
    –   Schema optimization
    –   Configuration file to integrate with Atlas II and OODT
    –   Add data processing capability (calibration, mosaicking, geometric processing, etc.)
    –   Map-based searches
    –   Integration with UPC
•   Data migration from Atlas I web-interface to Atlas II
    – FY2008: Voyager, Viking Orbiter, MPF, Clementine, Magellan
                                                      See: http://pdsimg.jpl.nasa.gov/Missions/index.html


                                                                            PDSMC April 3, 2008   7
    Unified Planetary Coordinates (UPC) Database
        • A geometry database system to unify coordinate systems for
          planetary image data
        • Designed in part to resolve a problem with coordinate system
          disparities in older planetary image data (esp. for Mars)
                           INSTRUMENT         LATITUDE    LONGITUDE       LONGITUDE      PRIME         DEM?
                                                          DIRECTION        SYSTEM       MERIDIAN

                        Viking 1               Ographic   Positive West   -180 to 180   EME 1950

                        Viking 2               Ographic   Positive West   -180 to 180   EME 1950




                  {
                        MGS MOC                Ographic   Positive West    0 to 360     IAU 1994

                        MGS TES                Ocentric   Positive West    0 to 360     IAU 1994

                        MGS MOLA               Ocentric   Positive East    0 to 360     IAU 1991

                        MO THEMIS/VIS          Ocentric   Positive East    0 to 360     IAU 2000        Yes

                        MO THEMIS/IR           Ocentric   Positive East    0 to 360     IAU 2000        Yes

                        MRO HiRISE             Ocentric   Positive East    0 to 360     IAU 2000        Yes



        • Provides significantly improved image positional and geometric
          information
                 – Calculates geometric parameters incrementally for each image
                 – Uses 3-D surface to project images where possible
Planetographic – Angle between the equatorial plane                                       Planetocentric – Angle between the equatorial plane
and a line that is normal to the reference ellipsoid                                                and a line from the center of the ellipsoid

                                                                                                           PDSMC April 3, 2008        8
                                    UPC Database
• A ‘geometry database’ for PDS data
   – Reads position & pointing from PDS image labels
      • Designed to be updated and expanded as needed
   – Converts to unified coordinates
      • Planetocentric latitudes
      • 0 to 360 degree positive east longitudes                   UPC
      • Mars/MGS: IAU 2000 Mars Coordinates                        Footprint
                                                                                              Corner
   – Calculates ‘improved’ pointing and location data                                         Footprint

      • Corner, edge positions every X lines, samples
      • Images projected onto 3-D surface model where possible
• Why?
   – Improves data search & retrieval (geographic,
     feature-based) for PDS image data
   – Enhances user ability to identify, combine, correlate
     and analyze cross-mission PDS data
   – Can easily be used by Atlas II, ODE and other search
     tools within and outside PDS (OGC-compatible)
   – Used to identify overlapping image sets to seed             Example: MOC WA Footprint
     control point and match point files
      • Will significantly enhance speed and quality of
        production of image mosaics in ISIS from data such
        as HiRISE and LROC

                                                                               PDSMC April 3, 2008   9
                                      UPC Status
• Database Population (Phase II Development)
   –   MGS/MOC WA, NA population complete (237,842 images)
   –   Clementine UVVIS and NIR in progress (957,144 images)
        • Upgrades to hardware now underway to improve through-put time
   –   Others staged for population
        • MRO/CTX (5,020 images)
        • MRO/HiRISE (157,727 images)
        • Viking (63,015 images)
   –   THEMIS IR and VIS now undergoing camera model improvement in ISIS
        •   To be added to UPC when completed (Aug 08?)
   –   Lunar Orbiter in progress
        • Camera model (ISIS3) and PDS-compliant label being developed

• Preliminary version of UPC released to JPL (IMG) and Wash U (GEO)
   –   MGS/MOC WA and NA database
   –   Workshop held at USGS on March 27, 2008

• MapServer interface operational
   –   Includes WMS, WFS and Raster support
   –   Allows users to specify desired acquisition parameters
   –   Displays detailed image footprint and basic information
   –   Now being tested with live, direct access to UPC database




                                                                           PDSMC April 3, 2008 10
               UPC Footprints: MGS/MOC



   UPC MOC
   Cumindex
     MOC
                      guinness@wunder.wustl.edu
  Offset is ~20 km




96 lon E, -36.5 lat

                                                  PDSMC April 3, 2008 11
Atlas II and UPC Integration
                                        ATLASII UPC INTEGRATION


                                                                                          AtlasII
                                                                                       Map Interface

                                                                            Unified Coordinate System
             AtlasII Query Forms            SQL query construction          Interactive Map Interface
             Constraint Specification
                                                                            Initial Implementation
                                                                            1) map selected point => db footprint => productId

                                                                            Features Added Later
                                                                            2) map bounding box => db footprint => productId list =>list
                                                                            file
                                                                            3) map footprint => db footprint => productId list => list file
                                                 AtlasII
                                                 Search
                                                 Engine




                                            Combined AtlasII SQL Query
                                            with UPC productId list as
      mySQL                                 subquery to forms constraints                     postgreSQL
   PDS meta data                                                                                  UPC
  (key - productId)                                                                         (key - productId)


                                                 AtlasII
                                                 Search
                                                 Engine




                                               AtlasII
                                            Shopping Cart




                                                                                                                                    PDSMC April 3, 2008 12
            Baseline Services: Restorations
• Restoration Activities
     – Clementine Near-Infrared (NIR)
        •   Peer Review complete for Release V. 01 of Clementine near-infrared 6-band lunar mosaic (100 m/pixel)
        •   Complements UVVIS cube, completes full 11-band Clementine multispectral dataset
        •   Final release of V 1.0 to PDS expected May 1, 2008
     – Apollo Photo Digitization (Metric, Panoramic, Handheld Cameras)
        •   USGS collaborators working with PI Robinson to evaluate scanned products to ensure quality and usability
        •   80 TB of data expected!
        •   Working with ASU to create mission, instrument, and data catalog files for PDS archive for Apollo image data
     – Lunar Orbiter Digital Data Archive
        •   Developed tiling scheme and labels for LO global mosaic and projected frames
        •   Working to create mission, instrument, and data catalog files for PDS archive of LO data
     – Viking Orbiter (VIS)
        •   Decompressed the full collection and loaded the database with the resultant metadata for AtlasII searches.
        •   AtlasII configuration and release pending
     – Support for Lunar Data Providers
        •   Support data providers (as needed) who respond(ed) to NASA PMDAP and/or LASER announcement of
            opportunity




                                                                                            PDSMC April 3, 2008 13
          Baseline Services: PDS-Wide
• Standards and ‘PDS2010’
   – Participating in ongoing testing of Label Validation Tools (Garcia,
     Isbell)
   – Participating in Physical Media Working Group ‘Tiger Team’ (Isbell)
   – Participated in ‘PDS2010’ ‘Data/Information Model’ and ‘User’ Tiger
     Teams (Isbell, Gaddis, et al.)
• PDS Community Service
   – Represented PDS at LPSC meeting in March 2008
     • Presented abstract and poster on PDS lunar data holdings in IMG (Garcia)
   – Representing PDS on NASA Working Groups (Gaddis)
     • Goal is to clarify and/or develop recommendations for standardization of
       cartographic products from PDS data
            – Lunar Mapping Project (ESMD; T. Lavoie, MSFC)
            – Lunar Geodesy and Cartography Working Group (ESMD; Archinal, Chair)
            – Geologic Mapping Sub-Group of the PGG Cartography and Geologic Mapping
              Working Group (L. Bleamaster, Chair)
            – ‘Planetary Cartography’ Working Group? (B. Bussey, Chair?)----TBD


                                                                     PDSMC April 3, 2008 14
                                                 Data Nodes
THEMIS DATA NODE (TDN)
•   Principal Investigator: Phil Christensen

•   Location: Arizona State University (ASU), Tempe, AZ

•   Operations
     – In full operational status since 2001

•   Data Holdings
     – ~5 TB now available to community
     – Full copy of data online at USGS (updated quarterly), served via Atlas II

•   Data Releases
     – Four per year (#21 to #24 in FY08); last in Jan 2008, next in April 2008 (this week!)

•   Data Access
     – ‘Map Tool’ Image Browser shows coverage for MO/THEMIS, VO, MGS/MOC NA, MRO/CTX & HiRISE
     – JMARS (Java-based) is mission planning tool for data visualization and integration
     – THMPROC Web site provides customized, processed THEMIS VIS and IR image products
            •   ISIS geometric processing s/w used (now undergoing refinement at USGS)
     – All of these tools will benefit from UPC database




                                                                                               PDSMC April 3, 2008 15
                                               Data Nodes
HiRISE DATA NODE (HDN)
•   Principal Investigator: Alfred McEwen

•   Location: University of Arizona, Tuscon, AZ

•   Operations
     – Now in full operational status, Eric Eliason is manager
     – MRO Extended Mission proposed to NASA (1 Mars Year, FY10 and FY11), ICD now in revision
     – Transition to PDS of data holdings will be facilitated by MRO/HiRISE data mission holdings in place at USGS

•   Data Holdings
     – 18,500 gigabytes (18 TB) now available to community

•   Data Access
     – JPEG2000 JPIP server allows efficient browse/roam/pan/zoom, used in the HDN and via the AtlasII
     – HiRISE Web Site (http://hirise.lpl.arizona.edu; tailored for HiRISE data products) and AtlasII (for generic search
       capability) provide access to all released products
     – Data also served by JMARS (ASU) and ODE (GEO/WashU)

•   Data Release
     – MRO project data release #1 in June 2007, #2 in September 2007, #3 in December 2007, #4 in March 2008
     – Color RDR products delivered October 2008
     – ~5 Months ahead of schedule!

•   Establishes procedures and guidelines that will benefit Data Node for LRO Camera

                                                                                                 PDSMC April 3, 2008 16
                                             Data Nodes
LRO Camera (LROC) DATA NODE (LDN)
•   Principal Investigator: Mark Robinson

•   Location: Arizona State University (ASU), Tempe, AZ

•   Operations
     – Plan modeled after HDN Operations, Ernest Bowman-Cisneros is manager
     – Transition to PDS of data holdings will be facilitated by LRO/LROC data mission holdings in place at USGS

•   Data Holdings
     – LROC EDR Product SIS in review (starts this week!)
     – LROC RDR Product SIS in development (near final), review will be combined with EDR review
     – Up to ~60 TB planned for Exploration Phase
            •   NAC EDR/CDR – 36 TB
            •   WAC EDR/CDR – 24 TB
            •   RDRs – 3 TB

•   Data Access
     – Web access, map-based data searches, parameter search, direct browse capability
          • See site for Apollo data, for example: http://apollo.sese.asu.edu/

•   Data Release
     – Quarterly deliveries planned (higher frequency expected after first release)

•   LRO to transition from NASA Exploration Systems to Science after Yr 1



                                                                                            PDSMC April 3, 2008 17
Extra Slides Follow




                      PDSMC April 3, 2008 18
      PDS IMG: Sub-Node Services Breakdown (FY08+)

Sub-Node        Unique Sub-Node Expertise                        Leverage on Institution                  FY07 Emphasis and Accomplishments

           Planetary Science (Moon, Mars, Icy bodies,   Planetary Cartography                             Mission Interface
USGS       Galilean satellites)                         Photogrammetry                                     MGS MOC, MO THEMIS, MER MI, Cassini
           Planetary remote sensing (spectroscopy,      Photoclinometry                                    RADAR, MRO HiRISE, LRO LROC, Messenger
           radar, thermal infrared)                     GIS                                                MDIS, LCROSS
           Planetary Cartography                        Planetary Geodesy                                 Baseline Systems and Tools
           Planetary Geologic Mapping                   Image Processing and Analysis                      Online Disk Array/FTP servers/ DB, Hardware
           Planetary Image Analysis and Calibration     ISIS software system                                System Evolution, MGS/MOC Web Site,
           NASA mission participation (e.g., MER MI     Participation in active NASA missions (Cassini,     Archive Media Research, UPC implementation
           team, New Frontiers Moonrise proposal)       Mars Odyssey THEMIS, Mars Global Surveyor         Standards and User Support
                                                        MOC and TES, MRO/HiRISE, Mars Express              Standards WG, IV&T testing, Map-a-Planet
                                                        HRSC, etc.)                                        Development, Upgrades and Maintenance
                                                                                                          Restoration
                                                                                                           Lunar Orbiter (digitization), Clementine NIR, Apollo
                                                                                                           cameras


           Data system development (active mission      Mission project management                        Mission Interface
JPL        and archive systems)                         Mission GDS/MOS                                    MGS MOC, MO THEMIS, MER ops & eng
           Database and Data Management System          Instrument flight software                         Prods, Cassini ISS & VIMS, MRO HiRISE, CTX,
           Development                                  Instrument calibration                             MARCI, ONC, MEX HRSC, Phx MARDI, MECA-OM,
           User interface development                   Industry imaging standards                         MECA-AFM, RAC, SSI, LRO LROC & LAMP, Messenger
           Image Processing and Analysis                Scientific animation                               MDIS, M^3, MSL cameras
           NASA mission participation (e.g., Mars       VICAR software system                             Baseline Systems & Tools
           Global Surveyor MOLA, Mars Odyssey           Education and Public Outreach                      AtlasII development and dataset integration,
           THEMIS, MER, Mars Express HRSC and           Participation in active NASA missions (Cassini,    Hardware system evolution, Web Site usability upgrade
           MARSIS, Cassini ISS and VIMS)                Mars Odyssey, Mars Global Surveyor, MER,          Standards & User Support
           Public Outreach                              MRO, Mars Express, Phoenix, MSL, etc.)             Standards WG, Label Validation Tool, M^3 archive format
                                                                                                           Photojournal maintenance
                                                                                                          Restoration
                                                                                                           Viking Orbiter, Lunar Orbiter (analog tapes)




                                                                                                                                   PDSMC April 3, 2008 19
                           What is the UPC?
• Starts from but does not replace the many existing instrument-specific databases

• UPC Ingestion Pipeline uses ‘Conductor’ software developed at UA for MRO/HiRISE
    – Process scalable to unlimited number of processing systems (now using 8 Linux and
       Mac OS X machines)
    – Uses local and/or remote (retrieved via wget) PDS-formatted source files

• Contains pre-computed geometry information for images, based on
     – ISIS and NAIF geometry subroutines
     – Best SPICE data available (including adjusted SPICE from geodetic solutions)
     – Single set of planet, coordinate, and other relevant definition files

• The UPC can and will be recomputed
     – To incorporate new images or new SPICE
     – To change the definition of useful data (ranges of i, , , and resolution)
     – To include additional search variables such as optical depth

• Users can specify desired ranges for
     –   Latitude, longitude, elevation or Cartesian coordinates (x, y, z)
     –   Resolution
     –   Incidence, emission, and phase angles
     –   Instrument, filter, etc.
     –   Time-of-day
     –   Other variables defined by user



                                                                                      PDSMC April 3, 2008 20
               What is the UPC? (2)
• Three areas of functionality
   – Availability and management of SPICE data
      • Four types and/or sources
          –   Predicts
          –   Nadir
          –   Reconstructed
          –   C-Smith-ed (“refined” SPICE products)
   – Generation of database for all instruments
      • Starts with best available SPICE data
      • Requires support in ISIS 3
          – Primarily a Camera Model
      • Adaptable to mission diversity and peculiarities
          – Numerous ingestion sources
               »   PDS labels (MER, Viking)
               »   Cumulative index tables (MOC)
               »   ISIS 2 & 3?
               »   Others
   – Diverse DB query tools
      • Applicable to different needs
          – Data processing, auto-seeding, mosaicking, cartography
          – PDS data searches
               » MGS/MOC dataset is one of the most improved


                                                                     PDSMC April 3, 2008 21
                The UPC Database
• Builds on SPICE database system derived from original mission
  kernels
   – Candidate images initially identified by UPC
   – Contents and/or paramaters designed to be refined and updated
     periodically in ISIS system
   – Results can be provided back to user community in the form of new
     SPICE kernels for included images

• Takes advantage of best available DEMs and SPICE kernels as
  well as advances in ISIS computing capabilities and techniques

• Full functionality and power of UPC developed and distributed in
  the ISIS system to science community




                                                         PDSMC April 3, 2008 22
              Why Use the UPC?
• External users benefit from:
  – Availability of image data via web interface
  – Compatibility with web GIS standards
  – ISIS public availability exports most internal capabilities to
    scientific community
  – Access to a growing library of data products
  – Access to most recent SPICE/Control Nets
  – Anticipate a large number of high-quality community-
    generated products
  – Provide system and/or data to other institutions like the PDS




                                                      PDSMC April 3, 2008 23
Footprint Example: HiRISE



                             Corner Footprint
  UPC Footprint




                  Original
 UPC




                                                PSP_002158_2035_RED4_0
                                                       PDSMC April 3, 2008 24
Footprint Example: MOC-WA



                  Corner Footprint
  UPC Footprint




                                         ab102402

                               PDSMC April 3, 2008 25
            UPC GIS Capabilities
• UPC provides binary predicates (or Geometry
  Relationships) using GEOS (JTS) functions and
  extensions in the database backend (PostGIS)
  – Binary predicate operations take two geometries (e.g.,
    MULTIPOLYGON) and returns a Boolean value indicating if
    the geometric relationship between them exists
  – Examples:
     • Equals(geometry,geometry)
     • Intersects(geometry,geometry)
     • Contains(geometry,geometry)




                                                 PDSMC April 3, 2008 26
             UPC GIS Geometries
   Point        MultiPoint      Polygon           MultiPolygon




LineString      LinearRing   MultiLineString   GeometryCollection




                                               PDSMC April 3, 2008 27
          GIS Analysis Methods
Intersection                                Difference                          Union




               Intersection of LineString
                     and Polygon                         Symmetric Difference




                                                                                PDSMC April 3, 2008 28
       GIS Predicate Example
• This example shows results of LineString and
  Polygon predicate functions




                                     PDSMC April 3, 2008 29
       UPC Population Process
• Initial population of the UPC is compute
  intensive
• A compute cluster of sorts is utilized to
  distribute processing tasks
• The process of downloading, computing,
  collecting and inserting the data into the UPC
  is called a pipeline
• Each process within the pipeline is called a
  segment of the pipeline


                                        PDSMC April 3, 2008 30
                       UPC Population (2)
• The pipeline is developed using the Conductor software developed
  by the Planetary Image Research Laboratory (PIRL) at the University
  of Arizona
   –   http://pirlwww.lpl.arizona.edu/software/Conductor.shtml
• Conductor is a Java application for managing queues of source files
  to be processed by sequences of procedures
• The queues for each segment of the pipeline are actually a set of two
  (MySQL) database tables
• One table contains the source file, the second contains the
  procedures required to process each source file completely through
  the segment
• One or more segments may exist in a single pipeline
• The last procedure in a segment passes the output source to the next
  segment in the pipeline




                                                                 PDSMC April 3, 2008 31
              UPC Population (3)
• Segments of a pipeline are run independently from
  one another on each compute node in the cluster
• Segments are distributed across one or many
  compute nodes
• One or more copies of the same pipeline segment
  may be run on a single node
• This system is scalable within nodes and across many
  platforms
  – We are currently run up to 12 different “nodes” around the
    clock
  – These nodes are both Linux and Mac OS X systems




                                                     PDSMC April 3, 2008 32
          UPC Population (4)
• The UPC pipeline is something of a hybrid
• It has 4 segments for each instrument in the
  UPC database
• Typically each instrument has its own
  Download and ISIS Ingestion segment
• All instrument data is processed through the
  GeomProc (geometric data) and Publish
  (database insert) segments



                                       PDSMC April 3, 2008 33
            UPC Population (5)
                  MOCNA
 DownloadMOC
                  MOCWA

                  THEMISIR
DownloadTHEMIS
                 THEMISVIS    GeomProc         Publish

DownloadHiRISE     HIRISE

 DownloadCTX        CTX

DownloadCLEM     CLEMENTINE




                                         PDSMC April 3, 2008 34
               UPC Population (6)
• Segment design has several criteria
  – Complexity of instrument
     • Some PDS EDRs do not have INSTRUMENT_ID
  – Number of images in instrument dataset
     • MOC has ~238,000 images, 96,618 of them are NA
     • Clementine has ~1.8 million
  – Ingestion into ISIS differs by instrument
     • SPICE initialization may also differ
  – Geometric processing and database publication (insertion) is
    consistant or sufficiently generalized




                                                        PDSMC April 3, 2008 35

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:11/5/2012
language:Latin
pages:35