Docstoc

NASA ROSES LiDAR Review

Document Sample
NASA ROSES LiDAR Review Powered By Docstoc
					      NASA ROSES ACCESS Project Review

          Science Data Systems for Satellite
              and Airborne LiDAR Data

                                23 February 2011


           C. Meertens and J. McWhirter, UNAVCO
            S-J. S.Khalsa and T. Haran, NSIDC/CU
             C. Baru and C. Crosby, SDSC/UCSD
            D. Harding and B. Blair, NASA/GSFC
                  and M. Hofton, U. Maryland



ACCESS: Advancing Collaborative Connections for Earth System Science
                    Science Data Systems for Satellite and Airborne LiDAR Data
                                           PI: Charles Meertens, UNAVCO

                                    Objective
• Provide an integrated system (NLAS) for web services-
  based access to distributed NASA airborne (LVIS) and
  satellite (ICESAT/GLAS) LiDAR data archives using
  OpenTopography technologies.
• Define enhanced quality control (QC) metrics and a
  standard data format to improve evaluation of data quality,
  data usability, and compatibility with commercial LiDAR
  software packages.
• Provide users access to point cloud and waveform LiDAR
  data, on-demand digital elevation models (DEMs) generation,
  and derivative products and visualizations.
                                                                  Portal provides user access to LiDAR point cloud, standard
                                                                     and on-demand digital elevation models generation.
         Approach                                                Key Milestones
• Collaborate to define an industry-based LiDAR data
  format for NASA LiDAR data; to define enhanced QC
  metrics; and to design web services.                          • Complete data format, QC, and web services
• Develop NSIDC ICESAT/GLAS web service.                          definitions.                                       12/10
• Develop UNAVCO/GSFC LVIS data repository system and           • Implementation of Data Web Services.               04/11
  web service.                                                  • Beta release of NLAS/OpenTopography Portal
• Provide integrated user access using OpenTopography             with connectivity to NSIDC and UNAVCO.             10/11
  Portal.
                                                                • Production release of NLAS Portal.                 03/12
• Science partner testing of enhanced data and QC products
  and portal access.

        Co-Is/Partners
S-J. S.Khalsa, NSIDC/CU; C.Baru, SDSC/UCSD; D.Harding,
B.Blair, NASA/GSFC; and M.Hofton, U.MD.                                     TRLin=5            TRLcurrent=5
19 April 2010                  Advancing Collaborative Connections for Earth System Science
                                                                                                              ACCESS
                                                                                  LiDAR Data Platforms


Distributed and Large Collections from a diverse set of LiDAR Platforms

                                       ~500 thousand points/day                   5 million points/day
                                       600 km altitude                            10 km altitude
                                       70 m footprint                             10 m footprint




              Satellite - Icesat                                    High Altitude Airborne
                                                            Laser Vegetation Imaging System ( LVIS)
                       500 million-1 billion/day
                       0.5 km altitude
                       20 cm footprint
                                                                                100-500 million points/day
                                                                                tripod
                                                                                1 cm footprint


 Low Altitude Airborne Laser
           Scanning
                                                   Terrestrial LiDAR Scanning
                                                                   NLAS Project Overview



 Objectives of NLAS Project
(NASA Laser Access System)
• Provide an integrated system (NLAS) for
  web services-based access to distributed
  NASA airborne (LVIS) and satellite
  (ICESAT/GLAS) LiDAR data archives using
  OpenTopography technologies.
• Define enhanced quality control (QC) metrics
  and a standard data format to improve
  evaluation of data quality, data usability, and
  compatibility with commercial LiDAR
  software packages.
                                                               Public Website
• Provide users access to point cloud and
                                                    (search for “NLAS Opentopography”)
  waveform LiDAR data, on-demand digital
  elevation models (DEMs) generation, and                Hosted at OpenTopography
  derivative products and visualizations.


                                                                  Project Collaborative Wiki
                                                                (uses Google sites nlaswiki)
                                                                     Project Partners



UNAVCO
PI: Charles Meertens, Project management
Software Engineer: Jeff McWhirter, Web services, API, and UI development

San Diego Supercomputing Center, University of California, San Diego
Co-PI: Chaitan Baru, Project management support
Science Specialist: Chris Crosby, Algorithm Development, Science Evaluation
Software Engineers: Vishu Nandigam, Minh Phan
          Web services, API, and UI development

National Snow and Ice Data Center, NSIDC, CU
Co-PI: Siri Jodha Singh Khalsa, Project management support, ESDSWG
Software Engineers: Terry Haran, Nitin Khanna
            Web services, API, UI and Visualization development

Goddard Space Flight Center
Co-PI: David Harding and Collaborator: Bryan Blair
         Define data format, QA/QC, generate new products, science evaluation

University of Maryland
Collaborator: Michelle Hofton
          Define data format, generate new products, science evaluation
                                                   Major Accomplishments




                      Major Accomplishments
• AGU NLAS presentation and NASA booth demo with end-to-end proof of
  concept for LVIS, ICESat GLAS, and TLS data
• Ingest of select LVIS and ICESAT/GLAS data sets into Open Topography
  portal
• Definition of initial new GLAS GLANL format and LVIS Comprehensive
  format
• NLAS/RAMADDA supports LVIS, LAS, GLAS and ASCII data formats as
  well as aggregations of those files
     • Provisional support for new ICEBRIDGE data sets – ATM, MCORDS,
       PARIS
• OpenTopography federated web service query to NLAS/RAMADDA for
  remote data repository collection summary metadata using ATOM caster.
• Public OpenTopography website at
  http://www.opentopography.org/index.php/about/nlas (sample LVIS
  collections, LVIS point cloud, DEM service)
                                                              Major Milestones




                          Major Milestones
• Complete data format, QC, and web services definitions.            12/10
    Draft definitions
    • Final definitions will evolve in stages over next two quarters
Develop file conversion software for LVIS, GLAS                     12/10
    Note major step to use NLAS/RAMADDA to provide common data
      repository and richer web services for LVIS and GLAS
Sample data sets ingested in OT; Presentations at 2010 AGU 04/11
• Implementation of Data Web Services.                               04/11
    Prototype
    Catalog of services (connectivity between UNAVCO/SDSC OT)
    • Full data service implementation at NSIDC, OT
• Beta release of NLAS/OpenTopography Portal
   with connectivity to NSIDC and UNAVCO.                            10/11
• Production release of NLAS Portal.                                 03/12
                                                Project Schedule Summary



               Project Schedule Summary
• Prototyping of NLAS workflow from data repositories to OT samples for
  LVIS and GLAS: slightly ahead of schedule
    • Expedited by decision to utilize RAMADDA framework for data
      repositories and universal, flexible data translation web service
• Definition of data formats and enhanced QC/QA: on schedule
• Definition of full data web services API and implementation : behind
  schedule
• Processing of complete datasets into comprehensive formats: behind
  schedule
• Science partner utilization of NLAS services: slightly behind schedule
• Publicizing NLAS project: ahead of schedule
                                                       NLAS System Overview


                                OpenTopography Currently:

   USERS                       • Provides users access to ten’s of billions of
                                 LiDAR point cloud measurements and in
                                 industry standard formats (e.g. ascii, LAS)
                               • Provides on-demand processing services to
                                 enable generation of digital elevation models
OpenTopography
                                 (DEMs), derivative products and visualizations.
                                NLAS will provide:
                               • Web Services access to distributed archives
                                 using a downloadable data repository package
NLAS RAMADDA                     (NLAS/RAMADDA)
                               • Integrated access to NLAS data via
                                 OpenTopography

LVIS                           • An extensible Java package for reading and
                 ICESat/GLAS
                                 writing various LiDAR formats
                               • Add full waveform access and enhanced quality
                                 control (QC) metrics for ICESat/GLAS and LVIS
                                                                          Open Topography




                                                             OpenTopography Search

                                                             • Map Interface
                                                             • Metadata about Dataset
                                                             • Access to download and
                                                             processing interface


                                                             • Public website
                                                             http://www.opentopography
                                                             .org/index.php/about/nlas




OpenTopography is funded through the National Science Foundation
                        Open Topography – Data Access



Data Downloading and Processing

                 User specifies
                 Download/Processing
                 Parameters

                  1.) Select area from Map
                 Interface
                  2.) Point Cloud
                   - Select format (ASCII or LAS)
                  3.) DEM Generation
                   -Select Gridding Method,
                 parameters, and format
                                      Open Topography - Results Page




                                  OpenTopography
                            On-demand Processing Products:

                            • Subset of point cloud data
                            • Digital Elevation (DEM) Map
                            •Visualization product and .kmz file
                            if desired
                            •DEM on browser embedded
                            GoogleEarth




LVIS example in Sierra Nevada
                                                            Open Topography - GLAS LiDAR Examples



                                                  ICESat/GLAS example

                                                  • Subset area of interest (below left)
                                                  • Download ASCII Lat, Long, Elev
                                                  (X,Y,Z)
                                                       • Example of subsequent import
                                                       into Excel and plot of shot
                                                       number vs elevation (below
                                                       right)




OT query of
NLAS/RAMADDA product
catalog using web
services with ATOM
casting just completed
(see beta server)
         Beta Server at: http://ot-beta.sdsc.edu:8080/gridsphere/gridsphere?cid=datasets
                                 Open Topography - Other LiDAR Examples



Terrestrial Laser Scanning example (NSF OT Development)
• Subset area of interest
• Download options include: ASCII Point Cloud, DEM, Hillshade &
Slope Products, kmz, and image
• Note that UNAVCO will implement TLS into RAMADDA for NSF
                                              Interoperability with Data
                                               Repository Web Service

                                          NLAS Interoperability

     Open Topography                      Visualization/Analysis Clients




                         HTML Interface


NLAS Web Service
                                               NLAS Web Service




                       NLAS/RAMADDA
                                                      NLAS – ICESat/GLAS GLANL/QCQA




ICESat Consolidated and Enhanced Altimetry Data Structure
Consolidation of parameters from 8 GLA product files
•Eliminating numerous redundant, invalid and unused parameters
•Producing a single data record for each laser shot
Selection of subsets from full ICESat GLA product files
•Currently ~160 unique parameters in products
•NLAS/GLANL 100 parameters in “long” subset for advanced users
•NLAS/GLANL 18 parameters in “short” subset for casual users
Definition of 25 additional derived parameters for improved ease of use
•e.g., Conversion of waveform-derived relative range offsets to elevations
Definition of 6 QA/QC parameters to guide selection of appropriate data
•Source of return: Earth surface, cloud, noise, indeterminate
•Cause when no geolocation result reported: no return detected, instrument anomaly
•Non-nadir data acquired during spacecraft maneuver
•Possible presence of forward scattering range delay
•Qualitative range precision assessment from S/N and signal fitting residual
•Data reliability: composite yes/no flag for 8 parameters effecting quality
                                                   NLAS/RAMADDA



RAMADDA Overview
• Data repository framework developed at Unidata/UCAR
   – Freely available Java based open source package
   – Extensible plugin architecture
• Basic content management functionality

NLAS/RAMADDA plugin to support LiDAR data services
• LiDAR data ingest, metadata harvesting and publication
• Services include:
   – Map overview, time series and waveform displays
   – Spatial subset, decimation
   – Conversion to multiple data format:
     original binary, LAS, ASCII CSV, …
   – Product generation - Image, hill shade image, KMZ
                                                           NLAS Data Repository – LVIS Example

                Collection Query
                                                  NLAS/RAMADDA (LVIS or GLAS – same
                ATOM Response
                                                        •Spatial queries
                                                                 package)
   Open                                                     •Subsetting




                                   Web services
Topography       Data Request                                                Metadata
                                                      •Data Conversion       Database
  Portal
                                                    •Product generation
                LAS, ASCII CSV,
                      etc.
                                                                             Metadata
                                                       LiDAR Tools           harvesting




                                                        File store             Ingest



                                                        LVIS Data System
Other Clients
                                                              LVIS              New
                                                             Product        Comprehensive
                                                                             NLAS/LVIS
                                                            Generation       Format (50
                                                                             parameter)
                                                           NLAS Data Repository – ICESat/GLAS
                                                                                      Example

                Collection Query
                                                          NLAS/RAMMADA
                ATOM Response
                                                      •Spatial queries
   Open                                                   •Subsetting




                                   Web services
Topography       Data Request                                                  Metadata
                                                    •Data Conversion           Database
  Portal
                                                  •Product generation
                LAS, ASCII CSV,
                      etc.
                                                                              Metadata
                                                     LiDAR Tools              harvesting




                                                       File store                Ingest



                                                  ICESat/GLAS Data System
Other Clients
                                                        GLAS Data           New GLANL
                                                          Pool            “long” 100 para
                                                                           “short” 18 para
                                                                         +25 additional para
                                                                             +6 QA/QC
                                                      NLAS/LiDAR Tools




NLAS/LiDAR Tools

Stand-alone Extensible Java package for reading and writing various
   LiDAR formats (made available via SourceForge)
    – LAS
    – LVIS (including waveform)
    – ICESAT/GLAS products
    – Other Icebridge formats – ATM, MCORDS, PARIS
    – Terrestrial Laser Scanning formats – text, PTX
    – Transforms global XYZ to Lat, Long, Elevation
    – …
                                          LiDAR Subset Form




Data Access direct from NLAS/RAMADDA
  (may run demo instead if time allows)
                                                                  LiDAR Waveform Data




   Full Waveform LiDAR Data
                                                                 Transmit Pulse

                                                                 Alternate Signal Start

                                                                 Outer Canopy Relief


                                                                 Canopy Inflection
                                                                 Standard Gaussian Fit

                                                                 Within Canopy Light
                                                                 Extinction

                                                                 Ground Relief
                                                                 Alternate Signal End
1064 nm laser pulse footprint
~ 70 m diameter spaced 175 m along profile
                                             Amplitude (volts)
 NLAS/RAMADDA Waveform Display




NLAS/RAMADDA
 Waveform Display
                                                                               Project Metrics



                                    Metrics Plan
•Custom and web log statistics from OpenTopography
http://www.opentopography.org/index.php/about/pcmetrics

•NLAS/RAMADDA Repository implementation includes web service API logging
•Web log analysis software will be used to parse and summarize information on service
requests (e.g. Domain analysis such as IP address, unique visitors, volume of data transferred,
number of points processed, jobs executed)
•UNAVCO will aggregate repository statistics and will be responsible for upload to NASA
metrics system
•Additional impact and software reuse metrics separate from the web logs are planned
•OT has guest user (no login) and login access that allows increased processing limits and a
custom workspace (mylidarjobs). NLAS will determine login and services access requirements
and apply to RAMADDA and OT. NLAS will also need to pass through user id or IP through web
service API and will bring this up with broader ESDSWG.

Example
2008 LVIS Sierra Nevada, CA:
Dec 2010 9 jobs    23,341,301 points
Jan 2011 4 jobs    2,186,947 points
Feb 2011 11 jobs 14,797,042 points
                                                        ESDSWG Participation



                 ESDSWG Participation
Siri Jodha Khalsa is the representative to the ESDSWG

Highlights:
 • Standards Process Group (SPG) activities
      •Participation in monthly teleconferences
      •Service on two SPG Technical Working Groups evaluating
      proposed standards and reviewer on third.
 • New Orleans meeting in October, 2010 (McWhirter also attended)
 • Incorporated Atom-based Service Cast output format into the NLAS
 web service implementation.

Feedback planned to ESDSWG:
 • Federated distributed search/processing from NLAS Repositories
 • Web service pass-through of user id/IP
 • Khalsa is tracking developments of a proposed US national lidar
 data encoding standard and will provide feedback to ESDSWG.
                                                        Status and Next Steps



Accomplishments
• LiDAR data repository framework
• NLAS/RAMADDA Data Repository
• LiDAR Data translation software tool
• End-to-end proof of concept
   – LVIS, ICESat GLAS, TLS
• New GLAS product
   – GLANL containing enhanced QC
Some Next Steps
                                                        GSFC LVIS
• Define NLAS web service API                      Antarctic Glacier Scan
   – Data and metadata discovery
   – Data service access
• Implement NLAS production servers for GLAS and LVIS data sets
• Implement full wave form service into OpenTopography
• Explore potential application of NLAS services to other data sets –
  ATM, TLS, etc.

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:38
posted:6/19/2011
language:English
pages:26