Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

HDF5 OPeNDAP Project - PowerPoint by 31sY3c3

VIEWS: 9 PAGES: 26

									             HDF5 OPeNDAP Project
                    Update and Demo

               MuQun Yang and Hyo-Kyung Lee
                     (The HDF Group)

                      James Gallagher
                     (OPeNDAP, Inc.)


10/17/2008           HDF and HDF-EOS Workshop XII   1
             OPeNDAP Review

   • A software framework that allows
     simple access to remote data
   • Data Access Protocol
   • Client-server model
   • Widely used by Earth Science
      Community


10/17/2008     HDF and HDF-EOS Workshop XII   2
                 OPeNDAP Concept
                Typical Remote Access


       View Data               FTP/HTTP                Remote Data
         (User)                                          (HDF4)
     Visualization
         tools
        Tools                                             Handler
     (gradsdap)
     (e.g. grads)                                      (hdf4_handler)

     Client Library                                       Server
      (libnc-dap)                                        (apache)

                      DAP Protocol (via http)

10/17/2008              HDF and HDF-EOS Workshop XII           3
               OPeNDAP HDF5 Handler


       View Data                                    Remote HDF5
         (User)

                                                    HDF5 Handler



                                                       Server
                                                      (apache)

                   DAP Protocol (via http)

10/17/2008           HDF and HDF-EOS Workshop XII          4
                        Challenges

               Mapping HDF5 to DAP
             • Compound Datatype
             • Groups
             • Object/Regional References




10/17/2008          HDF and HDF-EOS Workshop XII   5
             OPeNDAP HDF5 Handler with HDF-EOS5



       View Data                                         Remote
         (User)                                           HDF5
                                                        HDF-EOS5
     Visualization
         tools                                         HDF5 Handler
        (grads)

     Client Library                                       Server
      (libnc-dap)                                        (apache)

                      DAP Protocol (via http)

10/17/2008              HDF and HDF-EOS Workshop XII          6
             Challenges: HDF- EOS5

• Grid with No Geolocation data
     Clients expect Grid with Geolocation data
• Some Attributes stored as Extremely Long
   String(s) (e.g. StructMetada.0)
     Clients expect structured attributes




10/17/2008       HDF and HDF-EOS Workshop XII   7
             Tweaks for HDF-EOS5

   Added two default HDF5 handler
   configuration options:
   • --enable-eos-grid
   • --enable-eos-meta




10/17/2008       HDF and HDF-EOS Workshop XII   8
             Challenges: OPeNDAP Clients


       • Need special attributes on dataset.
       • Need shared geolocation variables.




10/17/2008         HDF and HDF-EOS Workshop XII   9
             Tweaks for OPeNDAP Clients

        Added two optional handler
        configuration options:
        • --enable-short-name
        • --enable-CF




10/17/2008         HDF and HDF-EOS Workshop XII   10
             OPeNDAP HDF5 Handler with HDF-EOS5



       View Data                                         Remote
         (User)                                           HDF5
                                                        HDF-EOS5
     Visualization
         tools                                         HDF5 Handler
        (grads)                                        w/ CF options

     Client Library                                       Server
      (libnc-dap)                                        (apache)

                      DAP Protocol (via http)

10/17/2008              HDF and HDF-EOS Workshop XII          11
             Day After Server Tweaks

             • Finally, Happy Clients!




10/17/2008            HDF and HDF-EOS Workshop XII   12
             Problems of Tweaking HDF5 Handler

      Remember that we added two optional
      HDF5 handler configuration options:
      1. --enable-short-name
      2. --enable-CF
                                          Causes ambiguity among
                                                variable names
                                           (e.g. /GroupA/ozone vs.
                                                 /GroupB/ozone)


                        Drops some key attributes
               (e.g. StructMetdata, HDF_ROOT_GROUP)


10/17/2008            HDF and HDF-EOS Workshop XII        13
             HDF5-Friendly OPeNDAP Client Library
          Groups
     View???                                          HDF5 Groups


       View Data                                         Remote
         (User)                                           HDF5
     Visualization
         tools                                        HDF5 Handler
        (grads)                                       w/ CF options
   HDF5-Friendly
   Client Library                                        Server
    (libnc-dap)
    (liboc-dap)                                         (apache)

                     DAP Protocol (via http)

10/17/2008             HDF and HDF-EOS Workshop XII          14
                Traditional Groups
             Example:OPeNDAP clientin HDF5
                library: It’s an attribute that I
                don’t understand. I’ll ignore it.


                                          HDF5-Friendly
                                          OPeNDAP client
                                          library: I was waiting
                                          for this key attribute to
                                          re-construct HDF5




10/17/2008            HDF and HDF-EOS Workshop XII          15
             Example: Reference in HDF5


             • Important for NPOESS
             • Object / Regional Reference
             • Map to DAP URL




10/17/2008          HDF and HDF-EOS Workshop XII   16
                   Example: dap2h5
  View Group/Ref.                                    Group/Ref.


       View Data                                      Remote
         (User)                                        HDF5
• HDF5-F. Client
  A test application for the client library
• It canApp.                  HDF5 Handler
   Library construct HDF5 from DAP output
        (dap2h5)
   HDF5-Friendly
   Client Library                                     Server
    (liboc-dap)                                      (apache)

                    DAP Protocol (via http)

10/17/2008            HDF and HDF-EOS Workshop XII         17
Our Client Library(prototype)


   • One more reason:
Help Clients to view Swath
                 Visualizing HDF-EOS5 Grids
             Visualizing HDF-EOS5 Swath Problem
         ???
      View Swath                                         Swath


       View Data                                        Remote
         (User)                                          HDF5
                                                       HDF-EOS5
     Visualization
         tools
         Tools                                        HDF5 Handler
        (grads)
      (gradsoc)                                       w/ CF option
   HDF5-Friendly
   Client Library                                        Server
    (libnc-dap)
    (liboc-dap)                                         (apache)

                     DAP Protocol (via http)

10/17/2008             HDF and HDF-EOS Workshop XII          19
  Demo: MLS swath via GrADS

• GrADS coupled with our client
  library
• The client library provides grid
 mapping from swath data
• GrADS displays swath through
 HDF-EOS5 specific client library
 API calls
             Summary of Client Library (Prototype)

   • Finished coding
   • Tested with the GrADS visualization
     client
   • Documented
   • Working on a demo DAP to HDF5
     tool by using the client library
     prototype
   • Will test with NCL if time allows
10/17/2008           HDF and HDF-EOS Workshop XII   21
                       Caution


• Our Client Library is ONLY A PROTOTYPE!
    It does NOT support all DAP data types
    It does NOT support all AURA files
    It does NOT support all Visualization clients




 10/17/2008      HDF and HDF-EOS Workshop XII   22
                Summary
• HDF5 Access via OPeNDAP
    Is Easy
    Is used by GES DISC to serve Aura
     files
• HDF5-Friendly OPeNDAP Client Library
    Serves HDF5 better(EOS swath)
    Benefits visualization clients

10/17/2008   HDF and HDF-EOS Workshop XII   23
              Future Work


 • HDF5 to DAP2 Mapping Document
 • Release HDF5-friendly OPeNDAP
   Client Library (Prototype)
 • URL: http://hdfgroup.org/projects/opendap




10/17/2008    HDF and HDF-EOS Workshop XII   24
                        Credits


                Mike Folk (The HDF Group)
                  Robert McGrath (NCSA)
    Peter Leonard, Daniel Kahn, Marghi Hopkins (ADNET)
Christopher Lynnes, James Johnson, Denis Nadeau, Fan Fang
                           (NASA)
                  Jennifer Adams (GrADS)
                    Dave Brown (UCAR)




10/17/2008        HDF and HDF-EOS Workshop XII   25
                Acknowledgement


      This work was supported basing upon the
      Cooperative Agreement with the National
      Aeronautics and Space Administration (NASA)
      under NASA grant NNX06AC83A,
      NNX08A077A and NNX06AG75A. Any opinions,
      findings, and conclusions or recommendations
      expressed in this material are those of the
      author(s) and do not necessarily reflect the
      views of NASA.


10/17/2008        HDF and HDF-EOS Workshop XII   26

								
To top