CUAHSI WaterOneFlow Web Services - PowerPoint

Document Sample
CUAHSI WaterOneFlow Web Services - PowerPoint Powered By Docstoc
					CUAHSI WaterOneFlow
   Web Services
    By Tim Whiteaker
   CE 394K.2 Hydrology
     1 February 2007
                  Outline
•   What are Web Services
•   What is WaterOneFlow
•   What is WaterML
•   How do we use HIS Analyst to access
    WaterOneFlow
    Programs Use Functions
• Functions are pieces of code that perform
  a specific task
               What’s the
                  result
               square root
                             = Math.SquareRoot(16)
                 of 16?

                                       Functions:
                   16
                                       Add
                                       Multiply
                   4                   SquareRoot
                              Math
                             library
     Program
   Web Services Let Computers
        Share Functions
• Local programs take advantage of remote
  resources
• Works just like local function
               result = Service.SquareRoot(16)
                                       Functions:
                  What’s the square root of 16
                                       Add
                                      Multiply
                                      SquareRoot
                            Math
                             answer
                        Thelibrary is 4
     Client                                         Service
     Program
  How Do Web Services Work
• SOAP (Simple Object Access Protocol) is
  a protocol for exchanging messages
  over a network

• WSDL (Web Services Description
  Language) is a language for describing
  what a web service can do
   How Do Web Services Work


                Server
         SOAP       WSDL
                (This is what I
                   can do)




Client               SOAP is like speaking the same language
                     WSDL is like a contract
          CUAHSI Web Services
            (WaterOneFLow)
• Standard mechanism for flow of hydrologic
  data between hydrologic data servers
  (databases) and users.
• Provides data access to
  –   USGS NWIS
  –   EPA STORET
  –   NCEP North American Model (NAM)
  –   NASA MODIS
  –   Daymet
  –   NWS ASOS
                    http://water.sdsc.edu/waterOneFlow/
Water Data Web Sites
         NWISWeb site output
# agency_cd Agency Code
# site_no USGS station number
# dv_dt       date of daily mean streamflow
# dv_va        daily mean streamflow value, in cubic-feet per-second
# dv_cd        daily mean streamflow value qualification code
#
# Sites in this file include:
# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC
#
agency_cd             site_no dv_dt   dv_va dv_cd
USGS 02087500                 2003-09-01       1190
USGS 02087500                 2003-09-02       649         Time series of
USGS 02087500                 2003-09-03       525         streamflow at a
USGS 02087500                 2003-09-04       486
USGS 02087500                 2003-09-05       733         gaging station
USGS 02087500                 2003-09-06       585
USGS 02087500                 2003-09-07       485
USGS 02087500                 2003-09-08       463
USGS 02087500                 2003-09-09       673
USGS 02087500                 2003-09-10       517
USGS 02087500                 2003-09-11       454
  Observation Stations Map for the US




    Ameriflux Towers (NASA & DOE)           NOAA Automated Surface
                                              Observing System




USGS National Water Information System   NOAA Climate Reference Network
              WaterOneFlow
• Consistent Query Format   • Consistent Data Format
     WaterOneFlow Methods
• (Almost) the same for all networks
• Methods
  – GetSiteInfo – Give me info about a site
  – GetVariableInfo – Give me info about a
    variable
  – GetValues – Give me a time series for a
    variable at a site
                      GetSiteInfo
• Input                              • Output
  – Network name and                     – Site name
    site code                            – Site location
  – Authorization token                  – Variables measured at
                                           the site
                                             • Variable code
                                             • Period of record

                                             where

          response = service.GetSiteInfo("NWIS:08158000", "")
                 GetVariableInfo
• Input                              • Output
  – Network name and                      – Variable name
    variable code                         – Variable description
  – Authorization token                   – Variable units




                                                 what

          response = service.GetVariableInfo("NWIS:00060", "")
                      GetValues
• Input                              • Output
  – Network name and                    – Site info
    location (e.g., site                – Variable info
    code)                               – Time series
  – Network name and
    variable code
  – Start datetime
  – End datetime
  – Authorization token               where           what

    response = service.GetValues("NWIS:08158000", "NWIS:00060", _
            "2006-12-31T01:00:00", "2006-12-31T08:00:00", "")
                              when
    The “Where” in GetValues
• Works for sites, or geographic locations
• NetworkName:SiteCode
  – NWIS:08158000

• GEOM:POINT(Longitude Latitude)
  – GEOM:POINT(-113 35)

• GEOM:BOX(WLon SLat,ELon NLat)
  – GEOM:BOX(-108 45,-107 46)
    Extract – Transform – Load
      Locations    GetSiteInfo          Data
                                        STORET
                   GetVariableInfo
Variable Codes     GetValues
                                          Data
    Date Ranges                      Data NAM
                                     NWIS
                     WaterML

                  WaterOneFlow          Data
                   Web Service       Repositories
Client
                   TRANSFORM         EXTRACT
 LOAD
      WaterML – The Output of
          WaterOneFlow
• Standardized for all data sources
• Formats
  – Object
  – XML (for programs that can’t handle objects)
• Defined in XML
           XML – A Primer
• The Extensible Markup Language (XML) is a
  W3C-recommended general-purpose markup
  language that supports a wide variety of
  applications. – Wikipedia

• XML represents data
• XML is both human and machine readable
      <site>
        <name>Mansfield Dam</name>
      </site>
             XML Structure
• Example – Let’s describe a streamflow site
                        element
                                      element name = “site”


                     <site></site>
         start tag     OR            end tag

                     <site>

                     </site>
                       OR


                     <site />
Elements can have Children

            child element

          <site>
element     <name>Mansfield Dam</name>

          </site>

                    element text
Elements can have Attributes


  <site>
    <name>Mansfield Dam</name>
    <siteCode network=“NWIS”>08154510</siteCode>

  </site>
       attribute name     attribute value
          XML Nesting

<site>
  <name>Mansfield Dam</name>
  <siteCode network=“NWIS”>08154510</siteCode>
  <location>
    <latitude>30.39</latitude>
    <longitude>97.91</longitude>
  </location>

</site>
                    WaterML
 • XML for describing water data
 • Defined at
http://water.sdsc.edu/waterOneFlow/documentation/schema/


 • WaterOneFlow returns data in WaterML
   format
  Point Observations Information Model
                      http://www.cuahsi.org/his/webservices.html
USGS        Data Source                                     WaterML Response

Streamflow gages           Network

                                                           SiteInfoResponseType
Neuse River near Clayton, NC         Sites

              Discharge, stage            Variables       VariablesResponseType
              (Daily or instantaneous)
                                                        TimeSeriesResponseType
                                                    Values
                         206 cfs, 13 August 2006      {Value, Time, Qualifier}

    •   A data source operates an observation network
    •   A network is a set of observation sites
    •   A site is a point location where one or more variables are measured
    •   A variable is a property describing the flow or quality of water
    •   A value is an observation of a variable at a particular time
    •   A qualifier is a symbol that provides additional information about the value
      WaterML Key Elements
• Response Types     • Key Elements
                       –   site
  – SiteInfo           –   sourceInfo
   GetSiteInfo
                       –   seriesCatalog
                       –   variable
  – Variables
   GetVariableInfo     –   timeSeries
                            • values
                       – queryInfo
  – TimeSeries
   GetValues
     SiteInfoResponseType
• Namespaces   Network


• queryInfo         Sites
• site                   Variables
                queryInfo
• Parameters sent to service
• URLs called (if external resource)
     queryInfo – Another Example



   user                      query URL
parameters
                    site
• siteInfo – info about the site
• seriesCatalog – catalogs of variables
  measured
              siteInfo
• Name
• Site Code
• Location
              geoLocation
• geogLocation – geographic coordinates
  – LatLon point
  – LatLon box
• localSiteXY – projected coorindates




                       European Petroleum Survey Group
                              i.e., NAD 83, etc.
 seriesCatalog
 name                web service location




series (variables)
                           original data source
                    series
• variable – what is measured
• valueCount – how many measurements
• variableTimeInterval – when is it measured
                 variable
• variableCode – global identifier
• variableName                    Sites

• units                          Variables
                                             Values
       variableTimeInterval
• TimeIntervalType – date range
• TimeSingleType – single measurement
• TimePeriodRealTimeType – last n days
                  ISO Time
• International Organization for Standardization
• 1998-03-01T14:30:00 =
   March 1, 1998, at 2:30 in the afternoon
            SiteInfo Response, Recap
queryInfo

 name

  code

location


site

seriesCatalog
 what

how many
 variables
  when
     VariablesResponseType
• variable – same as in series element
• Code, name, units               Sites

                                Variables
                                            Values
   Additional variable Children
• variableDescription
• valueType (enumeration)
  – Field Observation
  – Sample
  – Model Simulation Result
  – Derived Value
• generalCategory – e.g., “water quality”
• sampleMedium – e.g., “water column”
                          -- Many are optional
   TimeSeriesResponseType
• queryInfo
                           Sites
• timeSeries
                               Variables
  – sourceInfo – “where”
  – variable – “what”                 Values
  – values
                 sourceInfo
• SiteInfoType
  – Same as siteInfo element
  – code, name, location
• DataSetInfoType
  – For data continuous in space
  – LatLonPointType
  – LatLonBoxType
               variable
• Same as previous variable element
• code, name, units, etc.
                  values
• Each time series value recorded in value
  element
• Timestamp, plus metadata for the value,
  recorded in element’s attributes




      qualifier
                    ISO Time          value
        value Metadata Examples
•   qualifiers
•   censorCode (lt, gt, nc)
•   qualityControlLevel (Raw, QC’d, etc.)
•   methodID
•   offset
    –   offsetValue
    –   offsetUnitsAbbreviation
    –   offsetDescription
    –   offsetUnitsCode
    TimeSeries Response, Recap
queryInfo


 location


 variable




  values
  Conclusions about WaterML
• Consistent Format
• Includes lots of information
• How do I use it?

             HIS Analyst
                  HIS Analyst
 • Desktop apps that use WaterOneFlow
 • Examples for various application
   environments
 • HydroObjects – for apps that aren’t web
   service enabled
 • Workbook available at:
http://www.cuahsi.org/his/docs/HIS-workbook-20061130.pdf
           HIS Workbook
• Ingesting NWIS Data into Excel
• Ingesting STORET Data into Excel
• Ingesting Weather and Streamflow Data
  into ArcGIS
• Plotting MODIS Data with Matlab
• Ingesting NWIS Data using VB.Net
• Ingesting NWIS Data Using Java
Excel Demo
Weather Downloader
 Historical
Weather Data



Forecasted
Weather Data



  Streamflow


       Other
                    Recap
• Computers share functionality through
  Web Services

                          GetSiteInfo
                          GetVariableInfo
  HIS Analyst             GetValues

                WaterML
                                            Time Series
    Client                WaterOneFlow      Repository
                           Web Service