SeaDataNet V1

Document Sample
SeaDataNet V1 Powered By Docstoc
					Online data access in the SeaDataNet V1 system



                     by
   Dick M.A. Schaap – technical coordinator

              Oostende, June 08
  SeaDataNet Infrastructure versions and planning


• Version 0 = Continuation and maintenance of existing Sea-Search
systems with minor modifications

• Version 1 = All data centres involved in upgraded metadatabases and
transparent data access. Prototype ready by March 2008; all TTT partners
by end 2008; thereafter gradually migrating all partners – operational by
mid 2009.

• Version 2 = All data centres involved in Transnational Access and
Access upgraded. Plus viewing services and further virtualisation.
Operational by 2010.
              Discovery services


• EDMED - Data Sets
• EDMERP - Research
Projects
• CSR - Cruise Summary
Reports
• EDIOS - Monitoring
systems
• EDMO - Marine
Organisations
• CDI - Common Data
Index
SeaDataNet User’s portal schema V1
                                                                                                      User        Data
     General request              Metadata request                      Data request Status of     registration download
                                                                                     request




 Project info                 metadata                                           data               Registr.
  Standards            Cross search                                                 My transact.

                                                 CDI                                Requests
  Software            EDIOS         EDMED                          Shopping
                                                                                     status            AAA
                                                                    basket
 Vocabularies           CSR         EDMERP      EDMO                                manager

                                                       Organisation +
                                                                                                          User
                                                       data source id                                    Register

         CSR               EDIOS               CDI

                                                                          Ifremer Database



                                              EDMO
                           EDMED
                                                                           BODC Database


                                            + EDMERP
     at BSH             at BODC            at MARIS
                                                                           BSH Database
              Metadata & Data catalogues

     Entry point for access hits                                           ...
     services for metadata
     Data downloading services
                                                             Download managers in Data centers
     Visualization services (WMS) for V2
             Common Data Index CDI


• CDI is a fine-grained index to individual data
measurements (such as a CTD cast or moored
instrument record). This is in contrast to EDMED, which
is an inventory of data sets, each of which comprises a
large number of measurements.

• CDI paves the way for direct online data access
through file downloads or online data requests.

• Present CDI V0 system: > 340.000 CDI entries from
36 Data Centres from 29 countries

http://seadatanet.maris2.nl/v_cdi_v0/search.asp
Common Data Index CDI V0
Common Data Index CDI V0
Common Data Index CDI V0
                    CDI V0
• Central and homogeneous metadatabase, but for data
access the users are confronted with the various user
interfaces of the SeaDataNet data centres (which can
provide online search and access / ordering facility / e-mail
request)                  User
                          interface


                             CDI


              url                      url
                           url
    Data centre        Data centre           Data centre
       User               User                  User
     interface          interface             interface


      data               data                  data
                 CDI V1
• Central and homogeneous metadatabase plus
transparent access to the distributed data sets via a
unique user interface at the SeaDataNet portal
                          User
                        interface


                          CDI
      Central
       User          Shopping basket
      Register
                 Request Status Manager

     Download                             Download
     Manager            Download          Manager
                        Manager
      data                                  data
                          data
                CDI V1 System Modules

• CDI Metadatabase and User Interface:
   • Searching and browsing of metadata of data sets, managed
   by SeaDataNet Data Centres
• SeaDataNet Central User Register with User Roles db +
web service :
   • Details of users, their organizations and addresses, license
   period, Id-Passwords, SeaDataNet Roles of users
• SeaDataNet Shopping Basket:
   • Preparing a user request of multiple data sets, handling
   validation of users and roles via the Central User Register,
   and routing requests to the Request Status Manager
• SeaDataNet Request Status Manager:
   • Processing and administration of all requests and data
   deliveries (downloads), for users in communication with
   data centres. For data centres to oversee all transactions.
                    System Modules

  • SeaDataNet Data centre Download Manager:
       • Handles communication with SeaDataNet Request Status
       Manager and takes care that requested files are made ready
       for downloading by User (if cleared! ) in the agreed formats
       via a local website address.




          search
                                    Data download
           shop



                                   Data centre
      SeaDataNet                   Data centre
        Portal                     Data centre
                                   Data centre

Metadata + transaction data
           Order flow diagram

             Search in CDI directory

CDI Meta                                            Register
database                                             Users +
                    Results
                                                    Licenses

                 Requests list
Shopping
 Basket
               Submit Requests           User      User
               (specify filetype? )     Register   Roles
                          Log in       WS
                     User                           Info about
                     Check                         Conditions,
                                       NO          Registration
                                                        etc
                    YES
           User + Role + Requests
CDI V1 Search – home page
Search – Results page
Search – Detail page (full)
Search – records selected for Shopping basket
Shopping basket
Submit Shopping list => User Authentication
Authentication, Authorization and Administration
Single Sign On system set up for access to distributed system

User’s authentication information based on
     personal login / password

User must register to get one login, thereby agreeing with the
“SeaDataNet User Licence”

Central User Register, but management is decentralized

Authorisation based on “User Roles”
User identified and request for motivation and order confirmation
Request confirmed and link to RSM (also by e-mail)
                    YES
           User + Role + Requests


Shopping   Submit extended Request
 Basket      Form with motivation


            Requests + user info in
              RSM; Split by DC
               Status: Pending


             Confirmation email to
Request       User with RSM info
 Status
Manager                                   A   B   C   D
(RSM)
             Validation of access
             via business matrix      U                     Combi
                                      V                   + logic
                                      W
     Data access restrictions in CDI V1


CC              Collection cost charge
FE              Commercial charge
LI              Licence
MO              Moratorium
NA              No access
NC              Distribution cost
OG              Organisation
RS              Restricted
SR              Academic
UK              Unknown
UN              Unrestricted


      See Vocabulary L081 for definitions
        Multiple restrictions may apply
User Roles in AAA Register, of relevance
for data access

  SDRN02        Public
  SDRN03        Academic
  SDRN04        Commercial
  SDRN05        National and local government
  SDRN06        Pan-National government




     See Vocabulary C866 for definitions
           Multiple roles may apply
              Business Matrix


Restriction   A         B         C        D


Role
U
                  OK        TBD       NO       TBD

V
                  TBD       TBD       OK       NO

W
                  TBD       OK        NO       NO
               Validation of access                          Data Centre
               via business matrix
                                                     Download Manager
                                          Status: OK
                          Status: TBD                  Accepted requests in
          Status: NO
                                                        Download Manager
                                 E-mail to DC
                                   manager
                                                                           Check in SDN
                                        Status: OK
                                                                           User Register
Request
                                                                              OK
 Status
Manager                                                   Retrieving files +       Not OK
(RSM)                                                     preparing formats
                              Status:                  + giving URL for ZIP file
              Validate
                             Ready for
              in RSM
                             Download
          Access Denied



           Status:
           Access
           Denied
                                       Data Centre

                               Download Manager
                                   Retrieving files +
                                   preparing formats
                                + giving URL for ZIP file

                URL


Request                          Downloading data sets
 Status                             directly from
Manager                             Data Centres
(RSM)

          Status: Downloaded
Request Status Manager and Download Manager
RSM has Web service to communicate with the Download Managers, locally
installed

RSM keeps IP-addresses, EDMO codes, data manager e-mail, trigger
addresses and V0/V1 labels of all connected Data Centres in config file

Download Manager is Java programme, that must be locally installed

Functions DM:
   Communication with RSM Web service
   Retrieving and processing order lists after being triggered
   Double checking user via AAA Web service
   Retrieving data files, related to requested CDI’s, via local_CDI_ID
   Preparing data files in agreed and requested formats (ODV, MedATLAS,
   NetCDF)
   Placing data files in zip format for downloading
   Informing RSM Web service of order processing status and url’s
   Keeping a local log

First version DM = slim version => no NetCDF and use of pre-processed files
Second version DM = full version => includes conversion of file formats and
support for file management and database storage
Request Status Manager - Homepage
Submit Shopping list => User Authentication
User RSM – Overview of request per Data Centre and Status
User RSM : Status = Ready for Download + Download option
User RSM: Status = All
                Monitoring & Tracking

•All submitted requests, their users, the data sets concerned, and
processing of these requests is administered in the Request
Status Manager

•SeaDataNet Data centres can also log on to the Request Status
Manager and oversee all requests, handle ‘tbd’ requests and
prepare a number of online reports.
Data centre RSM: overview Standing Orders
Data centre RSM: overview To be Discussed Orders
Data centre RSM: overview History Orders
SeaDataNet : URLs
   CDI V1 retrieval

   http://seadatanet.maris2.nl/v_cdi_v1/search.asp

   CDI V1 RSM

   http://seadatanet.maris2.nl/request/welcome.html

   ---------------------------------------------------------------------------

   CDI V1 RSM Web service TESTER:

   http://seadatanet.maris2.nl/test/rsmtest.aspx


   WSDL:
   http://seadatanet.maris2.nl/rsm/RequestStatusManager.asmx?
   wsdl
SeaDataNet V1 Implementation planning

 Prototype is ready

 It includes a temporary set of CDI records and data sets, that are
 managed by RNODC

 Majority of CDI test records is ‘unrestricted’, but some records have
 other ‘data access restrictions’

 First version Download Manager installed => ODV ASCII files and use of pre-
 processed files


 See http://seadatanet.maris2.nl/v_cdi_v1/search.asp

 Use your SeaDataNet login - password
SeaDataNet V1 – Full implementation by TTT
partners in 2008
 Coming months all TTT partners will start installing the Download
 Manager and configuring their infrastructure for being a full V1 data
 centre.

    Preparing CDI V1 XML records following the upgraded CDI XML
    schema and using the new MIKADO software
    CDI must cover all available data sets
    Special attention to the ‘data access restriction’
    Installing the Download Manager software
    Filling the coupling table between the LOCAL_CDI_ID (in the CDI
    XML records) and the local Data sets
    Choosing between pre-processed data files OR conversion via the
    Download Manager
    Note: data files can come from a local file management system
    OR from a local database via specific queries

 Must be ready end 2008
SeaDataNet interim V1 Implementation by other
partners in 2008
 Coming months all other partners will also start populating the CDI V1
 database:

     Preparing CDI V1 XML records following the upgraded CDI XML schema
     and using the new MIKADO software
     CDI must cover all available data sets
     Special attention to the ‘data access restriction’ and LOCAL_CDI_ID

 Not yet installation of Download Manager, but data centres will be informed
 by the Request Status Manager by e-mail of all requests

 Data centres will process all requests manually using the Request Status
 Manager and indicating the URL where the user can download the data sets

 Must be ready end 2008;

 Effect is, that users will experience a common approach to all data centres.
 The difference is behind the scenes.
               Validation of access
               via business matrix

                                        Status: OK
                          Status: TBD
          Status: NO                                           Data Centre

                                E-mail to DC
                                  manager


Request
                                            Status: OK     Retrieving files +
 Status                                                  giving URL for ZIP file
Manager
(RSM)

              Validate
              in RSM                Status:
                                   Ready for
          Access Denied            Download

           Status:
           Access
                                                            interim V1
           Denied
                                        Data
                                       Centre


                                 Retrieving files +
                               giving URL for ZIP file

                 URL


Request                        Downloading data sets
 Status                           directly from
Manager                           Data Centres
(RSM)


                               Check by DC
                                Manager

          Status: Downloaded
                                        interim V1
SeaDataNet full V1 Implementation by other
partners in first half of 2009

 Early 2009 all other partners will start with installation and
 configuration of the Download Manager, using the experience from
 the TTT partners:

    Installing the Download Manager software
    Filling the coupling table between the LOCAL_CDI_ID (in the CDI
    XML records) and the local Data sets
    Choosing between pre-processed data files OR conversion via the
    Download Manager
    Note: data files can come from a local file management system
    OR from a local database via specific queries



 Must be ready mid 2009
SeaDataNet V1 Demonstration




 See http://seadatanet.maris2.nl/v_cdi_v1/search.asp

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:11/14/2012
language:Latin
pages:46