"Federated Service Oriented Information Management"
Federated Service Oriented Information Management Ahmet Sayar email@example.com Introduction Aim is utilizing distributed heterogeneous information and knowledge provided by different repositories and vendors in an efficient and robust manner. No agreed upon –useful- architecture framework for Federating Obtaining Analyzing Interpreting the heterogeneous distributed data/information for decision makers in scientific application domains Motivation SOA based on Web Services Information Sources are “Filters”: A service inputs DIKW (Data-Information-Knowledge-Wisdom Hierarchy) from Grid and outputs DIKW Web Services, easy to extend and federate. Easy to publish, located and bind. predictable input and output interfaces and defined by metadata Information management through ASIS (Application Specific Information System) framework in Science Domains such as GIS. Data and metadata concepts and formats A repository or sensor has or gets DIKW from "outside Grid"; it outputs DIKW Problem Recognition DB Coverage DB Vector data data DB Raw Data Bitmap netCDF data Image Binary jpeg Data data DB DB HDF5 XML DB data Information Bar Plots graphs DB images DB Knowledge Statistics data Wisdom Interactive Tools Decisions Problem Recognition Services like discovery and notification do not need to be made application specific. BUT If the domain changes then : choices, Database requirements, data format, core service requirements, attributes, and metadata context CHANGES ! What are the common concepts and characteristics for Data, Metadata, Query Language, Services, and Communication language, in order to drive information/knowledge from the heterogeneous data/information sources in Application Domains ? Overall Structure Solution ASL : Application Specific Language. XML based hierarchical data representation format. Cross language, platform and operating system ASVS : Application Specific Visualization System Last filter before the decision maker. Provides information/knowledge in human readable formats ASFS : Application Specific Feature Service. Stores and provides common data model (ASL) Treat binary and common data (in ASL) differently. ASFS AS AS Tool AS Service AS Tool ASVS Repository Display (generic) (user defined) (generic) AS “Sensor” Message Using ASL Overall Structure Solution -cont Common data (in ASL) is kept in ASFS. Enables interactive querying through GUI. Tentative architecture. In the DIKW world, everything is mixed as data and filters In a given domain every filter speaks in ASL ASVS both visualize information and provide a way of navigating ASFS and their underlying DB. ASVS can itself be federated and present output interface. GIS and Astronomy have some standards but not many others have Example (1): GIS Domain (OGC) FS-1 : Master Filter (WMS) Providing the available data list and Cap to ties capabilities abilithe end user clients - M tools a Raster Interactive eta-dat Vector data Data:b FS-2 : Web Feature Server data Provides vector data such as rivers, FS-4 (Minnesota) state and city boundaries in GML Data:a Discovering Service inte Data Handlin FS-3 : Web Map Server MD Filtering FS-2 FS-3 Data:b Data:c Provides image data in the form of Module -Core Service- jpeg, svg, png etc. Defined in its (Nasa) (CGL) rfaces capabilities file FS-1 (CGL) g FS-4 : Web Coverage Server Data:a Data:b Provides coverage (raster) data. Data:c Grided data, pixel info Query : No Standardh– gFilter specification Publis in – SQL PORTAL FS Data Encodings : GML, images Metadata : capability doc. No event notification – we use WSContext Data:a Data Data:b for asynchronous run Data:c capability Registry : WRS – MD Queryable Data in : WFS Interactive Decision Support Example (2): Astronomy Domain (IVOA) FS-1 : VOPlot Integrating, Interacting visualization tools DB FS-2 : SkyNode DB ADQL based SOAP interface DB returning VOTable based results FS-3 FS-3 : SIA FS-2 FS-4 2D sky projection, logically a grid of pixels encoded as a FITS image FS-1 FS-4 : SSA MD URL-based returning a dataset "document" (VOTable) Query : ADQL –extension of SQL Data Encoding: VOTable, FITS PORTAL Metadata : UCD, VOResource Event notification : VOEvent Data Registry : VORegistry capability QueryableData in : SSAP and SIAP, VOStore Interactive Decision Support Interactive Decision Support Tools - Interactive query, http://virtualsky.org (R. Williams et al.) - Interactive display, movie and animation - Integration to Application Science Simulations Issues To Be Discussed (1) Requirements for the domain metadata in capability What does capabilities do and need to have to federate filters? Requirements for the ASL (such as CML, GML) What does ASL need to have to federate the filters? Concept of data (such as feature, coverage) Common representation? Possible? To what extend? A common information management framework which can be applied to any domain. some instructions- any field, what needs to be done Issues To Be Discussed (2) Application level data/information federation Integrating the system with application science simulations. Creating interactive decision support tools utilizing integrated filter services. Tools for map animation, map movies, images Interactive query support to get further information on the image and/or animation. Enabling binding of services into pipelines with or without human intervention through metadata. Caching and load balancing to handle large scientific data in an efficient and robust manner (application based) Summary of SRB & Ogsa-DAI SRB Storage Resource Broker Uniform access to dist. heterogeneous data resources by attributes Catalog service is MCAT (Metadata Catalog Service) Resource and data location transparency Remote authentication authorization – user groups Not just for access, transferring and replicating Sample projects using SRB: BIRN and NASA IPG Ogsa-DAI Open Grid Service Architecture - Data Access and Integration Access to heterogeneous data via common interfaces on the grid. Catalog service is MCS (Metadata Catalog Service) OGSI-compliant Grid Components are Grid services. Resources should be registered. Sample projects using Ogsa-DAI : LEAD, MyGrid Discussions on SRB & Ogsa-DAI SRB Monolithic – does too much MCAT dependent MCAT has limited support for application-level metadata Need diff metadata for diff domain, and extensions for applications Not standard based – Not open source Not handling data based on DIKW hierarchy Ogsa-DAI At the data and Database level MCS dependent MCS has limited support for application-level metadata Need diff metadata for diff domain, and extensions for applications For Grid applications - GGF standards Data only in relational and XML database or ordinary files Not handling data based on DIKW hierarchy Our Work Compared to SRB & Ogsa-DAI Wisdom decisions, Ready to use knowledge and information information and extraction by the user knowledge -Central data access -Reusable components FS Filter Services with abstraction. Uniform access to specific ports and heterogeneous data FS FS interfaces sources -Distributed DIKW MasterSRB abstraction -Metadata : SRB Agents FS Ogsa-GDSF Ogsa/GDS FS FS SRB/MCAT, Ogsa- -Metadata in capability DAI/MCS document -Both provides -Metadata aggregators extensible metadata R R R R R R arch for diff domains -New metadata for different domains -SRB has “zone” Wisdom decisions concept address -User uses just Information/knowledge getData interface to similar issues but different Data access and query query Why are we different ? SOA (Service Oriented Architecture) Easy to extend Reusable components Cross platform and language. XML based hierarchical data representation Easy data integration Easy querying Human readable information Easy to access data – no command line Interactive tools On the fly query creation. Not only accessing data but also transforming through its path to end users. Ports to integrate application simulation to application specific information system (ASIS) Contributions Instructions how to build ASL and metadata in capability for the application sciences. Instructions how to build application specific information system (ASIS) federating multiple filters speaking ASL. Information grid (ASIS) formalization through capabilities metadata, defining all the data/information sources as interacting Web Service filters with standard metadata service ports. Optimize and enhance the distributed heterogeneous information management. THANKS firstname.lastname@example.org Ahmet Sayar