Get documents about "Grids Web Services The Global Operating system of the
Document Sample
Grids Web Services
The Global Operating system
of the World
Its Killer Applications
February 28 2005
Geoffrey Fox
Community Grids Lab
Indiana University
gcf@indiana.edu
e-Infrastructure
e-Infrastructure builds on the inevitable increasing performance
of networks and computers linking them together to support new
flexible linkages between computers, data systems and people
• Grids and peer-to-peer networks are the technologies that
build e-Infrastructure
• e-Infrastructure called CyberInfrastructure in USA
We imagine a sea of conventional local or global connections
supported by the “ordinary Internet”
• Phones, web page accesses, plane trips, hallway conversations
• Conventional Internet technology manages billions of
broadcast or low (one client to Server) or broadcast links
On this we superimpose high value multi-way organizations
(linkages) supported by Grids with optimized resources and
system support and supporting virtual (electronic) enterprises
• Low multiplicity fully interactive real-time sessions
• Resources such as databases supporting (larger) communities
A typical Web Service
In principle, services can be in any language (Fortran .. Java ..
Perl .. Python) and the interfaces can be method calls, Java RMI
Messages, CGI Web invocations, totally compiled away (inlining)
The simplest implementations involve XML messages (SOAP) and
programs written in net friendly languages like Java and Python
Web Services Payment
Credit Card
WSDL interfaces
Portal
Service Security Catalog
WSDL interfaces Warehouse
Web Services Shipping
control
Services and Distributed Objects
A web service is a computer program running on either the local
or remote machine with a set of well defined interfaces (ports)
specified in XML (WSDL)
Web Services (WS) have many similarities with Distributed
Object (DO) technology but there are some (important) technical
and religious points (not easy to distinguish)
• CORBA Java COM are typical DO technologies
• Agents are typically SOA (Service Oriented Architecture)
Both involve distributed entities but Web Services are more
loosely coupled
• WS interact with messages; DO with RPC (Remote Procedure Call)
• DO have “factories”; WS manage instances internally and interaction-
specific state not exposed and hence need not be managed
• DO have explicit state (statefull services); WS use context in the messages to
link interactions (statefull interactions)
Claim: DO’s do NOT scale; WS build on experience (with
CORBA) and do scale
Philosophy of Web Service Grids
• Much of Distributed Computing was built by natural
extensions of computing models developed for sequential
machines
• This leads to the distributed object (DO) model represented
by Java and CORBA
– RPC (Remote Procedure Call) or RMI (Remote Method
Invocation) for Java
• Key people think this is not a good idea as it scales badly
and ties distributed entities together too tightly
– Distributed Objects Replaced by Services
• Note CORBA was considered too complicated in both
organization and proposed infrastructure
– and Java was considered as “tightly coupled to Sun”
– So there were other reasons to discard
• Thus replace distributed objects by services connected by
“one-way” messages and not by request-response messages
Devices
Web services
resources
• Web Services build Humans
loosely-coupled,
Programs
distributed applications, Databases Computational resources
(wrapping existing codes
and databases) based on
BPEL, Java, .NET
service logic
the SOA (service
oriented architecture)
principles.
• Web Services interact by
message processing
exchanging messages in SOAP and WSDL <env:Envelope>
<env:Header>
SOAP format ...
</env:header>
• The contracts for the <env:Body>
...
message exchanges that </env:Body>
</env:Envelope>
implement those
interactions are described
via WSDL interfaces. SOAP messages
What is a Grid?
• You won’t find a clear description of what is Grid and how
does differ from a collection of Web Services
– I see no essential reason that Grid Services have different
requirements than Web Services
– Geoffrey Fox, David Walker, e-Science Gap Analysis, June 30
2003. Report UKeS-2003-01,
http://www.nesc.ac.uk/technical_papers/UKeS-2003-01/index.html.
– Notice “service-building model” is like programming language –
very personal!
• Grids were once defined as “Internet Scale Distributed
Computing” but this isn’t good as Grids depend as much if
not more on data as well as simulations
• So Grids can be termed “Internet Scale Distributed
Services” and represent a way of collecting services
together to solve problems where special features and
quality of service needed.
Community Resources
Grid Community databases have analogy to Television and the
News Web that allow individuals to communicate instantly with
each other via Web Pages and Headline News acting as proxies
N resources deposit information and N can view – Complexity
O(N)
Large and Small Grids
N resources in a community (N is billions for the world
and 1000-10000 for many scientific fields)
Communities are arranged hierarchically with real
work being done in “groups” of M resources – M could
be 10-100 in e-Science
Metcalfe’s law: value of network grows like square of
number of nodes M – we call Grids where this true
Metcalfe or M2 Grids
Nature of Interaction depends on size of M or N
• Shared Information O(N) Complexity Grids for largish N
• Complexity M2 Metcalfe Grids for smaller M < N
Grids must merge with peer-to-peer networks to
support both Complexity O(N) and M2 Systems
M2 Interactions
• Superimpose M2
“Grids” on the sea
(heatbath) of O(N)
“ordinary”
interactions
Architecture of (Web Service) Grids
Grids built from Web Services communicating through
an overlay network built in SOFTWARE on the
“ordinary internet” at the application level
Grids provide the special quality of service (security,
performance, fault-tolerance) and customized services
needed for “distributed complex enterprises”
We need to work with Web Service community as they
debate the 60 or so proposed Web Service specifications
• Use Web Service Interoperability WS-I as “best practice”
• Must add further specifications to support high performance
• Database “Grid Services” for O(N) Community case
• Streaming support for M2 case
Web Services WS-*
• Java is very powerful partly due to its many “frameworks” that
generalize libraries e.g.
– Java Media Framework
– Java Database Connectivity JDBC
• Web Services have a correspondingly collections of specifications
that represent critical features of the distributed operating systems
for “Grids of Simple Services”
– Some 60 active WS-* specifications for areas such as
– a. Core Infrastructure Specifications
– b. Service Discovery
– c. Security
– d. Messaging
– e. Notification
– f. Workflow and Coordination
– g. Characteristics
– h. Metadata and State
– i. User Interfaces
A List of Web Services I
• a) Core Service Architecture
• XSD XML Schema (W3C Recommendation) V1.0 February 1998,
V1.1 February 2004
• WSDL 1.1 Web Services Description Language Version 1.1,
(W3C note) March 2001
• WSDL 2.0 Web Services Description Language Version 2.0,
(W3C under development) March 2004
• SOAP 1.1 (W3C Note) V1.1 Note May 2000
• SOAP 1.2 (W3C Recommendation) June 24 2003
• b) Service Discovery
• UDDI (Broadly Supported OASIS Standard) V3 August 2003
• WS-Discovery Web services Dynamic Discovery (Microsoft,
BEA, Intel …) February 2004
• WS-IL Web Services Inspection Language, (IBM, Microsoft)
November 2001
A List of Web Services II
• c) Security
• SAML Security Assertion Markup Language (OASIS) V1.1 May
2004
• XACML eXtensible Access Control Markup Language (OASIS)
V1.0 February 2003
• WS-Security 2004 Web Services Security: SOAP Message
Security (OASIS) Standard March 2004
• WS-SecurityPolicy Web Services Security Policy (IBM,
Microsoft, RSA, Verisign) Draft December 2002
• WS-Trust Web Services Trust Language (BEA, IBM, Microsoft,
RSA, Verisign …) May 2004
• WS-SecureConversation Web Services Secure Conversation
Language (BEA, IBM, Microsoft, RSA, Verisign …) May 2004
• WS-Federation Web Services Federation Language (BEA, IBM,
Microsoft, RSA, Verisign) July 2003
A List of Web Services III
• d) Messaging
• WS-Addressing Web Services Addressing (BEA, IBM, Microsoft)
March 2004
• WS-MessageDelivery Web Services Message Delivery (W3C
Submission by Oracle, Sun ..) April 2004
• WS-Routing and Referral SOAP Routing Protocol (Microsoft) October 2001
• WS-RM Web Services Reliable Messaging (BEA, IBM, Microsoft,
Tibco) v0.992 March 2004
• WS-Reliability Web Services Reliable Messaging (OASIS Web
Services Reliable Messaging TC) March 2004
• SOAP MOTM SOAP Message Transmission Optimization Mechanism
(W3C) June 2004
• e) Notification
• WS-Eventing Web Services Eventing (BEA, Microsoft, TIBCO)
January 2004
• WS-Notification Framework for Web Services Notification with WS-
Topics, WS-BaseNotification, and WS-BrokeredNotification (OASIS)
OASIS Web Services Notification TC Set up March 2004
• JMS Java Message Service V1.1 March 2002
A List of Web Services IV
• f) Coordination and Workflow, Transactions and Contextualization
• WS-CAF Web Services Composite Application Framework including WS-CTX,
WS-CF and WS-TXM below (OASIS Web Services Composite Application
Framework TC) July 2003
• WS-CTX Web Services Context (OASIS Web Services Composite Application
Framework TC) V1.0 July 2003
• WS-CF Web Services Coordination Framework (OASIS Web Services Composite
Application Framework TC) V1.0 July 2003
• WS-TXM Web Services Transaction Management (OASIS Web Services
Composite Application Framework TC) V1.0 July 2003
• WS-Coordination Web Services Coordination (BEA, IBM, Microsoft) September
2003
• WS-AtomicTransaction Web Services Atomic Transaction (BEA, IBM, Microsoft)
September 2003
• WS-BusinessActivity Web Services Business Activity Framework (BEA, IBM,
Microsoft) January 2004
• BTP Business Transaction Protocol (OASIS) May 2002 with V1.0.9.1 May 2004
• BPEL Business Process Execution Language for Web Services (OASIS) V1.1 May
2003
• WS-Choreography (W3C) V1.0 Working Draft April 2004
• WSCI (W3C) Web Service Choreography Interface V1.0 (W3C Note from BEA,
Intalio, SAP, Sun, Yahoo)
• WSCL Web Services Conversation Language (W3C Note) HP March 2002
A List of Web Services V
• h) Metadata and State
• RDF Resource Description Framework (W3C) Set of recommendations expanded
from original February 1999 standard
• DAML+OIL combining DAML (Darpa Agent Markup Language) and OIL
(Ontology Inference Layer) (W3C) Note December 2001
• OWL Web Ontology Language (W3C) Recommendation February 2004
• WS-DistributedManagement Web Services Distributed Management Framework
with MUWS and MOWS below (OASIS)
• WSDM-MUWS Web Services Distributed Management: Management Using Web
Services (OASIS) V0.5 Committee Draft April 2004
• WSDM-MOWS Web Services Distributed Management: Management of Web
Services (OASIS) V0.5 Committee Draft April 2004
• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM,
Microsoft, SAP) March 2004
• WS-RF Web Services Resource Framework including WS-ResourceProperties,
WS-ResourceLifetime, WS-RenewableReferences, WS-ServiceGroup, and
WS-BaseFaults (OASIS) Oasis TC set up April 2004 and V1.1 Framework March
2004
• ASAP Asynchronous Service Access Protocol (OASIS) with V1.0 working draft
G June 2004
• WS-GAF Web Service Grid Application Framework (Arjuna, Newcastle
University) August 2003
A List of Web Services VI
• g) General Service Characteristics
• WS-Policy Web Services Policy Framework (BEA, IBM,
Microsoft, SAP) May 2003
• WS-PolicyAssertions Web Services Policy Assertions
Language (BEA, IBM, Microsoft, SAP) May 2003
• WS-Agreement Web Services Agreement Specification
(GGF under development) May 2004
• i) User Interfaces
• WSRP Web Services for Remote Portlets (OASIS)
OASIS Standard August 2003
• JSR168: JSR-000168 Portlet Specification for Java
binding (Java Community Process) October 2003
A List of Web Services VII
• j) Recent Updates …………………
• WS-Eventing important update of this notification specification
with IBM, Sun and others joining Microsoft et al. as authors
• WS-Enumeration supporting the splitting of a single entity (file
or stream) into multiple messages
• WS-Transfer supporting the creation, update (by get or put) or
deletion of a resource
• WS-Management competes with WS-DM to provide a Web
Service to manage resources
• WS-PolicyAttachment describes how to associate policies with
UDDI and Endpoints and how to integrate with WSDL
• WS-DAI is a Web Service of the OGSA-DAI Grid linkage with
databases
• WS-CIM is a Web Service rendering from DMTF (Distributed
Management Task Force) of the industry standard CIM
(Common Information Model) of metadata for computer devices
• The WS-* implicitly define an architecture
Database
Database Peers
Service Facing
Web Service Interfaces
Event/ Event/ Event/
Peer to Peer Grid
Message
Brokers
Message
Brokers
Message
Brokers
Peers
User Facing
Web Service Interfaces
A democratic organization Peer to Peer Grid
Importance of SOAP
• SOAP defines a very obvious message structure with a
header and a body
• The header contains information used by the “Internet
operating system”
– Destination, Source, Routing, Context, Sequence
Number …
• The message body is partly further information used by the
operating system and partly information for application
when it is not looked at by “operating system” except to
encrypt, compress it etc.
– Note WS-Security supports separate encryption for
different parts of a document
• Much discussion in field revolves around what is referenced
in header!
– e.g. WSRF adds a lot to header
Deployment Issues for “System Services”
• “System Services” are ones that act before the real
application logic of a service
• They gobble up part of the SOAP header identified
by the namespace they care about and possibly part
or all of the SOAP body
– e.g. the XML elements in header from the WS-RM
namespace
• They return a modified SOAP header and body to
next handler in chain
Header
WS-RM WS-……..
Body Handler Handler
e.g. ……. Could be WS-Eventing WS-Transfer ….
Messaging Structure
• Communication Services are messaging
(transport protocol, routing) using SOAP
protocol
Messaging
Service Service
itself itself
Process SOAP Process SOAP
Body Header Header Body
Customizable Handler
Chain processes
SOAP Header
Invoke Other Services
from Header or Body
Application Specific Grids Higher
Generally Useful Services and Grids Level
Workflow WSFL/BPEL Services
Service Management (“Context etc.”) Service
Service Discovery (UDDI) / Information Context
Service Internet Transport Protocol Service
Service Interfaces WSDL Internet
Base Hosting Environment
Protocol HTTP FTP DNS …
Presentation XDR … Bit level
Session SSH … Internet
Transport TCP UDP … (OSI
Network IP … Stack)
Data Link / Physical
Layered Architecture for Web Services and Grids
Working up from the Bottom
We have the classic (CISCO, Juniper ….) Internet routing the
flood of ordinary packets in OSI stack architecture
Web Services build the “Service Internet” or IOI (Internet on
Internet) with
• Routing via WS-Addressing not IP header
• Fault Tolerance (WS-RM not TCP)
• Security (WS-Security/SecureConversation not IPSec/SSL)
• Data Transmission by WS-Transfer not HTTP
• Information Services (UDDI/WS-Context not
DNS/Configuration files)
• At message/web service level and not packet/IP address level
Software-based Service Internet possible as computers “fast”
Familiar from Peer-to-peer networks and built as a software
overlay network defining Grid (analogy is VPN)
SOAP Header contains all information needed for the “Service
Internet” (Grid Operating System) with SOAP Body containing
information for Grid application service
Consequences of Rule of the Millisecond
• Useful to remember critical time scales
– 1) 0.000001 ms – CPU does a calculation
– 2a) 0.001 to 0.01 ms – Parallel Computing MPI latency
– 2b) 0.001 to 0.01 ms – Overhead of a Method Call
– 3) 1 ms – wake-up a thread or process
– 4) 10 to 1000 ms – Internet delay
• 2a), 4) implies geographically distributed metacomputing
can’t in general compete with parallel systems
• 3) << 4) implies a software overlay network is possible
without significant overhead
– We need to explain why it adds value of course!
• 2b) versus 3) and 4) describes regions where method and
message based programming paradigms important
Linking Modules
Closely coupled Java/Python … Coarse Grain Service Model
Module Module Service Service
Messages
B A B A
Method Calls 0.1 to 1000 millisecond latency
.001 to 1 millisecond
From method based to RPC to message based to event-based
publish-subscribe Message Oriented Middleware
“Listener”
Subscribe Publisher
to Events Post Events
Service B Message Queue in the Sky Service A
What is a Simple Service?
• Take any system – it has multiple functionalities
– We can implement each functionality as an independent distributed
service
– Or we can bundle multiple functionalities in a single service
• Whether functionality is an independent service or one of many
method calls into a “glob of software”, we can always make them as
Web services by converting interface to WSDL
• Simple services are gotten by taking functionalities and making as
small as possible subject to “rule of millisecond”
– Distributed services incur messaging overhead of one (local) to
100’s (far apart) of milliseconds to use message rather than method
call
– Use scripting or compiled integration of functionalities ONLY
when require <1 millisecond interaction latency
• Apache web site has many projects that are multiple functionalities
presented as (Java) globs and NOT (Java) Simple Services
– Makes it hard to integrate sharing common security, user profile,
file access .. services
Grids of Grids of Simple Services
• Link via methods messages streams
• Services and Grids are linked by messages
• Internally to service, functionalities are linked by methods
• A simple service is the smallest Grid
• We are familiar with method-linked hierarchy
Lines of Code Methods Objects Programs Packages
Methods Services Component Grids
CPUs Clusters Compute
Resource Grids Overlay
MPPs and Compose
Grids of Grids
Federated
Databases
Databases Data
Resource Grids
Sensor Sensor Nets
Component Grids?
• So we build collections of Web Services which we
package as component Grids
– Visualization Grid
– Sensor Grid
– Utility Computing Grid
– Person (Community) Grid
– Earthquake Simulation Grid
– Control Room Grid
– Crisis Management Grid
• We build bigger Grids by composing component
Grids using the Service Internet
Electricity Gas CIGrid
Flood CIGrid
… CIGrid …
Flood Services Gas Services
and Filters and Filters
Collaboration Grid Portals Visualization Grid
Sensor Grid GIS Grid Compute Grid
Data Access/Storage
Registry Metadata
Core Grid Services
Security Notification Workflow Messaging
Physical Network
Critical Infrastructure (CI) Grids built as Grids of Grids
USArray
Seismic
Sensors
a
Site-specific Irregular a
Scalar Measurements Constellations for Plate
a
Ice Sheets Boundary-Scale Vector
Measurements
Volcanoes
PBO
Greenland
Long Valley, CA
Topography
1 km
Stress Change
Northridge, CA
Earthquakes Hector Mine, CA
Repositories Sensors Streaming Field Trip Data
Federated Databases Data
Database Database
Sensor Grid
Database Grid
Research SERVOGrid Education
Compute Grid
Customization
Data
Filter
? GIS
Discovery Grid
Services
From
Research
to Education
Services Research Services
Simulations Analysis and Education
Visualization Grid
Portal Computer
Geoscience Research and Education Grids Farm
CERN LHC Data Analysis Grid
Related docs
