Integration of Portals and Web Services

Document Sample
Integration of Portals and Web Services Powered By Docstoc
					        Integration of Portals and
              Web Services
            NCSA Expedition Meeting March 15 2002

            PTLIU Laboratory for Community Grids
   Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn
                   Computer Science, Informatics, Physics
                 Indiana University, Bloomington IN 47404


6/16/2010               uri="           1
              What is a Web Service I
    A web service is a computer program running on either the local
     or remote machine with a set of well defined interfaces (ports)
     specified in XML (WSDL)
    In principle, computer program can be in any language (Fortran
     .. Java .. Perl .. Python) and the interfaces can be implemented in
     any way what so ever
       • Interfaces can be method calls, Java RMI Messages, CGI Web
         invocations, totally compiled away (inlining) but
    The simplest implementations involve XML messages (SOAP)
     and programs written in net friendly languages like Java and
    Web Services separate the meaning of a port (message) interface
     from its implementation
    Enhances/Enables Re-usable component model of ANY
     electronic resource
6/16/2010   uri="" email=""   2
             What is a Web Service II
   Web Services have important implication that ALL
    interfaces are XML messages based. In contrast
   Most Windows programs have interfaces defined as
    interrupts due to user inputs
   Most software have interfaces defined as methods which
    might be implemented as a message but this is often
    NOT explicit
                                       WSDL interfaces                                            Credit Card

                                  Security                         Catalog

                                WSDL interfaces                                                   shipping
6/16/2010   uri="" email=""   3
            What is a Web Service III
   “Everything electronic” is a resource
     • Computers; Programs; People
     • Data (from sensors to this presentation to email to
   “Everything electronic” is a distributed object
   All resources have interfaces which are defined in XML for
    both properties (data-structure) and methods (service,
    function, subroutine) (Resources are Services)
     • We can assume that a data-structure property has
       getproperty() and setproperty(value) methods to act as
   All resources are linked by messages with structure, which
    must be specifiable in XML
   All resources have a URI such as unique://a/b/c …….
6/16/2010   uri="" email=""   4
       Classic Grid Architecture                                                                               Resources

      Database                                                                Database

                                                                Content Access
                                Composition                                                        Middle Tier
       Netsolve                                                                                    Brokers
                                                                             Security              Service Providers

        Middle Tier becomes Web Services

6/16/2010         Clients                                                Users and Devices
            uri="" email=""   5
            Raw                                 Raw Data                              Raw Data
                                              (Virtual) XML Data Interface

                                               Web Service (WS)

               WS                                        WS                                         WS

                                           XML WS to WS Interfaces                        etc.

    WS                      WS                           WS                           WS                              WS

                         (Virtual) XML Knowledge (User) Interface

                           Render to XML Display Format
                                                                                                 (Virtual) XML
             Clients                                                                           Rendering Interface
6/16/2010   uri="" email=""        6
                      WSDL Abstractions
   WSDL abstracts a program as an entity that does
    something given one or more inputs with its results
    defined by streams on one or more outputs.
   Functions are defined by method name and
    methodname(parm1,parm2, … parmN)
      • Where parameters are “Input” “Output” or both
   In WSDL, we will have a Web Service which like a
    (Java or CORBA Program) can be thought of as a
    (distributed) object with many methods
      • Instead of a function call, the “calling routine” sends an
        XML message to the Web Service specifying methodname
        and values of the parameters
      • Note name of function is just another parameter
6/16/2010   uri="" email=""   7
        Details of WSDL Protocol Stack
   UDDI finds where programs are                                                    UDDI or WSIL
     • remote( (distributed) programs
       are just Web Services                                                                 WSFL

   WSFL links programs together
    (under revision?)
   WSDL defines interface (methods,                                                  SOAP or RMI
    parameters, data formats)
   SOAP defines structure of message                                               HTTP or SMTP
                                                                                   or IIOP or RMTP
    including serialization of information
   HTTP is negotiation/transport                                                           TCP/IP
   TCP/IP is layers 3-4 of OSI                                                    Physical Network
   Physical Network is layer 1 of OSI
6/16/2010   uri="" email=""   8
            Database                                                 JXTA

                                 Web Service Interfaces

                                      Integrate P2P                              Event/
                   Brokers   Peer to Peer Grid
                                      and Grid/WS

                                  Web Service Interfaces


6/16/2010                                                                            Peer to Peer Grid
             uri="" email=""
            Examples of Web Services I
    OGSA (Open Grid Service Architecture)
       • Integrate Web Service and Grid Concepts and allows Globus
         to be implemented as Web Services
    Audio-Video Conferencing as a Web Service
       • Integrates H323, SIP, JXTA (etc.) protocols by mapping to
         single XML Interface
       • Provides VRVS reflector model from Messaging Web Service
    Messaging or Event Web Service provides intelligent
     routing and buffering of messages
    Performance as Web Service (yesterday)
       • Allow access to distributed monitoring data, analysis, models,
         and final benchmarks with interoperable XML interfaces
6/16/2010    uri="" email=""   10
            Examples of Web Services II
   Computing as a Web service
      • More on this today – job submittal, status, composition, data services,
        visualization etc.
   Education as a Web Service
      • One of easiest to do as object standards well defined (IMS) and little
        performance issues
      • Grading, Homework submission, registration, assessment etc.
   Universal Access and Web Services
      • As Web Services allow multiple implementation of a particular interface,
        one can adjust to needs of particular clients (PDA v. versus, impaired
        sight etc.)
      • Can build custom implementations of certain web services for particular
        communities but re-use others
   Collaborative Web Services
      • As interfaces all message based, relatively easy to share Web Services
6/16/2010    uri="" email=""   11
               Portals and Web Services
   Web Services allow us to build a component model (see
    CCA) for resources.
   Each resource naturally has a user interface (which
    might be customized for user)
   Web Service <--> Portlet
   Natural to use a component model for portal building
    displayed web page from collection of portlets
      • So can customize each portlet and customize which portlets
        you want
   Apache Jetspeed seems good open source technology
    supporting this model
      • JSP model is better than say a client-side Java integration in
        that also message based so this is “Portal as a Web Service”

6/16/2010   uri="" email=""   12
              Component –based Portal
    Web Services provide a component model for the
     middleware (see large “common component
     architecture” effort in Dept. of Energy)
    Should match each WSDL component with a
     corresponding user interface component
    Thus one “must use” a component model for the portal
     with again an XML specification (portalML) of portal

6/16/2010   uri="" email=""   13
        Meta-Data and Web Services
    Enriching resources with meta-data is critical idea
       • Enables one to identify and link resources around the globe
       • Allows one to find out “meaning” of a Web service not just
         syntax of interface
    Semantic Grid implies linkage of Grid/Web services
     enabled by meta-data leading to “digital brilliance”
     phase transition
    We can experiment with Semantic Web techniques for
     specifying meta-data RDF DAML OIL
    These encompass both straightforward enriched data
     as well as Artificial Intelligence assertions

6/16/2010   uri="" email=""   14
                            RDF in a Nutshell
                    Subject                                                                  Object
   The essence of RDF is a statement which has a subject and
    predicate and an object
   The “subject” has the property “predicate” with value “object”
   The Web Page has the title “RDF Model and Syntax”
   Mary had a husband called John
   Statements can specify metadata but more generally any facts are
    expressible in this way
   One can associate multiple objects with a subject and so build up
    a general node of a tree with parent as subject and objects as
   DAML and OIL extend RDF with more powerful relations and
6/16/2010   uri="" email=""   15
            An Example of RDF and Dublin Core
   <rdf:RDF xmlns:rdf=“”
   <rdf:Description about=“”>
   <dc:Title>D-Lib Program - Research in Digital Libraries</dc:Title>
    <dc:Description>The D-Lib program supports the community of people with
    research interests in digital libraries and electronic publishing.
    <dc:Publisher>Corporation For National Research Initiatives</dc:Publisher>
   <dc:Subject>
      • <rdf:Bag> <rdf:li> Research; statistical methods</rdf:li>
        <rdf:li> Education, research, related topics</rdf:li>
        <rdf:li> Library use Studies </rdf:li> </rdf:Bag>
   </dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type>
   <dc:Format>text/html</dc:Format>
   <dc:Language>en</dc:Language>
   </rdf:Description> </rdf:RDF>
6/16/2010    uri="" email=""   16
                     What could one do?
    Get applications together and decide on set of common
     web services
       • Service and methods within services
       • At “Grid Computing” and “Application” level
    Implement generalized meta-data service at
     application level – generate, access
       • Role of RDF DAML OIL IMS etc.
    Define both middle tier (WS) and user interface
     (portal) components
     Agree on pilot projects demonstrating both
     interoperable WS/Portal components and complete
     applications built from WS/Portal components
    Feed lessons into W3C and Grid processes
6/16/2010   uri="" email=""   17
            Collaborative Web Services
   First note that there are two distinct concepts
   Collaboration as a Web Service
     • Such as “Audio-Video Conferencing” as a Web
       Service or “Text Chat as a Web Service”
   Collaborative Web Services
     • Here we view a Web Service as specifying a
       (distributed) object and wish to share an object
     • Object could be a Web page, a Job status form, a
       scientific visualization, a PowerPoint slide etc. (not
       all of these are Web Services but all should be)

6/16/2010   uri="" email=""   18
 Why Web Services for Collaboration
    Well everything is meant to be a Web Service but also:
    Web Services are MUCH EASIER to make
     collaborative than other objects because all input and
     output is defined by uniform XML messages
       • You need to teach your message service about collaboration!
       • Note local applications are NOT Web Services – input is
         things like “user mouse click” represented by “method
         events” (UI program interrupts) not “XML message events”
    The elegance of collaborative web services suggest that
     it could be easiest to make object X (such as
     PowerPoint or SVG) collaborative not by traditional
     direct methods but by converting to a Web Service
6/16/2010   uri="" email=""   19
     CollabWS: Collaboration as a Web Service
      This includes Audio-Video conferencing, Instant
       Messengers, shared applications and tools
      There are basic standards H323 and SIP
      There are peer to peer collaboration approaches like
       JXTA from Sun
      There are various commercial products – Groove
       Networks effectively defines some shared application
      CollabWS defines a superset of collaboration
       properties and subservices in XML/WSDL
      Then you provide several bindings of collabws.wsdl
       to new transports and old ones (binary H323)
6/16/2010   uri="" email=""   20

6/16/2010   uri="" email=""   21
   H323                                                       SIP

                                                                                      H323                   JXTA
6/16/2010   uri="" email=""   22
                A Typical SIP Message
    Via: SIP/2.0/UDP
    To: Bob (
    From: Bob (;tag=456248
    CSeq: 1826 REGISTER
    Contact: (sip:bob@
    Expires: 7200
    Contact-Length: 0
   Initially build a wrapper that accepts such messages
    and converts to …..

6/16/2010   uri="" email=""   23
     Collaboration as a Web service - now

6/16/2010   uri="" email=""   24
                                        SIP as XML
   <sipasxml>
    <via> SIP/2.0/UDP</via>
    <to> Bob ( </to>
    <from tag="456248" > Bob ( </from>
    <cseq> 1826 REGISTER </cseq>
    <contact> (sip:bob@ </contact>
    <expires> 7200 </expires>
    <contact-length> 0 </contact-length>
   So this is either an internal format to which we map SIP
    messages or it is an external format used for Collaboration
    Web Service
6/16/2010   uri="" email=""   25
     Future Collaboration Web Service

6/16/2010   uri="" email=""   26
 Messaging/Events as a Web Service
   We can implement messaging subsystem (between
    WSDL resources) with either direct messages or by a
    queued system where you publish messages to queues
    and subscribe as receiver to particular queues
   There are many different publish/subscribe models
      • JMS is a cluster of central servers
      • JXTA is a very dynamic Peer to Peer model where pipes are
        queues and topics (metadata) are service advertisements
   Implement JMS API with JXTA protocol – different
    WSDL bindings here have different fault
    tolerance/reliability semantics
      • Could use JMS as long distance “carrier” between JXTA peers
      • JXTA provides higher performance than JMS for nearby
6/16/2010   uri="" email=""   27
                              Event Web Service
    Routing                                                                       Filter                    workflow
                                  Source Matching

             Web                                     (Virtual)                                  Web
            Service 1                                 Queue                                    Service 2

                       WSDL                                                            WSDL
                       Ports                                                           Ports
   Filter is mapping to PDA or slow communication
    channel (universal access) – see our PDA adaptor
   Workflow natural as all messages “intercepted” by
    Event Web Service
   Routing illustrated by JXTA
   Destination-Source matching illustrated by JMS
6/16/2010      uri="" email=""   28
                              Broker Network
                                                              Software multicast
                                                              P2P: Brokers are clients




6/16/2010   uri="" email=""    29
       Narada JXTA/JMS Integration
   Is a network of event brokers which can reliably deliver XML
    specified events
   Using openJMS selection module, becomes a distributed or
    conventional Java Message Service
   Linking special JXTA
    proxies, it can link
    JXTA communities
   Think of JXTA JMS and
    Narada as different
    bindings to
    web service

6/16/2010   uri="" email=""   30
            Education as a Web Service
     Can link to Science as a Web Service and substitute educational
     “Learning Object” XML standards already exist from IMS/ADL – need to update architecture
     Web Services for virtual university include:
     Registration
     Performance (grading)
     Authoring of Curriculum
     Online laboratories for real and virtual instruments
     Homework submission
     Quizzes of various types (multiple choice, random parameters)
     Assessment data access and analysis
     Synchronous Delivery of Curricula
     Scheduling of courses and mentoring sessions
     Asynchronous access, data-mining and knowledge discovery
     Learning Plan agents to guide students and teachers
6/16/2010    uri="" email=""   31
            CMS (Content Management)
                as a Web Service
   For meta-data, build federated database as a Web
      • Storage, update, lookup of objects
      • Registration of different control engines
   For content, if W3C DOM and Java API as for SVG
    and “decent” XHTML browsers, then can wrap DOM
    as a web service for all interface functions
      • Can of course do this for IE5 but document object model
   Build collaboration systems as sharing Web Service
    interfaces (not COM or other object API as proposed
    by Groove Networks)
6/16/2010   uri="" email=""   32
       Integration of XML/Web Services
              into an Enterprise I
   You are the CIO of a little known state’s (call it Xanadu)
    education system and been tasked to position it as a 2005 leader
    in technology enhanced education
   Your initial review reveals a multitude of incompatible existing
    information systems distributed over Xanadu’s 50 campuses
   Your cradle to grave community education mandate (which got
    the Governor elected) implies integration of all aspects of
    education in your information system
   You note that performance issues are much less important in
    this application than interoperability (it currently takes months
    to get anything done so what's an extra millisecond or so at a
    WSDL interface) , so even in current primitive/immature state
    Web services look attractive
6/16/2010   uri="" email=""   33
            Enterprise Web Services II
   We start a requirements analysis and a catalog of existing “raw
    resources”. This will lead to the clean internal XML(WSDL)
    interfaces XIF and to the external XML interfaces (XEF) to the
    raw resources
   Xanadu Internal Framework (XIF) built on top of XML
    Messaging, Security etc. Services
   We find that The Grid Forum community has defined computing
    interfaces which we will adopt to support infrastructure needed
    for use of simulations and data repositories in curriculum
   We find that Microsoft, the Liberty Alliance and the Grid have
    Security schemes but are relieved that with a little bit of XSLT
    (transformations) we can map between them and can define
    authentication & access control as a Web Service
   We join the IMS consortium and adopt their standards for
    learning meta-data and user related information
   We adopt the AICC and ADL SCORM standards for “learning
    objects” (course pages)

6/16/2010    uri="" email=""   34
        Enterprise Web Services III
   We define Xanadu educational agents which use XIF plus
    RDF/DAML-OIL to be able to capture educational goals
   We expect commercial standards compliant solutions will be
    available for most other needed Educational web services
   We find that we must interface with a dreadful collection of
    external raw resources coming from legacy PeopleSoft SAP
    Oracle and Microsoft products
      • As our new model is object (XML) and open standards based we build a
        separate XII (Xanadu Information Infrastructure) supporting XIF objects
   We analyze structure of legacy systems and their relation to XIF
    and define the external framework XEF which captures much of
    the raw resources
      • This XEF Schema is extensible to capture quirks of each legacy system
      • We build a set of Web services that convert XEF into XIF
   We inform vendors that we will only install XIF compliant
    systems in all future purchases
6/16/2010   uri="" email=""   35
            WSDL from a Gateway View

6/16/2010   uri="" email=""   36
                  7 Primitives in WSDL
   types: which provides data type definitions used to describe the
    messages exchanged.
   message: which represents an abstract definition of the data being
    transmitted. A message consists of logical parts, each of which is
    associated with a definition within some type system.
   operation– an abstract description of an action supported by the
   portType: which is a set of abstract operations. Each operation
    refers to an input message and output messages.
   binding: which specifies concrete protocol and data format
    specifications for the operations and messages defined by a
    particular portType.
   port: which specifies an address for a binding, thus defining a
    single communication endpoint.
   service: which is used to aggregate a set of related ports

6/16/2010   uri="" email=""   37
 Discussion of 7 WSDL Primitives
   types specify data-structures which are equivalent to arguments of methods
   message specifies collections of types and is equivalent to set of
    arguments in a method call. Note that it is an “abstract method” in Java
   operation is a a collection of input output and fault messages; there are
    4 types of operation one-way(service just receives a message), request-
    response(RPC), solicit-response, notification (services pushes out a
   portType represents a single channel that can support multiple
    operations. It is “abstract” as specified as a set of operations. It is
    equivalent to a “interface or abstract class” in Java
   binding tells you transport and message format for a porttype (which
    can have multiple bindings to reflect say performance-portability
   port combines a binding and an endpoint network address (URL) and is
    like a “class instance”
   service consists of multiple ports and is equivalent to a “program” in
6/16/2010   uri="" email=""   38
            Gateway and Web Services
   We will use the Gateway Computing Portal as an
    example (
      • It is largely built using CORBA with a Java Server Pages
        front end
   Several capabilities have been interfaced using WSDL
      • Job Submission (11 Methods including execute local and
        remote command, copy files etc. as well as Submit Job)
      • Manage WebFlow Session (67 Methods)
      • Generate Batch Script (just 1 method but two
        implementations developed – one at SDSC and one at Indiana
        – with UDDI to manage)
      • Each is one service – could have used finer grain services
      • Sample files are at
6/16/2010   uri="" email=""   39
                     Gateway Architecture
           As needed DoD approved Kerberos/SecureID security,
            does not usually link to standard Grid technology (Globus)

         Pre WSDL                                                                              Post WSDL

         backend                                                                                            Add Castor
                                                                                                            Java to XML
                         or submit to local
                         queue system                                                              Will replace
                                                                              CORBA                historical CORBA
                        Corba middle tier                                                          By EJB
                        SECIOP with                             WSDL
                        Java modules
                                                                         SOAP wrapper
                  JSP SSL                                                 JSP Servlet

6/16/2010      uri="" email=""   40
6/16/2010   uri="" email=""   41
            SOAP and Gateway Portal I
      Having specified service in WSDL, the run-time is
       implemented in SOAP which is “just” an XML header
       (info needed by transport – empty here) and body
      Here is SOAP transported by HTTP message
      This is execLocalCommand WSDL operation to run
       one particular command (ls) on current WebFlow
                                                                      HTTP Header

                                                                            Argument of operation

                                          SOAP EnvelopeSpecify ls as
                                                        and body
             uri="" email=""   42

                                                                                                              and body

                                                        SOAP and Gateway
                                                            Portal II

                                                        And this is the result of ls
                                                         sent back to client in
                                                         SOAP over HTTP

6/16/2010   uri="" email=""   43
            WSDL Message Example
<message name="submitRequest">
   <part name="xmljob" type="xsd:string"/>
<message name="submitResponse">
   <part name="response" type="xsd:string"/>

   For the batch script service, we pass the XML description
   of the job as a string and get back the script as a string.
   In general, any XML primitive or complex types can be
   used in messages.
       We could improve our service by defining a BatchScript
       complex type.

6/16/2010   uri="" email=""   44
            WSDL portTypes Example
 <portType name="BatchScriptServicePortType">
    <operation name="batchGen">
      <output message="tns:submitResponse"
      <input message="tns:submitRequest"
A portType corresponds to a Java class, so if we
compile this WSDL to make client stubs, we will
generate a class.

6/16/2010    uri="" email=""   45
            WSDL SOAP Binding Example
<binding name="BatchBinding" type="tns:BatchScriptServicePortType">
  <soap:binding style="rpc"
  <operation name="batchGen">
     <soap:operation soapAction=""/>
       <soap:body use="encoded“ namespace="urn:BatchScriptService"
       <soap:body use="encoded" namespace="urn:BatchScriptService“
  </operation>         note binding’s “type” attribute points back
                                    to the portType tag by name

6/16/2010    uri="" email=""   46
            WSDL Ports and Services
  <service name="BatchScriptService">
     <documentation>BS stands for Batch Script
     <port binding="BatchBinding”
       <soap:address location=
   </service>      ports are concrete implementations of
  </definitions> portTypes and point back to a particular
                   binding (by name).
                                 They also point to the specific location of a
                                 server that implements the service.
                                 A service is a collection of one or more ports.

6/16/2010   uri="" email=""   47
     What is a Virtual XML Interface
   We can specify interfaces in XML but we are not required to
    implement in XML.
   Example 1:We aren’t likely to change syntax of mail
    Reply-to:Geoffrey Fox <>
    To: Geoffrey Fox <>
    Subject: A Test for Tutorial

    A simple mail message
    Geoffrey Fox FAX 8128567972
    Phones Cell 315-254-6387 Home 812323919
   But we could specify and indeed store in XML with transport
    done using conventional SMTP.
   So conventional mail is easy to give a virtual XML specification
    for with
    name:value      becoming
    <name>value</name> but this is not only way
6/16/2010   uri="" email=""   48
                                      Mail in XML
    <mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” >
          <reply-to email=“” >Geoffrey Fox</reply-to>
          <to email=“” >Geoffrey Fox</to>
          <subject>A Test for Tutorial</subject>
        <message whitespace="collapse"> A simple mail
        <signature personuri=“ssn://123/45/6789” whitespace=“preserve” >
          Geoffrey Fox FAX 8128567972
          Phones Cell 315-254-6387 Home 812323919 </signature>
    Such an interface could be used by “User Messaging as a Web service” which
     could perhaps integrate News Groups, e-mail, text chat, instant messenger,
6/16/2010   uri="" email=""   49
                    Virtual XML for MPI I
    In days gone by, we used to study:
     MPI_SEND(buf, count, datatype, dest, tag, comm)
     [ IN buf] initial address of send buffer (choice)
     [ IN count] number of elements in send buffer (nonnegative integer)
     [ IN datatype] datatype of each send buffer element (handle)
     [ IN dest] rank of destination (integer)
     [ IN tag] message tag (integer)
     [ IN comm] communicator (handle)
    Now our SPMD code is a Web service running in some
     large scale parallel machine – perhaps integrated with
     an Apache server running on node
       • Need to support distributed and shared memory
    We want to define all interfaces in WSDL – at least all
     of them which could be “external” ports

6/16/2010   uri="" email=""   50
                       Virtual XML for MPI II
     datatype becomes an XML Schema specifying structure of
      transmitted data – could either be a URI for Schema or its
        • There would be a built-in schema suitable for handling standard
          Fortran/C datatypes
     buf is a handle to an XML instance of Schema datatype
     count is still an integer (xs:integer where xs is Schema
      namespace xmlns:xs=
        • For some bindings count is implied by buf
     dest becomes a URI with special notation such as
      legacympi://procnum/local/nnn for processor numbers
     tag becomes an instance of (new) mpitag Schema
     comm becomes an instance of (new) mpicommunicator Schema

6/16/2010   uri="" email=""   51
                      Virtual XML for MPI III
   So one defines mpi.wsdl with all 100 or so methods
    defined as (abstract) operations
     • We will define several bindings including the classic HPCC
       method calls for Fortran C C++ and Java with either
       distributed memory or shared memory mode
     • For C++ and Java we would implements bindings for either
       “object” or “packed consecutive byte” interpretation of buf
   Note one can use both static and run-time compilation to map
    XML specified services into particular implementations
   MPI_INIT() might be implemented with JXTA (peer-to-peer
    advertisments) to find nodes; the communicator might be
    transmitted by SOAP while buf transmission uses MPICH
   This brings Web Services to MPI but also MPI to web
6/16/2010   uri="" email=""   52