Reusable Components for Grid Computing Portals

Reusable Components for Grid Computing Portals Marlon Pierce Community Grids Lab Indiana University Grids Today and Tomorrow    Grid software enables loosely coupled, globally distributed computing  “Virtual Organizations”. What does that really mean?  Specific services such as global authentication, resource allocation management, aggregated information services  Centered around a few wire protocols and service implementations What‟s next? Open Grid Service Architecture  Use XML (WSDL) to provide a service definition language.  Extend WSDL to support metadata about services. What Is Missing?    Grids are designed to enable Virtual Organizations.  Inter-organizational collaboration But we must also support the Real User  Provide access to the Grid from any computer (or anywhere).  Provide user interfaces to Grid services.  Provide customizable front ends that contain the service front ends. Grid Computing Environments  Browser-based Web portals Grid Computing Environments  Organizations setting up Grids have seen the value of developing user environments, or Grid Computing Environments.   28 articles in November-December 2002 issue of Concurrency and Computation: Practice and Experience. IPG Launchpad, HotPage, Alliance Portal, and others  World-wide development community interacts through the GCE research group in the Global Grid Forum.  G. Fox (IU), D. Gannon (IU), and M. Thomas (TACC) cochair. Reusability of components Common frameworks  Grid portal technology is coming of age   Example GCE: Gateway Portal   Developed for DOD supercomputing centers (ARL and ASC MSRCs). Support source-restricted (commercial or otherwise) applications  Ansys, Abaqus, ZNSFlow, Fluent  Developed to support typical, if simple, high performance computing services   Batch script generation, job submission and monitoring, file management and transfer. Do it all securely Characteristics of Portals    Framework contains user interfaces to the services. Backend services accessed through service proxies. The convergent/emergent architecture is a three tiered model. Grid and Web Protocols Portal Client Stub JDBC, Local, or Remote Connection Portal User Interface Database Service Database Portal Client Stub Grid Resource Broker Service HPC or Compute Cluster Portal Client Stub Information and Data Services Grid Information Services, SRB The three-tiered architecture is a standard for accessing Grid and other services. Sharing Portal Services  Given that everyone builds essentially around the same architecture     How do I build a client to interact with someone else‟s services? How do I build a compatible service implementation? How can I take someone else‟s end-to-end solution and plug it into my portal. How do I avoid reinventing basic services like login, view customization, access restrictions on interfaces.  To explore possible solutions, we chose to implement a new portal project, QuakeSim, around the Web services and Portlet models. QuakeSim Portal  A number of simulation methods for studying earthquakes are being developed by GEM consortium including:    Simplex, Disloc (JPL) Virtual California (UC-Davis) PARK codes (Brown) Need to manage information about distributed data sources: multiple databases, sensors, simulated data. Need to organize, manage information about multiple code installation sites. Need to simplify access to data, use of codes, and use of visualization/analysis tools for broad range of users Need to link together  As codes become more robust and accepted, problems emerge:      NASA funded activity to develop SERVOGrid Interoperability framework User Interface HTTPS Server with Client Applications and Stubs SOAP WSDL Interface DB Service 1 Job Sub/Mon And File Services DB Service 2 JDBC Operating and Queuing Systems Host 2 JDBC DB DB Host 1 Host 3 Computing Portal Grid Web Services   We have built a suite of general purpose Grid Web services for managing distributed applications. Core Computing services define general purpose functions:   Ex: job submission, file transfer, job monitoring, management of jobs and results Described as a GridShell as plays same role to Grid that Shell does for UNIX on a single machine  Application Grid Web services include metadata about applications.   Built on top of core services. Original application NOT changed  We have developed a toolkit that allows one to convert general software packages into Grid Web Services and manage application collections Application Grid Web Services   AGWS are designed to make scientific applications (i.e. earthquake modeling codes) into Grid Resources AGWS services are described by two XML Schemas:  Abstract descriptors describe application options. Used by the application developer to deploy his/her service into the portal.  Instance descriptors describe particular user choices and archive them for later browsing and resubmission. User Application Selection and Submission Generate script for job submission Select desired application and host Administer Grid Portal Provide information about application and host parameters Select application to edit Portlets for Reusable Portal Components      What we found was that groups did not really want to use common interfaces so much as share end-to-end services (user interfaces-client stubs-service implementations). Portlets/containers provide a simple way to do this. The container implements all portal specific services  Manages user customizations, logins, access controls Container treats all web content as generic „portlet‟ objects.  Controls which portlets are displayed and how they are arranged. Portlets and containers are implemented in Java  Tomcat webapp Value of the Portlet Approach  With portlets, we have a common infrastructure for managing content.   I don‟t have to reinvent login, user customization services. But I may choose to add my own service implementation in a well defined way.  Content (and service user interfaces) are added in a well defined way  Edit an xml registry file. Portlet Implementations  Several groups (IU, TACC, NCSA, UMich) are using Jetspeed  Open source portlet implementation from Jakarta Add custom services for message boards, chats, etc. Develop specific portlets to Grid services (like GridFTP). Build general purpose portlets to support needs of Grid service interfaces    We extend it to    Session state conversations, multipage content, security Bridge to legacy JSP and non-Java Web interfaces Local Portlets Java COG API Java CoG Kit Remote Interfaces CoG Grid Protocols Grid Services GRAM, MDS-LDAD MyProxy Portal Proxy Portlets Grid Services Other Services HTTP Stubs SOAP Teamlets Service CHEF API Services Jetspeed Internal Services The Grid Portal Consortium's initial architecture aggregates multiple services into a single portal using portlet containers. Portlet Longevity   Portlets have become popular in commercial enterprise servers The portlet API is being standardized through the Java Community Process.  Participants include IBM, Oracle, BEA, and others.  We anticipate or will contribute to building the open source reference implementation of the standard. Portlets and Portal Stacks   User interfaces to Portal services (Code Submission, Job Monitoring, File Management for Host X) are all managed as portlets. Users, administrators can customize their portal interfaces to just precisely the services they want. Message Security, Information Services Aggregation Portals User facing Web Service Ports Application Grid Web Services Core Grid Services Future Developments  User interfaces and services need to get much more sophisticated, intelligent.   Case-based reasoning interface for Earthquake simulation codes. More standard collaboration services as portlets  Whiteboards, chat interfaces  Ubiquitous access in a standard fashion  Portlet repositories to allow community sharing of reusable components. More Information    My Email: marpierc@indiana.edu Gateway homepage: www.gatewayportal.org More publications: http://ptlportal.ucs.indiana.edu.