Building-a-Grid-Portal-for-TeraGrids-Big-Red

Document Sample
Building-a-Grid-Portal-for-TeraGrids-Big-Red Powered By Docstoc
					 Building Grid Portals with
 OGCE: Big Red Portal and
          GTLAB
Mehmet A. Nacar, Jong Youl Choi, Marlon
         Pierce, Geoffrey Fox
        Community Grids Lab
          Indiana University
                     Introduction
This is a two-part talk on OGCE activities
   Part 1: using OGCE’s current grid portal release to build a basic
    gateway for IU’s BigRed
      Jong Youl Choi
      Zero to Done in 2 months. YMMV
   Part 2: simplifying the process for building portals like this using
    tag libraries.
      Mehmet A. Nacar
      Finishing up ~2 years of software development
Big Red supercomputer is part of TeraGrid at
 Indiana University
  2048 cluster nodes, 4 terabyte memory
For more information on Big Red, see the Monday
 tutorial.

                                                                       2
              Portlets for Big Red
 We have developed several portlets for Big Red based on
  the OGCE software
    Improved batch job submission for LoadLeveler (IU) and PBS
     (NCSA).
    MEME job submission
       Interactive and batch
       Includes both GridFTP and GRAM clients
       Job tracking.
    MOAB queue monitoring for both entire machine and the specific
     user.
    Access to existing IU resources like the IU Knowledge Base.
 These portlets also work with other TeraGrid resources.
    We tested across IU and NCSA resources.
              Meme Portlet




The MEME portlet uses OGCE portal libraries to
upload and download files, submit jobs, and monitor
their progress.                                       5
            Dashboard Portlet




The dashboard portlet allows users to track jobs on the
selected resource. The user can view either his own set
of jobs or get information on all submitted jobs.
                                                          6
OGCE IFrame Portlet can be used to
integrate external sites.
OGCE GPIR portlet can interoperate with
TeraGrid GPIR services.
Manage TeraGrid MyProxy credentials with
the OGCE ProxyManager portlets.
OGCE file management client portlets
interact with TeraGrid GridFTP servers.
In addition to MEME, general purpose batch and interactive job
submission to GRAM, WS-GRAM is supported.
                     Using TeraGrid
                       Resources
 Although utilizing Big Red, the portlets can be used with
  any other gateway in TeraGrid
 Certificate username and local system user names may be
  different.
    To avoid confusion, we submit a whoami command to retrieve
     correct username at first place.
    Aside: Why can’t we have just one user name for all TeraGrid
     resources?
    You can--it’s your grid certificate DN:
     http://communitygrids.blogspot.com/2007/06/teragrid-single-sign-
     on.html
      Works with GSI enabled ssh.
 Dashboard portlet facilitates this schema by using GRAM
  service.
                                                                 13
                   Problems with
                 Portlet Development
 Grid portlets typically wrap each single Grid capability in a
  separate portlet
 Problem is that Grid portlets need to combine these operations
    Portlets are entire web applications, so we need a component model for
     portlets: reusable portlet parts
 JSF uses Inversion of Control pattern to implement its Model-
  View-Controller architecture
    Remove dependencies on the Servlet API.
    Backing beans are just beans, so can be reused more easily outside of
     web and portlet applications.
 JSF also provides an extensible framework (tag libraries)
 Apache JSF portlet bridge allows you to convert standalone
  JSF applications (development phase) into portlets
  (deployment phase).
                    Grid Tag Libraries
                   and Beans (GTLAB)
 GTLAB provides common components for building portlets
  using tags and reusable parts.
 The goal of GTLAB to simplify Grid portlet development
   Enable rapid development
 GTLAB capabilities include Grid operations with XML based
  tags within Java Server Faces (JSF) framework.
 Grid tag libraries are built using JSF custom component
  development techniques
 Grid tags are interfaces to backing Grid beans
   End users pass values to Grid beans by using tag attributes.
 We build on Java CoG 4’s abstraction layer.
 Each backing Grid bean has equal capability with a portlet
  application in case of Grid portlet approach.

                                                                   15
                        GTLAB Example
 • Grid tags are associated with Grid services via Grid beans
     • Grid Beans wrap the Java COG Kit (version 4)
 • We show an example JSF page section below.
     • This allows you to develop new Grid portlets with no additional Java code.
<html>
 <body>
  <f:form>
    <o:submit id=”test” action=”next_page” />
   <o:myproxy id=”pr” hostname=”gf1.ucs.indiana.edu” port=”7512”
         lifetime=”2” username=“mnacar” password=”***” />
   <o:jobsubmit id=”task” hostname=”cobalt.ncsa.teragrid.org”
         provider=”GT4” executable=”/bin/ls” stdout=”tmp/result
   stderr=”tmp/error” />
    </o:submit>
  </f:form>
 </body>
                                                                                    16
</html>
Grid Tags          Associated Grid Beans Features

<submit/>          ComponentBuilderBean   Creating components, job
                                          handlers, submitting jobs

<handler/>         MonitorBean            Handling monitoring page actions


<multitask/>       MultitaskBean          Constructing simple workflow


<dependency/>      MultitaskBean          Defining dependencies among sub
                                          jobs
<myproxy/>         MyproxyBean            Retrieving myproxy credential


<fileoperation/>   FileOprationBean       Providing Gridftp operations


<jobsubmission/>   JobSubmitBean          Providing GRAM job submissions


<filetransfer/>    FileTransferBean       Providing Gridftp file transfer


                   ResourceBean           Describes common properties
                                          among all tags and beans. Passing
                                          values given by standard visual
                                          JSF components.
                       How to integrate GTLAB
Multi-staged task
                                                                                                     Multitask
                                                                                                      provides a
                              Input
                                                                                                      simple Directed
                                                                                                      Acyclic Graph
    Task A
File Operation
                    depends         Task B
                                 File Transfer
                                                                                                      (DAG)
    (mkdir)
                                                                                                     This example
                                                 depends       Task C
                                                                                                      demonstrates a
                                                           Job Submission                             composite Grid
                                                                                                      job using multi-
                                                                        depends        Task D         staged multitask
                                                                                    File Transfer
                                                                                                     GTLAB handles
                                                                                                      lifecycle of DAG
                                                                                  Output              within JSF
                                                                                                      application
                                                                                                                 18
                    How to prepare
                   application pages
 Developers embed Grid tags snippet into JSF page
   These components are non-visual and are not displayed in
     HTML.
 Resource bean provides bridging with form inputs and GTLAB
  framework.
   <h:outputText value="Taskname: "/>
       <h:inputText value="#{resource.taskname}" />
       <o:multitask id="multi" persistent="true" task
      name="#{resource.taskname}" />
 Dynamic values to Grid tag attributes are provided by Resource
  bean.
 Only visual component is <o:submit/> tag that is associated
  with action method of GTLAB.
                                                           19
                 GTLAB Dashboard Portlet
                       Example
<o:submit id=”track” action=”list_page” />
 <o:multitask id=”dashboard” taskname=”track” persistent=”true” >
    <o:myproxy id=”proxy” hostname=”gf1.ucs.indiana.edu”        lifetime=”2”
              username=”#{resource.username}” password=”#{resource.password}” />

      <o:jobsubmit id=”jobA” hostname=”cobalt.ncsa.teragrid.org”
             provider=”GT4” executable=”/bin/whoami”
             stdout=”tmp/result”
             stderr=”tmp/error” />

     <o:jobsubmit id=”jobB” hostname=”cobalt.ncsa.teragrid.org”
            provider=”GT4” executable=”/bin/showq”
            stdin=”tmp/result” stdout=”tmp/list”
            stderr=”tmp/error” />

       <o:dependency id=”depend” task=”jobB” dependsOn=”jobA” />
    </o:multitask>
</o:submit>
                                                                           20
                   Tracking and
                  Managing Jobs
 GTLAB manages lifecycles of jobs and monitor their
  status.
 Grid operations are usually batch processes
   We provide callback mechanism to follow up the jobs
   GTLAB creates handlers for jobs and persistently stores them.
 GTLAB handlers manages the job events such as stop,
  cancel or resuming the running jobs.
 GTLAB provides archive for job metadata and allows
  managing the archive
   Handler tag helps to organize user’s job repository
   <o:handler id=”delete” action="#{monitor.delete}" >
      <f:param id="task" name="taskname“ value="#{task}"/>
    </o:handler>
                                                                21
          GTLAB Related Work
Grid Portlets 1.3 of GridSphere
  Now they are trying to decouple with GridSphere. It’s
   called Vine (Portlet Vine) as separate project
  Grid Portlets 1.3 provide API and UI tags to build Grid
   portlets
RSF (Reasonable Server Faces)
  Derived from JSF, but it separates HTML pages and
   backing beans
  RSF provides non-visual components unlike JSF
  Beans can be contained by Spring like containers.
     Lifecycles of beans managed by Spring
OGCE portlets
  Packages Velocity, JSP and JSF portlets
  Provides portlet package for several Grid applications
   such as Globus, Condor, SRB and GPIR
             More Information
GTLAB version 1.0 Beta release available at
  http://grids.ucs.indiana.edu/users/manacar/GTLAB-
   website
See link from main OGCE web site
  http://www.collab-ogce.org
Contact OGCE: discuss@ogce.org
Contact me: mpierce@cs.indiana.edu
Contact Mehmet: mnacar@indiana.edu
                                                23
    Conclusion and Future Work
We have described a science gateway to IU’s
 Big Red supercomputer.
  Initial effort was to build the portal based on OGCE
   portlets
Described the process for creating Grid portlets
 using GTLAB.
  We gained rapid development by using reusable
   components
We are investigating abstracting our approach to
 support different workflow engines.
Investigate integrating AJAX with our session
 management and monitoring.
  Please come to the Web 2.0 BOF Tuesday evening.
                                                      24
          Acknowledgments
This work is supported by the National
 Science Foundation’s Information
 Technology Research (NSF grant ITR-
 0428774, 0427264, 0426867 VLab) and
 Middleware Initiative (NSF Grant SCI
 0330613) programs.

				
DOCUMENT INFO
Shared By:
Tags: Build, ing-a
Stats:
views:3
posted:12/1/2009
language:English
pages:25
Description: Building-a-Grid-Portal-for-TeraGrids-Big-Red