ian foster by youssefadham


More Info
									IEEE DISTRIBUTED SYSTEMS ONLINE 1541-4922 © 2004 Published by the IEEE Computer Society
Vol. 5, No. 2; February 2004

Ian Foster on Recent Changes in the Grid
Mark Baker

Ian Foster is a senior scientist and head of the Distributed Systems Lab at
Argonne National Laboratory as well as the Arthur Holly Compton Professor of
Computer Science at the University of Chicago.

Foster is perhaps best known for his work as the originator, along with Carl
Kesselman and Steve Tuecke, of the Globus Toolkit. This open source software
provides a unifying framework for wide-area distributed computing; it consists of
a number of components, including ones for security, resource management,
communication, and data management. Foster and Kesselman first introduced
many of the concepts that underlie the Grid in their book The Grid, first published
in 1999. Since then, the Globus Toolkit has been in the vanguard of Grid
software infrastructure implementations, which has made it a key component in
the realization of the Grid.

Since its first release, the Globus Toolkit has undergone several changes and
transformations. Not least of these was the move toward a service-based
instantiation, which was based on the Open Grid Service Infrastructure, with the
release of Globus Toolkit v3 in July 2003. Much to the surprise of many in the
Grid community, on 20 January 2004 at GlobusWorld, Foster and his colleagues
proposed a further evolution of OGSI concepts into the Web Services Resource

IEEE Distributed Systems Online editor Mark Baker contacted Foster just after
the announcement of WSRF to discuss the recent changes and to get an idea of
   how these will affect the Grid community. (The accompanying sidebar offers a
   key to the acronyms mentioned here.)

   Mark Baker: OGSI-based Grid services was a major jump compared to the previous Grid
   architecture. The WSRF's introduction is yet another significant change to the Grid's
   plumbing. Apart from the obvious potential of convergence with Web services, who or what
   were the greatest influences on the move to adopt WSRF?

   Ian Foster: The WSRF proposal is a refactoring of OGSI concepts to align better with Web
   services. The need for this refactoring became apparent when Web services vendors indicated
   that while they recognized the importance of OGSI concepts, they would not adopt OGSI as it
   was then defined. This response threatened to prevent the ubiquitous support for Grid
   infrastructure that was a primary motivator for OGSI. So, while we in the Globus Alliance had
   little enthusiasm for revisiting the OGSI design (we'd much rather be developing higher-level
   services than fiddling with infrastructure), we concluded that some refactoring was justified if it
   could affect the convergence of Grid and Web services. We teamed up with Web services
   architects to study this issue, and the result is WSRF.

   The changes from OGSI to WSRF are primarily syntactic but also represent some useful
   progress. The separation of OGSI functionality into six independent specifications simplifies
   adoption, the use of WS-Addressing is a step forward, and less aggressive use of XML Schema
   and WSDL 2.0 features will facilitate the use of available tooling. Do these benefits justify the
   disruption associated with a revision to the Grid infrastructure specification? At a purely
   technical level, it's debatable, but the fact that we now have strong support in the Web services
   community for Grid infrastructure is a huge achievement: it means we'll see WSRF support in
   core Web services products, which is wonderful news for the Grid community. Also, we
   shouldn't overemphasize the magnitude of the change: while there's certainly some work
   involved in converting from OGSI to WSRF, it doesn't appear to be substantial, and we're
   working to ease this transition.

   Clearly, the planning and design of WSRF started some time ago. Many in the Grid
   community are concerned about the secrecy of this process and how few parties, apart from
   those under a nondisclosure agreement saw what was planned until WSRF was announced
   at GlobusWorld on 20 January 2004. Why was this secrecy necessary? Also, do you have
   concerns about a process that used to be open, and how it now appears to be similar to those
   we'd typically expect from the commercial world?

   Work on WSRF started late in the summer of 2003, following feedback on OGSI from the Web
   services community. Fortunately, we were successful in engaging senior Web services
   architects in this work. Unfortunately, they were only prepared to engage with us in a closed
   setting. Fortunately, thanks to much hard work, we were able to conclude this process quickly
   and then promptly release the specifications for public comment.
IEEE Distributed Systems Online February 2004
   I wouldn't argue that this process was ideal. However, we should judge the process by its
   product. After a few months of closed discussions, we have a set of specifications that are
   sufficiently evolved to allow informed debate. These specifications have been well received by
   OGSI authors and by many in the Web services community. I can imagine better processes, but
   I can also imagine far worse outcomes.

   WSRF was motivated at least partly by a desire to make Grid services more palatable to the
   Web services community. In doing so, the concept of a Grid service seems to have
   disappeared altogether. Is this a fair statement? So why did we need Grid services two years
   ago but not now?

   No, the concept of a Grid service has not disappeared. The essential requirement addressed by
   OGSI and WSRF the critical functionality missing from Web services from a Grid
   perspective is the ability to create, address, inspect, discover, and manage stateful resources.
   In OGSI, these stateful resources are called Grid services; in WSRF, they are called WS-
   Resources. A Grid service had an identity, service data, and lifetime management mechanisms;
   a WS-Resource has a name, resource properties, and lifetime management mechanisms. The
   terms have changed, but not the need, concepts, or mechanisms. Both OGSI and WSRF provide
   the mechanisms required for the really important specifications, those that define the
   overarching Open Grid Services Architecture. Work on OGSA continues, and is affected only
   slightly by these changes.

   When OGSI was first announced, the gross features had already been planned in much
   secrecy and the Globus project was a long way along the OGSI roadmap. The Grid
   community for the most part forgave you, and invested a lot of effort to understand OGSI's
   implications and revise project plans to take account of them. Will they do it again?

   Your opening statement isn't correct: OGSI wasn't a secret process. From 2000 on, we saw
   growing interest within the Grid community in Web services as an implementation technology
   for grids. In response, a small team from the Globus Alliance and IBM put forward a draft
   OGSI specification in February 2002. Only after intensive review within a GGF working group
   over one and a half years was the final OGSI specification produced in June 2003. In my view,
   this is an excellent example of both open standards development and proactive engagement by
   the Globus Alliance to address emerging community requirements.

   The WSRF work is motivated by the same goals and is following the same process. The Grid's
   success requires broadly adopted standardized infrastructure for creating, addressing,
   inspecting, discovering, and managing stateful resources. OGSI defined the required
   mechanisms. WSRF adapts the presentation somewhat to facilitate adoption within commercial
   Web services tooling. It's a bump in the road but not a major one, and the benefits are
IEEE Distributed Systems Online February 2004
   Many projects have been pursuing an OGSI convergence track for some time. Some have
   been developing Grid services; is this work likely to be wasted? Others have stuck with Web
   services, hoping that the transition to Grid services will be fairly painless once a widely
   deployed OGSI infrastructure is in place. It looks like they'll be wise to wait for WSRF, but
   for many, their projects' lifetimes are too short. Yet others stuck with GT2 and are now likely
   to bypass GT3 altogether. Some are revisiting alternatives to Globus. How do you convince
   the aggrieved that the long-term benefits of yet another upheaval will outweigh the pain? Are
   we due a massive backlash against the Grid? Will there be a Grid winter to rival the AI

   Interest in e-science and e-business continues to grow rapidly: for example, recently there have
   been several announcements by Oracle, IBM, Sun, and others and initiatives such as the US
   Cyberinfrastructure program and new EU programs. So does recognition of the critical role
   middleware plays as a means of enabling secure, reliable, interoperable resource sharing and
   management. Thus, I'm not concerned about a backlash: the Grid is happening, regardless of
   what pain those of us who are engaged in building it may be suffering in the process.

   I don't want to trivialize the "upheavals" that you talk about, but I also think that they're often
   overemphasized. When the Globus Alliance first delivered OGSI support in GT 3.0, we
   committed to supporting GT's pre-Web services components, and many users continue to use
   them. Indeed, we continue to recommend their use for production services, because the WS-
   based components have some way to go before they achieve equivalent quality. WSRF is an
   incremental change from OGSI, and we'll continue to support OGSI for some time to come.

   Distributed systems developers must look carefully at available technologies and decide which
   is best for their purposes. Neither the Globus Toolkit, OGSI, nor WSRF is helpful in all
   situations. However, I'd be careful about jumping too quickly to the conclusion that "my
   problem is different or trivial, so I'll develop my own custom middleware." Distributed
   computing is complex at many levels, from security to reliability to credential management
   policy, and interoperability is always challenging. There are many advantages to working
   within a standard framework that addresses pressing problems such as single sign-on, remote
   deployment of executables, computation management, and data movement. Users who adopt
   GT will likely receive help with the issues and also the benefits that accrue from participating in
   an international community of developers and users.

   What are the timescales you envisage for agreement on the proposed WSRF standards, and
   do you think the GGF is viable as a standards body?

   There's much discussion of the WSRF specifications on discussion lists set up by the Global
   Grid Forum's OGSI working group. The specifications are likely to be submitted to Oasis
   [Organization for the Advancement of Structured Information Standards] soon, with a formal
IEEE Distributed Systems Online February 2004
   liaison process to GGF to ensure that Grid requirements are addressed. The reason for this
   approach is simple: WSRF is not a set of exclusively Grid standards but a set of WS standards
   that happen to be highly relevant to Grid and WS standards are typically handled in Oasis or
   W3C. The timetable for completion after submission is unknown, but I'd hope that the
   considerable effort already devoted to OGSI within GGF will allow discussions to proceed

   Some have expressed the view that taking WSRF to Oasis means that GGF isn't a viable
   standards organization. That's not my opinion. GGF has already produced Grid standards that
   are being widely used for example, GridFTP and GSI. Other important specifications are in
   the works, such as DAIS. Naturally, Grids also build on standards produced in other
   organizations such as IETF, W3C, and Oasis . Some GGF participants also participate in these
   other organizations, with, for example, OGSI authors acting as invited experts on the W3C
   WSDL working group. The GGF CMM and Oasis WSDM WGs are joining forces. We can
   expect to see more such cross-fertilization. These developments should increase GGF's
   relevance as a forum not only for Grid standards but also for ensuring that Grid requirements
   are integrated into other standards organizations' work.

   Could the OGSI specification, which has several implementations apart from the Globus one,
   retain sufficient adherents to have an independent future once Globus has jumped ship and
   gone with WSRF?

   The Globus Alliance is committed to evolving the Globus Toolkit to WSRF for reasons I
   alluded to earlier. My understanding is that the developers of other OGSI implementations such
   as pyGlobus, OGSI.NET, OGSI::Lite, and Unicore are making the same move, as are the
   designers of major OGSA-related specifications such as WSDM (management), DAI (data
   access and integration), and WS-Agreement. Thus, while there's no reason why people shouldn't
   continue to work with OGSI (we'll continue to support GT 3.x's WS-based components for
   some time, depending on resources and user demand), they should plan to evolve to WSRF
   within the next year.

   Many scientists are still using library-style APIs and don't understand Web or Grid services.
   How long will there continue to be a GT2-style layer?

   I fully expect scientific problem-solving environments to become increasingly service oriented
   as communities learn to pool expertise not by sharing programs or data but by creating services.
   This concept is at the core of the international Virtual Observatory initiative, for example. At a
   smaller scale, the US Fusion Collaboratory project has had great success with providing remote
   access to simulation codes, thus avoiding the need for remote users to download and install
   complex applications. More generally, Web services provide a strong framework for building
   large-scale distributed systems by allowing independent developments to communicate through
   rigorous interfaces.
IEEE Distributed Systems Online February 2004
   That said, many communities and users still need specific functionality (for example, access to
   remote computers or data) that is naturally implemented in terms of GT2-like libraries for
   creating and managing remote computations, transferring data, and so forth. That's why client-
   side APIs for those functions are provided in GT3 and GT4 and why these APIs will continue to
   be supported as long as there's significant demand. Indeed, my expectation is that the set of such
   APIs supported within GT and by GT-based tools will grow, as more higher-level and
   application-specific Grid tools are developed.

   A shortcoming of the OGSI specification is that it didn't say enough about certain issues to
   enable two independent yet compliant implementations to interoperate. For example, the
   security model in GT3 was a nonstandard mix of GSI and WS-Security. WS-Addressing
   should help avoid a repeat of the handle resolution obstacle. Is the Globus Alliance taking
   steps to avoid repeating such obstacles to interoperability?

   The OGSI specification is appropriately focused on WSDL interfaces for Grid services.
   Interoperability involves many other issues, including security and reliable messaging. We're in
   close communication with developers of other OGSI/WSRF implementations and hope to start
   testing interoperability soon. That work should contribute to the development of an
   interoperability profile for OGSI/WSRF implementations.

   Ultimately, interoperability requires the standardization of a wide range of WS and Grid
   standards. The full value of adopting Web services as a basis for the Grid will be achieved only
   when such standards are in place and in a manner that meets Grid requirements. Thus, it's
   critical that we engage with the broader Web services community and with organizations such
   as the WS-Interoperability Forum on these standards, to ensure that we have full
   interoperability of WSRF-compliant services from providers of not only Grid but also Web
   services middleware.

   How do you envisage constructing an information service for a WSRF Grid? Are there any
   specific mechanisms provided (such as a monitoring and discovery service), or will we use
   existing registry technologies?

   A beautiful feature of OGSI/WSRF is its deep integration of information service mechanisms.
   Any Grid service (or WS-Resource in WSRF) can declare service data (resource properties) that
   can be discovered, pulled, or pushed using standard mechanisms. The WS-Notification
   specification introduced with WSRF moves us another important step toward robust, feature-
   rich information services, by establishing interfaces that allow the exploitation of widely used
   message-oriented middleware. Thus, for example, a programmer who develops a file transfer
   service need only define some appropriate service data (resource properties) for that service to
   become immediately discoverable and be monitored.

IEEE Distributed Systems Online February 2004
   Given this underpinning, it becomes possible to define a wide range of information service
   components for discovery, monitoring, archiving, fault detection, and so forth. The Globus
   Alliance is developing an initial set of such components, including an archive service and a
   registry service designed to support rapid turnover of state providers. First versions will be in
   GT 3.2 and more functional versions in GT 4.0.

   What would you suggest developers do until dependable implementation of WSRF is
   available? Also, when are we likely to see WSRF components in languages other than Java?

   With respect to the Globus Toolkit, the software situation is as follows. GT3 is a solid
   implementation of GT's pre-WS components and an early implementation of WS-based
   components. GT 3.2, which is about to be released, will provide significant improvements to
   GT's WS-based components in terms of usability, robustness, and performance while
   continuing to support GT's pre-WS components. GT 4.0, to appear in the third quarter of 2004,
   will introduce support for WSRF while also continuing to improve the usability, robustness, and
   performance of GT's WS-based components and continuing to support GT's pre-WS

   Other OGSI implementations are also planning to evolve to WSRF, but I don't know schedule
   details. Stephen Pickles recently announced on the OGSI WG mailing list that he has almost
   completed the conversion of his OGSI::Lite system to WSRF.

   Given this context, here is my advice.

   If you're working with pre-WS GT components and are happy with them, as is the case with
   many production Grids, stick with them. They're supported in GT3 and will be supported in
   GT4. We'll continue to support GT's pre-WS components as long as there's substantial demand
   and we have the resources to do so likely well into 2005.

   If you're working with WS-based GT components using GT 3.0 today, switch to GT 3.2 when it
   becomes available to obtain significant improvements. Then evolve to WSRF when it makes
   sense to you. We'll continue to support GT 3.x as long as there's substantial demand and we
   have the resources to do so likely well into 2005.

   If you're just getting going with Web services and the Grid, start working with GT 3.2 and plan
   to evolve to GT 4.x some time over the next year.

   Finally, coming from the UK, I know that the UK e-Science programme, which started in
   2002, is very tied into Globus technologies for its infrastructure. In hindsight, did the UK e-
   Science programme start two years too early?

IEEE Distributed Systems Online February 2004
   No. The UK e-Science Programme started at a perfect time to allow the UK to take a clear role
   in developing the Grid vision. For example, without UK e-Science funding, the OGSA-DAI
   work would have started much later or perhaps not at all, the UK wouldn't lead in data services,
   and the Globus Alliance would never have expanded to include the University of Edinburgh.

   I'd like to address one misconception that I think is relevant to this question. Work on e-science
   is about creating the environment needed for 21st-century science. Naturally (to use a
   construction analogy), we'd like simply to move into a finished building, and some users seem
   to expect Globus to be that building. However, as its name suggests, the Globus Toolkit is a set
   of tools, not a complete solution. In the hands of skilled developers, these tools can deliver
   impressive results when used appropriately to address authentication, discovery, remote access,
   and other distributed computing challenges. But success requires more than a scientist and the
   Globus Toolkit: it demands a partnership between scientist and Grid technologist, and/or the
   availability of higher-level tools that provide specific vertical solutions such as the
   GriPhyN/PPDG virtual data toolkit, Access Grid, and portal tools.

   So, overall, developments such as the Grid require invention and understanding. As a result of
   experiences gained, we're improving our understanding of what will be required to build robust,
   reliable, global grids. As an example of how such partnerships can work, consider the Network
   for Earthquake Engineering Simulation Grid, due to be completed this year. The Neesgrid team
   has built on GT3 OGSI software to create a sophisticated, distributed experiment control and
   collaboratory system. Some work will be needed to evolve this OGSI software to GT 4.0, but
   Neesgrid designers feel that this change is a minor cost relative to the tremendous progress
   that's being made in terms of integrating collaboratory tools, deploying at Neesgrid sites, and
   training the community. For example, last July Neesgrid was used to perform a major multisite
   structural engineering experiment that linked experimental facilities in Colorado and Illinois,
   simulation systems in Illinois, data archives, and human participants at dozens of sites across
   the US. This is new science that couldn't have been accomplished without Neesgrid.

   As a second example, the US Grid3 system has been using GT pre-WS components to run from
   500 to 2,000 data analysis tasks continuously since November 2003 on a Grid spanning 27 sites
   in the US and Korea for a range of physics, biology, and computer science projects. Again,
   substantial application results are being delivered, thanks in this case to a partnership between
   end users and the developers of the GriPhyN/PPDG Virtual Data Toolkit, which uses GT
   mechanisms to coordinate the scheduling of data-intensive workflows over many sites. In
   addition, much experience is being gained in terms of what it means to create and run an
   operational grid.

   I'd like to thank Ian for taking time out to answer the questions put before him. Ian will no
   doubt be discussing the issues brought up in this interview and others at the Global Grid
   Forum's next event, GGF10, in Berlin, Germany, on 9-13 March 2004 (see www.ggf.org for
IEEE Distributed Systems Online February 2004
     Mark Baker is a reader in distributed systems at the School of Computer Science,
   University of Portsmouth, UK, and a visiting professor of computer science at the Cavendish
   School of Computer Science at the University of Westminster. Contact him at

                      CMM: Common Management Model

                      DAIS: Data Acquisition for Industrial Systems

                      GGF: Global Grid Forum

                      GSI: Grid Security Infrastructure

                      IETF: Internet Engineering Task Force

                      MDS: Monitoring and Discovery Service

                      Neesgrid: Network for Earthquake Engineering Simulation Grid

                      Oasis: Organization for the Advancement of Structured Information Standards

IEEE Distributed Systems Online February 2004
                      OGSA: Open Grid Services Architecture

                      OGSI: Open Grid Service Infrastructure

                      W3C: World Wide Web Consortium

                      WG: Working group

                      WS: Web services

                      WSDL: Web Services Description Language

                      WSDM: Web Services Distributed Management

                      WSRF: Web Services Resource Framework

IEEE Distributed Systems Online February 2004

To top