XML: The EAI Killer
By Ronan Bradley, PolarLake CEO
Application integration approaches using Enterprise Application Integration (EAI)
products have not produced a solution that can be easily maintained and
extended to handle the changing business requirements. This article discusses
how XML and Web Services, coupled with the Service Oriented and Event Driven
architectural approaches, provides an opportunity to build fundamentally more
Application integration is at the heart of most enterprise IT projects. This is not
new: a whole generation of Enterprise Application Integration (EAI) products have
attempted to solve the generic problem. These products have been successfully
implemented and deployed within many enterprises and across many industries.
However, in doing so, the users of these products discovered that they were
difficult and costly to implement.
Users of EAI products also found that making changes were potentially even more
expensive. A measure of the scale of this problem comes from Gartner, Inc.,
which estimates this maintenance cost to be 60 to 65 per cent of the money spent
on the original application integration itself. Clearly, software doesn’t rust and bits
don’t fall off over time: maintenance in this context is adaptation and ongoing
Let’s leave aside the whole ease of use issue around EAI products and focus on
the maintenance issue. In my view, the heart of the problem with solutions based
on the EAI products (and indeed with the common alternative of writing your own
integration framework) is lack of flexibility: flexibility to change as the IT and
business environment changes, and flexibility to be extended easily with low
impact on the existing solution.
Decoupled architectures: A route to flexibility
Decoupling is generally regarded as the key to flexibility. Decoupling allows
change in one system to be achieved without breaking other systems that interact
with it. This control of the ripple effect of change directly addresses the
maintenance issue identified by Gartner and increases the level of reuse, as
existing applications are reused in new integrations.
There are two types of decoupled architecture:
· Service Oriented Architectures (SOA): Very much flavor of the month
and closely associated with Web Services, the SOA approach consists of
defining service definitions corresponding to the capabilities each
application wishes to allow other systems to access. This architectural
approach has been around for a number of years, but has gained new
impetus with the Web Services momentum. The trick is to define a service
that will be used by both existing consumers and future consumers.
· Event Driven Architectures (EDA): The event driven architecture
involves defining a dictionary of messages which applications can either
send or receive, typically corresponding to business transactions and
running over messaging systems such as IBM’s MQ-series™ or Tibco’s™
products. To integrate a new application, you decide which of the existing
messages you will send and receive and create the necessary processing
logic. The trick with this approach is to messages that will be reused in
this manner and to carefully control the number of messages in use.
There is actually a lot of common ground between the two approaches, although
they have typically been implemented on top of different middleware products.
XML and Decoupled Architectures
XML as a universal data format is a perfect vehicle for defining services for SOA
and for defining messages for EDA. This fact has been recognized by end-users
who are using XML-based Web Services and XML-based messages with existing
messaging systems. Unfortunately most vendors are still either in the SOA camp
or the EDA camp, with weak or no support for the other approach. (It is
interesting to see the emergence of the Enterprise Service Bus architecture,
defined in December 2002 and heavily promoted by Gartner since then. This
architecture attempts to span both SOA and EDA, matching what PolarLake has
supported in its products for some time.)
When defining the XML documents, I believe that it is important to ensure that
the documents are technology ‘vanilla’ and describe only the business payload. At
every stage, the business and technical information should be carefully partitioned
in the document. If technical or implementation details are entwined in the
definition, it undermines the decoupling. Sticking to this approach allows the XML
documents to act as a new layer corresponding to the actual business
transactions, which can sit on top of the existing software stacks. This leads to a
good rule of thumb:
The XML documents encapsulating your business logic should be
architecture neutral (i.e., capable of delivery within a SOA or EDA and over
any software stack)
There is a natural consequence to maintaining this separation of business content
and implementation. Because the documents are abstractions of the underlying
capabilities, there is now a gap between the documents and the underlying
interfaces. This gap is an inevitable side effect of the abstraction and enforces the
loose coupling of the applications. It can be bridged by changing the application
to process new types of information (which is typically unacceptable) or by adding
a new layer to bridge the gap, mediating between document and application.
XML and the flexible enterprise
Applications, once deployed, usually don’t change over their lifetime: some
features may be added, but the fundamental capabilities are typically quite static.
How the application interacts with other applications does change, often
significantly and rapidly. The way a mainframe application interacts with other
systems today is very different to the way imagined ten or twenty years ago
when it was designed. In the past, this has been driven by architectural changes
such as the move to client-server, three-tier and distributed. We don’t know
whether such significant changes in architecture will continue to happen.
However even if the architectures don’t change, business requirements will
continue to change, and business changes will require greater technology
changes, as the enterprise becomes more and more integrated. Within an XML-
based architecture, these changes will impact on the types of XML documents
used and the way they flow around the network and applications. This leads to
another rule of thumb:
Assume that the set of documents you define will change and extend over
time and that the way your applications process the documents will also
This means that the way we handle XML, specifically how we integrate with the
applications and how we define the flow of documents between applications, is
also liable to change.
Implementing XML-based architectures
Accepting the two rules of thumb has a radical impact on the way we need to
build integrations. The need to define documents independent of implementation
introduces a gap between the data representation in XML and the application
view. The need for flexibility means that this gap must be bridged in a way that
can be easily modified and extended.
In a previous article in this series on XML and application servers
[http://www.polarlake.com/resources/opinions/01.shtml], I talked about the four
dimensions to this problem:
· Validation against business definitions
· Enrichment with additional information
· Transformation into the correct format before being mapped into the
· Exception handling must be managed intelligently
In the absence of a product such as PolarLake, many developers solve this
problem by creating their own code-based framework. Unfortunately, this is an
expensive and time-consuming task with most of the effort focused on generic
middleware-type code, not business logic.
The real trick is to solve the problem in a way that can be easily modified and
extended to handle new types of documents (corresponding to new service
definitions or messages) and new processing paths. I believe that this can only
be done with a flow-centric approach, which separates the processing functionality
(often requiring coding) from the flow control.
This is the approach taken in PolarLake’s products with our XML Circuits™ which
allow the flow of the XML to be laid out using graphical software, with the flow
decisions based on Xpath-based selection rules. It then becomes an easy task to
add or modify the flows of XML to the business logic.
What does it all mean?
As I have said before, with any new technology standard the key is to recognize
how much of the problem it solves and figure out how to get the rest done. The
concepts behind decoupled architectures are reasonably well understood and
tested. XML provides a great basis for defining the services and messages that
flow around the enterprise. The challenge is implementing these architectures.
This requires an approach that is XML-centric and fundamentally flexible, such as
that taken by PolarLake.