Chapter 18 – Message Brokers
The Preferred EAI Engine
Message brokers bridge many different platforms and development
solutions than until now have been more isolated than open.
The services provided by message brokers can be put into several
distinct categories: message transformation, rules processing,
intelligent routing, message warehousing, flow control, repository
services, directory services, management, APIs, and adapters.
An advantage of brokers is that they “mirror the business”, allowing
for non-invasive integration, and automating functions currently
Brokers build on top of existing middleware technology, and could be
called the “middleware of middleware” (See Fig 18.1).
Fig. 18.1 Message brokers are able to integrate many different
types of middleware, as well as applications and databases.
Message Broker is based on asynchronous, store-and-forward
messaging. While its standards are not yet well defined,
major vendor’s implementations include at least these three
essential components (Fig 18.2):
• message transformation layer
• rules engine
• and intelligent routing mechanism
An application publishes a message to the broker, while
another application subscribes to the message. Applications
need NOT be session connected, which alleviates scalability
problems of other integration solutions.
Moreover, brokers also able to transform and convert the
payload, reformat the message, and route to any number of
targets (which determined by the rules engine).
All brokers must offer “any-to-any” and “many-to-many”
messaging capabilities – publish/subscribe is an effective
model to achieve both goals.
They provide value-added services from the uppermost layer in
the ISO model – the application layer.
Fig. 18.2 Message Brokers have three primary
components: the message transformation layer, the
rules engine, and the intelligent routing mechanism.
Broker could interface with source or target at the data level,
application interface level, method level, and the user
The strength of message brokers is that they are able to link
any number of different types of systems, adjusting to the
differences between all source and target systems by
exposing an API, and sometimes an adapter. (Fig 18.3).
Fig. 18.3 Message Brokers provide services to applications
through an application programming interface, or an
Message Transformation Layer
The Layer understands the format of all messages being passed
among the applications and changes them “on the fly” –
data from source is restructured to target(s) format (Fig
18.4). It is done through parsing and pattern-matching
methods that are constructed to describe any message
Information required for transformation is generally stored in a
repository that keeps track of the source system, the format
of the message, the target system, and the desired format
of the target system.
Format incompatibilities resolved through schema conversions;
most data conversion are automatic, except for conversions
between numeric and alphanumeric formats. All
conversions must take place dynamically.
In rear cases, conversion cannot be done (e.g. length/value of
the message is unknown); rules-processing capabilities of
brokers allow resolution of such situations.
Fig. 18.4 The Message Transformation Layer
Rules Processing is an application development environment
supported by the message broker to address the special
requirements of integrating applications. In effect, it is an
application between applications, one that does nothing
more than provide the logic for sharing information.
Routing of message to more than one target, performing
computations or look-ups on data as its being transformed,
etc. are examples of Rules Processing.
Rules are created programmatically, primarily with script
languages, and with virtually no bound to their flexibility.
Also known as “flow control” or “content-based routing” builds
on capabilities of both rules layer and the message
Broker can identify message source and route it to proper
target(s) transforming it if required (Fig 18.5).
Fig. 18.5 Intelligent Routing means identifying the message
and sending it on to the proper destination.
Message Warehouse is a database that is able to store
messages that flow through the message broker (Fig 18.6).
Provision of message persistence is facilitated to meet
• Message Mining
- Capability for extraction of business data to support
• Message Integrity
- Warehouse provides persistent state for message traffic
• Message archiving
- Capability to store messages for archiving/other purposes
- Enabling to diagnose the health of the EAI solution through
analysis of message traffic
Fig. 18.6 Message Warehousing
Repository is a database of information about source and target
applications, which may include data elements, inputs,
processes, outputs, and the inter-relationships among
applications. The goal is to keep track of sophisticated
information about source/target applications (e.g.
metadata, schemas, security and ownership perms).
In essence, repository is a master directory for the entire EAI
domain, and will ultimately become the enterprise metadata
repository, able to track all systems and databases
connected to the Message Broker.
Directory Services provide a single point of entry for
applications and middleware, utilizing naming service
standards. They are nothing more than a way to classify
resources on the network in a way consistent with any other
method of classification (e.g. DNS, X.500, etc.).
Adapters are layers between the message broker interface and
the source or target application – for example, a set of
“libraries” that map the differences between two distinct
interfaces – the message broker interface and the native
interface of the source or target application – and that hide
the complexities of that interface from the end user or even
from the EAI developer using the message broker.
For example, broker may have adapters for applications (SAP,
Baan, PeopleSoft), databases (Oracle, DB2, Sybase) and even
to specific brands of middleware. They are also getting
“smarter” with intelligence placed in adapters running on
source/target systems for better event capture.
There are two types of adapters in the world of message
brokers: thin adapters and thick adapters; and adapters
may behave in two different ways: dynamic and static.
They are simple API wrappers, or
binders, that map the interface of
the source or target system to a
common interface supported by
the message broker (Fig. 18.7).
Their advantage is simplicity of
implementation and greater
Disadvantage is impact of
performance without increase of
functionality, and considerable
programming effort is required –
thin adapters are nothing more
then another interface software.
Fig. 18.7 Thin Adapter
In contract, these provide a lot of
software and functionality between
the message broker infrastructure
and the source or target applications
(Fig. 18.8). Because the abstraction
layer and the manager manage the
difference between all the
applications requiring integration,
there is almost no need for
The user sees only a business-like
representation of the process and
the metadata information as
managed by the abstraction layer
and the adapter.
Although Thick Adapters are very Fig. 18.8 Thick
complex to develop, greater Adapters place an
complexity of EAI will ensure further abstraction layer on
sophistication, requiring no top of the
programming and providing as easy, application
business-like method to view interface.
Static and Dynamic Adapters
Static Adapters are the most common today, have to be
manually coded with what’s contained within the source
and the target systems (e.g. they must be configured by
hand to receive data form a source schema).
Dynamic Adapters are able to “learn” about the source/target
systems connected to them through a discovery process
that the adapters go through when first connected to the
application or data source. Equally important – they can “re-
learn” if changes occur.
Using the API
In addition to adapters, message brokers leverage APIs as a
mechanism to access the services of source/target systems.
It is similar to traditional message-oriented middleware,
however broker could integrate multiple applications, while
MOM is primarily for linking applications in one-to-one
Message Brokers use a hub-and-
spoke topology. As a hub,
Broker rests between the
source and target applications
in topology of a star (Fig.
18.9). While this topology
considered traditional, others
include bus and multi-hub.
In the bus configuration, the
broker sits on the network
bus and provides services to Fig. 18.9
other systems connected to Hub-and-Spoke
the bus (Fig. 18.10). Such Configuration
topology is effective when
brokers play smaller role
Fig. 18.10 Bus
Multi-Hub Configuration – a number of brokers are linked, with
source and target applications linked to any of the brokers
in the topology (Fig. 18.11). The advantage is ability to
scale, making it possible to integrate a virtually unlimited
number of source and target applications.
Fig. 18.11 Multi-Hub Configuration
As message brokers mature and become more sophisticated,
there is a clear movement away from the simple hub-and-
spoke configuration toward the multi-hub configuration.
Issues must be resolved in orchestrating load-balancing
and implementing fail-safe mechanisms within the multi-
Future of EAI and Brokers
Vendors need to standardize core features as include layers of
value – added software, with thin adapters giving way to
thick. There is drive to hide the complexity behind
abstraction layers and provide business “face”.
Given this reality, message brokers may turn out to be more
about business automation than middleware.