Quality Assurance in EAI projects

Document Sample
Quality Assurance in EAI projects Powered By Docstoc
					Quality Assurance in EAI projects

Author: Pradeep Vengayil

This document is intended to describe general understanding of basic testing, overview of
enterprise application integration describing various levels of integration, testing challenges in
enterprise application integration projects, typical enterprise application integration QA
framework and test automation. In the testing challenges section an attempt is made to evaluate
the existing process and the suggestion to overcome the challenges in EAI testing with a brief
explanation about testing methods. Even though the testing methodology may vary based on
specific project or application requirements, a commonly used enterprise application integration
testing framework is given in the section- Typical QA framework for EAI testing.
WHITE PAPER                  Quality Assurance in EAI projects

                                                Table of Contents

Introduction......................................................................................................................... 3
Software Testing ................................................................................................................. 3
Enterprise Application Integration (EAI) ........................................................................... 3
Testing challenges in EAI projects ..................................................................................... 5
Typical QA framework for EAI testing .............................................................................. 9
Test automation in EAI..................................................................................................... 11
Reference .......................................................................................................................... 12
Conclusion ........................................................................................................................ 12
About the Author .............................................................................................................. 13
About Wipro Technologies............................................................................................... 13

© Wipro Technologies                                                                                                 Page 2 of 13
WHITE PAPER           Quality Assurance in EAI projects

Testing is one of the critical stages of software development life cycle and is mandatory to
improve and evaluate the functionality and performance of complex software systems.
Traditionally, the testing of such systems proceeds in a series of successive phases, known as
unit, integration and system tests. This step-wise approach on one hand allows testers to master
the inherent complexity of the task, by subdividing the test object into smaller, more manageable
pieces, and, on the other hand, enforces a disciplined procedure.

Enterprise Application Integration is a widely accepted model for integrating various applications
in the enterprise. Even though participating applications are system tested well before integration,
most of the issues come out from the interfaces. To avoid major reworks or design changes in the
enterprise systems it is a good practice to have proper testing methodologies well in advance.

Software Testing
Basic software testing has two important aspects:
1. Ensure that the system meets all the customer requirements and
2. Ensure that system will not misbehave when an unexpected input is received.

Good testers think technically, creatively, critically, and practically:

Technical thinking: The ability to model technology and understand causes and effects. This
includes things like knowledge of relevant technical facts and the ability to use tools and predict
the behavior of the systems.

Creative thinking: The ability to generate ideas and see possibilities. You will test only in ways
that you can imagine testing. You will look only for problems that you imagine can exist.

Critical thinking: The ability to evaluate ideas and make inferences. This includes the ability to
detect and eliminate errors from your thinking, to relate product observations to quality criteria,
and to build a compelling case for a particular belief or suggested course of actions.

Practical thinking: The ability to generate ideas that are practical and put these ideas into

Enterprise Application Integration (EAI)
EAI is the process of coordinating the operation of various applications across an enterprise.
Typically, an enterprise has existing legacy applications and databases and wants to continue to
use them while adding or migrating to a new set of applications that makes use of the Internet, e-
commerce, extranet, CRM, Billing and other new technologies based on requirements. EAI may
involve developing a new total view of an enterprise's business and its applications, seeing how
existing applications fit into the new model, and then devising ways to efficiently reuse what
already exists while adding new applications and data. .Now a days EAI is commonly used while
defining new enterprise wide systems which makes use of multiple systems (for example CRM,
Portals, Billing, Data warehousing etc.), those are proven in the market.

© Wipro Technologies                                                                    Page 3 of 13
WHITE PAPER           Quality Assurance in EAI projects

Different levels of integration that EAI involves are Business Process Integration (BPI),
Application Integration and Data Integration:

Business Process Integration (BPI): A business process contains sequence of interactions. For
example a customer contacts a supplier to place an order. A sales representative takes the order
and passes it on to accounting and manufacturing and so on. Each interaction in a process
typically involves the exchange of information and may include human-to-system interactions.
When integrating business processes, a corporation must define, enable and manage the processes
for the exchange of enterprise information among diverse business systems. This allows
organizations to streamline operations, reduce costs and improve responsiveness to customer
demands. Elements here include process management, process modeling, and workflow, which
involve the combination of tasks, procedures, organizations, required input and output
information, and tools needed for each step in a business process.

Quality assurance or evaluation of the Business Process Integration is done in a broad level rather
than individual unit level. Test scenarios have to be developed by considering the end users and
business potentials. Most of the testing required for the integrated solution falls on the end-to-end
testing phase or end-to-end system integration testing.

Application Integration: At this level of integration, the goal is to bring data or a function from
one application together with that of another application that together provide near real-time
integration. Application Integration is used for, to name a few, B2B integration, implementing
customer relationship management (CRM) systems that are integrated with a company's backend
applications, web integration, and building Web sites that leverage multiple business systems.
Custom integration development may also be necessary, particularly when integrating a legacy
application with a newly implemented ERP application.

Most of the time communication between various systems fails due to the mismatch on the
expected format or value of data. System Integration testing (SIT) plays a vital role in terms of
identifying the interface issues. SIT is one of the most challenging tasks in terms of software
testing. This is the starting point of handshaking between various systems and involves lot of
coordination and follow up activities between various systems.

Data Integration: In order for both Application Integration and Business Process Integration to
succeed, the integration of data and database systems must be tackled. Prior to integration, data
must be identified (where it is located), cataloged, and a metadata model must be built (a master
guide for various data stores). Once these three steps are finished, data can then be
shared/distributed across database systems.

Usually verification of data integration is done implicitly along with the system integration and
end to end testing.

Standards of Integration: In order to achieve full Data Integration, standard formats for the data
must be selected. Standards of Integration are those that promote the sharing and distribution of
information and business data – standards that are at the core of Enterprise Application
Integration. These include COM+/DCOM, CORBA, EDI, JavaRMI, and XML.

Platform Integration: To complete the system integration, the underlying architecture, software
and hardware, the separate needs of the heterogeneous network must be integrated. Platform
Integration deals with the processes and tools that are required to allow these systems to

© Wipro Technologies                                                                    Page 4 of 13
WHITE PAPER           Quality Assurance in EAI projects

communicate, both optimally and securely, so data can be passed through different applications
without difficulty. For example, figuring out a way for an NT machine to pass information
reliably to a UNIX machine is a large task for integrating an entire corporate system.

Testing challenges in EAI projects
Even though the success of any project is a collective responsibility of all the members of the
team, many times the responsibility of the delivery is in the hands of the Quality Assurance (QA)
team. QA team should be having the commanding capacity to accept or delay the delivery based
on the quality of the deliverables; hence testing is a challenging task in terms of over all project

EAI QA is one of the most neglected parts even though EAI methodologies got main stream
market acceptance. Most often EAI testing is delayed until the end of the project due to several
reasons like less time, unavailability of standard testing strategy, insufficient tools and expertise
to accomplish the task. However, delaying QA activities might impact severely due to the
difficulties involved in testing a complex integration involving many end systems. Identifying
bugs in a later stage is much more expensive than identifying the same in an early stage.

It is a known fact that stability of an integrated system lays on the stability and reliability of the
middleware platform it is built up on. When the complexity increases, it is essential that all the
loopholes have to be closed. Following a traditional QA lifecycle like unit tests, integration tests,
systems tests, to acceptance tests will not work for EAI projects easily, since EAI QA involves
system-to-system integration. Most of the IT testing tool do not work well for testing EAI
projects since the EAI architecture is multi protocol message oriented and simulation of
applications are required rather than virtual users.

Few key EAI testing challenges are as follows:

•   Integration project architecture is message based and the failure points are generally in the
    messages, hence test design should focus on the messages and message failure, which require
    good understanding of system architecture and the interfaces. Testers should be technically
    capable to understand the system design and architecture.

•   Since most of the testing tools are focused on the functionality testing and performance
    testing of applications, developers need to create stubs or elaborate error-prone traces for
    testing purpose. Some products provides test stub and test harness exclusively for Integration
    testing using their products.

•   Integration projects involve many different applications. This poses both organizational and
    technology challenges. Organizationally, teams need to think not just about their code, but
    how it affects both input and output components. In most cases application development
    teams don't have the vision or the technology to validate their systems operation in the overall
    enterprise. The entire verification responsibility of EAIis in the hands of testing team and
    hence the team should be well equipped with the thorough understanding of handshaking

•   An integration project may also include many different technologies, such as XML, MQ,
    middleware brokers, TCP/IP and others. Integration test teams need tools that work with

© Wipro Technologies                                                                     Page 5 of 13
WHITE PAPER          Quality Assurance in EAI projects

    many technologies so that they can validate the technology handoffs, as well as the business
    logic. Test team should be well versed with the messaging standers and the validation process

•   It is almost impossible in a large integration project to orchestrate end-to-end tests with
    application test tools. The risk involved in this can be mitigated by performing tests early in
    development phase.

•   When the integration solutions streamline development and architecture of an existing
    solution the complexity, paths and permutations still remain and need to be tested. Testing
    has to be done for all of the complex business logic, which will be now under another layer of
    integration-ware after the streamlining process

•   Other major hurdles in the EAI testing could include components developed in different
    programming languages; Difficulties in Test Case generation and Test result comparison,
    Base lining, Generating Test Data, Identifying test bed, proper testing framework and testing
    tools are not readily available.

•   When it comes to the actual interface testing, availability of integrating applications for
    testing is always a big concern. This usually requires lot of co-ordination and follow ups

•   Transaction testing across multiple systems requires complete understanding of the business
    requirements in order to verify all the transactions across the interfaces

•   When the communications between various systems are asynchronous, it is difficult to ensure
    the guaranteed delivery of the message. Special attention has to be given for the verification
    of handshaking mechanisms to ensure the proper message delivery

•   Security related testing is very importance when the transaction that flows through the
    integration layer is critical. All the error scenarios have to be captured very well along with
    the possible positive scenarios to avoid most of the security related issues

•   A minor change in the message set could cause huge impact in related transactions. To avoid
    this EAI require more regression testing and impact analysis when there is a change in the
    message sets compared to other applications

•   Verification of failover mechanisms and recovery mechanisms are of more important in case
    of integration projects. It is a difficult task to manage and control multiple applications for
    this purpose

•   Higher dependency on availability of end applications to perform different levels of testing.

•   Test cases involving timers, events and re-producing such scenarios is complicated.

An effective EAI testing requires a good understanding of the entire systems that are involved
(going to involve in future) in the message path as well as the integration architecture.

Agile methodology with incremental component development, builds and deployments is a good
process for EAI development and testing. Enterprise integration should be broken down into
business integration domains to reduce the complexity and risk of the undertaking. This

© Wipro Technologies                                                                    Page 6 of 13
WHITE PAPER          Quality Assurance in EAI projects

incremental approach requires a matching incremental QA strategy, which places an additional
strain on the environmental requirements and dependencies on end system availability – the
testing environments are required for a long period of time. Also, the frequency of regression
tests increase to ensure that new builds do not break the ones further along in the QA process or
in production.

In a complex software development scenario, traditional scenarios like sequential or iterative
model face problems in accepting continuous changes. Agile methodologies are developed for
accepting the changes quickly and smoothly. It is a collection of values, principles and practices
that incorporate iterative approach for development, test and feedback for further changes. Here
software is developed in incremental, rapid cycles, with each release building on previous
functionality. Each release is thoroughly tested, which ensures that all issues are addressed in the
next iteration. When we follow this approach for EAI project each components of EAI will be
available for multiple round of testing in the early stages of the development cycle. Availability
of each component for testing in the early stages will ensure the strength and stability of the
application by eliminating the hidden issues in a later stage.

Performance testing is also very importance to ensure the system stability. The architecture of the
EAI solution needs to provide the infrastructure for performance, as well as the selected enabling
technology. A simple way of measuring the performance is, run some tests to see if the system
performs well under a variety of conditions and measure the following, How well does it perform
at 100 users, 500 users, 1000 users, and even 10,000 users? What about processing load? What
happens to performance as you extend the EAI solution over time?

A well defined process for quality assurance always help to minimize major risks involved in the
EAI testing area. Formulating a strategy based on the type of the project is also very important.
Following process in a commonly accepted model can be a good solution to overcome the major

•   Test requirement analysis
•   Good estimation
•   Identification of proper QA strategy well in advance
•   Establish a test plan.
•   Proper test cases design considering the various interfaces
•   Based on test cases; generate test data for each test case
•   Proper testing bed.

Test requirement analysis: This is one of the early stages of the project. Along with the
implementation requirements special stress has to be given for the testing considerations. This can
be a part of the business requirement analysis by identifying the business critical areas. In a broad
way, requirement analysis should cover functional requirements, non-functional requirements,
interface requirements, features to be tested, features not to be tested, assumptions, dependencies,
limitations, acceptance criteria and system hardware requirements. The study of the existing
systems (if any) and the existing business requirements has to be considered during this analysis.
Overall analysis should give more importance for the interface requirements since EAI will be the
core component of integration. An experienced test requirement analyst can contribute more on
the architectural design of the integrating system, since he/she must be having the good
understanding of the weaker sections of the system. A test analyst will be considering the whole
system as a component while a developer will be focused on to an individual component of the
system. It is always a good practice to bring test analyst together with the business analyst during

© Wipro Technologies                                                                    Page 7 of 13
WHITE PAPER           Quality Assurance in EAI projects

the initial analysis. Dedicated testers bring benefits like, focus on customer usage over technical
implementation and focus on uncovering flaws over confirming completeness.

Test Estimation: An estimate should accurately predict and guide the projects future. It should be
realistic by including all the tasks that can be reasonably anticipated. It forecast what, based on
the current knowledge, is most likely to happen and it should reveal the risk of the test area, so
that mitigation plan can be done properly. Estimate should be actionable, which gives clear
ownership of tasks by committed individual contributors and to see assigned resources and
known dependencies. One more important aspect of estimate is that, it should be flexible, what if
the deadline and resources constraints are immovable? The estimate must accommodate project
realities. Work breakdown structure is a good way of estimation. It is a hierarchical
decomposition of the project into stages activities and tasks. Stages for the test estimation are
Planning, Staffing (if applicable), Test environment acquisition and configuration, Test
development and Test execution (including find/fix/retest cycles). Identifying the actual duration
of each task require good background knowledge and experience. It is always better to involve
the team for a better estimation. Feed back and suggestion from each member has to be
considered for finalizing a good estimate.

QA Strategy: Defining a proper test strategy is one importance success factor in the project.
Strategy has to be defined by considering various factors including the type, schedule and the
budget. Test strategy should prove that the system requirements and architecture can be validated
by the defined tests. A proper test methodology has to be defined here. It is very essential to
prove that the team acquired a thorough understanding of the existing system or the new system
before formulating the strategy. Usually a brief explanation of the overall architecture has to be
presented in the strategy and each interfacing requirements has to be explained with the main
verification points. In terms of individual components and integration of each component to
develop the system, strategy should be defined to explain what type of test is most suitable? How
many cycles of test have to be carried out? What are the different phases of tests? And how the
overall progress of testing happens?

Test plan: A test plan is particularly important because of the difficulty in testing an EAI
solution. Most of the source and target systems are business critical, and therefore has to be
considered here. It should bring more visibility on the test design and execution based on the
defined strategy. Test plan should give consideration for the following points.

•   Describe and justify the test strategy (including proposed test coverage) in relation to
    Technical requirements and technical risk

•   Describe and justify any special requirements or entry criteria that must be met in order for
    the test project to proceed, as well as any exit or process for determining when to stop testing

•   Support the initiation and organization of the test project, including preparations, staffing,
    delegation of responsibilities, facility acquisition, task planning, and scheduling.

•   Support daily management and evaluation of the test project and test strategy

•   Identify and manage any risks or issues that may impact the project

•   Specify the deliverables of the test project, and the delivery process.

© Wipro Technologies                                                                    Page 8 of 13
WHITE PAPER           Quality Assurance in EAI projects

For designing the test case with respect to the individual component, knowledge of message
format and interface contracts are essential. Test case design will vary based on the type of test. A
unit level test case should have a complete mapping with the different message formats that the
component is going to handle. When the test phase progress from unit to system and then to
integration, the requirement coverage should be taken care with proper mapping.

Testing Steps can be described by the following flow diagram.

                                   Testing Steps

                               Test Case Identification

                                  Test Case Definition

                              Test Case Documentation

                              Test data setup / Test code

                                       Test Execution

                         Test Result               Test Coverage
                          Analysis                   Analysis

                        Defect fix and              Test Report
                         Re-testing                 Generation

                                   Regression Testing


Test cases identification has to be done based on the level of requirement that applies to the type
of test. Once the test cases are identified, defining and documentation can be done based on the
method of testing. For generating the test input, if the test harness or test stubs are not available
appropriate tools have to be developed for the test execution. Test result analysis and test
coverage analysis are done to make sure that the test coverage is adequate.

Regression testing is a very important part of all EAI testing, since a minor change in the flow or
the structure of the message may cause lot of impacts on the interfacing applications. Some basic
test cases which cover all the basic functionalities have to be identified for execution after each
code drop or modification in the code.

Typical QA framework for EAI testing
As already mentioned, when a proper quality assurance process is in place, testing related
activities can be integrated into the project lifecycle starting from the beginning of the project.
This will help to design the test cases based on the actual user requirement and not the design
specification that may be defective. Test cases should be part of the requirements that drive

© Wipro Technologies                                                                     Page 9 of 13
WHITE PAPER           Quality Assurance in EAI projects

system design that are understood by the users, analysts and developers, such as test cases derived
from use cases and event flows. When the test cases are driven by requirements, test execution
will be more complicated due to the distributed nature of EAI. This requires a dedicated test bed
or a QA integration framework throughout the test execution phase.

Some of the basic requirements that contribute to the QA integration frameworks are.
• Standardized interfaces and messages for testing
• Recording and re-playing tools
• Simulators or test harness to replace the actual applications
• Stubs or loop backs to acknowledge the data output and data input
• Validation tools, to validate the expected behavior based on the base lined data

A typical integration scenario and corresponding QA framework is explained below. Consider an
application takes orders over the web and distributes them to ERP, CRM and Billing systems.
The scenario for testing the web-based application in this integration would consist of the
following steps.

                Billing                       Web                     Validation
                (Simulate /                   Input

                          Common Message Bus

                CRM                         ERP
                (Simulate /                 (Simulate /
                Reply)                      Reply)

                                 Fig 2

•   Enter orders into the web application – could be driven based on HTTP messages or from a
    GUI-based test tool (record/replay). Web application can be replaced with a standard message
    generator or test harness in the early stages of testing. Messages can be defined based on the
•   The web application would publish messages to the middleware common message bus which
    will be available to the respective target applications like ERP, CRM and billing system
•   The QA system would simulate the ERP, CRM and Billing systems by replying to the web
    application’s messages based on a prior recorded session – replaying the messages in the
    proper sequence. Stubs or loop back can be used to simulate the target application response
    for each request.
•   The QA validating system would record the messages sent by the web application and
    received from the systems participating in the test (simulated or live). This system can be
    used for the automatic data comparison based on the base line data.

© Wipro Technologies                                                                Page 10 of 13
WHITE PAPER           Quality Assurance in EAI projects

•   Manual verification can be done if the data comparison is more complex or message format is
    constantly changing

Quality Assurance Integration (QAi): Complexity of an EAI project is greater than other
projects knowing the fact that EAI basically integrates different applications together, for the
smooth business flow. Importance of specialized testing on integration projects can be a good
solution for facing the challenges in EAI. This can be named as QAi (Quality Assurances

Good software testing is a challenging intellectual process. In early days first impacting area of
any software budget cut is the testing team, but the situation changed over a period due the
underlining fact QUALITY. It is impossible for any product to exist in the highly competitive
world with poor quality, so quality assurance is an integral part of anything.

An ordinary tester can assure the quality of a product by considering it as a black box. When he
understands the internal complexity and the fundamental architecture of the product, we can name
the tester as specialized. It is quit obvious that when multiple systems operates in different
environments with different languages, interacts using a common middleware, the knowledge of
the middleware architecture and the interacting sub-systems are essential, more over the
communication between multiple systems are based on messages.

Essentials for QAi: When a developer does a testing the attitude will be to pass the test case with
out damaging the built up product, so there are chances of hiding instability and loopholes in the
product, which is going to fail in production.

Dedicated testers bring two benefits:

•   Focus on customer usage over technical implementation
•   Focus on uncovering flaws over confirming completeness

It is fun to read the word construction by destruction, and this can be seen only in software.
Attitude of a real tester is always destructive construction, and this can be developed only when
the tester is independent. When the tester acquires knowledge of design and the overall
architecture, the result will be a purified product with stability and reliability in the production.

Following are the essential requirements for a member in QAi

•   Capability to understand the business scenarios technical architecture
•   Thorough knowledge on testing methodologies
•   Good knowledge on test automation tools
•   Testing attitude
•   Good knowledge on various middleware integration (EAI) tools
•   Basic Unix skills
•   Basic development experience

Test automation in EAI
Many testing tools are available in the market for test automation of GUI based applications;
most of these tools are following the capture/replay method for the functional verification.

© Wipro Technologies                                                                    Page 11 of 13
WHITE PAPER           Quality Assurance in EAI projects

Since EAI projects involves various applications talking to each other, a combination of EAI
testing framework with GUI based testing tool can be used.

In a typical EAI project as shown in Fig 2, when various systems are talking to each other
through a common message bus, an input application can be automated using a capture/replay
tool and external applications like CRM, ERP, Billing etc has to be simulated using loop backs
(program which returns valid reply to the interfacing application based on the input) with all the
interface trace logs enabled. Along with the test result, failures and errors in the trace log has to
be verified carefully.

Since major failures happen with the interface rather than the external errors, it is always
suggested to do manual functional verification and automation for the regression test cases. Errors
visible in the trace logs can be verified for finding the cause of the failures.

Standard testing tool for testing the EAI projects are rare in market. Test automation is a mix of
capture/reply tool with the EAI framework.

EAI testing can be automated by writing automation scripts for the following activities

•   Start related components
•   Inject Message
•   Open the output file
•   Compare the output file with expected output file

Make a batch file to automate all of the above mentioned steps sequentially. Comparison of the
output file with expected output file can be done in the following ways.

•   Write a test data verifier code using simple assert type methods or even can use Junit™
•   Generate a Verifying output flat file containing few passes/fails

It is ideal to avoid expensive tools for verifying EAI module in an EAI project

1. World Wide Web
2. Lessons Learned in Software Testing: by Kaner
3. General books on testing methodologies

As the Information Technology sector is navigating briskly to the Integration Technology line
with large number of integration tools getting released in the market, it is quite obvious that there
is a huge business potential hiding in the quality assurance area of enterprise application
integration sector. A standardized testing framework for integration project which should be able
to use with flexibility across platforms and applications is definitely going to help the industry in
many ways.

© Wipro Technologies                                                                    Page 12 of 13
WHITE PAPER          Quality Assurance in EAI projects

About the Author
The author, Pradeep Vengayil, posses a graduation with B.E. in Electronics and communication.
He has been working with Wipro Technologies ( for the last 5 years. Your
comments are welcome to

About Wipro Technologies
Wipro is the first PCMM Level 5 and SEI CMMi Level 5 certified IT Services Company
globally. Wipro provides comprehensive IT solutions and services (including systems integration,
IS outsourcing, package implementation, software application development and maintenance) and
Research & Development services (hardware and software design, development and
implementation) to corporations globally.

Wipro's unique value proposition is further delivered through our pioneering Offshore
Outsourcing Model and stringent Quality Processes of SEI and Six Sigma.

For further information visit us at:

For more whitepapers logon to:

© Wipro Technologies                                                              Page 13 of 13

Shared By:
Tags: White, Paper