Web Services Testing - A Primer by ranjitm


									White Paper

Web Services Testing
A Primer
Last Updated: 7th May, 2007

DCOM and CORBA traditionally achieved what web services are now offering but with an exception to interoperability, which they later provide in a true sense. In addition, COM+, DCOM (Distributed Component Object Model) implementations from Microsoft were resourceintensive and were native to a specific Microsoft OS flavor. In other words, the consumer and provider were mandated to be on the same version of the OS. CORBA (Common Object Request Broker Architecture) was Object Management Group’s specification for interoperability between nodes with an objective to facilitate object level communication between different nodes in a distributed environment. However, ORBs (Object Request Brokers) present between the client and server came from different vendors and there was always a competition among the vendors providing the ORB. In addition, CORBA was tightly coupled against the loosely coupled web services and furthermore, a CORBA implementation required the client and server to share the same interface (IDL- Interface Definition Language) and run the same ORB at both ends. In a CORBA implementation, if different vendor ORBs interop???, higher level services such as security and transaction management cannot be extended. All these factors made the Interop factor quite elusive. On the contrary, web services are strictly loosely coupled, firewall friendly, stateless, URL based as well as language/platform/vendor independent. EDI is also a traditional approach which was used in the B2B ecommerce industry where the messages (data) are exchanged via EDI specific-formats, such as UN/EDIFACT and ANSI X12 instead of XML and the network was private for the most part. In addition, EDI was designed for large corporations that can conduct business via private networks and are not agile, in that internet is not an acceptable mode. Small businesses can take advantage of the affordable internet connections and capitalize on the benefits of web services thereby facilitating a transparent capitalist economy. The focus of this article is to highlight the importance of this emerging technology and how it is being adapted across companies. However, the crux is on how to test systems that are exposed to web services or consume them. The reader shall understand the basics of web services, the underlying pitfalls and the testing strategies involved.

What is a Service?
A Logical Abstract View
According to the W3C a web service is a software system designed to support interoperable machine-to-machine interaction over a network. It has an interface that is described in a machine-processable format such as WSDL. Other systems interact with the web service in a manner prescribed by its interface using messages, which may be enclosed in a SOAP envelope, or follow a REST approach. These messages are typically conveyed using HTTP, and normally comprise XML in conjunction with other web-related standards. Software applications written in various programming languages and running on various platforms can use web services to exchange data over computer networks, like the internet, in a manner similar to inter-process communication on a single computer. This interoperability (for example, between Java and Python, or Microsoft Windows and Linux applications) is due to the use of open standards. OASIS and the W3C are the primary committees responsible for the architecture and standardization of web services. To improve interoperability between web service implementations, the WS-I organization has been developing a series of profiles to further define the standards involved. For a list of publicly available services, visit www.xmethods.com

Testing Methodology
At a very basic level, one needs to consider how the remote method calls take place via SOAP, how the data is passed, how to test the transport layer, discovery methods, response data, scalability, performance, integrity etc. Testing Approach: Test Case Design, Test Execution shall be Top-Down and Bottom-Up respectively. Documenting test cases for the individual services can be a tedious task and subject matter experts can take part in unit testing the individual service so that they are error-free in the down stream process. This approach facilitates meeting the high level business requirements and can also be used to create the data contents for the exchanged message. This will avoid the major pitfall of delivering a technically acceptable solution which fails to deliver business value. Once this is done, the data must be formatted into messages (XML) that express service request and replies from consumer and provider respectively.

App_WhitePaper_Web_Services_Testing_1v04 Page 2 © 2007 AppLabs

Testing categories:
Test the Transportation Layer
Due to the problems introduced by latency and unreliability of the underlying transport, testing the Transport Layer (HTTP/HTTPS/MQ/JMS) – protocols which form the backbone in shipping the messages around. This testing can be done in isolation initially wherein the transport is tested with messages sent over the transport as data files instead of the actual data and consequently integrate with either side (consumer/provider) once the application is complete. As seen in the figure below, there are test messages sent between the Claims and Adjudication Services.

been completed using the entire web services involved in a unit of work. Example: If a purchaser has to consume 3 different web services involving insurance, banking and government regulations, it has to be all or none activity analogous to a ‘two phase commit’. The test cases focus on verifying that incomplete transactions are rolled back for the user/ application using the services in question. This may also be applicable for SOAP servers implementing Time to Live logic. XLANG verifies the two phase commit for distributed transactions wherein if a transaction is incomplete, express it explicitly. Additional benefits are not limited to:
Ñ Sequential and parallel control flow constructs; Ñ Long running transactions with compensation; Ñ Custom correlation of messages; Ñ Flexible handling of internal and external exceptions.

Functional Integration
Verifying the flow of the application, where the remote method calls to the SOAP server(s), is followed in a specific order as per business rules which additionally involve verifying the nested functionality from the WSDL. This may include testing for interoperability as sanity tests which validate the agreed upon semantics between the services. Example: Claims processing for an Insurance Company has to follow a series of actions. A sample flow would be:
Ñ Process Claims - generate a claims number; Ñ Adjudicate - examine the facts and determine the loss; Ñ General Ledger - update the ledger database.

Test in Isolation
Isolation can be achieved by storing the message in a data file. For instance, a message received by a consumer can be saved to a data file or a file. The emphasis should be more on testing the web service as an independent entity regardless of its dependency on other web services. The messages must be based on a specific standard and some of the industries have already started these activities, namely Association for Cooperative Operations Research and Development (ACORD), regulatory standards such as Health Insurance Portability and Accountability Act (HIPAA) for healthcare related transactions. Interoperability can be verified using tools available such as WSS (Web Services Studio) for a MS.NET implementation. The new Web Services Studio can be used to automatically generate and compile SOAP proxy classes and C# clients.

Verify Complex Undo Activity
Testing the fragility of distributed systems if incompatible updates are introduced to any participant may involve validating a two phase commit if a transaction has not

A single WSDL document (Web Services Description Language) may group together the definitions of several web services. This would contain multiple portType (class) element definitions containing multiple methods which need to be executed in a specific order. In the above example, if the WSDL document is grouping Process Claims, Adjudicate and General Ledger together, it becomes critical for the methods contained in them to be executed in a specific order without compromising business rules. A more pragmatic approach would be analogous to EDI benchmarking (Tracking) which is important when multiple entities are involved who work in a dependable manner. Typically global unambiguous URIs (Uniform Resource

App_WhitePaper_Web_Services_Testing_1v04 Page 3 © 2007 AppLabs

Identifiers) a.k.a URL’s which are analogous to primary key pointers for transaction completion can be verified for tracking purposes. They tell us that ‘This server was accessed and this command was issued ...’ For EDI transactions, there is a centrally available vendor who is responsible to investigate the success/failure of transactions. For web services, this portion needs to be replaced by using URI’s which is a fundamental feature available with the Web Architecture and can be leveraged by web services because of the fact that the later is built upon the former. An Administrative module can implement this wherein the service provider can track the transactions and if required share the information based on business needs. Each time a new transaction is initiated, a new URI is generated to unambiguously identify that transaction, much like a Primary Key in a database. However, while a database key may only be unambiguous within a particular database; a URI is globally unambiguous, which means that it can be conveniently transmitted to others without loss or confusion of meaning.

attack and need certain special measures in designing the services themselves. This may include using Message Level Security instead of Transport Level Security where the information related to security is encapsulated in the message, which makes it more flexible for the intermediate processes.
Ñ Even more risky for services using HTTP as opposed

to HTTPS In cases like these, verify that the authentication data is encrypted or hashed. This test case can verify appropriate messaging as per the two phase commit logic identifying a failed hacking attempt on the XML packets contained in the SOAP envelope. This may be applicable to both synchronous as well as asynchronous hashing algorithms. In low volume transactions, this might still be a low priority test case when the decrypt key is sent via a non-electronic fashion. An additional test case would be to consider environments where transport security is provided via HTTPS and the web services themselves do not really provide authentication models.. Upon completion of the transaction, if the session is not closed then a potential hacker could use the session ID to steal the information that becomes critical for the business. At a very basic level when consumers are requesting a web service, it is important to identify (with evidence) that the consumer is indeed the right one. ???? The right consumer? XKMS by Microsoft and Verisign facilitate security of transactions by way of digital certificates. These allocate the verification to a service so that thin or mobile clients do not have to carry out the smarts. An interesting fact about Microsoft’s Passport service was that it did not receive an overwhelming usage/response. At time of writing only 83 sites are using this service out of which 23 are Microsoft sites. A trade-off can be made to this test case where the encrypted data, if it does not have a significant financial implication, can be categorized as a low priority test case. Essentially, the security for web services is at a nascent stage which was same for the w3 when it started.

Load and Performance
These test cases involve a lot of guess work while allocating hardware resources. When the actual business processes change, these become a difficult area requiring the guess work to be redone and different loads to be configured on different SOAP servers. If all the web services reside on a single server this would be fairly simple. Usually this is not the case and the load is scattered on different servers and the balancing needs to be adjusted based on the effected SOAP server after understanding the peak loads of the different services residing on the different SOAP servers. This can be further broken down into Performance Testing covering scalability, throughput (time to connect, time to first byte and time to last byte) on individual services. Occasionally, a good test case would be to simulate the lack of shared memory between the ‘caller’ and ‘object’. Traditional load/performance testing tools can be used to test the individual services in isolation after benchmarking typical loads thus verifying concurrent access.

Once of the goals for web services is to be transport independent. That is to say the services can be provided/ consumed via any transport layer such as HTTP, HTTPS etc. In general, when HTTPS is used, the XML messages are secured to an extent. When intermediate processes need to access the messages, they are vulnerable for an

Miscellaneous Test Cases Not Limited to:
Ñ What happens if two logical WSDL documents define

the same service differently?

App_WhitePaper_Web_Services_Testing_1v04 Page 4 © 2007 AppLabs

Ñ Compatibility tests involving the usage of different

communication protocols (HTTP/HTTPS/JMS/MQ)
Ñ Compatibility of the SOAP message with frameworks.

In future, companies shall concentrate more on their core business and use services provided by experts in a specific area without re-inventing the wheel making Service Oriented Architecture (SOA) a reality. This will call for more stringent testing methodologies considering the scope involved in testing these services in an integrated manner. The Darwinian web services platform has certain standards still being driven by companies like Microsoft, Sun, BEA, Oracle, HP and IBM which is now governed by the XML protocol activity group under W3C. There is a significant amount of confusion involved while it evolves as an emerging and promising technology. Aptly enough, industry veterans observed that ‘If something is confusing, you are paying attention!’

For example, various versions of .NET frameworks involving garbage collection processing etc.
Ñ Verify that the infrastructure and business logic are

separated in the message which is more design verification
Ñ Write a client in a different language other than that

used in creating the web service and make calls to the web service

Several Challenges Ahead
Ñ To discover services that have web services description

language free of XML
Ñ Makes unnecessary for the service provider to advertise

and have an XML overhead which is specifically useful for e-shops tailored for parents
Ñ Web services are a culture themselves and companies

need to adapt. An example would be to have the willingness to wrap a legacy system as a web service for external as well as internal use

App_WhitePaper_Web_Services_Testing_1v04 Page  © 2007 AppLabs

To top