White Paper
The Testing Approach to Stock Exchange Applications
Last Updated: 25th May, 2009
Introduction
From the days of physical scrip to dematerialized stocks and from the days of simple equities to structured products, stock exchanges throughout the world have witnessed enormous change in their functioning. And ever since the introduction of online trading, the financial market has undergone a rapid change, leading to complexities and risks. Hence arises the need for a rigorous testing process, especially in terms of latency and performance, which this paper discusses about.
services and maintenance of liquidity. Another kind of risk involved is the exposure limit maintenance; this is being achieved by Online Position Monitoring at the Client, Dealer and Market level.
External Interfaces: As the exchange deals with a
Key Functionalities of Stock Exchange
The Stock Exchange has now transformed into a much advanced and sophisticated market place. A number of innovations have changed the whole trading scenario, but all the changes are still within the basic stock exchange structure. The key areas, as shown in the figure below are Core Services, Risk Management, External Interfacing and Recent Trends.
number of interfaces like Interface Management, Third Party Applications and Market Data Providers, the maintenance of proper interfacing is a crucial part of the trading services. There are a number of issues that the Stock Exchange has to deal with in this area as the volumes are large and the latency is high, the protocols and message formats that are dealt with are different in most of the cases.
Recent Trends: Along with the basic services, every
stock exchange needs to outperform using the financial innovations supported by the technological advancements which are; Algorithmic Trading, Dark Pools and DMA (Direct Market Access). The amount of analyzing capabilities that the recent machines and languages provide is utilized for the development of these algorithms. Strategic Investment algorithms are being developed to harness the technological advancements
Identification of Risks – Need for Testing
The technological advancements, product innovations, changes in the functioning of the application, time criticality in the development and the deployment of the application, have alleviated the complexity level, which in turn has put the application at higher risks. To mitigate these risks, the first step that needs to be followed is identifying the risks; the better the identification of risks, the lesser the chances of failure.
Fig 1: Functionalities of Stock Exchange
Risks that can occur to a trading application are:
Core Services: The Core Services comprise of the
primary and essential services of the exchange- the Primary market and Secondary market activities from listing of products to the clearing and settlement.
Risk Management: The other crucial service provided
Business Risks
The risks in this category focus on the basic activities covered under the Core Services of the Exchange - the Order Management System, Execution Management and Clearing & Settlement. Risks would occur because of improper functioning of different exchange products, the different order types involved and the way the orders and products behave during different trading sessions. Execution Management risk would involve order matching based on the price time priority, also, the allocation algorithms have to work fine. Another risk would be the correct functioning of the Trade Management functions like busting trades, trade correction, reinstatement etc.
by the exchange involves Regulatory Compliance Adherence. The Regional Regulatory Authorities provide rules and guidelines for the financial services industry which needs to be followed and adhered to, so as to provide a clean and transparent market place for the investors. It becomes a function of the exchange to follow these guidelines and adhere to the changes that are done for better governance. Surveillance also is a process which needs to be done for the clarity of
AppLabs.com
App_WhitePaper_Stock_Exchange_Applications_1v00 Page 2 © 2007 AppLabs
The trading application should adhere to the laid regulation compliance. This would be a compliance risk if the system does not comply with the set adherence guidelines like MIFID, RegNMS laid by the regional regulatory authorities. Due to the complexity of the functions, the business risks will also involve the integrated behaviour of the front, middle and back office. The market data at all times is a business risk as wrong information may lead to huge financial losses. Thereby the market data needs to be intact in order to avoid the business risk. In a nutshell, all the functional aspects of the trading application needs to be given a thought to achieve the maximum number of identified risks. The more the number of identified risks the better will be the capability to address the problems in the application.
Fig 2: Types of Testing
Technical Risks
Technical Risks would primarily include the technological integration of the business logic, integrated functioning of the different interfaces and gateways, latency, new technologies and security. The time criticality makes latency one of the major risks where there are systems available with minimal latency and every millisecond has a monetary impact. The regulatory framework, which lays down a roadmap keeping in mind all the players of the market for a gradual switching over to the new system, requires running of parallel systems concurrently along with the provided tools to interchange the data. The risk would be the integration of the two systems and then the fall back to one system in case the other fails to deliver at any given time. Introduction of new products in the market is another technical risk as they have to match up with the current trading scenario. The new technologies used for these product developments also have to be given equal importance as they should interface with the legacy systems. The interfaces involved also are at risk- their failure to deliver means loss. System security is yet another risk that needs to be addressed as most of the vulnerabilities lay behind the technological loop holes of the system.
Business Approach
The business approach of testing would mainly comprise of identifying the loose ends in the business model of the application that would commonly aim at the trade rules adherence, complex product functions, and business rules and logic. To proceed with, there could be two approaches - the top down approach and the bottom up approach. The top down approach starts from creation of business process maps, further drilling down to business scenarios, test conditions and test cases; the bottom up approach moves in the opposite direction wherein the test cases are developed, the test scenarios are mapped and finally the coverage is confirmed using business process mapping. When we combine the two approaches we finally get to the best approach which would have the Drivers and Stubs being used for creation of the test artifacts. To understand the approach a little better there could be two separate teams who develop the artifacts in the different top down and bottom up approach and then a set of skilled people map these artifacts to one another so as to minimize the gaps of both the approaches. This business approach will have minimum gaps and thus would provide a functionally healthy product.
Approach for Testing
To mitigate the risks, a framework of the approach is required, which should include the following testing types, Functional, Interface, Performance and Security. The right approach would be to drill down to areas and then focus on the areas with different approaches.
Fig : The Hybrid Approach
AppLabs.com
App_WhitePaper_Stock_Exchange_Applications_1v00 Page © 2007 AppLabs
Technical Approach
The technical approach should aim at the correct exchange of data, security provided to that data, the speed with which the data flows, the matching speed of the orders, storage capabilities and in case of any lapse, a backup provided for all. To the question if the backup provided is safe enough to rely on, and does it work at the same speed and security levels as the real system, the suggested approach will actually map each technological part to a different kind of testing.
log verifications which is a rigorous task and needs to be handled carefully.
Security Testing: Data Security
Security testing includes threat analysis and vulnerability analysis wherein one has to create threat models keeping in mind all the interfaces and leak points. In terms of exchange it would be all the internal and external interfaces to which the application is dealing with. The Vulnerability Analysis includes identification of critical components and configurations. After the plan is identified using the tools or doing a static analysis, then reviews have to be done. It also includes a source code review; in case the source code is not available, some other alternatives may be used. The network used for the clients and firms to access the application should be robust without any leaks and threats. In terms of volumes and amount, the impact of any threat as understood is huge, so testing around these lines especially networks, interfaces etc. would prove good and safe.
Interface Testing: Correctness of Data Exchange
The interface testing addresses the data correctness needs of the system and functioning of the interfaces. The complexity of integrating system lies in the dissension of the system protocols and message types. In a real time market the reporting and feeds have to function perfectly. For this reason, while testing, the test environment has to be almost similar to the real scenario which involves real time market simulators. The correctness to your approach would depend on the correctness of these simulators, so the simulator selection has to be perfect. The typical interface diagram would appear as in the figure depicted below comprising of various Incoming and Outgoing messages through different Gateways via a number of protocols:
Performance Testing: Storage and Latency Related Issues
In any trading application speed is the most important aspect which needs to be taken care of. As the response times are critical and the new generation systems work at lightening speeds, they must be accurate enough to match the standards. Performance testing would include testing of the main trading application, its sub systems and the different interfaces connecting these subsystems. To be able to perform a standard level of performance testing on a system, one needs to note all the major timings which effect the trading, like the peak trading times simulating such data and then creating different sets of scenarios which will address all the business rules as well. Order matching is only a part of trading, but there are a number of things that take place apart from that are order cancel, trade cancel, order modification etc. The standard performance testing scenarios should include these functions along with the trading peak timings.
Fig : Interfaces and Gateways of an Exchange
The internal interfaces of the exchange are easy to test as the message types are uniform across, but the actual problem lies with the external feeds where the protocols may be different and the message types need to be tested thoroughly. Most of the exchanges have adapted to the FIX messaging protocol, one of the widely used and understood protocols. There are a number of proprietary protocols used for security reasons as well, which need to be understood for testing purposes. The process generally involves technical knowledge of the back end and connectivity. Interface testing thus requires a robust testing which involves front end test runs followed by the back end
Failover/Recovery Testing: Backup and Recovery Issues
Failover Recovery is a part of performance testing wherein the focus is to check the backups provided apart from checking only the main system’s performance. This testing is important even in case of system migration and even when there is a requirement of running parallel systems. The technological and domain advancements at times ask for complete change of the trading application, in such cases, both the system need to work hand in hand and in case one fails the other should be in a condition to take up the full load. At all times there is a backup to all the
AppLabs.com
App_WhitePaper_Stock_Exchange_Applications_1v00 Page © 2007 AppLabs
elements in a trading application which is generally called the standby, and in case the primary fails, the standby should be in a position to function in the same manner as the primary. The issues that need to be addressed are the storage capacity of the system, and the time the system can take up the same load. In terms of the trading application a check needs to be done for all the Servers like Matching Engine, Trade Capturing, Login and Log out etc. The scenarios need to be drafted in such a way that they cover all the impossible mishaps that may occur to the system. The recovery timing is also very important because in general the backup systems are not fully equipped to take up too much load.
Conclusion
Stock exchange applications need to be functionally and performance wise perfect, and as the intricacies involved are more, the only way out is to figure out the best approach of testing based on the requirements. In addition, these systems are generally flexible in order to be able to change agreeably whenever required, so the approach needs to address these concerns as well. In order to have a reliable system in place it is always better to properly test it so as to avoid humiliation and embarrassment of a production failure resulting into financial, business and reputational losses.
AppLabs.com
App_WhitePaper_Stock_Exchange_Applications_1v00 Page 5 © 2007 AppLabs