White Paper
Scenario Testing Approach for Trading Systems
Last Updated: 12th May, 2009
White Paper
Introduction
The new Market regulations have brought major alteration to the structure of the equities markets throughout Europe. As a result, there are many new private exchanges evolving to compete with the established exchanges. Hence, to stay competent exchanges built technologically and functionally robust trading systems. Generally trading systems are functionally complex and it is equally challenging to test the functional aspect of the system. This white paper aims to address some of the benefits from the AppLabs scenario testing approach for trading systems.
Authoring and Execution of Scenarios
Authoring the scenario test case is quite a challenge, such that a thorough domain and functional understanding is mandatory; AppLabs Subject Matter Experts (SME’s) have successfully provided a specialized service to various Exchange clients across the spectrum. Our SME’s comprehensive understanding of the trading system, ability to make realistic assumptions; time and dedication help them to successfully author the scenarios. These scenarios are designed to handle various trading sessions like Opening, Normal Trading and Halt where it majorly involves large order book with complex pricing, calculations and trade details. Developing these scenarios requires a meticulous approach of setting up the precondition parameters, order book building and calculation of trades, etc. These scenarios are executed during various trading sessions such as Auction, Regular Trading and Halt where the execution of scenario test cases includes almost all the applications of the trading system. Where as the scenario testing approach saves time, ensures better coverage and it could be an integral part of the regression testing cycle. It also involves a lot of diligence to run these scenarios as there need to be more pragmatic steps such as pre-conditions set up of number of parameters, validation of the orders and trades accuracy. Execution of the scenarios will not require a complete system understanding but yet to analyze the trades, a SME support is required. A functional scenario test case can only be developed after the thorough understanding of Business requirements/Use cases. Typically a Trading Scenario is characterized by the presence of a series of order books, montage views and execution tables. The order book is moved from one market state/session to another and mimics the order book in a live environment. A standard trading scenario contains the following attributes:
Objective: A test objective encompassing complete
Trading Systems and the Complexity
Trading system comprises of set of rules where the Buy and Sell side orders for equities would be allowed to match and execute against the matching engine, thus it determines the state of an equity order towards a trade or cancellation. Trading systems are known for their intricate nature, so it is mandatory to have strong domain expertise individuals during the various stages of the project such as planning, development and testing. In the development stage, trading systems demand a solid understanding of technical analysis, the ability to arrive at empirical decisions and a thorough knowledge of how parameters and instruments work. During the testing phase, it is imperative to be familiar with the trading system domain and acquiring these skill sets can be an uphill task.
Trading Scenarios
Generally a functional – (unit level) test case is simple and deals with a single piece of function where it would only cover a minor portion of the whole functionality. So this is fine at a sanity testing standpoint. But as the trading systems are built with complex set of rules and so as to validate the system, AppLabs has arrived at a more innovative approach of building real time trading scenarios. These are called as complex scenarios where it is developed in the Table/Matrix form. The complex scenarios are developed in the Excel spreadsheets. As the scenario involves more complex calculations and numbers, Excel is handpicked as a more suitable tool. A typical exchange scenario depicts exactly the real time trading scenarios during the various sessions of a trading day.
functionality
Pre-conditions: Setting up the required market state
like Trading Session, Tick Size, Threshold Limits and so on
AppLabs.com
App_WhitePaper_Scenario_Testing_Approach_Trading_Systems_1v00 Page 2 © 2007 AppLabs
White Paper
User & Instrument Definition: The users and instruments
following screenshot refers to typical order entry in both
required to execute the test case
Order Books: Order Books that serve both as an input
and output for a sequence of steps
Montage: Montage of the market at various stages
validating the Market states
Execution and trade Tables: Containing the various
executions
Buy/Sell side. Each limit order type should be entered with the following details:
Quantity (e.g:1000 – Number of stocks) Price (e.g:100.50 – Price value of the stock) Time in Force (e.g: Day – Valid till day)
Firstly, a trading scenario should comprise of an objective to start with e.g. ‘To verify imbalance during the Auction session’. Once the objective is set the scenario should be designed accordingly. The above mentioned objective deals with the Auction session i.e. prior to regular trading session. Once arrived at the trading session, decision needs to be taken on the allowed order types during that session. In order to accommodate, the negative cases invalid order types during the opening session can be added. Tick size for the whole scenario should be taken in to account as it holds the key in pricing the stocks and it is going to be static throughout the scenario. For instance, when a stock has a tick size of 0.50 and the reference price stands at 100.00 then the stock price can move in the increments/decrements of 0.50. Thus the pre-conditions of the scenarios such as Tick-size, Session and Reference price should be defined. The most significant step in scenario development is building the order book with various order types and this is referred as order book building process. Generally trading system consists of various order types such as limit, market, iceberg, pegged, discretionary et al. So the various order types should be defined as input. The
Capacity (e.g: Agency/Principal – Order on behalf of the
client/own order) Thus each order type should be entered with some additional attributes along with the basic order properties as mentioned above. Throughout the scenario each input is assigned with a unique reference number to track the specific order in the order book and trades. Buy side orders are assigned as B01, B02… and sell side orders are numbered as S01, S02….This would act as a unique sequence number for every order in the book. Output - orderbook should be designed based upon the price and time priority rules of the trading system. Output should exactly meet the business requirements of the trading system. This should be in line with the order book in the real-time trading environment. For instance, an incoming Sell-Market order results in full trade/execution then the contra side – Buy order (B01) will move to the execution/trades column. Consecutively, the B01- Buy order will be removed from the output – orderbook, resulting in re-prioritizing the order book. Thus, the scenarios are designed to depict the live trading system.
AppLabs.com
App_WhitePaper_Scenario_Testing_Approach_Trading_Systems_1v00 Page © 2007 AppLabs
White Paper
Benefits
The real-time testing strategy has always been effective and it has resulted in the identification of bugs or issues. Complex scenarios are considered as a value addition act and these scripts are highly regarded by the customer. From the functionality standpoint these scenarios have majorly helped in validating the trading system, approved the system as bug-free and it effectively resulted in the successful launch. More than 70% of critical/high priority issues were discovered out of the scenario testing. Test Scenarios also ensure better coverage and also have significant time efficiency as they capture several spec points in a single scenario and are designed to test a major functionality completely.
Conclusion
By following this scenario testing approach, the organization will reduce the risk of non-compliance and also benefit from having a functionally robust trading platform. This white paper has highlighted the significance of scenario testing approach that organizations should address when implementing a new trading system. By understanding the detailed requirements of the trading system, organizations would better able to formulate a comprehensive functional testing approach to meet their business objectives.
AppLabs.com
App_WhitePaper_Scenario_Testing_Approach_Trading_Systems_1v00 Page © 2007 AppLabs