Docstoc

model-driven-sw-testing.ppt - IBM

Document Sample
model-driven-sw-testing.ppt - IBM Powered By Docstoc
					                Automated Model Driven
                   Software Testing
                              Rational Users Conference
                                       Tel Aviv
                                      June 7 2004
                                     Alan Hartman
                        IBM Israel – Haifa Research Laboratory



Rational Israel Users
Conference
                           Agenda
       •    AGEDIS Project Overview
       •    Architecture
       •    Methodology
       •    Interfaces
       •    Tools
       •    Future Plans



Rational Israel Users
Conference
                        AGEDIS Overview
       • Automated model-based test Generation and
         Execution for DIStributed systems
       • Methodology and tools for model-based testing
       • Open interfaces
       • Mixture of academic and industrial partners
       • Three phase timetable of experiment and
         development
       • November 2001- February 2004
       • Partially funded by the European Commission
Rational Israel Users
Conference
                        Consortium Partners
       •    IBM Haifa Research Lab
       •    Oxford University
       •    VERIMAG/IRISA
       •    Imbus
       •    France Telecom
       •    IBM UK
       •    Intrasoft International


Rational Israel Users
Conference
              Methodology & Architecture
       • UML based
         component testing
       • Compliant with RUP
       • High degree of
         automation
       • No silver bullet



Rational Israel Users
Conference
        Automated Testing Methodology




Rational Israel Users
Conference
         Automated Testing Architecture
                                                                               Simulator
            GUI & Productivity Aids


                                        Model                   Intermediate
                                                   Compiler
                                                                   Format      Generator

                                      Generation
                                      Directives                                 Abstract
                                                   Analyzers                    Test Suite

                                                   Visualizer      Suite
                                                    Editor       Execution
                                                                   Trace       Execution
                                      Execution
                                      Directives
Rational Israel Users
Conference
                                   Benefits
 • Starting from specification
       – Involves testers early in the development process
       – Teams testers with developers
       – Forces testability into product design
 • Building behavioural model and test interface
       – Finds design and specification bugs - before code exists
       – The model is the test plan - and is easily maintained
 • Automated test suite generation
       – Coverage is guaranteed - increases testing thoroughness
       – Matches coverage goals to testing budget
       – Zero test suite maintenance costs
 • Automated test suite execution
       – Finds code and interface bugs
       – Includes a framework for the testing of distributed applications
       – Reduces test execution costs
Rational Israel Users
Conference
                           Interfaces
       •    UML Profile for AGEDIS
       •    Test Generation Directives
                                           UML   IF
       •    Test Execution Directives
       •    IF Model Execution Interface   TGD
                                                       ATS
       •    Abstract Test Suite
                                                 SET
       •    Suite Execution Trace          TED




Rational Israel Users
Conference
                        User Modeling Interface
       • The AGEDIS Modeling Language is a
         profile for UML 1.4:
             – UML Class diagrams - structure
             – UML Object diagrams - snapshots
             – UML State diagrams – behaviour & test
               purposes
       • Annotated with an action language – IF

Rational Israel Users
Conference
                 Test Generation Directives
       • Test Purposes – stereotyped system level
         state diagrams
       • Coverage Directives – stereotyped system
         level state diagrams
       • Coverage without tears – 4 levels –
         parameters to the test generation algorithm


Rational Israel Users
Conference
                   Test Execution Directives
       • XML Schema
             – Model translation from model abstractions to
               SUT concrete implementations
             – Test architecture
                   • Host addresses
                   • Delays, timeouts
                   • Prologues, epilogues



Rational Israel Users
Conference
           IF - Model Execution Interface
       • IF Encoding of the model
             – Classes
             – Objects
             – State machines




Rational Israel Users
Conference
             Test Suite and Trace Interface
       • XML schema – for test execution and
         tracing
       • Model description
             – classes : constants, types, control & observable
               signatures
             – a special class is defined for the tester
             – object identities
       • Test Suite - set of test cases
       • Test Trace – record of executed test cases
Rational Israel Users
Conference
                    Test Case and Test Trace
       • Interaction graphs - between the tester
         and the SUT
       • Associated verdicts (Pass, Fail,
         Inconclusive)
       • Alternate paths for non-determinism in
         test cases
       • Expected results in test suite
       • Observed results and verdicts in trace
Rational Israel Users
Conference
                        Abstract Test Suite




Rational Israel Users
Conference
                                  Tools
        •    User Interface
        •    Modeler & Model Compiler
        •    Model Simulator                                 S
                                                        C    G
        •    Test Generator




                                              UI & RG
        •    Test Execution Engine
                                                        FA
        •    Test Suite/Trace Viewer/Editor
        •    Feedback & Analysis                        VE
                                                             EE
        •    Bug Reporter
        •    Report Generator


Rational Israel Users
Conference
                        GUI




Rational Israel Users
Conference
                Modeling Tool & Compiler
       • Objecteering UML
         modeling tool
       • Tool profile to convert
         to XML
       • General purpose XML
         to IF compiler
             – Written in Java, with
               XMI in mind as a
               future input format

Rational Israel Users
Conference
                        Model Simulator




Rational Israel Users
Conference
                             Test Generator
       • Based on GOTCHA and TGV
       • GOTCHA
             – uses Murphi specification language
             – explicit traversal of state space
             – extensive coverage criteria
       • TGV
             – language independent simulator
             – focus on distributed applications
             – explicit test purposes as sequences of
               interactions

Rational Israel Users
Conference
                        Test Execution Engine
         – Input: Abstract Test Suite and Test
           Execution Directives
         – Output: Suite Execution Trace
         – Multiple platform (e.g. Linux,
           Windows) support
         – Multiple external interface
           (Java,C++,C) support
         – Test distribution + centralized
           logging
         – Interactive test case execution
         – GUI to view execution progress,
           inputs, and outputs
Rational Israel Users
Conference
           Execution Engine Architecture
                        I/O

             1                  1 on each host   1..N on each host       Object

                                                                     .
                 Test Suite        Host            Process           .
                  Driver          Manager          Manager           .
                                                                         Object




                              Network

Rational Israel Users
Conference
                        Feedback Tools
  • Coverage analysis
        – Detect uncovered areas of the model in
          either test suite or test trace
        – Create test purposes to reach them
        – Invokes a functional coverage tool from
          www.alphaworks.com
  • Defect analysis
        – Clustering of defects
        – Feature extraction from clusters
        – Create test purposes to reproduce the bug
Rational Israel Users
Conference
                        AGEDIS Future Plans




Rational Israel Users
Conference
                          Future Plans
       • Exploitation Activity
             – Available under license for non-commercial use
             – Available as a service
       • Incorporation in wider Model Driven SE
         Effort
             – Integrate methodology with RUP
             – Migrate to Eclipse, XDE
             – Focus on business integration domain

Rational Israel Users
Conference
                              ModelWare
       • New European Project
       • Model driven software engineering
       • Eclipse based tools for UML modeling, and the
         full MDA lifecycle
             –   Model driven architecture
             –   Platform independent and platform specific models
             –   Model simulation
             –   Model transformation


Rational Israel Users
Conference
                         Thanks to:
 • A. Ramfos., S. Liapis, N.        • K. Nagin, O. Edelstein, A.
   Giannelos, A. Hondouridakis,       Kirshin, S. Olvovsky, M.
   M. Sardis, K. Bechrakis, V.        Berg, L. Raskin, T.
   Akousi-Krivki                      Shiran, C. Sacharen, M.
 • K. Dussa-Zieger, J. Trost, B.      Barshay, D. Neimer
   Nossem, T. Linz, B. Mattern,     • L. Mounier, M. Bozga, Y.
   J. Hofer, H. Raessler, T.          Lakhnech
   Rossner,                         • T. Jeron, E. Demairy, V.
 • I. Craggs, I. Griffiths            Tschaen
 • Y.-M. Quemener, D. Vincent,      • J. Davies, A. Cavarra, C.
   T. Heuillard, N. Moteau            Crichton, J. Woodcock,
 • A.Bertolino, A. Wills, S. Reid     M. Field

Rational Israel Users
Conference

				
DOCUMENT INFO