PowerPoint Presentation

Document Sample
PowerPoint Presentation Powered By Docstoc
					              Software Test Automation
                   - An Overview

                      By Thrineshwara M.R.

“Test automation solutions may be driven by business needs and in order to
succeed, product design should be driven by the testability of the product”
   Challenges faced in testing
   What is test automation?
   How to decide about automation?
   Principles of Test Automation
   Basic stages in test automation
   Preparation of automated test suite
   Advantages & Disadvantages
   Application/Suitability of test automation
   Automation Framework Design
   Strategy for test automation
   Simple examples
   Popular tools used for Test Automation
Challenges faced in testing

   Multiple products and multiple releases
   Repetition of test cases across multiple test instances or
   Deployment of large number of personnel
   Very large cycle time
   Increase in test preparation time with increase in number of
    test instances

Test Automation helps in overcoming the above
What is test automation?

   Elimination of manual operations during testing by
    use of software
   Replacement of people by computers
   Faster execution
   Centralized management of test preparation and test
   The actual time taken by the devices to execute
    certain commands remains same even after
How to decide about automation?
Try to find answer to the following questions:-
   Can the test sequence of actions be defined?
        Repetition of sequence of actions such as Acceptance test, Compatibility
         tests, Performance tests and Regression tests
   Is it necessary to repeat the sequence of actions many times?
   Is it possible to automate the sequence of actions?
        See if automation is not suitable for a particular sequence
   Is it possible to "semi-automate" a test?
        This can speed up the test execution time
   Is the behavior of the software under test the same with automation
    as without?
        See if Performance tests are impacted by automation
   Do you need to run the same tests on multiple hardware
        All non-UI tests should be automated
Principles of Test Automation
   Controllability
       The ability of the automation software to send commands or input
        data to the device under test (DUT) or to get required service from
        the DUT software
            Possible either fully or partially
            Most often decides the scope of automation systems

   Observability
       The ability of the automation software to look into specific
        parameters or outputs generated by the DUT for analysing the
        behaviour of the DUT
            Possible either fully or partially
            Most often decides the efficiency of automation system
Basic stages in test automation

   Test initialization
   Test execution
   Data capture & Analysis
   Decision making & Reporting results
   Cleaning
Preparation of automated test suite

   Selection of manual test cases
   Test suite generation
   Scripting of test cases
   Dry run of test scripts & debugging
   Final validation of test scripts
Preparation of automated test suite
Advantages & Disadvantages
   Advantages
       Elimination of repetitive manual operations
       Saving of time in time consuming operations such as
        initialization, data capture, analysis and cleaning
       Execution of testing continuously for several days round
        the clock
       Elimination of possible human errors
       Drastic saving in test execution cycle time
       Improvement in product quality due to better test
Advantages & Disadvantages
   Disadvantages
       Manual intervention needed if the test execution breaks
        due to error condition
       Inability to automate all the test cases available
       Additional effort needed to automate test cases and
        validate them
       Choice of a suitable automation framework or tool is
       Analysis and decision making is hard coded hence special
        observations to be made will not help
       Difficult to carry out exploratory testing
            Simultaneous learning, test design and execution
            Ad-hock testing (hence measurement is not possible !!)
            Unscripted testing
GUI Automation
   TC creation is expensive
   More staff needed in order to achieve the same breadth of
   Most of the available technical staff are tied up in
   Automation can delay the testing , adding more cost
   GUI changes force maintenance of tests
   Regression testing has less power
   Maintainability is a core issue since the main payback is in
    next release
   Unsolicited events cause more trouble
   Multiple ways in, multiple ways out
Accessibility Test Automation - Challenges

   Interpretation of guidelines without common sense
   Highly difficult to check content issues
   Highly difficult to check code issues
       Ensuring that text is real text and isn't embedded within images
       Making sure that the site functions without the use of JavaScript
        or Flash
       Providing equivalent text links if using server-side image maps
       Ensuring that the structure within the HTML reflects the visual
        appearance (e.g. headings are labeled as headings within the HTML
   Using outdated guidelines
   Most guidelines aren't properly checked
   Warnings are misinterpreted
Application/Suitability of test automation
   Large number of test cases leading to longer test execution
   Repetitive nature of many test cases or test execution
   Good amount of testing in User interfaces (GUI) (but
   Time consuming operations such as test configurations, re-
    initialization, s/w downloading, remote operations and
    multiple test platforms
   Manual operation is proving costly and producing longer test
    cycle time
   Test cases are stable and do not change frequently
Not to replace human beings but to make it faster and
 more efficient
Strategy for test automation
   Test automation is a fulltime effort, not a sideline.
   The test design and the test framework are totally
    separate entities.
   The test framework should be application-
   The test framework must be easy to expand,
    maintain, and perpetuate.
   The test strategy/design vocabulary should be
    framework independent.
   The test strategy/design should remove most testers
    from the complexities of the test framework
Automation Framework Design (1/2)
What is a Framework?
A set of assumptions, concepts, and practices that constitute a work platform or support for automated
   Test script modularity framework
        Create small, independent scripts that represent modules, sections and functions of the application under
        Using small scripts create larger test cases
        This framework applies principle of abstraction or encapsulation in order to improve the maintainability
         and scalability of automated test suites
   Test library architecture framework
        Similar to above but it divides the application-under-test into procedures and functions instead of scripts
        Requires the creation of library files (APIs, DLLs, and such) that represent modules, sections, and functions
         of the application-under-test
        The library files are then called directly from the test case script
   Key word driven testing framework
        Application independent
        requires the development of data tables and keywords, independent of the test automation tool used to
         execute them and the test script code that "drives" the application-under-test and the data

[ Courtesy: http://www.ibm.com/developerworks/rational/library/591.html ]
Automation Framework Design (2/2)
   Data driven testing framework
        Here test input and output values are read from data files (data-pools, ODBC sources, CSV
         files, Excel files, DAO objects, ADO objects, and such) and are loaded into variables in
         captured or manually coded scripts
        Variables are used for both input values and output verification values
        Navigation through the program, reading of the data files, and logging of test status and
         information are all coded in the test script
        Similar to table-driven testing in that the test case is contained in the data file and not in the
        Unlike in table-driven testing, though, the navigation data isn't contained in the table
        In data-driven testing, only test data is contained in the data files
   Hybrid test framework
        a combination of all of the above techniques, pulling from their strengths and trying to
         mitigate their weaknesses
        The one which evolves over time and multiple projects
        Most commonly implemented framework

[ Courtesy: http://www.ibm.com/developerworks/rational/library/591.html ]
Examples – The general structure of test automation

             read a test-case line
             parse out test-case data
             build up input arguments
             call the method-to-test
             if actual value = expected value
                    record a "pass" result
                    record a "fail" result
      end loop
Examples – ssh connection & Ping test
Manual operations:
  [root@prakash-laptop ~]# ssh
   root@'s password:
   Last login: Wed Feb 27 10:30:26 2008 from
   [root@pankajsharma ~]#

Automation Scripts in Tcl:
   package require Expect

   spawn ssh
   expect "password: "
   send "root123\n"
   expect "#"
   [ add scripts for running ping command here]
    [ add scripts for data capture… here ]
   [ add scripts for verification… here ]
Tools used for test automation
   GUI test automation tools
       WinRunner
   Performance test automation tools
       LoadRunner, IXIA, Smartbits
   Functional test automation tools
       EAST, ApTest, TTCN
   Usability test automation tools
       TestQuest, MobileComplete
   API level test automation tools
       Proprietary tools

Web site for details on scripting languages:
Are you ready to automate & free up yourself ?
Have you become redundant after automation ?

Shared By: