Test Automation - PowerPoint by HC120930213610


									             GUI Test Automation
             Sharing the Good, the Bad, & the Ugly
                       By Pierre Veragen

10/10/2006              SQuAD - GUI TEst Automation   1
 What are we going to cover today?
•   The Different Kinds of Test Automation
•   Why looking at GUI Test Automation
•   GUI Test Automation -- an Investment
•   Forces Affecting GUI Test Automation
•   Case Study 1 -- Record & Playback
•   Case Study 2 – Improved Record & Playback
•   Case Study 3 – “No-Record-Playback” Approach
•   Lessons Learned
•   Review
•   QA

10/10/2006         SQuAD - GUI TEst Automation   2
 The Different Kinds of Test Automation
         (our current approach)
      Component           Main Test Tool       Target Code              Notes
        Type                                    Coverage
        Classes             Unit Test              High                  nUnit
                                                                  Test Driven Design

             API          FitNesse, API
                           driver, .Net,     High code coverage when combined with UT,
                              VB6…                  but it is not what drives the test
                                              Test scenarios and test cases are driven by
                                                the expected usage of the system (Use
         GUI             GUI test tool                Cases…) or its components

              In all cases use code coverage to pinpoint weaknesses in the testing
                           Do not try to test all the layers through a GUI

10/10/2006                          SQuAD - GUI TEst Automation                             3
   Why Consider GUI Test Automation?

Our world today:    What Test Automation can bring:
• Frequent          • Instant feedback from regression
• Frequent builds     testing
• Frequent          • An excellent complement to
  deliveries to       manual testing
  customer          • Repetitive, reliable testing, ( if
                      designed right)
“To catch bugs
  early, we must    • Good coverage for data-oriented
  test often”         verification tasks

10/10/2006          SQuAD - GUI TEst Automation        4
GUI Test Automation - an Investment…
Cost:                  Benefits:
• Licenses             • Catch bugs early when they happen…
• Training               get the “Duh” effect (hidden ROI)
• Consulting           • Free QA resource for other tasks
                       • Forces to write down what we actually
• Building a library
• Writing scripts      • Can start small, then expand on it

And now the main

10/10/2006             SQuAD - GUI TEst Automation               5
…GUI Test Automation - an Investment
      App     App loc   Line of Test Code                        Scripts
                        Shared       App                Qty     Time to      Code
                                    Specific                      run      coverage

      A-      106,000   10,000       200,000            3,000   20 hrs       TBD
     Tool 1                           Done
      B-      130,000   10,000        27,000             1       ½ hr        29%
     Tool 1                           Done
      B-      130,000   20,000        29,000             8      100 hrs      30%
     Tool 2                           Done
      C-      300,000   20,000       2,600               1      8 min        TBD
     Tool 2                         Starting

10/10/2006                SQuAD - GUI TEst Automation                              6
   Forces Affecting Test Automation
    “We may not know the details of future changes, but we can
                 guess where it is going to change”

• Details of control will change
  >> attach names, enabled, visible, list…
• GUI Behavior will change
  >> added message, multilingual, new forms…
• Varied OS, machine speeds, network speeds, file sizes
• Requirement updates
• Members of the test automation team

See handouts for design strategies and more

10/10/2006              SQuAD - GUI TEst Automation              7
 Case Study 1 “Record & Playback”

• Demo 1
• Code duplication
• Maintainability issues

10/10/2006       SQuAD - GUI TEst Automation   8
             “Record & Playback”
The reality:
• Is it really a productivity feature?
• AUT is always on the move
• Every time you record you duplicate code
  ! maintenance cost !
• Pretty soon we will record again to add clicking
  on a message box … in hundreds of places

10/10/2006         SQuAD - GUI TEst Automation       9
                           “Record & Playback”
         Actual Recorded Script:

         Option Explicit

         Sub Main()
         ' Begin Record on Friday, April 14, 2006 at 4:41:34 PM by veragep

             ' Attach to Application=EXPLORER.EXE Caption='F:\SQuAD Presentation on D'
             Window("Application=EXPLORER.EXE Caption='F:\SQuAD Presentation on D'").Attach
                ListView("Index=1").Select "TinyApp.exe"
                ListView("Index=1").Select "TinyApp.exe", tpMouseDoubleClick

             ' Attach to Name=Form1 Application=TINYAPP.EXE
             DotNETForm("Name=Form1 Application=TINYAPP.EXE").Attach
                DotNETForm.Size 304, 192
                DotNETForm.Move 22, 29

         ' End Record on Friday, April 14, 2006 at 4:41:46 PM by veragep

         End Sub

10/10/2006                                                     SQuAD - GUI TEst Automation    10
                    Case Study 2
             “Encapsulate Recorded Code”
• Demo 2
• Goal: limit duplicating code

Code approach:
• Think of GUI functionality
• Pick methods names and arguments easy to remember

DB approach:
• Move name details and sequence of actions to DB

10/10/2006            SQuAD - GUI TEst Automation     11
   “Encapsulate Recorded Code”
    Option Explicit

    Sub Main()
      StartApp "G:\SQuAD Presentation\Tiny App\bin\TinyApp.exe"
    End Sub

    Sub StartApp(strFullAppName As String)
      Shell strFullAppName, vbNormalFocus
    End Sub
    Sub Button1Click()
    End Sub
    Sub Button2Click()
    End Sub
    Sub Form1Attach()
      DotNETForm("Name=Form1 Application=TINYAPP.EXE").Attach
    End Sub

10/10/2006                                                 SQuAD - GUI TEst Automation   12
    Case Study 3 “Development Project”

• Demo of an existing system
• Demo writing a script

10/10/2006      SQuAD - GUI TEst Automation   13
             “Development Project”

10/10/2006         SQuAD - GUI TEst Automation   14
             Development Project…
Business Requirements:
• Mimics the User’s Actions (mouse and keyboard
• Provides a Scripter Façade
• Encapsulates Attach functionalities
• Designed for maintainability and scalability
• Has timely error reporting (as they happen)
• Provides a framework easily maintainable and
• Provides database-driven tests for data and
  behavioral verifications

10/10/2006         SQuAD - GUI TEst Automation   15
             …Development Project
Good coding practices:
• Encapsulate
• No Code Duplication
• Divide your library into App-specific and shared

10/10/2006         SQuAD - GUI TEst Automation       16
    “No-Record-Playback” Approach
The Golden Rules:
• Record only to understand how the test tool thinks and
  how it differentiates one control from another
• Never duplicate code
• Choose your testing goals wisely
• Choose names for reusability

The Benefits:
• Fully encapsulated code – one place for the Click code,
  one place with the actual control name…
• Easy to upgrade and expand if well designed

10/10/2006           SQuAD - GUI TEst Automation            17
   Iterative Approach to Implementation

• List what needs to be tested >> Test Scenarios at a
  high level, using an organized thought process
  (Risk, UC, Requirements…)
• Earmark what could done through test automation
• Prioritize
• Set some reasonable goals (like a Build Smoke
  Test) to validate the automation approach (tool,
  encapsulation techniques, library design…)
• After the first script is running…
• Reconsider your solution

10/10/2006          SQuAD - GUI TEst Automation     18
   Be Selective in What to Automate…

• Repetitive to boredom
• Where we spend most of the time
• Test Scenario that would catch issues as
  they happen

10/10/2006      SQuAD - GUI TEst Automation   19
  … Be Selective in What to Automate
• Cost is 2 to 5 times the time it takes to perform a
• Some tests are nearly impossible to do by hand
• Run script 5 to 20 times before it becomes fully
  stable (it takes longer if more complex)
• Build stability when it does not matter (with daily

10/10/2006         SQuAD - GUI TEst Automation      20
             Avoid the Easy Traps
• When starting the test automation project:
     – Record now, clean up later
     – “Don’t build/run automated tests now because
       the GUI is changing”

• After the test automation project is more
     – Automated testing is enough
     – Volume is quality…

10/10/2006          SQuAD - GUI TEst Automation   21
             The Top Deadliest Pitfalls
• Using Record & Playback
• Having no programming experience in the
• Looking at it as a back-burner project
• Ignoring the fact that you are building a

10/10/2006           SQuAD - GUI TEst Automation   22
         The Next Deadliest Pitfalls
• Being disconnected from reality:
     – The test passes because the script ran
       without errors
     – No crashes, therefore it passes!
     – Capturing baseline values automatically
       without a manual review
• Outsourcing the whole test automation
• Trying to automate everything

10/10/2006          SQuAD - GUI TEst Automation   23
             Review for Success…

• Consider the building of this test automation
  framework as a development project (requirement,
  architecture…) with a development team (SME,
  Automation Engineer, and Tester)
• Pick a small initial goal like automating the Smoke
  Test, then a basic Regression System Test
• Do not attempt to automate everything, think ROI
• Manage timers and synchronization issues in an
  encapsulated way, early on
• Test your own test tool

10/10/2006          SQuAD - GUI TEst Automation         24
             … Review for Success
• Think maintainability as the AUT will change
• Think reusable components (script / library)
• Think scalability: use a library architecture to
  easily adapt to other applications being tested
  and other environments
• Get a developer involved from the get go
• Focus on simple, focused, and robust code

10/10/2006         SQuAD - GUI TEst Automation       25
• Extra Material for GUI Test Automation
     – Expect changes and what to do about them?
• Test Automation – The Manager Version

• Web sites
     – Fit and FitNesse
     – Compuware

10/10/2006          SQuAD - GUI TEst Automation    26

To top