Fundamentals of Software Testing by KyleEfaw

VIEWS: 59 PAGES: 33

									Acceptance Testing for
       ROME
          Pete Castle
    Test & Quality Manager
                Agenda
•   What is software testing/ Who does it?
•   Why software testing is important
•   Some fundamentals of testing
•   Test Plans & Scripts
•   Sample Testing Techniques
          What is software Testing?
•   “Software testing is an empirical technical investigation
    conducted to provide stakeholders with information
    about the quality of the product or service under test”
    Professor Cem Kaner - Director of Florida Tech's Center
    for Software Testing Education & Research

•   Empirical - derived from experiment, experience, and
    observation
•   Technical - Having special skill or practical knowledge
•   Investigation - A detailed inquiry or systematic
    examination
         Why testing is Important
•   All Software has defects (bugs)
•   All software products are „prototypes‟ (in my view)
•   Software products are getting larger and more
    complicated - Vista 40% larger than XP @ over 50
    million LOC
•   Software Engineering is not as mature as other
    disciplines e.g. Civil Engineering
•   Software is written by people – people make mistakes
•   Software testing looks to find the most important defects
    as early as possible – increasing confidence that the
    software meets specification
     Who‟s involved in testing?
• Requirements Analysts – Inspections, Peer Reviews
• Developers – Code Inspection, Unit Testing
• Testers – System & Integration Testing
• Trainers – Training materials production
• Users – User Acceptance Testing
• Project Managers – Scheduling, Resourcing, Risks,
  Issues, Defect Stats
• Everybody is responsible for quality - NASA
     Fundamentals of Software Testing

           Plan     Specify   Execute   Record   Check




•   Software testing needs planning, tests need specifying,
    once executed they need results recording, and post
    completion should be easily auditable
    The importance of a planned approach
•    Important to map out a strategy that will give the greatest
     level of confidence in the product
•    „Ad hoc‟ testing may find errors, but may not be cost
     effective
•    Testing should focus on areas where defects are most
     likely
•    All testing should have a reason
     •   Question “Is a test that doesn‟t find an error a good test or not?”
•    Essential to plan what needs to be done and then
     itemise how it is to be achieved.
                    Testing Mantra
• Mantra - Spiritual conduit, words or vibrations that instil
  concentration in the devotee.

• Test as early as possible

• Gather as much knowledge of the application under test as possible

• Look for vulnerabilities

• Build „Bug Taxonomies‟ (Classification)

• Use Quicktests (and publicise the fact)
                    Testing Mantra
• You can always think of another test – but should you?
    • Concept of „Good enough Testing‟
    • Practicality over dogma
    • Everybody has responsibility for „shipping‟ the product


•   Record all tests/defects/issues/recommendations

• Testers are not the sole arbiters of quality
    • Testing only shows problems exist – not their absence


• Never, ever, ever make it personal
    • Defects are issues with products and process not people
    • Good working relationship is essential for good products
Document Hierarchy - Test Plan
           What is a Test Plan - 1
   Test plan is
       tool to help plan the testing activity
       product to inform others of test process
   Includes
       Document control
       Objectives
       Scope
       Approach – Schedule, Priorities, Deliverables,
        Resources, Responsibilities
       Risks/Contingences
       Sign-off/Approval
       What is a Test Plan - 2

• Produced by Test Lead/Project Manager
• Published to Project/Programme
• Not constrained by format – living document
• Enough information to be used by anyone to test
  the product
          Rome Test Plan

 Ready  for review
 Written by Tim Wells
Document Hierarchy -Test Scripts
                 Test Scripts
• Test Script - Is a collection of test cases for the
  software under test (manual or automated)
• Test Case - A set of inputs, execution
  preconditions, and expected outcomes
  developed for a particular objective, such as to
  exercise a particular program path or to verify
  compliance with a specific requirement.
      Pre-conditions/Information
•   Browsers – IE, Firefox, Safari
•   O/S – Linux, Windows
•   Access Control – Logins, Roles
•   Test Data requirements
•   Date/Time considerations
•   Other document references
                Example Test Script - 1
  •    System Test of input of numeric month into data
       field


Ref.    Field/Button   Action       Input        Expected Result                                Pass/Fail


001     Month          Enter Data           0    Data rejected. Error Message 'Invalid Month'   Fail


002     Month          Enter Data           1    Data Accepted, January Displayed               Pass


003     Month          Enter Data           06   Data Accepted, June Displayed                  Pass


004     Month          Enter Data           12   Data Accepted, December Displayed              Pass

005     Month          Enter Data           13   Data rejected. Error Message 'Invalid Month'   Fail
                 Example Test Script - 2
Search
Researcher
Page
Test    Reqs                                                                                                            Pass/
Ref.    Ref.     Function      Inputs                   Expected Result                       Actual Result             Fail
2.001   REF003   Search        1. Forenames = John      All UCL researchers with              427 matches - paging
                 Researchers   2. Surname = <Blank>     forenames starting Pete displayed     working correctly, data
                               3. eMail = <Blank>       in alphabetic order, 23 records per   displayed correctly and
                                                        page                                  in reasonable time (5
                                                        List comprises Name, Department,      secs)
                                                        Occupation Type
                                                        All data items hyperlinked
                                                                                                                        Pass
2.002   REF003   Search        1. Forenames = <Blank>   All UCL researchers with surnames     61 matches - paging
                 Researchers   2. Surname = Smith       starting Smith displayed in           working correctly, data
                               3. eMail = <Blank>       alphabetic order, 23 records per      displayed correctly and
                                                        page                                  in reasonable time (5
                                                        List comprises Name, Department,      secs)
                                                        Occupation Type
                                                        All data items hyperlinked
                                                                                                                        Pass
               Example Test Script - 3

                                                                   New/      Covered   Pass           Date
No.   Type         Scenario        Test                 To test?   Change?   (Y/N)     / Fail   Who   tested
 39   Admissions   Basic Licence   Basic Licence        Y          Y         Y         P        SM    06/01/08
                   Admissions      Admissions - Setup


 40                                Basic Licence        Y          Y         Y         P        SM    06/01/08
                                   Admissions -
                                   Criterion Setup

 41                                Basic Licence        Y          Y         Y         P        SM    06/01/08
                                   Admissions -
                                   Admissions Policy

 42                                Basic Licence        Y          Y         Y         P        SM    06/01/08
                                   Admissions - ADT
                                   Import from EMS

 43                                Basic Licence        Y          Y         Y         F        SM    06/01/08
                                   Admissions - ASL
                                   Export to EMS
               Test Scripts
 Readable
 Accessible
 Usable  by anyone – standards
 Can vary depending upon the type of
  testing being undertaken
           Testing Techniques

•   Quicktests
•   Negative Testing
•   Integration Testing
           Techniques 1- Quick Tests
•   Quicktests - Investigation more important than
    Confirmation

•   A quicktest is a cheap test that has some value but
    requires little preparation, knowledge, or time to perform

•   Focus on common coding errors
              Techniques 1- Quick Tests
•   Things that have failed before – Defect data
     •   Tab order (particularly when adding new functions)
     •   Addresses (BFPO, new Post Codes)
     •   Short cut keys

•   Boundaries – Ages, Dates, Values, Increments, Page Breaks

•   Interrupts, Duplications, Ordering of tasks
     •   Generate Order before setup – no cost codes, cost centres, suppliers,
         budgets
     •   Exit/Interrupt before completion

•   Consume resource (Dog Piling)
     •   Never close a window
     •   Never exit an option
              Techniques 1- Quick Tests
•   Force all error messages
     •   Informative messages - Spelling
     •   Debug information?


•   Capacity/Files – Files to fill all available space, large files, empty
    files, incorrect format

•   Dependencies – e.g. same student many functions
     •   Integration Quicktest


•   Comparisons – screens, data, reports
     •   Tools e.g. Beyond Compare, Screen Capture, Redgate Toolset, InCtrl3
               Negative Testing

•   Testing the system using negative data – to generate
    exceptions that cause the software to fail
•   Going against knowledge of „How the system should
    work‟
•   For Example - Testing the password where it should be
    minimum of 8 characters - so test it using 7 and 9
    characters
•   Emphasis on breaking not confirmation
                 Negative Testing
•   Embedded single quote and other „special‟ characters e.g. John‟s
    Car, John & Erin, 99%, Straße (German Addresses)
•   Required Data Input – Don‟t
•   Field Size – Shoe test (also Quick Test)
•   Field „Types‟ – Characters in numeric field
•   Boundaries (Upper/Lower) – underage job applications, 101 lines on
    an order with a maximum of 100 lines
•   Invalid dates e.g. 31/04/08
•   Addresses – BFPO, Hong Kong Addresses, „New Post Codes‟
•   Web Session Testing – Access web page but not logged in
•   Switch off during upgrade – what happens, does the application
    know there is a problem?
       Integration Testing (In the large)

•   “Testing performed to expose faults in the interfaces and
  in the interaction between integrated components and
  products” Sue Myler – Integration Team Lead
• Usually scenario based rather than low level test cases
• Relies upon testers having system knowledge & testing
  expertise – ability to think „outside of the box‟, develop
  new tests during testing
• Relies on successful unit, integration in the small and
  system testing
• Can mimic business processes
     Integration Testing (In the large)

Integration Test Cases
    • 3 Applicants
       • 1 applies for 1 post
       • 1 applies for 2 posts - also applies for the same
         post twice (by accident)
       • 1 applies for 3 posts
       • do their records appear correctly across ROME
    • Delete a Vacancy – what happens to that applicant
      records?
         Integration Testing (In the large)

•   Short list applicant for post he entered twice, deleting
    one application
•   Invite for interview but candidate withdraws
•   Candidate then re-applies
•   Data exported, ROME updated, then re-exported, does
    data appear correctly in target application
                                   Test Scenarios
                                                                    New/      Covered   Pass           Date
No.   Type         Scenario         Test                 To test?   Change?   (Y/N)     / Fail   Who   tested
 39   Admissions   Basic Licence    Basic Licence        Y          Y         Y         P        SM    06/01/08
                   Admissions       Admissions - Setup

 40                                 Basic Licence        Y          Y         Y         P        SM    06/01/08
                                    Admissions -
                                    Criterion Setup
 41                                 Basic Licence        Y          Y         Y         P        SM    06/01/08
                                    Admissions -
                                    Admissions Policy
 42                                 Basic Licence        Y          Y         Y         P        SM    06/01/08
                                    Admissions - ADT
                                    Import from EMS
 43                                 Basic Licence        Y          Y         Y         F        SM    06/01/08
                                    Admissions - ASL
                                    Export to EMS
 44                                 Basic Licence        Y          Y         Y         F        SM    06/01/08
                                    Admissions - ATF
                                    Import from EMS
 45                                 Basic Licence        Y          Y         Y         F        SM    06/01/08
                                    Admissions - ATF
                                    Re-Import from EMS
                                    with additions and
                                    amendments
                            Review
•   Software Testing is important for increasing confidence that the
    software meets specification
•   To get the best results from testing certain fundamentals should be
    followed
•   Testing is part of software development
•   Different software testing techniques enhance our ability to test
•   Many different types of software testing exist – which we can
    combine into single test cases/scenarios
     Test Example – Data Entry Screen

•   Create Test cases to negatively test (break) the
    data entry screen
      Description             Data/Validation

      Title                   20 Chars, mandatory, pick list
                              provided
      Forename                25 Chars, mandatory
      Surname                 25 Chars, mandatory
      email Address           50 Chars, mandatory, validated
      Home Telephone Number   25 Chars
                Next Steps
 ROME    - Kick off meeting
    Testing required who/when
    Test Script Template
    Mantis - Issue/Defect Logging

								
To top