Software Quality and Testing

Document Sample
Software Quality and Testing Powered By Docstoc
					March 30, 2004
Exploratory Testing
Testing Approaches
   Analytical
   Information-driven
   Intuitive
   Exploratory
       Design the tests and test concurrently
       Learn the system and test it as you go
       Structure creative testing
       Think while testing
       Risk-based
       Analogous to Extreme Programming
Exploratory Testing Tasks
   Explore
       Elements of the product
       How the product should work
   Design Tests
       Which elements
       Speculate on possible quality problems
   Execute Tests
       Observe behavior
       Evaluate against expectations

   All with test design techniques best suited for the
Exploratory Testing Practice
   Used to probe for weak areas
   Especially useful when
       Weak specifications and requirements
       Little domain knowledge
       Time pressures
   Less appropriate when
       Well-defined test requirements
       Strong need for regression testing
       Repeatable over releases
            Cost of maintenance
            Few new test cases
   Decompose the product into elements
       Areas of function that you can test in 1-2 days
       Define charters
          Decomposition into units that can be tested in 1-2 hours

   Quality criteria
       Capability, reliability, usability, performance, installability,
        compatibility, …
   Select test techniques
   Provides clear mission of why this test
   Suggests what and how it should be
    tested, as well as problems to look for
   Not a detailed plan, but should be as
    specific as possible
   Might include risks, documents and
    desired output
Cost of Maintenance
   Estimates of percentage of total life
    cycle cost: 40% - 90%
Software Maintenance Types
   Adaptive maintenance: changes needed as a
    consequence of operation system, hardware, or
    DBMS changes
   Corrective maintenance: the identification and
    removal of faults in the software

   Perfective maintenance: changes required as a
    result of user requests
   Preventive maintenance: changes made to
    software to make it more maintainable
Why adaptation?
   Lehman’s Law: if a program doesn’t
    adapt, it becomes increasingly useless
       Example: programs that didn’t adapt to the
   The majority of maintenance is
    concerned with evolution deriving from
    user requested changes.
Lehman’s Second Law
   As an evolving program changes, its structure
    tends to become more complex.
       Extra resources must be devoted to preserving the
        semantic and simplifying the structure.

   For most software, nothing has been done
    about it, so changes are increasingly more
    expensive and difficult.
How is Maintenance Initiated?
   Most common: user request
   Types of requests
       Bug reports
            Accepted
            Working as designed
            Documentation fix
       Feature requests
       New environments
Steps for handling a change
   Understand the problem
   Design the changes
   Analyze impact
   Implement changes
   Update documentation
   Regression test
   Release
   What is a patch?
       Quick fix that doesn’t go through the full process
   When should it be used?
       Error that is preventing use of the system
   Problems with use
       Multiple patches can be order dependent
       Users can barely track which ones have been
       Code version explosion
       Permanent fix may or may not be compatible
Problems of Maintenance
   Organizational
       Alignment with objectives
       Cost benefit analysis
   Process
       Impact
       Documentation
       Regression testing
   Technical
       Building software that is maintainable
Different Objectives over Time
   At release: bug-free
   Six months later: competitive or
    competition-leading features
   Two years later: reduce maintenance
Outsourcing of Maintenance
   Outsourcing: inside or outside the company

   Advantages
       Less expensive
       Uses less skilled people
       Frees developers for the next release
   Disadvantages
       Changes can be slower
       May still require developer help
       Code can become less maintainable
Cost Benefit Analysis
(Risk Analysis)
   Will this problem reduce the number of
    programs that I sell?
   Will this problem impact future sales/
   How many people will it affect?
   How important are the customers it will
   Is it a “show stopper” or an annoyance?
Process Issues
   Impact analysis
   Updating documentation
   Reengineering

   Legacy systems
   Reverse engineering
Impact analysis
   How much of the system will it impact?
   How complex are the changes?
   Heuristics:
       Percentage of requirements that are
       Where was the defect injected
        (requirements, design, code)
Updating documentation
   A requirement defect can impact
       Requirements
       Use cases
       Architecture
       Design
       Code
   Where in the process do you update?
       No matter when, there is a period of inconsistent
   Code gets messy over time
       Extreme programming re-factoring
   At some point, quality suffers
       Changes slow
       Fixes introducing errors
   Need to invest in the code!
       Rules as to when to rewrite a module
       Abstractions: variables -> methods
       Harder: when is REDESIGN needed?
Legacy Systems
   Existing systems that are still useful
       May not want to invest in enhancements
            Future functions will use new process
       May not be able to easily modify
            Unsupported language or libraries
            Lack of skills
            No source code available!
Handling Legacy Systems
   Incorporation
       Business as usual
   Encapsulation
       Accessed from new system
       Adapters
            Wrapper around the legacy system
            Adapters in new system
Technical Issues
   Building maintainable software
   What features have we talked about in
    code and documentation that help?
Technical Issues
   Building maintainable software
   What features have we talked about in
    code and documentation that help?
       Well documented code
            Names, headers, style, …
       All the documentation
            Architecture, design documentation, use cases,
             requirements, …
Today’s Tip
   You need to make sure that when you
    turn in your final project, everything is
       Contract can’t be changed
       Other documents
For your enjoyment…
   “Yes, the world needs bankers and artists and
    even professional athletes. They, among
    countless others, create the breadth of
    society and culture. But if you want
    tomorrow to come – if you want to spawn
    entire economic sectors that didn’t exist
    yesterday – those are not the people you turn
    to. It’s technologists who create that kind of
    future.” (Neil deGrasse Tyson, Director of Hayden Planetarium)

Shared By: