Black Box Testing - DOC

Document Sample
Black Box Testing - DOC Powered By Docstoc
					Black Box Testing
Also known as functional testing. Software testing technique whereby the internal workings of the item being tested are not known by the tester. For example, in a black box test on software design the tester only knows the inputs and what the expected outcomes should be and not how the program arrives at those outputs. The tester does not ever examine the programming code and does not need any further knowledge of the program other than its specifications. The advantages of this type of testing include:
   

The test is unbiased because the designer and the tester are independent of each other. The tester does not need knowledge of any specific programming languages. The test is done from the point of view of the user, not the designer. Test cases can be designed as soon as the specifications are complete.

The disadvantages of this type of testing include:
  

The test can be redundant if the software designer has already run a test case. The test cases are difficult to design. Testing every possible input stream is unrealistic because it would take a inordinate amount of time; therefore, many program paths will go untested.

For a complete software examination, both white box and black box tests are required. Black box testing takes an external perspective of the test object to derive test cases. These tests can be functional or non-functional, though usually functional. The test designer selects valid and invalid input and determines the correct output. There is no knowledge of the test object's internal structure. This method of test design is applicable to all levels of development - unit, integration, system and acceptance. The higher the level, and hence the bigger and more complex the box, the more we're forced to use black box testing to simplify. While this method can uncover unimplemented parts of the specification, you can't be sure --that all existent paths are tested. Black Box Testing is testing without knowledge of the internal workings of the item being tested. For example, when black box testing is applied to software engineering, the tester would only know the "legal" inputs and what the expected outputs should be, but not how the program actually arrives at those outputs. It is because of this that black box testing can be considered testing with respect to the specifications, no other knowledge of the program is necessary. For this reason, the tester and the programmer can be independent of one another, avoiding programmer bias toward his own work. For this testing, test groups are often used, "Test groups are sometimes called professional idiots...people who are good at designing incorrect data." 1 Also, do to the nature of black box testing, the test planning can begin as soon as the specifications are written. The opposite of this would be glass box testing, where test data are derived from direct examination of the code to be tested. For glass box testing, the test cases cannot be determined until the code has actually been written. Both of these testing techniques have advantages and disadvantages, but when combined, they help to ensure thorough testing of the product.

Advantages of Black Box Testing
  

more effective on larger units of code than glass box testing tester needs no knowledge of implementation, including specific programming languages tester and programmer are independent of each other 1

  

tests are done from a user's point of view will help to expose any ambiguities or inconsistencies in the specifications test cases can be designed as soon as the specifications are complete

Disadvantages of Black Box Testing
     

only a small number of possible inputs can actually be tested, to test every possible input stream would take nearly forever without clear and concise specifications, test cases are hard to design there may be unnecessary repetition of test inputs if the tester is not informed of test cases the programmer has already tried may leave many program paths untested cannot be directed toward specific segments of code which may be very complex (and therefore more error prone) most testing related research has been directed toward glass box testing

Testing Strategies/Techniques
         

black box testing should make use of randomly generated inputs (only a test range should be specified by the tester), to eliminate any guess work by the tester as to the methods of the function data outside of the specified input range should be tested to check the robustness of the program boundary cases should be tested (top and bottom of specified range) to make sure the highest and lowest allowable inputs produce proper output the number zero should be tested when numerical data is to be input stress testing should be performed (try to overload the program with inputs to see where it reaches its maximum capacity), especially with real time systems crash testing should be performed to see what it takes to bring the system down test monitoring tools should be used whenever possible to track which tests have already been performed and the outputs of these tests to avoid repetition and to aid in the software maintenance other functional testing techniques include: transaction testing, syntax testing, domain testing, logic testing, and state testing. finite state machine models can be used as a guide to design functional tests According to Beizer 2 the following is a general order by which tests should be designed: 1. 2. 3. 4. 5. 6. Clean tests against requirements. Additional structural tests for branch coverage, as needed. Additional tests for data-flow coverage as needed. Domain tests not covered by the above. Special techniques as appropriate--syntax, loop, state, etc. Any dirty tests not covered by the above.

Black-box and white-box are test design methods. Black-box test design treats the system as a "blackbox", so it doesn't explicitly use knowledge of the internal structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque-box, and closed-box. White-box test design allows one to peek inside the "box", and it focuses specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box and clear-box. What is a Black Box Testing Strategy? Black Box Testing is not a type of testing; it instead is a testing strategy, which does not need any knowledge of internal design or code etc. As the name "black box" suggests, no knowledge of internal logic or code structure is required. The types of testing under this strategy are totally based/focused on the testing 2

for requirements and functionality of the work product/software application. Black box testing is sometimes also called as "Opaque Testing", "Functional/Behavioral Testing" and "Closed Box Testing". The base of the Black box testing strategy lies in the selection of appropriate data as per functionality and testing it against the functional specifications in order to check for normal and abnormal behavior of the system. Now a days, it is becoming common to route the Testing work to a third party as the developer of the system knows too much of the internal logic and coding of the system, which makes it unfit to test the application by the developer. In order to implement Black Box Testing Strategy, the tester is needed to be thorough with the requirement specifications of the system and as a user, should know, how the system should behave in response to the particular action. Various testing types that fall under the Black Box Testing strategy are: functional testing, stress testing, recovery testing, volume testing, User Acceptance Testing (also known as UAT), system testing, Sanity or Smoke testing, load testing, Usability testing, Exploratory testing, ad-hoc testing, alpha testing, beta testing etc. These testing types are again divided in two groups: a) Testing in which user plays a role of tester and b) User is not required. Testing method where user is not required: Functional Testing: In this type of testing, the software is tested for the functional requirements. The tests are written in order to check if the application behaves as expected. Stress Testing: The application is tested against heavy load such as complex numerical values, large number of inputs, large number of queries etc. which checks for the stress/load the applications can withstand. Load Testing: The application is tested against heavy loads or inputs such as testing of web sites in order to find out at what point the web-site/application fails or at what point its performance degrades. Ad-hoc Testing: This type of testing is done without any formal Test Plan or Test Case creation. Ad-hoc testing helps in deciding the scope and duration of the various other testing and it also helps testers in learning the application prior starting with any other testing. Exploratory Testing: This testing is similar to the ad-hoc testing and is done in order to learn/explore the application. Usability Testing: This testing is also called as ‘Testing for User-Friendliness’. This testing is done if User Interface of the application stands an important consideration and needs to be specific for the specific type of user. Smoke Testing: This type of testing is also called sanity testing and is done in order to check if the application is ready for further major testing and is working properly without failing up to least expected level. Recovery Testing: Recovery testing is basically done in order to check how fast and better the application can recover against 3

any type of crash or hardware failure etc. Type or extent of recovery is specified in the requirement specifications. Volume Testing: Volume testing is done against the efficiency of the application. Huge amount of data is processed through the application (which is being tested) in order to check the extreme limitations of the system. Testing where user plays a role/user is required: User Acceptance Testing: In this type of testing, the software is handed over to the user in order to find out if the software meets the user expectations and works as it is expected to. Alpha Testing: In this type of testing, the users are invited at the development center where they use the application and the developers note every particular input or action carried out by the user. Any type of abnormal behavior of the system is noted and rectified by the developers. Beta Testing: In this type of testing, the software is distributed as a beta version to the users and users test the application at their sites. As the users explore the software, in case if any exception/defect occurs that is reported to the develop

Black Box Testing is testing technique having no knowledge of the internal functionality/structure of the system. This testing technique treats the system as black box or closed box. Tester will only know the formal inputs and projected results. Tester does not know how the program actually arrives at those results. Hence tester tests the system based on the functional specifications given to him. That is the reason black box testing is also considered as functional testing. This testing technique is also called as behavioral testing or opaque box testing or simply closed box testing. Although black box testing is a behavioral testing, Behavioral test design is slightly different from black-box test design because the use of internal knowledge is not illegal in behavioral testing. Advantages of Black Box Testing
      

Efficient when used on Larger systems As the tester and developer are independent of each other, test is balanced and unprejudiced Tester can be non-technical. There is no need of having detailed functional knowledge of system to the tester. Tests will be done from a end user's point of view. Because end user should accept the system. (This is reason, sometimes this testing technique is also called as Acceptance testing) Testing helps to identify the vagueness and contradiction in functional specifications. Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing 4

    

Test cases are tough and challenging to design, without having clear functional specifications It is difficult to identify tricky inputs, if the test cases are not developed based on specifications. It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult Chances of having unidentified paths during this testing Chances of having repetition of tests that are already done by programmer.

Bridging the Gap between Black Box and White Box Testing Black box and white box tests represent two broad categories of test types. Neither in isolation can accurately depict the quality of the system. Together, however, they give testers a much clearer perspective on system quality. I Thought We Were Testing Software, Not Boxes The "box" in black box and white box testing refers to the system under test; the color refers to the visibility that the tester has into the inner workings of the system. With black box testing, the tester has no visibility into those inner workings. The tester sees only the interfaces exposed by the system. By contrast, white box testing offers the tester full visibility into how the system works. Think of a soda vending machine. A black box test would involve inserting the money into the machine and verifying that a soda drops out and that correct change is given. A white box test might involve opening the back panel on the machine and manually triggering the switch that drops the soda. Black box testing is sometimes referred to as functional or behavioral testing, and it offers numerous benefits. In the first place, a black box test validates whether or not a given system conforms to its software specification. In implementation, black box tests introduce a series of inputs to a system and compare the outputs to a pre-defined test specification. They test not only individual system components, but also the integration between them. The tests are architecture independent. They do not concern themselves with how a given output is produced, only with whether that output is the desired and expected output. Finally, as they require no knowledge of the underlying system, one need not be a software engineer to design black box tests. White box testing is sometimes referred to as structural testing. Because white box tests involve the individual components of a system, they require an implicit knowledge of the system's inner workings. In implementation, white box tests introduce a given set of inputs to a component or individual function of a system and compare the outputs to an expected result. Testing is generally not done through a user interface, but by using the debugging features of the given development environment. How to Choose a Black Box or White Box Test Black box testing is generally performed by QA analysts who are concerned about the predictability of the end-user experience. Their job is to make sure that the application meets customer requirements and that the system does what it's designed to do. But black box tests offer no guarantee that every line of code has been tested. Being architecture independent, black box testing cannot determine the efficiency of the code. Finally, black box testing will not find any errors, such as memory leaks, that are not explicitly and instantly exposed by the application. This stands in sharp contrast to white box testing, in which, given an infinite amount of time, all lines of code can be tested and clues as to the code's relative efficiency can be ascertained. Generally, developers whose time is relatively more expensive than that of QA analysts perform white box testing. White box testing proves insufficient, however, for situations in which testing isolated components may not reveal integration errors with other components. The Tools Rational Software has been producing tools for automated black box and automated white box testing for several years. Rational's functional regression testing tools capture the results of black box tests in a script format. Once captured, these scripts can be executed against future builds of an application to verify that new functionality hasn't disabled previous functionality. Rational also offers white box testing tools that 1) provide run-time error and memory leak detection; 2) record the exact amount of time the application spends in any given block of code for the purpose of finding inefficient code bottlenecks; and 3) pinpoint areas of the application that have and have not been executed.


For as long as these tools have been available, the black box testing tools have resided almost exclusively on the QA analyst's machine, while the white box tools have been purchased primarily by developers. There are several reasons for this divide. In the first place, QA analysts traditionally do not have a coding and debugging environment at their disposal. And even if they did, the argument goes, most analysts would not understand the information output by the tools. Rational believes that this is an artificial barrier, and that QA work can be improved via white box testing tools that do not require access to source code or a development environment. Now, Rational uses a patented Object Code Insertion (OCI) technology to instrument an application's executable files. No source code is required. This approach to the problem also enables Rational tools, Purify Quantify and PureCoverage, to perform white box testing on third party code and controls, for which source code may not otherwise be available. With the introduction of Rational Test Studio in early 1999, white box testing became integrated with black box testing; since then, QA analysts have been able to perform white box tests at the same time as black box tests. With Rational's OCI technology eliminating the need for source code or a development environment, QA analysts now have visibility into the "black" box. While QA analysts are running their functional black box tests, structural white box tests for memory leaks, code bottlenecks, or measuring code coverage can also occur, that development teams as the intended audience. In essence, the QA group can now undertake a job which previously had to be completed by an engineer. Given the relative average salary levels for these two populations, this is clearly an efficient optimization. Great Minds Don't Think Alike, They Think Together I don't think it's too much of a stretch to claim that the technological advancement of Rational's tool set is like a marriage of the white box and black box testing roles. That's why I began this article with the anecdote about my own recent marriage. We're still different people, my wife and I, but our goals are more focused now, and we're working together toward some common objectives. Similarly, the more testing professionals can share their detection tools and information base, the more quickly and accurately they can ascertain the overall quality of a software application. Testers performing traditional black box tests can leverage these same scripts to reap information from their testing that was previously only available to the white box tester on the development team. In so doing, QA analysts have the opportunity to lighten the load on the development team at very little expense. Earlier, I said that "given an infinite amount of time, all lines of code can be tested." Obviously, no software team has the luxury of an infinite amount of time. However, if one reduces the goal of white box testing to finding memory leaks and application crashes throughout the code base, then the time needed for white box testing becomes much more manageable. By putting a common tool set on both the developer's and the QA analyst's desktops, Rational has not only brought black box testing and white box testing a step closer together, but it has also brought developers and QA analysts closer together. This more unified team may be the greatest benefit of all.


Shared By:
Tags: Black, Testing
Description: Black Box Testing