VIEWS: 2 PAGES: 31 POSTED ON: 8/18/2012
Models for Software Reliability N. El Kadri SEG3202 Definitions • 1991 IEEE standard: the probability of failure-free software operation for a specified period of time in a specified environment • The quality of the product improves over time, and we talk about reliability growth • Software-reliability growth problem: estimating and predicting the reliability of a program as faults are identified and attempts made to fix them. • Fault density - The number of faults per thousand lines of executable source code. 2 Using Reliability Models 3 Taxonomy of Software Reliability Models 4 Time-dependent Reliability Growth Models 5 Time-dependent Reliability Growth Models 6 Discussion • One problem with models is an overwhelming number of models has been proposed to address the issue of software reliability assessment. • We must be aware that no single model can be recommended universally to users under any circumstances. • The best models may vary from time to time and differ form application to application. 7 Time-Independent Reliability Models: Fault Injection Estimates the number of faults N in the system in the case where we know the outcomes of the already detected faults. 1. Insert into a software module a certain number A of faults 2. Proceed with its testing 3. Count the number of failures due to injected faults (f) Count the number of failures due to inherent faults (i ) 4. Estimate the number of remaining faults => 8 Time-Independent Reliability Models: Fault Injection Example: 1. Insert into a software module A=25 faults 2. Proceed with its testing (results: 32 failures) 3. Count the number of failures due to injected faults (f=17) 4. Count the number of failures due to inherent faults (i=15) 5. Estimate the number of remaining faults 9 Fault Injection: Confidence Levels about the Number of Faults 1. If not all artificial faults have been found (f<A), then: If all artificial faults have been found (f=A), then: 10 Fault Injection: Example (cont.) 1. Confidence levels about the number of faults Let us set E to 10. f < A => 11 Fault Injection: Example 2 1. Assume all artificial faults have been found (f=A) 2. Given E=10, certain confidence level λ and i<E, how to determine the number of faults to be injected? 12 Time-Independent Reliability Models: Input Domain 1. Software reliability depends on how software operates on a certain input domain. 2. This point of view relates to selecting software test cases over the input domain according to how software is used • Software usage information includes the environment information where software is used, as well as the information on the actual frequency of usage of different operations, functions, or features that the system offers. • The usage information is quantified through operational profiles 13 Input Domain: Equivalence Classes Software Module 14 Operational Profiles • Methodology to develop operational profile: 1. Determine customer profile or usage context profile. 2. Determine user profile. 3. Determine system modes and their profile. 4. Determine the functional (requirements) profile. 5. Determine the operational (implementation) profile. 16 Operational profile • A set of relative frequencies (or probabilities) of occurrence of disjoint software operations during its operational use • A software-based system may have one or more operational profiles. • Operational profiles are used to select test cases and direct development, testing and maintenance efforts towards the most frequently used or most risky components. 17 Operational profile • Construction of an operational profile is preceded by definition of a customer/user profile, a system mode profile, and a functional profile. • Profiles are constructed by creating detailed hierarchical lists of customers, users, modes, functions and operations that the software needs to provide under each set of conditions. • For each item estimate the probability of its occurrence and thus provide a quantitative description of the profile. • If usage is available as a rate (e.g., transactions per hour) it needs to be converted into probability. 18 User Profile - Example 19 System Mode 20 System Modes Example We can identify five system modes: 1. Normal Traffic load 2. High Traffic load 3. Start/Restart 4. Administration 5. Troubleshooting The first three are only relevant to the subscriber user type, the fourth to the operator user type and the fifth to the customer care system user type. 21 Functional Profile 22 Operational Profile 23 Operational Profile 24 Input Domain: Equivalence Classes Software Module 25 Input Domain: Equivalence Classes & Operational Profiles 1. Each Path through the Operational Profile hierarchy defines an equivalent class: • Path E1: subscriber, normal traffic, MO SMS, originating MO SMS • Each path Ej has its associated probability Pj stating that the inputs will come from it under normal operation of the system • Assuming the inputs are independent from each other, the probability of a path is a product of the corresponding operational probabilities, i.e.: for E1 the associate probability is P1 = 0.9*0.6*0.05*0.95 = 0.35 26 Input Domain Model: Reliability Estimation 1. Assumption: c equivalent classes Ei 2. With each class comes its operational profile 3. Let Pi be the probability stating that the inputs will come from Ei under normal operation of the system 4. nj is the number of test cases sampled from the jth input domain Ei, where fj out of them resulted in software failures 5. The estimated reliability is computed as: 27 System reliability • Knowing the reliability of individual components Ri, one can easily compute the reliability of some architectures as follows: (a) Series configuration: (b) Parallel configuration: 28 Classes of Reliability Models and their main Features 29 Availability 31 ANSI/IEEE 982.1-1988 Includes guidance for the following: • Applying product and project measures throughout the software life cycle • Optimizing the development of reliable software with respect to constraints • Maximizing the reliability in its actual use environment • Developing the means to manage reliability in the same manner that cost and schedule are managed 32 More Information 33
"Models for Software Reliability"