Agenda - INCOSE by gegeshandong


									Better Specifications
      What is a Specification?
• A Statement of the Customer’s Needs
• In the Form of Required Characteristics of a
• A Component of a Legal Contract Between
  Customer and Supplier
• Subject to Verification of Compliance
      Specification Organization
•   MIL-STD 490
•   Should Follow Functional Decomposition
•   PUI Organization can Help
•   Performance Requirements Not
    Decomposed Further When Can Allocate
    to Next Level of Hierarchy
             Ground Rules
• Say What You Mean Precisely and Simply
• Only “SHALL” Imposes a Contractual
• Do Not Use Subjective Adjectives (large,
  rapid, modular, nominal,optimum, efficient)
• Define Verification Criteria as part of
  Requirement Statement Generation
• Provide Definitions
       Some Characteristics of Good
•   Clear
•   Necessary
•   Applicable to only One System Function
•   Not Redundant
•   No Conflict With Other Requirements
•   Not Biased by any Particular Implementation
•   Verifiable
•   Traceable
•   Achievable
    Partial List of “Poor Words”
• Adequate – Not verifiable
• And – Possible multiple requirements
• Appropriate – Not Verifiable
• Best Practice – Not verifiable
• But not limited to – Unspecified super set, not
• Easy – Not verifiable
• For Example – Not verifiable
      More Not Verifiable “Poor
•   Including      •   Monitor
•   Large          •   Sufficient
•   Many           •   User Friendly
•   Maximize       •   Quick
•   Minimize       •   Effective
•   As a minimum   •   Normal
•   Rapid          •   Provide for
                   •   Should
     Parent Child Relationships
         Specification View
3.2.x Power-up Missile
3.2.x.1 Receive External Power
3.2.x.2 Perform SBIT
3.2.x.2.y Computer SBIT
3.2.x.3 Go to Standby
            Some Problems
• Specifying Capability – Does not Ensure
  Delivered Unit Will Perform as Desired.
  – “…the unit’s design shall be capable …”
• Design Could be Correct but Unit
  Improperly Fabricated.
         Problems (Continued)
• Absolute Performance Parameter
  –   Variability Exists (Profound Knowledge)
  –   Tolerance Range Required
  –   Values, not Percentages
  –   Define Origin for Time Requirements
       Problems (Continued)
• Use of “Support”
  – “…The XXX shall support …”
• How do You Verify Compliance?
    A Well Written Specification is
•   A Design Description
•   A Statement of Work
•   A Test Plan
•   A Concept of Operations Description
•   A Novel
Electric Water Heater Controller
•   What is the Required Function?
•   Output = 3000 W; 70°<T<100°
•   Output = 2000 W; 100°<T<130°
•   Output = 1000 W; 120°<T<150°
•   Output = 0;       150°<T
•   Is this Requirement Set Valid?
                       Requirement Examples
  Good requirement statement:
         “The system shall provide a water flow
         rate of 500 gallons per minute ±10 gallons
         per minute.”
  Lousy requirement statement:
         “The system should provide a flow rate to
         the maximum extent possible.”

 Courtesy of Reed Integration, Inc.

IBIT Response Time
The XXX shall complete IBIT and transmit the test results to the MC within 5 minutes.

CBIT Coverage
The XXX design shall allow CBIT to test and status functions on a non-interfering
continuous basis.

IMU Operating Time
The IMU shall operate 10 minutes for tactical operation.

Acceleration Measurement Range
The IMU Accelerometer provides velocity measurement data along the three orthogonal axes
for accelerations not exceeding 50g along any axis.

IBIT Coverage
The XXX design shall allow IBIT to test no less than 99 (TBR) percent of the electronic function
critical missile faults. The following minimum functions shall be tested:
            Transmission of software version in Read Only Memory to MC.
            Determine internal temperature of package
                    Good Requirements Statements
1.     The electrical power subsystem shall provide 28 +4, -2 vdc to the system.
2.     The weight of individual items shall comply with MIL-STD-1472 paragraph
       xxx.yyy.1 titled “Single person lift requirements”.
3.     The command generator process shall receive inputs from the operator,
       and generate commands in the format specified in IFS 10345.
4.     The alignment error caused by structural deformation due to loads and
       thermal gradients shall be no greater than 0.01 deg.
5.     The vehicle shall receive and authenticate commands from the ground
       station, and provide an acknowledgement that the command has been accepted
6.     The vehicle shall perform as specified within the temperature range
       of –25 to +65 deg Celsius.
7.     The system shall have the capability to accept no less than 25 targets.
     Courtesy of Reed Integration, Inc.
      Some Famous Failures
• Titanic – Rivet Quality Verification
• Tacoma Narrows Bridge – Requirement not
• Apollo 13 – Failure to Change Spec and
  Verify Compliance
• IBM PC Jr – Requirement not Valid
• See Bahill & Henderson, Systems
  Engineering, Vol 8, Nr 1, 2005, pp 1-14
• Most Specs are Invalid and not Verifiable
• Poor Specs lead to Program Problems
• Product Assurance Begins with Ensuring
  Valid and Verifiable Specifications
• If it is not in the contract it will not be
      Compliance Verification
• Operational Definitions
• Compliance Verification Criteria
  – Source
  – Where Documented
• Integration Into Program
       Operational Definition
• A definition by which one can do business
• Without benefit of lawyers
• Need – A blanket, 50% wool, 50 % cotton
• Requirement Statement
   – “The blanket shall contain a 50/50 blend of wool and
     cotton fibers.”
• Possible Solution
   – Take a wool blanket and a cotton blanket
   – Cut each in half
   – Sew a wool half to a cotton half
• Question – Has requirement been satisfied? How
  do you know?
             Better Statement
• “The blanket shall contain a homogeneous
  blend of wool and cotton fibers.”
• Questions
  –   How do we define homogeneous?
  –   How do we define wool and cotton?
  –   What method will we use?
  –   What results are acceptable?
• Method
  – Pick 3 blankets at random from a lot of 200
  – For each blanket pick 10 locations randomly distributed
    across the surface of the blanket.
  – Cut a one inch diameter circle sample at each location
    using a ruler to measure diameter.
  – Subject each sample to fiber classification in
    accordance with XYZ standard.
  – Measure the weight of each fiber type using a calibrated
    laboratory scale.
        Solution (Continued)
• Criteria
  – If the proportion by weight of wool to cotton or
    cotton to wool is at least 45/55 for all samples,
    accept the blend as homogeneous.
• We can do business!
• Every specification requirement statement
  containing SHALL must have an associated
  criterion for verifying compliance.
• No requirement statement is complete until
  the verification criteria are defined
• If you can’t figure out how to verify
  compliance, it should not be a requirement.
          Verification Methods
•   Inspection
•   Demonstration
•   Analysis
•   Test
•   Method identified in Verification Cross
    Reference Matrix
     Verification by Inspection
• Physical Properties
  – Weight
  – Center of Gravity
  – Moments of Inertia
• Dimensions
• Appearance
• Will have tolerance or standard
    Verification by Demonstration
•   Maintenance times
•   Speed
•   Load
•   Transportation
•   Environmental qualification
•   Go/No go, (Bernoulli Process)
       Verification by Analysis
•   Kill probability
•   Performance range
•   Effectiveness
•   Efficiency
           Verification by Test
•   Accuracy
•   Compatibility/interactions
•   Reliability
•   Timing
•   Functioning
•   Mechanical Properties
             Problem Areas
• Stochastic parameters
  – Reliability
  – Accuracy (variability)
• Origin definitions
  – From where do we begin measuring
• Event definitions
Accuracy Compliance

Requirement:Sigma < Sigma0
    Use hypothesis test
       Alpha = 0.05
             Hypothesis Test
        for Accuracy Compliance
           Ho: Sigma=Sigma0
           H1:Sigma < Sigma0
State         Accept Ho          Reject Ho
Ho True       No Error           Type 1 Error
Ho False      Type 2 error       No Error
      Operational Characteristic
•   P[Accept Ho| False] vs. Sigma|Sigma0
•   β vs Sigma|Sigma0
•   Depends on Sample Size
•   Depends on α
                                     OC Curves. Alpha = .05





P[Accept Ho]






                     0   0.2   0.4            0.6             0.8   1   1.2
                                         Sigma/Sigma 0
                                   CEP Confidence Limit vs Sample size

CEP Upper Limit/CEP


                      1.6000                                              C=0.9
                      1.4000                                              C=0.85

                               0       5     10    15    20    25    30
                                              Sample Size
Risk of not satisfying the requirement is high
  – Demonstrated Accuracy is much better than
    actually required
  – A very large sample is obtained with accuracy
    close to the requirement.
  In any case, Demonstrated Accuracy Must be
    better than the Requirement
               Better Way
Specify a tolerance around a target point and a
  minimum number of samples.
Compliance verification criterion is clear.

Note: Will still have to evaluate risk of
 unacceptable performance as a function of
 tolerance and sample size.
 Origin and Definition Problems
• Requirement:
  – “The XYZ shall activate within 10 milliseconds
    after receipt of an activation command”
• Problems:
  – What event corresponds to “activate?”
  – What is the definition of “receipt?”
  – When does one start and stop measuring?
• Section 4, Quality Assurance Provisions,
  should contain the compliance verification
• The verification cross reference matrix
  should should identify the verification
  method and reference the paragraph that
  contains the criteria
         Verification Example
Specification Statement

3.1 Blanket Material Properties – The blanket fabric shall
be a homogeneous blend of 50% wool and 50% cotton

 Verification Cross Reference Matrix

 Para   Title                            Method      Procedure
 3.1    Blanket Material Properties      T           Para 4.3.1
  .               .                       .               .
  .               .                       .               .
  .               .                       .               .
       Verification Example (Cont’d)
Section 4.0

4.1 Inspection Procedures
4.2 Demonstration Procedures
4.3 Test Procedures
4.3.1Blanket Material Properties Select three blankets at random from each lot in accordance with
     paragraph 4.x.y (randomization procedure) For each blanket, select ten locations randomly distributed across the
     surface of the blanket in accordance with paragraph 4.x.z. Cut a 1.0  0.1 inch diameter circular sample from the blanket at each
     of the selected locations. Subject each sample to fiber classification by weight in accordance
     with Procedure XYZ. Declare the sample compliant if the classification result is at least
     45%/50% wool to cotton OR cotton to wool. Declare a blanket compliant if at least nine of the ten samples are
     compliant. Declare the lot compliant if all sampled blankets are compliant.
      Program Implementation
• Since every requirement containing “shall” is a
  contractual requirement, compliance must be
• A procedure for collection, documenting, and
  displaying compliance information should be
  defined and managed. Requirements Allocation
  Database is a Possibility.
• Criteria should be developed in conjunction with
  the requirement.
• A master plan (Verification Plan) to integrate
  compliance verification activities into
  development tests and other activities can be
  developed, but criteria must be provided to the
• Requirement compliance verification
  requires operational definitions.
• Requirements are not complete without
  compliance criteria.
• Compliance verification can be easy or
  hard, depending on how the requirement is

To top