Docstoc

Powerpoint - CSSE Website

Document Sample
Powerpoint - CSSE Website Powered By Docstoc
					Requirements Volatility
Workshop



Rick Selby
USC Center for Systems & Software Engineering
http://csse.usc.edu
February 15, 2007
                            Attendee List


Rick Selby          NGC                         Rick.selby@ngc.com
JoAnn Lane          USC CSSE                    Jolane@usc.edu
Sue Koolmanojwong   USC CSSE                    Koolmano@usc.edu
Scott Rigby         Raytheon                    Scott_rigby@raytheon.com
Mike Barker         NAIST                       Mbarker@computer.org
Jim Lambert         CISCO                       Jlamber@cisco.com
Tom Schroeder       BAE Systems                 Tom.schroeder@baesystems.com
Robert Culbertson   CISCO                       Rculbert@cisco.com
Gary DeGregoro      Motorola Labs               Gary.degregorio@motorola.com
Ray Hunnicutt       Lockheed Martin             Ray.hunnicutt@lmco.com
Karen Owens         The Aerospace Corp.         Karen.Owens@aero.org
Da Yang             Institute of Software CAS   Yangda@itechs.iscas.ac.cn
                           Attendee List


Dan Ligett       Softstar              Ligett@softstarStystems.com
Ray Madachy      USC CSSE              Madachy@usc.edu
Anna Warner      Boeing                Anna.warner@boeing.com
Barbara Park     Boein                 Barbara.park@boeing.com
A Winsor Brown   USC CSSE              Awbrown@usc.edu
George Hulin     DSC                   g.huling@alumni.duke.edu
Rosaline Lewis   The Aerospace Corp.   Rosalind.lewis@aero.org
Barry Boehm      USC CSSE              Boehm@usc.edu
Di Wu            USC CSSE              Diwu@usc.edu
Vu Nguyen        USC CSSE              Nguyenvu@usc.edu
Rod Robertson    Boeing                Rodney.robertson@boeing.com
Workshop Guidelines

 Product: briefing, preferably with notes

 Topics should include:
   – Most critical success factors in each area
   – Current best practices for addressing them
   – Areas for further research
       Rated 0-10 on value and difficulty of
        research



                                                  4
Requirements Critical Success Factors
 Technical
   – (tech)Investigate req interdependencies and dependencies
   – (tech)Demistify the notion that requirements are independent
   – (tech)Requirements gathering-what are the important attributes
     that should be captured
   – (tech)Want virtual model and understand attributes
   – (tech and people)Better ways to prioritize requirements,
     considering risk, cost factors, stakeholder utility function and how
     to reconcile stakeholders interest
   – (tech and people)Same as above with greater emphasis on
     change
   – (tech)Specify requirements/not implementation; users often want
     to specify implementation
   – (tech)What are the critical/driving requirements




                                                                            5
Requirements Critical Success Factors (continued)

 Technical
   – (tech)Relate requirements work back to COSOSIMO and
     COSYSMO
   – (tech)Missing rich enough vocabulary to model ―this whole thing‖
     (wants, needs, problems, issues, business drivers that are
     referred to as requirements)
   – (tech)Need modeling approach other than English
   – (tech/tool)How do we automatically measure requirements,
     requirement volatility, progress, scope
   – (tech)Accurately assess/predict impact of changes
   – (tech)Standard classification method to capture types of
     requirements and level of requirements
   – (tech)Classify requirements volatility types and define metrics to
     measure volatility
   – (tech)Address differences in req vol across market segments


                                                                          6
Requirements Critical Success Factors (continued)

 Technical
   – (tech)Function points necessary but not sufficient—
     miss technical performance and constraints
   – (tech)Capabilities versus requirements
   – (tools/tech)Tools: relate estimation inputs to the
     outputs (for example, inputs req lead to sloc output
     instead of sloc to effort)
   – (tech)Global collaboration—state requirements in a
     way that they can be understood globally and
     implemented locally
   – (tech)Specify the behavior for off-nominal conditions
     and events
   – (tech)Detect the confliction between requirements


                                                             7
Requirements Critical Success Factors (continued)

 Tool
   – (tech/tool)How do we automatically measure
     requirements, requirement volatility, progress, scope
   – (process/tool)Requirements traceability from the
     beginning (who initiated and why, which application,
     which test)
   – (tools)Investigate tools available for requirements
     process (measurement, tracking)
   – (tools/tech)Tools: relate estimation inputs to the
     outputs (for example, inputs req lead to sloc output
     instead of sloc to effort)
   – (process/tools)Need a good way to move back and
     forth between reqs and models


                                                             8
Requirements Critical Success Factors (continued)

 Product
   – (prod)Define requirements in a sufficient enough way
     to cost them, but high level enough to provide
     flexibility (R. Turner comment)




                                                            9
Requirements Critical Success Factors (continued)

 Process
   – (process)Capture requirement up front using elicitation in a way
     that avoids future change
   – (process)Plan for requirements to change
     (evolutionary/incremental development)
   – (process)Late binding of requirements-need to specify in a way
     to promote late binding
   – (process)Involve all key stakeholders and ensure their
     accountability for the requirements
   – (people/process)Treat requirements process as a negotiation
     and learning process
   – (management/process)Minimize requirements volatility by
     managing decision volatility
   – (management/process)Link decisions and requirements
   – (process)Need to identify assumptions as well as requirements



                                                                        10
Requirements Critical Success Factors (continued)

 Process
   – (process)Be clear what is an assumption and what is a
     requirement
   – (people/process)Treat requirements process as a negotiation
     and learning process
   – (process)Standard on requirements volatility-how to measure,
     what to measure
   – (process/methodology)Look at process used to development
     requirements, especially across enterprises
        Modeling and sim
        Environment (process, tools, people)
        Instrumentation
        Increase propensity to accept changes in the requirements
         process
   – (process)Steal from agile, tests are one of the better forms for
     capturing requirements

                                                                        11
Requirements Critical Success Factors (continued)

 Process
   – (process/management)Get immediate feedback from
     development and test teams that requirements can be
     implemented and tested
   – (process/management)Write requirements in
     alignment/context with business drivers
   – (process/tool)Requirements traceability from the
     beginning (who initiated and why, which application,
     which test)
   – (process/management)Keeping the multiple
     stakeholders engaged across the life cycle
   – (process/management)Integrate measurement and
     modeling of requirements changes


                                                            12
Requirements Critical Success Factors (continued)

 Process
   – (management/process)Improve/investigate
     methodology for requirements development by
     incorporation of meta models, semantics, and
     ontology
   – (process)Using value based systems engineering we
     need to identify the 10-20 key driving requirements for
     large scale systems
   – (people/process)Collaboration with customer to
     prevent later volatility (using analog of collaborative
     design in the requirements space)




                                                               13
Requirements Critical Success Factors (continued)

 Process
   – (process)Moving from requirements to models too
     early leads to premature codification of solution(s)
   – (process/tools)Need a good way to move back and
     forth between reqs and models
   – (process)25% req change results in 100% change in
     the solution space
   – (process)Requirements completeness for context at
     various points in time




                                                            14
Requirements Critical Success Factors (continued)

 People
   – (tech and people)Same as above with greater
     emphasis on change
   – (people) Committed, representative, available,
     collaborative, knowledeable (CRACK) stakeholders
   – (people/process)Treat requirements process as a
     negotiation and learning process
   – (people/process)Treat requirements process as a
     negotiation and learning process
   – (people)Make stakeholder incentives explicit
   – (people)Function point modeling to provide another
     way to look at requirements



                                                          15
Requirements Critical Success Factors (continued)

 People
   – (people)Data modeling (part of FP modeling) to
     analyze requirements
   – (people/management)Change management and
     expectation management
   – (people/org)How do we convince customers to think in
     smaller steps
   – (people/org)Customers tend to think 5-10 yrs in the
     future, others tend to think 9 months ahead
   – (people/process)Collaboration with customer to
     prevent later volatility (using analog of collaborative
     design in the requirements space)
   – (people)Experience of team


                                                               16
 People
   – (people)Diverse representation in the systems engineering
     organization: role, function, people, cultures
   – (people/org)Managing stakeholder continuity/volatility/high
     variance of req interpretation
   – (people)Ensure that you understand your customer (familiar with
     customer)
   – (people)Impact of cognitive makeup of req candidates using
     standard psych tests (Toyota evaluates critical thinking, problem
     solving, decision making—40 hours of testing prior to
     interviewing)
   – (people)UML never represent customer space, they could, but
     they don’t—the first one you see is taken as a design model (any
     time you make an abstract model, it is assumed by others that it
     is how you plan to design the solution)—how do you convince
     people that there are abstract models that are useful to define the
     problem without defining the solution and to treat them as such


                                                                           17
Requirements Critical Success Factors (continued)

 People
   – (people)People involved in eliciting and analyzing
     requirements
       Well versed in domain and the trade space for
        solutions leads to more effectiveness in eliciting
        and analyzing requirements
   – (people)Having domain knowledge and experience in
     extracting requirements leads to




                                                             18
Requirements Critical Success Factors (continued)

 Management
  – (management/process)Minimize requirements volatility
    by managing decision volatility
  – (management/process)Link decisions and
    requirements
  – (management)Use models to understand effect of
    requirements change on the process
  – (people/management)Change management and
    expectation management
  – (process/management)Get immediate feedback from
    development and test teams that requirements can be
    implemented and tested



                                                           19
Requirements Critical Success Factors (continued)

 Management
  – (process/management)Write requirements in
    alignment/context with business drivers
  – (process/management)Keeping the multiple
    stakeholders engaged across the life cycle
  – (process/management)Integrate measurement and
    modeling of requirements changes
  – (management/process)Improve/investigate
    methodology for requirements development by
    incorporation of meta models, semantics, and
    ontology




                                                    20
Requirements Critical Success Factors (continued)

 Methodology
  – (process/methodology)Look at process used to
    development requirements, especially across
    enterprises
      Modeling and sim
      Environment (process, tools, people)
      Instrumentation
      Increase propensity to accept changes in the
       requirements process




                                                      21
Requirements Critical Success Factors (continued)

 Organizational
   – (people/org)How do we convince customers to think in
     smaller steps
   – (people/org)Customers tend to think 5-10 yrs in the
     future, others tend to think 9 months ahead
   – (people/org)Managing stakeholder
     continuity/volatility/high variance of req interpretation




                                                                 22
Requirements Critical Success Factors (continued)

 (citation)Software requirements: the Requirements Set
  (Ferdinandi) (about categories of requirements)




                                                          23
Research Projects
Vc

 V: 1,7,2 D:2,7,2 Establish relationships between capabilities and
  requirements 3
 V: 5,6,2 D: 0,4,5 How to support client/customer in requirements
  elicitation 4
   – for naïve/non-expert clients
   – when the elicitor does not have domain knowledge
 V: 2,5,5 D: 0,10,0 Define and evaluation a taxonomy of knowledge
  needed in req elicitation 2
 A) V: 11,3,0 D: 0,8,6 Develop improved/standardized methods for
  measuring and evaluating reqs/reqs changes and evaluate the
  benefits on experimental projects 7, 22/14=1.57
 V: 7,7,0 D: 1,8,4 Investigate use of boundary objects for modeling
  requirements in collaborative teams 5
 V: 5,6,4 D: 0,5,9 Build tools to visualize and investigate the
  relationships between decisions and requirements, with emphasis on
  managing changes that are resulting from decision change 5


                                                                       24
Research Projects (continued)

 B)Define set of rules/metrics that can accurately
  predict/reflect 13 ; D: 0,4,13 21/17=1.24
   – requirements quality
   – decision quality
   – Req traceability with code and design (bi-directional)
   – Requirements completion
   – Requirements omissions
 Develop checklists/tools help one identify missing reqs 4
 Translate English requirements set into a virtual
  representation 4
 C)Develop knowledge management tool to categorize
  requirements and based on this, develop cost
  estimate/risk predictions 6; D: 0,4,11 19/15=1.27
 Make function point models into a useful form of
  boundary objects 1


                                                              25
Research Projects (continued)

 D) Define a representation and accompanying new tools
  for reqs that enables automated measurement and
  analysis 6; D: 0,3,11 17/14=1.21
 Investigate team org and roles that are effective at reqs
  elicitation, analysis, pruning, and prioritization 5
 E) Extend USC COCOMO research to relate reqs as
  inputs to SLOC as output and verify on real projects 6;
  D:0,7,8 22/15=1.47
 Investigate levels of requirements and develop factors to
  estimate number of reqs at various levels (requirements
  normalization to get consistent count of reqs at a given
  level) 4
 Investigate the effects of communicating reqs across
  cultures, especially globally 3


                                                              26
Research Projects (continued)

 Small to medium projects: does a very structured process have an
  impact on 5
   – Req quality and req volatility
   – quality and cost of end product
 Define and develop helper for ensuring that behavior for undesirable
  events and off-nominal conditions are captured in the reqs 1
 Tool to identify potential emergent behaviors resulting from
  requirements 3
 F) Develop an approach for costing by capabilities instead of
  requirements 7; D:0,6,8 20/14=1.42
   – Want to build a system ―like that one‖ – reqs def by analogy
 Investigate feasibility of function points as an alternative to
  COSYSMO reqs input 2
 Define/develop Tools to identify ―similar requirements‖ to enable
  detect redundancy, conflicts, and opportunities for reuse 5



                                                                         27
Research Projects (continued)

 Compare contribution of people vs. process to the quality
  of the developed requirements 3
 Define and develop a way to measure the quality of
  developed requirements 3
 G) Develop meta model, semantics (categorization of
  terms, naming conventions, nomenclature) and
  ontologies of requirements 7; D:0,8,6 22/14=1.57
 Investigate the efficiencies of various requirements
  derivation techniques for deriving high quality/complete
  requirements 3
 H) Define the minimum required steps for lean
  requirements-related processes 7; D:4,9,2 32/15=2.13
   – Small projects
   – Large projects

                                                              28
Research Projects (continued)
 Develop tool to capture requirements associated with
  selected/reused/COTS components 1
 Identify preferred methodology for incorporation of legacy
  requirements into new projects 2
 Investigate the extent to which we can simulate the user’s problem
  space (virtual model)—executable model 3
 I) Define a mechanism for capturing reqs using test procedures (test-
  centric requirements development) 6; D: 0,12,1 25/13=1.92
 Define more specific completeness criteria for passing LCO/LCA
  milestones 4
    – Investigate continuous milestones with continuous/tightened
      criteria
 Identify and do good write-ups of case studies on outstanding
  requirements and processes in industry. 5
 Identify best practices for reqs development 4
 Identify preferred approach for seamless incorporation of se and sw
  eng life cycles, with emphasis on software life cycle within se life
  cycles 3


                                                                          29
Research Projects (continued)

 Define and develop an incremental sys eng life cycle
  model 3
 J) Develop a model to predict reqs volatility 14; D: 0,3,13
  19/16=1.19
   – Based on a given set of reqs, problem domain, and
     stakeholders
   – With emphasis on the velocity with which reqs were
     developed, changes made, defects injected and
     defected
 Capture the types and provide predictors for latent req
  defects 1




                                                                30
Top 10 Research Project

 Develop a model to predict reqs volatility 14
   – Based on a given set of reqs, problem domain, and stakeholders
   – With emphasis on the velocity with which reqs were developed,
      changes made, defects injected and defected
 Define set of rules/metrics that can accurately predict/reflect 13
   – requirements quality
   – decision quality
   – Req traceability with code and design (bi-directional)
   – Requirements completion
   – Requirements omissions
 V: 11,3,0 D: 0,8,6 Develop improved/standardized methods for
  measuring and evaluating reqs/reqs changes and evaluate the
  benefits on experimental projects 7
 Develop knowledge management tool to categorize requirements
  and based on this, develop cost estimate/risk predictions 6



                                                                       31
Best Practices:

 What are the current best practices for addressing risk
  related to requirements volatility
   – Postponement of requirements
   – Iterative requirements development
   – Clarity of requirements
   – Prioritize requirements
   – Prioritize by feasibility
   – More agile process
   – Capture justification and history of requirements
   – Architect product to withstand change
   – Better standards for requirements documentation
   – Agile prototyping
   – Involve many stakeholders, especially downstream
     customers
   – Hierarchical classifications (Layered—577 MBASE)

                                                            32
Best Practices (continued)

   Prioritize early and at high levels on the requirements hierarchy
   Change propagation and traceability
   Requirements traceability
   Over-engineer (just in case) and create a resilient architecture that
    anticipates change
   Visible and understandable reqs
   Measure requirements and volatility
   Assess and communicate impact to all stakeholders
   Training
     – Processes
     – Domain
   Checklists
   Make explicit win conditions and incentives of stakeholders
   Triage requirements—if some reqs change, it does not matter, if
    others change, it matters a lot


                                                                            33
Best Practices

   ROI analysis for alternative requirements
   Use tools
   Rely on standards to a larger extent
   Conduct sensitivity analysis up front
   Categorize requirements, capabilities, project,
    performance, interface, evolution, future




                                                      34
Ease / Value


Project Letter                                                                           Difficulty                   EASE          Value:
  Research Project desc.   main Key Words, Phrases and Concept                       3           2       1                          TopTen


  (starts with)                                                                    Easy Medium        HardWeighted Normalized Votes
 aDevelop improved ...     Measure & evaluate; reqts/changes; assess benefits        0           8       6       22          1.57            7
 bDefine set of …          Rules/Metrics related to reqts. Qualities                 0           4      13       21          1.24        13
 cDevelop KM tool …        Categorize reqts.; cost est.; risk prediction             0           4      11       19          1.27            6
 dDefine a representation … Tools; Automated measurement & analysis                  0           3      11       17          1.21            6
 eExtend USC COCOMO … Reqts to SLOC                                                  0           7       8       22          1.47            6
 fDevelop an approach …    Costing by capabilities; reqts. definition by analogy     0           6       8       20          1.43            7
 gDevelop meta-model, …    Meta-models, semantics & ontologies                       0           8       6       22          1.57            7
 hDefine the minimum …     Steps for lean requirements processes                     4           9       2       32          2.13            7
 iDefine a mechanism …     Test-centric reqts. development                           0          12       1       25          1.92            6
 jDevelop a model …        Predict reqts. volatility                                 0           3      13       19          1.19        14




                                                                                                                                                 35
Ease / Value Chart


                         16
                                     j
                         14
                                         b
                         12
   Value (# in top 10)




                         10

                          8
                                                    f           a                              h
                                                                g                   i
                          6
                                    d    c          e
                          4

                          2

                          0
                           1.00   1.20       1.40             1.60           1.80       2.00       2.20
                                                        Ease (1=Difficult)




                                                                                                          36

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:21
posted:6/4/2010
language:English
pages:36