Optimizing IV&V in Mature Organizations
Dan McCaugherty, Frank Gmeindl, ProLogic, Inc. Christopher Fuhrman, ETS
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29
Summary Slide
• Objective • Related work • Research approach • Success Measures • Major Accomplishments to Date (1/2) • Open issues and concerns
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29 2
Objective (1)
• What optimization strategies can be taken by IV&V planners, knowing the appraised level of maturity of the development organization?
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 3
Objective (2)
• What relationship exists (if any) between the level of effort spent on “managerial” IV&V activities and the assessed level of maturity of the development organization?
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 4
Objective (3)
• Provide a set of guidelines to help IV&V planners tailor activities to a development organization’s mature processes
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 5
Related work (1)
Ratio of IV&V effort non-technical (managerial) vs technical issues [1] 3-4% 6-8% 10-15% IV&V Unable to <25% IV&V estimate nonIV&V technical IV&V IV&V Level 5
somewhat unpredictable
Level 1
[1]
Level 2
Level 3
Level 4
R. O. Lewis, Independent verification and validation: a life cycle engineering process for quality software. New York: Wiley, 1992.
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 6
Related work (2)
• IV&V Cost Estimation, NAVAIR
– Need for similar data (IV&V Task Details, Man-hours, Scalars)
• Baseline Cost Model, ProLogic
– Re-use of WBS, planning strategies
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 7
Research approach (1)
• Analyze the CMMI Level 2 and 3 “typical work products” from IV&V perspective (FY03)
• Identify evaluation criteria for determining their usefulness to IV&V
• Analyze the work products according to the criteria
2003-07-29 8
Technical Interchange and Initiative Review, NASA IV&V Facility
Research approach (2)
• Examine actual project planning data for one or more IV&V projects done with mature developers (FY04)
– See if or how the CMM/CMMI work products were used in the actual IV&V activities (prove useful relationship) – Calculate ratio of effort spent on “managerial” vs technical IV&V activities
• Provide guidebook for IV&V planners (FY04)
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29 9
Success Measures
• Reduced level of effort spent on non-technical IV&V activities • Ability to tailor IV&V activities in relationship to a developer’s mature processes based on an appraised maturity level • Ability to avoid unnecessarily redundant V&V activities between internal V&V and independent V&V
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29 10
Schedule of Deliverables
Deliverable Due Date (FY) Quarter
Draft Report: Analysis of CMMI PAs from IV&V perspective Final Report: Analysis of CMMI PAs from IV&V perspective Define approach and selection of project for pilot study
03 04 04
Q2 Q1 Q1
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 11
Major Accomplishments to Date (1/2)
• Adaptation of research initiative for the CMMI rather than the CMM • Draft analysis of CMMI PAs and work products in the context of IEEE 1012
– Criteria for analysis of artifacts:
• Useful for IV&V planning? • Useful/necessary in IV&V technical tasks? • Possibility of compromising technical independence?
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29 12
Major Accomplishments to Date (2/2)
• Two articles submitted to NASA Software Engineering Workshop (SEW)
– Optimizing IV&V in Mature Organizations
– Software V&V and the RUP
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 13
Prioritized list of CMMI PAs
Criteria for IV&V activities H=high M=medium L=low Process areas (CMMI-SW Level 2 and 3) 1. Project Planning (PP) 4. Requirements Management (REQM) 5. Configuration Management (CM) 7. Measurement and Analysis (MA) 8. Organizational Process Definition (OPD) 11. Integrated Project Management (IPM) 12. Risk Management (RSKM) 13. Requirements Development (RD) 14. Technical Solution (TS) 15. Product Integration (PI) Planning Execution Support Conflict
H M/H
M M
L
L M L L L L L
L/M L/M L M N/A L
M/H H
L L L M
H
M M
H
L/M L L
H
M M
H H M/H
H H M/H
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 14
Open issues and concerns
• Availability of IV&V project planning data for an IV&V project with a mature developer (i.e., one that has been assessed/ appraised at a minimum of level 2 in CMM/CMMI)
– Typical IV&V metrics are related to defects, etc.
• Existence of appraised developers (yet to be identified)
– GSFC in-house projects, e.g., Global Precipitation Measurement, might “count” as a mature developer if they have had a CMM/CMMI appraisal
Technical Interchange and Initiative Review, NASA IV&V Facility 2003-07-29 15
Questions?
Technical Interchange and Initiative Review, NASA IV&V Facility
2003-07-29 16