Get documents about "COCOMO SCORM Interactive Courseware Project Cost Modeling
Document Sample
COCOMO SCORM
Interactive Courseware Project Cost
Modeling
Roger Smith & Lacey Edwards
SPARTA Inc.
Orlando, Florida
rdsmith@modelbenders.com and
Lacey.Edwards@sparta.com
1
COCOMO-SCORM Project: Proposed
Solution
Proposed Technical Concept:
“Every complex project requires project managers and sponsors to calculate the
expected level of effort, duration, and cost of the project. Developing web-based
content to the Sharable Content Object Reference Model (SCORM) is complex
enough that an algorithm is necessary to assist both the government and developers
in estimating the size of the project. SPARTA and Anteon propose to create an
interactive courseware estimation tool based on widely used Instructional Systems
Design (ISD) methodologies and on the principles contained in the well-established
and widely accepted COCOMO II model for software project estimation.”
Development
Decision Points Interactive
and Test
Courseware Cost
Estimation Tool Estimates
2
COCOMO Model Family
Software Cost Models DBA COCOMO
Other Independent
2004 Estimation Models
COCOMO 81 COCOMO II COCOTS COSYSMO
1981 2000 COINCOMO 2000 2002
2004
COSoSIMO Costing Secure
2004 System 2004
iDAVE COPLIMO COPSEMO Security
COQUALMO
2003 2003 1998 Extension 2004
1998
COSCOMO COPROMO CORADMO
2006 1998 1999
Software Extensions
Legend:
Model has been calibrated with historical project data and expert (Delphi) data
Model is derived from COCOMO II
Model has been calibrated with expert (Delphi) data
Dates indicate the time that the first paper was published for the model
COCOMO II Tool Example
User’s
Perspective
Quantitative
Qualitative
5
COCOMO II Modeling Methodology
Analyze existing
literature
Step 1 Perform Behavioral
analyses
Step 2 Define relative
significance, data,
ratings
Step 3 Perform expert-
judgment Delphi
assessment, formulate
a priori model
Step 4 Gather project data
Step 5 Determine Bayesian
A-Posteriori model
Step 6
Gather more data;
refine model
Step 7
6
ADDIE Process
Job/Task
Design Plan Storyboards Trials Reliability
Analysis
Needs Instructional Programming Pilot tests Validation
Analysis Media Design
Report Multimedia Training Plan
Learning
Analysis SCORM Plan Testing
Situation
Analysis
Technical
Analysis
7
COSCOMO Drivers
(27 Input Variables)
• Size • Platform EM
– Source Lines of Code (SLOC) – Platform Volatility (PVOL)
– Design Modification (DM) – Bandwidth (BAND)
– Code Modification (CM) • Personnel EM
– Integration (IM) – Senior Capability (SCAP)
– Assessment (AA) – Developer Capability (DCAP)
– Understanding (SU) – Personnel Continuity (PCON)
– Unfamiliarity (UNFAM) – Applications Experience (APEX)
• Scale Drivers – Platform Experience (PLEX)
– Development Flexibility (FLEX) – Development Tools Experience
– Process Maturity (PMAT) (DTEX)
– Precedentedness (PREC) • Project EM
– Arch/Risk Resolution (RESL) – Lifecycle Tools (LIFE)
– Team Cohesion (TEAM) – Multisite Development (SITE)
• Product Effort Multipliers (EM) – Required Development Schedule
– Required Reliability (RELY) (SCED)
– Product Complexity (CPLX)
– Required Reuse (RUSE)
– Documentation (DOCU)
8
Quantitative Size Variables
How many hours of courseware will be in the final product?
Total Hours of
Courseware
How are the courseware hours above divided across the different levels of instruction?
Level of Instruction
L1 L2 L3 L4
Hours of
Courseware
Weights 1.0 1.3 1.8 2.5
When considering content, media, and code, what percentage of the final product will be …
Brand new?
Reused after some modification?
Reused without modification?
9
EM Effect on Project
15 Effort Multipliers Set to Nominal
RELY CPLX RUSE DOCU PVOL SCAP DCAP PCON APEX PLEX DTEX LIFE SITE SCED BAND
H H H H
H H H L L L L L L L H
Course Project
Hours L L
Cost
L L L H H H H H H H L L L
Extra High H
Very High
High
Nominal 1.0
Low
Very Low L
10
Qualitative Project Variables
Extra
Very Low Low Nominal High Very High High
APEX 1.22 1.1 1 0.88 0.81 N/A
BAND N/A N/A 1 1 1 1
CPLX 0.73 0.87 1 1.17 1.34 1.74
DCAP 1.34 1.15 1 0.88 0.76 N/A
DOCU 0.81 0.91 1 1.11 1.23 N/A
DTEX 1.2 1.09 1 0.91 0.84 N/A
Effort LIFE 1.17 1.09 1 0.9 0.78 N/A
Multipliers PCON 1.29 1.12 1 0.9 0.81 N/A
PLEX 1.19 1.09 1 0.91 0.85 N/A
PVOL N/A 0.87 1 1.15 1.3 N/A
RELY 0.82 0.92 1 1.1 1.26 N/A
RUSE N/A 0.95 1 1.07 1.15 1.24
SCAP 1.42 1.19 1 0.85 0.71 N/A
SCED 1.43 1.14 1 1 1 N/A
SITE 1.22 1.09 1 0.93 0.86 0.8
FLEX 5.07 4.05 3.04 2.03 1.01 0
Scale PMAT 7.8 6.24 4.68 3.12 1.56 0
Drivers PREC 6.2 4.96 3.72 2.48 1.24 0
RESL 7.07 5.65 4.24 2.83 1.41 0
TEAM 5.48 4.38 3.29 2.19 1.1 0
Nominal/Standard Levels 11
EM Effect on Project
15 Effort Multipliers Other Than Nominal
BAND
SCED H
SITE
PCON
H H
DCAP APEX TOOL
L PLEX LTEX
L L L
Est.
ACAP
PVOL L L L
Cost
L L
H L
DOCU H
H H H
H
RUSE H H
H L H
RELY CPLX
H
H L
L
Course Nom.
Hours Cost
L L
12
Data Collection Form
COURSE AND CONTACT IDENTIFICATION
Name of
Course:
Organization:
Person
Surveyed:
Contact
Information:
SIZE AND REUSE
What is the level of SCORM-conformance of this course?
none 1.0 1.2 2004
Q1 SCORM
How many person-months of effort went into building the course? (Usually calculated as Duration * # People * Average % Effort from Each person.)
Person-Months People Assigned Duration Months Hours/Month
Person-Months
Q2 of Effort
How many hours of courseware will be in the final product?
Total Hours of
Q3 Courseware
How are the courseware hours above divided across the different levels of instruction?
Level of Instruction
L1 L2 L3 L4
Hours of
Q4 Courseware
Weights
Q5 When considering content, media, and code, what percentage of the final product will be …
Brand new?
Reused after some modification?
Reused without modification?
0%
(see Excel file for entire form…) 13
COSCOMO Estimates
Course Person- Person- Time to Time to
Name Months Months Develop Develop
Reported Estimate Reported Estimate
Defense 78 75 13 17.5
Nuclear
Weapons
School
AF 24.3 105.3 7.0 19.1
Accounting
Liaison
Course
Explosive 6.0 4.8 2.0 7.7
Hazard
Awareness *
Next …
* Survey was not completed. All remaining variables were estimated. 14
Defense Nuclear Weapons School
Cumulative Effect of Effort Multipliers
2.00
Very Low 1.80
Low
1.60
Nominal
High 1.40
Very High
1.20
Extra High
1.07 1.07 1.07
Cumulative Multiplier 1.00 1.00 1.00
0.93
0.80 0.79
0.70
0.60
0.56
0.46
0.40 0.42
0.38 0.38 0.38 0.38
0.20
0.00
RELY CPLX RUSE DOCU BAND PVOL SCAP DCAP PCON APEX PLEX DTEX LIFE SITE SCED
15
Building PRED(30) for COSCOMO
Total Project Cost (PM)
DNWS
-3.8%
78
ALC
+333%
24
EHA
-20%
6
-30% 0% +30%
% Variance of Estimated Value
16
Reliability = PRED(30)
• Reliability of COCOMO family of models is often
measured by the percentage of test cases that it will
estimate within 30% of the actual project costs
– e.g. If a project requires 300 person-months to
complete, then its PRED(30) range would be (210 to
390)
– If the model estimates 70% of its test cases within this
range then the model’s PRED(30) = 70%
• COCOMO Family Model Levels
– COCOMO II: PRED(30) = 64%
– COSYSMO: PRED(30) = 56%
– COSCOMO: PRED(30) = unknown at this time
17
Delphi Calibration Survey
• Using Delphi Method to gather independent opinions
about the level of effort associated with each setting on
each variable.
– Collect individual responses
– Calculate group average
– Present averages to each person and allow them to
change their responses if they would like
– Convene group work session to settle on acceptable
values
• Round 1 of Delphi Conducted Via Private Survey
• Round 2 Conducted in Seminar Form
18
Conclusion
• COCOMO II has been modified to include variables that are
important to SCORM courseware developers
• Variable weights are being adjusted by Delphi surveys of experts
• Model is being calibrated through the collection of historical
project information
• Current results show that this approach is feasible
– It is to early to determine the degree of success that will be
achieved
• Historical project data collection is essential, but it is also very
difficult to get access and cooperation from the people with this
information
– Have currently collected data on 7 projects
– Targeting 40+ projects to calibrate the model appropriately
19
Related docs
