COCOMO SCORM Interactive Courseware Project Cost Modeling by tii14306

VIEWS: 111 PAGES: 18

									          COCOMO SCORM
Interactive Courseware Project Cost
              Modeling

      Roger Smith & Lacey Edwards
              SPARTA Inc.
            Orlando, Florida
     rdsmith@modelbenders.com and
       Lacey.Edwards@sparta.com




                                      1
                     COCOMO-SCORM Project: Proposed
                               Solution
Proposed Technical Concept:
“Every complex project requires project managers and sponsors to calculate the
expected level of effort, duration, and cost of the project. Developing web-based
content to the Sharable Content Object Reference Model (SCORM) is complex
enough that an algorithm is necessary to assist both the government and developers
in estimating the size of the project. SPARTA and Anteon propose to create an
interactive courseware estimation tool based on widely used Instructional Systems
Design (ISD) methodologies and on the principles contained in the well-established
and widely accepted COCOMO II model for software project estimation.”



                                                                 Development
       Decision Points                Interactive
                                                                   and Test
                                     Courseware                      Cost
                                    Estimation Tool               Estimates



                                                                                 2
                        COCOMO Model Family
 Software Cost Models                      DBA COCOMO
                                                                        Other Independent
                                               2004                     Estimation Models
 COCOMO 81           COCOMO II                                     COCOTS              COSYSMO
    1981               2000                COINCOMO                 2000                 2002
                                              2004
                                                                  COSoSIMO          Costing Secure
                                                                    2004             System 2004




                     iDAVE            COPLIMO              COPSEMO                 Security
 COQUALMO
                      2003              2003                 1998               Extension 2004
   1998


                           COSCOMO                COPROMO              CORADMO
                             2006                   1998                 1999
Software Extensions

       Legend:
       Model has been calibrated with historical project data and expert (Delphi) data
       Model is derived from COCOMO II
       Model has been calibrated with expert (Delphi) data
            Dates indicate the time that the first paper was published for the model
               COCOMO II Tool Example



  User’s
Perspective

Quantitative




Qualitative




                                        5
                   COCOMO II Modeling Methodology



Analyze existing
literature

Step 1    Perform Behavioral
          analyses

          Step 2    Define relative
                    significance, data,
                    ratings
                    Step 3     Perform expert-
                               judgment Delphi
                               assessment, formulate
                               a priori model
                               Step 4     Gather project data
                                         Step 5     Determine Bayesian
                                                    A-Posteriori model
                                                    Step 6
                                                                Gather more data;
                                                                refine model

                                                                Step 7
                                                                                    6
                            ADDIE Process




Job/Task
            Design Plan        Storyboards   Trials        Reliability
Analysis

Needs       Instructional      Programming   Pilot tests   Validation
Analysis    Media Design
            Report             Multimedia                  Training Plan
Learning
Analysis    SCORM Plan         Testing
Situation
Analysis

Technical
Analysis
                                                                           7
                         COSCOMO Drivers
                         (27 Input Variables)
•   Size                                 •   Platform EM
      – Source Lines of Code (SLOC)            – Platform Volatility (PVOL)
      – Design Modification (DM)               – Bandwidth (BAND)
      – Code Modification (CM)           •   Personnel EM
      – Integration (IM)                       – Senior Capability (SCAP)
      – Assessment (AA)                        – Developer Capability (DCAP)
      – Understanding (SU)                     – Personnel Continuity (PCON)
      – Unfamiliarity (UNFAM)                  – Applications Experience (APEX)
•   Scale Drivers                              – Platform Experience (PLEX)
      – Development Flexibility (FLEX)         – Development Tools Experience
      – Process Maturity (PMAT)                  (DTEX)
      – Precedentedness (PREC)           •   Project EM
      – Arch/Risk Resolution (RESL)            – Lifecycle Tools (LIFE)
      – Team Cohesion (TEAM)                   – Multisite Development (SITE)
•   Product Effort Multipliers (EM)            – Required Development Schedule
      – Required Reliability (RELY)              (SCED)
      – Product Complexity (CPLX)
      – Required Reuse (RUSE)
      – Documentation (DOCU)

                                                                                  8
                      Quantitative Size Variables

How many hours of courseware will be in the final product?
Total Hours of
Courseware

How are the courseware hours above divided across the different levels of instruction?
                  Level of Instruction
                  L1                 L2                L3                    L4
Hours of
Courseware
           Weights                 1.0                1.3                1.8                2.5


When considering content, media, and code, what percentage of the final product will be …
                 Brand new?
                 Reused after some modification?
                 Reused without modification?




                                                                                                  9
                               EM Effect on Project


                             15 Effort Multipliers Set to Nominal

         RELY CPLX RUSE DOCU PVOL SCAP DCAP PCON APEX PLEX DTEX LIFE   SITE SCED BAND
               H    H                                                   H         H
          H              H     H    L    L    L    L    L    L    L          H


Course                                                                                  Project
Hours               L          L
                                                                                         Cost
          L    L         L          H    H    H    H    H    H    H     L    L    L




                                                                                        Extra High H
                                                                                        Very High
                                                                                            High
                                                                                         Nominal       1.0
                                                                                             Low
                                                                                        Very Low L
                                                                                                             10
                     Qualitative Project Variables

                                                                            Extra
                           Very Low Low         Nominal High      Very High High
              APEX              1.22      1.1           1    0.88      0.81 N/A
              BAND         N/A       N/A                1       1         1          1
              CPLX              0.73     0.87           1    1.17      1.34       1.74
              DCAP              1.34     1.15           1    0.88      0.76 N/A
              DOCU              0.81     0.91           1    1.11      1.23 N/A
              DTEX               1.2     1.09           1    0.91      0.84 N/A
  Effort      LIFE              1.17     1.09           1     0.9      0.78 N/A
Multipliers   PCON              1.29     1.12           1     0.9      0.81 N/A
              PLEX              1.19     1.09           1    0.91      0.85 N/A
              PVOL         N/A           0.87           1    1.15       1.3 N/A
              RELY              0.82     0.92           1     1.1      1.26 N/A
              RUSE         N/A           0.95           1    1.07      1.15       1.24
              SCAP              1.42     1.19           1    0.85      0.71 N/A
              SCED              1.43     1.14           1       1         1 N/A
              SITE              1.22     1.09           1    0.93      0.86        0.8
              FLEX              5.07     4.05        3.04    2.03      1.01          0
  Scale       PMAT               7.8     6.24        4.68    3.12      1.56          0
 Drivers      PREC               6.2     4.96        3.72    2.48      1.24          0
              RESL              7.07     5.65        4.24    2.83      1.41          0
              TEAM              5.48     4.38        3.29    2.19       1.1          0

 Nominal/Standard Levels                                                                 11
                                     EM Effect on Project
                            15 Effort Multipliers Other Than Nominal
                                                                                                     BAND
                                                                                                SCED  H

                                                                                         SITE
                                                        PCON
                                                                                          H      H
                                                 DCAP          APEX               TOOL
                                                         L            PLEX LTEX
                                                  L             L                  L
                                                                                                            Est.
                                          ACAP
                                   PVOL                                L    L                         L
                                                                                                            Cost
                                           L                                                     L
                                    H                                                     L
                            DOCU                         H
                                                  H             H                  H
                             H
                     RUSE                                              H    H
                      H             L      H

         RELY CPLX
               H
          H                  L
                      L
Course                                                                                                      Nom.
Hours                                                                                                       Cost
          L    L




                                                                                                                   12
                         Data Collection Form
                                COURSE AND CONTACT IDENTIFICATION


      Name of
      Course:

      Organization:
      Person
      Surveyed:

      Contact
      Information:



                                                   SIZE AND REUSE


      What is the level of SCORM-conformance of this course?
                          none                           1.0                  1.2              2004
Q1    SCORM

      How many person-months of effort went into building the course? (Usually calculated as Duration * # People * Average % Effort from Each person.)
                      Person-Months People Assigned Duration Months Hours/Month
      Person-Months
Q2    of Effort

      How many hours of courseware will be in the final product?
      Total Hours of
Q3    Courseware

      How are the courseware hours above divided across the different levels of instruction?
                        Level of Instruction
                        L1                 L2                L3                    L4
      Hours of
Q4    Courseware
      Weights

Q5    When considering content, media, and code, what percentage of the final product will be …
                       Brand new?
                       Reused after some modification?
                       Reused without modification?
                                                                           0%



 (see Excel file for entire form…)                                                                                                                       13
                                            COSCOMO Estimates

           Course                   Person-                   Person-    Time to    Time to
           Name                     Months                    Months     Develop    Develop
                                    Reported                  Estimate   Reported   Estimate
           Defense                  78                        75         13         17.5
           Nuclear
           Weapons
           School
           AF                       24.3                      105.3      7.0        19.1
           Accounting
           Liaison
           Course
           Explosive   6.0                                    4.8        2.0        7.7
           Hazard
           Awareness *
           Next …


* Survey was not completed. All remaining variables were estimated.                            14
                Defense Nuclear Weapons School

                                         Cumulative Effect of Effort Multipliers


                        2.00

Very Low                1.80
Low
                        1.60
Nominal
High                    1.40
Very High
                        1.20
Extra High
                                                  1.07   1.07   1.07
Cumulative Multiplier   1.00      1.00     1.00
                                                                       0.93

                        0.80                                                  0.79
                                                                                     0.70
                        0.60
                                                                                            0.56
                                                                                                   0.46
                        0.40                                                                              0.42
                                                                                                                 0.38      0.38   0.38   0.38


                        0.20


                        0.00
                               RELY   CPLX    RUSE   DOCU   BAND   PVOL   SCAP   DCAP   PCON   APEX   PLEX   DTEX       LIFE   SITE   SCED




                                                                                                                                             15
Building PRED(30) for COSCOMO

          Total Project Cost (PM)
               DNWS
               -3.8%
                       78




                                      ALC
                                     +333%

                       24
       EHA
       -20%
                       6


   -30%           0%          +30%

   % Variance of Estimated Value
                                             16
               Reliability = PRED(30)


• Reliability of COCOMO family of models is often
  measured by the percentage of test cases that it will
  estimate within 30% of the actual project costs
   – e.g. If a project requires 300 person-months to
     complete, then its PRED(30) range would be (210 to
     390)
   – If the model estimates 70% of its test cases within this
     range then the model’s PRED(30) = 70%
• COCOMO Family Model Levels
   – COCOMO II: PRED(30) = 64%
   – COSYSMO: PRED(30) = 56%
   – COSCOMO: PRED(30) = unknown at this time



                                                                17
           Delphi Calibration Survey


• Using Delphi Method to gather independent opinions
  about the level of effort associated with each setting on
  each variable.
   – Collect individual responses
   – Calculate group average
   – Present averages to each person and allow them to
     change their responses if they would like
   – Convene group work session to settle on acceptable
     values
• Round 1 of Delphi Conducted Via Private Survey
• Round 2 Conducted in Seminar Form




                                                              18
                         Conclusion

• COCOMO II has been modified to include variables that are
  important to SCORM courseware developers
• Variable weights are being adjusted by Delphi surveys of experts
• Model is being calibrated through the collection of historical
  project information

• Current results show that this approach is feasible
   – It is to early to determine the degree of success that will be
     achieved

• Historical project data collection is essential, but it is also very
  difficult to get access and cooperation from the people with this
  information
    – Have currently collected data on 7 projects
    – Targeting 40+ projects to calibrate the model appropriately




                                                                         19

								
To top