Joint ADL Co-Lab BAA COCOMO-SCORM Content Complexity

Document Sample
Joint ADL Co-Lab BAA COCOMO-SCORM Content Complexity Powered By Docstoc
					       Joint ADL Co-Lab BAA

         COCOMO-SCORM:
Content Complexity Measurement Tool
        Contract #: N61339-05-C-0115



          Interim Project Review
              09 March 2006



                                       1
         COCOMO-SCORM Project
             IPR Agenda
                March 9, 9:00 – 11:30


• JADL Co-Lab (9:00-9:15)
   – Purpose of Meeting
   – Goals of this program
• Sparta/Anteon (9:15-11:15)
   – Project Management (Roger Smith)
   – Research COCOMO Variables (Lacey Edwards)
   – Research ISD Cost Items (Kelly Ward)
   – Create COCOMO-SCORM Model (Lacey Edwards)
   – Data Collection (Roger Smith)
   – Model Calibration (Roger Smith)
• Questions and Discussion (11:15-11:30)
• Conclusion (Roger Smith)


                                                 2
                    Project: COCOMO-SCORM
BAA Research Interest Statement:
“Web-based and SCORM content is difficult to measure in terms of complexity and
development time; however, most people would agree that there are certain qualities
that make some content more difficult to develop than other content (like level of
interactivity). What is needed is an algorithm that takes into account all of the
decision points an instructional designer and later the software developer makes
when designing and developing SCORM content. Being able to quantify this in a
comparative numerical measure would make development and test cost estimates
more meaningful and accurate; thereby, reducing risk to a content development
program.”
Proposed Technical Concept:
“Every complex project requires project managers and sponsors to calculate the
expected level of effort, duration, and cost of the project. Developing web-based
content to the Sharable Content Object Reference Model (SCORM) is complex
enough that an algorithm is necessary to assist both the government and developers
in estimating the size of the project. SPARTA and Anteon propose to create an
interactive courseware estimation tool based on widely used Instructional Systems
Design (ISD) methodologies and on the principles contained in the well-established
and widely accepted COCOMO II model for software project estimation.”              3
               Project Description


• Create an interactive project estimation tool for
  ISD/SCORM content
• Algorithmic foundation for the tool is the COCOMO II
  algorithm developed for software projects by the
  research team at the University of Southern California
  and led by Barry Boehm
• ISD methodology estimated will be the ADDIE model
  developed at Florida State University and adopted by
  the U.S. Armed Services
• Resulting algorithm and tools will estimate the person-
  months required to create a SCORM conformant
  learning product
    – Dollar costs can be derived outside of the model by
      applying industry or specific company labor rates



                                                            4
                     Problems to Solve


 • Enumerate SCORM products and processes
       that contribute to the staffing and duration
       of a project
 •    Identify mathematic and logical
       relationships between these items
 •    Quantify the level of contribution of each
       item to project cost
   •   Adjust the algorithm constants for SCORM
       courseware
   •   Define a bounded set of conditions under
       which the algorithm can be relied upon
   •   Validate the algorithm for various sets of
       data within the bounded conditions



                                                      5
             Applicability & Value to
                   Community

• Consistent, objective, and reliable estimation tool for
  SCORM content and projects
• First step in formalizing an estimation method in the
  ADL community
• Create a tool that other projects can apply, modify, and
  mature
   – COCOMO II has been evolving for 25 years.
   – COCOMO-SCORM tool from this BAA will be the first
      step in the long evolution and improvement of a tool for
      this community




                                                                 6
                     Project Cost Estimation
                                                  Dollar Costs
    Project
 Requirements




Quantitative Input
                        Qualitative Adjustments
  Reuse                                             Person
     Level of                                      Months to
   Instruction                                      Develop
        Course
         Hours




                                                    Time to
                                                    Develop
                                                                 7
Project Management




                     8
               Team: Personnel


• SPARTA
   – Roger Smith, PM
   – Lacey Edwards
   – Seth Lytle
• Anteon
   – Kelly Ward
   – Denise Stevens
   – Tatjana Pitts
   – Tim Richey




                                 9
                  Team: Collaboration


• Continuous Research Collaboration
     – Daily telephone and email exchange
• Bi-weekly Meetings
     – Face-to-face exchange of ideas and progress bi-weekly
• Monthly Sub-contract Report
     – Monthly written report of progress, status, and future plans
• In Process Review
     – Pre-government meeting preparation and collaboration
• Training 2006 Presentation
     – March 7, 2006
• I/ITSEC 2006 Paper Preparation
     – Abstract Submitted: February 2006
     – Draft Paper Due: June 2006
     – Final Paper Due: October 2006



                                                                      10
                                Schedule: Phase 1
                  Task                      Duration       Start Date     Finish Date

1.0 Algorithm Development
1.1 Project Management - Algorithm       33 weeks       10/10/05        6/9/06

1.2 Project Kick-off Meeting             2 days         10/17/05        10/18/05
1.3 Research COCOMO Variables            10 weeks       10/10/05        12/16/05

1.4 Research ISD Cost Items              10 weeks       10/10/05        12/16/05

1.5 Develop COCOMO-SCORM                 10 weeks       12/19/05        2/24/06
Algorithms
1.6 Prepare and Conduct In-Process       1 week         3/6/06          3/9/06
Review
1.7 Create COCOMO-SCORM Software         11 weeks       3/3/06          5/19/06
Model
1.8 Create Prototype User Interface      10 weeks       3/6/06          5/12/06

1.9 Create Algorithm Documentation       2 weeks        5/22/06         6/2/06

1.10 Prepare and Conduct Final Project   1 week         6/5/06          6/9/06
Review
1.11 Validation Tests                    Interspersed   12/19/05        6/2/06

Total                                    33 weeks       10/10/05        6/9/06

                                                                                        11
                      Deliverables: Phase 1
   Deliverable                         Description                        Delivery Date


DI-MGMT-81117    Technical and Management Work Plan.
                 1) Project Schedule
                                                                     12 October 2006
                                                                                          
DI-MGMT-80227    Contractor’s Progress, Status and Management        Monthly        Progress
                 Report.                                             Reports
                 1) Monthly progress reports (Due: 7th day of each   07 November 2005     
                 month)                                              07 December 2005     
                 2) In-Process Review presentation material          09 January 2006      
                                                                     07 February 2006     
                                                                     07 March 2006        
                                                                     07 April 2006
                                                                     08 May 2006

                                                                     IPR Material
                                                                     01 March 2006        
DI-MCCR-80700    Computer Software Product End Item.                 09 June 2006
                 1) COCOMO-SCORM Application Software
                 2) COCOMO-SCORM User Documentation




DI-MISC-80711A   Scientific and Technical Report.                    09 June 2006
                 1) Project Final Report


                                                                                               12
                          Option: Phase 2


                          Task                      Performance Period


2.0 Professional GUI Develop


2.1 Project Management - GUI                       16 weeks


2.2 Create Professional GUI                        10 weeks

2.3 Interface with COCOMO-SCORM Spreadsheet        2 weeks



2.4 Create GUI Documentation                       2 weeks


2.4 Prepare and Conduct Final Project Review and   1 week
Demonstration

2.5 Create Final Report                            1 week

Total                                              16 weeks



                                                                         13
Research COCOMO Variables




                            14
      COCOMO II Research Approach


• Studied the definitive source on COCOMO II: Software
  Cost Estimation with COCOMO II by Barry Boehm et.al.
   – Model form and variables
   – Methodology used to create the COCOMO II model

• Researched how related models (COSYSMO,
  COPSEMO, etc.) modified COCOMO II for their
  purposes
   – COSYSMO workshop
   – Relationship with Dr. Ricardo Valerdi

• Surveyed software implementations of COCOMO II



                                                         15
                        COCOMO Model Family
 Software Cost Models                      DBA COCOMO
                                                                        Other Independent
                                               2004                     Estimation Models
 COCOMO 81
    1981            COCOMO II
                       2000               COINCOMO
                                                                   COCOTS
                                                                    2000
                                                                                      COSYSMO
                                                                                         2002
                                                                                                 
                                              2004
                                                                  COSoSIMO          Costing Secure
                                                                    2004             System 2004




 COQUALMO
   1998     
                     iDAVE
                      2003
                                      COPLIMO
                                        2003              COPSEMO
                                                             1998                 Security
                                                                                Extension 2004


                             COSCOMO
                               2006
                                                  COPROMO
                                                    1998
                                                                       CORADMO
                                                                         1999          
Software Extensions

       Legend:
       Model has been calibrated with historical project data and expert (Delphi) data
       Model is derived from COCOMO II
       Model has been calibrated with expert (Delphi) data
            Dates indicate the time that the first paper was published for the model
               COCOMO II Tool Example



  User’s
Perspective

Quantitative




Qualitative




                                        17
                        COCOMO II Drivers
                        (29 Input Variables)
•   Size                                •   Personnel EM
      – Source Lines of Code (SLOC)          – Analyst Capability (ACAP)
      – Design Modification (DM)             – Programmer Capability (PCAP)
      – Code Modification (CM)               – Personnel Continuity (PCON)
      – Integration (IM)                     – Applications Experience (APEX)
      – Assessment (AA)                      – Platform Experience (PLEX)
      – Understanding (SU)                   – Language/Toolset Experience
      – Unfamiliarity (UNFAM)                   (LTEX)
      – Requirements Evolution (REVL)   •   Project EM
•   Product Effort Multipliers (EM)          – Use of Software Tools (TOOL)
      – Required Reliability (RELY)          – Multisite Development (SITE)
      – Database Size (DATA)                 – Required Development Schedule
      – Product Complexity (CPLX)               (SCED)
      – Required Reuse (RUSE)           •   Scale Drivers
      – Documentation (DOCU)                 – Development Flexibility (FLEX)
•   Platform EM                              – Process Maturity (PMAT)
      – Execution Time Constraints           – Precedentedness (PREC)
        (TIME)                               – Arch/Risk Resolution (RESL)
      – Main Storage Constraints             – Team Cohesion (TEAM)
        (STORE)
      – Platform Volatility (PVOL)
                                                                                18
           COCOMO Effort Multipliers
    Factor/Multiplier    Very Low    Low     Nominal   High     Very High Extra High
Scale Factor Weights
    1      FLEX               5.07    4.05      3.04     2.03        1.01      0.00
    2      PMAT               7.80    6.24      4.68     3.12        1.56      0.00
    3      PREC               6.20    4.96      3.72     2.48        1.24      0.00
    4      RESL               7.07    5.65      4.24     2.83        1.41      0.00
    5      TEAM               5.48    4.38      3.29     2.19        1.10      0.00
Effort Multiplier Values
    1      RELY               0.82    0.92      1.00     1.10        1.26       N/A
    2      DATA                N/A    0.90      1.00     1.14        1.28       N/A
    3      CPLX               0.73    0.87      1.00     1.17        1.34      1.74
    4      RUSE                N/A    0.95      1.00     1.07        1.15      1.24
    5      DOCU               0.81    0.91      1.00     1.11        1.23       N/A
    6      TIME                N/A     N/A      1.00     1.11        1.29      1.63
    7      STOR                N/A     N/A      1.00     1.05        1.17      1.46
    8      PVOL                N/A    0.87      1.00     1.15        1.30       N/A
    9      ACAP               1.42    1.19      1.00     0.85        0.71       N/A
    10     PCAP               1.34    1.15      1.00     0.88        0.76       N/A
    11     PCON               1.29    1.12      1.00     0.90        0.81       N/A
    12     APEX               1.22    1.10      1.00     0.88        0.81       N/A
    13     PLEX               1.19    1.09      1.00     0.91        0.85       N/A
    14     LTEX               1.20    1.09      1.00     0.91        0.84       N/A
    15     TOOL               1.17    1.09      1.00     0.90        0.78       N/A
    16     SITE               1.22    1.09      1.00     0.93        0.86      0.80
    17     SCED               1.43    1.14      1.00     1.00        1.00       N/A

                                                                                       19
                     COCOMO Project Variation
                          Factor/Multiplier    Very Low    Low     Nominal   High     Very High Extra High
                      Scale Factor Weights
 SF Flex:                 1      FLEX               5.07    4.05      3.04     2.03        1.01      0.00
                          2      PMAT               7.80    6.24      4.68     3.12        1.56      0.00
 22.6% Higher             3      PREC               6.20    4.96      3.72     2.48        1.24      0.00
                          4      RESL               7.07    5.65      4.24     2.83        1.41      0.00
 9% Lower                 5      TEAM               5.48    4.38      3.29     2.19        1.10      0.00
                      Effort Multiplier Values
                          1      RELY               0.82    0.92      1.00     1.10        1.26       N/A
                          2      DATA                N/A    0.90      1.00     1.14        1.28       N/A
                          3      CPLX               0.73    0.87      1.00     1.17        1.34      1.74
                          4      RUSE                N/A    0.95      1.00     1.07        1.15      1.24
                          5      DOCU               0.81    0.91      1.00     1.11        1.23       N/A
                          6      TIME                N/A     N/A      1.00     1.11        1.29      1.63
                          7      STOR                N/A     N/A      1.00     1.05        1.17      1.46
                          8      PVOL                N/A    0.87      1.00     1.15        1.30       N/A
EM Flex From:             9
                          10
                                 ACAP
                                 PCAP
                                                    1.42
                                                    1.34
                                                            1.19
                                                            1.15
                                                                      1.00
                                                                      1.00
                                                                               0.85
                                                                               0.88
                                                                                           0.71
                                                                                           0.76
                                                                                                      N/A
                                                                                                      N/A
115.6 times larger        11     PCON               1.29    1.12      1.00     0.90        0.81       N/A
                          12     APEX               1.22    1.10      1.00     0.88        0.81       N/A
17.6 times smaller        13     PLEX               1.19    1.09      1.00     0.91        0.85       N/A
                          14     LTEX               1.20    1.09      1.00     0.91        0.84       N/A
                          15     TOOL               1.17    1.09      1.00     0.90        0.78       N/A
                          16     SITE               1.22    1.09      1.00     0.93        0.86      0.80
                          17     SCED               1.43    1.14      1.00     1.00        1.00       N/A



                                                                                                             20
                       COCOMO II Model Form

                                                   * P EMi
                                                       17
                     PM =       A*(Size)E
                                                       i=1

                                                             5
                                    E = B + .01*S SFj
                                                         j=1

Where:
   PM = effort in Person Months
   A = calibration constant derived from historical project data
   Size = Adjust Source Lines of Code
   E = represents diseconomy of scale (composed of 5 scale factors)
   EM = effort multiplier for the ith cost driver. The geometric product results in an
   overall effort adjustment factor to the nominal effort.

    B = calibration constant derived from historical project data
    SF = scale factor for the jth cost driver. Provides organization specific
    adjustments to the size of the project.

                                                                                         21
               Reliability = PRED(30)


• Reliability of COCOMO family of models is often
  measured by the percentage of test cases that it will
  estimate within 30% of the actual project costs
   – e.g. If a project requires 300 person-months to
     complete, then its PRED(30) range would be (210 to
     390)
   – If the model estimates 70% of its test cases within this
     range then the model’s PRED(30) = 70%
• COCOMO Family Model Levels
   – COCOMO II (2000): PRED(30) = 69%
   – COSYSMO: PRED(30) = 56%




                                                                22
                           PRED(30)


  COCOMO II                                    COSYSMO



            69%
                                                    56%




-30%   0%         +30%                      -30%   0%     +30%

                   % Variance of Estimated Value


                                                                 23
Research ISD Cost Items




                          24
                            ADDIE Process




Job/Task
Analysis

Needs
            Design Plan        Storyboards
Analysis
                                                           Reliability
            Instructional      Programming   Trials
Learning
            Media Design                                   Validation
Analysis
            Report             Multimedia    Pilot tests
                                                           Training Plan
Situation
            SCORM Plan         Testing
Analysis

Technical
Analysis


                                                                           25
             SCORM Cost Variables


•   Current Estimation Practices


          – Historical Data


              – Top Down


              – Bottom Up


      – Industry Standards

                                    26
             SCORM Cost Variables


• Significant Factors:


          – Types and quantity
            of media
                                  – Levels of
                                    interactivity




                         – Entry point into ADDIE
                           process

                                                    27
                        COCOMO II Drivers
                        (29 Input Variables)
•   Size                                •   Personnel EM
      – Source Lines of Code (SLOC)          – Analyst Capability (ACAP)
      – Design Modification (DM)             – Programmer Capability (PCAP)
      – Code Modification (CM)               – Personnel Continuity (PCON)
      – Integration (IM)                     – Applications Experience (APEX)
      – Assessment (AA)                      – Platform Experience (PLEX)
      – Understanding (SU)                   – Language/Toolset Experience
      – Unfamiliarity (UNFAM)                   (LTEX)
      – Requirements Evolution (REVL)   •   Project EM
•   Product Effort Multipliers (EM)          – Use of Software Tools (TOOL)
      – Required Reliability (RELY)          – Multisite Development (SITE)
      – Database Size (DATA)                 – Required Development Schedule
      – Product Complexity (CPLX)               (SCED)
      – Required Reuse (RUSE)           •   Scale Drivers
      – Documentation (DOCU)                 – Development Flexibility (FLEX)
•   Platform EM                              – Process Maturity (PMAT)
      – Execution Time Constraints           – Precedentedness (PREC)
        (TIME)                               – Arch/Risk Resolution (RESL)
      – Main Storage Constraints             – Team Cohesion (TEAM)
        (STORE)
      – Platform Volatility (PVOL)
                                                                                28
                         COSCOMO Drivers
                         (27 Input Variables)
•   Size                                 •   Platform EM
      – Source Lines of Code (SLOC)            – Platform Volatility (PVOL)
      – Design Modification (DM)               – Bandwidth (BAND)
      – Code Modification (CM)           •   Personnel EM
      – Integration (IM)                       – Senior Capability (SCAP)
      – Assessment (AA)                        – Developer Capability (DCAP)
      – Understanding (SU)                     – Personnel Continuity (PCON)
      – Unfamiliarity (UNFAM)                  – Applications Experience (APEX)
•   Scale Drivers                              – Platform Experience (PLEX)
      – Development Flexibility (FLEX)         – Development Tools Experience
      – Process Maturity (PMAT)                  (DTEX)
      – Precedentedness (PREC)           •   Project EM
      – Arch/Risk Resolution (RESL)            – Lifecycle Tools (LIFE)
      – Team Cohesion (TEAM)                   – Multisite Development (SITE)
•   Product Effort Multipliers (EM)            – Required Development Schedule
      – Required Reliability (RELY)              (SCED)
      – Product Complexity (CPLX)
      – Required Reuse (RUSE)
      – Documentation (DOCU)

                                                                                  29
                   COCOMO-SCORM Drivers
                     (24 Input Variables)
•   Size of Product                       •   Personnel EM
      – Hours of Courseware                    – Senior Capability (SCAP)
      – Level of Instruction                   – Developer Capability (DCAP)
      – New, Reused, & Modified                – Personnel Continuity (PCON)
        Courseware                             – Applications Experience (APEX)
      – Requirements Evolution (Content        – Platform Experience (PLEX)
        Discarded)                             – Development Tools Experience
•   Product Effort Multipliers (EM)               (DTEX)
      – Required Reliability (RELY)       •   Project EM
      – Product Complexity (CPLX)              – Lifecycle Tools (LIFE)
      – Required Reuse (RUSE)                  – Multisite Development (SITE)
      – Documentation (DOCU)                   – Required Development Schedule
•   Platform EM                                   (SCED)
      – Bandwidth (BAND)                  •   Scale Factors
      – Platform Volatility (PVOL)             – Precedentedness (PREC)
                                               – Development Flexibility (FLEX)
                                               – Arch/Risk Resolution (RESL)
                                               – Team Cohesion (TEAM)
                                               – Process Maturity (PMAT)


                                                                                  30
Create COCOMO-SCORM Model




                            31
                   COCOMO-SCORM Drivers
                     (24 Input Variables)
•   Size of Product                       •   Personnel EM
      – Hours of Courseware                    – Senior Capability (SCAP)
      – Level of Instruction                   – Developer Capability (DCAP)
      – New, Reused, & Modified                – Personnel Continuity (PCON)
        Courseware                             – Applications Experience (APEX)
      – Requirements Evolution (Content        – Platform Experience (PLEX)
        Discarded)                             – Development Tools Experience
•   Product Effort Multipliers (EM)               (DTEX)
      – Required Reliability (RELY)       •   Project EM
      – Product Complexity (CPLX)              – Lifecycle Tools (LIFE)
      – Required Reuse (RUSE)                  – Multisite Development (SITE)
      – Documentation (DOCU)                   – Required Development Schedule
•   Platform EM                                   (SCED)
      – Bandwidth (BAND)                  •   Scale Factors
      – Platform Volatility (PVOL)             – Precedentedness (PREC)
                                               – Development Flexibility (FLEX)
                                               – Arch/Risk Resolution (RESL)
                                               – Team Cohesion (TEAM)
                                               – Process Maturity (PMAT)


                                                                                  32
                      Quantitative Size Variables

How many hours of courseware will be in the final product?
Total Hours of
Courseware

How are the courseware hours above divided across the different levels of instruction?
                  Level of Instruction
                  L1                 L2                L3                    L4
Hours of
Courseware
           Weights                 1.0                1.3                1.8                2.5


When considering content, media, and code, what percentage of the final product will be …
                 Brand new?
                 Reused after some modification?
                 Reused without modification?




                                                                                                  33
                     Qualitative Project Variables

                                                                            Extra
                           Very Low Low         Nominal High      Very High High
              APEX              1.22      1.1          1     0.88      0.81 N/A
              BAND         N/A       N/A               1        1         1          1
              CPLX              0.73     0.87          1     1.17      1.34       1.74
              DCAP              1.34     1.15          1     0.88      0.76 N/A
              DOCU              0.81     0.91          1     1.11      1.23 N/A
              DTEX               1.2     1.09          1     0.91      0.84 N/A
  Effort      LIFE              1.17     1.09          1      0.9      0.78 N/A
Multipliers   PCON              1.29     1.12          1      0.9      0.81 N/A
              PLEX              1.19     1.09          1     0.91      0.85 N/A
              PVOL         N/A           0.87          1     1.15       1.3 N/A
              RELY              0.82     0.92          1      1.1      1.26 N/A
              RUSE         N/A           0.95          1     1.07      1.15       1.24
              SCAP              1.42     1.19          1     0.85      0.71 N/A
              SCED              1.43     1.14          1        1         1 N/A
              SITE              1.22     1.09          1     0.93      0.86        0.8
              FLEX              5.07     4.05       3.04     2.03      1.01          0
  Scale       PMAT               7.8     6.24       4.68     3.12      1.56          0
 Factors      PREC               6.2     4.96       3.72     2.48      1.24          0
              RESL              7.07     5.65       4.24     2.83      1.41          0
              TEAM              5.48     4.38       3.29     2.19       1.1          0

 Nominal/Standard Levels                                                                 34
                                                   EM Effect on Project
                                   15 Effort Multipliers for Nominal Project
                                   Cumulative Effect of Effort Multipliers



                            RELY   CPLX RUSE DOCU BAND         PVOL SCAP     DCAP PCON APEX       PLEX   DTEX   LIFE   SITE   SCED
                                                                                                                                     Extra High
Low
                    1.80

                                                                                                                                     Nominal
nal
                    1.60


High
                                                                                                                                     Very Low
High                1.40

lative Multiplier
                    1.20


                            1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00
      Course        1.00
                                                                                                                                       Project
      Hours                                                                                                                             Cost
                    0.80




                    0.60




                           Project Cost                        =             A             *      Course Hours
                           (Person Months)                            (Constant)                     (Student Hours)


                                                                                                                                                  35
                                       Answers Effect Project Cost
                                       Cumulative Effect of Effort Multipliers
                                15 Effort Multipliers for More Realistic Project
                               RELY CPLX RUSE DOCU BAND PVOL SCAP DCAP PCON APEX PLEX DTEX                            LIFE   SITE SCED
Very Low                3.50

Low
Nominal                 3.00

High                                                                            2.66
                                                                                                                      2.50
Very High                                                                                2.39
                        2.50                                             2.31                                 2.29           2.25   2.25
Extra High                                                                                                                                 Project
                                                                                                2.10   2.10
Cumulative Multiplier                                                                                                                       Cost
                        2.00

                                                                  1.63

                                                    1.41   1.41
                        1.50
                                             1.15
                               1.00   1.00
          Course        1.00
                                                                                                                                           Nominal
          Hours                                                                                                                             Cost

                        0.50



             Project Cost                            =               A            * EMi                 * Course Hours
             (Person Months)                                  (Constant)               (Modifiers)                   (Student Hours)


                                                                                                                                                     36
                                                Range of Cost Variation
                                   Cumulative Effect of Effort Multipliers
                                                                                                                                    Max = X
                                                                                                                                     115 ?
                           RELY   CPLX RUSE DOCU BAND         PVOL SCAP     DCAP PCON APEX       PLEX DTEX     LIFE   SITE   SCED
Low
                    1.80


nal
                    1.60


High
High                1.40

lative Multiplier
                    1.20



      Course        1.00
                           1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00   1.00
                                                                                                                                     Nominal
      Hours                                                                                                                           Cost
                    0.80




                    0.60




                    Flex From:
                    X times larger to
                    Y times smaller                                                                                                 Min = Y
                                                                                                                                    0.05 ?
                                                                                                                                              37
  SF Effect on Project
All Scale Factors Set to Nominal




        FLEX PMAT PREC RESL TEAM
          L    L    L   L     L


 SW                                Project
 Size                               Cost

         H    H    H    H    H




                                              Very Low    L
                                                   Low
                                               Nominal
                                                  High
                                             Very High
                                             Extra High   H   0.0 (always)
                                                                             38
   SF Effect on Project
Scale Factors Other Than Nominal




                                   TEAM   Est.
                                     L
         FLEX                             Cost
           L           PREC RESL
                         L   L
                PMAT
  SW              L                       Nom.
  Size                                    Cost
                                    H

          H
                        H    H

                 H                                Very Low    L
                                                       Low
                                                   Nominal
                                                      High
                                                 Very High
                                                 Extra High   H   0.0 (always)
                                                                                 39
               Range of Scale Effect


                                                 Max = X
                                                  1.226


                      FLEX PMAT PREC RESL TEAM
                        L    L    L   L     L


               SW                                Project
               Size                               Cost

                       H    H    H    H    H


                                                 Min = Y
                                                  0.91
Flex:
22.6% Higher
9% Lower

                                                           40
                        COSCOMO Model Form

                                                   * P EMi
                                                       15
                     PM =       A*(Size)E
                                                       i=1


                                                             5
                                    E = B + .01*S SFj
                                                         j=1

Where:
   PM = effort in Person Months
   A = calibration constant derived from historical project data
   Size = Adjust Source Lines of Code
   E = represents diseconomy of scale (composed of 5 scale factors)
   EM = effort multiplier for the ith cost driver. The geometric product results in an
   overall effort adjustment factor to the nominal effort.

    B = calibration constant derived from historical project data
    SF = scale factor for the jth cost driver. Provides organization specific
    adjustments to the size of the project.

                                                                                         41
Data Collection




                  42
                 Projects Collected


•   Defense Nuclear Weapons School, Dean Marvin
•   Air Force Accounting Liaison Course, Gary Twogood
•   Explosive Hazard Awareness Course, Larry Helms
•   (more to follow …)

•   Leads:
•   Capt Scott Loller, Air Force Distance Learning
•   Angela Lindsey, Joint Forces Staff College
•   Maj Chris Edwards, UK Training Advisory Group
•   Patricia Mulligan, Formerly AFAMS




                                                        43
                         Data Collection Form
                                COURSE AND CONTACT IDENTIFICATION


      Name of
      Course:

      Organization:
      Person
      Surveyed:

      Contact
      Information:



                                                   SIZE AND REUSE


      What is the level of SCORM-conformance of this course?
                          none                           1.0                  1.2              2004
Q1    SCORM

      How many person-months of effort went into building the course? (Usually calculated as Duration * # People * Average % Effort from Each person.)
                      Person-Months People Assigned Duration Months Hours/Month
      Person-Months
Q2    of Effort

      How many hours of courseware will be in the final product?
      Total Hours of
Q3    Courseware

      How are the courseware hours above divided across the different levels of instruction?
                        Level of Instruction
                        L1                 L2                L3                    L4
      Hours of
Q4    Courseware
      Weights

Q5    When considering content, media, and code, what percentage of the final product will be …
                       Brand new?
                       Reused after some modification?
                       Reused without modification?
                                                                           0%



 (see Excel file for entire form…)                                                                                                                       44
                                            COSCOMO Estimates

           Course                   Person-                   Person-    Time to    Time to
           Name                     Months                    Months     Develop    Develop
                                    Reported                  Estimate   Reported   Estimate
           Defense                  78                        75         13         17.5
           Nuclear
           Weapons
           School
           AF                       24.3                      105.3      7.0        19.1
           Accounting
           Liaison
           Course
           Explosive   6.0                                    4.8        2.0        7.7
           Hazard
           Awareness *
           Next …


* Survey was not completed. All remaining variables were estimated.                            45
                Defense Nuclear Weapons School

                                         Cumulative Effect of Effort Multipliers


                        2.00

Very Low                1.80
Low
                        1.60
Nominal
High                    1.40
Very High
                        1.20
Extra High
                                                  1.07   1.07   1.07
Cumulative Multiplier   1.00      1.00     1.00
                                                                       0.93

                        0.80                                                  0.79
                                                                                     0.70
                        0.60
                                                                                            0.56
                                                                                                   0.46
                        0.40                                                                              0.42
                                                                                                                 0.38      0.38   0.38   0.38


                        0.20


                        0.00
                               RELY   CPLX    RUSE   DOCU   BAND   PVOL   SCAP   DCAP   PCON   APEX   PLEX   DTEX       LIFE   SITE   SCED




                                                                                                                                             46
                        AF Accounting Liaison Course

                                          Cumulative Effect of Effort Multipliers



Very Low
Low                             RELY         RUSE          BAND          SCAP          PCON          PLEX           LIFE         SCED
                         2.50
Nominal
High
Very High
                         2.00
Extra High
Cumulative Multiplier
                         1.50

                                                                1.25
                                   1.10    1.10                        1.09
                                                  1.05   1.05
                         1.00

                                                                              0.77

                                                                                     0.59                               0.60
                         0.50                                                               0.53          0.51   0.51          0.54   0.54
                                                                                                   0.43




                         0.00




                                                                                                                                             47
                        Explosive Hazard Awareness
                                        Cumulative Effect of Effort Multipliers



                               RELY CPLX RUSE DOCU BAND PVOL SCAP DCAP PCON APEX PLEX DTEX                         LIFE   SITE SCED
Very Low                2.50
Low
Nominal
High                    2.00

Very High
Extra High
Cumulative Multiplier   1.50
                               1.26
                                                           1.15
                                      1.10
                                             1.04   1.04
                                                                  1.00
                        1.00                                             0.85
                                                                                0.75
                                                                                       0.67   0.67
                                                                                                     0.61          0.61
                                                                                                            0.56          0.55   0.55

                        0.50




                        0.00




                                                                                                                                        48
Model Calibration




                    49
Building PRED(30) for COSCOMO

          Total Project Cost (PM)
               DNWS
               -3.8%
                       78




                                      ALC
                                     +333%
                       24
       EHA
       -20%
                       6


   -30%           0%          +30%

   % Variance of Estimated Value
                                             50
                            Calibration of Variables

                                                                            Extra
                           Very Low Low         Nominal High      Very High High
              APEX              1.22      1.1          1     0.88      0.81 N/A
              BAND         N/A       N/A               1        1         1          1
              CPLX              0.73     0.87          1     1.17      1.34       1.74
              DCAP              1.34     1.15          1     0.88      0.76 N/A
              DOCU              0.81     0.91          1     1.11      1.23 N/A
              DTEX               1.2     1.09          1     0.91      0.84 N/A
  Effort      LIFE              1.17     1.09          1      0.9      0.78 N/A
Multipliers   PCON              1.29     1.12          1      0.9      0.81 N/A
              PLEX              1.19     1.09          1     0.91      0.85 N/A
              PVOL         N/A           0.87          1     1.15       1.3 N/A
              RELY              0.82     0.92          1      1.1      1.26 N/A
              RUSE         N/A           0.95          1     1.07      1.15       1.24
              SCAP              1.42     1.19          1     0.85      0.71 N/A
              SCED              1.43     1.14          1        1         1 N/A
              SITE              1.22     1.09          1     0.93      0.86        0.8
              FLEX              5.07     4.05       3.04     2.03      1.01          0
  Scale       PMAT               7.8     6.24       4.68     3.12      1.56          0
 Factors      PREC               6.2     4.96       3.72     2.48      1.24          0
              RESL              7.07     5.65       4.24     2.83      1.41          0
              TEAM              5.48     4.38       3.29     2.19       1.1          0

 Nominal/Standard Levels                                                                 51
                   Delphi Survey


• Using Delphi Method to gather independent opinions
  about the level of effort associated with each setting on
  each variable.
   – Collect individual responses
   – Calculate group average
   – Present averages to each person and allow them to
     change their responses if they would like
   – Convene group work session to settle on acceptable
     values
   – Repeat if necessary
• Surveys distributed to:
   – Dean Marvin, Kelly Ward, Denise Stevens, Tim Richey,
     Tatjana Pitts



                                                              52
Conclusion




             53
                   Completed Work

• Technical and Management Work Plan (DI-MGMT-81117)
• Researched COCOMO II
   – COCOMO published literature
   – COSYSMO seminar and relationship with Dr. Ricardo Valerdi
• Researched SCORM Cost Drivers
   – ADDIE process steps
   – SCORM project cost drivers
• Created COCOMO-SCORM Algorithm
   – Transformed COCOMO variables into COSCOMO tool
   – Created new variables for COSCOMO




                                                                 54
                      Current Work


• Creating COCOMO-SCORM Software Model
   – Functional software tool
• Data Collection
   – Collect historical project data
• Model Calibration & Validation
   – Calibrate model to fit historical data and expert knowledge
• Build Community
   – Create a community of interest around the method and tool




                                                                   55
                         Next Steps


• Complete COCOMO-SCORM Model
    – Microsoft Excel implementation
    – Accompanying Documentation
• Create Prototype GUI
• Final Project Review
• Phase 2 (optional)




                                       56
                        Right on Track

Task                            Status

1.1 Project Management          In Progress

1.2 Project Kick-off Meeting    Complete

1.3 Research COCOMO Variables   Complete

1.4 Research ISD Cost Items     Complete

1.5 COSCOMO Algorithms          In Progress

1.6 In-Process Review           In Progress

1.7 COSCOMO Software Model      In Progress

1.8 Prototype User Interface    Not Started

1.9 Algorithm Documentation     In Progress

1.10 Final Project Review       Not Started

1.11 Validation Tests           In Progress

                                              57

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:8
posted:9/4/2010
language:English
pages:57