COTS Project Types CeBASE COTS effort

Document Sample
COTS Project Types CeBASE COTS effort Powered By Docstoc
					COTS Project Types
 CeBASE COTS research

Ye Yang, Jesal Bhuta, Dan Port
                COTS Definition

• For the purposes of these classifications, we
  use the following definition of COTS
“A software product, developed by a third party (who controls
  its ongoing support and evolution), bought, licensed, or
  acquired for the purposes of integration into a larger system
  as an integral part, i.e. that will be delivered as part of the
  system to the customer of that system (i.e. not a tool),
  which might or might not allow modification at the source
  code level, but may include mechanisms for customization,
  and is bought and used by a significant number of systems
MBASE Milestone Elements
MBASE Milestone Elements (2)
         COTS Based Systems
• A COTS Based System (CBS) is any system that
  more than 50% of a systems functional
  requirements are implemented through use of COTS
  via glue code, tailoring, and assessment effort.
  – Glue code is the custom development needed to integrate
    COTS packages within an application external to the
    packages themselves
  – Tailoring effort is the activities relating to adapting
    COTS packages internally for use within an application
  – Assessment effort is the suitability evaluation and analysis
    of the desired attributes of the COTS packages for an

     * The above definitions are compatible COCOTS definitions
               CBS Approaches
• There are a variety of approaches to developing
  COTS based systems (CBS).
   – Find and use COTS packages as is (“turnkey”)
   – Find and adapt COTS packages as needed (“adaptation”)
   – Use COTS as components in custom built application
• These approaches differ considerably and driven
  primarily by the systems shared vision and desired
  characteristics & priorities (DC&P’s).
   – Effort distributed between
      • Glue code
      • Assessment
      • Tailoring
CS577B COTS Effort Distribution

                     Source: CS577 Project
                     Effort Data
                     CBS Types
•   Roughly, a CBS project can be classified
    as either
        •   Assessment Intensive
        •   Tailoring Intensive
        •   Application Intensive

•   These classifications provide an overview
    of the potential risks, attributes, and
    development effort priorities.
    –   Use for strategic and tactical development
          Assessment Intensive

• An assessment intensive CBS project will have
   – The DC&P’s covered without much modification (or
     tailoring) by a single COTS framework or through the
     simple integration of several.
   – The primary effort is focused on identifying collection
     of candidate COTS products and evaluating (i.e.
     assessing) a feasible set of products whose existing
     capabilities directly address a desired operational
      Tailoring intensive CBS

• A tailoring intensive CBS project is where
  – The DC&P’s are covered by a single (or very
    few) COTS framework(s).
  – The primary effort is on adapting a COTS
    framework whose existing general capabilities
    can be customized (i.e. tailored) in a feasible
    way to satisfy the majority of a systems
    capabilities or operational scenarios.
          Application Intensive
• The application intensive CBS project will have
   – Some nontrivial DC&P’s covered by COTS, and some
     that must be covered by custom development.
   – The primary effort is to identify COTS packages that
     can feasibly used as components to address subsets of
     the required system capabilities.
   – Typically there are significant “buy versus build”
      • Critical factors are: available COTS products, schedule
        limitations, budget limitations, and desired current and future
          Effects of CBS Types
• The CBS type effects many aspects of a
  –   MBASE development guidelines
  –   Glue code, assessment, tailoring effort
  –   Risk factors
  –   Requirements flexibility
  –   System evolution and maintenance
  –   Cost factors
  CBS project Type Guidelines
                         System shared vision
              Desired characteristics & priorities (DC&P’s)

                  Relation of candidate COTS
                      products to DC&P’s

                        II              III           IV

Standard           Application          Tailoring-            Assessment
MBASE              -Intensive           Intensive             -Intensive
Application        CBS                  CBS                   CBS
Guidelines         Guidelines           Guidelines            Guidelines
   CBS project Type Selection Matrix

Evaluate with respect to business case and organization attributes
Assessment Intensive CBS

         Ye Yang
    Assessment Intensive CBS

• Assumption:
  – COTS packages available to cover most of
    desired system capabilities identified according
    to system shared vision among stakeholders.
  – A considerable number of COTS package
    combination needs to be assessed against
    established evaluation criteria.
    Assessment Intensive CBS-
         Characteristics 1
• The primary effort is focused on identifying
  collection of candidate COTS products and
  evaluating a feasible set of products whose
  existing capabilities directly address a
  desired operational concept.
  – About 60-80% of COTS related effort is
    assessment effort, and accounts for 14% of
    overall effort.
    Assessment Intensive CBS-
         Characteristics 2
• Need high flexibility in requirements and
  design due to COTS uncertainty
  – COTS capabilities compose requirements
  – Requirements cannot be fixed (or “hard”)
    before final COTS selection
  – We use Desired Characteristics & Priorities
    (DC&P’s) to guide requirements gathering and
    Assessment Intensive CBS-
         Characteristics 3
• System architecture is focused on a COTS
  package configuration model, since the
  class model, object model and interaction
  model are not available for most COTS
  – Example: Rework effort for CS577 2001-2002
    team8’s SSAD
    Assessment Intensive CBS-
         Characteristics 4
• A business case analysis for each COTS solution
  is highly recommended for the detailed COTS
   – COCOTS Model for effort and schedule estimation
      • Assessment, glue code, tailoring effort tradeoffs
   – Return on Investment analysis
      • Value, risk, cost tradeoffs
   – COTS specific risks and mitigation costs
      • vendor volatility, upgrade lifecycle, etc.
• Example: Dot-Project infeasibility in CS577A
  2001 team 8
   Identifying Assessment CBS

• You have an assessment intensive CBS
  when you have the previously stated
  – High requirement volatility, high domain
    adaptability, low developer resources…
  – Business case shows feasibility of focusing
    effort on COTS assessment (rather than glue
    code, tailoring, or build from scratch)
       Assessment Intensive CBS
          Project Guidelines

Assessment   Assess COTS                      Yes
-Intensive   candidates with        Singl        Follow Tailoring-Intensive
CBS          respects to            e                 COTS Guideline
             DC&P’s                 COT
                                    No, COTS Mix

                               Develop glue         Integrate,       Evolve as
                               code; Tailor         test,            tailored
                               COTS mix to          transition       COTS mix
                               DC&P’s               IOC

• Project profile
  – USC Collaborative Services System (team 8
    CS577 2001-2002)
     • DC&P’S
        –   Project management
        –   File management
        –   Online Discussion Board
        –   Group Calendar
Example- COTS Assessment Steps
• Plan: set the evaluation effort scope and provide a baseline
  for evaluation process.
• COTS Identification: search for candidate COTS packages
  based on the set of DC&P’s
• Establish evaluation criteria based on functional,
  performance, architectural, and financial characteristics of
• Conduct multiple cycle of evaluation to collect evaluation
  data against the defined evaluation criteria.
• Perform Hand-on Experiments on COTS to verify vendor’s
• Compare and contrast the evaluation data and make COTS
  final selection.
              Major risks-1

• Many new non-technical activities
  introduced into software engineering
  – Establishing evaluation criteria,
  – Planning and executing evaluation procedures,
  – Collecting and analyzing data from evaluation
  – Making COTS recommendations, which
    require training and experience.
                  Major Risks-2

• Within the COTS evaluation process:
   – You may miss some possible COTS candidates.
      • Stay as broad as possible when doing the initial search for
   – You might not include all key aspects for establishing
     the evaluation criteria set.
      • Judgment of weight distributions require experienced and
        knowledgeable people
   – Introducing new COTS candidates is likely and require
     re-planning or contingency planning.
      • Limits on schedule and budget must be accounted for
              Major Risks-3

• You may not get all the features of certain
  COTS products as advertised by its vendor.
  – Detailed assessment beyond literature review or
    vendor provided documentation should be
    performed in the form of hands-on experiments.
  Example - Comparison of results from initial
     assessment and detailed assessment

Evaluation Requirement                    COTS candidate 1-    COTS candidate 2-    COTS candidate 3-
                                          eProject             Blackboard           iPlanet collaborative
                                          Initial   Detailed   Initial   Detailed   Initial     Detailed
Users can upload/modify/download/delete   **        *****      **        **         *           None
authorized files

Project management, including planning    **        *****      **        ***        None        None
group tasks, timelines and tracking

Message Board                             **        *****      **        ****       None        None
User Interface                            **        *****      **        **         **          *
Admin Interface                           **        *****      **        ****       **          None
Group Calendar                            **        *****      **        **         **          *

         Detailed analysis provides greater assurance of COTS
         characteristics with respect to vendor documentation
         (although at significant effort)
               Major Risks-4

• The organization may not have the ability or
  willingness to accept the impact on its
  operation due to imposed COTS
  – Example:
     • CS577 2001-2002 team 8:
        – COTS: Blackboard
        – Required operational change: User access mode
             Major Risks-5

• Difficulty in finding suitable time for key
  personal meetings to discuss COTS
  assessment may incur significant and
  unpredictable schedule delay
  – Based on analysis from student project weekly
    effort reports, team interaction and client
    interaction are the top-2 activities for
    assessment intensive CBS projects.
Tailoring Intensive CBS

       Jesal Bhuta
      Tailoring Intensive CBS

• Assumption:
  – COTS packages have some sort of a
    tailoring/development interface using which the
    COTS package can be adapted to the DC&P’s
  – Little or no assessment is required (if required it
    is already complete) in selecting the COTS
     • CS 577 2001 – 2002
        – Team 20 (577 MBASE in BORE)
        – Team 21 (Quality Management Through BORE)
      Tailoring Intensive CBS
          Characteristic 1
• The primary effort is on tailoring the
  existing adaptable capabilities of the COTS
  package to satisfy the majority of a systems
  core capabilities or operational scenarios.

  – CS 577 2001 – 2002
        – Team 20 (577 MBASE in BORE)
        – Team 21 (Quality Management Through BORE)
       Tailoring Intensive CBS
           Characteristic 2
• Need moderate flexibility in requirements and
  design due to limitations in adaptability of COTS
   – Capabilities (and adaptability) of the COTS packages
     may only partially satisfy the system requirements
   – System requirements may need to be adapted to fit
     within COTS limitations
      • Significant source of risk, focus on DC&P’s to manage
        requirements change
      • Example: QA in BORE security levels requirement change
       Tailoring Intensive CBS
           Characteristic 3
• The focus of the system architecture depends upon
  the type of the COTS package tailoring required in
  order to satisfy the DC&P’s
   – GUI Based Tailoring (ex. Spearmint)
      • No focus on system architecture
   – Parameter Based Tailoring (ex. Windows Media
      • Little or no focus on system architecture
   – Programmable based Tailoring Interface (ex.
      • Moderate focus on system architecture
     Tailoring Intensive CBS –
          Characteristic 4
• The Tailoring intensive CBS development
  usually require some amount of time for
  developer training to get acquainted to the
  COTS package.
  – A Learning Curve must be accounted for during
    the system development process
    Identifying Tailoring CBS

• You have an tailoring intensive CBS when
  you have the previously stated
  – Business case shows feasibility of buying and
    tailoring COTS packages to implement major
    system capabilities
Tailoring Intensive CBS Project

Tailoring-    Get acquainted       Tailor COTS   Test,         Evolve as
Intensive     with the COTS        Framework     Transition,   Tailoring-
CBS           framework            to DC&P’s     IOC           Intensive CBS

             Ensure that the
             core features to be
             used, function at
             an acceptable level
                      Risk 1

• Version change may result in re-tailoring of
  COTS product
  – A constant feedback by the client to the vendor
    about the features used and enhancements
    required may help in ensuring that the features
    used by the current CBS continue in the future
    versions of the project
     • Ex. “The LOGO element specifies a graphic file to
       display as a logo. Not supported after Windows
       Media Player version 6.4.”
                    Risk 2

• Inadequate vendor support may result in
  significant project delays
  – Hiring consultants with experience in using
    COTS product (Non-CS577 mitigation)
  – Ensuring an efficient Phone/Email support at
    least during the development phase
                      Risk 3

• Faulty Vendor claims (COTS “surprises”)
  may result in Significant Delays
     • (Ex) Oracle intermedia claims to search for all
       unicode characters however the search is not carried
       as specified
  – Pre-testing of core features to be used before
    beginning of the actual development
                      Risk 4

• COTS package incompatibilities
  (integration clash)
  – In case of multiple COTS packages being used
    to implement the DC&P’s there may exist
    incompatibilities between COTS solutions
     • Pre-testing of compatibilities between packages to
       be used before beginning of the actual development
                     Risk 5

• Added complexity of un-used COTS
  – Adaptable and Tailoring capable COTS
    packages usually come with additional
    complexity and cost that is incurred in the
    process of making the package tailorable
     • Ex Microsoft Office Package
    Assessment/Tailoring Intensive CBS
          Life Cycle and Mbase
             Identify the Type of CBS   Identify the COTS
             Narrow the list of         product (s) that shall
             Vendors depending on       implement the system
             available information

        Inception                 Elaboration                Construction 1             Construction 2
                        LCO                      LCA                             CCD                       I OC
                                                         Train the Developers          Implement the low          Support personnel
                           Develop and Test
Obtain a shared                                          to use the COTS               priority                   must be trained to
                           Functional Prototypes
Vision amongst                                           product                       requirements of            work with the
                           to ensure the
stakeholders.                                                                          the system                 COTS API
                           functioning Core
Identify why to go                                       Implement and test                                       Maintaining
CBS (window of                                           the Core requirements                                    constant contact
opportunity,                                             of the system                                            with the developer
                           Collect Detailed
independence from                                                                                                 in terms of
                           Information and
evolutionary                                                                                                      specifying
                           Evaluate the COTS
maintenance, …).                                                                                                  evolution
                           vendors short-list
Develop and collect                                                                                               requirements
Information on
available COTS
product in the market
(RFI). Make a list of
products which can
satisfy the Core
requirements and
undergo a primary
evaluation them
Application Intensive CBS

         Dan Port
    Application Intensive CBS
• Assumptions:
  – COTS packages are available to satisfy
    significantly valuable subsets of system
    capabilities or project limitations dictate use
    (e.g. schedule, skill, or complexity factors)
     • Buy vs. build cost-benefit ROI
  – Integration overhead is minimal with respect to
    custom development
  – Low assessment and tailoring effort required
    Assessment Intensive CBS-
         Characteristics 1
• The primary effort is focused on identifying
  system components and requirements that
  may be risky to custom implement and
  mapping them to a collection of candidate
  COTS products.
  – CS577 2001-2002 Team 15 DART’s use of
    MySQL and Tomcat
      Application Intensive CBS-
           Characteristics 2
• COTS expected to implement system requirements
• Low requirements volatility and adaptability
• Significant glue code effort expected
• COTS packages used are not specialized to a
  particular application domain (generic capabilities)
• COTS components dependent on application to
• Many possible COTS packages may apply
• Many evolutionary requirements with respect to
  custom components and anticipated COTS features
   Identifying Application CBS

• You have an assessment intensive CBS
  when you have the previously stated
  – Business case shows feasibility of buying
    components to implement major system
    capabilities and focusing effort on COTS
    integration (glue code)
                Application Intensive CBS
                   Project Guidelines

                                                             Tailor COTS

Application-     Assess COTS          Choose best mix of   Develop glue code,     Integrate, test,
Intensive CBS    candidates with      COTS, custom         integrate COTS,        transition IOC
                 respects to DC&P’s   components           custom components

                                                             Develop custom     Evolve as
                                                             components         application-
                                                                                intensive CBS

• Project profile
  – CS577
     • Team 15 DART 2001-2002
        – Tomcat, MySQL
     • Team 17 Web-mail 2000-2001
        – Tomcat
     • Team 8 Fulltext Title Search 2000-2001
        – MySQL, Tomcat, Zope
     • Team 9 Student/Staff Directories 2001-2002
        – Zope
               Major risks-1

• Spending too much effort locating appropriate
  COTS products (too little value for cost)
• Overly optimistic expectation of COTS quality
• COTS package incompatibilities (integration clash)
• Added complexity of un-used COTS capabilities
• Imposed black box testing of COTS components
• Inadequate COTS assessments
            Major risks-2

• Overly optimistic COTS package learning
• Compromising hard requirements to make
  use of particular COTS
• COTS “surprises”, capability shortfalls
• Application dependent on COTS vendors
  (legacy support, upgrades, bugs, etc.)
CBS Type Comparisons
1     Team Interaction                     7      Feasibility Rationale Document
2     Client Interaction                   8      COTS Final Selection
3     COTS Initial Filtering               9      Control and Monitoring
4     Life Cycle Planning                  10     COTS Tailoring
5     Project Web-site                     11     Transition and Support Planning
6     Training and Preparation             12     Critical Component Implementation

    Source: CS577 2001-2002 Project Effort Data
CBS project Type Comparisons