Service Oriented Architecture (SOA) Implications to End-to-End by qfa20129

VIEWS: 20 PAGES: 14

									Service Oriented Architecture (SOA)
    Implications to End-to-End
            Assessment



                           Brian Eleazer
                           Brian Hall
                           Robert Kohout
                           Joint Systems Integration Center
                           U.S. Joint Forces Command
                           757-203-4421 / 4453 / 7598
                           John.eleazer@jsic.jfcom.mil
                  outline
•   Definitional Grounding
•   Elements of SOA Implementation
•   Notional “V” Approach to SOA
•   Lessons Learned and Considerations




                                         2
UNCLASSIFIED
                             Definitional
• SOA Services are autonomous, reusable
  components that provide specific business
  or mission capability.
• Two key conditions:
      – Autonomous, reusable components
               • Exposed, accessible information
               • Availability, Quality of Service (QoS)
      – Specific business/mission capability
               • Orchestrated usage
               • Measurable operational utility
                                                          3
UNCLASSIFIED
         Elements of SOA Implementation
                                                                 Operational Perspective

                         • End-to-end orchestrated usage of multiple services & systems to
Each layer presents      specific tasks
                         • Access of mission-critical data and information to allow
development & testing    commanders and users to adapt to changing mission needs

challenges …
                                                      Systems / Services Perspective
                         • Loosely coupled components with minimum development
                         assumptions about when, why, or under what environment
      …characteristics   conditions invoked
                         • Provide functionality abilities to allocated requirements and
and considerations       specifications




                                                          Infrastructure Perspective
                         • Multiple, independently constructed, ‘adopted’ products
                         configured to interact via persistent network
                         • Means to connect
                                •Registration, subscription, and discovery characteristics
                         • Cross-domain networks


                                                                                             4
 UNCLASSIFIED
                           Characteristics and Challenges
                                                                                            • De-centralized ownership and control; variety of
                                                                                              providers, infrastructure, and consumers
                                                                • Combination of            • Lack of design information; subtle engineering design
                                    Operational Perspective
                                                                  humans, applications,       limitations
• End-to-end orchestrated usage of multiple services &            web-services,             • SLA and Web-Service Definition Language (WSDL)
systems to specific tasks
• Access of mission-critical data and information to allow
                                                                  networks/back-office,       does not ensure behavior and desired expectation
commanders and users to adapt to changing mission                 databases, business       • Complex end-to-end execution; variations in
needs                                                             rules                       mission, configuration, and business activities
                                                                                            • Aggregated failures, difficultly in root-cause analysis
                                                                                              & assigned correction

                                                                                            • Unknown context, lack of usage understanding
                                                                • Web-service               • Unanticipated demand, impact to QoS & load
                      Systems / Services Perspective              implementation            • Customized implementation of WSDL, Simple Object
                                                                • Message formats             Access Protocol (SOAP), Extensible Markup
• Loosely coupled components with minimum
development assumptions about when, why, or under
                                                                • Service Level               Language (XML), Universal Descriptive Discovery
what environment conditions invoked                               Agreements (SLA)            and Integration (UDDI) …
• Provide functionality abilities to allocated requirements     • Functionality, QoS,       • Lack of stimulation or modeling service behavior
and specifications
                                                                  Conformance to open       • Misunderstanding data exchanges, lack of common
                                                                  standards                   data model
                                                                • Multi-services            • Second-order and un-intended consequences
                                                                  constructs                • Multiple provider schedules, testing, and increments
                                                                                            • Independent releases; un-defined regression testing
                                                                                              parameters
                           Infrastructure Perspective

                                                                • Registration &            • Response time & latency
• Multiple, independently constructed, ‘adopted’ products
                                                                  Discovery                 • Limited technical information (closed design or
configured to interact via persistent network
• Means to connect                                              • Transform / Translation     proprietary)
   •Registration, subscription, and discovery characteristics     of meta-data              • Complexity in configuration, administration, and
• Cross-domain networks
                                                                • Security authentication     security protocols
                                                                • Message interaction       • Dynamics in releases, patches, service packs
                                                                • Data bases & data         • Cross-domain variations, implementation of
                                                                  stores                      standard, meta-data attributes                5
   UNCLASSIFIED
                  Horizontal Composition
                                                                         Conduct Joint / Component  
                                                      2.2.3              Parallel Planning            OP 6.2.9       Coordinate Personnel Recovery
                                                                           Conduct Joint / Component  OP 2.6         Provide Intelligence Support to Plans




Traditional view toward
                                                              2.2.3        Parallel Planning                         Gain and Maintain Situational 
                                                                                                              OP 6.2.9      Coordinate Personnel Recovery
                                                                                                      OP 2.5 OP 2.6  Understanding (SU)



                                                                                                                                                                                  Operational Perspective
                                                                                  Conduct Joint / Component                 Provide Intelligence Support to Plans
                                                                      2.2.3       Parallel Planning   OP 5.3         Prepare Plans and Orders
                                                                                                                            Gain and Maintain Situational 
                                                                                                                      OP 6.2.9    Coordinate Personnel Recovery
                                                                                                              OP 2.5 Overcome Operationally Significant 
                                                                                                                            Understanding (SU)

                                 Joint Publications
                                                                                                                      OP 2.6      Provide Intelligence Support to Plans
                                                                                                      OP 1.3.1OP 5.3 Barriers, Obstacles, and Mines
                                                                                                                            Prepare Plans and Orders
                                                                                                                                  Gain and Maintain Situational 
                                                                                                                            Overcome Operationally Significant 
                                                                                                                      OP 2.5      Understanding (SU)
                                                      2.2.4           Develop & Analyze COA           OP 2.6OP 1.3.1 Provide Intelligence Support to Plans
                                                                                                                            Barriers, Obstacles, and Mines
                                                                                                                      OP 5.3      Prepare Plans and Orders
                                                                                                                     Overcome Operationally Significant 
                                                                                                                                  Overcome Operationally Significant 
                                                            2.2.4          Develop & Analyze COA      OP 1.3.1       Barriers, Obstacles, and Mines
                                                                                                              OP 2.6OP 1.3.1Provide Intelligence Support to Plans
                                                                                                                                  Barriers, Obstacles, and Mines
                                                                                                                     Gain and Maintain Situational 
                                                                                                                            Overcome Operationally Significant 




integration
                                                                  2.2.4                               OP 2.5 OP 1.3.1
                                                                                  Develop & Analyze COA              Understanding (SU)
                                                                                                                      OP 2.6Barriers, Obstacles, and Mines
                                                                                                                                  Provide Intelligence Support to Plans
                                                                                                                     Analyze and Compare Theater Courses of 
                                                                                                                            Gain and Maintain Situational 
                                                                                                                                  Overcome Operationally Significant 
                                                                                                      ST 5.3.1.2  ActionUnderstanding (SU)
                                                                                                              OP 2.5  OP 1.3.1    Barriers, Obstacles, and Mines
                                                                                                      ST 6.2.7       Conduct Personnel Recovery
                                                                                                                            Analyze and Compare Theater Courses of 
                                                                                                                                  Gain and Maintain Situational 
                                                                                                                     Integrate Theater Intelligence Capabilities 
                                                                                                              ST 5.3.1.2  ActionUnderstanding (SU)
                                                                                                                      OP 2.5 
                                                                                                                     with Operations
                                                                                                      ST 2.8  ST 6.2.7      Conduct Personnel Recovery
                                                                                                                                  Analyze and Compare Theater Courses of 
                                                                                                      OP 5.3.5       Analyze Courses of Action
                                                                                                                            Integrate Theater Intelligence Capabilities 
                                                                                                                      ST 5.3.1.2  Action
                                                                                                              ST 2.8 Compare Courses of Action
                                                                                                      OP 5.3.6              with Operations
                                                                                                                      ST 6.2.7    Conduct Personnel Recovery
                                                                                                                OP 5.3.5     Analyze Courses of Action
                                                                                                                                   Integrate Theater Intelligence Capabilities 
                                                      2.2.5

                                                              2.2.5
                                                                         Select & Brief COA

                                                                               Select & Brief COA
                                                                                                          OP 5.3.7    Select or Modify Course of Action
                                                                                                                      ST 2.8 Compare Courses of Action
                                                                                                                OP 5.3.6 
                                                                                                                      OP 5.3.5 
                                                                                                                OP 5.3.7     Select or Modify Course of Action
                                                                                                                      OP 5.3.6 
                                                                                                                                   with Operations
                                                                                                                                   Analyze Courses of Action
                                                                                                                                   Compare Courses of Action
                                                                                                                                                                                   OVs, BPMs
                                                                      2.2.5          Select & Brief COA               OP 5.3.7     Select or Modify Course of Action




Development of Operational-to-                                                 Tasks, UJTs


Tactical level of operations

                                                                                                                                  Systems / Services Perspective

Integration of systems,
applications, technical
exchanges, and components         System Views
                                                                                                                                                                                               Service
                                                                                                                                                                                               Exchange
across development activities                                                                                                                                                                  Matrix




                                                                                                                                                      Infrastructure Perspective
Success dependent upon                                                                  Meta-data &

understanding and agreement                                                             Data Stores


on foundation

                                    Component Model
                                                                                                                                                                           Coding               Network




                                                      Horizontal integration                                                                                                                                6
  UNCLASSIFIED
                      Vertical Decomposition
                                                                                                                                                                                                                   Improvement
                              Priority Need                      2.2.3

                                                                       2.2.3
                                                                                    Conduct Joint / Component  
                                                                                    Parallel Planning
                                                                                      Conduct Joint / Component  
                                                                                      Parallel Planning
                                                                                                                 OP 6.2.9 
                                                                                                                 OP 2.6

                                                                                                                 OP 2.5 OP 2.6
                                                                                                                                Coordinate Personnel Recovery
                                                                                                                                Provide Intelligence Support to Plans
                                                                                                                                Gain and Maintain Situational 
                                                                                                                         OP 6.2.9      Coordinate Personnel Recovery
                                                                                                                                Understanding (SU)
                                                                                                                                                                                                                     Criteria
                                                                                                                                                                                             Operational Perspective
                                                                                             Conduct Joint / Component                 Provide Intelligence Support to Plans




An Integrating views
                                                                             2.2.3           Parallel Planning   OP 5.3         Prepare Plans and Orders
                                                                                                                                       Gain and Maintain Situational 
                                                                                                                                 OP 6.2.9    Coordinate Personnel Recovery
                                                                                                                         OP 2.5 Overcome Operationally Significant 
                                                                                                                                       Understanding (SU)

                                            Joint Publications
                                                                                                                                 OP 2.6      Provide Intelligence Support to Plans
                                                                                                                 OP 1.3.1OP 5.3 Barriers, Obstacles, and Mines
                                                                                                                                       Prepare Plans and Orders
                                                                                                                                             Gain and Maintain Situational 
                                                                                                                                       Overcome Operationally Significant 
                                                                                                                                 OP 2.5      Understanding (SU)
                                                                 2.2.4           Develop & Analyze COA           OP 2.6OP 1.3.1 Provide Intelligence Support to Plans
                                                                                                                                       Barriers, Obstacles, and Mines
                                                                                                                                 OP 5.3      Prepare Plans and Orders
                                                                                                                                Overcome Operationally Significant 
                                                                                                                                             Overcome Operationally Significant 
                                                                       2.2.4          Develop & Analyze COA      OP 1.3.1       Barriers, Obstacles, and Mines
                                                                                                                         OP 2.6OP 1.3.1Provide Intelligence Support to Plans
                                                                                                                                             Barriers, Obstacles, and Mines
                                                                                                                                Gain and Maintain Situational 
                                                                                                                                       Overcome Operationally Significant 
                                                                             2.2.4                               OP 2.5 OP 1.3.1
                                                                                             Develop & Analyze COA              Understanding (SU)
                                                                                                                                 OP 2.6Barriers, Obstacles, and Mines
                                                                                                                                             Provide Intelligence Support to Plans
                                                                                                                                Analyze and Compare Theater Courses of 
                                                                                                                                       Gain and Maintain Situational 
                                                                                                                                             Overcome Operationally Significant 
                                                                                                                 ST 5.3.1.2  ActionUnderstanding (SU)
                                                                                                                         OP 2.5  OP 1.3.1    Barriers, Obstacles, and Mines
                                                                                                                 ST 6.2.7       Conduct Personnel Recovery
                                                                                                                                       Analyze and Compare Theater Courses of 
                                                                                                                                             Gain and Maintain Situational 
                                                                                                                                Integrate Theater Intelligence Capabilities 
                                                                                                                         ST 5.3.1.2  ActionUnderstanding (SU)
                                                                                                                                 OP 2.5 
                                                                                                                                with Operations
                                                                                                                 ST 2.8  ST 6.2.7      Conduct Personnel Recovery
                                                                                                                                             Analyze and Compare Theater Courses of 
                                                                                                                 OP 5.3.5       Analyze Courses of Action
                                                                                                                                       Integrate Theater Intelligence Capabilities 
                                                                                                                                 ST 5.3.1.2  Action
                                                                                                                         ST 2.8 Compare Courses of Action
                                                                                                                 OP 5.3.6              with Operations
                                                                                                                                 ST 6.2.7    Conduct Personnel Recovery
                                                                                                                           OP 5.3.5     Analyze Courses of Action
                                                                                                                                              Integrate Theater Intelligence Capabilities 
                                                                 2.2.5

                                                                         2.2.5
                                                                                    Select & Brief COA

                                                                                          Select & Brief COA
                                                                                                                     OP 5.3.7    Select or Modify Course of Action
                                                                                                                                 ST 2.8 Compare Courses of Action
                                                                                                                           OP 5.3.6 
                                                                                                                                 OP 5.3.5 
                                                                                                                           OP 5.3.7     Select or Modify Course of Action
                                                                                                                                 OP 5.3.6 
                                                                                                                                              with Operations
                                                                                                                                              Analyze Courses of Action
                                                                                                                                              Compare Courses of Action
                                                                                                                                                                                              OVs, BPMs
Mission-to-task                                                                  2.2.5          Select & Brief COA               OP 5.3.7     Select or Modify Course of Action




                                                                                          Tasks, UJTs
requirements driving
prioritized functionality      Requirement
                                statements
specifications                                                                                                                                                                                                          Functionality
                                                                                                                                             Systems / Services Perspective

Functionality dependent
upon SOA interactions,
standards, discovery                          System Views
                                                                                                                                                                                                          Service
                                                                                                                                                                                                          Exchange
                            Interactions,                                                                                                                                                                 Matrix
mechanisms                  Standards, &
                             Discovery                                                                                                                                                                                 Infrastructure
                                                                                                                                                                                                                     capability/capacity
Success dependent on                                                                                                                                             Infrastructure Perspective
an iterative vertical                                                                    Meta-data &
                                                                                         Data Stores
communication,
alternative analysis,
trade-offs, and
                                               Component Model
engagements                                                                                                                                                                           Coding               Network

                              Vertical
                              integration                                                                                                                                                                                      7
   UNCLASSIFIED
        Standard “V” System Engineering
                 Development

Concept of Operations                     Replicated operational environment       Operational Testing
   • Requirements                               feedback                             • Acceptance
                                                                                     • Sustainment
                Requirements              Meeting system requirements
                                                                                  System Integration,
               Decomposition &
                                                                               Verification, and Validation
                Architectures
                                          Meeting standard profiles

                            Detailed Design                        Component Testing
   Pr




                              • Allocated                            • Interfaces and
     og




                                                                                                       n
                             Requirements                        standards conformance




                                                                                                   tio
        ra
          m




                                                                                                    a
                                                                                                 gr
               De




                                                                                               te
                fin




                                                                                             In
                    itio




                                                                                           &
                                          Implementation
                        n




                                                                                          st
                                    • Acquisition & Development




                                                                                        Te
                                         Time and $$$                                                      8
UNCLASSIFIED
FOCUS
                       Notional SOA Implementation




                                                                                                                            Capability
                                                                                                                            Improved
           JSIDS / Operational Sponsor                                   Operational Testing / Exercises

         Concept of Operations              Replicated operational environment                    Operational Testing
            • Requirements                                                                          • Acceptance
                                                         feedback                                   • Sustainment


                  Requirements Decomposition &
                          Architectures                                          System Integration, Verification, and Validation
                                                   Meeting system requirements
        Pr
          og




                                                                                                                        n
                                                                                                                     tio
            ra




                                                                                                                     ra
              m




                                                                                                                 t eg
              De




                                    Detailed Design                           Component Testing




                                                                                                               In
                fin




                              • Allocated Requirements                    • Interfaces and standards




                                                                                                          t&
                   itio




                                                                                  conformance




                                                                                                           s
                       n




                                                                                                        Te
  Trade-offs
                                                       Implementation                                  JFCOM & Joint
                                                 • Acquisition & Development
                                                                                                       Assessment
                                             Acquisition Agent & Developer                             Engagement at
                                                                                                       Touch Points
                                                         Time and $$$
           Focus & Maturity                                                                      Improved Capability
                                                                                                                                9
  UNCLASSIFIED
             Lessons Learned and Consideration
Lessons Learned:




                                                                 FOCUS
                                                                 FOCUS
                                                                            JSIDS / Operational Sponsor                                      Operational Testing / Exercises




                                                                                                                                                                                         Capability
                                                                                                                                                                                         Improved
                                                                          Concept of Operations                                                                    Operational Testing
• Need common theme to develop centralize plan                               • Requirements                  Replicated operational environment                      • Acceptance
                                                                                                                                                                     • Sustainment
                                                                                                                          feedback
      •“unwillingness of the services … to agree to a
      joint command and control modernization that is                              Requirements Decomposition &
                                                                                           Architectures                                          System Integration, Verification, and Validation
                                                                                                                    Meeting system requirements
      centrally managed” (Senate language)




                                                                         Pr
                                                                         P
                                                                           og




                                                                                                                                                                                     n
      •“committee concerned that [DoD] has been




                                                                                                                                                                                  tio
                                                                             ra




                                                                                                                                                                                  ra
                                                                               m




                                                                                                                                                                                eg
                                                                               De
                                                                                                     Detailed Design                            Component Testing




                                                                                                                                                                             Int
      unable to develop a rational plan” (House




                                                                                  fin
                                                                                               • Allocated Requirements                     • Interfaces and standards




                                                                                                                                                                            &
                                                                                      iitio




                                                                                                                                                                           st
                                                                                                                                                    conformance




                                                                                           n
                                                                                           n




                                                                                                                                                                         Te
      language)                                                    Trade-offs
• NECC program did not establish an focus imperative                                                                     Implementation
                                                                                                                   • Acquisition & Development
  linkage direct requirement issues* to test/assess to
                                                                                                              Acquisition Agent & Developer
  baseline                                                                                                                Time and $$$

      •Test constructs not based on warfighter utilization
      of GCCS FoS or specific shortfall mission
      objective                                              Establish an operational Imperative
• The to-be migrated modules fell short of what the
  warfighter actually uses in the field today for mission    • Determine outcome-based focus
  accomplishment                                                          – Measurable joint issue
• Doctrinal level operational architecture develop
  concurrent and post-selection of modules and               •      Associate with task / mission
  functional to be enhanced.                                        objectives
• Late products and decision compressed time                              – Pre-defined business processes
  required to ‘flesh-out’ integrated architectures and
  cases                                                                   – Characteristics of specific Joint
• Operational response times for mission executions                         Scenario
  were not used as testing benchmark due to SOA                           – Identify ‘the user’ and ‘intended
  immaturity and integration with legacy baseline                           environment’
                                                             * NECC Requirements Integration Document (NRID) or the Global
                                                             Information Grid Requirements Integration Document (GRID) issues
                                                                                                                                                                                                  10
   UNCLASSIFIED
             Lessons Learned and Consideration
Lessons Learned:




                                                               FOCUS
                                                                          JSIDS / Operational Sponsor                                      Operational Testing / Exercises




                                                                                                                                                                                          Capability
                                                                                                                                                                                          Improved
                                                                        Concept of Operations                                                                    Operational Testing
• Maintain traceability of module development to “As-                      • Requirements                  Replicated operational environment                       • Acceptance
                                                                                                                                                                   • Sustainment
                                                                                                                        feedback
  Is” Functional Transition Plan (FTP) to identify and
                                                                                 Requirements Decomposition &
  integrate to current systems and functions.                                            Architectures                                          System Integration, Verification, and Validation
                                                                                                                  Meeting system requirements
• The decomposition of requirements and functional




                                                                       Pr




                                                                                                                                                                                    ion
                                                                         og
                                                                         o
  business processes (Mission Capability Area (MCA)




                                                                           ra




                                                                                                                                                                                  at
                                                                            am




                                                                                                                                                                                gr
                                                                                                                                                                              te
                                                                             De
                                                                                                   Detailed Design                            Component Testing




                                                                              ef




                                                                                                                                                                            In
                                                                                in
                                                                                             • Allocated Requirements                     • Interfaces and standards
  Business Process Models (BPM)) must be reusable




                                                                                                                                                                            &
                                                                                  itio




                                                                                                                                                                         st
                                                                                                                                                  conformance




                                                                                      n




                                                                                                                                                                       Te
  elements with trace-able to mission-task based                 Trade-offs
  driven requirements.                                                                                                 Implementation
                                                                                                                 • Acquisition & Development

• Minimize parallel processes: developers and                                                               Acquisition Agent & Developer

  operational subject matter experts working with                                                                       Time and $$$


  incomplete or not vetted information; i.e., interfaces,
  data sources, and component dependencies.
• Fluid baselines force fluid integration objectives and    Provide an integration construct
  plausible test objectives                                 • Reflect critical business processes in
• Developer specifications and waiver based on                architectures
  ‘engineering mission threads’ devoid on operational
  usage; increase risk consequences in mission usage        • Develop an initial Operational Concept
• Lack of up-front integrated architecture (mission           (Ops Con)
  thread) assessment resulted in assortment of limited         –Define mission profiles & conditions
  modules that are less enable to execute mission              –Identify intended environment (as-is) and
  tasks (i.e., operational demonstration of end-to-end          benchmarks
  mission thread across enterprise and service
  infrastructure)
                                                            • Collaboratively, define
• ‘Tailoring’ of DoD Architecture Framework (DODAF)            –Dependencies and linkage to FTP
  provided inadequate community understanding and              –Program recognition of mutual multi-
  details to support integration and testing.                   program objectives (SoS requirements)

                                                                                                                                                                                          11
   UNCLASSIFIED
             Lessons Learned and Consideration
Lessons Learned:




                                                               FOCUS
                                                                          JSIDS / Operational Sponsor                                      Operational Testing / Exercises




                                                                                                                                                                                          Capability
                                                                                                                                                                                          Improved
                                                                        Concept of Operations                                                                    Operational Testing
• Interdependencies between program and core                               • Requirements                  Replicated operational environment
                                                                                                                        feedback
                                                                                                                                                                    • Acceptance
                                                                                                                                                                   • Sustainment

  enterprise services, capabilities, and infrastructure
                                                                                 Requirements Decomposition &
  created fundamental risks and coordinate challenges                                    Architectures
                                                                                                                  Meeting system requirements
                                                                                                                                                System Integration, Verification, and Validation



  (i.e., findings, resolutions, and synch schedules)




                                                                       Pr




                                                                                                                                                                                    ion
                                                                         og
                                                                          gr




                                                                                                                                                                                  at
                                                                            am
• Evaluate services in terms of their maturity to support




                                                                                                                                                                                gr
                                                                             m




                                                                                                                                                                              te
                                                                                                   Detailed Design                            Component Testing




                                                                             De




                                                                                                                                                                            In
                                                                               fiin
                                                                                             • Allocated Requirements                     • Interfaces and standards




                                                                                                                                                                            &
                                                                                   itiio




                                                                                                                                                                         st
  an engineering thread and/or operational thread                                                                                                 conformance




                                                                                                                                                                       Te
                                                                                        n
    • Include: understanding and evidence to meet                Trade-offs
                                                                                                                       Implementation
      enterprise interdependencies, linkage to FTP, and                                                          • Acquisition & Development


      other development dependencies.                                                                       Acquisition Agent & Developer

                                                                                                                        Time and $$$
• Design and selection to high level and “button-logy”
  abstractions exasperate issues to spread over abstract
  Mission Capability Areas (MCAs) and designs required
  to support specific focused operational missions.         Assess Maturity & Trade-offs, iteratively
• Not grouping services for task-driven assessment
  decrease ability to evaluate trade-off
                                                            • Evidence-based: inclusive of technical
    • Resulted in difficulty executing any one of more        maturity, limitations, and feedback
      than 800 Master Scenario Events List (MSELs)            –Experimentation, demonstrations and
• The key to ABC (Adopt-before-Buy, Buy-before-
  Create) approach requires ‘assessment’ of ‘product’          Prototyping
  adaptation in commercial best practices, architectures,   • Associate candidate solutions to focus
  and standards (& past performance) for C2.
• Piloting and promoting services with priority #1 and        –Use integrated architecture to identify
  priority #2 issues invites subsequent test headaches         gaps, shortfalls, outliers
• A collaborative ‘’sand-box; requires technical and
                                                              –Review candidate solutions; acquisition
  usage maturity to successfully access critical
  information, collaborate, sharing knowledge products,        efforts, schedules, funding, and risk
  and testing / validation infrastructure                     –Develop Mission-focus Analysis of
                                                               Alterative to support acq/user decisions
                                                                                                                                                                                          12
   UNCLASSIFIED
              Lessons Learned and Consideration
Lessons Learned:




                                                                   FOCUS
                                                                              JSIDS / Operational Sponsor                                      Operational Testing / Exercises




                                                                                                                                                                                              Capability
                                                                                                                                                                                              Improved
                                                                            Concept of Operations                                                                    Operational Testing

• Joint Engineering / Analyst and Testers require timely                       • Requirements                  Replicated operational environment
                                                                                                                            feedback
                                                                                                                                                                        • Acceptance
                                                                                                                                                                       • Sustainment


  access to program and technical information for pro-                               Requirements Decomposition &

  active engagement and mitigation                                                           Architectures
                                                                                                                      Meeting system requirements
                                                                                                                                                    System Integration, Verification, and Validation



• Full visibility required in piloting and testing events for




                                                                           Pr




                                                                                                                                                                                        ion
                                                                             og




                                                                                                                                                                                      at
                                                                               ra




                                                                                                                                                                                    gr
                                                                                 m
                                                                                 m
  collaboration and assess issues, second order effects,




                                                                                                                                                                                  te
                                                                                                       Detailed Design                            Component Testing




                                                                                 De




                                                                                                                                                                                In
                                                                                   fiin
                                                                                                 • Allocated Requirements                     • Interfaces and standards




                                                                                                                                                                                &
                                                                                       itiio




                                                                                                                                                                             st
                                                                                                                                                      conformance
  and impacts




                                                                                                                                                                           Te
                                                                                            n
                                                                     Trade-offs
• Early opportunity for operational SMEs and developers                                                                    Implementation

  discussions increase understanding on how technical                                                                • Acquisition & Development

                                                                                                                Acquisition Agent & Developer
  operations and technical services will be operationally                                                                   Time and $$$

  orchestrated and invoked.
• Without information exchange details, personal
  relationships and emails are not sufficient to
  communicate exchange paths required across JTF                Evaluate, Plan, and Perform early
  nodes and user-role actors necessary for testing                integration and feedback
    •Impromptu Orchestration Team working groups can
    not guarantee on-the-fly table top analysis, feedback       • Identify ‘white-box’, composite, and
    and thoroughness detailed to evaluate test cases              rapid feedback testing opportunities
• Use operational mission thread (OMT) matrix
  approaches to pro-actively trace operational activities-
                                                                  –Synchronize program schedules
  to-functionality-to-service to identify ‘holes’ in              –Plan for within Test and Evaluation
  information and gaps                                             Master Plan (TEMP)
• Lack of early assessment, table-top or limited pilot,
  prevents critical feedback and increases integration            –Anticipate joint context & use cases
  failures. Late issue discovery present root-cause             • Anticipate / Plan for regression testing
  determination challenges in larger end-to-end venues.
  Moreover, late issues are difficult to correct and              criteria – ToR and responsibilities
  integrate (likely across programs) late in development        • Use pre-defined business activity for
  cycles
                                                                  simulation & interaction modeling 13
   UNCLASSIFIED
                              Summary
• Less-cost, faster cycle, and right objectives
   – Requires clear objectives, actively balance, and meaningful
     engagement
   – If you don’t have a specific focus – you won’t meet it


• Provide an integration construct
   – If you don’t plan for integration – it won’t integrate later

• Assess maturity and trade-off
   – Use evidence-based, informed decisions
   – It seems all programs share similarities: behind schedule, over cost

• Plan and conduct early integration mitigation and feedback
   – Anticipated in schedule and program test strategy
   – Mutual perspective leads to success

                                                                       14
  UNCLASSIFIED

								
To top