Slide 1 Naval Postgraduate School (PowerPoint)

Document Sample
Slide 1 Naval Postgraduate School (PowerPoint) Powered By Docstoc
					ASW METOC Metrics: VS07 Data Collector
            Workshop
                  Bruce Ford
            Clear Science, Inc. (CSI)
          bruce@clearscienceinc.com

                 Tom Murphree
        Naval Postgraduate School (NPS)
              murphree@nps.edu




             Brief for VS07 Training
                  19 June, 2007
Visualizing the Metrics Collection Process


     METOC                METOC                        Operational                 Operational
    Forecasts *         Observations                     Plans                     Outcomes




                 METOC                                               Operational
               Performance                                           Performance
                 Metrics                                               Metrics




                               Metrics of METOC
                             Impacts on Operational
                                 Performance



  * or other products     Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Overall Goals


• Collect that data


• Examine the process




                Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
General Data Collection Plan
•   Aboard each CV
    •  Mimic NOAT data collection role by collecting:
       •   Discrete data
       •   Verifying in situ data
       •   Recommendations (tactical and mitigation)
       •   Recommendation outcomes
       •   Customer measures of success
    •  Investigate other potential data sources, methods, customer
       measures of success, etc.

•   At RBC
    •   Monitor watch officer actions and log for potential data collection
    •   Monitor NOAT interactions
    •   Observe product generation process

•   Embedded with deployed MPRA TSC (MOCC)
    •  Mimic MPRA data collection by collecting:
       •   Data from each GREEN and PURPLE
       •   Discrete forecast elements
    •  Investigate other potential data sources, methods, customer
       measures of success, etc.
                        Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
General Data Collection Plan (continued)
•   Embedded with each major staff
    • Mimic NOAT data collection role by collecting all:
      • Recommendations (tactical and mitigation)
      • Recommendation outcomes
      • Customer measures of success
    • Investigate other potential data sources, methods, customer
      measures of success, etc.




                     Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Data Collector Responsibilities

•   Prior to exercise

    •   Train on data collection procedures (what you are doing now)

    •   Help improve data collection forms (spreadsheets)

    •   Participate via email in discussion/form
        preparation/arrangement-making, possible conference call

    •   Become familiar with proposed data collection system for
        NOATs and MPRA metrics nodes (briefs provided)




                              Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Data Collector Responsibilities
•   During the exercise

    •   Check in with RBC metrics coordinator when settled on scene
        and provide SIPR email address if available

    •   Record data on specially-prepared paper forms and
        spreadsheets
        • From METOC products
        • From briefs (be present at as many as possible!)
        • From interactions with staff
        • Process improvement recommendations
           • Metrics collection process
           • METOC support process
        • Record personal observations and suggestions
        • Assess how eventual metrics interface and process will
           impact future deployed NOATs
        • Strive to understand the ASW warfighter and how their
           success is measured

                             Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Data Collector Responsibilities
•   During the exercise (continued)

    •   Participate in twice-daily metrics collector chats (hosted by
        the RBC). Details TBD and will be sent to everyone.

    •   Submit spreadsheets to RBC for archival as soon as
        completed
        • Timing will differ by NOAT and NOAD/MOCC

    •   Stay vigilant regarding the overall goals and methods
        • Ask questions!
        • Out-of-the-box thinking is encouraged!

    •   Remain available to answer questions from metrics data
        coordinator (RBC)

    •   Keep in mind how the VS07 metrics effort fits within the
        larger ASW metrics project
        • Record and comment on processes (METOC and metrics)
        • Collect METOC impactseton VS07 DataWarfighter bruce@clearscienceinc.com
                              Ford B. al.,
                                           ASW Collection, May 07,
Data Collector Responsibilities
•   After the exercise

    •   Remain available to answer questions expeditiously via email
        or phone regarding data collection

    •   For those at SSC, plan on post exercise meeting TBA.




                         Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Brainstorming
•   Understanding how the VS07 metrics collection fits with the
    larger ASW metrics project

•   Metrics data collector day
    • Where and when to collect the best data
    • Positioning ourselves to collect obvious and subtle impacts

•   Data collection forms – Go over tab-by-tab
    • Improving the forms prior to the exercise
    • Improving them on the fly during the exercise




                     Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
  Overview of ASW Metrics Data Collection System
NOAT Metrics Node                                   Customer
                                                                                                    MPRA Metrics Node
                                                                     MEP
                                       Recco        Measures
                                                                    Builder            Green
                        Contact     Information     of Success
                      Information                                                                 Purple


            In Situ
             Data                                                                                          Freeform
                                                                                                           Data Entry


FCST/Anal                                                                                                          Quality
                                                                                                                   Control

                                                             Metrics
                                                             Server
   Exercise
  Intentions
                                                                                                             Watch
                                                                                                             Officer
                                                                                                              Log
             Planning
             Impacts


                                                                                                  NOAT
                             Exercise                                                             Survey
                             Outcomes               Flag                      Objective
                                                  Exercise                      Data
                                                  Records                     Collection
R&A Metrics Node                                                                                      RBC Metrics Node
METOC Data Source          Non-METOC Data Source


                                         Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Questions?




Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Backup slides




Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Why Bother With Metrics?

1. Develop and transition to operational use systems for:
   a. collecting data from METOC units and their customers
   b. quantifying METOC performance and impacts on customer
      operations
   c. modeling and predicting impacts of METOC support on
      war fighting operations

2. Identify methods for improving quality and efficiency of METOC
    support.

3. Recommend:
   a. focus directions for METOC resources
   b. methods to incorporate METOC into OPNAV assessments




                    Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Definitions

   Metrics: Objective, quantitative, data based measures of products
    and services. Examples:
     Metrics of product quality
     Metrics of effects of products on customers

   METOC Metrics: Metrics of METOC organization’s products and
    impacts. Two main types:
     Performance metrics: metrics of capacity, readiness, quality,
       efficiency / return on Investment
     Impacts metrics: impacts on warfare customer operations
       (planning, execution, post-assessment)

   Methods for Generating METOC Metrics
     Collect / analyze real world data on METOC and customer ops
     Model METOC and customer ops




                     Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
Hierarchy of Metrics
                                              Higher Level
   Metric Symposium                        (Navy-wide SLD accuracy)   CNMOC/Fleet
   Focus Space                                                          MetricsLarger Spatial and/or
                                                                                      Temporal Scale
                                                                                     (Exercise forecast location)
                                                               Directorate
                                                                 Metrics

                                                          NOAC
                                                          Metrics
    Performance
 (Temperature and salinity                                                                                   Impacts
       accuracy)                              Exercise                                              (Number of positively identified
                                               Metrics                                                      submarines)




                                       NOAT
                                       Metrics

                             Ind. Forecast
 Smaller Spatial
 and/or Temporal                Metrics
      Scale                                    Lower Level
 (Point forecast location)                   (MOATs SLD accuracy)



                                    Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
VS07 Opportunity
•   Offers the opportunity to simulate a larger metrics system by
    collecting and analyzing data as an established metrics program
    would

•   Offers an excellent opportunity to develop, test, and improve
    data collection and analysis process from multiple perspectives:
    • NOAT
    • RBC
    • MPRA
    • Exercise

•   Two key types of data to collect
    • Performance data (METOC and customer)
    • Impacts data
    • Process data




                     Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com
VS07 Data Collection Sites

Stationing of Data Collectors in Priority Order

1. Aboard each CV/ASWC NOAT

2. At RBC

3. Embedded with CTF-74

4. Embedded with deployed MPRA TSC (MOCC)

5. Embedded with 3rd Fleet staff




                      Ford B. et al., VS07 Data Collection, May 07, bruce@clearscienceinc.com

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:3/12/2012
language:English
pages:18