Docstoc

Project Experimental Plan Document Template

Document Sample
Project Experimental Plan Document Template Powered By Docstoc
					   Capabilities Description for RPC




  CAPABILITIES DESCRIPTION

                FOR

RAPID PROTOTYPING CAPABILITY
FOR EARTH-SUN SYSTEM SCIENCES

             Version 1.0

       By RPC Project Team
     Mississippi State University
       Robert Moorhead, P.I.
        David Shaw, co-P.I.

            May 3, 2006




            Page 1 of 24
                                   Capabilities Description for RPC


                                        TABLE OF CONTENTS


1.     Scope ................................................................................................................... 3
1.1.     Document security .......................................................................................... 3
1.2.     Identification ................................................................................................... 3
1.3.     Document overview ........................................................................................ 3
1.4.     System overview ............................................................................................. 3
2.     References ........................................................................................................... 6
2.1.     Referenced documents .................................................................................... 6
2.2.     Acronyms and Definitions .............................................................................. 6
3.     Current situation.................................................................................................. 7
3.1.     Background, objectives, and scope ................................................................. 7
3.2.     Operational policies and constraints ............................................................... 8
3.3.     Description of current situation ...................................................................... 8
3.4.     Modes of operation for the current situation................................................... 9
3.5.     Users and other involved personnel ................................................................ 9
3.6.     Support environment ....................................................................................... 9
4.     Justification for and nature of changes ............................................................... 9
4.1.     Justification of changes ................................................................................... 9
4.2.     Description of desired changes ..................................................................... 10
4.3.     Priorities among changes .............................................................................. 11
4.4.     Changes considered but not included ........................................................... 11
5.     Concepts for the proposed system .................................................................... 11
5.1.     Background, objectives and scope ................................................................ 11
5.2.     Operational policies and constraints ............................................................. 13
5.3.     Description of the proposed system .............................................................. 13
5.4.     Modes of operation ....................................................................................... 19
5.5.     User classes and other involved personnel ................................................... 19
5.6.     Support environment ..................................................................................... 19
6.     Operational scenarios ........................................................................................ 19
6.1.     Scenario for idealized experiment evaluating data sources .......................... 20
6.2.     Scenario for idealized experiment evaluating models .................................. 21
7.     Summary of impacts ......................................................................................... 22
7.1.     Operational impacts ...................................................................................... 22
7.2.     Organizational impacts ................................................................................. 22
7.3.     Impacts during development ......................................................................... 23
8.     Analysis of the proposed system....................................................................... 23
8.1.     Summary of improvements ........................................................................... 23
8.2.     Disadvantages and limitations ...................................................................... 24




                                                   Page 2 of 24
                             Capabilities Description for RPC



1. Scope

The Rapid Prototyping Capability (RPC) is being developed by the Mississippi Research
Consortium (MRC) and its partners under contract to NASA Stennis Space Center.

Consider the following definitions in this document.

      “Current” means as of Fall 2005.

      The “proposed system” will be delivered by December 2007, and thus, will be
       operational by January 2008.

The development period (20062007) will include infrastructure development and
performing of example experiments that will validate the RPC requirements and verify
the infrastructure implementation.


   1.1. Document security

This document is not restricted.


   1.2. Identification

This document is identified by the title and effective date.


   1.3. Document overview

This document describes the capabilities of the Rapid Prototyping Capability. The
format of this document follows the outline of a Concept of Operations document, per
IEEE Standard 1362-1998. Major sections are (1) a description of the situation as of Fall
2005, (2) justification for change, (3) operational concepts for a proposed Rapid
Prototyping Capability, planned for January 2008, that includes both the activities of
people and supporting technology, (4) idealized operational scenarios for classes of rapid
prototyping experiments, (5) a summary of the impacts of the proposed system, and (6)
an analysis of the improvements and limitations of the proposed system.


   1.4. System overview

Within the systems engineering approach to NASA’s Earth Science Application Plan to
move science results through an evaluation phase and into implementation, the Rapid
Prototyping Capability fulfills a need to reduce the amount of time that has typically been
required to consider the utility of new or future data streams on model outcomes. The
basic functionality as shown in figure 1 consists of receiving inputs from the Solutions


                                        Page 3 of 24
                            Capabilities Description for RPC


Networks research results activities, performing an RPC experiment, and producing
information for use in potential follow-on Integrated Systems Solutions activities.




 Figure 1. In a fully implemented state, the SN shall enable NASA science results to
 be harvested, allowing the selection of results to move forward for RPC evaluation.
 Critical to the selection of an effort for RPC evaluation are the needs to identify
 successful science results, baseline the conditions of the current system, identify new
 or future NASA data streams that might mitigate or enhance the current solution
 system, define the existing model schema, develop an RPC evaluation team, identify
 stakeholders in the solution, identify a pathway to ISS, and develop the data and
 model resources needed to conduct the RPC evaluation.


Figure 2 depicts a functional diagram of the capabilities of a fully-functional Rapid
Prototyping Capability. A typical RPC experiment consists of due-diligence systems
engineering of an idea for transition from research to operations. Central to the due-
diligence process, science results and partner needs are considered and environmental
simulation models are selected along with existing data for baseline condition
representation. Also, the RPC process identifies desired synthetic/simulated data from
future sensors and/or simulation model output for enhanced/mitigated solution
evaluation. A collaborative effort is followed to integrate model and data sources into the
RPC for systematic evaluation. Model owners, agency experts, RPC developers and
model scientists, and research evaluators collaborate to evaluate the performance of the
baseline model against simulations that leverage existing NASA data sources, model
derived data, and/or simulated data sets. The result of the experiment is an evaluation of
whether a specified NASA science result is likely to yield an operational benefit for a
partner agency and an identified pathway to a potential ISS implementation.




                                       Page 4 of 24
                            Capabilities Description for RPC




Figure 2. RPC Overview.

Figure 2 illustrates the RPC as a functionally implemented node in the context of a
workspace at NASA Stennis Space Center with a collection of technology tools for a
multidisciplinary team to perform an RPC study in a face-to-face working environment.

Although interfaces to configuring an RPC activity and evaluating outcomes are expected
to occur interactively in real time, it is not expected that derivation of synthetic data,
integration of new model-base capabilities, or model computational processing will occur
on an interactive, real-time basis. The RPC capabilities are envisioned as supporting an
evaluation workflow, some components of which will occur on a real-time or
synchronous basis and some of which will involve asynchronous activities and follow-up
task completion prior to proceeding to the next step in the workflow.

The capabilities of the RPC include critical infrastructure components that interface with
and provide access to the NASA Enterprise Architecture tools through METIS, access to
the network of NASA data products and technology resources through the “SSC RPC
Gateway,” access to a variety of local and distributed modeling capabilities, and access to
simulated or synthetic data sources through OSSEs, ART, and other technologies to be
implemented to provide enhanced data sources for assimilation into and RPC evaluation.




                                       Page 5 of 24
                            Capabilities Description for RPC


2. References

   2.1. Referenced documents

[1] “IEEE Guide for Information Technology --- System Definition --- Concept of
Operations (ConOps) Document,” IEEE Standard 1362-1998.

[2] “Extending NASA Earth-Sun System Research Results through a Systems
Engineering Capacity” (Working Document), version SEC_V_5.

[3] Mississippi State University, “ Rapid Prototyping Capability for Earth-Sun System
Sciences” (proposal), Principal Investigators Robert J. Moorhead and David R. Shaw.

[4] NASA Stennis Space Center, “Statement of Work: Solicitations for Proposals through
the Mississippi Research Consortium (MRC) for Competitive Selection of Projects to
Establish and Evolve Applied Sciences Systems Engineering Capacity in the Functional
Areas of Solutions Networks, Rapid Prototyping Capability and Integrated Systems
Solutions,” Sept. 9, 2005.

[5] NASA Rapid Prototyping Workshop, Hampton, Virginia, April 1920, 2006.
Presentation materials.


   2.2. Acronyms and Definitions

DAAC           Distributed Data Active Archive Center
DSS            Decision Support System
DST            Decision Support Tools
ISS            Integrated Systems Solutions
MRC            Mississippi Research Consortium
OSSE           Operational Spacecraft Simulation Experiment
R2O            Research to Operations
RPC            Rapid Prototyping Capability
SN             Solutions Networks

All acronyms may be interpreted as singular or plural.

Consider the following definitions.

      “Experiment” means an evaluative study for NASA conducted using the RPC. A
       typical study evaluates an innovative combination of data sources and scientific
       models.

       “Data source” means any source of data used in an RPC experiment, such as
       Distributed Data Active Archive Center (DAAC) data products, other spacecraft
       observations, and simulated spacecraft observations (OSSE)



                                      Page 6 of 24
                            Capabilities Description for RPC


      “Model” means a scientific model used in an RPC experiment. Many models
       provide predictive simulations of natural phenomena.

      “Systems engineering” means the application of engineering principles to analysis
       and implementation of the transition from research to operations.

       “Experiment team” means a multidisciplinary team of scientists and systems
       engineers that facilitates and conducts an RPC experiment.

      “Steering committee” means a NASA organizational mechanism for approving
       and prioritizing RPC experiments.

      “Technical working group” means a NASA multidisciplinary team to oversee the
       technical evolution of the RPC infrastructure.



3. Current situation

   3.1. Background, objectives, and scope

The following is quoted from the RPC SOW [4].

NASA strategic goals and corresponding objectives are defined in “The New Age of
Exploration” released by the Agency in February 2005. The guiding objectives for the
NASA Earth-Sun System Division Applied Sciences Program are:

National Objective 5
Study the Earth system from space and develop new space-based and related capabilities
for this purpose.

NASA Objective 14
Advance scientific knowledge of the Earth system through space-based observation,
assimilation of new observations, and development and deployment of enabling
technologies, systems, and capabilities, including those with potential to improve future
operational systems.

NASA Objective 15
Explore the Sun-Earth system to understand the Sun and its effects on Earth, the solar
system, and the space environmental conditions that will be experienced by human
explorers, and demonstrate technologies that can improve future operational systems.

The NASA Applied Sciences Program implementation strategy is aligned with these
objectives. The results of NASA Earth-Sun system science include, but are not limited to
the following.

      NASA research spacecraft and their observations of the Earth-Sun system;


                                      Page 7 of 24
                                      Capabilities Description for RPC


        NASA models and their predictive capabilities for weather, climate and natural
         hazards; and

        Published improvements to scientific knowledge of the Earth-Sun system.


NASA results are extended in two ways: 1) through the transition of research results to
operational utilization, and 2) through projects that extend NASA research results into
integrated system solutions for specific application areas of national priority (identified in
the NASA Earth Science Applications Plan).

NASA has designed the system engineering approach shown in Figure 3 to applications
comprising the process steps of Evaluation, Verification and Validation (V&V), and
Benchmarking.


                    Identify
                                                            Design &                                       Improved
   Research      Requirements            Ev aluate                           Verify &
                                                           Implement                         Benchmark
   Capability          &              NASA System                         Validate NASA
                                                            Integrated                        DSS with     Operational
                Specifications         Components                           Outputs as
                                                             System                          NASA Inputs
   Selection    of DSS Inputs &       as DSS Inputs                        DSS Inputs                        System
                                                             Solutions
                    Outputs



                    Refine               Refine              Refine          Refine            Refine




                                  Evaluation                                V&V           Benchmarking
       Use of Systems Engineering principles leads to                    The Verification and Validation(V&V)
       scalable, systemic, and sustainable solutions and                 phase includes measuring the
       processes, which in turn contribute to the                        performance characteristics of data,
       success of the mission, goals, and objectives of                  information, technologies, and/or
       each National Application.                                        methods, and assessing the ability of
       The Evaluation phase involves understanding the                   these tools to meet the requirements of
       requirements for and technical feasibility of Earth               the DSS.
       science and remote sensing tools and methods                      In the Benchmarking phase, the
       for addressing DSS needs.                                         adoption of NASA inputs within a DSS
                                                                         and the resulting impacts and outcomes
                                                                         are documented in a Benchmark Report.



Figure 3. Systems engineering process used by the NASA Applied Sciences Program [2].


   3.2. Operational policies and constraints

The current situation conforms to normal NASA organizational relationships and
operations.


   3.3. Description of current situation



                                                      Page 8 of 24
                              Capabilities Description for RPC


The current situation is characterized by ad hoc transition from research to operations.
NASA sponsors a wide variety of projects that are aimed to facilitate the transition of
specific science results to operations. Each has a different scope, covering various parts
of NASA’s systems engineering approach. Each has customized immediate goals. They
tend to be time-consuming (i.e. not rapid). The lack of a common infrastructure
conducting such experiments tends to aggravate costs, because each experiment must find
its own infrastructure.


    3.4. Modes of operation for the current situation

(Not applicable.)


    3.5. Users and other involved personnel

Stakeholders of the current NASA research to operations (R2O) process include the
following.

   Science Principal Investigators

   Partner agencies

   NASA Headquarters (Applied Science Program)

   NASA Centers


    3.6. Support environment

Not applicable currently, because of the ad hoc approach.


4. Justification for and nature of changes

    4.1. Justification of changes

Figure 4 shows the importance of trying to demonstrate the utility of innovative solutions
prior to initiating budget support for operational use. If a partner agency waits until a
mission is flying to evaluate the utility of the science results from the mission, then it will
be too late to take full advantage of the mission.




                                        Page 9 of 24
                                 Capabilities Description for RPC




Figure 4. Research solicitation vs. mission cycle [2].
The applications community needs to begin to explore potential applications early using
Operational Spacecraft Simulation Experiment (OSSE) prior to the beginning of a
mission. By using this resource in conjunction with the RPC, the operations community
could potentially accelerate the use of the application [2].

The RPC will significantly accelerate the evaluation of mission science results, and
thereby make budget support for a mission possible much earlier than under the current
situation.


   4.2. Description of desired changes

Figure 5 depicts the proposed approach for transitioning NASA science results to
operations. This Capabilities Document addresses only the Rapid Prototyping Capability.

   Research supply                                                             demand
  and Analysis                         Applied Sciences
                                           Program                                      Operations
    Program
                                 Crosscutting                     National           Government
                                  Solutions                     Applications          Agencies
                                                                                         &
                                                                                      National
                                                                                    Organizations



                      Scientific                   Rigor
        NASA
                                        Rapid                     Integrated              Societal
      Earth-Sun      Solutions
                                     Prototyping                    System
       System        Network                                                              Benefits
                                      Capacity                    Solutions
      Research
                    Uncertainty                    Analysis


                                    •Evaluation • Verification •Benchmarking
                                                      and
                                                   Validation

Figure 5. Transitioning from research results to operations and societal benefits [2].


                                          Page 10 of 24
                            Capabilities Description for RPC




“Solutions Networks systematically examine the portfolio of results from NASA funded
research in the seven science focus areas of the Earth-Sun System Division to find
candidates that may be transitioned from research to operations or that could be
integrated into solutions with specific decision support systems” [4].

“Rapid Prototyping Capability [experiments] systematically evaluate research
capabilities, based on the use of specific research results in a simulated operational
environment in order to evaluate components and/or configurations that could be
considered for verification, validation, and benchmarking for transition from research to
operations and/or into an integrated system solution” [4].

“Integrated System Solutions extend the benefits of NASA research by following a
rigorous systems engineering process with our federal partners to evaluate (if necessary),
verify, validate, and benchmark the assimilation of NASA research results into their
decision support system(s)” [4].


   4.3. Priorities among changes

(Not within the scope of this document.)


   4.4. Changes considered but not included

(Not within the scope of this document.)


5. Concepts for the proposed system

   5.1. Background, objectives and scope

A Rapid Prototyping Capability within NASA is needed to accelerate the transition from
research to operations [3]. Operational models have strict requirements that must be
verified and validated before they can become accepted by the operational partner
agency.     Currently, Operational Spacecraft Simulation Experiments (OSSE) are
completed before any potential uses are investigated. This leads to insufficient time to
investigate, verify and validate, and benchmark the science results from a sensor. By
beginning rapid prototyping during the development of the OSSE, the time for transition
from research to operations can be reduced. The rapid prototyping concept has been
successfully applied in industry, at NOAA, and at DoD. The proposed Rapid Prototyping
Capability is intended to fulfill a similar purpose in the NASA research to operations
flow.




                                      Page 11 of 24
                            Capabilities Description for RPC


The proposed RPC infrastructure will allow multidisciplinary experiment teams,
including model developers and owners, to systematically evaluate research capabilities,
based on using NASA science research results in a simulated environment. Components
and/or configurations that could be considered for an Integrated Systems Solution project
will be evaluated. Results of NASA Earth-Sun system science include but are not limited
to the following.

      NASA research spacecraft and their observations of the Earth-Sun system

      NASA models and their predictive capabilities for weather, climate, natural
       hazards, etc.

      Published improvements to scientific knowledge of the Earth-Sun system

Figure 6 depicts the place of the Rapid Prototyping Capability in the context of the
NASA Applied Science research-to-operations flow. One should note the wide variety of
stakeholders who have an interest in this critical capability.




                                       SN   RPC    ISS
                                       N




Figure 6. RPC problem space [5, Marley].




                                     Page 12 of 24
                           Capabilities Description for RPC




   5.2. Operational policies and constraints

An approved plan for an RPC experiment is the key input from Solutions Network. We
envision the following operational process.

   1. The Solutions Network activity will identify NASA science results that have good
      potential to add value to partner DDS. This will result in proposed RPC
      experiments. Other activities may also propose RPC experiments.

   2. The RPC steering committee will approve and prioritize proposed RPC
      experiments from time to time.

   3. A multidisciplinary RPC experiment team will be formed.

   4. Necessary resources will be scheduled, including the RPC infrastructure, the RPC
      workspace, and relevant external resources.

   5. The RPC experiment team will conduct the experiment.

   6. Results of RPC experiment will be useful when formulating a solicitation for the
      follow-on Integrated Systems Solutions effort.


   5.3. Description of the proposed system

The RPC shall provide an extensible architecture for defining the baseline operation of
current applications, rapidly integrating new data sources, creating model-ready input
data, integrating environmental simulation models, configuring model runs with typical
data inputs (baseline) and new sources of data, and evaluating the performance of the
environmental simulation model comparing the results of the model in its typical
(baseline) configuration against results derived from the use of new data sources. The
RPC shall be modular in its design and implementation and shall provide the ability to
develop extensible use cases for environmental simulation models which have been
integrated within the RPC framework. A suite of tools and functions will be implemented
within RPC modules for managing the creation of scientific data from NASA and other
data streams (SDM), manipulating scientific data to provide model ready input data
through an interoperable geoprocessing engine (IGE), configuring and managing
environmental simulation model runs as well as conducting or managing model
computation through the model manager (MM), and providing an environment for
generating performance metrics in a workbench (PMW) environment to evaluate model
performance with various inputs.




                                    Page 13 of 24
                            Capabilities Description for RPC


NASA Stennis Space Center has designated a work space, shown in Figure 7, to host
teams running RPC experiments. Although the RPC infrastructure will support virtual
work groups to some extent, face-to-face meetings of interdisciplinary teams with readily
available technology support will be a vital part of RPC experiments.




         Figure 7. RPC room at NASA/SSC.



As noted in the RPC System Overview (section 1.4),

       “The RPC capabilities are envisioned as supporting an evaluation workflow, some
       components of which will occur on a real-time or synchronous basis and some of
       which will involve asynchronous activities and follow-up task completion prior to
       proceeding to the next step in the workflow.”

Figure 8 provides an illustration of the typical workflow and personnel components of an
envisioned RPC process that moves in stages RPC Selection to RPC Preparation to RPC
Evaluation all supported by ongoing RPC Development (including evolution and
maintenance) capabilities which must underpin the the RPC to support new designated
evaluation. Communications and collaboration are key components that will ensure that
needed models and data are successfully integrated, implemented, and RPC systems
capabilities are configured to manage the evolving integrated capabilities.




                                     Page 14 of 24
                           Capabilities Description for RPC




Figure 8. The capabilities of the RPC are expressed in a step-wise workflow that begins
with a selection process, is followed by preparation activities, and ends with an
evaluation, all of which are underpinned by activities of the RPC developers.



                                    Page 15 of 24
                            Capabilities Description for RPC


The RPC shall comprise a collection of capabilities that will enable experiments to
efficiently evaluate the usefulness of various existing or future (simulated) data streams
or model derived output data streams to enhance a decision support system or simulation
model. Figure 9 depicts the functional structure of the Rapid Prototyping Capability. The
following section describes key modular component blocks of this diagram.




     Figure 9 Functional overview of the RPC showing modular component
     blocks of the system and indicating interactions and interfaces between
     major modules.


                                      Page 16 of 24
                            Capabilities Description for RPC


Science Data Manager (SDM): NASA Science Data are selected for evaluation use in
specific models or DSS tools. The SDM accesses the data from the NASA Earth-Sun
observation sources. The sources may include the Distributed Active Archive Centers
(DAAC’s), Observing System Simulation Experiments (OSSE), and other data that may
be simulated through the Application Research Toolkit (ART) and similar systems. In
this component of the architecture, the metadata will be developed and maintained and
will be in compliance with the NASA Enterprise Architecture (EA). The SDM will
supply input to the Earth Observation section of the Earth-Sun System Architecture Tool
(ESAT).

Interoperable Geoprocessing Environment (IGE): The IGE will employ semantic
definitions to encode processing tasks that conduct needed geoprocessing and formatting
to provide model-ready input data.

                                 Semantic Definitions:
           •   Define geoprocessing methodology
           •   Give meaning to the terminology of the environmental model
           •   Remove uncertainty about the creation of information
           •   Allow the IGE to represent instances of the concepts
           •   Allow the IGE to produce descriptions of instances (parameters)

The IGE will comprise capabilities that include “libraries” of semantic definitions that
encapsulate processing concepts, systematic encoding of model knowledge into metadata,
and knowledge handling infrastructure. The fundamental capability of the IGE is to
translate through geoprocessing, manipulation, and formatting the data products that
measure the appropriate geophysical parameters into model-ready data inputs needed for
the models being exercised. The data from the NASA sources will be processed to extract
the specific information needed for the model(s) under study. The suite of atmospheric
and land models, such as the NASA Land Information Systems (LIS) and the Weather
Research and Forecasting (WRF) model, require a suite of initialization and forcing fields
that are routinely produced by operational centers such as NCEP and ECMWF as well as
by the NASA Data Assimilation activities that involve other regional and global models.
The RPC will facilitate timely access to all these data sources via appropriate protocols
and mechanisms. The implementation of the access and data transport mechanisms as
well as the schedule and volume of data transferred will be coordinated with the
operational and research agencies as necessary.

The IGE component will be a major contributor to performing experiments rapidly. A
significant capability to be understood is the reusability of configured geoprocessing
tasks to provide model-ready input data to a model that has been fully integrated into the
RPC. It is this “reuse” capability that will enable the rapid evaluation of new data types.
By associating existing geoprocessing workflows with new data types, the rapid
assimilation of next-generation data into configured models should be readily achievable.

The IGE includes two sub-components; Model Data Resource Manager and Data
Assimilation Tools. The Data Assimilation Tools will include transformation,



                                      Page 17 of 24
                            Capabilities Description for RPC


manipulation, extraction and other pre-processing tools, needed to produce the
geophysical parameters input needed for the Model Manager component of the
architecture. The requirements of the model and the processing steps required to produce
the input will be stored in a database and used to ensure the quality of the data produced.

The Model Data Resources Manager will provide documentation and cataloging of the
model requirements in a database to identify requirements that are common to multiple
models, and speed development of additional prototypes that require the same input data.
The database developed will supply input to the Data Products section of the NASA
Enterprise Architecture tool.

Model Manager (MM): Science results and partner agency needs drive the selection of
models and decision support tools to be evaluated in the RPC environment. In the RPC
workflow process, the selection of a particular evaluation requires that model capabilities
to perform the evaluation are available and configured for the desired data stream to be
considered in the evaluation. The selection task should trigger the following critical
capabilities questions:

           1) Is the desired model or decision support tool integrated into the RPC
              model manager along with the necessary “model knowledgebase?”
           2) Are the data desired for evaluation configured for ingest and assimilation
              within the RPC?
           3) Are the needed geoprocessing tasks to create model ready data from
              desired evaluation input data streams fully implemented?

The MM component of the proposed architecture also has two sub-components that
correspond with the Knowledge Base: the Model Base Manager and Model Results
Resource Manager. These components correspond to the Model and Analysis Systems
and Model Outputs/Predictions parts of the Knowledge Base. The Model Manager
component will be coordinated with the Earth System Modeling Framework (ESMF) to
contribute to the building of an infrastructure designed to increase the performance,
portability, interoperability, and coordination in the modeling community.

Performance Metrics Workbench (PMW): Environmental simulation models, decision
support tools, and custom data product outputs are systematically stored and tested
against baseline outputs and outputs created from contrasting test configurations.
Performance parameters are computed and examined utilizing visualization methods as
well as tabular, graphical, and statistical measures. The PMW will provide tools for
assessing whether there is significant improvement in a model’s performance. The
results of the model using NASA data will be compared to the current results and the
difference evaluated. A key component of the PMW will be the visualization of the
model results. The Model Scenario Comparison Toolkit will provide capabilities that
include quantitative tools to evaluate model performance. These tools include the
statistical analysis of the numerical results to determine the significance of any
differences in outcome.




                                      Page 18 of 24
                            Capabilities Description for RPC


    5.4. Modes of operation

The RPC as a whole will have a single mode of operation, customized for support the
experiment at hand.

On a technical level, each tool will have its own mode of operation.


    5.5. User classes and other involved personnel

Stakeholders of the proposed Rapid Prototyping Capability include the stakeholders of
the current NASA applied science systems engineering process, plus the organizations
involved in implementation and support of the RPC.

   Science Principal Investigators

   Partner agencies

   NASA Headquarters (Applied Science Program)

   NASA Centers

   NASA Stennis Space Center

   Mississippi Research Consortium team

       Mississippi State University

       University of Mississippi

       Science Systems and Applications Inc. (SSAI)

       Institute for Technology Development (ITD)


    5.6. Support environment

To be determined. Contracts have not been awarded for the period after December 31,
2007.


6. Operational scenarios

This section provides scenarios that show how the Rapid Prototyping Capability could be
used to perform experiments that will facilitate the transition of NASA science results to
operational benefits through partner agencies. Each scenario is a use case.




                                       Page 19 of 24
                            Capabilities Description for RPC


   6.1. Scenario for idealized experiment evaluating data sources

Use Case: Perform experiment using multiple data sources and a single model
--------------------------------------------------
CHARACTERISTIC INFORMATION
Goal in Context:           Evaluate whether a specified NASA data source, compared to a
baseline data source, can add value to a specified numerical model
Scope: This use case includes activities by people, processing by computational
resources, and communications.
Level: Top level
Preconditions:
                           1. Complete RPC experiment plan
                           2. Formation of RPC systems engineering team for this
                                experiment, including data and modeling specialists
                           3. Access to appropriate external resources (e.g. data, models,
                                host computers)
                           4. Availability of RPC node(s) for needed timeframe
Success End Condition: Completed RPC evaluation report
Failed End Condition: Experiment not performed
Primary Actor:             RPC systems engineering team,
Trigger:                   Management approval and schedule
----------------------------------------
MAIN SUCCESS SCENARIO
1. Verify access to appropriate external resources.
2. Acquire baseline data set.
3. Acquire or produce experimental data set.
4. Preprocess baseline data set, if necessary, for compatibility with the model. (e.g.
     transformation, manipulation, extraction, scaling, interpolation, reformatting)
5. Preprocess experimental data set for compatibility with the model.
6. Run the model for the baseline data set.
     6.1. Configure the model for the baseline data set.
     6.2. Run the model.
     6.3. Acquire baseline model run results.
7. Run the model for the experimental data set.
     7.1. Configure the model for the experimental data set.
     7.2. Run the model.
     7.3. Acquire experimental model run results.
8. Compare results of model runs.
9. Analyze the uncertainty of modeling results.
10. Evaluate potential value of experimental data source to target DSS.
11. Publish scientific results of the experiment in a peer-reviewed venue.
12. Submit results to the ESAT developers.
----------------------
EXTENSIONS
Some experiments will have a more complex configuration of experimental conditions.
--------------------



                                      Page 20 of 24
                            Capabilities Description for RPC


SUB-VARIATIONS
(none)
----------------------
RELATED INFORMATION (optional)
Priority:                  This scenario is a core RPC function.
Performance                Target: a few weeks to a few months depending on experiment
details
Frequency:                 One scenario per experiment
Superordinate Use Case: (none)
Subordinate Use Cases: (none)
Channel to primary actor: RPC node facilities
Secondary Actors:          (specified by RPC experiment plan)
Channel to Secondary Actors: (specified by RPC experiment plan)
----------------------------


   6.2. Scenario for idealized experiment evaluating models

Use Case: Perform experiment using baseline data sources and multiple models
--------------------------------------------------
CHARACTERISTIC INFORMATION
Goal in Context:           Evaluate whether a specified model or model configuration,
compared to a baseline model, using baseline data sources, can add value to a specified
decision support application.
Scope: This use case includes activities by people, processing by computational
resources, and communications.
Level: Top level
Preconditions:
                           1. Complete RPC experiment plan
                           2. Formation of RPC systems engineering team for this
                                experiment, including data and modeling specialists
                           3. Access to appropriate external resources (e.g. data, models,
                                host computers)
                           4. Availability of RPC node(s) for need timeframe
Success End Condition: Completed RPC evaluation report
Failed End Condition: Experiment not performed
Primary Actor:             RPC systems engineering team,
Trigger:                   Management approval and schedule
----------------------------------------
MAIN SUCCESS SCENARIO
1. Verify access to appropriate external resources.
2. Acquire baseline data sets.
3. Preprocess baseline data set, if necessary, for compatibility with the baseline model.
     (e.g. transformation, manipulation, extraction, scaling, interpolation, reformatting)
4. Preprocess baseline data set for compatibility with the experimental model.
5. Run the baseline model for the baseline data set.



                                      Page 21 of 24
                            Capabilities Description for RPC


     5.1. Configure the baseline model.
     5.2. Run the model.
     5.3. Acquire baseline model run results.
6. Run the experimental model using the baseline data set.
     6.1. Configure the experimental model for the baseline data set.
     6.2. Run the model.
     6.3. Acquire experimental model run results.
7. Compare results of model runs.
8. Analyze the uncertainty of modeling results.
9. Evaluate potential value of experimental model to target DSS.
10. Publish scientific results of the experiment in a peer-reviewed venue.
11. Submit results to the ESAT developers.
----------------------
EXTENSIONS
Some experiments will have a more complex configuration of experimental conditions.
--------------------
SUB-VARIATIONS
(none)
----------------------
RELATED INFORMATION (optional)
Priority:                  This scenario is a core RPC function.
PerformanceTarget: a few weeks to a few months depending on experiment details
Frequency:                 One scenario per experiment
Superordinate Use Case: (none)
Subordinate Use Cases: (none)
Channel to primary actor: RPC node facilities
Secondary Actors:          (specified by RPC experiment plan)
Channel to Secondary Actors: (specified by RPC experiment plan)
----------------------------


7. Summary of impacts

   7.1. Operational impacts

The Rapid Prototyping Capability will be instrumental in the shift from the current ad hoc
approach to a systems engineering to transitioning research to operations.


   7.2. Organizational impacts

The following are impacts on organizational issues that will need resolution by the time
the RPC infrastructure has been developed (January 2008).

      NASA will need to establish an organizational mechanism for approving and
       prioritizing RPC experiments. In this document, we call such an organization a



                                      Page 22 of 24
                            Capabilities Description for RPC


       “steering committee.” An organizational mechanism for balancing the interests of
       multiple stakeholders will be essential to the long-term success of RPC.

      Each RPC experiment will need a temporary multidisciplinary team to facilitate
       and conduct the experiment. In this document, we call such an organization an
       “experiment team.” Each team will exist only for the duration of the experiment.

      The RPC infrastructure will need a multidisciplinary team to oversee the technical
       evolution of the infrastructure. In this document, we call such an organization a
       “technical working group.”



   7.3. Impacts during development

The experiments that will be conducted during development of the RPC (January 2006–
December 2007) will, in their own right, be useful to NASA for facilitating the transition
of research to operations.

The development of RPC (January 2006–December 2007) will not interfere with
NASA’s current research-to-operations transition efforts.


8. Analysis of the proposed system

   8.1. Summary of improvements

The proposed Rapid Prototyping Capability is targeted to provide the following
improvements.

      The RPC infrastructure will provide a suite of tools that will be useful for rapid
       conduct of RPC experiments across a range of observational systems and science
       models. Such a coordinated set of tools is not currently readily accessible to a
       RPC experiment team.

      The RPC facility at NASA Stennis Space Center will provide a place for an RPC
       experiment team to work face-to-face, and thus, expedite progress on the
       experiment. Such a facility is currently not commonly used by ad hoc
       experiments.

      The RPC experiments conducted during development (January 2006–December
       2007) will illustrate the long-term potential for rapid effective RPC experiments.




                                      Page 23 of 24
                            Capabilities Description for RPC


   8.2. Disadvantages and limitations

The proposed Rapid Prototyping Capability has the following limitations.

      Although simulations of future planned data streams is time consuming and
       expensive, one of the primary drivers for the RPC is the need to understand the
       usefulness of next-generation data products. Therefore, a central aspect of the
       RPC is the use of simulated or synthetic data streams to employ in applications
       for comparison against baseline outputs. In-so-far as is possible, existing tools
       that enable the rapid simulation of next-generation data streams will be employed
       to arrive at “rapid” answers to the usefulness questions that surround next-
       generation satellite sensor data streams and derived scientific data products.

      Substantial programming of software, such as major modification and recoding of
       existing models to make them RPC compliant, is outside the scope of RPC
       experiments, because such programming efforts will defeat the goal of “rapid”
       results.

      RPC experiments will perform due-diligence systems engineering of limited
       prototypes for the purpose of providing evidence of the practical value of NASA
       science results. In contrast, the Integrated Systems Solutions projects will address
       engineering issues relevant to verification and validation of the solution to fully
       meet the needs of partner agencies.

      The proposed system (January 2008) will consist of two computational nodes, one
       at NASA Stennis Space Center and one at Mississippi State University. The
       infrastructure will be designed for straightforward replication of nodes at
       additional networked sites.

      The proposed system (January 2008) will have one physical meeting facility at
       NASA Stennis Space Center. The design will allow straightforward replication at
       other locations.




                                      Page 24 of 24

				
DOCUMENT INFO
Description: Project Experimental Plan Document Template document sample