Get documents about "Data Stewardship NOAA's Programs for Archive_ Access_ and
Shared by: liuhongmei
-
Stats
- views:
- 2
- posted:
- 8/13/2011
- language:
- English
- pages:
- 15
Document Sample
Data Stewardship
NOAA’s Programs for Archive, Access,
and Producing Climate Data Records
John Bates, NOAA’s National Climatic Data
Center
Mitch Goldberg, NOAA’s Office of Research
and Applications
International TOVS Study Conference 14
1 May 25-31, 2005
Outline
• Background
• Defining Data Stewardship
• NOAA’s Comprehensive Large Array data
Stewardship System (CLASS)
• NOAA’s Scientific Data Stewardship (SDS)
Program
• Conclusions
• Need for International Partnerships - GEO
International TOVS Study Conference 14
2 May 25-31, 2005
International TOVS Study Conference 14
3 May 25-31, 2005
Principles of Data Stewardship
1. Archive and access to fundamental measurements,
products and metadata - CLASS
2. Data archaeology and improved use – CLASS/SDS
3. Careful monitoring of observing system performance
for long-term applications - SDS
4. Generation of authoritative long-term records through
validation of the calibration process, reprocessing,
product generation and the blending of in situ and
satellite measurements - SDS
5. Provide state of the environment information for
decision makers and place the current state in its
historical context - SDS
International TOVS Study Conference 14
4 May 25-31, 2005
CLASS Goals and Vision
NOAA's National Data Centers and their world-wide clientele of
customers look to CLASS as the sole NOAA IT infrastructure project in which all
current and future large array environmental data sets will reside. CLASS
provides permanent, secure storage and safe, efficient access between the Data
Centers and the customers.
– A web-based data archive and distribution system for NOAA’s
environmental data
– A combined process to both reengineer legacy data storage and
access systems and blend new and efficient technologies to ensure
the stewardship of existing (e.g., POES, GOES, NEXRAD, in-situ)
and rapidly approaching large-array data sets (e.g., NPP/NPOESS,
EOS, METOP, NEXRAD).
– An aggressive plan to safeguard, enhance, expand and automate
NOAA’s capability to ingest, store, quality control, preserve, and
access its vast environmental data holdings.
– A focused effort to ensure the Information Technology (IT)
infrastructure is in place and working, before the arrival of
significantly larger and more complex environmental data (e.g.,
5 NPP/NPOESS, and GOES-R). International TOVS Study Conference 14
May 25-31, 2005
Synergy of CLASS & SDS
+ =
CLASS + Scientific Data Stewardship = Climate Products and Information
CLASS – NOAA’s new archive; Comprehensive Large-Array data Stewardship System
International TOVS Study Conference 14
6 May 25-31, 2005
Background
• NOAA’s Scientific Data
Stewardship rooted
in NRC dialogue and
reports
• NOAA/NRC SDS leads
– Bates
– Goldberg
International TOVS Study Conference 14
7 May 25-31, 2005
Key Elements of a Successful
CDR Program
CDR Organizational
CDR Organizational CDR Generation Elements
CDR Generation Elements Sustaining CDR Elements
Sustaining CDR Elements
Elements
Elements •• High accuracy and stability of
High accuracy and stability of •• Available resources for
Available resources for
•• High-level leadership
High-level leadership FCDRs
FCDRs reprocessing CDRs as
reprocessing CDRs as
council
council •• Pre-launch characterization of
Pre-launch characterization of new information becomes
new information becomes
•• Advisory council to
Advisory council to sensors and lifetime monitoring
sensors and lifetime monitoring available
available
represent climate research
represent climate research •• Thorough calibration of sensors
Thorough calibration of sensors •• Provisions for feedback
Provisions for feedback
community and other
community and other from scientific community
from scientific community
•• Well-defined criteria for TCDR
Well-defined criteria for TCDR
stakeholders
stakeholders •• Long-term commitment of
Long-term commitment of
selection
selection
•• Fundamental Climate Data
Fundamental Climate Data resources for generation
resources for generation
•• Stakeholder involvement and
Stakeholder involvement and
Record (FCDR) Teams
Record (FCDR) Teams and archiving of CDRs and
and archiving of CDRs and
feedback for TCDRs
feedback for TCDRs
•• Thematic Climate Data
Thematic Climate Data associated data
associated data
•• Well-defined criteria for TCDR
Well-defined criteria for TCDR
Record (TCDR) Teams
Record (TCDR) Teams validation
validation
•• Use of in-situ data for validation
Use of in-situ data for validation
Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors
together with the ancillary data used to calibrate them.
Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by
blending satellite observations, in-situ data, and model output.
International TOVS Study Conference 14
8 May 25-31, 2005
Defining Scientific Data Stewardship
Notional Functions of
Scientific Data Stewardship for Climate
Scientific Data Stewardship
Real time and retrospective management of climate data
Network Performance Monitoring Climate Data Records Model Re-
analyses
Observations & Metadata
Reference Climate
Analyses and Archives Data Sets Quality
Quality Control (Reprocessing) Products
Feedbacks Climate
Analyses
Stewardship Teams
International TOVS Study Conference 14
9 May 25-31, 2005
Defining CDRs
Climate Data Records
Data (Direct & Remotely Sensed)
Time-tagged
Geo-Referenced
Sensor Data
Records (SDRs)
Converted to Homogenization
Bio-Geophysical and Calibration
Variables
Environmental
Data Records Fundamental
Climate Data
(EDRs)
Records (FCDRs)
Climate
Converted to Data
Bio-Geophysical Records or
Variables Homogenized
Time
Series
Thematic Climate
Data Records
(TCDRs)
International TOVS Study Conference 14
10 May 25-31, 2005
NOAA CDR Reprocessing -
Climate Central Requirements
– For reprocessing, SDS NOAA/CLASS Users
requires an IDPS-like system,
(Climate Central) to process: NOAA
SDS
RDR SDR CLIMATE
Multi-satellite FCORs & TCDRs ACQUIRE RDRs
CENTRAL
IMPROVED SDRs
– SDS interdependent with MULTISATELLITE FCDRs
CLASS, e.g., large data set
I/O PROCESS TCDRs
– CLASS and SDS mutually
PROVIDE
dependent ASSESSMENTS
IMPROVEMENTS
– Requires NASA & W&W NDE
linkage for enterprise solution
Products and Services
11 Provide FCDRs and TCDRs
NOAA’s Scientific Data Stewardship Program
FY06
Governance and Management Structure EDSM $2.5M
C2D2 $1 to $1.5M
Scientific Data NOAA Climate Program
Stewardship & Climate Board NOAA
C2D2 NCDC
Program ORA
NOAA Observing (5-10%)
Management System Council
Scientific Data NOAA
Climate
Stewardship Working
Science External
Advisory
CLASS Group (1-2%)
Board
Working Group
Research Climate Data Science Teams Research Operational CDR Generation & Data Mgmt. NOAA,
to CDR Generation CLASS Other
FCDR Teams TCDR Teams Operations
NOAA Agencies
Observations R&D Products NOAA IT
Other Agencies Infrastructure External
Scoping and
Universities operators Scientific
Requirement Services
Systems Theme Areas Private Sector (~90%)
International TOVS Study Conference 14
12 May 25-31, 2005
12 Currently exists FY05 FY06
NOAA’s Scientific Data Stewardship
Thematic Climate Data Records (TCDRs)
Sea Surface Temperature
NASA Future:
NOAA Research NPP/NPOESS
Operational Satellites
Satellites
Aqua NPP
TRMM
AMSR-E VIIRS/CMIS
AVHRR TMI
Reconfigurable & Scalable
NOAA
•Mandates L1b DATA Climate
•Mission Requirements •Climate Observations
•Capabilities & Analysis
MAIN PROCESSING SYSTEM Central
•Capability Gaps ID’d •Climate Forcing • Data from multiple satellite and
•Alternative Solutions •Climate Prediction
•Solution Selection
•Ecosystem Applications instrument types
•Regional Decision
•Acquisition & Support • Buoy, Ship & Ice Data L1-
L1-L3 DATA CLASS
Operations Approval
Data
NOAA Management & Staff Community Agreed
Standards and Protocols
Users
NOAA Scientific Science
PPBES Climate Data In Situ Data
Community
Program Stewardship
International TOVS Study Conference 14
13 May 25-31, 2005
13
Conclusions
• NOAA SDS is primary USA program for CDR
development
• NOAA leadership for sustained program is
required to fill lawful mandates
• User involvement in all aspects of SDS
International TOVS Study Conference 14
14 May 25-31, 2005
Need for International Partnerships
Group on Earth Observations GEO
International TOVS Study Conference 14
15 May 25-31, 2005
