Docstoc

Bennett

Document Sample
Bennett Powered By Docstoc
					Curating, and providing access to,
             EO data

   Victoria Bennett, Bryan Lawrence and
          colleagues, CEDA, STFC
      Rutherford Appleton Laboratory




              NCEO Annual Science Meeting
                 Oxford, 2-4 Sept 2009
                      Outline



Introduction to CEDA: NEODC/BADC
Highlights, example activities
Why archive your data?




              NCEO Annual Science Meeting
                 Oxford, 2-4 Sept 2009
                                      CEDA Objectives
                        (Centre for Environmental Data Archival)


•Operate key data centres: BADC, NEODC, IPCC-DATA.

•Exploit project opportunities to develop potential new services.

•Participate in environmental research programmes.

•Participate in appropriate curation and informatics activities.

•Develop, maintain, and share data management expertise.




                            NCEO Annual Science Meeting
                               Oxford, 2-4 Sept 2009
                                   Objectives - Data

Acquire, curate and provide access to:
• atmospheric and earth observation data from bodies such as the Met
Office, ECMWF and ESA.
• data collected in the Atmospheric and Earth Observation fields,
including that by related NERC facilities.


Acquire, curate and actively manage:
• key ancillary information for data management (metadata, controlled
vocabularies etc)




                          NCEO Annual Science Meeting
                             Oxford, 2-4 Sept 2009
                            Statistics 08/09

                                   BADC         NEODC
Total Users                           11,731        884
Identifiable Users                       1924       178
Downloading Data
Total Download Volume                   36TB       12TB

Total Number of Files            12,000,000      750,000
Downloaded
Helpdesk Queries                         3418       404




                 NCEO Annual Science Meeting
                    Oxford, 2-4 Sept 2009
           Examples: highlights & activities


•IPCC CMIP5
•Data Manipulation and Delivery
•(A)ATSR Archive
•MetOp data
•ARSF data archive




               NCEO Annual Science Meeting
                  Oxford, 2-4 Sept 2009
                                                 IPCC CMIP5
                                                  (Climate Model Intercomparison Project)

Supporting the data archive for
the IPCC AR5:
•MoU signed with PCMDI, MPIM.
•New storage commissioned (0.5 PB,
more due soon, ~ 1 PB by next year).
•Developing the “official” CMIP5
questionnaire.
•Developing procedures for data
ingestion (due to start soon). Working
with Met Office on a range of issues.




                                   NCEO Annual Science Meeting
                                      Oxford, 2-4 Sept 2009
                                Data Manipulation and Delivery




   Two views of the same HIGEM data: one in a new WMS                       Part of the UKCP
  portal client, and one in Google Earth, both served from a               interface (that went
    prototype Web Map Server (WMS) deployed at BADC.                        live in June 2009)


Aim: To deploy services which hide the increasing volume and heterogeneity of data from potential
users. Data users should be able to extract geographical, temporal and vertical subsets of data
according to their requirements in common formats, plot them, and carry out a range of “prescribed”
manipulations



                                      NCEO Annual Science Meeting
                                         Oxford, 2-4 Sept 2009
                              QESDI
  (QUEST Earth System Data Initiative)




NCEO Annual Science Meeting
   Oxford, 2-4 Sept 2009
                                                  (A)ATSR Archive




17- Years of ATSR1,2 and AATSR SST, reprocessed to consistent format, and
             using new algorithms (at RAL and ESA respectively).



                            NCEO Annual Science Meeting
                               Oxford, 2-4 Sept 2009
                                            NEODC Data Acquisition


Data Inputs required for NCEO science:
•Common requirement: easy access to Eumetsat MetOp
data (IASI, GOME-2 and AVHRR-3 instruments).
•Initially 16 TB IASI (L2 and L1b) data, 6 TB AVHRR-3
(L1b) data and 4 TB GOME-2 (L1b) data for 2007-2008.
•Archive will grow forward in time.
•Bulk orders of data on 800GB LTO-4 tapes, which have
been ingested and made available




                              NCEO Annual Science Meeting
                                 Oxford, 2-4 Sept 2009
                                     ARSF Archive
Airborne Remote Sensing Facility; Highlights for 2008/09:
    – 41 users downloaded digitised aerial photos, and 50 users
      downloaded data from other ARSF instruments (most popular
      NEODC dataset!)
    – New higher volume instruments coming online ...
    – New programme this year of rescuing older data (physically: 25
      boxes containing 1500 CDs, 700 DAT tapes, 600 Exabyte tapes, and
      electronically: migrating to HDF format with better metadata).




                         NCEO Annual Science Meeting
                            Oxford, 2-4 Sept 2009
                         Why archive your data?

• Data collection/dataset generation is expensive
• Long-term safekeeping
• Future re-use
• Knowledge exchange – commercial/educational exploitation
• Sharing data for scientific cooperation, but with agreed
  restrictions or limitations
• Academic recognition
• “Because NERC data policy says so”




                       NCEO Annual Science Meeting
                          Oxford, 2-4 Sept 2009
                             NCEO showcase datasets;
                                   examples:

•   Surface reflectance and aerosol optical depth from (A)ATSR
•   Land surface temperatures from (A)ATSR/SEVIRI
•   SST from (A)ATSR/AVHRR
•   Climatologies of phytoplankton/primary production
•   Multi-year O3 profiles from GOME-1, SCIA, GOME-2
•   Snow mass retrievals
•   Time series of ice thickness (Envisat, ICESAT, GRACE, Cryosat-2)
•   Global budget of volcanic sulphur and aerosol emissions


• & model output..
                              NCEO Annual Science Meeting
                                 Oxford, 2-4 Sept 2009
                       Datasets “appropriate for archival”

1. are useable (good documentation, useable format, unrestrictive
conditions of use, appropriate quality)

2. contain generally useful parameters (likely to be used outside the project
that created them)

3. are referred to in academic publications

4. have historical importance

5. are unique

                     Some, or all, of these should apply...


                                NCEO Annual Science Meeting
                                   Oxford, 2-4 Sept 2009
               Benefits of properly archived data

Benefits include...
• Visualisation
• Citation
• Discovery




                      NCEO Annual Science Meeting
                         Oxford, 2-4 Sept 2009
Reading e-SC/
CEDA work to
visualise NCEO data
archived at NEODC




                      NCEO Annual Science Meeting
                         Oxford, 2-4 Sept 2009
                                         OJIMS
OJIMS – Overlay Journal Infrastructure for
Meteorological Sciences




                    NCEO Annual Science Meeting
                       Oxford, 2-4 Sept 2009
                                     Conclusion

Lots of good reasons for data archival..

Benefits rely on following formatting and
  documentation rules (& making data available!)

NCEO needs “showcase datasets”

     badc.nerc.ac.uk           www.neodc.rl.ac.uk

                     NCEO Annual Science Meeting
                        Oxford, 2-4 Sept 2009

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:2
posted:3/7/2012
language:English
pages:20