Burnett_USA_ by lanyuehua

VIEWS: 5 PAGES: 85

									    WMO
                       WMO Regional Association V
                           Fifteenth Session
                            Bali, Indonesia
                         30 April – 6 May 2010

                 Proper Data Management
          Responsibilities to Meet the Global Ocean
          Observing System (GOOS) Requirements

                        Dr. William Burnett
               Data Management and Communications
               U.S. NOAA/National Data Buoy Center



1               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
           NOAA and Climate/Ocean Observations
    WMO


•    There is an increasing demand for global climate change & ocean information,
     services and products – this includes observations from the Indonesian GOOS
     (InaGOOS) and the Indian Ocean equatorial array (RAMA),

•    Partnerships enhance ocean observations in the region and also build the regional
     capacity to apply these observations to understand climate risk management, coastal
     resilience, ecosystems, MPAs, and other socio-economic benefits,

•    NOAA is interested in advancing a strong, equitable and mutually beneficial
     collaboration with Region V for capacity building, socio-economic applications and
     ocean observations in the region, and

•    Training & education is important for countries building the next generation ocean
     observatories. NOAA is pleased to be able to provide USA training & educational
     opportunities through Memorandums Of Understanding or Agreements.




2
                   International Cooperation
    WMO             Tsunami/RAMA cruise




                                              RV Baruna Jaya III
                                                  Sept 2007
          Formal bilateral agreements between NOAA and agencies in:
              Indonesia--signed in 2007
              India--signed in 2008
              Japan--signed in 2008
              France--planned in 2009
              ASCLME (9 East African countries)


3
                The Initial Global Ocean Observing System for Climate
          Status against the GCOS Implementation Plan and JCOMM targets
    WMO



                                Total in situ networks   61% October 2007
                                                                   87%



                                                                   100%



                                                                   59%



                                                                   81%



                                                                   100%




                                           73%                     62%
             48%     34%


                                                                          Milestones
                                                                            Drifters 2005
                                                                             Argo 2007



4
          RAMA: Implementation Status
    WMO




          Resource Formula:
          Partners provide ship time
          NOAA provides most equipment




              57% of sites occupied by March 2010 (26 of 46)



5
                     NOAA’s Contributions to
    WMO
                   Indonesian GOOS (InaGOOS)
          NOAA is developing a replacement for the ATLAS system
           Measurements comparable to ATLAS
           Use more commercially available components
           Prototype deployment targeted for October 2010
           Deploy near existing ATLAS RAMA moorings for comparison
           As per IA, 2 systems would eventually serve InaGOOS and RAMA




                  15-20 Days




6
          Indonesia MMAF Visit to
    WMO
              NDBC July 2008




7
                 Currents Status
    WMO
          More global ocean observations


                                http://www.osmc.noaa.gov




8
               Current Status
    WMO
          Expansion in free data sets




9
           Current Status
     WMO
           “Climate-gate”




10
     WMO
              Proper Data Management

• More than just placing a meteorological,
  oceanographic or geophysical instrument in the water
  or on the land,
• More that just collecting an observation, and
• More than just disseminating the data via a data portal




11
     WMO
                 WMO Strategic Thrusts

1.Strategic Thrust –

Improving Service Quality and Service Delivery

2. Organization-Wide Expected Results –

Enhanced capabilities of Members to deliver and improve
access to high quality weather, climate and water and related
environmental predictions, information and services in response
to user’s needs and to enable their use in decision-making by all
relevant societal sectors.




12
     WMO
                        Conclusion
           Any and all atmospheric, oceanographic
           and geophysical observations will be
           considered as a “climate” or high-quality
           observations – and should be treated as
           such.




13
     WMO
              GCOS Climate Monitoring Principles
1.  Assess impact of new systems or changes to existing systems prior to implementation.
2.  Ensure a suitable period of overlap for new and old observing systems.
3.  The details and history of local conditions, instruments, operating procedures, data processing
    algorithms and other factors pertinent to interpreting data (i.e., metadata) should be
    documented and treated with the same care as the data themselves.
4. Regularly assess quality and homogeneity of data as a part of routine operations.
5. Integrate into national, regional and global observing priorities the needs for environmental
    and climate-monitoring products and assessments, such as IPCC assessments.
6. Maintain operation of historically-uninterrupted stations and observing systems.
7. Focus on data-poor regions, poorly observed parameters, regions sensitive to change, and
    key measurements with inadequate temporal resolution as high priorities for additional
    observations.
8. Specify to network designers, operators and instrument engineers at the outset of system
    design and implementation the long-term requirements, including appropriate sampling
    frequencies.
9. Promote the conversion of research observing systems to long-term operations in a carefully-
    planned manner.
10. Data management systems that facilitate access, use and interpretation of data and
    products should be included as essential elements of climate monitoring systems.




14
                      Seven Data Management Laws
     WMO



1.         A quality descriptor will accompany every real-time observation distributed
           to the ocean community.
2.         Subject all observations to some level of automated real-time quality test.
3.         Sufficiently describe the quality flags and quality test descriptions in the
           accompanying metadata.
4.         Observers should independently verify or calibrate a sensor before
           deployment.
5.         Observers should describe their method / calibration in the real-time
           metadata.
6.         Observers should quantify the level of calibration accuracy and the
           associated expected error bounds.
7.         Manual checks on the automated procedures, the real-time data collected
           and the status of the observing system must be provided by the observer
           on a time-scale appropriate to ensure the integrity of the observing
           system.




15
     WMO
                        Recommendation

           Region V should begin to implement proper data
           quality techniques into their newly developed
           marine observation platforms - now – before the
           instruments are placed in the water.




16
                         National Data Buoy Center
     WMO



     To provide a real-time, end-to-end capability beginning with the
      collection of marine atmospheric and oceanographic data and
       ending with its transmission, quality control and distribution.
                               IOOS Partners             Tsunami Warning Centers        Weather Forecast Offices/
         NDBC
                                 Platforms                                               River Forecast Centers
     & other NOAA
     observations
                                                                        NWS Global
                                                   NDBC             Telecommunication
                                               Data Assembly           System (GTS)
                                                                        Operational
                                                   Center                Bulletins


                                                                                                 National
                                                                                               Centers for
                                                                                              Environmental
                                                                  Emergency
                                                                                                Prediction
              Oil & Gas Platforms                                  Managers

                                                                                  National
                                          HF Radars                        Environmental Satellite,
              Voluntary Observing Ships                                           Data, and
                                                         Public
                                                                             Information Service
                                                                            (NCDC, NODC, NGDC)


                       DATA COLLECTION                   DATA DELIVERY

17                         Electrolyte to Satellite to Website
     WMO
                       Location of NDBC
Stennis Space Center
Mississippi
-      NDBC




18
           NWS/NDBC Ocean Observing System of Systems
     WMO
                  Weather Buoys that have in place for > 30 Years




19
               NDBC’s Ocean Observing Systems
     WMO




111 met/ocean buoys
4 ocean/waves buoys
49 C-MAN stations
39 DART stations
55 TAO buoys + 4 current profiler moorings
1000+ Voluntary Observing Ship vessels
20              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
           Standard 3m & 6m Buoys




21         Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO       Other NDBC Platforms: DART®
     • Deep-ocean Assessment and
       Reporting of Tsunamis (DART)
     • 39 Stations Since March 2008
     • ~230 Ship Days - Contract or
       NOAA Vessel




22                Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                  Other NDBC Platforms: TAO

     • Tropical Atmosphere Ocean (TAO)
       Climate Buoys
     • 55 equatorial Pacific buoys
     • 4 Ocean Current Profilers
     • 234-278 ship days a year
     • Typically a NOAA vessel




23                 Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                           Growth of NDBC Observing Systems
                            1999 to 2009 - The Era of Explosive Growth
           300
                                                               Katrina
           250
                                                         Tsunami
           200                                                                                                 Weather
                                                                                                               & Hurric.
           150
                                                                                                                  TAO
           100
                                                                                                                 DART
           50
                                                                                                                C-MAN
            0
             1980       1990          2000        2002       2004        2006          2008        2010


                 • 51 CMAN Stations                                             •   49 CMAN Stations
                 • 50 Weather Buoys                                             •   96 Weather Buoys
                                                                                •   15 Supplemental Hurricane Buoys
                  101 Observing Systems                                         •   55 TAO Climate Buoy Systems
                   2 system Types with similar sensors                          •   39 DART Tsunami Systems
                                                                                    254 Observing Systems
                  ~ 12 % in Severe Environments
                                                                                     5 system Types with diverse sensors
                  • USCG Provided all Ship Days
                                                                                    ~ 25 % in Severe Environments 24
                                                                                    • Challenge Obtaining Ship Days
24
                                         Engineering
     WMO




     • Mooring, power system, station design
     • System validation, analysis, evaluation
     • Technology refresh, prototype testing
25                   Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                             Station/Equipment Prep




      •    Buoy repair
      •    Mounts, cable fabrication
      •    Sensor repair, refurbishment and calibration
      •    System integration and testing
26                     Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                                          Logistics




           •   Field service planning
           •   Arranging ship support
           •   Providing dockside services
           •   Shipping equipment worldwide
           •   Inventory management
27                    Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                     Field and At-Sea Maintenance




      •    Multi-disciplined team – technicians, engineers, USCG, NOAA Corps
      •    Worldwide dockside and at-sea repairs and testing
      •    Buoy deployment, retrieval, exchange
      •    Vessels of opportunity – USCG, OMAO, Commercial
28                   Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                            Data Assembly Center




           •   Oceanographers, meteorologists, IT specialists, programmers
           •   Observations ingest, processing, analysis
           •   Data processing and QC algorithm development for new systems
           •   Observation dissemination and web display
29
           •   Management of station configuration and metadata
                          NDBC WIS / WIGOS
     WMO
                          Data Assembly Center
     • 24/7/365 support of
           – Data Quality Control
           – Communications
           – Operations
     • Daily Operations Brief
     • More than 700 stations
       currently supported
           –   NDBC Buoys
           –   NDBC C-MAN Stations
           –   NOS Stations
           –   DART
           –   TAO
           –   VOS
           –   Partner Stations

30
           Hurricane/DART Cruise 2010                      M/V HOS MYSTIQUE

     WMO




           Team: Artalona, Kendrick, Obenhaus, Stinson, Tretbar

     23 Apr        Winds: WNW 15-25 kt                        Scattered Showers   Service 41048

                   Wind waves: 3-5 ft     Swell: Negligible
     24 Apr        Winds: VAR 5-10 kt                                         En route to 41047

                   Wind waves: 1-2 ft     Swell: N 5-7 ft @ 8-9 sec
     25 Apr        Winds: S 10-15 kt                                          En route to 41047

                   Wind waves: 2-3 ft     Swell: NNE 3-4 ft @ 9-10 sec
     26-27         Winds: SW 15-20 kt                                            Service 41047
      Apr                                                                     En route to 41046
31                 Wind waves: 3-5 ft     Swell: NNE 3-4 ft @ 9-10 sec
                                             NOAA/NDBC
     WMO
                                          Data Assembly Center
                                     10



           Million of Observations
                                            Total Observations flowing through DAC
                                     8

                                     6

                                     4

                                     2

                                     0
                                          2003    2004     2005     2006      2007        2008        2009
                                                                           Active reimbursable partners
                                                                           National Marine Sanctuary Program    Kennedy Space Center
                                                                           U.S. Marine Corps                    Goddard Space Flight Center
                                                                           Army Corps of Engineers              U.S. Coast Guard
                                           22%           Mission
                                                 3%      Reimbursable       NOS/NWLON and other NOAA Obs
                                                         NWLON
50%                                                                         Independent IOOS observing partners
                                                         IOOS Partners      Gulf of Maine Ocean Observing System       Stevens Institute (NJ)
                                            25%                             International SeaKeepers Society           Long Island Ferry Boat
                                                                            University of South Florida                Louisiana State University
                                                                            Texas General Land Office                  University of Connecticut
                                                                            Louisiana Universities Marine Consortium   UNC and UNC-W
                                                                            Skidaway Institute of Oceanography         Caro-COOPS
                                                                            Scripps Institution of Oceanography        Forrest Oil
                                                                            Chesapeake Bay Observing System            Shell Oil
                                                                            Monterey Bay Aquarium Research Inst        Oregon State University
                                                                            University of Southern Mississippi


32
33
                                                 Expected Number of Stations



                                                                                                                     WMO
                    Go




                                    0
                                        5
                                            10
                                                   15
                                                        20
                                                             25
                                                                   30
                                                                        35
                                                                               40
                                                                                    45
                                                                                         50
                  MY MOO
                        S         S
                  LIS OU
                         FE     ND
                   ST RR
                         EV Y
                              EN
                                  S
                             DC
            CB          US P
                 In t        A
                     erp CE
                           rat
                               i
                          CB ve
            CA C O S
                RO O R
                       -C M P
                          O
                    US O PS
       DA                 CN
          UP                   M
              HI CO S
                  N
                      SE MPS
                          A
                               LA
                    L       US B
        MA LSU UMC M
           RI
              NE W AV O N
                   R
                                                                                                                    Stations Reporting




                        EN CIS
                             ER
                          TA G Y
                        T BS
                  SH COO
              NW E LL N
             NW S C OI L
                   S entr
                                                                                                                    Number of IOOS Partner




                        Ea al
                            s te
                                                                                              Reporting




                         GL rn
                              E
                        U RL
                           MI
                         MB CH
                        C ARI
                  O R ICO R
                         -C        E
                   U OO
                                                                                              Missing




             SC HA S
                  RI         W
                     P           A
                NE PS C II
                      R          D
                 N RS IP
            PR ERR WQ
                 Se S M
                      is m e
                            ic t
                                Ne
             NW                     t
                   S ICO
               Wo       Ala N
                             sk
              DI ods an
                  MA Ho
                                                                                                          Missing
                                                                                                                        Reporting




                        R/ l e
           Ev              CC
              erg MA CP
                   lad RE
                        es X
                             N
                                                                                                                                    NOS Availability




     EP                 KA .P.
                                                                                                          15




        A/
                                                                                                                        207




           Go                US
               vt. BP T
            Pe of M Inc.
               t ro
                    bra ex ic
                          sU o
                               SA
           U.S. IOOS Partner Support
     WMO
           http://www.ndbc.noaa.gov/pstat.shtml

                                                  Providing real-time
                                                  quality control
                                                  information to IOOS
                                                  Partner Platforms...
                                                  Including QC flags and
                                                  analyst comments via
                                                  the website.




34
     WMO

                         Web June 2010 Web
                     Apr 2010 - Pages Viewed Pages Viewed
                     Apr 2009 - June 2009 Web Pages Viewed
           4500000

           4000000

           3500000

           3000000

           2500000
                                        `
           2000000

           1500000

           1000000

            500000

                 0
                   ay

                   ay

                   ay

                   ay
                    pr

                    pr

                    pr




                                                       un

                                                       un

                                                       un
                     r

                     r




                                                        n
                  Ap

                  Ap




                                                     Ju
                 -A

                 -A

                 -A

                 M

                -M

                -M

                -M




                                                    -J

                                                    -J

                                                    -J
               1-

               8-




                                                   3-
               6-
              15

              22

              29




                                                  10

                                                  17

                                                  24
              13

              20

              27



                                                             35
35
                         WIGOS
     WMO
             Regional Marine Instrument Center
Joint WMO-IOC Technical
Commission for Oceanography and
Marine Meteorology (JCOMM)
Regional Marine Instrument Center
(RMIC) Training Workshop on April
13-15, 2010 for WMO Regional
Association IV (RA-IV) at NDBC.
Representatives from ten countries
participated in this first Workshop.
Countries represented were:
Bahamas, Costa Rica, NL Antilles,
Guatemala, Belize, Canada, France,
Barbados, Morocco, and China.




36
           OceanSITES DAC / Global DAC
     WMO




37
                             OceanSITES Data Management
                                                                                •Maintains specific OceanSITES platforms,
     WMO
            Observations in any format – may or may not be quality controlled   •Determines what observations are released to
                                                                                GTS,
                                                                                •Assures that the platform is available and provides
               Cronin            Send                             McPhaden      reliable information,
                                                Pattabhi
       PI




                                                                                •Provides the DAC with the observations in any
                                                                                format the DAC is willing to take, and the metadata
                                                                                necessary to serve as an OceanSITES platform, and
                                                                                •QC post-recovery data according to OceanSITES
                                   FTP, Flash Drive, CD                         agreed procedures.

                                                                                •Sets up the OceanSITES server according to the
                             Formats observations and provides QC               approved specifications,
      DAC




                                                                                •Guarantees data availability from the PI,
                 EuroSITES                  WHOI
                                                                    MBARI       •Translates the data to the OceanSITES format,
                                                                                •Quality Controls real-time data according to the
                                                                                minimum OceanSITES agreed procedures,
                                                                                •Provides the observations via the GTS (if requested
                                          FTP                                   by the PI),
                                                                                •Provides the data on a FTP server for access by
                                                                                the GDACs
                        Provides access to data, checks formats
               GDAC




                                                                                •Provides centralized access to the DAC data
                             IFremer                       NDBC                 •Ensures no data are excluded at the GDAC level,
                                                                                and full high-frequency data sets are available,
                                                                                •Keeps only the best version of the data. Additional
                                                                                products like interpolated data are separate
                                          FTP                                   optional sets,
                                                                                •Check all files daily using the “File Checker”
                                                                                software,
        Technical Coordinator             User                                  •Maintains the OceanSITES catalogue, and
                                        Requests                                •Synchronizes the catalogues with the second
38                                                                              GDAC periodically ( at least daily).
              “WIS-Data Discovery, Access and
     WMO        Retrieval (DAR)” Overview

                                                     Table of
                                                     Contents
                     Observations


                                          SOS        Data Values
                               Data     (Sensor
           Quality
                              Provider Observation
           Control                       Service)


             Metadata                                Metadata




39
                                International Tsunameter
     WMO
                                  Data Assembly Center
                                                             http://www.ndbc.noaa.gov/dart.shtml

                                                             The worldwide tsunami observation network also
                                                             requires a real-time, data assembly center to
                                                             provide continual monitoring and quality control
                                                             of Deep-ocean Assessment and Reporting of
                                                             Tsunamis (DART®) water pressure/height
                                                             observations. The Data Assembly Center
                                                             monitors the various real-time transmission of
                                                             DART® messages depending on the operating
                                                             mode of the bottom pressure recorder.




           Transmission of real-time water level heights
           occurs when the tsunami detection algorithm is
           triggered by a seismic event, when interrogated
           by the NOAA Tsunami Warning Centers (TWCs)
           or NDBC, or at pre-scheduled intervals.




40
                    Tropical Atmosphere Ocean Data
     WMO                    Assembly Center
                                             http://tao.noaa.gov




           •       Processing of Automated Distribution Service messages from Service Argos. It uses both the TAO
           calibration database and calibration files to convert raw data to engineering units and also calculates
           buoy positions. An automated real-time QC is performed for gross error checking and then the TAO
           database is updated with the corrected data.
           •       The TAO Real-time Data Monitoring Subsystem supports daily, weekly, and monthly QA/QC
           activities by providing on-demand data checking functionality to the DAC. In addition to the automated
           gross error checking, the real-time data monitoring subsystem provides on-demand reports for once-
           daily, thorough examination of all current buoy data and detailed review of the real-time data.




41
              U.S. Voluntary Observing Ship (VOS)
     WMO
                                     The United States Voluntary Observing Ship Project Mission
                                     The mission of the Voluntary Observing Ship (VOS) project is
                                     two-fold: (1) to collect and disseminate critical real-time
                                     maritime weather observations through the recruitment and
                                     support of ships to fulfill National needs and International
                                     agreements supporting commerce, forecasts and warning
                                     programs, and the Safety Of Life At Sea (SOLAS) worldwide,
                                     and (2) to define the global climate and help measure extreme
                                     weather events, climate variability, and long-term climate
                                     changes. VOS operates at no cost to the vessel, with
                                     communication charges, observing equipment and reporting
                                     supplies furnished by the National Weather Service.


                      Port Meteorological Officers

              Port Meteorological Officers (PMOs) support
            observing programs aboard Voluntary Observing
           Ships. They are responsible for recruitment of new
             vessels as observers, and also for ensuring the
              quality of observations from vessels actively
                       participating in the program.


                  http://www.vos.noaa.gov


42
                            High Frequency Radar
     WMO




                                          NDBC became the National HF Radar
                                          Node on 28 February 2007. NDBC will
                                          receive HF Radar radials from all HF
                                          radar sites and generate vectors.




           http://hfradar.ndbc.noaa.gov



43
               Oil and Gas Partners
     WMO

                                                        Apply quality control
                                                        to real-time ADCP
                                                        data from deep-
                                                        water oil platforms
                                                        and rigs.




           http://www.ndbc.noaa.gov/maps/ADCP_WestGulf.shtml


44
           QC Matrix
     WMO




45
     WMO
           NDBC High Level Data Flow




46          Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                              NDBC Data Center
     WMO




           • 10 Gigabit Ethernet technology that delivers multi-gigabit bandwidth to all resources while
           maintaining a 100% uptime status.
           • Designed around high-end enterprise class systems. Minimum standards include Dual-Core 1.8Ghz
           systems, 4GB RAM, redundant Gigabit network interface cards, and redundant power.




47
                                  Primary Data Sources
     WMO



                              Data Feed                                          Type Data Received

           Geostationary Operational Environmental               Weather Buoy, C-MAN, Forecast and VOS
           Satellites (GOES)

           IRIDIUM Satellite System/Router Based                 DART Data
           Unrestricted Digital Interworking Connectivity
           Solution (RUDICS)
           IRIDIUM Satellite System/Department of                Hurricane buoys, TAO refresh, AIS data, limited
           Defense (DoD)/Short Burst Data service                weather buoy

           Service ARGOS, Inc. (ARGOS) satellite system          Buoy positioning via LUT, TAO legacy data


           Integrated Ocean Observing System Participants        Partner data via FTP and XML
           (IOOS)

           HF Radar                                              High Frequency Radar Data


           GTS/GODAE                                             Non-NDBC marine observation data in support
                                                                 of the web and OSMC



48                            Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
           Basic Real-time Processing and
     WMO
                   Web System
                                                  INTERNET
           NDBC at Stennis Space Center, MS                                NWSTG at Silver Spring, MD




                         DODS      WEB03-04    COMMS02                NDBC-LOAD            GATE



                                       WEB01                              SS01             SS02
                  NDBC DMZ



                                                                         COMMS01
                                                                             Silver Spring, MD


                                        DB01     RT01
                  NDBC LAN

                                                                NOAANet



                                                                                      NESDIS
                                The Basic Real-time Processing and Web System




49             Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
              Tsunameter Data Flow
     WMO




50         Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                        TAO Data System Architecture
     WMO




                        Iridium SBD                         Argos ADS                       NWSTG GTS




                                                  Real-time Data Ingest and Dissemination
                          Delayed Mode Analysis




                                                                                             Console Interfaces



                                                                                                                  DAC Analysts
                                                         Automated QC and Alerts
           Scientists




                                                      Database and File Management


                                                         Public Web Presentation



                                                                Public Users




51                        Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                              Ocean Surface Currents- speed and direction
     WMO
                           NDBC processes FTP transmissions and displays
                           vectors on NDBC Web site via the WWW
                           http://hfradar.ndbc.noaa.gov/




           HF Radar
           Stations
           located along
           US Coast




52
                       Partner Data Processing
     WMO

                                            Regional
                                            Observatories



                      ftp*
                                                                    Public
                                                  NDBC
                                               Dial-A-Buoy


                                               Wx. Channel
                                               Local Media




     * Via XML formatted files or the NDBC Meteorological and Oceanographic
       Data Exchange Module (MODEM) Kit - request from MarineObs@noaa.gov

53
                    Data Quality at NDBC
     WMO
                           1970s - 2005
• Started with NDBC in 1970s
    • Focus on “Top Five”
    Wind Direction, Wind Speed, Atmospheric Surface Pressure,
    Air Temperature and Waves
• Algorithms perform check at NWS Gateway
• Quality Assurance Group at
NDBC provided daily check of
marine observations – usually
8 hours to two days after
dissemination
• Approx. 110 platforms




54
                    Data Quality Control:
     WMO
               The Last Line of Defense, after…
       •   Sensor Evaluations
       •   Individual Sensor Calibrations
       •   Payload Software Testing
       •   Burn-In
       •   Data Evaluation at deployment
       •   Partner Data?




55               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                    Quality Control of Observations
     WMO

           • No QC Done Onboard the Buoy or C-MAN
           • Automated QC Done in Real-Time at NWSTG
           • Hard-flags:
               – Stop the Release and Archive of Data unless Analyst
                 Overrides.
                   • Override in advance – Storm Limits, or
                   • Remove before archive
           • Soft-flags: Climatology-based, Advisory
           • Handbook of Automated Quality Control Checks and
             Procedures of the National Data Buoy Center
           •   http://www.ndbc.noaa.gov/NDBCHandbookofAutomatedDataQualityCo
               ntrol2009.pdf




56                     Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
           Moored Buoy Observations
     WMO




57         Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
              Quality Control Process Flow
     WMO


           • Real-Time:
             – System Parameters (e.g., power)
             – Message Integrity
             – Automated Hard-Flag
             – Automated Soft-Flag
           • Post-Release, Pre-Archive: Data
             Analyst




58              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                             Hard-Flags
     WMO
                   by hierarchy - highest to lowest:
           •   T Transmission parity error
           •   M Missing sensor data
           •   W Wave message is short, checksum or parity errors.
           •   E Spectral Density are exceeded or are in error
           •   D Delete measurement (“permanent failure”)
           •   S Invalid statistical parameter (e.g., mean > max)
           •   V Failed time continuity.
           •   L Failed range (climatological) limits
           •   R Related measurement has failed a hard QC check
               (e.g., WVHGT fails → Periods failed).



59                   Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                        Soft-flags for Waves
     WMO


      • a: Measurement is above monthly, regional limit.
      • b: Measurement is below monthly, regional limit.
      • c: Measurement has been adjusted, or corrected.
      • f: Measurement failed hourly time continuity.
      • m: High frequency spikes detected in the wave spectrum.
      • p: Failed wave height to wave period comparison test.
      • q: Swell direction is from an improbable direction.
      • w: Failed wind direction verses wave direction check.
      • x: Wind wave energy is too high for prevailing wind
           speed.
      • y: Wind wave energy is too low for prevailing wind speed.




60               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
              It all happens at the NWSTG
     WMO




     Most other organizations: NWSTG is a big router




                      For up to
                      5 minutes




61
             Why we halt the data at NWSTG
     WMO



           • Needed to process waves
           • Data can be corrected:
             – Can recover incorrect payload parameters
             – Sensor offset or position drift
           • Quality Control
           • Add computed fields
           • Assign WMO code forms, routing
             identifiers



62               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
                   Configuration Control File
     WMO
                       for Each Station:
       •   GOES ID
       •   Codes and Routing Identifiers
       •   Test or Operational Mode
       •   Sensors Failed
       •   Sensor Hierarchy
       •   Sensor Scaling
       •   Station and Sensor Elevations



63               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                 Derived Measurements
       • Pressure Tendency & Trace
         Characteristic
       • Extrapolated 10- and 20-m wind speeds
       • Swell and wind wave estimates
       • Wind Chill
       • Ice Accretion
       • Heat Index



64            Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                    Output Formats
       • FM-13 for moored buoys
       • C-MAN code for coastal stations:
           – National code, but well-suited for coastal
             obs.
           – Contains water level & waves
           – Based on FM-12 land synoptic code
       • FM-64 TESAC (temp., salinity, current
         profiles)
       • FM-65 WAVEOB (spectral wave data)


65                Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                Real-Time Quality Control
     • Purpose: Remove gross errors
     • Data rejected: Virtually certain to be degraded
     • Typical causes: Transmission errors, power
       degradations, broken cables
     • Limitations: Won’t detect minor errors, biases




66              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
           Monthly Climatology Check




67
     WMO
                               Limit Checks

     • Classic way of catching a sensor that “pegs”.
     • However, extreme, but valid, data has been
       withheld.
     • Climatology based
     • Can be overridden before expected storm




68              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
                   Time Continuity Check

•     Dependent on time since last valid observation.
•     Max. allowed value = 0.58*Std.dev.*SQRT(TimeDiff)
•     Works well for normally distributed measurements
•     Std. dev. Chosen 50% higher climatic standard dev.




69              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
           Time Continuity Check




70
     WMO
                    Time Continuity Check

       • Maximum allowable values in one hour:
           – Sea level pressure                            12.2 hPa
           – Air Temperature                               6.4 deg. C
           – Water Temperature                             5.0 deg. C
           – Wind Speed                                    14.5 m/s
           – Wave Height                                   3.5 m




71               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
               If data fails time continuity,
     WMO
                 it may still be released if:
      • Pressure: Both pressures < 1000 hPa
      • Wind Speed: Both pressures < 995 hPa or it
        agrees within 2 m/s of duplicate
      • Air Temperature: Either wind speed > 7 m/s or
        wind direction change > 40 deg.
      • Wave Height: Current wind speed > 15 m/s
      • Manually disabled in front of a hurricane




72             Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
               Internal Consistency Checks
     • If battery voltage < 10.5 V, pressure not released.
     • Significant wave height and dominant period set to
       zero if significant wave height < 0.25m. Without
       this, large periods can result from nearly calm
       seas.




73              Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
               Internal Consistency Checks

     • If dew pt. > air temp, set dew pt. = air temp. (RH can
       read slightly more than 100%)
     • If ratio of gust to wind speed > 4, don’t send wind
       speed or gust.




74               Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
     WMO
             Post Release Man-Machine Mix
       • Pre-done Graphics produced:
          – Flagged data first
          – Then all data
       • Tools:
          – Time series plots
          – Spectral Wave Curves
          – Wind Wave scatterplots
          – Surface weather plots




75             Regional Association V, Fifteenth Session, Bali, 30 April – 6 May 2010
           Visualization Tool Suite
     WMO
                  Top Level




76
           Visualization Tool Suite
     WMO
                 Area Level




77
     WMO
           Pre-done Graphics




78
           Pre-done Graphics
     WMO




79
           Pre-done Graphics
     WMO




80
            Time Series Plots:
     WMO
           Wind Speed Problem




81
                   Time Series Plots:
     WMO
           Comparison with GFS Initial Analysis




82
           Position Plots
     WMO




83
                      Data Monitoring
     WMO
                  What does the future hold?
           • Implementing new Unidata WX analysis
             software with improved vis. tools
           • Look at only suspect or bad data
              - real time QC associated with a database
              - more sophisticated logic in algorithms,
             extended trend analysis
              - model fields used in real met and wave QC
           • Flags assigned based on quality score
           • Distribute and archive data with flags



84
     WMO




             Terima Kasih !
               Bill.Burnett@noaa.gov
              National Data Buoy Center
           Stennis Space Center, MS 39529




85

								
To top