DATA PROCESSING QUALITY CONTROL (DPQC) DATA FLOW AND QUALITY by qws18475

VIEWS: 22 PAGES: 4

									 DATA PROCESSING QUALITY CONTROL (DPQC) DATA FLOW AND
               QUALITY CONTROL SYSTEM

                                  Michela Sunda, Mirko Santuari

                SERCO-Italia Spa, Via Sciadonna 24, 00044 Frascati (RM), Italy,
                         Email: msunda@serco.i, msantuari@serco.it



ABSTRACT                                                The DPQC group is furthermore responsible for
                                                        many other activities related to the operational
The products monitoring of the ESA satellite
                                                        mission support.
instruments is nowadays performed, by the
                                                        The DPQC tasks for the ENVISAT mission
DPQC consortium on behalf of ESA, . The
                                                        support can be summarized in the following:
DPQC involves expertise of different specialized
                                                        •    Daily and long term monitoring of
companies. The scope of the consortium is the
                                                            instrument health
quality check of the operational product of all
                                                        •    Daily and long term monitoring of products
ESA missions with special attention to the
                                                        •    Anomaly investigation and performance
ENVISAT and ERS instruments. Furthermore
                                                            assessment
the DPQC is also responsible for the instrument
                                                        •    Calibration activities and delivery of
calibration and the operational processor
                                                            auxiliary data files
upgrades. In order to accomplish to this task a
                                                        •    Maintenance and support for IPF evolution
big amount of products and auxiliary files must
                                                            and implementation
be daily collected from the ground segment and
                                                        •    Mission support, reporting and attendance
stored. Besides this several routine algorithms
                                                            to progress meetings
run for the generation of quality control reports
                                                        • Answer to general requests from users.
and calibration files. These activities must be
performed daily with the operational constraints
                                                        2. DPQC OVERVIEW
of the non-stop service. This task needs a
significant effort in terms of data flow                The DPQC is responsible for the quality check of
organization and system architecture design. The        the products of several ESA missions (ERS,
scope of this poster is to present the data flow        ENVISAT and LANDSAT). In particular this
and system architecture which will be put in            poster is focused on ENVISAT instruments and
place within the DPQC in support to the ESA             environment. The DPQC is also responsible for
satellite mission. The main task of this system         the instrument calibration and validation, as well
will be the quality control and the calibration         as the operational processor upgrades. In order to
files generation and dissemination for all the          accomplish to this task a large volume of data
considered instruments. The system will allow           products and other files (Auxiliary files, Mission
for easy and flexible adaptation to the new             Files,CTIs, Flight Data) are collected by DPQC
missions and new types of products and offers a         daily.
homogenous platform for quality control.                The data sources and the facility that interacts
                                                        with DPQC are many and with different
                                                        functionality:
1. DPQC IN SUPPORT TO ENVISAT                            PDS (Payload Data control Segment) as source
MISSION                                                 of NRT products and auxiliary data files;
                                                        • FOS(Flight Operations Segment ) as source
The quality assessment activities of ESA Earth
                                                             of DMOPs and other flight data;
Observation missions (ERS, ENVISAT, GOCE
in the future ) are performed by the DPQC               • AOIPs (Announcement of Opportunity
organization today, a Serco-Datamat consortium               Instrument Providers) as source of Auxiliary
of specialized companies which provides this                 Data Files;
service to ESA.                                         • RGT as source of Mission Planning, MPL’s
                                                             File and CTIs.

_____________________________________________________
Proc. ‘Envisat Symposium 2007’, Montreux, Switzerland
23–27 April 2007 (ESA SP-636, July 2007)
                                        Fig. 1 – DPQC Overview




2.1 DPQC Data Flow Description                        Protocol (FTP). The server is polled
                                                      continuously in order to collect the latest files
2.1.1 ESRI015 DDS Receiver                            available. The data is stored in the local raid of
DPQC gets Envisat Data products disseminated          the DPQC environment.
via DDS (Data Dissemination System). The
incoming data on the ESRI015 local disks are          2.1.5 RGT (Reference Operations Plan
transferred via File Transfer Protocol (FTP) onto     Generation Tool)
the local raid of DPQC environment.                   The Mission Planning files (MPL) and CTI
                                                      tables are collected from the RGT server via File
2.1.2 Processing and Archiving Centres                Transfer Protocol (FTP). The server is polled
The products processed offline at the various         continuously in order to collect the latest files
PACs are received on exabyte or on CD or via          available. The data are stored in the local raid of
ftp (systematically or on request) by DPQC. The       DPQC environment.
data available on-line on the AMS of the various
PACs are retrieved via PCF Data Manager.              2.1.6 ADS (Auxiliary Data Server)
                                                      The Doris information are collected from the
2.1.3 Payload Data Control Segment                    ADS server via File Transfer Protocol (FTP).
The data products distributed via ground links        The server is polled continuously in order to
are retrieved via File Transfer Protocol (FTP).       collect the latest files available. The data are
The servers are polled continuously in order to       stored in the local raid of DPQC environment.
collect the latest files available. The data is       The Auxiliary Data Files and the CTIs generated
stored in the local raid of the DPQC                  within the DPQC through IECF are transferred
environment.                                          into the Envisat Auxiliary Data Server (ADS) for
                                                      dissemination in the operational processing
2.1.4 FOS (Flight Operations Segment)                 chains.
The Flight Data and Mission Files data are
collected from the FOS server via File Transfer       2.1.7 AMS (Archive Management System)
The AMS is used by DPQC for archiving               2.1.9 SCICAL
purposes. All off-line data processed inside the    SCICAL is a tool for SCIAMACHY in-orbit
DPQC environment is archived in a partition of      calibration based on level0 NRT data. SCICAL
Esrin AMS allocated to DPQC. On the other           is hosted on the SCICAL server. For each
hand the AMS accepts retrieval request from         SCIAMACHY ADF produced by SCICAL the
DPQC through Data Manager in order to retrieve      IECF is responsible for recovering the files
Envisat products and Auxiliary files processed in   automatically by ftp and delivering them to ADS
the various processing chains (NRT and Off-         server.
line). The retrieved data is stored in the DPQC
Raid. Finally, the GFE (Generic Front End) of       2.1.10 MICAL
each Envisat AMS automatically forwards to          MICAL is a tool for MIPAS in-orbit calibration
DPQC all header product file and auxiliary files    based on level0 NRT data. MICAL is hosted on
before archiving them in the AMS for quality        the MICAL server. For each MIPAS ADF
control purposes.                                   produced by MICAL the IECF is responsible for
                                                    recovering the files automatically by ftp and
2.1.8 DPQC (Data Processing Quality Control)        delivering them to ADS server.
Infrastructure
The Data Processing Quality Control                 2.1.11 Envisat FTP servers
Infrastructure is the environment, where the
products quality control, the routine calibration   The DPQC is in charge to make additional data
and validation and the long-term sensor             available to the users:
performance monitoring for the Envisat mission           ASAR Transponder control file that
are performed. It is being set-up and operated by       contains ASAR transponders pointing
the DPQC team through a certain number of               information transferred to ESTEC for usage
tools, instrument specific (IATs) or mission            in the operational transponders.
related (QUADAS, QUARC, IECF).                           RA2 USO correction auxiliary files to the
The Envisat Instrument data is systematically           users
monitored for anomaly detection; the relevant            MERIS Calibration Data are extracted
parameters are extracted and stored in a database       through the METRIC CFI tool. The purpose
for trend analysis and long-loop sensor                 of the METRIC tool is to perform data
performance assessment. The actual file itself is       extraction and spatial compression over
stored in the file system. Following an anomaly         specified sites and following site type
detection, the products are further checked with        specific radiometric and geographic criteria
the instrument specific tool. The outputs are           for the absolute calibration of MERIS by
daily reports or cyclic reports, anomaly reports.       vicarious methods.
The DPQC is also responsible for calibration
activities. The data collected from the different   2.1.12 AOIPs
sources are processed trough specific algorithm     (Announcement of Opportunity
functions of IECF (ENVISAT Instrument                Instrument Providers)
Engineering and Calibration Facility) and fed
back to the ground segment. In particular, IECF     The AOIPs provide regularly the IECF with
performs generation of auxiliary data files to be   specific Auxiliary Data Files files on a particular
delivered to PDS for the corresponding usage in     ftp server. The
the processing chains and generation of the         IECF polls these servers as necessary and collect
calibration tables (CTIs) to be delivered to the    the latest files available. The scope of IECF is to
RGT for the corresponding usage in the              validate and disseminate them to the ADS server.
processing chains.                                  In some cases it applies also specific calibration
                                                    algorithm in order to generates the final ADFs.
                               Fig. 2 – DPQC Data Flow Description.




3. DPQC NETWORK CONFIGURATION
The DPQC network configuration schema shows how the servers in the network are configured,as well as
the network interfaces. The DPQC Servers are connected to the Gb data network aswell as to the PCF
Monitoring and Control network (100 Mbps). They have been moved behind the Envisat PDS firewall for
easier access to the data products processed in the PDS.

                               Fig. 3 – DPQC Network Configuration

								
To top