Get documents about "Oxygen Objectives - PowerPoint
Document Sample
Image Information Mining
ESA Perspective
and
European Coordination Group
Sergio D’Elia
Head of Service Support and Ground Segment Technology
EO Programme Directorate
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 1
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 2
EO support for real problems
Need: intervene on environment after oil spill
ERS SAR catalogue Data
Many actors and services
•Parallel catalogue search (PBytes)
Envisat ASAR catalogue
RadarSat catalogue
knowledge
•EO data access Ship
EO data detection
•Information extraction EO data
Oil spill
detection
Wave / wind data;
•Higher level processing & model Information
fusion with non-EO data / (KBytes)
information
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 3
New Technology Opportunities
Need: intervene on environment after oil spill
Service Support
ERS SAR catalogue Data
Many actors and services
•Parallel catalogue search Environment
(PBytes)
Envisat ASAR catalogue
RadarSat catalogue
knowledge
•EO data access Ship
Knowledge-
EO data detection
based Information Mining /
•Information extraction centred Earth Observation
Oil spill
EO data detection
Wave / wind data;
•Higher level processing & Information
model Grid –
fusion with non-EO data / Processing On Demand (KBytes)
information
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 4
SSE Capabilities
Service Provider Easy service publication, chaining,
/ Consumer Web discovery, activation, monitoring
Services
Extend GS Interface vs. providers
Non-EO Domain • Parallel access to distributed catalogues
Services (GIS) • Order submission
• Satellite data dissemination
SSE
EO Domain Services
Services • Remain by the providers
• Defined via SLA (incl. QoS)
• Automatic or manual
Catalogue EO Ground Seg.
Order Services • Subscription or occasional
Delivery • From any domain (e.g.: GIS)
Tasking
Processing Based on
Archive • Open standards (WS, BPEL, SOAP, ...)
• Neutral ESA “Moderation”
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 5
SSE Use Opportunities
Test / automate
Public Restricted internal procedures
Access Access (chain own services)
Internal Use
SSE Portal
Cooperate /
Services
Restricted Use exploit synergies
A1 A2 with other SPs
A3
Provide services
Restricted Access to selected users
Provide services:
Public Access from one / more SPs
manual / automatic
simple / chained
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 6
Processing On Demand
Capabilities
• 40 TBytes & 128 processors
(2.6 GHz each; based on Grid technology)
• Process large data sets
• Interoperable with ESA catalogues / archives
Status / Plans
• Operational with following services:
MERIS Global Vegetation Index (JRC Level 3 processor)
Mapping of ice sheets at Earth Poles from ASAR Global Monitoring data
Temporal aggregation of MERIS RR global data sets (L3 processor prototypes)
Ozone validation (comparing ERS-2 GOME data against LIDAR measurements)
• Add new user processors and integrate with SSE
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 7
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 8
IIM Technology Objectives
Support / automate information extraction
• Interactive Information Discovery (Probabilistic Information Mining )
• Content Based Image Selection (also via SSE)
• Feature Access (also via SSE)
• Time series handling
Explore combined use of
• Feature Extraction Algorithms
• Probabilistic Information Mining
Address related issues
• Semantics (ontology / terminology)
• Knowledge management / discovery
• Information mining from heterogeneous sources
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 9
ESA Approach on IIM
IIM Technology Operational Method Capabilities Project
Interactive Information Discovery, KIM, (*
)
Scene Understanding KIMV
Interface
Content Based Image
SSE
Probabilistic IM Non-Interactive Selection Services MIMS
Non-Interactive
Content Based Image
Interface
Selection Services KEO,
SSE
Component-based
Processing Environment
Feature Access PIMS-DLR
Services
Feature
Interactive /
Extraction
Non-Interactive
Algorithms (*) Interactive KIM capabilities
available in all projects
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 10
Related ESA Projects vs. Area
KEO IIM-TS
Time Series
KEO, PIMS-DLR
Component-based Proc. Env.
KES-B
IIM Services
Crater, ICDY, SURF KEO, MIMS
Feature Extraction Components
DPF KEI
Probabilistic IM KIM, KES, KIMV
2001 2002 2003 2004 2005 2006 2007 2008 2009
Registration
MIR, UGEIP KEO, UGEI, MIR-E
OrthoServ
Orthorectification
KES, KES-B, KEO KEO, SDD, EOS
Semantics
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 11
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 12
Knowledge-based Information Mining
KIM permits to interactively train the system to search an entire image
collection for large or well characterised areas
KIM elements
• Ingestion (Primitive Feature extraction / clustering per image collection)
• Client / Server for:
System training (and training refinement) from sample images
Application of training to the entire image collection
Semantic labelling of trained features (for reuse)
Output
• Image identifiers
• Feature maps / GIS objects
KIM
Data Information
Ingestion Database Client
Mining
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 13
KIM User Interface
A few clicks’ training
Applied to all ingested images
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 14
Primitive Features
Spectral Spectral signature
Texture Structural information extracted from the images by applying the
stochastic auto binomial model of Gibbs Marcov Random Fields (S0: from
full resolution images; S1: from sub sampled images)
DCT Discrete Cosine Transform: transforms signals and images from the
spatial domain to the frequency domain
EMBD Enhanced-Model-Based-Despeckling: performs a high quality despeckling
of SAR images
Area Area of the objects detected with the segmentation process
Compactness Compactness of the objects detected with the segmentation process
Spectral Mean value of the radiometric information of the image inside the closed
Mean area detected by the segmenter
Spectral Variance of the radiometric information of the image inside the closed
Variance area detected by the segmenter
Hu Moments Hu-Moment Invariants: shape information conveyed by the contour points.
Hu moments are invariant to scale, rotation and translation (the first 4 out
of 7 invariant moments as shape descriptors have been used).
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 15
Test Collections
Sensor Product Primitive Features
Envisat MERIS RR1
Spectral, DCT, Texture S0,
Landsat 5 TM / Texture S1
CEOS Syscorrected
Landsat 7 ETM
ERS-1/ERS-2 Intensity, Texture S0, Texture S1,
GEC
SAR EMBD
Ikonos Pancromatic GeoTIFF
Spectral, Texture S0, Texture S1,
Area, Compactness, Spectral
Spot 5 HRG DIMAP
Mean, Spectral Variance, Hu
Generic RGB image Moments
Various
(jpg, TIFF, GeoTIFF …)
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 16
MERIS Cloud-free Subscription Service
User FTP MERIS RR
Archives
Subscription e-mail
Ingestion
Orders
Cloud-free KIM
Service Catalogue
Information
KIM
SSE
User
Information
Authorisation Mining
MIMS
Service n Expert
SSE = Service Support Environment
KIM = Knowledge-based Information Mining
MIMS = MERIS Information Mining Services
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 17
KIM / SSE: MERIS Cloud-free Service
KIM / MIMS
• Ingest systematically MERIS RR products from the ESA rolling archives
• Too many cloud objects:
cloud cover % computed for each image cell (112x112 pixels)
• Special search in the “cloud cover” database:
cloud cover < user defined threshold within user area of interest
SSE
• Activates KIM “cloud cover” database search
• Provides notification and product identifiers (via e-mail)
• Permits to download MERIS RR products (via FTP) from rolling archives
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 18
KEO Positioning
Service Provider Component-based Processing Environment
/ Consumer Web for information extraction
Services
by authorised users
Non-EO Domain
via chained internal / external services
Services (GIS)
SSE
EO Domain
Services KEO
Catalogue EO Ground Seg.
Order Services
Delivery
Tasking
Processing
KIM
Archive
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 19
KEO Concept
Catalogue IF Discovery /
Chaining / Client
Information Management
Service IF
Knowledge
Information base
Control
SSE Engine
Descriptions
KEO
Algorithms
Support
Feature Extr.
Information Extraction Probabilistic
Data
KIM
Information
Ingestion Database Client
Mining
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 20
KEO Local / Remote Contributions
Chaining Remote
Create, Check Contributions
Chain, Portfolio
Publish Database
Discover Data
Components
Control Describe Processing
Terminology
Engine Components
Reference Data Sets
Create,
Maintain, Doc.s Data Processing Create,
Browse Components Components Maintain
Local FEAs
KIM
Contributions & Functions Training
Web Services / FTP
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 21
Image Information Mining
Objectives
• Image search also by content
• Discovery in image archives
• Interactive image interpretation
• Extract / access features
Available
• Feature Extraction Algorithms survey
• Knowledge-based Information Mining tool
• MERIS SSE Cloud-free Service
Plans
• Distributed Component-base Processing Environment linked with SSE
• Extend to other missions (e.g.: TerraSAR)
• Ontology / Terminology
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 22
KEO Phase 2 Plan
Objective
Prototype of a Component-based Processing Environment
to automate extraction and provision of information
by authorised users
Capabilities
Handling of local Reference Data Sets
(incl. Documentation, Data Components, auxiliary data, …)
Use of remote Data Components
Description, handling, discovery and use of local or remote
Processing Components
Service creation, chaining, publishing and use
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 23
Relations with Other Projects
KIM reused in
• ESA projects: MIMS, KEO, PIMS-DLR
• STREAM (FP6 Project on “Technology to Support Sustainable Humanitarian
Crisis Management”)
KEO – SSE architectures converging
• Easy KEO publishing onto SSE of internal services for external access
• Easy KEO use / chaining of external SSE services
KEO – SSE architecture applied to / considered for
• PIMS-DLR, STREAM, EUSC Reference Facility
• Heterogeneous Mission Accessibility (harmonisation of ground segments)
• Wide Information Network (WIN) FP6 Integrated Project on Risk Management
Other similar approaches emerging
• E.g.: EOFrame from VTT
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 24
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 25
Semantics
Objective
• Permit easy, semantic identification from non-EO domains (using non-EO domain
terms) of relevant EO
Products / services
Processing components
Approach
• Identify an ontology
As simple as possible
With limited dependencies from evolution / changes
Supporting multiple domains
Permitting a (partial) Web Service implementation
Supporting multiple languages (outside ESA implementation)
• Test the implementation in specific cases
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 26
ESA Path on Semantics
ESA activities on EO ontology / terminology started in 2003 with
the KES-B project (http://earth.esa.int/rtd/Projects/KES-B/index.html), aimed at
easing:
• Information extraction (permitting semantic description / search of related processors)
• User exploitation of EO products / services (permitting their semantic description / search)
KES-B results analysed for
• Ontology simplification / generalisation
• Implementation stability (minimise changes when applied to the real world)
Resulting approach being verified for:
• A processing environment (KEO: http://earth.esa.int/rtd/Projects/KEO/index.html)
• Implementation for coastal applications in cooperation with Mississippi State
University (SDD)
Ontology to be extended in future to the Service Support
Environment
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 27
KES-B Knowledgebase
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 28
KES-B Ontology
Definitions
Product = data or information
packed for user
Service = (controlled) product
provision
Services
Processing
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 29
Minimum EO Ontology
Domain
Product Service
Processor
Unstable: many changes are possible below “Domain”
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 30
Separation to Improve Stability
Domain
Leaves define
Classification Ontology Domain “Service Categories”
(single, shared, stable, multi-lingual) (Service Category) (improving stability)
Semantic links
Portfolio Ontology Domain
(local, specific, dynamic, selected language) (Service Category)
Product Service
Processor
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 31
Classification Ontology: Thesaurus?
ISO 2788
Documentation – Guidelines for the establishment and development of monolingual thesauri
Broader Terms (BT)
Term
Related Terms (RT) Related Terms (RT)
Synonyms (NPT)
(not linking Terms) (linking Terms)
Definition (DEF)
Narrower Terms (NT)
Definition, Synonyms and “not-linking” Related Terms
provide many entry points
with a minimum number of links (graph complexity)
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 32
Thesaurus Example
DOMAIN DOMAIN DOMAIN DOMAIN DOMAIN DOMAIN DOMAIN
LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 5 LEVEL 6 (SERVICE CAT EGORY)
Env ironment
M arine Env ironment
C oastal Env ironment
Marine Pollution
Oil Spill Oil Spill Detection
Oil Spill M ov ement Forecast
Oil Seepage Oil Seepage Detection
Algae
Algae Bloom Algae Detection
Offshore Env ironment
Marine Pollution
Operational Oceanography
Land
Land C ov er
Land U se
Env ironment Degradation
Marine Pollution
Air Pollution
Deforestation
Dsertification
Water Quality
Resources
M arine Resources
Oil Seepage
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 33
Thesaurus Example
SYNONYMS RELATED TERMS
Algae PRYMNESIOPHYTES, Chrysocromulina polylepis, Chrysocromulina leadbeateri, Phaeocystis cf. phytoplankton, phytoplankton concentration, planktonic algae, brown
puochetii, Phaeocystis cf. globosa, Prymnesium parvum, Prymnesium patelliferum, Emiliania sea, yellow-brown sea, red-brown sea, red sea, green-yellow sea,
Huxley (non-toxic), DIATOMS, Amphora coffeaformis, Chaetoceros convolutus, Pseudonitzschia animal zooplankton, chlorophyll pigments, mineral nutrients
pseudodelicatissima, Pseudonitzschia seriata, Rhizosolenia, Skeletonema costatum,
SILICOFLAGELLATES, Dictyocha speculum, DINOFLAGELLATES, Gymnodinium galatheanum,
Gyrodium aureolum, Amphidinium operculatum, Alexandrium excavatum, Ceratium furca (non-
toxic), Ceratium lineatum (non-toxic), Cochlidinium heteroblatum, Dinophysis acuminata, Dinophysis
acuta, Dinophysis norvegica, Dinophysis rotundata, Dinophysis caudata, Dinophysis fortii,
Karlodinium micrum, Pfiesteria piscida, Pfiesteria shumwayae, Prorocentrum minimum, Scrippsiella
trochoidea, HETEROTROPHIC DINOFLAGELLATES, Noctiluca scintillans, RAPHIDOPHYTES,
Chattonella verruculosa, Fibrocapsa japonica, Heterosigma akashivo, CYANOBACTERIA,
Nodularia spumigena, CYANOPYTES, Anabaena, Aphanizomenon, Microcystis aeruginosa,
CHRYSOPHYTES, Aureococcus anaphogefferens
Algae Bloom algal blooms, harmful algal blooms, HAB, algae spreading, harmful algae spreading, toxic algae biomass, chlorophyll-a, eutrophication, climate changes,
blooms, toxic algae spreading aquaculture, coastal pollution, coastal zone, biodiversity, detritus,
dissolved organic matter, fisheries, food, Gelbstoff, Gilvin, hazard,
human illness, salmon, ocean chemistry, ocean circulation, ocean
colour, ocean optics, phytoplankton, poisonous mussels, health,
recreation, red tides, risk, sea surface temperature, SST, sediments,
shellfish, suspended matter, tourism, water type, water leaving
radiance, water quality, water transparency, wild life, yellow
substance
Algae detection algae detection, algae monitoring, algae observation, algae spreading, algae forecast, algae bloom
detection, algae bloom monitoring, algae bloom observation, algae bloom spreading, algae bloom
forecast
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 34
Multi-domain Thesaurus
Resources Environment
Marine Resources Marine Envir. Land
Coastal Envir. Offshore Envir.
Marine Pollution Operational Oceanography
Oil Spill Oil Seepage Algae
Algae Bloom
Oil Spill Detection Oil Spill Mov. Forecast Oil Seepage Detection Algae Bloom Detection
Product Service Service Service Service Service Product Service
Processor System A System B System C System D System E Processor System F
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 35
Distributed / Shared Functions
Knowledge Knowledge
Management Discovery
Classification Documents
User User
Terminology Knowledge Internet
Database Preferences
base Logs
User Data Terminology Preferences Knowledge
Management Management Management Access
Web Services
Search / Navigate
Management Management Management
Search / Navigate
Portfolio Portfolio Portfolio
Database Database Database
System A System B System C
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 36
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 37
European
Image Information Mining
Coordination Group
S. D’Elia – ESA / ESRIN
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 38
IIMCG History
“Voluntary” organisation focusing on
Research and technological activities for automated and
user-centred extraction of information from EO images
Created on May 26, 2003 by
• ASI - Italian Space Agency
• CNES - French Space Agency
• CNR - Italian National Research Council
• DLR - German Aerospace Center
• EC-IST - Information Society Technology
• ESA/ESRIN - European Space Agency / Research Institute
• ETHZ - Swiss Federal Institute of Technology Zurich
• EUSC - European Union Satellite Centre
Extended to EARSC
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 39
Achievements (after 3 years and 9 meetings)
Good cooperation
• Free exchange of information and experience
• Common definition of IIM roadmap
• Contributed to ESA technology plan definition
• Agreement on specific research projects
Practical actions (of the group or of members)
• Organised 4 Conferences with ESA, EUSC, IEEE-GRSS, OGC sponsorships
• Presented tutorials and short courses at IGARSS 03
• Supported special sessions on IIM at IGARSS 03, 04, 05 and 06
• Introduced IIM in Master and PhD Programs of several universities
• Multiple installations of KIM prototype and parallel evaluation
• Started analysis of possible reference data sets
Raised wide interest (see CNES/DLR/ENST Competence Center)
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 40
Table of Contents
Background
IIM at ESA: Objectives and Approach
IIM at ESA: Results and Plans
Semantics
European IIM Coordination Group
Links
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 41
Useful Links
Feature Extraction Algorithms
• Survey results (2004) http://earth.esa.int/rtd/Documents/SURF_TN.pdf
KIM demonstrator
• Client / documentation http://kes.esrin.esa.int/kes/
• Questions to EOHelp@esa.int
MERIS Cloud-free Subscription service under test
• Authorised user access at http://services.eoportal.org
KEO
• Phase 1 presentations http://earth.esa.int/rtd/Projects/KEO/index.html
IIMCG
• Information (under revision) http://earth.esa.int/rtd/IIMCG
ESRIN Ground Segment Research Projects
• Information at http://earth.esa.int/rtd/
ESA-EUSC 2006 EUSC - November 27-29, 2006 – Slide 42
