CAS by shuifanglj


									Center for Advanced Sensors
          David J. Russomanno, Ph.D.
                Professor and Chair
 Department of Electrical and Computer Engineering
           Herff College of Engineering
            The University of Memphis

            Knowledge Fusion Workshop
                December 1, 2005
                  Annapolis, MD
Center for Advanced Sensors (CAS)

           NVL (NVESD)   ONR

           Army Research Lab
          University of Memphis

           VU              UAH
•   CAS Background
•   CAS Projects Overview
•   Imaging Sensors Research
•   Knowledge Engineering Research
  Center for Advanced Sensors
• Established in April 2005 via a cooperative
  agreement between ARL/ARO and The
  University of Memphis
• Enhanced by over 25 years of collaboration
  between The University of Memphis and the
  U.S. Army’s NVESD, Redstone Technical
  Test Center (RTTC), and ARL
• PIs reside in Electrical and Computer
Electrical & Computer Engineering
• 120 EE and 70 CpE undergraduate students
    38% African-American
• 50 MS and 15 PhD graduate students
    increasing efforts to recruit US citizens into graduate programs
• 11 Faculty, 1 Post-Doc, 3 Adjuncts
    8 research-oriented faculty
    3 teaching- and service-oriented faculty
    ongoing search for tenure-track Assistant/Associate Professor
• Research Focus Areas:
    imaging sensors and electronic devices
    biomedical imaging and devices
       – Whitaker Foundation, American Heart Association
    intelligent information systems
       – NSF
• Anticipate  $2M in FY 05-06 research expenditures
Center for Advanced Sensors
                Sensor Simulation   Know ledge
                    Test-Bed        Engineering


       Lightw eight
     Optical Systems
         Sensor Testing/                   Diamond
           Calibration                    Technology

                UAH                     VU
Center for Advanced Sensors
Projects at The U of M
•   Performance Modeling with Image Processing Enhancements – Halford,
      Establish perception laboratory and provide support for models
      Establish methodology for incorporating image processing techniques into existing
       performance models
      Modify/adapt existing sensor models to account for image fusion
      Model performance enhancement by anomaly detection ATRs

•   Performance Modeling of Advanced Architecture Systems – Griffin
      Model short-to-mid range transmission in the THz
      Model antenna coupling
      Model optics and interaction with focal plane coupling system

•   Knowledge Engineering – Russomanno
      Sensor ontology development
      Ontology-driven algorithms
      Application Development
Center for Advanced Sensors
Projects at VU and UAH
•   VU – Bio-Optics of Vision, IR Display, Bio-Optic Sensor Electrodes – Bonds,
      Research natural (living organism) visual sensory representation as performed by
       neural assemblies, recording isolated neural activity across a network of cells
      Determine the coding of visual signals by cell populations and achieve a working view
       of the neural code to achieve superior electronic approaches to rapid AI night vision
       image signal processing
      Engages multiple disciplines, including (1) visual neuroscience,(2) imaging,
       processing and detection, (3) automated decision making/classification and (4)
       statistical analysis

•   UAH – Wavefront Sensors – Reardon
      Existing wavefront sensors tend to be computationally intensive
      Desirable to reduce or eliminate computational steps
      Research utilizes a purely optical means of decomposing the wavefront into known
Imaging Sensor Modeling
                                             Detector             Display
                                             Array &
                                Optics       Cooler
                                                                            Human Vision


                                           Visible               – Sunlight (Digital Video)
                                           SWIR                  – Laser Illuminated
                                           Infrared              – “Thermal Glow” (FLIR)
  Target and Background                    Submillimeter         – “Molecular Glow” (THz)
                                           Millimeter & Longer   – Obstruction (RADAR)
 Role of Models
                                                                                          Test & Evaluation
     Sensor                 Model                 Model        Model
    Concepts             Development            Validation   Applications
    Technology                                  Field Test                  Trades
       Push                                        Data

    Component                                                            HITL
                                                Perception                                     Training
    Technology                                                           Experiments
      models            1
      Theory /         0.6

     Literature        0.4
                                                                            War Gaming
                             0   10   20   30

                                                                            Decisions            Product

Sensor Performance Models - NV-Therm, Acquire, Search, I2, TV, Laser
 Basic                 Applied            Development              Single Device                Production
Research              Research                                     Development                  & Fielding
   CAS Perception Laboratory
Indigo Merlin LWIR and
MWIR cameras

LightSpace 1024Z
3 Dimensional Monitor

          Large set of Military visible, LW,
          and MW images
          Synthetic Image Generation
Effects of band-limited noise, blur
on minimum contrast for ID

• Perception tests performed include:
   TOD (Triangle Orientation Description)
   ID of military vehicles

• Assessment of perception as a function of
  range for imaging sensors in various
Vehicle ID
Perception Lab Experiments
Urban Operations and AT/FP
 • Provide the sensor design and analysis models for the military user,
   system designer, and the war gamer in the urban environment [TV, I2,
   SWIR, MWIR, and LWIR]

                           Two Main
                          Search and                     Night Imagery
                                           Day Imagery
                           Target ID


NVESD Twelve Target Set
Phenomenology & Radiometry
• Objective: Provide state-of-the-art radiometric field
  measurements to the NVESD research community and
  other government agencies
• Extensively used in sensor modeling and
• Not significantly exploited once sensor is
• Incorporate aspects of phenomenology into
  sensor ontologies to enhance confidence in
  acquired sensor data?
   Source of Background/Context Knowledge
  Speckle Simulation
       Electric Field Intensity Transfer               Power Transfer
               random                                        EBCCD
E1     Pe j

                Collection Optics                                    x

                            E ' ( x, y )  E ( x, y ) * *h( x, y )
                                          P' ( x, y ) | E ( x, y ) |
                                                                     '   2
Field Results & Simulation
Turbulence Field Results
Third Generation FLIR Modeling
• Objective: To provide sensor designers, analysts, and war gamers with
  a physics-based, multi-spectral modeling capability in support of the next
  generation of FLIR sensors

 Band 1

 Band 2
                                                           Fused Image
Anomaly Detection ATR Input
Anomaly Detection ATR Output
THz Imagers – Concealed Weapons Detection
            Knowledge Engineering Activity
Constellations of heterogeneous sensors                                         Vast set of users and applications

                                                        Network Services

                                                    Sensor Web

             Surveillance                              Application Services

                                            • Distributed self-describing sensors and
                                            related services
                                            • Link sensors to network and network-
                                            centric services
            Chemical           Biological   • Common encodings, information models,
            Detectors          Detectors    and metadata for sensors and observations
                                            • Access observation data for value added
                                            processing and decision support
                        Sea State           • Users on workstations, web browsers,
                                            and mobile devices
 Ontologies (Prior Work in
 implementing Plausible Inference*
 on the Semantic Web)
                ow nedBy                                   partOf                              ow nedBy
     Car                      Grover             Trunk                  Car             Car                    Grover

partOf                                  containedIn                           containedIn
                         ownedBy                               containedIn                           neither
    Wheel                                      SpareTire                               Ernie

                                        We implemented, using Semantic Web
                                        infrastructure, a symbolic algorithm that
           Ri                           reads/parses RDF and determines when the
                                        composition of Ri and Rj is nonempty and
                                        whether it is a subset of Ri or Rj or neither.
*see [Huhns, 89] [Russomanno, 03], [Russomanno, 06]
          Requires Richer Relation Semantics
<!-- Definition of causedBy property -->

           <rdf:Property rdf:ID="">

                       <rdfs:comment>causedBy(Event1, Event2) means Event1 is caused by Event2.
                           It is the property used for object-object event
                           causation. Note that value "no" has been asserted for temporal
                           because the range element precedes the domain element.</rdfs:comment>

                      <rdfs:domain rdf:resource=""/>
                      <rdfs:range rdf:resource=""/>
CAS Knowledge Engineering
• Activity:
  Build an ontology-based, knowledge repository of
  representative sensors
   Logical data model analysis and ontology design
   Population of an ontology with sensor instances
   Construction of Ubiquitous Sensing Prototype Test Bed

• Objective: Tap in-house sensor expertise to
  develop knowledge models
                                                                         Ubiquitous Sensing Environment
               Connecting to infrastructure
    Discovering, extracting, collecting, and sharing data
                                                                     ...                                 Sensor Node
                                                                                          Base Station       S             S                 S
                                               Other Sensors,                                  M             M             M                 M           S
…                                             Sensor Netw orks,
                                                                        Gatew ay:
                                                                   Bridges the sensor
                                                                   netw ork to another
                                              Other Intelligence                                                                Sensors:                 S
                                                                   netw ork or platform                                                              S
                                                  Sources                                                                       Acoustic                 O
                                                                                                                               Temperature               R
                                                                                       Mote:             S             S        Radiation                N
                                                                                  Microcontroller                               Vibration                E
Dynamic and Heterogeneous Environment                                         Mesh Netw orking S/W
                                                                            Low Pow er RF transceiver
                                                                                                         M             M        Chemical
                                                                                                                                Biological       S
                                                                              Battery/solar pow ered                                ...                  O
                                                                                                                                                 M       R
CAS-Russomano et al.:
                                                                                                         S             S                 S

                                                                                                         M             M                 M
1. How do we expose sensor data, meta data, capabilities, and                                                                                                S
     services within a network-centric environment?
   ● Raw byte streams? Semantic Web?                                                                                                                         M

2. How do we discover and process data from multiple sensor
     sources for various tasks?
   ● Need explicit declarations and search strategies
3. How do we utilize sensors and other info sources for
     automated reasoning?
   ● Need shared understanding of semantics
4. How do we distribute tasks and processing across the
   ● Must handle vast numbers and varieties of data sources,
     adaptive networks and applications
Ontologies needed for remote
sensing through In-situ sensors
    U of M’s OntoSensor*
•        Work in progress … attempting to take advantage of local expertise in
         “traditional” physics-based sensor models and phenomenology

•        OntoSensor defines a set of concepts, taxonomies and relations
         common to many sensors

•        Seeks to leverage Semantic Web infrastructure: OWL, OWL-S, Rules,

•        Create sensor profiles that commit to OntoSensor with attributes,
         properties, and services

•        Deploy aspects of OntoSensor in applications ASAP
              Incrementally develop “deeper” knowledge models as familiarity with
               physics-based sensor models evolves

* see [Russomanno, 05a] [Russomanno, 05b]
OntoSensor Leverages Existing Work

• Sensor Markup Language (SensorML)
   Open Geospatial Consortium Initiative
      U of M is a (non-voting) member of the OGC
   Sensor Ontologists should be aware of this effort
   Developed from a Software Engineering rather than a Knowledge
    Engineering Perspective
   Not an ontology (UML models with XML realization only)
      No formal semantics
      Loose generic model
   May provide a good organizational framework within which an
    ontology can be created
   OGC revisions are ongoing (watching from the sidelines …
    incorporating aspects of SensorML into OntoSensor as
    SensorML Conceptual Models*

                                             Possible SensorML Applications Include:
                                             • Coincident search for relevant data (i.e., data discovery)
                                             • On-demand processing of data products
                                             • Dynamic on-demand fusion of disparate sensor data
                                             • Pre-mission planning
                                             • Onboard applications
                                             • Autonomous operation/target recognition
                                             • SensorWeb communications of location and targets
                                             • Direct transmission of data and processing information to remote sites
                                             • Determination of sensor footprint with time
                                             • Intelligent retrieval and co-registration of sensor data
                                             • Visual fusion of disparate sensor data
*SensorML material from [OGC 04-019, 2004]
Identifier & Classifier
•   Identifier includes a type definition
    (e.g. shortName, longName,
    serialNumber, noradID, missionID,
    etc.) and a codeSpace which takes a
    URI. The codeSpace identifies the
    authority source (typically an online
    Dictionary or Registry) for the value.

•   Classifier object provides a means of
    providing several classification tags
    with the description. For instance, a
    single sensor might be classified as
    “remote observing”, “infrared
    detector”, “airborne”, “civilian”, and
    “atmosphere observing”. Such
    classifications could assist in sensor
SensorML Capabilities
• hasCapabilities
  property provides
  information that might
  be useful for sensor

• includes properties for
  and taskableProperty
SensorML Interface
• Sensors might support
  the IEEE-P1451
  interface for low-level
  data transfer, or
  perhaps a specific Web
  service interface for
  tasking the sensor or
  retrieving data
• Each component, platform,
  sensor, and sample has its
  own local CRS (specified
  by the hasCRS property),
  which must ultimately be
  related to some geodetic
  CRS (e.g. latitude,
  longitude, altitude). The
  process by which this
  occurs is dependent on the
  LocationModel used.
SensorML Parameters
• Parameters are used within
  various SensorML classes
  as values for various
  properties of the type,
  dataComponent, as well as

• Example properties include
  latitude, speed, stepAngle,
  wavelength, or timeStamp.
SensorML Response Model
• ResponseModel is a particular
  type of ProcessModel that
  describes the sensor’s response
  to some phenomenon

• Describes the process by which
  a sensor measures a
  Phenomenon and converts that
  observation to an output Product

• ResponseModel provides
  information regarding the
  sensor’s sensitivity to a
  phenomenon and the quality of
  its measurements
Logical Data/Knowledge Model
U of M’s
    OntoSensor effort includes Agent Shell
•    Objectives
      Locate sensors via intelligent search
      Query specs/capabilities of a sensor and/or network
      Task sensors via abstract service specifications

•    Goal: Achieve these objectives at the agent level
     without the agent having to deal with proprietary
     software designed for each individual sensor

•    Important for adaptive fusion algorithms without a
     priori knowledge of specific sensors
    Summary of Accomplishments through 2nd Quarter
•   Logical Data Model Analysis: Analyzed the 1.0.0 (beta) specification of the
    Sensor Model Language (SensorML) for In-situ and and Remote Sensors
    published by the Open GIS Consortium (OGC).

•   Logical Data Model Design: Started implementation of a prototype sensor
    ontology (OntoSensor) using the SensorML specification (in-part), IEEE
    Suggested Upper Merged Ontology (SUMO), International Organization for
    Standardization 19115, and constructs of the Web Ontology Language

•   Sensor Repository Population: Skeletal data/knowledge about several
    sensors and motes have been instantiated.

•   Implementation: Coded initial version of Semantic Web Expert System Shell
    (SWEXSYS) capable of querying OWL knowledge bases. Includes
    Dempster-Shafer, voting fusion and other elementary fusion algorithms.

•   Prototype Construction: Started design and configuration of ubiquitous
    sensing prototype test bed.
                                                                                                                                 … 433 MHz TBD
    Prototype Construction (CAS-Memphis)
•   Sensor nodes are built using low-cost                                             Sensor
                                                                                 Instances and/or
    magnetic, passive infrared, light,           Semantic Web
                                                                           LAN     Aggregations
                                                                                     (in OWL)
    temperature, humidity, barometric            Expert System                       Commit to                     Sensor Node
    pressure, and acoustic sensors provided          Shell
                                                  (SWEXSYS)       ES 223
                                                                                                    Base Station       S             S           S
    by Crossbow Inc., and others                                                                         M             M             M           M
•   Wireless software provided by Crossbow’s                                                             G
    mesh networking protocol                                                                                                                             S
•   Mote packets are stored in a Postgress        OntoSensor
                                                 Know lege Base
    database for subsequent extract to              (OWL)         ES 231                                           S             S                           E
    Semantic Web data repositories                                                                                                       916 MHz             T
                                                                                                                   M             M                           W
•   MWIR and LWIR cameras from FLIR, Inc.                                                                                                            S       O

•   SWEXSYS can read sensor repositories in                                                                                                          M       K

    the OWL knowledge base                                                                                         S             S           S

•   Possible SWEXSYS Applications:                                                    Sensor
                                                                                                                   M             M           M
                                                                                 Instances and/or
        Thermal camera calibration using                                            (in OWL)
                                                                                     Commit to
                                                                                                                   Sensor Node
         wireless MEMS sensors (in situ) in                                         OntoSensor
                                                                                                    Base Station       S             S           S
         field of view for ground truth                                                                  M             M             M           M
        Map pixel to GPS location using                                                                 G
         wireless MEMS sensors (in situ) in                                                                                                              S
         field of view for ground truth                                                                                                                  M
         Ontology-driven algorithms to specify                                                                     S             S       2.4GHz              E
         sensor states based on information                                                                        M             M
         from base stations, perceived                                                                                                               S       O
         environment and overall objective of                                                                                                        M       K
         agent                                                                                                                                               2

        Increase confidence in target
         identification via incrementally
         acquired evidence and/or
         data/knowledge fusion
Typical specs
included in
repository that
commits to
  Knowledge Engineering 3rd Quarter Plans
• Add more detail about sensors to ontology
  (emphasize low-cost sensors that comprise
  the sensor networks first, followed by more
  sophisticated imaging sensors)              Sensor

                                                             presents       described by   supports

• Increase number of sensor
                                                   Service                  Service                    Service
                                              Profile                        Model                    Grounding

  instances/repositories that commit to      What the
                                           service does
                                                                          How it w orks
                                                                                                       How to
                                                                                                      access it
  OntoSensor in a network-centric environment

• Explore capturing sensor services into the
  ontology using OWL-S
Application Development (ASAP)
• Support ARL ongoing application development
• Persistent Threat Detection System (PTDS) sensor
  markup using OntoSensor
   MWIR, Color Video, B/W Video, Acoustic Sensor, etc.,
    suite on single platform
• PTDS interfaces to Counter Rocket and Mortar
  (CRAM) system
• Project: Implement PTDS meta data spigot & ad-
  hoc query support using OntoSensor concepts to
  support future interoperability and possible fusion
      CAS & Affiliated Personnel
• CAS-Memphis                  • ARL
     Carl Halford, CAS Director      Steve Murrill, CAS Program Manager
     Steve Griffin                   Gary Wood
     Aaron Robinson                  Bill Ruff
     David Russomanno                Ronnie Sartain
• CAS-VU                              Greg Sztankay
   A.B. Bonds                 • NVESD
   Jim Davidson                    Ron Driggers
• CAS-UAH                           Eddie Jacobs
   Pat Reardon                     Phil Richardson
   Joe Geary                  • ONR
                                    Keith Krapels
1.   [OGC 04-019, 2004] Open GIS Consortium Inc. (2004) “Sensor Model Language
     (SensorML) for In-situ and and Remote Sensors,” Version 1.0.0 (beta), M. Botts
2.   [Huhns, 89] M. Huhns and L. Stephens (1989) “Plausible Inferencing Using Extended
     Composition,” Proceedings of the Eleventh International Joint Conference on Artificial
     Intelligence, Detroit, MI, pp. 1420-1425.
3.   [Russomanno, 03] D.J. Russomanno and C. Kothari (2003) “An Implementation of
     Plausible Inference for the Semantic Web,” The International Conference on
     Information and Knowledge Engineering (IKE03), CSREA Press, Las Vegas, Nevada,
     pp. 246-251.
4.   [Russomanno, 06] D.J. Russomanno (to appear in 2006) “A Plausible Inference
     Framework for the Semantic Web,” Journal of Intelligent Information Systems.
5.   [Russomanno, 05a] D.J. Russomanno, C. Kothari and O. Thomas (2005) “Sensor
     Ontologies: From Shallow to Deep Models,” The 37th Southeastern Symposium on
     Systems Theory, IEEE Press, Tuskegee, Alabama, pp. 107-112.
6.   [Russomanno, 05b] D.J. Russomanno, C. Kothari and O. Thomas (2005) “Building a
     Sensor Ontology: A Practical Approach Leveraging ISO and OGC Models,” The 2005
     International Conference on Artificial Intelligence, CSREA Press, Las Vegas, Nevada,
     pp. 637-643.

To top